Does it seem like this proposal would cover want you want in ‘more sound channels’ type features?
And does it seem like a system you’d be happy to add support for?
I’ve updated the proposal very slightly by making the Unicode strings optional, and put the result at frobnitz.co.uk/zmachine/1.2/draft2.html
I do actually like your single gestalt for all streams, but it needs to note the gestalts that are in current use even if it depreciates them at the same time. Same for stream six - it may not be needed any more, but it shouldn’t be reused.
I also agree that all new features should have gestalt checks.
I’m finding that doing this for Frotz will be fairly simple, once I get some basic threading code worked out. None of this seems workable with the DOS version. Indeed, I’ve frozen DOS Frotz’s sound capabilities at AIFF samples only. More than that is too much to ask of DOS.
So far, as far as I’m aware, no attempt to make a 1.2 Standard has been widely agreed on, and no 1.2 Standard exists.
I’m not happy with the idea of being required to document the features of every game that used non-Standard functionality, even if that functionality was documented in another New Standard proposal. If your game uses non-Standard features, that’s fine, but they’re still non-Standard.
(Obviously if the general opinion is that I’m wrong, and the Standard needs to acknowledge these other gestalt ids, I’ll add them)
I feel like I should explain my thinking behind Output Stream 5 a bit more, so someone can tell me if I missed something.
Dannii’s draft for a 1.2 standard mentions (among other things) four new features: Transcripts Protocol, @parchment opcode, Output Stream 5 (eval() stream) and Output Stream 6 (Vorple Stream).
Now, I’ll admit I haven’t looked very deeply into any of these, but:
the draft for the Transcripts Protocol states “The basic system will involve interpreters periodically sending logs of a player’s input and the story file’s responses to a web server.”
the @parchment opcode currently has only the ability to send text to be sent to eval()
Output Stream 5 also seems to be a way to output text to eval()
Output Stream 6 seems to be sending some other hints to the interpreter in text form (html or something?)
Aside from the fact that we surely don’t need two different ways to send text to one function that most interpreters won’t have access to, it seems to me that essentially all four of these operations are basically doing the same thing. That is, they are all sending textual output to the interpreter, that the interpreter may not understand, in the hope that the interpreter does understand it, and does something clever with it.
My idea is that all four of these features can be accomplished with just one output stream. Output Stream 5 therefore gives any text sent to it to the interpreter in the hopes that the interpreter knows what it is, and does something clever.
Now, obviously, the chances are fairly high that the interpreter won’t know what to do with whatever random text the game decides to spew out, but that’s okay, the interpreter is allowed to do nothing if it doesn’t understand. A slightly more difficult problem is how the interpreter can tell the difference between, say, code to be sent to eval() and nicely formatted data for the Transcripts Protocol.
And my answer is, not my problem.
My thinking is that it would be sensible for any use of output stream 5 to begin with an agreed (between game writers and interpreter writers) sequence (as for instance ‘JSEVAL:’ or ‘SCRIPTPROTOCOL:’) which lets the interpreter know what the game is expecting of it. Maybe something less ugly, but I feel it would be sensible to use meaningful words to reduce the likelyhood of two systems with different features using the same ID sequence.
But that’s up to the people implementing the features (on both the game side and the interpreter side). The Z-Machine Standard Does Not Care what is being sent to stream 5, or what is being done with it.
And then I thought, what about letting the interpreter talk back to the game? Hence Input Stream 2. I’m still less sure about Input Stream 2, though.
Yeah, a string of text where the first word is the element’s name and the rest is the class name applied to the element. It’s not pretty and I’m glad it’s gone.
Oh, this would be marvelous. I had the impression that sending meaningful data back to the game wouldn’t be possible so I have this horrible hack where the interpreter sends information through hidden commands. If using input stream 2 doesn’t break existing interpreters I’d be very interested in this option.
Well, as it stands right now, it would be an input stream like any other input stream, which means the text would be sent to @read. This means a maximum of 255 characters per call to @read, but you could call @read as many times as you want. Also all the text would be reduced to lower case. These things are why I’m not completely sure about input stream 2.
I suppose an alternative to this behaviour, maybe more useful, it to have the interpreter look at the header extension for an address, and then have a table at that address which the text is printed to when input stream 2 is active. This would allow for more text to be input with fewer operations, and preserve capitalisation. It would remove the lexical analysis that @read does, though.
Talking to myself again. Sigh.
Even if we did this, the interpreter would have to wait for a @read instruction. Maybe if we just specified that text is not lower-cased when coming from input stream 2? Does that break anything? Does it help anything? Am I thinking about this too much?
For story-terp pairings, sure. But I’m squeamish about under-specification for the sake of unpaired code. For one, if I’m writing a story feature that would be implemented one way under terp X and another under terp Y, how do I figure out what to write to stream 5? Or, if I don’t code a check, and someone puts that story in terp Z, how does it know what to skip and what to interpret?
As things stand, the spec is shrugging its shoulders and asking (at the least) interpreter authors to come to a common agreement on these points. But that seems odd; the purpose of the standard is to document such agreements.
On the ‘how does it know what to skip and what to interpret’ bit, well, that’s why I suggested that games should send codes to indicate what the text is for. The idea that the game would be sending different output to different interpreters depending on what feature they support hadn’t occurred to me. That is a problem.
I still like the general idea of just one stream to send out text for the interpreters to interpret. I’ll grant that maybe it needs a better way to specify what we’re asking for, and a way to indicate that the request was not understood.
The reason for under-specifying the nature of text sent was that this seems like something people might want to add more to fairly often, and so far the Z-Machine Standard has been updated every ten years or so. A separate document of codes might work better. I don’t know. I think the main problem is letting the game know if things worked. Anyone got any ideas? I’ll think some more about it myself.
Okay, then we seem to be on the same page. I agree that the range of message kinds that can be sent on stream 5 should be open and not nailed down by the spec. I only wanted to make sure that the forms that protocol queries and message delimiting take get fleshed out and documented.
The other IF I/O system that does something similar is Textfyre’s channel I/O. If one were to follow its design, message types would be indicated by a word’s worth of Latin-1, which would fit in a second argument to a @gestalt. I don’t know that being limited to two characters is the best idea, but it does seem that the gestalt system is the best home for these checks. Maybe it would work for the second argument to be the address of a ZSCII string instead.
I feel I should stay right out of the input-stream discussion, but here are my other detail notes.
Unicode strings:
The “word-767 is the length” notion is gross. It uses all the remaining Z-characters to store not very much information, and then you need to chain Unicode groups if you have more than 257 characters anyway.
Also, this fails to encode characters above U+FFFF (unless you get into UTF-16).
Counterproposal: ZSCII code 1023 means “Unicode escape”. The interpreter starts at the next word and reads bytes (not words), decoding UTF-8, until it reaches a zero character (U+0). It then switches back to normal decoding at the next byte.
Decoding UTF-8 is not painful; see the gli_parse_utf8() function in github.com/erkyrath/cheapglk/bl … cgunicod.c . Note the convenient fact that a zero byte in UTF-8 will always represent a zero character, never a part of some longer character.
This plan would save space for mostly-Latin text, because ASCII encodes one byte per character. It would cost only one extra byte (the terminating null) for Greek, Russian, Arabic, Hebrew (two bytes per character). It would take 50% more space for CJK languages (three bytes per character). I don’t think large blocks of emoji (astral plane) are likely.
I would also drop the “invert bytes to obfuscate” thing because, honestly, why. We’ve had ztools forever. If you do want to keep this, specify that the interpreter should xor bytes with $FF before UTF-8 decoding.
A rough corner of this plan is that the word-alignment of the Z-code decoder can shift (if the UTF-8 string has an odd number of bytes, including the null). Is that a problem? I can (just about) imagine it tripping up an old-school interpreter that swaps high memory from disk. If this is a problem, specify throwing on a second null byte as padding.
I’ve updated again, with a third proposal at frobnitz.co.uk/zmachine/1.2/draft3.html
Moved sound channel availability checking to @gestalt, added a method to check for output stream 5 format availability. Completely rewrote Unicode Strings as per zarf’s advice.
Regarding the streams:
-
Could you clarify ``the first word’’ in 7.1.2.3.1? Without context, I would read that as the first two bytes. Or maybe that’s what you meant, though then a note under gestalt 8 would be nice.
-
When a story is done sending a message under one identifier, how does it switch to another? I.e., what signifies the end of a messaage? Or can it not switch?
-
What’s the reason for ``only Latin-1 characters’’ in identifiers?
And a couple of typos: 10.2 says that input stream 2 is input stream 3, and the gestalt opcode was changed in the Gestalt section without a corresponding update in the opcode summary.
On a different note, 10.2.2 reads ``It has no way of finding out which input stream is currently in use.’’ That’s the one thing I’d like to see changed, so I guess I’ll air the suggestion while I’m posting.
This is unfair - up until now the 1.1 standard was just as much a draft. I’m not even sure if anyone had used 1.1 features yet.
You’ve changed the number of arguments to @gestalt - I don’t think that’s a problem for EXT opcodes, but could someone confirm?
You can’t change the meaning of gestalt selectors that are already in use. Parchment shipped with @gestalt a couple of years ago, and its selector 1 is for the standard revision. Having an interpreter revision number is useful - make it 2, or something else.
The transcripts protocol and @parchment are both dead I think. Depreciate their selectors. Gnusto does implement @parchment, but it has been phased out in Parchment, so I guess you could reuse it for @font_size. If Juhana no longer needs a second stream for Vorple then it’s selector and stream can be depreciated, but not reused. I still think it would be wise to document the zoom opcodes. The $200* selectors can be depreciated now that the standard is being updated.
I think the idea of using a tagged stream 5 is probably a good one. Your spec doesn’t however mention a memory buffer for it to write into. Input streams are useful, but only when used for commands - not everything that JS would want to return is a command. Remember games cannot tell which input stream is in use. You also don’t cover overlapping streams, including with stream 3 and further uses of stream 5.
It is absolutely essential that there be a record of stream 5 identifiers! If these are not standardised it will be unusable.
What I recommend is keeping selector $30 around for untagged uses of stream 5 (even if its depreciated.) An interpreter can either support untagged JS, or tagged uses, but not both. They can switch which one they support based on which gestalt the game asks for, but once it has decided it can’t change. This will allow both games and interpreters to be backwards compatible. Alternatively, make the multi use tagged stream be stream 7.
You need a selector for @font_size.
Why is a selector needed for timed input?
If you’re going to have a selector for undo, you might as well make it useful and get it to indicate how many undo slots are available. Perhaps it could even return a number? Or -1 for unlimited (or unknown but more than 1).
Does anyone really want to mess around with knowing whether unicode characters can be printed? Pretty much every interpreter these days lets the user select the font, so the answer will be determined by the font.
What’s the reason for making @font_size a brancher?
Experience from the Glulx side of the universe say “No.” This is a feature from the Glk gestalt call, but I’m pretty sure nobody has ever seriously used it.
I don’t know much about the previous 1.2, so I’m not sure how it compares to process 1.1 went through.
I do know Kevin Bracey and I spent about three years of discussion on the Z-Machine list getting to a consensus for 1.1 (which was the version labeled draft 9), at which point, it was uploaded to the archive (accepted). The actual updating of the main doc failed to occur (I seem to remember Graham asking for volunteers, but it never came to pass until now) and a bit after that the standalone doc was re-titled from “Z-Machine Standard 1.1 Proposal” to “Z-Machine Standard 1.1 Document” (which also canonized Blorb and Quetzal).
I checked before I did it. EXT opcodes can have four arguments. Actually, eight, I think, but there’s rarely a need for that.
Overlapping streams work like they do now. If you don’t want the data sent to multiple streams, don’t turn multiple streams on. It’s perfectly reasonable to allow a game to send text to the screen, and memory, and stream 5 at the same time.
The input streams thing, yeah, I still don’t know what I’m doing about that. I want the functionality. @read takes text sent to the input stream and puts it into memory (lowercased, but maybe we can fix that). Why is that not good enough? (I’m serious, I need to know what’s needed so I can try to work out what to do).
Surely the game knows which input stream is in use if it keeps track of that internally?
Even an interpreter which supports changing the font size needs to be able to tell the game “No, I can’t make the font any smaller.” Interpreters which can’t change the font size at all signal this by never branching (except when the size argument is 0. It’s always possible to set the font to the default size). Given this, I’m not sure we need to have a gestalt.
There’s a selector for (almost) everything there’s a header flag for.
Yeah, good idea.