I’d like to start this thread to list and discuss any and all forms of undefined behavior in the z-machine as a corollary to the ongoing Standard 1.2 effort.
I have a decent list around somewhere that I put together over time, but for now I’ll just list the things that are fresh in my mind. Please feel free to add/comment/discuss/rant at your leisure.
Echoing input when output stream 1 is off, or 3 is selected.
Updating the status line in V3 games when output stream 1 is off, or 3 is selected.
Nested read/read_char - Calling read or read_char from an interrupt. This is a tricky one as nested reads are completely unmentioned in the standard, but are actually required. Border Zone calls a read from the timeout interrupt of another read…This should at least be mentioned in the standard. The undefined part is whether or not it is possible to return from the original timeout interrupt. Border Zone only allows RESTORE, RESTART, or QUIT as outcomes of the subsequent read, preventing the original from ever being reached. Infocom’s interpreters react badly when allowed to return through a specially crafted story file. My recommendation is that while any level of nested read/read_char calls be allowed, returning in any code path to any but the most recent to be an error.
Story length and exact cut-off for memory addresses. While a valid routine or string address must be within the range for a packed address of the appropriate version, this is actually smaller than the implied limit of 128K, 256K, or 512K, e.g 8*65535 = 524280, not 524288. I’ve always assumed the limit is the larger e.g. 524288. But it’d be nice to see it specified somewhere, especially given the lovely KB vs Kib world the hard drive manufacturers have given us. KB = 1024 bytes…there I said it. Footnote: The limit can actually be circumvented. I’ve built huge story files in the dozens of megabytes range and successfully executed code from all throughout them in bog-standard Frotz. (Edit: My memory may be exaggerating here although I know they were several megabytes at least.)
Should output stream 4 record the true original input (untranslated unicode in the case of extra characters, and the exact timing of input), or the effects as influenced by the game at that time (zscii after unicode table translation, actual timeouts), because these may not be constant across playthroughs.
Sound in V4 - seemingly illegal. Does anyone check?
var forms of op2 instructions - I think the intent has always been that je and set_colour are the only exceptions and all other 2ops require exactly two operands at all times, but it’d be nice to see this spelled out. I’ve never seen an interpreter allow more than two operands for any of the math operators, and quite a number of instructions would make no sense with more or fewer operands, e.g. throw, set_attr, test_attr, etc. The instructions call_2s and call_2n are potentially ambiguous, but if they are true var, then why the need for the call_var opcodes? I think it should be specified that even the var forms of these only take (and require) two operands.
The value of catch - only specified in quetzal, but makes saves potentially cross-interpreter incompatible if not specified. My first interpreter went a very different route and only assigned an optional word to a frame if catch was used, so the actual call stack could go higher than 65535 and still use catch/throw properly as well as preventing the use of throw without a previous catch (theoretically in most terps you don’t need catch if you know in advance how many frames are going to be on the stack). yeah…seemed like a good idea at the time. Don’t judge me.
Throwing from an interrupt routine while a read/read_char is active to a frame below the read operation. Yikes…never actually tried this one.
Changing the result store variable during routine execution. Yeah, another “Why would you do this?” item. In my case it affects me only because my preferred implementaion does not resolve the final store variable until after the routine returns. Most terps I believe do it when the routine starts.
Game modification of header values. Most header bits are off limits according to the standard, but there are Infocom stories which twiddle some of these in-game. “Suspect” comes to mind.
je with zero or one oeprand. je with one operand has been deemed both legal and illegal in the standard. je with no operands isn’t ever mentioned, but is also possible. Infocom’s interpreters don’t all do the same thing here. Some branch, some don’t. it’d be nice to see this clarified.
Not sure what you’re suggesting here. A string that starts at address 524280 and contains exactly eight bytes is obviously legal. A string that starts at address 524288 is not valid because you can’t encode that address.
Are you getting into the question of a string/routine that starts below the memory limit but extends past it?
That’s always been my take too, but the file length field in the header suggests otherwise. You can’t reach 524288 since the file length field maxes out at 524280. Oops. Now we’re having fun with non-zero ‘padding’ on the end of our file that actually contains important data/code. Frotz definitely doesn’t enforce a limit anywhere near here.
As I’ve said in some other thread, I’m also thinking about setting up an IF spec site on the IF Archive somewhere. Z-machine spec documents would be duplicated there.
After considering the file length in the header I’m thinking the lower number should be the limit and a string or routine that begins on the last possible packed address (for all versions except 6 and 7, which vary because of the offsets) could never be valid because it would be only one byte long. The file sizes listed in 1.1.4 could be made more specific.
Edit: Argh! Wrong, wrong, wrong. Even knowing there is an issue here, I couldn’t get it right.
To illustrate: In V8 the max file length is 8 * 65535 = 524280. But this means the valid addresses are 0 to 524279. The maximum packed address is completely invalid as it would represent 524280.
There are a couple of things I noted down while working on an interpreter.
In all versions but 6, startup occurs outside of the context of a function. Several opcodes just assume they’re in a function, though (all the returns, catch, throw, and check_arg_count, at least). At the very least I’d say interpreters should be advised to detect this and halt with an error message.
The behavior if text is written beyond the edge of the upper window: wrap, or cut off/extend beyond the edge? Interpreters differ here. For example, Lectrote (so probably ifvms.js) and Unix Frotz wrap the text; Windows Frotz and Fizmo cut it off. An interesting test case is My Angel.
In section 8 of the standard it says that scrolling is never applied to the upper window and in the remarks it suggests that wrapping and buffering in the upper window is incorrect. Edit: A reading of Infocom’s ezip spec says that window 1 never scrolls and printing beyond the right-hand margin is not displayed. Perhaps the clipping of text in the upper window could be made more explicit in the standard though, instead of being in the remarks only.
Returning from the starting routine in V6 or the ‘functionless’ context of all other versions is in fact illegal according to the standard (sections 5.4 and 5.5). Catch and check_arg_count should just return zero, while throwing to the ‘main’ routine via “throw 0” would then also return from that routine because of the behavior of throw (section 15, description of throw opcode) and so is also illegal, which agrees with the standard.
It would also be nice if unit tests could be created for any of these issues. I’ll very happily receive pull requests for Praxix for non-UI/interactive stuff, though we should probably make a new one for UI/interactive stuff. Clipping would be an easy thing to test, just print “should not be visible” after a screen’s-width of spaces.
On wrapping/clipping, section 8 says only the following:
In Versions 3 to 5, text buffering is never active in the upper window (even if a game begins printing there without having turned it off).
Some ports of ITF apply buffering (i.e. word-wrapping) and scrolling to the upper window, with unfortunate consequences. This is why the standard Inform status line is one character short of the width of the screen.
I read this as saying that the problem is with word wrapping more than character wrapping, though you’re probably right that clipping is the best option of all.
@cas Yeah I don’t think it would be possible with Glk/HTML to have input wrap.
I was looking over Infocom’s xzip spec and noticed that CATCH and THROW were never intended to work from within interrupts. That would take care of #9 if it were added to the standard.
When accepting unicode (extra characters) as line input, it is ambiugous whether or not the conversion to zscii should happen before or after input is lower-cased.
This gives us two scenarios -
Before case change: Requires both upper and lower case versions of the unicode character to exist in the table if you want to allow case insensitive input. Possible weirdness if any are missing.
After case change: Only lower case letters needed in extra character table (unless upper case is needed for output). This means an upper case unicode character will always be valid input if the lower case character is. Maybe this is what the author wants, maybe not.
My understanding is that unicode characters that are not in the extra character table are not valid for input. If this is correct, then only the first case is conforming.
I know of nothing in the standard that would dictate one case over the other. It all depends on when the transition to lower case is made.
In the first case, a story file with non-matched upper and lower case characters in the extra characters table could lead to case-sensitive input in a way that can’t happen with ascii characters.
Agreed, but this case sensitivity is already in the standard for read_char.
My thinking was that check_unicode returns bit 1 set for a character “if and only if the interpreter can receive it from the keyboard” which I took as a requirement for the character to be in the extra characters table. Thinking about it, this is probably more strict than what is mandated.
Of course read_char is a different beast anyway, accepting characters that read won’t (at least for anything more than line terminators). It seems clear that read was always intended to be case-insensitive.
My interpreters have always done the case conversion first, and it wasn’t until long after my first one that I realized it could be handled differently. I’ve not looked, but I’m curious how the case conversion is handled in other interpreters, especially since the exact zscii values are story dependent (and although unlikely, perhaps missing entirely!)
Do they do a double lookup like this:
Check the unicode char is a valid zscii extra char
Convert that to lower case
Check the lower case value is a valid zscii extra character
Use this second zscii value
The standard says only function key codes can be in the terminating characters table, which would remove any ambiguity regarding terminating characters (maybe this is why), but see the other recent thread which points out that at least one Infocom game contains non-function key codes in the terminating character table (although it is a slash, which still doesn’t present a case problem).
I think I’m going to stick with lowering the case first, then doing zscii conversion. It doesn’t require a double lookup, it is what frotz does, and it removes any problems with the terminating characters table as well. Presuming we allow arbitary characters here in contravaention of the standard, then upper case terminating characters, just like upper case dictionary entries, would never be matched.
(Edit: My memory may be exaggerating here although I know they were several megabytes at least.)
