That’s not quite the whole question though. Z-machine operands are 16-bit, but addresses aren’t. The memory map extends beyond $FFFF, so somewhere in the interpreter, addresses are stored in 24-bit or 32-bit fields.
Of course @loadw isn’t meant to read up in high memory. I guess that’s what that particular Praxix test was about. Do you convert the operands to 32-bit values and then perform 32-bit addition, or do you perform 16-bit addition and then convert the result to a 32-bit address?
This is key. I went back and looked at my old C# interpreter, which is the last time I ran Praxix I think. In that I treated the operands as unsigned 16-bit values, but C# promotes to 32-bit integers automatically when adding. I then cast that back to 16-bit before reading the address (effectively wrapping).
In my newest interpreter in Rust, I was converting the array and index into 24-bit address values before doing the addition, thus no wrapping, which would mean failure in Praxix.
Thanks to everyone’s comments and ideas in this thread. I appreciate the dialog.
@fredrik Yes, that Praxix test was the one I was thinking of. Thanks for finding it. My original implementation of loadb (etc) was trying to make Praxix happy, but it broke Lost Pig. I suspect (?) that this problem might be restricted to games > z5?
@zarf The opcode in question is 29277: 30 00 00 0e LOADB #00,(SP)+ -> L0d
I am running a debug version of Yazmin, in which I inserted copious logging to match up opcode-by-opcode what I’m processing vs. what Yazmin is doing. Because Yazmin runs Lost Pig perfectly, even when constrained to the same tiny screen dimension I’m forced to on Pico-8. (I initially thought the tiny screen layout was affecting print calculations )
In Yazmin and mine I saw the first occurence of strangeness at that line, and when I investigated the parameters being passed I saw 29277: loadb(00000 09014)
So the baddr == 0x0000 and n == 0x9014
My terp took 0x9014 to mean -28652 and that was the source of my issues.
In this case, the positive offset is clearly necessary. This particular case could be trapped by checking if the addition of the two is < 0. But it raised a concern in me that there could be games (presumably limited to more recent ones) with weird edge cases where either adding or subtracting n could point to a valid memory region.
I’m just not completely clear on this point, but I will note that Klooster writes in the opcode section, “All operands are assumed to be unsigned numbers, unless stated otherwise.” and loadb (etc) are not stated otherwise. Ignoring Praxix for the moment and going with that information got Lost Pig working (at the expense of Praxix crashing during Array tests)
@Mike_G The first operand is a byte address, so it will never be signed.
Well the second operand is supposed to be an element in an array - not an arbitrary memory offset, so presumably it was always intended to be positive too. Clearly things have not turned out that way. Negative addresses make about as much sense as negative element indexes. Often, z-machine specifics are not straightforward due to the “Specification by implementation” way they were first created, followed by the whole reverse engineering thing and followed by later extensions.
Edit: Not to mention people hitting edge cases that Infocom never did in their games.
What about a case like loadb ( 0xFF00, 0x01FF)? With 16-bit wrapping this returns a positive 0x00FF. Seems like it should be trapped as an error, but the case you cited seems like it should be too if the second operand is considered signed, but not if it is treated as unsigned.
I’ve always interpreted the zmachine standard this way.
As you see, arr3 is an array with address $94E6. (Which prints as a negative integer, because it’s over $7FFF, but it’s a valid address in addressable memory.)
The line arr3->0 = 99; compiles as @storeb $94e6, 0, 99 This is the “obvious” way to refer to this memory address.
(I’m using the -~S switch, by the way, so that Inform compiles array accesses directly to opcodes. In strict mode there’s an extra layer of error-checking which obscures what’s going on.)
The line val = 0->arr3; compiles as @loadb 0, $94e6 -> L00. As you see, interpreters do take this as a reference to the same address.
This came up recently in a Glulx discussion (with 32-bit values, obviously, not 16-bit). My answer was “Eww, don’t do that, it’s not supported.” (Even though it happens to work in some Glulx interpreters.)
In Z-code? I know we’ve all agreed that this is not a way to access Z-machine memory addresses above $FFFF. But I don’t know if we’ve had a discussion about whether it’s guaranteed to access address $00FF. I feel like it’s risky, but that’s just me.
I heartily agree with that. I am concerned with trapping would be errors though.
Agreed, although I’ve always felt like this might have been a missed opportunity to do so.
It probably is risky, but isn’t the example of @loadb 0, $94e6 -> L00 relying on low end wrapping if we are considering that second operand as signed? Seems like the standard could use some clarification here.
@Mike_G Yes, I see what you mean now. I somehow didn’t notice that particular note in the spec doc, because this is what is written about loadb in Klooster,
“Store byte in the byte at baddr+n.”
That says nothing about arrays or indices, and given that generic definition Praxix isn’t wrong, per-se. If it truly is strictly about “here’s the start of an array, and here’s the index into it” then I totally agree with you on the handling.
loadb ( 0xFF00, 0x01FF)
In my project, all address manipulation occurs in a 32-bit space so I hadn’t really considered 16-bit wrapping until now. As you say, standards clarification would be nice on the point.
@zarf I appreciate you doing the extra work of running a real test.
I suppose the approach here then, to keep both Praxix and Lost Pig happy, would be to perform the signed addition first. If that value is less than $0000, perform an unsigned addition. Perhaps assert then that the value is within the dynamic memory threshold stored at $0e (in Lost Pig’s case it is up to $a63a). That seems… safe, I think? (maybe reverse it into the more common positive case as the first attempt)
I can’t imagine a case where $8000 and above doesn’t cross either low or high dynamic memory boundaries as a general trigger for this ambiguity event. If dynamic memory alone were of size $10000 or higher, that would imply to me a significantly large game file that probably exceeds what I could load anyway. Maybe…
From Infocom’s original ZIP spec (consistent with all of their later specs):
GET table,item 2OP:15/VAL
Interpreting the table pointed to as a vector of words, returns the item’th element. In other words, returns the word pointed to by item times two plus table. (Tables begin with element zero.)
From the z-machine standard 1.1:
loadw
2OP:15 F loadw array word-index → (result)
Stores array–>word-index (i.e., the word at address array+2*word-index, which must lie in static or dynamic memory).
The existing behavior that has been adopted does not match at all the description given in either Infocom’s documentation or the standard itself. It is therefore surprising and surprise is not at all desirable in any standard.
Clearly whatever is actually in use wins by being the de facto standard, but the written standard needs to be updated to reflect that reality. If some edge cases should be considered “undefined behavior” or outright errors, then the standard needs to spell that out, or state what the expected behavior really is.
I’d argue that if wrapping array+index below zero is legal, then I see no compelling reason wrapping above 65535 shouldn’t also be legal. This implies there are no error cases that can be generated by these opcodes other than falling outside of dynamic memory after 16-bit wrapping, and neatly seals up the possibility of these opcodes ever being used to access memory beyond 64k, which is agreed upon my most terp authors if I am not mistaken.
Lost Pig is one of the games that I distribute in a collection, built with Ozmoo. As far as I know, it’s fully working. And Ozmoo passes the Praxix suite.
Here’s the Ozmoo code to calculate the final address for loadb and storeb:
No ifs or different ways depending on case, just 16-bit addition, which wraps around in case of an overflow, and this addition can be used for signed or unsigned addition with 2-complement. In a higher level language where numbers may have a lot more bits, this would be the same as
final_address = (start_address + index) % $10000
As Zarf mentioned, loadb can access all of readable RAM (all of the first 64 KB of the memory space), while the final address for storew must fall within dynmem.
As for dynmem size, it can’t be larger than $FFFF, so 65535 in decimal. The word at $0E in the header points to the start of static memory, so static memory must start no later than $FFFF.
The lesson I’m seeing here is that while Z-machine addresses can be 24-bit, Infocom’s interpreters used 16-bit math everywhere they possibly could. (Because, like Ozmoo, they were written in 6502 assembly or similar!) They only broke out the larger fields where they absolutely had to, e.g. the code that dealt directly with string printing and function decoding.
All the corner cases we’re looking at derive from this approach. Our spec documents and test suites follow on from that.
@fredrik Thank you for the sample code and explanation. I see now that I was overthinking things, letting the larger address space needs of the z-machine confuse me about the computational math restrictions of the lower-level opcodes. If I simplify and let the cpu handle things, rather than trying to “help” it, I should be in better shape.
@zarf Your phrasing gives me a more concrete frame of reference when thinking about how things work internally and why. Even if the current state of things has expanded into larger address spaces, the core mathematics are still dictated by the 16-bit numbers of the z-machine’s inception. A creaky foundation perhaps, but it is what it is.
EDIT: Yep, that’s what I was missing. I was spending too much effort converting addresses to my internal representation and computing relative offsets. All I actually needed to do was let 2’s complement 16-bit math do it’s natural thing.
A bit late to the party here - but just to add to the agreement I also let things go their natural course. I use straight-up 16 bit unsigned for both @loadb and @loadw as per below. I drop any overflow after the calculation, even for word. This is the code from all versions of vezza/MxZVM for the two load functions, noting the store are done the same way:
;Load and store operations
; @loadb
z_ldb: ld hl,(v_arg1)
ld bc,(v_arg2)
add hl,bc
call ZXPK64i ;peek64
ld c,a
ld b,0
jp ret_bc ;return the value in BC
; @loadw
z_ldw: ld hl,(v_arg1) ;Array
ld bc,(v_arg2) ;Offset
add hl,bc
add hl,bc
ld e,0
call ZXPKWD ;Read word into bc, ZPC increment not required.
jp ret_bc ;return the value in BC
;