Since a couple years ago I’ve been thinking about interpreter autosaves and how they could be improved. Two things I didn’t like were that they didn’t store any versioning details, and that they usually involved multiple files. So I would now like to propose a file format for versioned autosaves: Versioned autosaves | Dannii’s IF Pages
The format itself is very simple, it’s just an IFF file with a Vers chunk storing version numbers in JSON. Easy to write, easy to read.
I’ve also sketched out a basic C API for a VM to use to read/write its data.
At the bottom are some questions I still have. Some of them more concern the process around autosaves, such as whether there should be a single way for naming autosave files, rather than a format-specific way, and whether Zarf’s hook functions need changing to work with the proposal.
After the Async Parchment update is merged into master I’m going to begin trying to implement this. No doubt I’ll discover aspects of it that won’t work yet. But ideally this would be a format that could work for all interpreters, so I’d appreciate feedback from everyone.
The current autosave implementations are “internal”, in that
they’re not visible to users as files
they’re not portable between interpreters
These facts go together, I guess. You can’t see them, you can’t export them, it doesn’t matter what format they have. (In the case of Quixe autosaves, they’re not files at all, but JS objects.)
Versioning within an interpreter’s format may well be a good idea, but doesn’t have to be specced if there’s no exchange. On the flip side, if we want these files to be portable between interpreters, I’d think a lot more speccing would be necessary. “Components are free to add whatever other chunks they need” isn’t going to be sufficient.
Right, the internalness of autosaves is why I don’t think we need to specify how an autosave is named. You could use just the Z-Machine header, or you could md5 the whole storyfile. We probably don’t need to be consistent there.
On the other hand, it would be helpful if the same autosave code in Glulxe can be used in Gargoyle, Spatterlight, Parchment etc. They might need different autosave-locating code, but they shouldn’t need different autosave read/writing code. (I’m not sure to what extent that’s currently the case.) I’m not envisioning that an autosave could be interchanged between Glulxe, Git, and Quixe, or between GlkApi.js and RemGlk.
Each layer of VM, Glk, and GlkOte could get version incompatibilities at different times. I was originally thinking that each layer would store its own version, but I think now it’s simpler to store them together, which also allows us to investigate and see the specific implementation of each layer.
For interpreters with a choice of format VMs, it would be very good to be able to find out which VM to load. The format I’m proposing here would make that very easy. Then if the default VM is changed or if the user changes their preferences we’ll still be able to load old autosaves. (Terps might need an “override autosave VM” setting.)
Lastly, I’d like to add a feature to Parchment to share a link with a snapshot. This would be effectively sharing an autosave between computers, and would also depend on being able to identify which VM needs to be used.
Yes, me either. :/ I know Spatterlight has autorestore but I haven’t looked at the code.
Hm, yeah. Hadn’t thought about that.
The other obvious problem is that the hook API between the layers is already so complicated that I can’t remember how it works! Every time I touch it, I have to figure it out. This is not a prime candidate for adding more abstractions to.
(Adding a “whoops, wrong version, this autorestore fails” check is conceptually easier. So the versioning itself should not be a big deal, but I am really hoping not to have to rethink the hooks.)
Looking at Glulxe, it constructs its own autosave paths in get_autosave_basepath. For the reasons in my last post, I think it would be better if it was the library’s responsibility to do so. So I’m thinking of a function like this:
strid_t autosave_find_file(glui32 format, strid_t gamestr, glui32 writemode);
You’d pass it a format ID (we’d have to predefine these, Z-Code can be 1, Glulx can be 2, etc) and a stream for the storyfile and whether you want to read/write. If the library can locate an autosave for the game then it returns a stream to the file. The format ID means the library could also handle unknown formats (just md5 the whole file).
Actually it would be better to just pass in the storyfile format into autosave_set_vm! Then it only needs to be provided once.
But the game stream won’t be changing… should we also pass that into autosave_set_vm?
Similarly, for use by a multi-interpreter launcher to identify which VM to load, we could have this function:
void autosave_get_versions(glui32 format, strid_t gamestr,
void *ptr, glui32 *len);
Give it a format ID and a stream for the storyfile, and if it finds an autosave then it will read the autosave, get the versions chunk, and copy it to a new allocation which it returns. The launcher would have to mfree it once it’s finished.
Pinging @Angstsmurf in case you have any thoughts.
Who implements autosave_get_versions and who calls it? Can you sketch how and why it would be used (called? implemented?) by the launcher? I thought I could follow along previously but now I’m stumped.
I have to second that this tangle of hooks is already too complicated to keep in one’s head when not actively working on it.
Oh yeah, that one doesn’t really make sense. Usually the Glk library and the launcher will be closely integrated, so no need to specify how they interact.
I agree it’s quite complicated. But I think that goes with the territory for a pioneering approach. While Glulxe+RemGlk/iOSGlk work, that complexity is taken into Glulxe’s own codebase, such as when it handles constructing paths and serialising the dispatch layer.
I’m positing here that a more verbose description of the process and the responsibilities of each layer will end up conceptually simpler and easier to grasp than what we have now. The dispatch layer’s serialising code might be completely portable, for example.
I’ll work towards making a detailed outline of the process an interpreter would follow.
Pinging @Angstsmurf in case you have any thoughts.
I don’t really have any comments except to say that I’m currently busy breaking the standards by adding lots of extra information to the Bocfel autosaves, in order to support my non-standard Z6 hacks. The Spatterlight version of Bocfel now has two separate, incompatible, code paths for autosaves, one by me, one by Chris Spiegel.