To avoid cross-platform surprises, I suggest that the list of invalid characters is the same across platforms, so that those characters are removed everywhere and not just on Windows where it has to be done.
Maybe this is stretching the original proposal a bit, but could there be some mechanism to ask the interpreter what it would name a file? For instance, if a story can’t find some data it needs, it might want to tell the player the filename it was looking under.
What is the expected behavior if a game ships with a .glkdata file?
- Library reads from the common directory (only), writes to the common directory (only), fails to open file
- Library reads from the common directory, then game directory, then writes to the directory where the file was found
- Library silently moves data files to the common directory, then reads / writes from there
I feel like you’re basically telling us “do it like Zoom does it” in the name of data-sharing and cross-library compatibility, without really offering any solid guarantees to authors that this new functionality will be reliable. If libraries couldn’t even agree on file extensions, how will they agree on where common data files should be stored? Absent agreement, Zoom will write its files in one place, Gargoyle in another, and even if both libraries are now looking for .glkdata files, they’ll look in different places and won’t come up with the same result.
In actual practice, I do a lot of data file sharing, but it’s with the same game running under the same interpreter on different platforms, synched via Dropbox. Moving files out of the game’s directory and into some unsynchronized cache would be a big step backwards in ease of use for me. It also complicates the obvious, low-effort solution for sharing data between games: have the player store the games in the same directory, or copy the data files from one directory to the other.
The most logical place to store a game’s data files is in the directory with that game. Anything else is going to need a more rigorous treatment.
I agree with Ben, they should be saved to the same location as the story file. That then will make it easy to share or protect the files.
Could the extensions be .data and .save instead? Glulx allows other io systems, and you should be able to play a story in fyrevm and then in gargoyle and have them access the same data files. Of course the other io system could just be designed to access .glkdata and .glksave, but i just don’t see the need for glk to be in the extension.
I can’t guarantee that, because I don’t know all platforms, but I can add more. Are :*? really invalid in Windows filenames, though, or are they just hard to type in the shell?
Too much of a stretch (and we’re considering the idea that the interpreter might look under several filenames, anyhow). What would the player do with this information? “You need the data file for Hipposaurus Rex” is about as actionable as “You need HippoData.glkdata”.
You’re overreading what I wrote. I’m not mandating or recommending any policy about whether data files should be kept with the game file or pooled somewhere. All I’m saying is that if external files are pooled somewhere, then data files should have a common namespace across games, whereas save-game files shouldn’t.
The interpreter might want to map file types to itself (e.g., double-clicking on a save file) and it’s hopelessly impractical to do that for .data and .save.
Potentially, but this isn’t a real-life case yet. If someone does set a game up this way, I don’t see that it’s harder to agree on one set of filenames rather than the other.
Well, the best thing about Windows is how well it runs Windows programs. Including old, badly written Windows programs. That’s the one feature other operating systems can’t beat. So Microsoft are generally very wary about changing things that might break old software. (I need Windows to run all my old Windows software. If Windows wouldn’t run that old software anymore, then I’d have to replace that software and I’d just end up using my Mac for everything.) 
Well, for the sake of clarification, by ``filename’’ I meant full path. In particular I was thinking of my change from Zoom to Gargoyle, and how it took me a bit to figure out that data files were being looked for in the directory enclosing the story file, not ~. Knowing the directory difference is what made the error message actionable. But depending on how things go, maybe that will be a moot point.
But right now the draft reads
Perhaps that wording needs to change?
I hope this can be covered by interpreter documentation. Reading a full path into game code seems like a pitfall, if not a security risk. (A full path on iOS would be unrecognizable to most users, and you don’t even want to know how Quixe handles it.
I have updated the draft spec to include:
The library should take the given filename argument, and delete any characters illegal for a filename. This will include all of the following characters (and more, if the OS requires it): slash, backslash, angle brackets (less-than and greater-than), colon, double-quote, pipe (vertical bar), question-mark, asterisk. The library should also truncate the argument at the first period (delete the first period and any following characters). If the result is the empty string, change it to the string “null”.
…
Some programs will look for all files in the same directory as the program itself (or, for interpreted games, in the same directory as the game file). Others may keep files in a data-specific directory appropriate for the user (e.g., ~/Library on MacOS).
If a game interpreter uses a data-specific directory, there is a question of whether to use a common location, or divide it into game-specific subdirectories. (Or to put it another way: should the namespace of named files be per-game or app-wide?) Since data files may be exchanged between games, they should be given an app-wide namespace. In contrast, saved games should be per-game, as they can never be exchanged. Transcripts and input records can go either way.
Good point.
Thanks. This is much clearer to me.
Rethinking the glk_stream_open_resource_uni() call a bit… would it be better for the binary/text flag to be encoded in the Blorb file, rather than being supplied at stream_open time?
The difference is only relevant if you’re reading Unicode data; it tells the interpreter whether to expect four-byte Unicode values or UTF-8. It makes sense to get this flag from the Blorb file, since the data chunk will definitely be one or the other.
(The Blorb chunk type would be TEXT or BINA, I suggest.)
Well, I’ve written test cases and an implementation (only in cheapglk so far), so the spec as currently written looks stable.
I’ve also written a Python blorb-adjusting tool. It doesn’t do everything, but it lets you examine blorbs and add, delete, and extract chunks to/from them. See eblong.com/zarf/blorb/blorbtool.py
I’ve added a few paragraphs to the Blorb spec describing the data-resource side of this from the underside: eblong.com/zarf/blorb/blorb.html
Each data file is stored as one chunk, with chunk type ‘TEXT’ or ‘BINA’ (denoting text or binary data). The format and contents are up to the game to interpret.
This feature was designed to support Glulx, but data resources can be accessed by any game format if the interpreter supports them.
For Glulx games (and any other game format which uses the Glk API), the data format must follow the conventions described in the Glk spec. (eblong.com/zarf/glk/, “Resource Streams”.)
To summarize: if the data file is opened via glk_stream_open_resource(), then it will be read as a stream of bytes; text will be assumed to be encoded as Latin-1. If it is opened via glk_stream_open_resource_uni(), then a ‘TEXT’ chunk will be assumed to be a stream of characters encoded as UTF-8; ‘BINA’ will be assumed to be a stream of big-endian four-byte integers. If read by lines (glk_get_line_stream(), etc), resource text should use Unix line breaks in all cases.