An asciicast is worth a thousand words, so click the box below.
WebAssembly is a new VM designed for security, rapid downloads and JITting, and native-like speeds. It was originally designed for web browsers but several standalone implementations now exist.
As a way to gain more experience with WebAssembly, I’ve been working on a WebAssembly target for the Inform 6 compiler.
What works so far:
Printing a string
Arithmetic
Bitwise operators
Logical operators
for, while, and do loops
break and continue
if and else
Routine calls
You can have any number of parameters as long as it’s two.
return
rtrue and rfalse
What doesn’t yet:
Most of the features of the print statement
Debug info
Inform-style debug information not tested and will need adapting
native web assembly debug information not implemented
Omitting unused functions
Pretty much everything past the first chapter of DM4
objects
classes
global variables
Huffman decompression
The jump statement
Some parts are stubbed out, including (most shamefully) the number of functions generated.
I thought I’d share the approach I used for I/O on the online Quill Adventure player tool I developed recently for playing ZX Spectrum Quill games online (take a look at my website link through my user details if you’re interested).
The backend of this was developed in the language Rust, targetting WebAssembly. Although I initially handled I/O just as character streams, it quickly became apparent that I needed something a bit more. Even on the fairly simplistic games that the Quill generates, there were still features, such as sound, that couldn’t be as easily handled through a character stream.
The approach I took was for the back-end to generate a JSON array as a string. Each element of the array could contain a different action.
I could then generate a range of actions, including things such as:
FreeText - can just rendered to the front-end output device directly
Debug - any debug information that I did not want to render to the front-end
Pause - a command instructing the front-end to pause for a number of milliseconds
Save - an array of bytes containing the data to write to a save file
Beep - the pictch and duration of a sound
Admittedly, my Quill Player is an interpreter, not a compiler but you may be able to apply similar concepts to the I/O runtime supporting the compiled code.
The advantage of course of using a JSON format is that the front-end component can take those elements that it likes and ignore the rest. My Javascript-based web interface outputs the FreeText onto a display canvas, outputs the Debug into the Javascript console and when seeing Save allows the user to select a filename to save to. At the same time I’ve developed a command-line interface using the same back-end (handy as I can write and test things easily on the move on a mobile phone). This can take the same output format and render it appropriately to this interface. Both interfaces currently ignore the ‘Beep’ command, but in principle the web interface could use this to generate sound. In the future, this is easy to extend. I may at some point get around to extracting Illustrator-format graphics and then I can easily add these to the output actions. The front-end components will ignore actions that they don’t understand, so the command line interface would carry on working as usual and just ignore the graphics commands.
I’m not so familiar with generating WebAssembly from C. I’d recommend Rust to anyone that hasn’t tried it yet, although I appreciate that there is quite a learning curve and it’s not really as mainstream as C yet. But it’s got great support for JSON; just include the ‘serde’ library, annotate output data structures with the [Serialize] attribute and it will automatically generate JSON from them.
I’ve thought a lot about it, which is paradoxically why it’s taken so long to reply. There’s a lot to summarize.
In the long term I’d like it to be pluggable, using the WebAssembly dynamic linking mechanism if possible. For the short term, I’ll just support WASI, which is an immature libc-like standard interface that is supported by the major standalone WebAssembly engines but not yet by any browsers.
The mid term is the most challenging. It’s not currently possible for WebAssembly running on the main browser thread to block or yield. There are several proposals in the pipeline to address this, but it’s impossible to say how long that will take or what it will look like.
Emscripten, the C/C++ compiler to WebAssembly, uses a few different techniques to work around this limitation. They’re all based on the fact that the WebAssembly module’s memory stays the same after the code is finished running and code can be called again, reminiscent of TSRs back in the DOS days. We can emulate what Emscripten does with some challenge, and may have to in order to implement saving anyway.
Implementing a Vorple-like system where the Inform code generates JavaScript to run is fairly straightforward. You provide a function for the code to import that just takes a string and evals it. The only complication is that there isn’t a standardized way to pass strings, so you have to take a pointer, fish the characters out of memory, and turn it into a JavaScript string yourself.
Similarly, it should be possible for the Inform-produced module to import an interface source compatible with the one used by Glulx Inform.
Being able to call browser APIs directly (instead of through JavaScript) is an eventual goal for WebAssembly. It will still take some work on the Inform side to enable that.
Overall, I think there is a lot of opportunity to be flexible both on the author and on the users’ sides.
Thanks for the detailed reply. This is obviously a complicated and evolving area.
For a GUI-based IF interpreter, you generally want the VM to run in a background thread while the UI runs on the main thread. Is that a sensible approach here?
(I get that there’s a lot of use cases that aren’t “compile an IF game and then play it”, but I’m focusing on that one for now.)
I’d like to chip in a couple of points that might be helpful;
I’m currently building interpreters for webassembly via emscripten. I’m not generating the WASM directly, only via transpiled C/C++.
I have had the same problems with how to block the interpreter and other I/O issues:
My architecture separates the GUI from the interpreter, usually by running the “back-end” in another thread. I could not get threads to work under emscripten. They are meant to work(?), but i had all sorts of problems and they didn’t work for me.
I reworked my front/back architecture to optionally support co-op multi-tasking. ie co-routine yielding.
Turns out emscripten supports co-routines as “fibers”. This worked well for me with WASM.
emscripten comes with simple file IO that appears to live in the download “package”, not sure where writes end up. Browser storage??
Initially, i had everything in the same package. Obviously this was bad for the initial download size. Especially if you have a selection of fonts and worse when you have sounds and pictures. Basically you need async fetching!
This was fixed when i switched to “sokol” as my renderer interface where now i use the sokol “fetch” interface, which i think calls out to JS to provide async fetching IO. Now i can demand load fonts, pictures etc.
sokol also fixed text input from mobile. I had a problem where it would not pop up the virtual keyboard, making it basically broken on mobile. I mention this as this is a common problem!
I’ve found Asyncify works quite well for blocking the interpreter. In Emglken there’s only one actual async function, getc, but dozens or hundreds of other functions which call it also need to be handled. Emscripten takes care of that for you.
And it might be possible to run the Asyncify algorithm on wasm code generated by Rust?
Emglken also has a custom file system which then access GlkOte’s Dialog library. Writing custom file systems would be possible for other storage models. But if you’re writing your own IO with Rust then you could just do whatever you want, it would be even simpler than this.
Thread support under Emscripten is getting better. Until fairly recently you had to enable experimental options in some browsers, but that’s not necessary anymore. Still, the architecture of the web means that most web API calls can only be done on the main thread, so your have to arrange for that to happen either in your C or JavaScript code.
Theoretically you can run binaryen’s standalone wasm-opt tool in asyncify mode on it. As a practical matter, how well that will work depends on whether binaryen produces the WebAssembly constructs used by the Rust compiler. I’ve never seen it have a problem with any WebAssembly 1.0 instructions, but newer ones can come out of wasm-opt severely suboptimal or wrong.
For example:
(The code in question is almost verbatim from an Inform 6 for loop.)
Of course, there’s nothing stopping the Rust compiler from implementing Asyncify itself. The algorithm is pretty straightforward.
