Folly - a Z-machine with handwritten input

Playing Project Announcements
,
1

Folly is a Z-machine interpreter I’ve been working on for the reMarkable tablet. It’s designed to look as much as possible like a printed book – and you handwrite your input directly on the page, like so:

folly-screenshot
folly-screenshot1137×877 51.1 KB

You could think of it as a sort of interactive e-reader: a lightweight, text-centric interface, organizing contents into pages instead of the classic infinite scroll. I’ve made some effort to map Z-machine conventions over to typographic ones – rendering the status line as a running head, putting block quotations inline with the text, etc. I’m naturally rather biased, but I find it to be a pleasantly calm and immersive way to play IF; e-ink is nicer to read on than a glowing screen, and on a tablet / stylus the handwritten commands feel natural and direct.

The project is still in alpha, and installing custom software on the reMarkable takes a bit of hacking… but if you have the hardware, some comfort with the command line, and an appetite for early adoption, I’d certainly be interested in what you think of it! Releases are available on Github, and you’re welcome to DM me for help getting set up.

Technical details

The actual Z-machine powering the thing is a fork of Encrusted, which was discussed here a few years back. The fork adds support for higher Z-machine versions (v3-5 and v8), fixes a few bugs, and makes it easier to embed as a library in another app. I imagine nobody is wildly excited by yet another Z-machine implementation, but it might be useful for anyone else who wants to hack on an IF project in Rust.

Folly grabs some metadata from the Z-code to help process input – it uses the game dictionary to help steer the handwriting recognition towards more likely words, and the “extra unicode characters” in the header are available in the onscreen keyboard. It also uses some heuristics to clean up and structure the output text; extracting status lines and quote boxes from the upper window, or stripping the > so we can display it in the margin. This is necessarily a bit brittle, but the patterns are consistent enough across games that it works well for the vast majority of story files I’ve tried.

Most of the rest is UI code; the existing library support for the tablet was pretty limited, so everything from the text layout and handwriting recognition is custom-built. Some technical problems (and many bugs) but not super IF-relevant.

Screenshots

char-input
char-input1404×1872 73.7 KB
game-play
game-play1404×1872 195 KB
game-menu
game-menu1404×1872 112 KB

Questions I have

If you have thoughts or insight, I’d be particularly interested to hear from you about:

  • Anything like this been done before that I should know about? Handwritten input for IF, book-like interfaces, or anything that presents Z-machine content in unfamiliar ways.
  • What game should I bundle with a public release? If Folly is going to be accessible to those who don’t already have a large personal collection of Z-code files, it would be nice to bundle a story or two with the app itself. Desiderata include:
    • Available or compilable as Z-code.
    • An explicit and permissive license. (Many games are on IFDB as freeware, but it’s often not clear to what the exact terms are and whether stuff like redistribution is okay.)
    • Relatively friendly to beginners.
    • “How to play IF” content is awesome, but less so if it refers too heavily to the keyboard. (“Type x me and press return” is helpful at a keyboard but actively confusing if you expect to be writing something out with the pen.)
  • What games should I test with? Praxix etc. are passing, but don’t tend to stress the upper window much, and that’s where the most fragile UI code in Folly is.

Thanks!

21 Likes
2

This is a very cool hack.

The Z-machine on a Livescribe pen is some previous art on handwritten input.

4 Likes
3

This looks fascinating! And it also raises a lot of questions.

Bronze is always the game I recommend for this. The manual describes its license. Elsewhere I thought I saw Emily Short say that it’s available under a Creative Commons license, but I can’t find any direct evidence of that in the manual or in the web site containing its source code.

But… can your app do HTTP requests? If so, you might as well have the player download games directly from the IF Archive, ensuring that you’re not copying them yourself, hopefully circumnavigating any copyright risk for any of these unspecifically “freeware” games.

What would you expect players to do if the game prompts the player to press a key? For example, in your first screenshot of Curses, it asks the player to “press SPACE to begin.” How would the player do that with the pen?

I see you’ve also provided a screenshot of the “HELP” menu in Curses, which raises a similar question for me: how would the player interact with that menu using a pen? (Worse, most games ask players to type “HELP” at the start of the game to get help, dumping them into that menu system; the worst case for newbies.)

4 Likes
4

Not to derail, but have you tried proposing some of the enhancements you gave to encrusted back upstream? I’d love to be able to play some Z8 games with the Web encrusted version!

5

This is neat – thanks! Quite charming that you end up with a physical transcript at the end of a session.

Thank you - I’ll look into it!

Playing through the first few moves with novice mode on, I can see it suggests commands but doesn’t go into detail on how to enter them. Great!

Not yet but I could teach it! I’d been hoping to cut out the extra step, but it might be a useful feature anyways… more direct than downloading a game just to transfer it to the tablet. Will consider.

Yeah, this has been an annoying problem.

For the menu screenshot, you can see that it’s basically thrown up its hands and put up an onscreen keyboard, which can be tapped with pen or finger. In the first Curses screenshot, it’s popped up a single [ space ] button for the user, since typically when read_char is used in that context that’s the expected input. But the keyboard can be toggled from the button in the margin if that’s not right.

The keyboard is literally the last thing I’ve put in, since it seems like something of an admission of failure. But it does at least ensure that the user never gets stuck, and it should only be necessary to call up the keyboard where the handwriting recognition or the heuristics aren’t working. (In the menu case, for example, I should be able to autodetect the menu items and just let folks tap on the one they want… which is what literally everyone who has tried this for the first time tries to do anyways! We’ll see.)

The reason the tutorial content makes me nervous is because if it refers to a keyboard in a context where Folly “wants” you to use handwriting input… either it just confuses, or users think they need to use the keyboard for this part for some reason, and either way that’s a bad experience.

Does that all make sense? Definitely something I’ve gone back and forth over during development…

Yes - but unfortunately the original project seems dead, and I haven’t been able to get in contact with the author via email.

I’ve tried to structure the fork so that it would be possible to get the cool web stuff from the original project working again, but I haven’t been interested in doing it myself. If anyone’s interested in trying that out, though, please let me know… would be happy to try and make that work!

2 Likes
6

This is incredibly cool!

So wait, can people write apps for the remarkable? Any info on how?

1 Like
7

Yes, though note that it’s very much a “homebrew” software environment: it’s not officially supported by the reMarkable folks, though they don’t discourage it and have provided a toolchain etc. for the community to use. There’s a bunch of resources on the community wiki, a pretty active Discord server, and a package manager for existing apps and tools.

4 Likes
8

Thanks a lot for the info.

1 Like
9

For a book interface, the Silverlight implementation of Jack Toresal and The Secret Letter was done like a book. I just confirmed can still run it in Internet Explorer on Windows, but I don’t think any modern browser supports Silverlight. After you unblock the zip file, run the MSI file. You’ll get a Textfyre folder in the start menu and run the app. You’ll get asked by Windows to pick a browser for .htm files and you need to choose Internet Explorer here and accept the ActiveX plugin. (relax - there are no trojans here).

2 Likes
10

Thanks for the link! Haven’t managed to get this running on Linux yet, but the screenshots help give an impression at least.

For Folly, the bookish interface is something of a necessity. E-ink has a very low refresh rate, and filling a page fill top-to-bottom involves a lot less redrawing than to have a whole page scroll upwards on every new line. Elements that can change size, like the upper window, are also annoying to manage if you don’t want to reflow text. And once you’ve done all that, typography history has lots of tricks for making a page-based interface information dense and readable. (One nice thing: old pages have the contents of the status line at the time, so as you flip back and forth you preserve the context which gets lost in a typical IF transcript.)

I suspect the book metaphor makes less sense on a PC, where the same constraints don’t apply. Less certain about devices like the iPad which live somewhere in between. My impression is that handwriting recognition would still be really positive – if you’re willing to require a stylus, which may limit the audience; little tweaks like inlining block quotations and pushing the > into the margin are probably also still good for readability; but infinite scroll is probably more natural there.