Inform 7 v10.1.0 is now open-source

Inform is a design system for interactive fiction based on natural language, and consists of a core compiler, together with extensions, kits and other resources, a number of outlying tools, and documentation, along with applications presenting the system in a friendly way on MacOS, Windows and Linux. This software had been used extensively since 28 April 2006, but by 2016 its source code was in considerable need of modernisation. In part that was wear-and-tear, but it was also the effect of years of experiment in which the code was often built without a full understanding of the concepts it was groping towards. In early 2016, then, a substantial work of renovation began. That work is now essentially complete, and the first results can be seen. The git repository “GitHub - ganelson/inform: The core software distribution for the Inform 7 programming language.” became public today, presenting the complete source and extensive technical documentation. And with that, the whole system was placed under the highly permissive Artistic License 2.0. This is an open-source licence recognised as such by, for example, the Free Software Foundation.

The Inform presented in the repository, 10.1.0 (see below for a note on version numbering), is not yet reflected in the three apps. Users can build the software and use it on the command line, but not yet in the familiar Inform UI. Philip Chimento (Linux), David Kinder (Windows) and Toby Nelson (MacOS) hope to release their respective apps in the week of Monday 16 May, if all goes well. Even then, these apps will feature the ability to select the version of Inform used, so that if v10.1.0 does not agree with your existing Inform project, you can easily revert that project to use 9.1, 9.2 or 9.3. So the rollout of the new Inform is ongoing rather than complete, and at this point we deserve only the five-minute music dance experience, not the waffle party.

The overhaul of Inform had three goals: to improve the clarity of code throughout, increasing consistency, reliability and test coverage; to document its workings in depth, enabling publication of the compiler and its satellite software as “literate programs”; and to build a more ambitious infrastructure for Inform to relate to software other than the project at hand, with a new build manager (“inbuild”) at the top level, and a new intermediate representation called “Inter” together with a new pipeline architecture for linking, optimisation and code-generation.

In the longer term, this may all prove to be quite significant. But it was a non-goal to expand or substantially change the Inform language itself. This is not because its design is frozen, but because the open-sourcing and renovation were an essential first step to further development. In the short term, then, most users should find the experience of using Inform about the same. But for some, there will likely be a little disruption, and especially for users of low-level extensions to what Inform does, which worked by what we used to call “template hacking”: that may especially affect non-English IF writers using language extensions, or ambitious projects such as Vorple. Apologies for that. But we expect this disruption to be fairly short-lived, and in the mean time, as noted above, no user will be stranded by upgrading, because it’s still possible to keep using 9.3 for now.

Publication of the “literate program” of Inform, another long-cherished aim, may offer a sense of how the methodology of “literate programming”, developed by Donald Knuth in the 1970s and 80s, might feasibly be used for medium-scale projects by the standards of today’s world. Inform weighs in at around 300,000 to 400,000 source lines of code (depending on definitions, as always with SLOC measures), and it is a project with real users and needs. This makes it a real-world rather than a toy example, and one which seems to be the largest “literate program” ever completed. Publishing it certainly required better scalable LP tools than have previously existed, and for those tools see the repository at “ganelson/inweb”, also public as of today. (“inweb” plays a role in managing the Inform repository’s non-trivial build process: so if you want to build a command-line Inform from source, you must first build inweb. But the instructions are, we hope, relatively quick and easy.) The third and final repository now opening is “ganelson/intest”, a flexible command-line tool for testing other command-line tools, which includes its own scripting language for test suites, “Delia”.

In literate programming, a program is both computer-readable and, in a different format, human-readable. For Inform that human-readable format is a website, rather than a printed book: it is part of the inform repository, alongside the codebase, and is served by Github Pages. (That website dynamically changes with the program, so that it will not fall out of date the way almost all technical documentation of large projects does.) In one sense this website makes for a varied sort of read. All kinds of fruity-for-a-compiler techniques turn up, from dimensional calculations in physics (Inform can resolve the “volt” in terms of SI units, for example) to Conway’s algorithm for English pluralisation. (A recent bug fix: the plural of “human” is “humans”, not “humen”.) In another sense, of course, the website is an endless morass of self-similar C code, and readers will no more read it cover-to-cover than they would an encyclopaedia. The idea is rather that anyone interested in what Inform is doing, and why it was set up that way, can get both an overview and a detailed presentation of literally all its functions.

The literate program is presented as a companion website to the code itself, and is part of the same repository, served by Github Pages and automatically updated as the software updates.

Turning now to the compiler tool-chain, the transition to the Inter IR was by far the largest single architectural change in Inform’s history. It enables entirely new developments, from peephole optimisation and other compilery recreations to the ability to have Inform compile its natural-language source down to C code: this in turn can then be compiled by clang or gcc to a stand-alone executable, or to code which can be linked into Unity projects, iOS apps or similar. It thus becomes possible, and we hope even convenient, for choice-based games for console, mobile or desktop to use Inform, if they wish, purely for world-modelling, conversation, or dynamic text generation. Such stories need not in any way resemble traditional command-parser IF. A number of simple demonstration projects show Inform source linked with C, and demonstrate straightforward communication between Inform phrases and C functions.

Inter opens other possible doors, too, and is essentially a programming language in its own right, with its own documentation, and with a command-line tool (called just “inter”) acting as a Swiss Army knife. In particular low-level IF code written in Inform 6 (the final version of Inform’s predecessor language, born on 28 April 1993) can be compiled by “inter” into what are called “kits” of Inter code. (In this way, even code which began as Inform 6 syntax can find itself translated to C: or, after a picturesque journey, can be recompiled back into Inform 6.) Kits are a new concept to Inform, and are a low-level analogue of Inform’s “extensions”. Both are libraries of code, but where extensions are high-level and written in natural language, kits are low-level and better reserved for close-to-the-metal business. Besides creating Inter code, kits can also contribute new “kinds” to the main compiler, using a formal mini-language for that purpose called “Neptune”.

Inform is supplied with several kits built in, notably BasicInformKit, WorldModelKit, CommandParserKit and EnglishLanguageKit. This is somewhat more modular than the “template files” system which existed before. In a complementary way, the old Standard Rules extension has divided into two, Basic Inform (which defines the language itself) and the Standard Rules (which defines the IF-style world model). It is now possible to have Inform projects which run Basic Inform only, a form of the language without rooms, things, vehicles, the command parser, actions, and so on, but still having phrases, rules, lists and the like: in effect, Basic Inform is a general-purpose programming language.

At the other end of the compiler - the front door, rather than the back - is a new build/package manager, “inbuild”. Though available as a command-line tool, where it can perform mildly entertaining party tricks such as printing out a dependency graph of all the extensions needed by a project (and their local availability), and can incrementally invoke “inter” to build kits as necessary, “inbuild” will mostly be invisible to the end user. Somebody clicking Go in the Inform apps is running “inbuild”, but without realising it.

The real point of “inbuild”, though, is to pave the way for a better collective approach to hosting and sharing Inform resources, and especially extensions. “inbuild” can manage multiple versions of an extension, using semantic version numbers to work out what is compatible with what; in future, it may acquire the ability to download these, perhaps from tagged release points at Github repositories. We are not there yet, but the present Public Library approach to sharing popular extensions (or kits, or website templates) may move to something more like a community registry, with “inbuild” playing a similar role to tools like “cargo” (Rust) or “swiftpm” (Swift). Looking at “inbuild” today, it seems over-engineered for what it does. These future aspirations are why.

A few other tools make their first public appearance today, too, notably “indoc”, which generates Inform’s documentation as web pages, in-app web pages, or ePub books.

Bug-tracking for the Inform project has also been modernised, with a new tracker set up at the Atlassian website and running under the popular Jira software. (By “popular”, I mean “widely used”: I wouldn’t wish to imply that anyone has a kind word to say about it.) In fact, this tracker is old as well as new. Hugo Labrande undertook the huge labour of migrating data from the old Mantis bug tracker, which ran on a private server and had become unusable because it was written in pHp now considered unsafe by private-server-hosting companies. The whole corpus of Mantis bug reports was preserved, including their commentary. Numerous bugs reported since the last release have been fixed, and others will be, but we are no longer going to have a “zero open bug reports or we don’t release” policy, in so far as we ever did: the project is just too big for that. We are all grateful to Brian Rushton, too, who has become the Inform project’s new bug-report curator, and has put in a great deal of time to get things in order.

A note on version-numbering, as promised above. Inform is sometimes called “Inform 7”, since it succeeded a lower-level C-like language whose last version was “Inform 6”. We now usually call it simply Inform, since the “7” is not meaningfully a version number any longer, except to make clear that we don’t mean I6. Instead, as the language matured in use, its successive releases were identified by four-character build codes with the shape “NLNN”. Those build codes still exist internally (as I type this, we’re at 6U93) but we have moved to semantic version numbers for the compiler, as for all other components of Inform. Past builds have been retrospectively version-numbered: so we can now see that 6M62 (December 2015) was really version 9.3 all along. See the file inform7/versions.txt for the full correspondence, if it really matters.

Finally, anyone still reading this far down the post may be asking: so what happens next? As noted above, the UI apps will be released in May, and at that point real users other than command-line hackers will be running the new code live. We have yet to decide what our release cadence will be, or whether we will in any formal way have “stable” and “unstable” builds, but of course anyone is free to pull from the repositories. At present we would prefer not to receive pull requests from outside the Inform team for new features or changes other than clear-cut bug or typo fixes; we have been focused more on getting to this point than in the world which comes after.

It remains only to thank the many people who have got us to this point: Emily Short and Andrew Plotkin, to whom Inform 7 was dedicated, and who still deserve that dedication; Philip Chimento, David Kinder and Toby Nelson, who have been tireless in working not only on the GUI apps but also in helping make the build apparatus work on multiple platforms, and coaxing the 2500-case test suite to a point where it more or less works across the board (right now, everything passes, but the last 5 cases to fix only began working on Windows four hours ago); Hugo Labrande and Brian Rushton, for their work on the bug tracker; Jason McIntosh, Liza Daly, Judith Pintar, Jacqueline Lott, and other fine folk at the Interactive Fiction Technology Foundation, and especially all those organising and taking part in the Narrascope conferences; Dannii Willis, Juhana Leinonen and other users helping to make Inform do more than it thought it could; Jimmy Maher and especially Aaron A. Reed, for bringing so many people into the tent; Sam Ashwell, Dan Schmidt and many other hospitable interlocutors over the years; Chris Klimas, who may be surprised to learn that he is something of an inspirational figure to me, in terms of how to bring a community along with software people are truly switched on by; the 482 people doggedly following me on Github in spite of the fact that, until today, I had not one public repository; and literally thousands of others who have commented or reported on Inform’s foibles since it began on 28 April 1993 - twenty-nine years ago today.

Graham Nelson, 28 April 2022

137 Likes

V. good. :)

21 Likes

Good things come to those who wait. (Or, as we say in German, “Was lange währt, wird endlich gut.”) This is an amazing accomplishment, and I’m excited to, at long last, take a peek into the compiler’s source code.

I’ll spare all of you the remainder of the “deep speech” stuff I’m not particularly good at, and simply say: thank you for all the work you have put into Inform over all those years.

11 Likes

Man, this is setting quite a high bar for first posts on IntFic!

Congratulations to Graham and all who helped get to today; I’m barely technically literate but am still aware this was quite the achievement. Despite my lack of context, I’m still enjoying dipping into the literate programming presentation – for those looking for it, the quick link is Inform: The Program.

25 Likes

I cannot wait to spend countless hours sifting through this; I’ve been fidgeting on the edge of my seat since you mentioned this happening today in email :sweat_smile:

7 Likes

This link seems to be offline.

2 Likes

Congrats on a tremendous accomplishment. Sometimes the hardest work is the least visible and most necessary.

6 Likes

The docs link is supposed to be https://ganelson.github.io/inform/.

7 Likes

It will be a lot of movement here from now on.
Every author, programmer, enthusiast will be offroad running and examining the code.

Congratulations.

5 Likes

Huh, weird, that appeared to be the link earlier! Dunno if things have moved around or I just messed something up – I noticed another broken link, where the LP page on extensions links to a Github 404 (I believe there are supposed to be easy ways to tag issues on Github but I don’t know how to do that so hopefully posting here is at least minimally helpful).

3 Likes

Github has little workers compiling the pages, so after a code change there may be a short non-availability while they are replaced.

I’m not an Inform fan but I’d like to say well done to everyone who had a part to play. This is a significant achievement and will reassure a lot of Inform authors.

2 Likes

I think secretly Graham saw that I followed him on Github the other day and he thought it would be courteous to have some public repos :).

But seriously, many thanks to Graham and the community.

6 Likes

And what a nice gesture, to decide to log in to the forum after a decade’s absence.
Kind of adds to the occasion :smile:

5 Likes

Thank you.

1 Like

Being a lifelong introvert, I like to keep my social media interactions evenly spaced over decade time spans. :smiley:

6 Likes

I was hoping that’s what the sudden burst of recent private github activity meant. Yippee!

7 Likes

That is kind of frustrating from github. Not showing activity when it’s on a private repo. They should just hint with shades of grey or something.

1 Like

Congratulations! Here’s to the next era of Inform!

4 Likes

This is very exciting! Have been looking forward to this.

2 Likes

I’ve recently visited some folks’ GitHub pages and saw something like “there were 20 commits to a private repo” - but I think we were in the same organization.

1 Like