I have been reading through some of the threads about parser fiction, specifically those about debugging parser code, or big MUD systems. I have never fallen in love with parser fiction, but I am intensely intrigued by their functioning. Especially reading about Inform 7, something is just not quite clicking yet in trying to analyse how it might work under the hood.
Now, I could go and Google it, and I have, and keep finding threads left and right that just suck me into one rabbit hole after another, so I am just going to ask here: From an architectural standpoint, how do parser systems generally work, and how are they generally programmed? Because it seems like there is some really interesting things to be learned from it.
You might find Notes from the Dawn of Time by Richard Bartle worth reading. Several of the articles (from âIntroduction to Parsingâ to âA Complete Exampleâ) discuss parsers in general and his MUD parser in particular. I took a lot of inspiration from it while developing my own system.
Having just gone through this process, I have good insight and tons of documentation on the Sharpee GitHub repository.
There are several elements of parser-if that all align.
Grammar - the system needs a way tokenize commands and make basic sense of the command and the tokens.
GIVE RED APPLE TO SNOW WHITE
action/word phrase/to/word phrase
This we store as the Unvalidated Command.
The parser picks up from here and tries to identify the action and slot the two word phrases if it knows the action. The two word phrases have to resolve to objects in the world the author has defined.
There will be an object âappleâ with an adjective âredâ. And another for âsnow whiteâ.
If everything is confirmed, we now have a Validated Command.
Now the tricky part. We now have a Command Validator that verifies those two objects are within reach and nearby.
Then we have a standard library of action processors that take the validated command and mutate the world model. In this case the red apple object is moved from the player to the Snow White NPC.
Then some reporting of what happened is either directly printed or in Sharpeeâs case a message is created with the details of what happened.
All parser IF works generally the same. Some of the above logic overlaps in different platforms. Meaning the grammar, parser, world validation, scope validation, and mutation are in different places in different systems. But it all comes out the same.
The parser IFs have internally rooms, items and characters. The playerâs input is either primitively interpreted as VERB OBJECT (2-word parser) or checked with pattern matching. Another thing which is quite useful is the virtual machine (VM). So the game logic and the operating system specific stuff are separated. The author needs to compile his game only once for all platforms. And the player needs just one program to run all games of the specific parser.
Fascinating! So, what seems to be implied is that a number of systems seem have some library of how NOUNS VERB in different ways and how they alter the world state, or am I off base there?
As @DavidC mentions, every parser system Iâve seen has a set of grammar definitions which match a sequence of tokens to a game action. The tokens can be either literal words or other things which the parser knows about like âname of an object in the gameâ. So:
PUT DOWN (object) â [âDROPâ action]
PUT (object) IN (object) â [âINSERTâ action]
Any sufficiently fully-featured parser system will have a way to modify or add to the grammar lines (if your game uses a new verb, for example) and in many cases a way to provide code for parsing new types of token, e.g. so that you can add a spell-casting system and then create a âspellâ token to use in a grammar line like âCAST (spell)â or âCAST (spell) ON (object)â.
The most common token is probably âname of an object that the player can seeâ. Many systems just give each object a list of associated words and match any number of them in any order (so if you create a BIG RED BALL, the player may be able to pick it up by typing GET BALL BALL RED BALL BIG RED BIG). Usually you want some way to refine this on an object-by-object basis, so that if you have a CAR and a CAR KEY, you can insist that the word KEY has to appear for the input to match the car key. (This is contentious, though, because some players expect any one word in the objectâs name to be enough to identify it.) But this brings me to my next point:
A lot of the time, the input will be ambiguous, so a significant part of sensible parser design is working out which interpretations are most likely. Your parser will probably have to iterate over various ways of matching the input against grammar lines and score them somehow. A classic example (assuming that your grammar includes both GIVE THING TO PERSON and GIVE PERSON THING) is GIVE HARPER LEE TO KILL A MOCKINGBIRD. To support this, some systems allow you to use special tokens like:
GIVE (inanimate object, preferably held) TO (person)
GIVE (person) (inanimate object, preferably held)
to help provide cues for the parser about which interpretation to take. Generally the story author should also be able to provide hints; in Inform 7, for example, you can include âdoes the player meanâ rules which allow you to flag actions as likely or unlikely:
Does the player mean taking something which is fixed in place: it is unlikely.
You also have to consider how the parser will provide useful error messages when the input doesnât match a grammar line. For example, say the player types PUT BOOK. This will fail to match PUT (object) DOWN and PUT (object) IN (object), but it also failed to match TAKE (object) or INVENTORY or WAKE UP. So you need some way to record the fact that the input was a âcloserâ match to the PUT grammar lines and then construct a meaningful response along the lines of âI only understood you as far as wanting to put the book somewhereâ.
Iâd say the right level of generalization (not too broad, not too specific) is:
Generally, a parser IF system has a list of patterns that it tries to match the playerâs input against. The game then takes the matched pattern and changes the world state based on it.
In the simplest systems, like the original Adventure, or the Scott Adams games, these patterns only have two pieces, so theyâre called âtwo-word parsersâ. In Adventure, one of these patterns might be [GET] [ROD], where [GET] matches GET, TAKE, GRAB, etc, and [ROD] matches ROD, METAL, RUSTY, STAR, etc. When the player types TAKE RUSTY, that pattern is matched, so the game rules then figure out what to do about it (check if the rod is in the location, and if it is, move it to the playerâs inventory).
More complex systems will have more complex patterns, like show [OBJECT] to [PERSON]. How exactly they match the playerâs input against this pattern can be complicatedâhow about SHOW MY COPY OF TO KILL A MOCKINGBIRD TO THE GUIDE TO THE MUSEUM?âbut in the end, the parser finds the pattern that matches, hands that to the game code, and the game takes it from there.
(For a pathological case, look at The Vambrace of Destiny, where each pattern is a single keypress! But that âpatternâ is dutifully passed through to the game rules, where itâs processed just like in any other parser game.)
I would say Dialogâs is one of the best out there at the moment, because itâs very good at handling ambiguity, and itâs very transparent for authors to mess with. But Informâs, for example, is better at handling units, punctuation marks, and real numbers (anything except whole words and natural numbers). For the most part, these parsers donât actually use standard algorithms like Earley or CKY, because theyâre overkill for this purpose; theyâre usually crafted by hand instead.
Okay, Iâm starting to catch on, I think. This may be a gross oversimplification, but it seems much of the parser fiction discussions I see are mostly about solving puzzles, and finding out what the parser actually responds to. Is the implied enjoyment there about prodding the world and see what the creator accounted for?
Though that may just be what is over reported on, because puzzles is what people get stuck on. What are some very strong applications of it? Iâve seen some classic fantasy applications, but is there a generally accepted prime example of immersive parser fiction?
I would say the big draw of parser games is systematicity. There are a huge number of options available in your average parser gameâeven in something like The Vambrace of Destiny, which completely eliminates combinatorial explosion, you end up with 26 options on every page! That many options in a choice game would get overwhelming.
But, the vast majority of them are useless in any given situation. Leaving all those hundreds of options implicit means players can think about which ones are relevant. A lamp can be picked up, dropped, turned on or off, but probably not climbed. Thatâs where a lot of the fun of the puzzles comes from: you know that TAKE works in this game, you know the lamp is portable, you infer you can pick it up.
Or, for a more real example: you learn a magic word that causes a treasure to teleport back to its starting location. You find a troll who will let you cross a bridge if you pay him a treasure. You can use the magic word to cheat the troll, which tends to give a satisfying âaha!â moment thatâs more subdued when the magic word is a prominent link on every page.
And then the followup: you need to go back across the bridge before cheating the troll, otherwise he demands another treasure!
The satisfaction comes from seeing something that works in a systematic way, learning that system, and applying it to your advantage. Whether thatâs magic words, or TAKE and DROP, or hacking computers. Parsers are very good at this kind of systematicity, and most parser systems have it built in at a very deep level.
As Sharpee has demonstrated, the VM aspect is no longer necessary. Using a cross-platform scripting language like Typescript allows Sharpee to be run anywhere with the same code.
I donât plan to offer a CLI and if authors want a stand-alone Electron app Iâll provide instructions, but Sharpee wonât provide the build mechanism. Running a Node app can allow for attack vectors. Iâve built an official Tauri story runner (Zifmia) to alleviate these concerns. One install runs all Sharpee stories. Or authors can publish to a simple web page that doesnât require a server. Saves are stored in LocalStorage. The browser is a convenient sandbox.
Isnât the scripting environment still serving the same purpose as the Z-machine or TADS interpreter, though? Youâre still producing more abstracted, platform-agnostic code thatâs then interpreted by different runners on each platform.
âParser gamesâ are really several ideas tangled up together. Thereâs the method of input per se, which gives the style its name. Thereâs the world model, i.e. the idea of rooms which the player can wander between, with items they can move about (a âmedium-sized dry goods simulatorâ). There are puzzles. And thereâs textual output, of course. None of these except the parser itself are specific to parser games, and actually a given parser game wouldnât engage in all of these to the same degree.
The way you phrased your initial question, the parser itself is almost irrelevant. That is, the parser is a minor, peripheral part of the architecture of a parser game. The key part is implementing the world that the playerâs commands act on. The most basic world model is a set of rooms and connections between them, and almost certainly a set of items which can be in a room, or carried by the player. Beyond this, it could all be smoke and mirrors! Some microcomputer games tried to fake even those things (with partial success).
This is the amazing thing about parser games: they commit to printing natural language descriptions of the world and your actions, and they commit to taking natural language commands, and thatâs it. Inside the black box, the game might be a tangle of if statements and global variables to track whether you left the door open and the oven on, or it might be the most elegant object-oriented Prolog-based rule thingy you ever saw. As long as it can print a reasonable response to KISS GRANDMA, both are equivalent. Any complexity in the implementation is purely an attempt to make programming those consistent responses easier.
Put another way, the same parser that can recognize REMEMBER INSERTING TWELVE COPPER PENNIES INTO COIN ACCEPTOR also recognizes LOOK. Itâs a totally upwards-compatible style of input and output. That is, any natural-language command is conceivable input for a parser game of any era. The puniest parser is equivalent to the best, until the game actually needs to accept a particular complex command.
What this means for the game is that it becomes a continual conversation between the author and the player, in which they alternately affirm that they have the same idea about the range of commands in this game, about the extent to which various aspects of the world are modeled. So I think youâre quite right that itâs about solving puzzles and finding out what the parser actually responds to, but less in the way of whittling down the possibilities than in the way of building them up. Itâs sort of like a crossword puzzle in that your entire knowledge of the world is potentially in play, and like a good crossword puzzle in that the author got into your head while writing, so that you can get into theirs while playing.
A big boon that authors have now is that systems like Inform, Dialog, and TADS offer quite a few facilities to elevate the consistency of the game above the lowest smoke-and-mirrors level. Rooms, connections between, items which can be carried, but also items which can contain other items, topics of conversation with NPCs, light, darkness, smells. But the smoke and mirrors are still there at the edges of any game, because not everything needs to be âfullyâ implemented, whatever that would mean. I would argue this is as characteristic of how parser games are programmed as any complex system a strong parser might have.
There are two sides to this, with the second side having evolved after years of dungeon crawls and puzzle-fests.
Narrative heavy storytelling emerged and the even the puzzles became more about story and less about treasures.
One of the styles of parser IF that stands out is system-centric. The author invented a set of physics that work regardless of the objects in the story, then implemented challenges based on the interactions of system and objects. Savior Faire, Haden Lands, and Galatea all revolve around a core system. This gives the player the wonderful challenge of pairing the story world with actions.
But the evolution of narrative led to stories on rails and even though that can be really boring, a good writer can capture things an open world system cannot. Photopia is a perfect example.
Puzzles are a big part of parser IF, but less today than 25 years ago.
I think that âpuzzlesâ as a concept kinda naturally arise out of the idea of an âadventure gameâ where the actions that a player might take are broad in scope, rather than being limited to a specific mechanic in the way that a shooter or platformer or whatever is.
I mean, if you want a game where you progress by doing a wide range of actions reflective of the wide range of actions that a human might do while experiencing an âadventure,â then the obvious problem is that there necessarily will, at any given time, be specific actions necessary in order to progress. So the player needs to be able to determine what those actions are, but you donât want to just directly tell the player what to do because thatâs boring, so you need to give them just the right amount of information to figure it out themselves, and congrats, youâve discovered Puzzlesâ˘.