Progress of the Parser

Technical Development
1

Here’s one for the more technical here to comment on, although i’m curious to get everyone’s thoughts on this (technical or otherwise).

A bit of scene setting first. The ZIL language, parser and all, and the Z-Machine were created circa 1980 so that’s now 40 years ago (yes, we are that old!) In the interview between Kevin Savetz and Steve Meretzky earlier this year, SEM was asked about the continued usage and his response was along the lines of “surprising, hard to imgaine that that is still the best way to make text games…” In the Get Lamp interview with Marc Blank, Blank states that “look at everything that’s happened in technology and yet this thing is still being used!”

As i’m typing away in Notepad++ and creating my ZIL, it actually strikes me that maybe at “ground level”; ie tools that can be used by any person, free or purchased, and not including AI software being created/used by games companies, that the parser systems (whether it be ZIL, Inform, Adventuron etc) actually still ARE the best way to make text games. I’m not getting into a Parser vs Other conversation here, that’s another topic, but since the essentials are to parse a sentence that a person has typed into a device and then output something plausible this is something the parser system does do … and we can add more and more variations, synonyms, modern colloquialism etc but essentially this IS the best we have.

NO CODE created the fantastic Stories Untold anthology (not played it yet? Go and play it NOW - it’s fantastic). The first game in the four is The House Abandon which is essentially a Spectrum +2 text adventure game, parser and all, and of course the underlying technology is Unity but even then to achieve the effect the best system for the job was the Parser (albeit coded within Unity).

Sticking with the premise of the player typing in ‘something’ in natural language, the system reading it and outputting something plausible … is there anything better than the parser? Anything more advanced? I’m not sure there is, even now.

This isn’t a bad thing by the way. I just find it fascinating!

Adam

1 Like
2

Can you clarify your question? Because that seems like a tautology to me. If you’re only talking about interfaces where the player types something in natural-ish language and the system responds, then what you have (no matter how advanced) IS a parser, right? Or at least that’s what we would call it?

Edit: I do think parsers have room to become significantly better than what we have now: just hand a random parser IF piece to someone who has never played one before and watch all the things they try to type that it doesn’t understand. As someone who didn’t grow up with text adventures (but did spend a lot of time playing the Discworld MUD as a college student) it’s fascinating to see the ways in which Inform7 is very advanced and the ways in which it’s extremely primitive and limited.

But making parsers better in a way that doesn’t lead to more bizarre misunderstandings nor makes it significantly more difficult to author does seem like a very challenging thing to do.

3

Hi Josh,

I’m thinking that Inform, TADS, ZILF etc as you say are both advanced and limited at the same time, but either way represent the best system currently available to the ‘person on the street’ for making IF/Text Adventures where you type in a command, the system reads it and outputs a description of what happens next.

I’m sure there’s probably an AI system out there that could read a text input, understand it and respond (I don’t know enough about the technologies to know whether that would still be classed as a parser or something new and AI-titled!) - but these aren’t to my knowledge available to the home user.

Does that help?

Adam

4

Modern natural language parsing is available, sure.

Try this: https://cloud.google.com/natural-language/#natural-language-api-demo – scroll down to “Natural Language API Demo”, type an IF-style command into the text box, hit “Analyze”, select the “Syntax” tab.

googlesnap
googlesnap1626×950 93.8 KB

This is a paid service but a small for-money indie game could easily afford to use it.

Now, integrating this with a game world model is not trivial. Most of the work that went into, e.g., the Inform parser is not about parsing per se – it’s about parsing into the terms of the game. (Game actions that can be applied to objects.) If the player expresses something that doesn’t fit into that model, you’re going to need a plan. This is where the usual “hey, better parsers are easy now” idea bogs down.

6 Likes
5

There is also the issue of the “parser” supporting languages that aren’t “English”.
eg. different grammar, punctuation, spelling, etc…

6

I’m trying to understand what people mean by a “better parser”. To me, the Infocom style parsers seem to have struck a pretty good compromise between allowing the player to express most reasonable actions without getting overwhelming in the process.

In a few Infocom games, you could use adverbs like “carefully” and it would make a difference. But I just found that to be surprisingly annoying. Even for interaction with other characters in the game I think I prefer the simplicity of a small set of well defined commands over some kind of natural language parser.

2 Likes
7

One avenue for improvement is to throw out superfluous parts of the player’s input, perhaps with a warning. You can make some progress on this if you have complete sentence diagramming (as in my link above).

For example: even if the game doesn’t have a meaningful distinction for “carefully”, it doesn’t have to reject EXAMINE PAINTING CAREFULLY. It could go ahead with EXAMINE PAINTING, and maybe also print “[You don’t need to use adverbs like CAREFULLY in this game.]”

Or: rather than requiring the author to list every possible adjective in the game source, it could detect adjectives and use them as-needed. If the player throws in an unexpected adjective (TAKE DISCARDED SWORD), as long as it doesn’t conflict with something else in scope, the parser could go ahead and tolerate the unrecognized word.

3 Likes
8

Another useful approach is to give better guidance when the player types something which isn’t a game command. Aaron Reed’s old parser-friendliness extension did some of this.

For example, the whole class of I DON’T KNOW WHAT TO DO, I AM CONFUSED, WHAT IS GOING ON could trigger a help response.

I AM A MIGHTY WIZARD or THE TROLL IS FRIENDLY are author-level statements about the world. Newbie players sometimes try inputs like this because they haven’t gotten the idea of imperative commands yet. A syntax parser could detect this whole class and trigger an explanation of how to interact.

I WANT THE LAMP is plausibly expressing an intent to take the lamp, even though it’s not noun-verb format.

1 Like
9

Note: historically, I’ve agreed with Torbjorn’s position! A focused set of imperative action commands is easy for an experienced player to manage and it’s easy for an experienced author to design for. It’s a sweet spot which we’ve relied on for a very long time.

That doesn’t mean we can’t think about changes. I’m throwing out some off-the-cuff ideas to encourage discussion.

2 Likes
10

Firstly, thanks for sharing the above link, this sort of stuff is really interesting to me.

On Torbjorn’s point… Yes, basically. In essence this is what I was thinking with the “what we’ve got still works well” sentiment of the original post.

That said…

THIS is also very interesting to me…

I’ve been trying to do this with ZIL/ZILF for a while and not cracked it yet, or even got started frankly.

You’re right because in essence this is how two people who don’t speak the same language fluently converse anyway ie they listen and pick up on the majority of key words in what the other person is saying and disregard the rest.

At the moment if you don’t nail the syntax the odds are the parser will give you a hard reject. If it could allow additional words to be inserted, parse the key words and ignore or identify the rest but proceed to respond anyway, that could work.

I’ve been to tinkering with allowing certain full sentences (by recognising the full sentence verbatim), also trying to get it to read for ‘key’ words in the input sentence and respond accordingly; which is fairly close to what @zarf was suggesting (if I’ve understood correctly).

So someone could type “for god’s sake attack the troll now please” and it would spot the key words and ignore the rest.

Needless to say, to date, I haven’t made much progress, but this is also due to time limitations.

Adam

11

For me it’s not about expanding the range of what you can say, but how you can say the limited set of things that the game understands. English in particular often has several (roughly equivalent) ways to say the same thing, and parsers usually only understand one particular phrasing. Of course now I can’t think of any examples…but I remember that being a big problem when I first started playing IF, and when I’ve tried to show parser IF works to other people.

2 Likes
12

Adding to my “To Do List” the noobs.zil file!

In here I’ll try to incorporate as many weird and wonderful sentence classes and synonyms as possible!

Letting my kids loose on text games is a great way to quickly gather a list of unusual alternatives! Earlier today in fact one of my son’s typed DESTROY instead of ATTACK. :smiley:

Adam

1 Like
13

eg. pick up the <small | tiny | puny | miniature | etc… > statue

14

Well, to be honest this is the easy part in my humble opinion. It’s just that it requires time and effort.

It’s literally a case of working through each action that you’ve programmed, adding synonyms to the command syntax code or if required creating new V-XXXX syntax strings.

I’m referring to ZIL now, I’m assuming it’s similar in Inform.

I’m terms of coding, dead easy. In terms of time and completeness, quite demanding.

Adam

15

Now what I want to get to is …

pick up the <small | tiny | puny | miniature | quite big | great big | etc… > statue

Adam

16

@JoshGrams @Greyelf

Actually I’m up for a bit of a challenge here! Give me some examples and/or scenarios that you think pose problems and I’ll try to code up the solution in ZIL?

Adam :grin:

1 Like
17

Sounds a bit like the Inform 7 language, actually, or maybe that old Stephen King story “Word Processor of the Gods”. It wouldn’t surprise me one bit if you were to announce that you were working on a game where this was the central mechanic. :slight_smile:

18

After the Infocom source code was made public, I started working my way through the games, checking for bugs etc. I’m still in the first third of them, but you can tell that they were experimenting with what the parser should respond to as well. Some games respond to the question “IS object IN object”, some treat “GIVE ME THE object” as another way of saying “TAKE object”, etc.

Of course, that’s more a matter of vocabulary than parsing.

1 Like
19

Yes, and to me it’s the time and completeness that are important here. What can the tools do to help? What if your development environment included a thesaurus with grammatical info about how the words are used, and could suggest alternate constructions, or even just build them automatically and show you the code for your approval or rejection? That sort of thing.

A bunch of things that I dislike about current parsers seem to be a direct consequence of the archaic VM technology used to run them. Like the whole thing about words or names being these immutable tokens that have no internal structure and maybe only even represent the first X characters of a word. I feel like we see a fair amount of problems like this Dialog thread. If you look for questions about trying to make dynamically-named objects in Inform I think you’ll find more similar problems.

You could do other interesting things if you removed that limitation, like letting users rename things. It’s very easy for authors to make things that are extremely hard to type. What if every game let the player work around that?

Other Inform limitations that seem crazy to me (even though as a programmer I can certainly understand how they might have come to be): I believe you can’t have an object that is both a container and a supporter. And you can’t do multiple identical items at all? And support for doing things with multiple items seems patchy and ad hoc? You can generally take all, but doing other things with multiple objects seems to work or not work based on whether the author decided to support it, rather than being built into the engine.

If you have the time to spare, I’d recommend trying out the Discworld MUD for an hour or two. It was started around the same time as Inform (I think it opened to the public in 1992?), but comes from a different background: constantly being worked on by a substantial group of people while other people are playing it. And they have a very serious system for getting new authors up to speed, and rules for how to make good content, and such. It’s far more dense than any other MUD I’ve ever seen. To me it makes most of them look like sparse half-finished prototypes. It’s more CRPG than IF, but also has lots of one-off set-piece puzzles (though many of them do use the stat system as well as just being puzzles) and hand-written interactions.

It has an alias system which allows you to not only rename objects but is essentially a tiny scripting language. It handles disambiguation and duplicate objects extremely well (there are maybe twelve different currencies in the game, and you can say things like “get Ankh coins from satchel” or “give three rhinu to Fred” or “sell 7 torches”). It does containers and liquids fairly well. And there are a whole bunch of different in-world languages which your character has to learn by practicing, which is super grindy but also hilarious.

I dunno. I think the differences are interesting and instructive. And the game is fun too, if you’re into that sort of thing. It gets grindy pretty quickly, but there’s probably a good 10 hours worth of walking around looking at and talking to and just messing with the world, even if you mostly ignore the RPG bits.

20

Inform has always supported multiple identical items and commands like TAKE FIVE GOLD COINS. (For small multiples – you don’t want to model 69105 leaves this way.)

You are correct about the other limitations. I could fuss about details but it’s a serious tangent. :)

Those are both very tricky roads from the view of playability, of course. No input is verbatim once real players show up.

My experience with keyword matching is that it’s only sufficient for recognizing simple noun phrases. (If you had a game where you entered “LAMP”, “ELVISH SWORD”, … then keyword matching would work well.)

Once you get into nouns and verbs, like the classic PLANT POT PLANT IN POT, you need to deal with structure or you’re going to be forever confusing nouns for verbs for adjectives. Or imagine dealing with TAKE GOLD COIN AND SILVER KEY, for example, in a room where there’s also a silver coin and a gold key.

I suggested the syntax parsing would be good for throwing out extraneous phrases: I REALLY WANT TO READ THE BOOK. IT IS TIME TO KILL THE TROLL. The Google service I mentioned can handle this. But if you try to deal with it at a keyword level, you’re going to match the player (“I”) or the last-mentioned object (“IT”) or whatever object might match “TIME”.

4 Likes