I was only talking about the part that breaks user input down to basic commands (actions in Inform’s terminology). Carrying out those actions is the rest of the game.
I guess I don’t understand how breaking a user’s command down into simple actions for the game to use involves the world model. All it involves is vocabulary. It gives the rest of the game actions to do and what to apply them to. A parser doesn’t know what a blue ball is, nor does it know that that is different from a red ball. It doesn’t know where the ball is, nor where the player is. It doesn’t know what the action is, just that it’s an action; a word it “knows.” It just breaks a sentence down and recognizes there’s an adjective associated with a noun, and feeds that along with whatever action is involved to the game rules that deal with that. And that’s where the world model comes in.
Dictionary isn’t world model.
1 Like
This may be true for some extremely primitive parsers, but not for the ones most of us here work with - I’m only familiar with the mechanics of the Inform parser, but it does “know” all these things (in the sense of “has internal representations of these concepts”). When the parser hits a word like “ball”, its translation of that input into a game concept depends on whether there are zero, one, or several things that could be referred to as “balls” within the scope of the player. Like, say there’s a red ball, a blue ball, and a tv showing a ball game - just using a dictionary and going word by word isn’t going to allow the parser to reduce “ball” to a single action (is ball a noun or an adjective?)
So the parsing mechanics include a complex model that does inevitably use elements of the world model to “understand” player input, i.e. translate it into game logic, by assigning different weights to different possible interpretations. The author has tools to intervene in how the parser makes its assessment by tweaking different aspects of the world model, but the mechanism itself lives within the parser. And I’m sure this is true of TADS as well.
Anyway none if this is really here or there, I don’t think, since from what you’ve said above it seems like you’re really talking about using ChatGPT as a “pre-parser” that would exist in between the player’s input and the actual parser (your example is of ChatGPT taking text and turning it into other text that could then be fed to an IF parser to produce an action).
3 Likes
Even better one that can transform “negotiate the vending of some cheesy comestibles” into [Player, Buy, Cheese].
1 Like
Yeah, I remember someone had a problem with a self-referential loop a few days ago because they didn’t realize that Inform 7 invokes the world model during command parsing, and so they accidentally made an infinite loop by calling command parsing during the world model section:
2 Likes
In any case, even if ChatGPT was an algorithm one could paste into their parser IF, it wouldn’t be usable for common people, you’d need a powerful computer, right? Or host it on a server with the required processing power, which would cost a lot.
We already have great interactive fiction parsers that can even work on 8-bit computers, so using ChatGPT seems a bit like a waste of energy for the gains it can bring.
5 Likes
Interesting news item on our local radio (2UE Sydney, Aus) this morning mentioned ChatGPT in relation to Universities being concerned about students using it to submit papers, etc that their plagiarism software wasn’t able to identify as to who had written the piece. - AI or real person.
And repeated on the TV news tonight too. Seems it has them worried.
5 Likes
Yes, it has us worried!
We did a bit of informal testing and found it could spit out a humanities-related essay not far below a 3rd class (which, for my institution, is really quite impressive).
2 Likes
I used ChatGPT to write a warning to my Software Engineering class about using ChatGPT or Copilot to write code.
6 Likes
I saw a memo Nick Montfort wrote to MIT colleagues about ChatGPT and teaching posted on Planet IF the other day - passing it along in case it’s of interest:
5 Likes
There actually is one way using GPT-3 might work (note this is GPT-3, not ChatGPT). You could conceivably train a model to rephrase long commands as standard parser IF commands. It would take several hundred examples and probably would have to be done separately for each game, but it might work.
4 Likes
Yeah, zarf mentioned this above, though it somewhat got buried:
I don’t think it would necessarily have to be done separately for each game, because for the most part, the grammar of parser IF commands (“Informese”, as I6 calls it) is fairly consistent between games. They might have some different verbs, but the underlying structure is pretty much the same.
That helps a bit with the training data problem—but probably not enough.
3 Likes
Once the Spring semester is well underway I might give it a shot.
1 Like
@artao: You’ve already demonstrated that it’s capable of translating natural English into “gamese” (the simplified English appropriate to a particular game development system’s parser), in a manner suggesting that the bot does use semantics in its parsing. (It initially assumed that the key was for the door and planned to break the case with the hammer, suggesting use of a generic world model. After being told that the key was for the case and the door was unlocked it modifies its plan accordingly, suggesting use of a hypothetical world model specific to the discussion.)
There are a few other testable hypotheses in this thread that could be performed by those with access:
-
Give it a scenario with a lamp and ask the interpretation or response to >GET LANTERN. (Perhaps best to try this with other items and reasonable synonyms for them, too, since the lamp/lantern association is almost certainly found in transcripts that were part of its training data.)
-
Explain the structure of action representations in a given system and ask it to translate player input into a representative format. (For example, the tuple
[actor, action name, noun, second noun]used by Inform – throwing inact_requesterif one wants to get tricky.) -
Give it a scenario and ask it to explain the world state in terms suitable for a particular development system (e.g. for Inform: list of objects and their kinds, state of relations such as containment/support, state of object properties such as open/closed/locked). Process a few actions and then ask it to explain the world state again, to see if it has modeled the state (or done the functional equivalent) correctly.
4 Likes
Do you folks generally agree that accepting more phrasings for commands will improve IF? I’m not in the habit of putting much effort into the commands I use, so I’m personally at a loss as to why I’d use a more complex phrasing than the obvious one. Is the idea to be a kind of automated thesaurus, for instances where the author has created some confusion about which terms the game will recognize? I can see that being useful, although a tool to help authors scan for these instances might be even more useful.
6 Likes
More synonyms comes to mind, perhaps also more “ignorable” words. Imagine if you will a short story about a guy who’s neck-deep into YouTube, and the world model includes a like button. In regular IF-ese, you might expect PRESS LIKE BUTTON to work, maybe PUSH. And I’m only specifying which button because there might be more. But what if you wrote SMASH THAT LIKE BUTTON? That’s what the guy in the video said to do, after all! But that’s not gonna work unless your game/library recognizes smashing a button as pressing it, and perhaps ignores the “that”.
We certainly have the space for a ton of synonyms.
Or I’m completely off the mark, should make like a horror and go back to lurking.
5 Likes
Do you folks generally agree that accepting more phrasings for commands will improve IF?
It is not about accepting more complex phrasings, but simply the fact that everybody will have a different idea of which phrasing is the obvious one.
4 Likes
While more synonyms and a “smarter” parser will help, there has always been a difficulty in translating the player’s intent into the correct action in the parser’s game world.
This isn’t so difficult with platformers, because nothing is being interpreted. The jump button means jump. Period. Plop an English copy of Crash Bandicoot in front of a preteen who doesn’t speak or read English, and they’ll likely still be playing passably well in 30 minutes from simple trial and error alone.
Parser IF can’t even get that same sort of clear ease of use from fairly literate folks who all speak the same dialect of English, let alone the complications added by those speaking other dialects of English and those who learned English as a second or third language.
With all that said, is AI the eventual solution to interpreting the bewildering variety of potential player commands into something a parser could reliably understand correctly? I have absolutely no idea whatsoever and I’m not sure how anyone else could be super confident either way about the relative utility of future AI applications in combination with a parser.
5 Likes
I’m not in the habit of putting much effort into the commands I use, so I’m personally at a loss as to why I’d use a more complex phrasing than the obvious one.
That’s true for everybody who is a regular here. Therefore, we don’t know what IF with a truly better parser would look like! Our design principles (for parser IF) haven’t changed much in thirty years, because the parser hasn’t changed much, because we don’t know what we’d do with a better one – it’s a bit of a closed loop.
EDIT-ADD: To say this differently: we have lots of experience creating IF games where all the player actions can be expressed in one obvious way – in Inform-standard syntax.
7 Likes
In other words, AI has the potential to alter Interactive Fiction paradigms, but it’s way too early to understand how. As Andrew said, we’re experts on Inform-style parser based IF.
AI is such a massive shift it requires an entirely new set of principles.
I assure everyone following this thread and following the overall impact of ChatGPT, every business and creative mind is contemplating the addition of AI to their interests.
It’s a fairly huge leap in technology easily comparable to the PC, the Internet, and Cloud.
4 Likes