Thanks for sharing, this is really interesting and has a lot of new information that hasn’t been very present in AI discussion in recent months. This does seem like a bigger step to a world-model AI.
I was thinking after reading this that it might be useful to define from a game viewpoint: what is the desired function of the AI? Feasibility and enjoyability change a lot depending on that function.
I can think of a few functions AI could have:
- Adding a command-interpreting layer over a traditional parser game. Here the AI’s sole use would be taking unrecognized commands and matching them to the most similar or likely recognized command. This is less an LLM problem and more of a classification problem, but could be approached with similar techniques. Game errors would still exist in a useful way (like ‘There is no object called that here’ kind of stuff) but typos or long sentences could be fixed (like if someone types ‘please go west’ it could direct it to GO WEST).
- Adding non-error, world-changing responses to every command. So if the programmer hasn’t implemented ‘DIG’ and the player types ‘DIG SAND’ in a desert. Since the AI is generating stuff on the fly here, this could end up with the AI going ‘off script’ and providing an entirely different story than the one programmed. AI Dungeon is basically nothing but this.
- Programming/writing the whole game itself (coming up with plotline and coding). Fortuna in recent parsercomp was closer to this.
I could see 1 be exciting and/or fun for drawing in new people; eventually I think it won’t be useful, because I’ve learned over time that:
- players eventually learn what commands your game expects and create a mental list of those commands and just reuses them, except for instances you require something special, and
2)players eventually prefer shorter forms of commands. So many parsers pride themselves on being able to understand complex sentences, but I’ve never seen a player say ‘I wish I could use longer and complete, grammatically correct sentences while playing’. Like, it’s cool to know that I could type “I take my journey to the land of the west’ and the game moves you west, but if you have a game with 40 locations and lots of movement you’re not going to type that every time! That’s why it’s more like ‘GO WEST,’ 'WEST”, or even just ‘W’.
So for those 2 reasons I think that AI assistance would mostly be useful for new players or for the beginner of the game.
I’m not really excited about the 2nd and 3rd kinds of games I listed above. In the 2nd game, if the AI is doing the heavy lifting, then the programmed game becomes irrelevant. You might as well just have the pure AI, like AI Dungeon. In the 3rd kind, AI is currently programmed to write what people expect in a given situation, so it tends to do bad at surprises or unusual things.