Use of LLMs in custom interactive fiction engines

I do verb and noun synonyms visually. The author can link any number of words or groups of words to an object and the parser resolves it to the object. The same for verbs. There is also the ability to add generic nouns and verbs in case you want to go off into the weeds and make something really custom. In my testing project I have a synonym “gas up” to also mean fill. This ability is how I was able to create a tutorial and have commands to take you in and out of it.

Getting a bit abstract, but I think this is relevant in any discussion of LLM use in IF: guess-the-verb, “playing the parser”, error messages etc. are constitutive elements of parser IF as an expressive medium. Without them, one could even argue that what we have is no longer parser IF, but something else. In other words, I don’t think the fact that not everything the player types in is accepted by the parser is a limitation to the full realisation of IF, but rather, it forms part of what IF is.

I really like Jeremy Douglass’ 2007 thesis on IF, Command lines, where he talks about “error and frustration as a form of IF aesthetics”. One plays IF through a process of interrogation, trying things, getting feedback from the parser, and adjusting their mental model of how the parser works and what they can type to change the state of this particular game. Obviously, there are cases where this is poorly done, which we call bad IF: where the player’s frustration seems excessive and, especially, arbitrary, not related to anything the game is trying to say.

But I think LLM-powered IF can also feel like bad IF, for a related reason. Not because it’s too restrictive, but because it doesn’t restrict and, therefore, also feels arbitrary. If I can type anything and get a response, the game is telling me less. I don’t form a mental model of how it works, and so it doesn’t affect me. LLM-powered IF feels inherently generic, rather than like something specifically crafted, including in its limitations, to say something.

I know the paradigmatic bad first experience of IF where you keep trying commands and getting uninformative error messages in response has traumatised a lot of people, and gets blamed for why IF isn’t popular in the way some people would like it to be, so there is interest in anything that could be a solution to that. But I think the solution is just to teach people to play IF, which yes, in my opinion means sticking to a restricted set of commands, and interrogating the parser to figure out what you can do with them. If they don’t see the appeal of that, I think they just don’t like parser IF, and there’s nothing wrong with that.

I think this is a really interesting point.

I haven’t tried recently, but I haven’t yet found a way to prevent an LLM from telling someone how to install debian if they ask. They’re too trained to just answer your questions

I don’t assume that this problem is non-overcomeable in the long term, but it is a barrier to making something focused with them right now I think.

I’m still learning about the genre in general, but I do think that it’s cool if you can teach someone new to play in the game or though the interface somehow, and I’m interested to possibly experiment with ways to do this.

I think one of my favorite things about IF so far is that it forces you to think critically about the world in order to imagine what might make sense to do, instead of, for example, in point and clicks ( which are still completely awesome ) where you sometimes feel like just clicking on everything on the screen because you know that’s how you have to be able to do something.

But all genres have their failure modes, IF, novels, point and clicks, 3D physics-based games, etc., they all give us different things we can do, as well as things that we can’t do. And sometimes what we can’t do is as much as a feature as it is a limitation.


As a meta sidebar, I’ve been pleasantly surprised by how active ( and not overactive :slight_smile: ) this community is, as well as how respectful y’all have been on controversial subjects in multiple threads that I’ve read some of / skimmed.

It’s really pleasant so far, thanks for being cool, folks. :slight_smile:

You made me think of three things.

1-A long time ago, someone made two games (I can’t remember who or the name of the games). In one of them, whatever you typed, it said “you can’t do that”. In the other one, whatever you typed, it said “That’s exactly what happens”. The point was to show that absolute freedom is the same as absolute constraint in games.

2-When you talk about error and frustrations, one thing I learned by studying past IFComp games is that error messages are some of the most frequently encountered parts of your game, and that strict care should be given them. People look for edges on purpose to find out what the rules are on a game, and having hard boundaries can make things more fun instead of less.

3-I think one issue is that ‘having grammatically correct text’ and ‘being a great game’ used to be really highly correlated because they meant that someone had worked hard on a game. Then we found a way to make everyone be grammatically correct, and some people expected that to automatically make the game great, but there was never any causation there. It’s like blue checkmarks on Twitter. They used to be a status symbol, ‘elite’, because you had to be at least semi-famous and go through verification. So people wanted them because they felt they’d be respected. Now they’re sold, and they’ve become the opposite sign: a blue check often means a porn bot or a scammer. I feel like the same thing has happened with LLM’s generating text that would previously have been associated with quality in games.

@talsvals
Those are really great perspectives on limitations being desirable and it goes way beyond LLM conversations. I think it’s worth repeating here that limitations breed creativity; and that goes for both author and audience.

https://jeremydouglass.com/dissertation.html

Thanks for the mention of Douglass’ paper. Looks like a good read. :+1:


Agreed. I would even say that it’s objectively generic.

I believe an LLM cannot respond with anything that deviates from the consensus of it’s training data (our written thoughts). It will never say anything worthwhile that’s remotely original… and that’s where I believe creativity lies. AI models are inherently cliché due to their probabilistic nature.

I am not a pro-LLM person, but I think this is flattening the discussion of the past few days into a straw man (straw chatbot) argument.

We can distinguish the problem of learning how the IF parser works (and what verbs to type) from the problem of learning how the game world works (and what actions are worth trying). We already take for granted that these are two separate things! Because the Golden Age(*) IF games were written with the assumption that players already knew Thing 1, and were excited to tackle Thing 2 for the new game.

(* I mean mid-to-late 1990s, when this community got started on Usenet.)

When talsvals said above:

guess-the-verb, “playing the parser”, error messages etc. are constitutive elements of parser IF as an expressive medium.

…that is strictly speaking not correct. “Guess-the-verb” is either (1) a failure to know how the parser works, or (2) a failure to understand what actions are meaningful in the game. In either case it’s a failure mode, something that the game author tries to avoid. (Either by better inclueing or by giving the player a parser tutorial.) Anything which reduces the incidence of either is an improvement to the game experience.

The important distinction you’re getting at is the difference between, e.g., “The rug is too heavy to carry” and “The rug is immovable.” The first provides a clue, the second destroys it. If an AI parser system (or any parser system) can’t reliably come down on the correct side of that line, that’s when it’s wrecking the expressivity of the game.

(It may be helpful to distinguish “parser error messages” (thing 1) from “game failure responses” (thing 2). We don’t consistently do that but it’s what I’m getting at for this discussion.)

This seems trivial to disprove, because it’s easy to get an LLM to say two contradictory things (with two different prompts), and they can’t both be the consensus.

I think we are using consensus in different contexts. I should have been more clear. I speak of a statistical consensus; not a consensus of actual human-level understanding and conventional wisdom.

I should have said something like, “it generates a statistical aggregate response based on the prompt (seed) weighted against the relational token weights in the training data”… but I got lazy, okay? :wink:

These are helpful distinctions, and you’re right, I think I conflated those things: knowing what the parser will accept, and knowing how to act in the game world. Typing a synonym to a verb that would be readily understood by a person but isn’t understood by the parser is 99% of times something people try to avoid in IF. Knowing that parsers work a certain way and that sometimes you can have trouble because of the way you’ve worded a command is, I maintain, part of “knowing how to play IF”, but definitely isn’t the core of what playing IF is or why people enjoy it.

I do think there is this idea that LLMs can be used to make IF more playable because they can generate on-the-spot responses to things the author hasn’t thought of. I think it’s worth questioning what the limits of that are. No one here is saying exactly that, but I think it is a relevant part of the imaginary of IF: fiction, but interactive! This idea that it would give the reader freedom, that they would control the narrative etc. and that this was a frustrated promise, that could still be realised. I think what I was trying to say in my post is that trial-and-error is how one plays IF, not a surmountable limitation of IF. But I agree there are important differences in figuring out how the parser works and how the game world works. Though I don’t know how completely separate they feel to me as a player, and I think that’s interesting and I’ll continue thinking about that.

I really like this sentiment and find it insightful. I don’t think it detracts from the discussion. I’m a sucker for paradoxical truths. :wink:

If I play a parser game and everything I do sounds interesting or possibly relevant, that sounds like a recipe for disaster.

I guess you could envisage some experiences where this facility allowed you to take a narrative in any direction. Like a fluid novel. Perhaps at some point LLM’s or their successors could achieve this.

But if you have puzzles then I think it’s a non-starter as it’s just going to bury the real clues in an avalanche of equally signal-bearing fluff. You can’t fire Checkov’s gun when it’s one of 30 hung on the wall.

AI Dungeon and its ilk do this, but it tends not to be very satisfying. Someone last IFComp posted a review of Penny Nichols, Troubleshooter (a comp entry that was a prompt to be pasted into an LLM) where they typed “do the macarena” for every input, and the game tied itself in knots to accommodate, eventually explaining that the player character had godlike reality-warping powers activated by dancing and thus the plot could progress normally despite the constant irrelevant inputs.

The friction is part of the fun!

I know. That LLM paste-in game inspired me to figure out how to get Gemini to utilize state variables, assuming one thread per turn and not doing the whole game in a single context window. I figured out it kind of works, but not only does it take a lot of effort to do, but it feels more like a TTRPG than like IF.

Of course, it could be automated, and maybe made more IF like, by being managed on the back end through API calls and automated back-end prompt tooling. I’m certain the real AI enthusiasts would be able to do that. And it probably hasn’t been done well yet. It’s probably worth trying, and I’m probably not going to try to do it. Probably best tried outside of the IF community, at any rate, out of respect for the medium.


In regard to the parser, I think it’s still true that the player’s decision to cooperate with the system is almost everything. I think convincing the player to really play along, to want to cooperate, is the key. And I guess the key question here is, can LLM context games convince the player to cooperate better than traditional parser IF?

That’s the first thing that comes to mind as I look at @tzbits concept in NotebookLM. It seems that convincing the player to choose to cooperate is even more important for an LLM-based system. The failure modes are different.

Games are about the things you can’t do just as much as the things you can.

People are already using LLMs for freeform roleplay scenarios, specifying characters and mechanics. LLMs are fluid to a fault, with sycophancy meaning the player can easily send the game off the rails and hijack it completely. The capacity for LLMs to extend beyond the original bounds is controlled hallucination. The inherent lack of firm boundaries is a serious problem. LLMs are generally not trained to do hard refusals except for safety or political reasons. People are also trying to implement RPG-inspired frameworks atop LLMs. If IF is a hard fit for LLMs, that’s probably a sign that the two don’t mesh well.

…or commercial. Don’t forget these are fundamentally corporate products. LLMs ironically can be very conservative when it comes to discussions tending toward copyright infringement.