Use of LLMs in custom interactive fiction engines

Authoring General Design Discussions
104

I’ve been a proponent of this idea for a while now (that if LLMs have a place in IF, it’s in processing user input and translating it into valid commands, not in producing output). But nothing has really come of it. The problem seems to be that even if players don’t need to know parser syntax, they still have to know what’s possible in the game—they need to know that taking things is usually productive, but flipping things upside down is not. Syntactic knowledge is not much help without semantic knowledge.

And that’s something best addressed by a tutorial, which knows exactly which actions are semantically sensible at any particular point in the game. A parser, no matter how smart it is, only knows syntax; that’s its whole purpose. If a player types LICK MR DARCY, generally the parser is expected to pass the appropriate action to the game world, which will reject it; it’s not the parser’s job to determine if it’s sensible or not, only whether it’s a valid verb and a valid noun.

105

There are two opposite problems here:

  1. If the LLM writes everything, the prose tends to be weak[1] and whole experience tends towards player-driven hallucination (what I call “lucid dreaming”) instead of staying closely tethered to the author’s vision and prose.
  2. If the LLM just parses input, the parser tends to end up writing checks the game engine can’t actually cash. The gap between “type anything!” and “actually, the game only understands these 17 actions under the hood” creates more friction than it lubricates.

What I’ve been saying for years is that we need is something in the middle: an architecture that does give the LLM genuine control over the world model and prose, but still shackles it tightly to authorial intent and human-authored prose.

(I’ve been tinkering with this for awhile, and recently made some architectural breakthroughs that genuinely blew my mind. I think releasing an actual game will be a lot more convincing than monologuing on Discourse, but to summarize, I have become convinced that (1) social simulation is the “killer app” for LLM-based IF; (2) the fundamental architectural keystone is correct information siloing [that is, simulating each character from their own perspective, rather than using the LLM as an omniscient narrator]; and (3) the remaining architectural problems turn out to be very different than anything people typically discuss.

For example: In one test session, my character needed to deal with two problems at once. While I worked on one, I asked some NPCs to work on the other. They went into a different room, autonomously had a whole debate in-character as their author-defined roles, agreed on a compromise, went and solved the problem, and came back to report what they did. I repeat: I was able to delegate puzzles to a committee.)

  1. Although, I’m convinced this problem is solving itself; default output is still bad, but models have improved so much in the last year that even a small local model like Gemma4-26B-A4B can now be coaxed into writing pretty decent prose. ↩︎

106

Is that fun for the player, though? My experience has been that clever puzzle-solving NPCs (LLM-generated or not) tend to be more fun for the author than for the player. There’s that story about how too-smart NPCs in Versu were gossiping behind the player’s back and sharing information they’d learned, but since players couldn’t see that happening, they reported it as a bug. All they saw was NPCs knowing things they shouldn’t. (And thus the devs removed the feature.)

107

The difference, I think, is that social simulation in LLM-IF can be so real that the player stops thinking of it as an engine at all. Real life is fully of social ambiguity. If an NPC does something bizarre, the player’s reaction is no longer “that seems like a bug,” but “woah, what is that dude’s problem? Was it something I said? What was he doing in the boiler room, anyway? Joanne was there, too—maybe I can ask her…”

This is conducive to very different sorts of games than traditional IF, of course. In the committee example, as a player, it felt totally natural (and convenient) to just ask someone else to tackle the problem. (One could imagine a whole co-op game based on this concept.) It was only when I read the engine logs that I realized how much effort the NPCs put into convincing each other and coordinating their actions.

108

We happen to be passing through the “AI is the new hammer” time. Everything just looks like a nail right now, including IF. I tend to agree that LLMs would work well for translating player input for parsers based around semantic slotting.

109

I think it’s ok if the NPCs get up to things in secret, but if the player doesn’t know about it, then it hasn’t happened. When the game state changes, there always has to be some sort of reporting back even if it’s subtle. “I will remember that,” or similar might be enough if you want t leave room for mystery. Create suspense, not surprise.

110

Part of me actually thinks that this can be used to make the player DOUBT what is possible in the game. Not for the main gameplay, otherwise you plant a lack of agency. But if you intelligently design the way the LLM selects from behaviour and content you prepared, it can keep the player in a state of wonderful uncertainty of what the edges of possible play are.

111

The problem with LLMs here is that they’re programmed to assume the player is always correct; and in many cases that’s true. However, if I type LICK MR DARCY in a game set on another planet, the LLM might well assume Mr Darcy in fact exists somewhere in the game world and produce a nonsense response. It all depends on how it’s implemented. For instance , the LLM could resolve the input by assuming that Mr Darcy is a flavor of ice cream, based on the context. If indeed ice cream is in the game somewhere, it could respond with something like “The cone has no Mr Darcy in it.”

112

I’m very glad I found this thread!

This is very close to what I’ve been working on for the past ten months, actually. :slight_smile: My engine is called TALYXR_, and I have a working demo (not publicly released) that I think is doing really well with both input interpretation and freeform NPC conversations.

I’ve just posted an introduction to TALYXR_ here, with some basic details about how it works.

The story, world, characters, locations, objects, actions, puzzles, etc. have all been pre-authored, including the responses to supported game actions.
Early on, I found that letting a language model handle the output of something “simple” like LOOK AROUND wasn’t giving me what I needed. It’s either too detailed, too minimal or focused on the wrong things. Parser games require a very specific type of feedback. Current models just aren’t reliable. So I just wrote those myself. It’s better, more fun and creatively satisfying too.
Generated text is used for freeform NPC dialogue, since I can’t pre-author every possible conversation, and for responding to attempts the game doesn’t support.

The basic flow is:

  • The regular (deterministic) parser handles commands it recognises.
  • Otherwise, a language model interprets the input and translates it into a command the parser can process.
  • If the intended action is supported, the engine handles it according to the game’s rules, using authored responses.
  • If the action isn’t supported, a generated response acknowledges the attempt and explains why it doesn’t work, without changing the world state.

Obviously, I am massively oversimplifying the principle, because it’s not that easy. Getting it to work consistently is considerably less simple.

Player Freedom

Player freedom is one of the hardest pieces of the puzzle. Even once you can distinguish supported actions from unsupported ones, you still have the main design question: where do you draw the line?

Take a library, which is an actual location in my game. You might have dozens of bookshelves, each holding dozens of books. A language model could generate incidental book titles only when the player examines a particular shelf. That would introduce another use for generated content, beyond dialogue and failed attempts. But it also opens up a whole set of questions.

It would be weird if the player could not take out any of those books. So, you allow it. And if the player can hold a book, they’ll probably want to read it. Do you then provide actual passages, or just a brief description?

Is the player able to rip out a page? Is that page then a seperate object? Is the player able to rip out 50 pages? Can I burn a them? Can I stand on a chair? Can I put the chair on the table and stand on the chair that is on the table? Can I stack chairs? How many?

I’m pretty sure you get the gist of it.

So, I’m trying to support as many reasonable actions as I can, although I have to draw the line somewhere before this becomes a physics simulator. :sweat_smile:

NPC interactions

Conversations are really cool and can sometimes feel real.

To get there, I’ve written extensive biographies and character descriptions, including what each NPC knows. The aim is to let you talk freely while keeping their responses consistent with those characters.

Keeping them from inventing facts is definitely one of the hardest parts. Language models really love to talk and come up with juicy details that you never asked for, which makes for a very lively interaction, but with a high risk of saying something that isn’t true within the story.

But language models are also notoriously bad at following many rules at once. The more rules you give it, the worse it’s going to perform. A big part of my approach is limiting the information the model receives to what the character should actually know, rather than giving it the whole story and asking it to keep secrets. That still leaves the problem of invented details, especially when players ask things I haven’t anticipated.

I still need to figure this out properly. In my current demo, I don’t have NPCs in the present. There are only flashbacks, which obviously are memories. If the player says or does something that is really weird or obviously not what happened, the flashback just distorts and rewinds to the last output by the NPC.

Apart from that, the NPCs have a “story backbone” which they follow. You can ask questions that are not relevant but are in character. The NPCs will respond to this question, but then they will pull the conversation back to the story backbone that I intended.

In flashbacks, that works really well. In the present, however, you cannot rewind. So if the player acts completely out of character, the NPC actually needs to deal with that.
I haven’t worked this out properly yet. My inclination is to let those actions have consequences, even if they take the player away from the intended path. If you scare away an important NPC, for example, that might close off part of the story. Could you repair the relationship? Find another way forward? Or would it lead to an earlier ending?

I want those consequences to make sense within the story. How much of that I can support, and which alternative outcomes I would need to author, is still something I need to work out.

The unreleased demo

Anyway, the demo needs more work before public testing, but I can already show examples from it. Is there a particular interaction or conversation you’d like to see?

Also I’d be interested to hear what would make this approach convincing to people/you.