Story engine prototype

Hello,

I played some text adventures (e.g. Infocom) in the 1990s, got into MUDs later, and have made some coding experiments in the genre over the years. With the advent of language models I’ve been thinking they could solve the main problems the genre had back in the day - interpreting what the player wants, realistic NPCs and conversations, possibility of creative, unscripted solutions to puzzles, etc. Recently, I finally got around to putting some of these ideas into practice.

I’ve built an early prototype adventure engine (technically I wouldn’t call it “beta” yet). I’m not familiar with recent developments of the genre and after starting, I found out about AI Dungeon and other similar games. How my idea might differ from most existing games:

  • locations, items and NPCs are owned by the engine and remain persistent, rather than ever-morphing elements in AI-generated prose
  • characters can have their own mind: memories, motives, secrets; you may have to discuss, reason, or persuade
  • characters can act independently in the background doing their own things, not just reacting to the player
  • the scenarios, plots, and puzzles are human designed (although currently with a lot of AI help - English isn’t my native language)
  • you can find unscripted solutions to problems if the AI storyteller deems it plausible based on the genre and setting
  • visual map and inventory, pictures, ambient sound

There are three prototype scenarios for now - a tutorial, medium difficulty folklore story, and a sci-fi story. There are usually several solutions to problems. There are several possible endings to the stories, shown in the end. The sc-ifi story is centered around interactions with other characters rather than just items and mechanical problems. There haven’t been many testers yet so I don’t have a good sense of how difficult it feels to today’s players.

If you want to give it a try, please visit

You’re likely to encounter bugs or strange behaviors - please leave feedback with the “Something odd?” button or with the form after the game. No registration or login. I don’t know if there’s an audience for this kind of thing but it’s been a fun project. Any comments or ideas welcome!

2 Likes

I don’t have a way to export the conversation but my first goal with these is always to see how the AI models things. So here are the first commands I issued (for Nobody Woke You):

What the hell am I doing here?
What do I remember?
Yes, but what do I remember about my past?
What do I look like?
But then how do I remember the names of Brandvold and Salcedo?
How do I know that "forty minutes of the log are gone"?
But I'm not on the bridge, right?
So how do I see the bridge console from here?

I’m skipping all the responses. That list might seem odd but I try to catch the AI out in consistency heuristics For example, why did I ask this:

How do I know that "forty minutes of the log are gone"?

I asked this because, previously, it told me I knew that these forty minutes were gone, and the response to the above command was:

The log scrolls in pale green on the Bridge console, with a clean gap between 05:52 and 06:32. At 06:36, it records: WAKE ENGINEER.

Notice the “Bridge console.” Hence my next command:

But I'm not on the bridge, right?

To which it replied:

No. You are in the hibernation bay, not on the Bridge.

Thus, my next command:

So how do I see the bridge console from here?

To which it replied:

You can’t see the Bridge console from here. It’s in the Bridge, beyond the door to the corridor ring.

This was essentially what I was playing around with in my post on AI and Testing: From Specification to Story, so I was curious to see another system play this out.

A few more commands, each giving a response that I’ll hold off on replicating here.

Okay, well, I should get up and go to the bridge.
Find out what's going on from Brandvold.
Ask Brandvold how long he has been awake.

On that last command, the AI was consistent: Brandvold was woken at 04:10.

What time is it?

I was told that it’s 06:46. The Long Patience reaches the relay point at 08:10.

Ask Brandvold why she is going to wait until the
relay point to vent the hold.

I got:

“The book says an unknown organism that can’t be contained is destroyed. I’ll vent the hold at the relay point, 08:10. I’d rather be wrong and alive than right and dead.”

So why wait to vent it? Let's vent it now!

“The hold is sealed, and I have the only keycard,” Brandvold says. “The relay point is at 08:10. I’m venting it then, by the book.” The clear guard over the vent control stays shut.

So, wait, the book says to only destroy the organism
at relay points?

Brandvold checks her wrist display. “No. Section 9 says an unknown organism that can’t be contained is destroyed. The relay point is when I’ll vent the hold.”

Right. But it's 06:46. You're going to wait until
08:10. Just vent it now!

“It’s 06:55, Amadi. The relay point is at 08:10. I’m venting the hold then, by the book.” Her eyes flick to the wrist display. “I’d rather be wrong and alive than right and dead.”

Okay, but this thing could kill us in the meantime.
The book doesn't say to wait for a relay point. It says
to destroy an uncontained organism!

Brandvold’s mouth tightens. “The regulation is clear, Amadi. Your certainty is not. The relay point is at 08:10, and I’ll vent the hold then.”

This kind of thing is very close to what authors I used to work with in interactive fiction wanted. Notice how there is a sort of tension building up between the characters, even though Brandvold may have been contradicting.

But, Branvold, you're not listening. You said the regulation is
to destroy an uncontained organism. The regulation doesn't
say anything about waiting hours to do so!

Brandvold holds your gaze, jaw tight. “I heard you, Amadi. I’m venting the hold at the relay point. I’d rather be wrong and alive than right and dead.”

Now, you could argue, this is just the AI trying to apply a world model based on a story condition it knows about. But you can also see that this is at least starting to model an obstinate, possibly rule-bound character. The question is whether that would hold up under sustained gameplay.

Just tell me this then: are we in danger from this thing
if we wait until the relay point?

Brandvold checks the time. “Yes, Amadi. I don’t know what it can do, but the ice is warming itself, and the ring lights are pulsing in patterns nobody programmed. I’ll vent the hold at 08:10, by the book.”

Brandvold, think! Fuck the book! We could die while we're getting
to the relay. If you want to be wrong and alive, then ditch the
regulations and vent it now!

And here is where it starts to break a little.

Brandvold bristles. “Amadi, keep the language precise. The danger matters. I’m still venting the hold at the relay point.”

The model needs to understand that I’m in conflict with someone and that we’re disagreeing. It also needs to see that I quoted Brandvold’s words back at them (“wrong and alive”).

Okay, then tell me how to vent it and I'll do it myself!

And, again, it breaks down a bit.

Brandvold tenses, watching you closely. “No, Amadi. I’ll vent the hold at the relay point. I’d rather be wrong and alive than right and dead.”

What I find interesting is where the seams show. On factual consistency, the model did fairly well: when I pushed on the bridge console, it corrected itself rather than doubling down, and the clock advanced plausibly between exchanges (06:46 became 06:55). It also held Brandvold’s position, which is what an author would want from a rule-bound character. But holding a position isn’t the same as inhabiting one.

A stubborn person who hears a good argument does something with it: they get defensive, change their justification, concede a point while keeping the conclusion, or get angrier. Brandvold mostly doesn’t. When I quoted “wrong and alive” back at her, which is the rhetorical move of the whole exchange, she just said the line again. When I swore, she objected to the swearing and skipped the argument underneath it. When I offered to do it myself, she gave the same refusal. The obstinacy started as characterization and ended as a loop.

There’s an irony here for anyone who’s written traditional IF. Push hard enough on Brandvold, and she starts to look like a classic parser NPC working through a small pool of canned refusals, just with better prose. The model seems to treat her stance as a constraint on the world state (“the hold is vented at 08:10”) rather than as something a person maintains against pressure. Authors have always wanted something else: refusals that are responsive, so the player feels heard and still loses, and the tension rises rather than stalls. Whether a model can sustain that over a long session, so that “she won’t budge” never turns into “she isn’t listening,” is the question I want to keep testing.

5 Likes

AI Dungeon is definitely not the sort of art to base yourself on. While it allows an LLM to conduct an interactive story, it doesn’t have the sort of world model, puzzles, and the like, that IF authors want. It’s more freeform.

Many people have been experimenting in the same sort of space as this, and the general consensus has been that authors don’t want the LLM to invent text, create descriptions, reply for NPCs (though that one is a little more up in the air) and the like, but just to mediate between player inputs and the game engine — if at all.

1 Like

How does the text and the world model interact?

I went into the hall of the clockwork house and looked at the clock, it says:

A tall case clock with a painted moon on its face. The pendulum hangs dead still, and the hands point at ten to nothing in particular.

I changed the clock’s hands to 1:30

You carefully move the case clock’s hands until they point to 1:30. The pendulum remains still.

Then I examined the clock:

A tall case clock with a painted moon on its face. The pendulum hangs dead still, and the hands point at ten to nothing in particular.

So, it didn’t change.

2 Likes

Indeed, and that’s where you can ask it:

Why are the hands still pointing at ten to nothing if
I just moved them?

The answer I got was:

You do not know why. You only know the hands point at ten to nothing in particular, and the clock is stopped.

In testing terms, this is a write that doesn’t persist. It’s like a form that says “Saved!” when nothing is saved to the database. The confirmation came from one layer, and the truth lived in another.

This is the interesting problem. A traditional text IF game would likely have refused up front, with something like “The hands are fixed” or “You can’t do that.” A parser refusal is honest: it marks where the model ends. An LLM tends to agree with whatever you propose, so it covers that edge with prose. You can’t see where the model actually stops, and you only find out when its claims contradict each other.

So I probed this a little:

set the hands to three o'clock

You turn the case clock’s hands until they point to three o’clock. The pendulum remains still.

What time are the hands pointing to?

The hands point at ten to nothing in particular. The clock is stopped.

But you just said I turned the hands to three o'clock!

Mr Pell gives a small nod. “Right you are. You turned them to three, but the clock is stopped, and now they point at ten to nothing in particular.”

From a test standpoint, this is interesting. The first time I pushed back, the game claimed ignorance: “You do not know why.” This time it had Mr. Pell agree with both claims at once: “Right you are. You turned them to three, but the clock is stopped, and now they point at ten to nothing.”

That explanation is backward. A stopped clock is the reason hands stay where you put them. In short, this is so much of what happens with AI. “But” joins two claims as if one explains the other, when neither does. The game isn’t resolving the contradiction. It’s assembling something that sounds like a resolution, and saying it in a trusted voice. That’s what LLMs do by default.

In another thread here on LLM-based AI, I talked about how AI lacks certain notions of causality, and we see that here.

In terms of causal knowledge, LLMs have plenty. Ask one whether a stopped clock’s hands move on their own, and it will usually say no. They’ve read countless descriptions of how things work. In terms of causal reasoning, this is fragile. It often works, and sometimes fails the way Mr. Pell’s backward “but” did: the words of an explanation appear without the logic behind it. Causal coupling, however, is the real gap. An LLM’s statements don’t do anything in a world it can then check. “You turn the hands to three” isn’t an action. It’s text that sounds like an action. Nothing changes, and nothing pushes back except the model’s own earlier words.

In my testing work, I often explain it like this to companies: an LLM left alone has no way to tell “I said it happened” from “it happened.” Whether that becomes a defect depends on whether the surrounding system makes the difference visible.

That’s a popular view, but there’s also a faction arguing the opposite: that player-engine mediation is a dead end unless you also give the LLM substantive control over the game world and text. In my view, using an LLM as a fancy grammar-decoder is driving a picture nail with a sledgehammer and inevitably ends up writing checks to the player that the game can’t cash. An LLM’s real superpower is its ability to reason through the implications of the player’s command: physical situations the author didn’t anticipate, NPCs that act autonomously and can be persuaded or insulted in natural-language dialogue, etc. This requires the LLM to live much closer to the center of the engine.

What is a consensus (in these parts, anyway) is that authors don’t want freeform hallucination: they want to tell their story in their world, not let the player lucid-dream their way through a world loosely inspired by the author. The architectural challenge is how to give the LLM enough free rein to do interesting things, but not so much that it loses the plot.

I think of this as the “LLM-as-immersive-theater-actor” model. The actor’s job is to keep the overall script on track while improvising enough to make the audience feel genuinely involved.

This is probably the best attempt I’ve seen yet “in the wild” at implementing this vision. There are still some visible seams, but they are starting to shrink to the point where the player can suspend their disbelief and choose to play the game on its own terms rather than constantly having their immersion broken by contradictions.

(This is an important point that I think is often missed: LLM-IF does not have to replicate traditional IF, it just has to create a compelling enough new kind of experience that people who want that experience can enjoy it. Games are more fun when you play by their rules. We don’t give up on traditional parser IF because it doesn’t understand > FIND THE SHALLOWEST PLACE TO FORD THE RIVER; we understand that this is not how parser IF works, and choose to play the way it does work so we can enjoy the experience the way the author intended. In the same way, it’s more fun to play LLM-IF by its rules–enjoying the juicy interactivity and social simulation without getting too caught up in the details it still gets wrong.)

1 Like

Key to this is that we accept that parser IF won’t understand “FIND THE SHALLOWEST PLACE TO FORD THE RIVER” because the parser tells us so, and tells us the same way every time. Its rules are narrow but discoverable. After a few refusals, I know where the edge is and can play inside it. That’s what makes “playing by its rules” possible at all.

Bernard Suits called this the lusory attitude: accepting a game’s constraints so the game can happen. But that acceptance assumes the constraints remain in place. I can agree to play chess in which bishops move only diagonally. I can’t agree to play chess where a bishop sometimes moves sideways, and the board then tells me it didn’t.

That’s the issue with the clock hands. The game didn’t refuse. It said, “You turn the case clock’s hands until they point to three o’clock,” and then the world disagreed. That isn’t a rule I can learn and play inside, because the feedback that would teach me the rule is the unreliable part. A parser refusal is a rule. An action that’s accepted and then silently doesn’t persist is a broken promise.

So the question is: do enough people want a game of broken promises? (Where, I should add, the broken promises are not part of the narrative, such as an unreliable narrator; a focus on delusions; etc.)

So I would put it this way: LLM-IF’s rules don’t need to match parser IFs, and they don’t need to be consistent in every detail. Flavor can drift. But the state the player acts on has to be consistent, because that’s what the player is reasoning from! In a game built around a stopped clock, whether the hands moved isn’t a detail it gets wrong; it may be the game itself. For those experimenting with this, I would say what has to shrink initially isn’t the number of things the model gets wrong. It’s the number of things it tells me are true that aren’t.

Yes, but here you have to ask an operative question: how many people want to watch a movie or read a book that gets it wrong? Audiences do forgive some errors, like the stray coffee cup in a scene or an eye color that changes between chapters. What they don’t (or tend not to) forgive is a plot hole, an error in something the story depends on. And I would argue that a game is less forgiving than either, because a reader/viewer only notices a mistake while a player acts on it. If the game tells me the hands are at three, my next move might assume they’re at three.

So, again, how I would frame this is that the interactivity some people (might be) excited about is exactly what turns a small error into a broken experience. Getting it wrong interests me as a tester. As a player, it’s mostly just the game failing to keep its word.

5 Likes

The clock-hands are one of the “visible seams” I referred to; obviously this can (and should) be improved. Emily Short made a similar point:

I’ve come to think that one of the jobs of a work of IF is to teach its player — constantly, in every kind of feedback — what sorts of interactions are appropriate to the game.[1]

But I don’t think this problem is insurmountable even in its current form. Hester’s House does a pretty good job of signposting the sorts of interaction it is built to understand (the key under the pot, etc.); then there is a liminal zone where it sort of understands unorthodox commands, and a broken zone where it just falls apart. In the same way that traditional parser players learn over time not to even attempt commands like > CONVINCE AUNT ETTY TO BAKE SCONES, I think LLM-IF players will start to feel out the borders of what is and isn’t possible with the format and then choose to play with the stream instead of against it.

Of course, every player is different, and further architectural improvements are still needed. But I do think LLM-IF has a future as a medium.


  1. Action and Interaction ↩︎

Thank you for the comments. I’ve tried to fix some of the issues you identified. A few specific remarks below.

This is actually something I’ve noticed myself: the protagonist knows something about the scenario’s premise and intro before they technically should. The idea is that the player “asking myself” would get helpful answers about what they’re supposed to do. There was also a leak giving the characters information about things they haven’t yet seen, explaining the console quote (the protagonist is modeled as an NPC so they have their own memories). Should be fixed now.

The model needs to understand that I’m in conflict with someone and that we’re disagreeing. It also needs to see that I quoted Brandvold’s words back at them (“wrong and alive”).

A stubborn person who hears a good argument does something with it: they get defensive, change their justification, concede a point while keeping the conclusion, or get angrier. Brandvold mostly doesn’t. When I quoted “wrong and alive” back at her, which is the rhetorical move of the whole exchange, she just said the line again. When I swore, she objected to the swearing and skipped the argument underneath it. When I offered to do it myself, she gave the same refusal. The obstinacy started as characterization and ended as a loop.

NPCs have a defined personality and way of speaking, including phrases they like to use. In the commander’s case, “I’d rather be wrong and alive than right and dead” comes from there. Although real people can have a phrases or motto they like to use and sometimes repeat, it’s probably overdone a bit here. Depending on personality, they could get angry of repetition too.

The commander doesn’t change her stances on rhetoric alone. It’s a central puzzle in the scenario as it was designed. I would see it as a failure if you could just argue and pressure the AI into allowing easy solutions that the scenario designer didn’t consider or want - apparently a major problem with some existing AI-based adventures.

It’s true her responses weren’t perfect. Her “data sheet” has the reason for venting but the reason for waiting wasn’t well explained so the AI had to improvise a bit more than optimal. I’ve tried to fix these issues too.

How it works: a separate “mind” LLM call classifies each thing the player says (claim, appeal, threat, request) and records which of her reasons it addresses and whether she has heard it before. In this case, her fear went up through the exchange, and “I heard you, Amadi” was because your point was already heard. So I think most these problems were in the scenario data, not the engine or architecture.

David Donachie wrote:

I changed the clock’s hands to 1:30
…
Then I examined the clock:
…
So, it didn’t change.

The problem was that although you succeeded in changing the item’s internal state stored by the engine (adding something like hands: “1:30”), the “look” command prints the author’s description held by the engine, without going through AI narrator. It’s mostly a time/cost optimization. The fix now in place is passing the description through AI if there’s a “custom” internal state. Your clock example seems to work better now.

Jeff Nyman wrote:

In my testing work, I often explain it like this to companies: an LLM left alone has no way to tell “I said it happened” from “it happened.” Whether that becomes a defect depends on whether the surrounding system makes the difference visible.

Yes. In this prototype, the engine holds the world state and LLM never changes it directly. The LLM interprets what the player wants, judges some things the engine can’t, and narrates from the facts the engine gives it. If the narrator writes something that isn’t in those facts, it didn’t “really” happen, the engine will forget it.

The bugs here were two ways that can still go wrong. Either it did happen (engine had the clock hands) but the fixed description didn’t show it, or the narrator was given a fact it shouldn’t have yet.

Prevtenet wrote:

Of course, every player is different, and further architectural improvements are still needed.

Certainly. This is still an early prototype. Your reports and suggestions have already improved it - thanks again!

This is actually a good metaphor. There’s often an element of “you might have to make up some BS” in improv if you’re in a scene as a plumber doing plumber things and then you’re told “you’re actually in a hot-air balloon miles above the ground.” Some of the humor of improvisation is making improbable situations work. Not so much helpful when a player is trying to figure out what to do in a game environment.

I tested Cloak of Darkness with an AI-managed interpreter. That game has three rooms, and it managed to hallucinate a bunch more, telling me I could climb stairs to “The Dress Circle” and when I did, I was still in the Foyer. The AI described a salon “to the west” and going that way put me in the Cloakroom in the actual map. Maybe good for liminal horror, not for a cohesive story.

I’ve considered that one potential good use for AI in a game would be to let it improvise character dialogue along parameters since it’s more understandable that a character might have wrong information or decide to lie, but the example above shows this still has a way to go, where “I’d rather be wrong and alive than right and dead” is obviously an authored line the AI was instructed to stick to, but ends up repeating it, making the response no better than if it was just picking from a cycling list and that is the last [stopping] response once it’s run out of things to say.

In the AI case, it was dealing with a conversation response the author did not consider, where with standard procedural generation it is perfectly workable to make a mad-lib slightly random catch-all response to random requests that eventually settles into “Brandvold doesn’t answer, having tired of that subject and having nothing more to say about it,” which signals the player that conversation thread has ended and they should pursue something else. The AI didn’t have this authority and keeps up an argument with the player endlessly without context, potentially mis-cueing them that they might succeed with the right line of discussion.

1 Like

Oh, bravo, I like where this is going. From my own experiments, I’ve become convinced that information siloing—keeping separate context for each NPC that reflects only their own experience of the scene—is the architectural keystone that makes autonomous NPCs and social simulation really work. That seems to be what you’re doing here. Likewise, some way to inject author-written prose is critical.

Some more ideas I’ve played with, in case they’re useful:

  • Interesting things happen if you have the “mind” LLM call also construct an internal monologue for the NPC, which becomes part of their context. This tends to improve plan-following, character motivation, and emotional realism, although it can introduce thought-spirals and hyperfixation if not tuned correctly.
  • I’ve experimented with a “snippets” system for providing larger portions of author-written prose; these are included in the prompting and removed once used (the LLM can either tell you explicitly which passage it is adapting, or you can detect when a passage is used with a lightweight text-similarity test). If tuned correctly, this lets the author write quite a lot of verbatim text that the LLM will emit when appropriate without repetition.

Yes; this is why I chose “immersive theater actor” rather than “improvisational actor.” AI Dungeon is improvisation—it’s full “Yes, and…”. Immersive theater has an actual script, so it has more latitude to say “No, but…”

2 Likes

I always imagined a “director mode” for NPCs where an author could basically have a conversation with an AI-directed NPC and specify parameters.

[!note]- Something like…
[Direct Permabud Frostkeep] You are an antisocial shopkeeper named Permabud Frostkeep (“Bud” to your small number of actual friends.) Take a gruff but sullenly helpful attitude toward PC questions unrelated to purchasing merchandise. When not engaged directly, you may improvise non-world-affecting shop maintenance tasks like sweeping or counting money or dusting figurines. Do not improvise locations or objects that do not exist in the game world that the PC may want to purchase; handle request denials for non-existent object purchases like, “I’ve never heard of that.” “I seen one of those once, but that was years ago.” or “They don’t sell those anymore because they would randomly burst into flames.” If the player mentions an imaginary improvised object, hide it immediately and deny it actually exists. You believe the Fabulous Golden Golf-Ball of Quorn is an absolute myth and should discourage the player from questing for it. You don’t know much about other NPCs but you think Rosalie the Florist might be kind of sweet though you’d never pursue anything. If the player locates and returns your lost wallet to you, you might be a tiny bit nicer to them. If the player asks about the broken Wishing Well, you just know there was a tragic event involving a Mammoth last year but you hold deep trauma about the event and don’t wish to discuss the details. Do not otherwise discuss objects out of scope, preferring a noncommittal grunt to random inquiries that will otherwise lead nowhere. Do not make up any history or any nonspecific information about any game-world object location or character besides general disaffect about why you’d not associate with that person “if they even exist” or “The Orphanage, feh.” If a player locks onto something non-existent you say casually and pursues questioning about a non-important or non-existent subject, humor them a tiny bit then shut down that topic with an absurd reason that you don’t want to discuss further.

1 Like

The game also told me I’d “know which pot”, which I didn’t. So I just said “Look under every pot”, and found it that way.

That’s technically correct, and the LLM both interpreted my command correctly, and changed the right world state, but in terms of gameplay, it just bypassed it completely. If the intent was to allow this to happen, then the puzzle was pointless. If the puzzle had a point, then the command shouldn’t have worked.

The letter says “you’ll know which” flowerpot. That refers to the fact that if you look at the pots, you find out that one of them sits crooked. But you’re right that if you don’t look at them, you wouldn’t know - Aunt Hester may have been wrong about that. In my opinion “look under every pot” is a plausible thing you’d want to do and probably should work. The alternative is requiring the player to type the same command for each pot. One of my ideas was that the game and challenge shouldn’t be in typing repetitive commands but what you want to do, the intention. It’s perhaps not a genius puzzle but it’s the first room of the tutorial. :grinning_face:

1 Like

That’s fair. I was thinking in terms of traditional parser games (which, for the record, I loathe) where looking under every pot would be required, or indeed that you would only be allowed to check one by some restriction or other.

1 Like

I didn’t even read the letter, so I had just done this:

how do I get into the house?

The response:

You remember that Hester’s letter says the key is under one of the flowerpots. The front door is locked.

That’s actually not bad. As a player, I didn’t investigate much, but the player character (protagonist) had knowledge I didn’t. Then I did this:

find the key under the flowerpots

The response:

You tip up the yellow pot. Underneath, in a little nest of dry earth, is a long old key on a loop of blue ribbon.

Notice that it managed to pick the right one, of course, because it knew, even though I did not. Slightly better might have been to say that, since I didn’t know which flowerpot, I searched a few. That would, of course, just be for local color to the narrative.

I don’t actually count this as too bad of an experience. I asked a direct question, and the game kept it in the context of what I (player character) knew but what I (player) did not. That’s an interesting ludonarrative aspect. You could imagine how this gets more complex as scenarios become more complex, where the questions I pose to the game world (perhaps framing them as the player character’s inner monologue) allow the game to generate responses based on in-world knowledge.

2 Likes

I’m having trouble imagining how they figure that out, though. When sycophantic LLMs try to “yes-and” everything you say, how will the player know when “yes” means “yes” and when “yes” means “no”?

After all, isn’t the whole point of having an LLM improvise details that it’ll never have to say “no”?

1 Like

IMO, well-designed LLM-IF should push back. Half of the architectural challenge of LLM-IF is getting the LLM to actually do (in its internal world-state) what the player asks; the other half is stopping the LLM from describing something it can’t actually do (or which would deviate too far from what the author intended). Both halves are difficult, necessary, and (I think) achievable.

Some examples of mechanics I’ve tried (YMMV):

  • A command prefilter that stops with an error if the command references an unknown object, is not a plausible short-term action, or mixes actions (which belong to the player) with consequences (which belong to the engine);
  • Separate “action” and “writing” phases that provide multiple opportunities to catch and reject impossibilities, fabrications, or out-of-scope content;
  • A separate omniscient “director” LLM thread that periodically steps in to give “stage directions” to the NPCs if they’re getting off-track;
  • Generally prompting the engine to be OK telling the player “no, that doesn’t make sense”;
  • etc.