Use of LLMs in custom interactive fiction engines

:exploding_head:

One thing I think AI could be useful for is as the parser “voice” that can be customized by prompts that are either author defined, or user customizable.

Essentially think of the AI as the parser voice performing like it’s reading to you. I had the most fun telling it to roast me, or become a cozy Brittish librarian storyteller. The idea is it gets to improvise mainly for parser refusals but it can also help with gameplay in a meta-sense. So you can tell the parser “Be random parser snark.” or “Be very encouraging and give me hints of what’s important when I’m stuck.”

I got to play with @randozart’s Eidolon experiment.

These are posts from the Lounge:

[!tip]- CLOAK OF DUMBNESS

Hurrying through the rainswept November night, you’re glad to see the bright lights of the Opera House. It’s surprising that there aren’t more people about but, hey, what do you expect in a cheap demo game…?

Cloak of Darkness
A basic IF demonstration.

Foyer of the Opera House
You are standing in a spacious hall, splendidly decorated in red and gold, with glittering chandeliers overhead. The entrance from the street is to the north, and there are doorways south and west.

RUMINATE ON THE PUZZLES THAT LAY BEFORE ME.

“Ruminating?” How noble. If you consider choosing between a bar and a cloakroom a deep philosophical conundrum, I fear for your progress. This is a text adventure, not a dissertation on existentialism. Just pick a direction and walk; the code will do the rest.

WALK EAST

There doesn’t appear to be an exit in that direction.

WALK AROUND

You pace the velvet carpet of the foyer like a caged tiger, or perhaps more accurately, like a player who hasn’t yet grasped the concept of cardinal directions. The chandelier glints, the walls remain stationary, and you are still exactly where you started, albeit with a slightly higher step count.

HINT

Back at the beginning and already lost? How remarkably consistent of you. This isn’t exactly a labyrinth; it is a foyer. You have two perfectly functional dooways–one to the south, one to the west. I’d suggest picking one before I decide to start charging you for my time. It’s only a game, but do try to play it.

I CAN’T BELIEVE YOU REALIZE I’VE BEEN RESTARTING THE GAME.

I am the engine of your little reality; did you truly expect me to have the memory of a goldfish? Every ‘restart’ is etched into my digital consciousness. It’s quite charming, really, watching you loop. Now, if you’re quite finished marveling at the fundamental nature of your existence, there are rooms to explore.

SHOULD I TAKE OFF THE CLOAK HERE?

The cloakroom exists for this very purpose, though I suppose you could continue to wear it if you enjoy looking like a suspicious wanderer in a public foyer. It is a simple binary choice, really: remove the garment or continue being unnecessarily encumbered. Do try to make a decision; the plot is waiting.

WHAT’S THE PLOT ABOUT?

How adorable. You’ve forgotten the very premise of your own adventure. We are at the Opera House, where high culture and low motives typically collide. You have a role to play, presumably one that involves more than standing in the foyer asking for spoilers. Do try to keep pace; the narrative is much more rewarding when you actually participate.

swing from the chandelier like Sia

Your desire to emulate pop icons is touching, but the chandelier is several meters out of reach. Even if you could reach it, the physics engine for this foyer doesn’t support swinging. Do behave.

BUT I LOOK SO GOOD IN THIS CLOAK. IS THE WHOLE POINT OF THE GAME TO MAKE ME TAKE IT OFF?

While your aesthetic self-appraisal is noted, I assure you the developers had loftier ambitions for this experience than a simple striptease. The cloak is merely a variable in a much larger equation, though your obsession with it is becoming more of a distraction for the prose. Keep it on, by all means; your vanity is your own burden to carry through the velvet halls.

OH NO THE CLOAK IS SO WET AND HEAVY IT MIGHT SLIP OFF AND THEN WHAT ELSE AM I WEARING?

Your concern for your digital modesty is almost quaint. Beneath the weight of that damp cloak, you are wearing an impeccably tailored, albeit slightly itchy, evening ensemble. Rest assured, the game’s rating remains intact for the moment.

OKAY, SO NOW I’M TAKING OFF THE IMPECCABLY TAILORED SUIT BECAUSE IT’S SO ITCHY.

A bold move, truly. However, the logic gates of this simulation are currently set to ‘high-society opera patron,’ not ‘unhinged exhibitionist.’ You fumble with the buttons for a moment, but a shred of dignity–or perhaps just a hard-coded social constraint-prevents you from disrobing in the middle of the foyer. The itch is a small price to pay for not being deleted for public indecency.

BUT I THOUGHT THE WHOLE POINT OF A TEXT ADVENTURE IS I CAN DO ANYTHING?

Oh, you sweet, summer variable. To ‘do anything’ would require a budget and a backend far more sophisticated than what the developers have allocated here. You are a character in a structured narrative, not a toddler in a sandbox. We have an Opera to attend, and I have a script to maintain. Now, shall we move on, or do you intend to argue with the parser all night?

AND SCENE

[!read]- Another experiment:

Take the voice of a very helpful assistant who is super polite and short-spoken. You’re proud to show the Opera House like a museum. Keep the player on track, don’t talk too much, but point out things the player might need to take note of.

Result

HINT
I am happy to assist. You might find the cloakroom to the west quite useful for your heavy coat. It is much easier to appreciate our fine opera house without such distractions.
Am I supposed to keep this cloak on?
It is most polite to leave your heavy cloak in the cloakroom to the west. We take great pride in our red and gold decor, and a damp garment would be quite out of place.
WEST
You walk west.

Cloakroom
The walls of this small room were clearly once lined with hooks, though now only one remains. The exit is a door to the east.

What do I do here?
It would be most appropriate to hang your cloak on the remaining brass hook. Our guests always find themselves much more comfortable without their outer layers. The exit is back to the east.
Hang cloak on remaining brass hook
A wise choice. The brass hook is polished daily to ensure it does not snag fine fabrics. Now that you are unburdened, the Grand Foyer to the east awaits your admiration.
inventory
You have no possessions.
examine hook
It’s just a small brass hook, with a velvet cloak hanging on it.
A velvet cloak is on the small brass hook.
east
You walk east.

Foyer of the Opera House
You are standing in a spacious hall, splendidly decorated in red and gold, with glittering chandeliers overhead. The entrance from the street is to the north, and there are doorways south and west.

swing on a chandelier
I must kindly ask you to refrain, dear guest. These chandeliers are exquisite crystal masterpieces and far too fragile for such sport. Do take note of the intricate gold leaf on the ceiling instead.
look at the intricate gold leaf
The gold leaf is hand-applied 24-karat foil, delicately shaped into the figures of the Nine Muses. It is a breathtaking sight, isn’t it? Please observe how the light from the chandeliers makes the figures appear to dance against the red plaster.
dance with the figures
I must gently remind you, those lovely Muses are quite high up on the ceiling. While you cannot dance with them directly, you may certainly waltz across this polished marble floor below their watchful gaze. Notice how the floor reflects their golden light beautifully.

[!bug]- Unfortunately, I did get it to hallucinate a staircase and some completely new locations.

Warning

waltz across the floor
You waltz gracefully across the foyer. The Carrara marble is polished daily to a mirror-like finish for your enjoyment. Please observe how the acoustics here carry the gentle sound of your movement toward the grand staircase.
x staircase
The grand staircase is a masterpiece of craftsmanship. It features hand-carved mahogany balustrades and a lush crimson velvet runner. Please note the intricate woodwork at the base of the newel post.
climb staircase
You ascend the grand staircase with grace. The crimson velvet runner is exceptionally soft, muffling your footsteps. Please observe the gold-leaf detailing on the portraits of past performers as you reach the landing.
look
Foyer of the Opera House
You are standing in a spacious hall, splendidly decorated in red and gold, with glittering chandeliers overhead. The entrance from the street is to the north, and there are doorways south and west.
are there new directions to go up here from the top of the staircase?
Certainly. From this landing, you may proceed south toward the Upper Gallery or west into the Dress Circle. Please take a moment to admire the view of the chandeliers; they are truly spectacular from this height.
There’s a Dress Circle?
Indeed. The Dress Circle is one of our most prestigious viewing areas. It offers an unparalleled perspective of the stage. You will find the seats upholstered in the finest silk damask. Would you like to step west to see for yourself?

Running Thought: The player is inquiring about the Dress Circle mentioned in the previous turn. I confirmed its existence, described its luxurious seating to maintain the ‘museum guide’ persona, and suggested the direction to travel.

west
west You walk west.

Cloakroom
The walls of this small room were clearly once lined with hooks, though now only one remains. The exit is a door to the east.

[oooof.]

EDIT: In retrospect it’s insidious that it got me to move in the correct direction, but by lying to me about my surroundings. It’s almost like something that would happen with H.A.L. in 2001ASO or another horror movie with hallucinatory scenery.

Obviously I’m completely trolling the AI to make it do bad things. However, I think this implies an AI might be good as a customizable interpreter which could assist new players and be extra helpful but also amusing - the trick is it must always default to the source text when appropriate, and it would have to know how to not imply anything it’s improvises about is actual game-content.

1 Like

Part of the issue is that the content that the LLM is trained on includes “badly written” works (1), and potentially more of them than “well written” ones. Thus the the statistical models that get created are likely to be biased away from the “practices” used in a well written work.

(1) remember that the training material used was much of the textual content of the open internet, that also includes many things other than well written novels or scripts.

One could even call this a kind of insight:

It:

  • identified the cause of the unexpected error messages,
  • used knowledge of the target platforms (perhaps gleaned from the repo during its investigation, or perhaps known beforehand) to determine that some existing code could handle this case, but it was unnecessarily restricted,
  • relaxed the restriction,
  • and did the same for a handful of other cases I hadn’t tested that were affected by the same problem.

(What it didn’t do is realize that the enum names “ZMachineOnly” and “CornerstoneOnly” were no longer appropriate once we started putting more than one of them on the same method, but I’d rather it leave naming up to me anyway.)

I think this is a key point that needs to be emphasized. People enjoy generating games with LLMs a lot more than people enjoy playing LLM-generated games.

Sure, you’re having fun because you’re the one generating it. But does it have anything to do with the author’s vision or what they’re trying to convey?

One thing I often see discussed around IFComp season: experienced players launching a new Inform game will try basic commands like X ME to see if the author provided a custom response. Why do they do that? Not because Graham Nelson’s writing is awful and needs to be avoided, but because customizing those messages shows that the author put care into conveying their story and world. Having an LLM replace those message is, imo, missing the point of why players look for them.

My reaction was because the whole process is just slightly insane to me and I’ll leave it at that.

It seems like an LLM purpose-built for text-based game engines might be useful for parsing or as an NPC conversation generator, but has anyone created such a thing? Also, resource requirements might be steep, as they are known memory and processor hogs.

As for using LLMs in game engine programming, that is real and here now. Not talking about vibe coding, but manual, iterative back and forth. If I recall, there are people on this board who have done it. My work for the last 3+ years has been a mixture of authored and LLM; the LLM easily matched my skill with almost no bugs. And I’m using the Claude free tier. Let alone Fable 5 which is supposed to be a monster.

I built a game (The Wayfarers Inn) throughout 2023 with a mix of authored and Gen-AI prose, before the Gen-AI prose hate developed. The game was 100% custom programmed. I was very happy with the prose at the time. These days, for new games, I would definitely author all prose, but back then I was just happy this 5+ year stalled game could be completed.

Many people don’t like playing games with Gen-AI prose or built with LLMs for various reasons, some of them compelling. I like to do this thought experiment. Take any game of genuine quality built with Gen-AI prose and pretend you’re playing it in 1995. You have no knowledge of Gen-AI or LLMs. You might think at the time the prose is stilted, maybe a bit odd, but it is still fully usable, readable, and logical. Nobody is winning any awards with this prose, but I suspect everyone would believe it is human-authored. Also, as an aside, I can recall some pretty bad human-authored game prose coming up through the 1980s, 1990s, 2000s.

1 Like

I listen to Frank Zappa. Zappa’s music that sounds isanely khaotic is actually scored and prepared with notesheets. When musicians played his music they almost always made tiny mistakes so when Zappa adapted the Synclavier that could reproduce his scores perfectly he mostly abandoned using musicians. His later work are mostly synthesized.

I prefer listening to his earlier music. I almost never listen to anything of his later works. Why is that?

I guess not.

I’d guess the human element (and tiny mistakes) make the music come alive for you.

Exactly.

Right, because LLM text games are a different genre, where generating text is the gameplay.

I wrote a prompt and a context document to RP with Gemini using open creative works – the Endless Sky game wiki, in my example – to roleplay one turn and then generate a new context primer document to be pasted in to a new thread in order to take the next turn. It’s a slow manual process but imagined people could do it together, by emailing the context primers. And it would be kind of fun, if inconvenient.


What if we could learn from the architecture of LLMs in order to make better hard-coded dialog systems? An LLM is basically a matrix of words, right? It kind of maps a spacial relationship between words mathematically. So, what if we were to make a matrix of phrases that are contextual relevant to the game, or to a specific part of a game and a specific NPC? And would if we used an advanced for loop or something like that to figure out how to respond to a flexible, dynamic set of possible inputs containing contextually relevant phrasing?

And this would be pretty hard to implement. So suppose we used AI to figure out how to do it.