Continuing the discussion from Creating a persuasion-based Detective story with AI elements in dialogue:
So to give some framing, I’ll sketch out the outlines of the relevant bits of the game design I’m working on (because it informs how I’m framing NPC decision-making).
In terms of genre, I guess you’d call it non-Lovecraftian cosmic horror. One of the central ideas, in terms of things I wanted to work on and why I’m writing it as an IF game, is that all of the major characters are dealing with regret. Specifically regret over the things they haven’t done, choices they didn’t make. The road not traveled, that kind of thing.
The game is set in a small town (a formerly mildly prosperous rural tourist trap that’s well past its prime). At the start of the game, all the NPCs have set “agendas”, which basically just means they each have their own routine: get up in the morning, go to work, that kind of thing. Think of Majora’s Mask or, more recently, something like The Outer Wilds. As the game’s narrative advances, conditions get progressively worse, and one of the things I want to model is the slow degradation of these normal behaviors. I don’t want it to be just scripted intervals: player does foo, that triggers a scripting flag and boom all the shops in town are closed.
Another core game design decision is that I want almost all puzzles and puzzle-like activities in the game to have one or more NPCs that can help the player solve them. Either by coaching (giving hints in conversation), or (at the player’s prompting) to just solve them outright. Basically as a guard against the player running up against a kind of puzzle they bounce off of and having their overall advancement blocked.
Here I’ll use made-up gameplay situations using a UFO crash landing and a zombie outbreak. That’s not what the game’s about, but it’ll be easier to talk about things in concrete terms, instead of making veiled references to actual gameplay situations.
So at some point in the game if the UFO has crashed outside of town, zombies are roaming the streets, and the player needs help from a biologist, say, at the crash site. But the biologist has boarded themselves up in their home and won’t leave.
I think the “traditional” approach to this sort of thing is to basically have scripting that encodes the stuff the biologist specifically cares about, and so “recruiting the biologist” becomes either something like a fetch quest (obtain the equipment from the overrun lab and return to the NPC) or a fill-the-affection-meter task (do a bunch of little things for the NPC, each of which increments a counter which eventually crosses some deterministic threshold past which they become an ally).
The approach I’m taking, which I touched on in the other thread, is instead to model each NPC’s overall drives and mental state, and then frame each decision in terms of what satisfies those needs and desires, and have NPCs make decisions based on their evaluation of what is currently most compatible with their overall wishes.
In terms of authoring interface, NPCs are built using a sort of “personality quiz”, loosely modeled after the one used in Ultima IV. Except instead of selecting a best match from a set of pre-defined archetypes, the quiz questions each apply adjustments to a nominal baseline model, making the NPC more sensitive or less sensitive to certain kinds of stimuli, as well as tuning their baseline mental state. Example:
Blockquote
Question 1
You’ve boarded up the windows, blocked the doors, and now you and the other survivors, hungry and tired, are huddled in the darkness listening to the howling things outside.
A: Quietly gather some supplies for yourself and hide the remainder.
B: Wait for dawn and then slip out for a resupply run without telling anyone.
C: Huddle with the rest and wait for whatever is coming.
D: Start directing the others, making sure they recognize you’re in charge.HELP or ? for help.
>
The kinds of adjustments should be fairly apparent (more greedy versus more risk tolerant and so on) and to be clear it’s an authoring tool rather than a “real” personality test. The main goal is to just mean that I don’t have to hand-write the numerical values (discussed below) directly.
There are also bits for defining what sorts of inhibitions, if any, the NPC has (specifically for allowing disinhibition to affect decision-making in some circumstances) as well as what kinds of impulsiveness, if any, the NPC is prone toward (generally things like is the NPC more likely to act impulsively if they’re felling good or when they’re felling bad, when they act impulsively does that just amplify their normal behavior or are they likely to do things they wouldn’t otherwise do, and so on).
What that produces is a baseline mental state, which is a vector (just a list of numbers) over a set of pre-defined mental channels: fear, anxiety, fatigue, pain, anger, shame, social isolation, and so on. Because this is a horror game, all of the channels are “negative”. That is, a large positive numeric value is “bad”, and generally positive mental states are negative numeric values. Everything is done using integer math because this is all in TADS and floating point in TADS is slow.
At this point I’m going to start using some jargon, and I want to give an explicit disclaimer. I’m a programmer who read a bunch of papers on psychology to gather material for writing a video game. I’m going to use a lot of psychological terms, but most of them are being used in a slightly different sense than they are in the literature, and many of then are used very differently.
With that in mind, the vector described above is called the “disposition”. It is the durable baseline the NPC tends toward. There’s also a mutable vector, the “activation”, which gets tweaked by interactions with other characters and the environment. In the normal course of things the activation regresses by itself to the disposition. At any given point, most decision-making uses a mental state vector that’s the mean of the activation and the disposition.
The mental state represents how important that channel is to the NPC. So a high fatigue value doesn’t mean the NPC is currently tired, it means that they react strongly to being tired.
There’s also a vector, again with exactly the same channels, for physical drives. These represent how [whatever] the NPC currently is. Some of these are tweaked monotonically by passing time (like hunger and fatigue) and are automatically reduced by in-game interactions (like eating and sleeping), while others are tweaked by feedback from decisions, by scripted events, and so on.
The drive and mental state together form, and sorry for more jargon, the desire pre-image. We’ll get to the desire vector in a bit, but it’s defined in terms of what things provide, called affordances. Affordances are things like comfort, social connection, and so on. The desire pre-image isn’t in those terms, it’s just the net mental state. By default it is computed using the drive vector as a scaling factor that grows the further the drive vector channel deviates from the baseline. That is, a small amount of hunger shapes decision making a small amount, a large amount of hunger shapes decision-making more.
To translate between the mental space and the above-mentioned affordance space each NPC has an embedding matrix. Each cell in the matrix corresponds to the combination of one mental channel and one affordance channel. The semantic meaning of each cell is “how much does this mental channel’s value change how much the NPC values what this affordance channel offers”. The desire vector is just the product of the desire pre-image vector and the embedding matrix. It can be read as how much the NPC currently cares about each affordance, as a result of their overall mental state and physical drives.
The simple version of decision-making is then to just treat the values of the desire vector as relative probabilities and use them to pick one (norm them so they sum to 1000, pick a number between 1 and 1000, figure out which option the number corresponds to), and there’s your decision.
But that decision would just tell you which affordance they most want satisfied. For choosing between different in-game options, each query starts out with a list of options, each option having an affordance space vector representing what it offers. That is, if you pick this option, these are the affordances (and weights) you’d get.
So now the simple model is to just compute the dot product of each option with desire vector, which results in a scalar (an integer) representing, roughly how strongly, overall, the NPC likes that option. The values for each option are then treated as relative probabilities (as described above), and an option picked based on that.
Of course there are additional complications. What is described above is acting directly on the result of projecting the net mental state through the embedding matrix, the raw desire vector.
But in addition to just computing the raw desire vector, there’s a (very rudimentary) feedforward network(-ish thing) that consists of a series of ordered layers, each of which takes the current desire vector as its input and can process it in various ways to produce an output vector which is then fed into the next layer. And the net desire produced by all the active layers combined produces the final desire vector passed to the sampler (the thing that rolls the dice to make the choice).
The layers handle things like feedback-based learning (developing “preferences” bases on the outcome of choices based on their actual reward, in affordance terms), modelling regret (NPC taking note of what they didn’t get as a result of making choice A instead of choice B, perhaps making them more likely to choose B in the future), adjusting the desire vector based on the NPC’s relationship with other characters if the option is a request from another character or if it involves another character, as well as some fancy stuff involving altered mental states that could affect decision-making (trauma resulting in paranoia, which then places ceilings and floors on some channels until a timeout or external trigger resets the state, and that kind of thing).
Right. That’s the basic mechanics. The main other component is a query builder for easier authoring of query options. As mentioned above, each option involves an affordance vector, representing what the choice offers (or rather what the NPC believes the choice offers).
But writing a bunch of math for every query is messy and error-prone, so instead most queries are built from a library of stock query option types. This covers generic things like agreeing or disagreeing with a general request based on who is asking, to situations like exploring a potentially dangerous situation versus going to ask for help, to confronting someone versus being evasive. These can be scaled, with the idea that, from an authoring perspective, they look something like an encounter in a board game with a bunch of skill checks: explore the creepy library after dark might be an “explore a dangerous place” task with a difficulty of 1, while exploring the ancient temple crawling with torch-bearing acolytes of the Dark God would be a “explore a dangerous place” task with a difficulty of 5.
Similarly, there’s an “appraisal” system, where NPCs “see” tags on objects sorta like they’re item cards in something like a board game: this is a “tool item, utility 1” versus this is a “quest item, utility 3” and so on, with per-NPC tuning values automagically converting items tagged in gameplay terms into affordance terms evaluable by the huge mess described above.
Anyway. That all sounds pretty heavyweight. And I’ve spent a lot of time hammering on it (and it isn’t done yet). But on a turn-by-turn basis even in the fairly sluggish TADS3 VM, on an ~10 year old laptop individual decisions (with an unrealistically large number of bells and whistles enabled for testing purposes) still only take on the order of ~100 μs. So even in my comically overdesigned worst case test scenario (400 NPCs navigating a 400 room maze) the decision logic still takes less time than T3 takes just updating the location of 400 objects.
That’s a lot of text. I’m willing to talk more about details if anyone’s interested. I’ll eventually make the T3 code available like almost all the other modules I’ve developed for this project, but I’m planning on keeping the NPC decision engine and the procedural map generation stuff private until after release, whenever that is. Don’t mind discussing the details, though.
)