Building a Conversation Simulating Engine (in RenPy/Python)

Authoring General Design Discussions
1

I am a sci-fi novellist about to make my first proper attempt at making IF (in the form of a RenPy VN). I have basic coding experience, so should be able to handle the modest demands of RenPy with practice.

Where I want to do things differently is to find a way to build a mechanic that simulates relatively free flowing conversation between characters. I would appreciate any feedback or suggestions on my thoughts so far.

The plan is to generate a set of around 100 small conversation based branching decision trees, each on a narrow topic (with say two branch points and four endings on average).

The difficulty lies on how to allow the player to explore all these options without getting overwhelmed or lost in excessive randomness. So lets say we limit the player to a choice of 3 new topic trees after the previous one is exhausted.

I need to balance narrative cohesion with player freedom, so this is my plan.
Choice 1. A random selection from a range of 2-3 hand picked topic trees that most directly flow from the previous topic and decisions.
Choice 2. A random selection from a range of 10-20 topic trees that are categorised into the same broad subject as the previous decision.
Choice 3. A random selection from all other topics (topic trees will be removed from the selection pool after being selected once for all options).

This is intended to replicate the feeling of having a “getting to know you” conversation (like a first date), focused on allowing the player to get a partial slice of the NPCs character’s backstory. In any single playthrough only a small percentage of topics can be chosen in the time allowed. Hopefully this creates an “iceberg” feeling around the characterisation, and encourages replay to explore the NPCs full backstory.

The workload seems manageable here- ~100 short dialogue trees, ~400 x 3 hand selected topic that are the best fit to follow on, broad categories of topic types. I plan to only make 2 characters this complex in the first game, with a few smaller models earlier in the game for me to achieve proof of principal. I have a professional programmer on board to help me get the new mechanic working (after which I should be able to adapt the code for new characters as needed).

The thinking behind this mechanic is one approach to avoiding exponential path bloat, by breaking the game into three distinct phases:

  1. Establish premise and sketch main character. Offer only a few choices off the main narrative (leading to death/failure of the MC- restarting hurts less 5 minutes in). Possible small versions of conversation simulation engine with low stakes to introduce the new mechanic.
  2. Establish major characters through conversation simulation engine.
  3. Plot focused climax, rapidly branching paths (using information/stats/flags accumulated from step 2 to influence outcomes).

Keeping the branching narrative to the very end limits the amount of time for excessive numbers of paths to accumulate. Separating characterisation from plot (via the conversation simulation mechanic) could be one approach to solving this issue. It might be worth noting I am aiming to create a VN with a 30-60 minute playthrough time (but designed to be rewarding for multiple playthroughs).

Any feedback/critique/suggestions are welcome.

5 Likes
2

It sounds like you’ve re-invented the “storylet” idea which has been kicking around here for the past few years. E.g. this thread: What is Storylet to you?

Emily Short has written extensively about this sort of structure, which she used in the recent Mask of the Rose (Failbetter Games). See this blog post and its attached and followup links: Storylets: You Want Them – Emily Short's Interactive Storytelling

5 Likes
3

That’s an intriguing idea, Haldane.

I believe Storylets (mentioned by Andrew) are like possible cards drawn from a deck. If the correct conditions are met, then the game would draw the best conversation candidate cards. I’d imagine you’d have to decide what variables to track in order to dictate where the conversation might go and what cards the game would determine to be the most appealing/logical to present to the player.

Like, a conversation card will show up if the player has shown enough empathy or liked/hated something the NPC likes/hates. It’s one thing to pick a random topic from the pool and it’s another to pick one or two that kind of relates to what was being discussed during (or near the end of) the previous conversation trees.

I’d imagine a more organic approach would be welcome with the initial starting topics being the actual random parts and the rest being procedural based on the choices made.

Food for thought.

3 Likes
4

Thank you both of you for those links. The model I have in mind is indeed a storylet system of sorts, though my method for giving some structure to the system is a little different to the models outlined.

I will study all the previous examples to see if there might be better ways of tying the conversational topic swarm together. I’m not sure the model I have in mind can be easily graphed since there is a small chance that any two topics can be connected- the best way to visualise it might be topics connected by paths of varying darkness- 2-3 black lines for the hand picked choices, dark grey lines for the 10-20 topics in the same broad category, and faint grey lines to every other possible topic. That. way of visualising the system might help immediately communicate the messy but partly structured tangle that a real conversation can be (where you are free to jump around to any topic, but in principle tend to follow a flow between related subjects).

Thank you for cracking my chicken and egg/not knowing the established terminology to search for more information conundrum. I’ll get busy reading and building small paper models to playtest my ideas.

1 Like
5

Here are some things i discovered that might be helpful;

Agency

I need to balance narrative cohesion with player freedom

Can your players choose to break out of conversation, do something else, then go back to conversation, picking up where they left off?

If not, people get frustrated and bored and feel they are trapped in conversation.

People don’t like to have to replay conversation trees they’ve already seen. They just stop playing.

Breaks

This is intended to replicate the feeling of having a “getting to know you” conversation (like a first date), focused on allowing the player to get a partial slice of the NPCs character’s backstory

For this, i invented “the break”. In some cases the player initiates conversation and in some the game character does. When the game initiates it, you need “the break”. This is an initial dialogue that automatically drops out of conversation, leaving the player to choose to do something or talk.

eg. Game drops out of conversation to provide agency;

You go inside the cottage.
Looking around inside the strange cottage you see a plethora of unusual things; _potions_, _jars_, _books_ as well as magic trinkets like charms and _some silver stars_.
Standing here, a strange slim woman dressed all in black stares at you with wild fiery eyes that feel as though they are burning into you. This makes you feel somewhat uncomfortable.
In the corner sits a large _black cat_, eyeing you suspiciously.
At the back of the cottage, you glimpse a rather _unusual stone_.
“Greetings traveller. I am Akiko, maker of magic potions and lucky charms. Who are you?”
(1) Tell her your real name
(2) I’d rather not say
? 2

> I’d rather not say.
“In that case, I’ll call you Chad.”
Akiko continues, “I have here some of the finest and most powerful _potions_. Pray, what is your trouble, as for sure I have the cure for it. Or are you here for better luck, well, specially for you I also stock some of the most amazing lucky charms. Perhaps one of these cost-effective, shiny _silver stars_ with handy self-adhesive backing. A veritable bargain.
Have a look around, take your time and browse. I’m sure you won’t be disappointed.” Akiko begins stacking some books.
(1) Talk to Akiko
(2) Go Out

Dialogue Blocks

It’s necessary to package the dialogue into separate “bubbles” or “blocks”. Some blocks are background info and some are directly related to the story.

It would be possible to have one giant dialogue definition, but this quickly becomes unworkable in practice.

For this i have choice inheritance. I don’t think anyone else does this. So the choices are authored into blocks and plugged together by choice-based inheritance. It’s a form of inheritance because some minor background blocks can also appear as parents to different conversations with the same character. And you don’t want to author them twice.

Also blocks of conversation become conditional on game state. Like when the characters is asleep and unable to reply or when they become angry, they might not want to talk about their personal hobbies or interests, for example.

2 Likes
6

This sounds very sensible and thank you for taking the time to share your approach.
I am going to build and playtest two models in my upcoming project.
Model 1.
Each storylet focuses on one particular small topic, and branches in 2x2 steps. Each of the four endings is crafted to marry up with the beginning of another storylet. The whole spaghetti junction of paths meanders forward through the possible topics in a fairly large range of possible combinations. It will be impossible to exhaust the whole storylet swarm in a single playthrough (so freshens up replays a little). None of the storylets is critical to the rest of the game in terms of variables, but the information might be of minor use to later decisions. The ~19x storylet swarm helps establish the main character, setting, world building and stakes. Each storylet can only be encountered once. The trigger for the conversation end will be when some threshold of insufficient remaining valid storylet choices is reached (or maybe just a timer as well to control story pacing).

Model 2.
Each storylet has a more substantial body on a particular topic (as in multiple lines of linear back and forth dialogue). The storylet ends in a three way choice- 1. Follow the most obvious narrative path from the preceding text with 2-3 choices, 2. Divert to a limited range of tangential storylets under the same broad topic, 3. Change the topic to a small selection from any storylet remaining. (plus the option to end the conversation at any time- I plan to have two characters available for conversation in this stage, so the player can spend more time with the one they like best. Investing more time in either character will unlock other options in the final heavy plot phase of the game.

I do question your point about how players hate skipping through dialogue they have seen before. I agree that in branching narratives people often skip until they reach an unexplored branch point (which I think detracts from the impact of that version of the story as a whole). Replays become an exercise in satisfying curiosity, rather than a complete story in itself.

I am reminded of a sexology study that found people got more bored watching the same erotic scene on repeat, but their interest jumped again when the same actors did something different. In contrast, kids watching blues clues find the spaced repetition of the same episode through the week gives them a sense of familiarity and control.
My theory is that you don’t need to change all the text for a replay to have value, just a sprinkling of unpredictable new elements mixed with familiar ones. Maybe this is a bit like rereading something on the scale of a novel, where sections that your mind skimmed over the first time come into sharper relief on a second look. An optional meter in each section which shows the percentage of dialogue options explored might be appealing to some players to give a sense of progress/completion.
I am hopeful that my approach of hand crafted storylet tangles, where it is virtually impossible to follow a simple playthrough guide to the “best” endings, will engage the player enough to make replays a balance of comforting familiarity with enough crumbs of novelty to make it feel rewarding.

That is the theory at least. I am a beginner to this style of writing (the last few years getting better at linear fiction only half prepared me for the challenges ahead). I’m confident I have the determination to push through to a finished product, and even if it isn’t perfect in the end I will be satisfied to have at least tried to do something novel. That is what I love about the interactive fiction world- so many experimental ideas are waiting to be tested out and you all seem like such lovely and fascinating people to share the journey with.

3 Likes
7

Both your storylet models sound interesting.

So really my points are not specifically about the specific features of storylets, but instead about authoring.

In principle, given any logic required, it’s possible to code it up and get it working, but my focus in this area is more about how to author it easily.

So, i hope someone else can correct me here or fill in details, but as far as i know systems like Inform do not have storylet specific authoring directives. I understand there are some useful extensions in this area but, i hope I’m correct in saying, Inform does not have storylets per se.

Sure you can code them. And you can create equivalent logic, but that’s like saying I can write IF if i have a C compiler.

What am i talking about?

Right, so in your case, it seems to me the problem is to, in advance, design and build a generic storylet authoring system as part of your IF initiative. Then use this to build your dialogue.

So, if this is Python, what will it be?

either’

  1. a Python library
  2. a translator that generates python from another definition
  3. something else

These are the questions to tackle.

2 Likes
8

I chatted to a programmer friend yesterday who is keen to help me overcome the technical hurdles. I plan to build the visual novel in RenPy (and learn the very manageable coding demands there) but the algorithm for selecting the range of possible storylets to present next is the key mechanic that will take a bit more skill than I am prepared to develop personally.

I just completed a smaller scale model (complete with writing) composed of ~20 storylets (each on a different focus topic). Inside each storylet was a 2x2 decision tree. The four end points linked back to other storylets, or terminated the storylet swarm. Writing it was a mind bending exercise.Often a storylet would initiate from 4 different termini, and choosing wording that smoothly linked to all of them was a fun linguistic challenge. Likewise steering the 2x2 choices to a preselected range of adjacent storylet topics was fun. It was almost like the conversation was constantly veering off in another direction (which real conversations do much of the time). I can see twine being a useful tool during the drafting phase (and for play testing to evaluate conversational smoothness/continuity).

Currently the programmer and I are working on a larger toy model with 100 storylets to get the programming working in RenPy. Each storylet is linked to two adjacent storylets (A1/A2), two close ones (B1/B2) and two distant ones (C1/C2). Based on my initial experience drafting the dialogue for the 20 storylet swarm this feels like a manageable level of narrative interstitching. The player will be presented with a choice of A*/B*/C* (one of each category randomly selected). My instinct says this will be enough randomisation to make each playthrough feel unique (only a 1/8 chance of seeing the exact same three options on a replay).

Your point on the authoring issue is right on the money. The raw data for the swarm will look like a 100 x 6 table of potential links, or a list of six potential links at the end of each storylet. I could probably program the latter approach manually into renpy, but that feels like a very messy solution which will mean duplicating a whole bunch of fiddly programming work every time I want to create a new conversation simulator with a different character. So the ideal is to come up with a python module that renpy can call to juggle all this information (maybe put the storylet swarm data in an external file to the renpy code). This level of programming is beyond me, but at least I have a pretty clear idea of the mechanics I want in the end.

The ideal authoring format for me would be the ability to write all my storylets, keeping track of how they can link to each other in a huge table. Then the program we develop is able to interpret that information to present the player with a convincing conversation with an interesting character that cannot be boiled down to a walkthrough guide since every replay will wander in random (but conversationally connected) directions.

As always- your interest and thoughts are much appreciated on this topic

9

This is indeed most interesting and i am intrigued by your approach. Personally, I would be worried about having to design the whole thing in a giant table. I tend to prefer more organic designs that can be be extended. I would be worried by the table approach because it might not be straightforward to extend or modify - say you needed to introduce another component of the conversation, prompted by a game design change, for example.

Nevertheless, I don’t think there is an easy way.

So;

Have you tested out your smaller 20 storylet model in a playable way?

Because if not, and you are agreeable, I could have a go to see if they could be accommodated by my conversational system (which is not python). The system is not specifically designed for “storylets” but I am interested to see if it can do them and it would be an interesting challenge.

This means you will have to send me your 20 storylet model. Unfortunately, they may be confidential to your project. If so I understand.

But, if you are prepared to share the (you could DM me). I will try to create a playable version on a private web page, which you could play out - or even a public page if you wish, to get others’ feedback.

Let me know. It would be an interesting experiment.

2 Likes
10

I’m looking everywhere and seeing no options for direct messaging you directly jkj yuio. Happy to email is to you (mine is shane.simonsen@icloud.com).

The early toy model of the ~20 storylet conversation isn’t something I am precious about sharing- it took me a few hours to draft it. It is rough and I expect it will need either polishing or redrafting in a different structure. I put it into twine so I can send you the html version to playtest. I am particularly interested to get other perspectives on how well the story structure fits in the unusual structure (I plan to playtest it myself today after leaving it rest for a few days).

Also- I am still on the hunt for a python programmer who I could potentially recruit to build the extra functionality into RenPy, so I am very open to suggestions of people to approach. I have a modest budget to compensate someone for the time spent. I am particularly hoping I can come up with a system that makes writing new conversation simulators relatively painless. My nightmare is having to hand code in RenPy in multiple locations every time I build a new conversation simulator.

I did a rough estimate today, and I am aiming for my largest conversation simulators to have 100 storylets, each about 200 words long (I’m viewing them each as a piece of flash fiction). That will mean each one contains about 20 k words of content with 200 hand crafted direct connections, and the first game I am planning has two such large conversation simulators. Add in a few smaller versions of the simulator plus general writing (and the plot focused decision tree at the end of the game) and I will be looking at about 80k words of writing work (doable- my last novel project was 160 k in the end).

1 Like
11

I’ve sent you an email. If you’re interested in letting me try out your 20 storylet model, i can give it a go.

1 Like