The Magic is in the Sound

Authoring General Design Discussions
1

I’m an audio guy. During the day I have ambient sounds and music playing to help me work, and at night if i’m struggling to sleep then the sounds of rain, rivers, waves, or binaural beats, help me relax. Similarly, whilst pictures in IF games are great, I feel like it’s the fusion of text and sound that truly ignites the imagination, creating a depth of immersion.

Imagine stepping into a dimly lit room described with rich, evocative language. The crackle of a distant fire, the whisper of the wind through ancient trees, and the soft, rhythmic thud of a heartbeat in the dark are not mere background elements but gateways to another world. These auditory cues form a tapestry that our minds weave into vivid, personal experiences.

Well-crafted soundscapes, ranging from ambient noise to ASMR, binaural beats, isochronic tones, and guided meditations, resonate with our inner selves, transcending the physical and tapping into the metaphysical. Each sound element serves as a brushstroke, painting a mental picture more powerful than any visual aid could achieve.

Consider the allure of ASMR videos, where the simple act of whispering or tapping can evoke a visceral, almost magical response. This phenomenon illustrates how our brains are wired to respond to sound in profound ways. Similarly, in interactive fiction, soundscapes can transform a simple narrative into a deeply immersive journey.

The beauty of sound lies in its ability to synchronize with our emotions, guiding us through the narrative landscape with an almost hypnotic pull. Ambience and carefully chosen audio cues can transport us to realms of tranquility, suspense, or wonder, enriching the storytelling experience beyond the constraints of the written word.

While visuals hold a certain allure, it is the synergy of text and sound that elevates interactive fiction to new heights. By engaging our auditory senses, we unlock a dimension of storytelling that speaks directly to the soul, allowing our imaginations to soar and our minds to explore new, uncharted territories.

Adam.

8 Likes
2

I agree, most of my major games in choice use music and sound effects, and I made a couple of I7 parser games with sound but many people either didn’t switch it on, or used an interpreter that didn’t support sound, or played in the browser - although @mathbrush did help me convert it with the Bisquixe web interpreter so it works now.

I know many people play text games because their isn’t sound, and may play their own music by preference. But I agree that I am very inspired by music and sometimes the vibe of the track-list I create informs the story. Lots of the mood of Cannery Vale was influenced by the music and even the titles. One of the tracks was eerie and had a title that mentioned mermaids so that scene was based on the music. The techno track “Harmful or Fatal” influenced much of the night-carnival scenes and I had fun using different segments of the track to build intensity, and there are volume and compression changes to indicate music playing inside a building that gets much clearer and louder when you enter.

Whenever the player is “authoring” their story I really loved how a loop of kalimba music blended with the sound-clip of typing to evoke writing-focus. There’s even a subtle audio clue connecting the writing and the briefcase in the story - the briefcase theme is the kalimba authoring theme only slowed way down - kind of to evoke the Protagonist in the novel is figuring things out and attempting to alter his fate just as the Author is, but isn’t up to speed. It’s dorky, but felt very cool.

The other issue is music and sound isn’t accessible to all players, so you never want to include something like a voicemail with critical information that is available in audio-only.

5 Likes
3

Same as graphics, i find that sounds really helps to bring out the feeling of being in a scene.

I strive to give all my locations background ambiences, then overlay effect Foley’s. It’s surprising just how much it adds, even from just a bit.

5 Likes
4

On the one hand, the addition of sound effects and ambient music would be nice…

On the other hand, I’m dependant on a screen reader and in-game sounds can often clash with synthesized speech in a way that makes the speech hard to understand and the soundscape hard to appreciate.

As such, I would like to try a game that follows the gameplay style of a text adventure, but which fully narrates the game with a human narrator for descriptions and voice acting for NPCs so things are properly balanced between narration and the soundscape instead of having the game and my screen reader clashing.

5 Likes
5

I’m on a constant search for guidance on how to create music of this kind.
Online culture promotes production of dense, extended, percussive tracks.

I think to accompany literature you need more short, subtle, sparse fragments and I haven’t found many good examples of that kind of composition.

3 Likes
6

One of my favourites is State Azure, they make wonderful, ambient soundscapes, some of them are generative so the soundscapes shift and change sometimes over 2, 3 or 4 hours. :slightly_smiling_face::+1:

Adam

1 Like
7

You’ve also got ambient sounds like hiss, hum, static and drone, I enjoy layering these up for some of my ambience videos. If you wanted to check them out then it’s the Analog Ambience collection on The Fog Productions channel on YouTube.

I’ve been heavily influenced by Kyle Edward Ball, his videos are atmospheric due to the layered audio.

Adam

2 Likes
8

I sometimes feel sound can be used as a story aid, something as simple as footsteps through different mediums as you choose walking directions and the background hum of birdsong can set the stage for tension. For example, entering a clearing and having the birdsong and/or music die out, leaving only the sound of your footsteps, triggers some sort of primal alert that things are suddenly too quiet and put us on edge. Probably a leftover due to survival, when surrounding birds and other small critters fall silent when in the presence of a dangerous predator.

5 Likes
9

There is a line with sound effects. I don’t want it to sound like a radio show or RoadRunner cartoon, but there is something viscerally satisfying about a good button-press click or a “jingle” when you collect loot that is subconsciously satisfying in a Pavlovian way when used well.

3 Likes
10

I always have waves or other white noise playing when I sleep if I can’t have my first choice - a good fan with metal blades.

I don’t see the attraction to ASMR. I’ve listened to a few and I always have the exact same reaction: It makes me want to punch someone. I don’t know why, but it bugs the living daylights out of me.

1 Like
11

If you’ve ever got that “pleasant shiver” feeling on the back of your neck when the barber is using the electric clippers, or someone breathed into your ear, or certain crackly sounds that cause unique sensations with your ear drum, or the tingly nerve twitches from someone barely touching your skin and just contacting the invisible hairs that makes them stand on end, that’s the entire goal of ASMR.

1 Like
12

I use a recording of the Irish coast (thanks, @Zed!) that helps me focus by 20 times.

Unfortunately, the ASMR I’ve heard is just horribly unpleasant and makes me flinch. The right kind of ASMR, however, could probably do a lot.

2 Likes
13

Haha! That made me laugh out loud reading that! Brilliant. :laughing:

I know what you mean, they’re not really for me either, I’m much more a pink/blue/green noise kinda guy. But I know ASMR is very popular and it still lends itself to what I wanted to convey about the power of audio to connect directly with your mind… Even if the result is an angry person (in fact that further makes the point!)

Adam

14

@pinkunz @HanonO

Yes! It’s best when it’s subtle, almost lofi.

Adam

1 Like
15

When I was at university many years ago, I would listen to the Mystery Theater by E. G. Marshall radio broadcasts on the way home in the evenings. The background music and effects added immensely to the experience.

I would frequently sit in the driveway until the story finished so I wouldn’t miss the ending. :wink:

5 Likes
16

That’s sort of my dream as well. (I’m not sight-impaired, thankfully, but I always find it much easier to focus on an audiobook rather than reading myself.) Unfortunately, doing this would preclude most all dynamic text generation/variation unless you resort to using some kind of text-to-speech service/system for those parts.

(E.g., Inform’s stock response when another actor picks something up is "[The actor] [pick] up [the noun].", which can theoretically come out to number of takeable things x number of actors variations. That could be a lot of voice lines, never mind the technical aspect of figuring out which one to play for any given occurrence of this response.)

3 Likes
17

I’m sure it will arrive in the not too distant future, a plug-in for a voice from ElevenLabs would do the trick.

18

That’s the advantage of text - it can be manipulated in real-time like clay, whereas produced audio performance is expensive and not as easily varied. While video-games can be fully voice-acted, it’s usually on AAA games or small projects. Audio production has been reduced to something that can be done on one computer, and synthesized speech has gotten a whole lot better, but most of it is intended to tell you where to turn off the interstate rather than giving a dramatic reading or performance.

Netflix had planned to do a lot of “interactive TV shows” after Bandersnatch and even developed the technology to facilitate it, but it turns out they would have to produce the equivalent of 4-5 full-length movies or a miniseries to create enough content for a 30-45 minute session of an interactive show and they found it unfeasible to do regularly.

3 Likes
19

The best I’ve managed to do (and it’s still iffy in places) is this…

Incidentally Keith gave me his permission to use his voice, it was a bit of an experiment, we connected up a few years ago when he made a really good radio play of Photopia on his XO channel.

Adam

1 Like
20

I think any sort of interactive audio IF needs to be well planned. This video above kind of made me think you could do something interesting with a live-streaming scenario like this. (I see you dropping in the Portishead…)

The livestream could be produced like a haunted location exploration being done with simultaneous “group chat” that is mostly simulated. The player is a chat participant and types comments and suggestions in the chat, so mostly the audio could go along its pre-planned way, but have subtle branch points. Like if the streamer says “there’s a painting on the wall” the player might suggest “look at the painting” in the chat (perhaps the simulated other people in chat might echo in support or contradict that suggestion adding more or less “weight” to the choice - they are essentially the “parser” giving feedback that one of the keywords the player mentioned is important) so if that choice is “weighted” enough in the chat by other repeating and mentioning it, the streamer notices it and audio will take a side branch where the PC actually stops to look at the painting and describes it. If it’s too late (the streamer has moved out of that room) the suggestion will either be ignored (as happens in live chat) or maybe there could be an alternate refusal audio clip added in “Hey, I saw you guys asking about the painting. I’m not in that room anymore, but if I get back there I’ll try to take a look at it…”

So essentially instead of fully-interactive, it’d be mostly a forward-moving script with “lane changes” programmed in that the player could affect, but there’s a diegetic reason for the game to ignore the player’s commands if they don’t make sense, don’t fit the story, or don’t apply at that time. The player isn’t directly controlling the PC but has some agency in the story by making suggestions requesting detail which may or may not be answered. At some point the player might have to type DON’T OPEN THE DOOR! enough times and convince the chat to also spam it to keep the player from dying. Or if the player typed GET OUT AND CALL THE POLICE enough, the story might exit early at the player’s suggestion.

3 Likes