IF with 'soundtracks' feasible?

Authoring Inform 7
1

I’ve been thinking for some time how fun it would be write the score and create the foleys for a text adventure. Graphics can kill the imagination but, at least for me, sound is a lot more evocative and alive, and WAY simpler to create. Plus my imagination is mostly graphical, not sonic.
(I’m using IF to pre-design a graphical point-n-click but I may never make it past the IF stage :slight_smile:
I’m implying that the sound is only for mood, not actual activity that would fight with the text for attention. I imagine footsteps and creaking doors is the most you could get away with.

Anyway, while being unemployed with tons of time on my hands, I started learning I7 and discovered with excitement that the basis for audio is available. (glulx and Multiple Sounds by Massimo Stella, for now)

My fantasy game to score would have to be Anchorhead, although the source would have to be available and convertible to glulx. Authors could have an audio artist ‘score’ their game if they are not multimedia skilled like some DIY types.

Anyway, I keep reading about memory limitations and such, despite living in a tera-byte world!
Does this sound feasible? I already made a simple two-room example and even Stella’s (limited) extension demonstrates the concept well.
But a full-sized game would have a CD’s worth of audio files. That seems ok to me when compared to mainstream PC games but maybe there is a major stumbling block with interpreters or performance?
I would not hesitate to require a player to use a specific interpreter to enjoy the game with sound.
(even better: I originally envisioned something that would be literally driven by the text of the console ie an external app that monitors the console text and is programmed to make sounds based on interpreting that text. Room banners being an obvious starting point. Then anybody could add sounds to existing compiled games.)
I’m curious about whether this is an attractive idea to others (plus the issues mentioned above).
thanks,
Matt

2

Hi Matt,

  • On the author side, the tech is easy.

If using Inform, you can include sounds in lossy ogg format or lossless AIFF format. ogg will do in the vast majority of cases and keep your file size way down.

The latest Glk spec has built in support for audio crossfading etc., though it still may require a bit of code fiddling to set up more than 2 sound channels. But I helped with Multiple Sounds and contributed code to it which should help a person set extra channels up. Multiple Sounds was written for the previous version of Inform and doesn’t cover the new crossfading capabilities. I’m not sure if the newer extension Music does cover them, but I think it is probably more flexible in any case.

The only real size limit for the most part is on how big a file you think players of your game are prepared to download.

  • On the player side (or in response to ‘is the idea attractive?’) there are a fair few hurdles.

The first one is that it’s still safe to say that, at least for parser-based IF, the majority of players are pretty skeptical of audio in these games or anti-it. You’ve got some people who move to turn off any sound as soon as they see it’s there, so it’s not worth thinking about them. But in the area of people who would consider it, there remains understandable skepticism about it being too distracting. These games require thought and reading and concentration, and people are picky about what sound or music (if it’s non-discrete) they can or will have playing at the same time they’re trying to do those things, for a range of personal taste or neuroscience reasons.

The second hurdle is that if you make use of the latest audio spec stuff, people can’t (today) play your game on a Mac, because no Mac interpreter is up to date with it.

The third hurdle is that the most up to date Mac interpreter (Gargoyle) has a bug that prevents audio cueing working well in the previous spec, anyway.

So that’s a series of restrictive factors that could limit desire or the audience for such an audio focused game in Inform. That would also be why there isn’t one yet. That doesn’t mean it can’t or won’t be done if it is designed well or plays to its own strengths.

-Wade

3

IMO, even with the caveats severedhand lists, it’s always worth writing the game you want to write. When it comes to releasing it, get some beta testers on various platforms and see what works and what doesn’t. If you can fix what doesn’t, yay. If it is a problem with certain machines or interpreter bugs, well that’s too bad for people on those platforms, maybe they’ll pressure the interpreter writers to fix the bugs, maybe they just won’t play your game.

If you’ve got an idea, get it out into the world and let it fly as it may :slight_smile:

4

What’s the state of audio support in quixe, though? That would moot the Mac stuff.

5

An additional factor, as I learned the hard way when I released “The White Bull,” is that some people may just plain not like your music. One reviewer complained that he didn’t like electronic music. The game is set in Greece, and he would have preferred to hear, like, acoustic bouzouki playing or something.

At the time, I didn’t respond, but I was thinking, “Well, gee, guy – would you like to send me about $10,000 to hire some musicians, rehearse them, and book time in a recording studio to do these bits of music? Because if you’re not willing to do that, you can just go sit on a sharp stick.” I have some lovely computer-based synthesizers, so that’s what I used.

The take-away is, if your audio is not what people are expecting (based on their experience of film soundtracks or whatever), you may provoke a negative reaction.

6

It’s definitely do-able. My IFComp entry last year TRANSPARENT had a score and sound effects.

The things you must consider:

You must provide for interpreters that don’t handle sound. The “Glulx Entry Points” extension is good for this. The commands that produce sound can throw errors in certain interpreters and you need to intercept that.

You want to provide a way for the player to turn sound on and off, and if the game relies on information in the sound effects such as speech, you want your game to intelligently provide descriptions of sound when sound is off (or by default along with the sounds, as your preference.) It’s true that people could just mute or turn down the sound if they don’t want it, but it’s a politeness to allow for this.

Other considerations: Sound files will swell the size of your gblorb. Transparent is huge due to the number of long music files. If you can plan for short sounds or looping music you can keep the file size down, but looping a too-short music file runs the risk of becoming repetitive and encouraging the player to turn sound off.

7

For Mac, Garygoyle probably is the best option for sound.

8

There’s no sound in Quixe yet. Zarf just gave us graphics for the first time the other day, which is cool.

Probably? There’s no other option! And the sound notifications don’t work properly, and its Glk spec is out of date. I say this because I want to make sure the last word on a tech topic in this topic is the technically correct one.

-Wade

9

Spatterlight for Mac also supports sound.

10

If you make a game today, it will most likely fail to run, or your saves will randomly fail or break, on Spatterlight, because Spatterlight is multiple Glk specs behind, not just in sound. It’s a kaput project.

Spatterlight is very good for playing some older format games (EG Adrift on a Mac) but you should consider it dead for a game made now.

-Wade

11

Hey, these are all great replies and they sound mostly positive:
-Yes, files could be huge, but normal downloaded games easily hit 100s of megabytes; I don’t see much resistance unless this happens to be a community of people playing IF games on DOS machines with 2400 baud modems. Is there? I tend to keep up with PC tech in my home so the idea of holding back to reach a larger audience is a mixed one for me.

-I also agree that the conflict of audio vs sounds in one’s head is a critical one and here is my take:
…In the spirit of the sharing philosophy amongst IF fans, the code would be available and another enthusiast is welcome to replace the sounds with their own and release it as a variation. I also realize that this seriously breaks away from the conventional relationship between text-only and IF, which used to be called text adventures. Changing to ‘IF’ in some ways killed that requirement and opened the doors to many experiments within the broader concept. And WHAT other game-writer’s forum is going to support any kind of text-dominated interaction except for the IF community? IF remains very traditional but a backwater amongst the bigger gaming world. Right now there is no middle ground to play with hybrids. (I even thought about how I might implement a totally real-time action game with nothing but text I/O, not turned-based. Bizarre idea? Yes. Your typing skills better be good!)

…Allowing to turn off the sound is a given, although it kind of misses the point! But I understand. I tend to put on atmospheric music when I play because it is one of the few game-playing activities that gives me that option. (although I did not find any acceptible ‘spooky circus’ music to go along with playing Ballyhoo recently.)

…Music IS subject to taste but people who create are committing to artistic choices with every project; and the critics will supply a thumbs up or down. That being said…
…Music could be an aspiration but I was mostly thinking ambience, not music (think rain sounds in the opening scene of Anchorhead). Part of the reason I have been conceiving audio with IF is that IF is mainly expository and there is a limit to how much exposition a player will accept. A normal story has total freedom to shape the perception of the reader with conciseness compared to the IF medium.

…Some advantages to sound: As soon as you enter a room, audio-ambience will immediately push you into a passive mental state that requires no effort. I can overlook a change in the text but I am far more likely to notice the clock is not ticking anymore. If I add text ‘the clock is not ticking’ really gives the show away because its presence is itself a clue that it (may) be important. Again, the clicking clock doesn’t have to be important, just another element of the ambience.
…By its nature, sound (compared to the other senses) is highly tuned to interpreting the 3D world around us. It is intrinsically dynamic and reveals the space around us in 360 degrees all at once.
(tidbit of interesting info: I used to work on cochlear implants and the most interesting thing I learned is that deaf people complain of loneliness and isolation more that blind people, despite the huge difference in handicapping! Apparently visual is good for physical interaction with the world around us but sound conveys more mood and feeling and, very important, what is happening behind us.)

Despite all that, I sometimes prefer to read a book with no music playing to allow the books ‘voice’ to dominate; I completely understand some desires to skip past all sensory import. Hell, I might even discover that I hate my own idea once I actually try to play it. Plus, everybody has heard the saying ‘everybody is a writer’, which purports a creative palette accessible to everybody, whereas I am guessing that there is definitely a lower tolerance for bad sounds because they’re harder to tune out. The quality and choice of even background non-music audio like rainfall or wind blowing could be irritating if your particular creative choice does not interlace well with a player’s imagination.

Well, that was a lot of babble on my part but good convo, thanks! There are probably a lot of IF fans who are mentally screaming NO!, don’t mess with the format! But isn’t art ultimately about exploration and pushing boundaries? There is ALWAYS friction in the artistic community.

12

I totally agree, my two major works had extensive sound, and I had more than one tester who told me they were very affected by the ambience of TRANSPARENT and were on edge entering the house. But just keep in mind that your game needs to work with no audio as well. ANCHORHEAD has unsettled people with plain text for a decade.