glulx sound observations

I just thought I’d report a few of my observations on using glulx sound in a game (which I am), since it’s not a very charted area yet.

The basic features - easily start, stop and loop sounds - are pretty neat.

If you want to address sound in terms of real time (EG play this for two minutes, turn this off in 30 seconds), that is currently not easy. You have to start to program the glulx timer, or hack extensions that interact with it. Your best bet here is probably to wait for (or start to fiddle with the i6 version of) the fabled DaMusix extension, which has audio timers built-in.

There is currently one issue which cannot be overcome by any user programming, as it would have to be addressed at the glulx source level (by moving the volume only at zero crossings of sound waves?). When you stop a channel which is playing, you will usually get a click in the audio. You will also get clicks when you change a channel’s volume level.

This also means you can’t program a clean sound fade-in or out from within your game unless the gradient of the fade is very shallow, because every time the volume is adjusted you can get a click, resulting in a crackly overall fade. I fiddled around with this a lot yesterday, tweaking the delay between steps in volume while fading and the size of the steps. In the end I did come up with a decent approximation of an exponential curve type fade, where the crackling will be faint on most people’s average speakers, but you can’t really eliminate it.

This may sound picky, but producing fades on-the-fly is probably the most commonly required activity in any game delivering audio, so if glulx is to support audio, I would say that removing the crackles that occur on volume changes is the one fix required for the future. Adding built-in fade support would be killer, but they’re actually easy to program so I’m sure folks can do that themselves.

Apart from that, I think all the tools are there for you to do what you want with your sound.

I agree that smooth volume adjustments are a highly valuable addition to the spec. I grumbled about the absence of fade-ins and fade-outs in TADS 3, and eventually Mike added support for them.

The point is, in IF the author cannot know in advance how long the player will remain in a room. When the player discovers the trapdoor beneath the Queen’s Scented Bedchamber and descends to the Ghoul-Packed Crypt, the music bed will quite likely need to change. There’s just no way to manage this without a smooth cross-fade.

Noted for the Glk to-do list.

Naively, I would have expected the Glk volume control to be fine enough for volume changes to be possible without crackling. Is this with a specific interpreter, or with the interpreter in the Inform 7 front-end, or all interpreters you’ve tried?

I tried Gargoyle on Mac, Glulxe and Wingit on a PC, and they all behaved the same. The Mac Inform front-end doesn’t support sound, so if you write on a Mac, you always have to make a release of your game to check audio stuff.

Could you post an example Glulx file to test with?

Also, can you try the Damusix demo and see if it has the same issues with volume changes and fades?

dl.dropbox.com/u/2759298/damusix_demo.blb

I tried DaMusix. It does have the same (subtle) issues. Also it uses very gently sloped fades, which click less often or not at all.

I put together a sound demo with Inform 7. You can download it here:

aeriae.com/no/GlulxSoundDemogblorb.zip

It lets you jam the volume on and off, run multiple fades in a row, etc. In an ideal situation, none of these actions would ever cause a click, but as is, sometimes they will. The steeper a fade, the more clicky it can be.

The appearance of these clicks etc. is entirely dependant on the nature of the sound wave at the moment the volume is changed. For instance, if you ‘volume off’ at a particular moment in a piece of audio and no click happens, and you supernaturally had the ability to ‘volume off’ at that exact same moment in the audio when it came around again, it also would generate no click the second time. The same goes for a moment that would produce a click, etc.

I’ve pasted source code of the demo below. If you wish to compile it and play with it yourself, you will need to first have the following 2 extensions in your inform kit:

Multiple Sounds
inform7.com/extensions/Massimo%2 … Sounds.i7x

Real-Time Delays
inform7.com/extensions/Erik%20Te … Delays.i7x

Btw, Damusix uses the glulx timer in such a way that the user can keep performing actions while fading, etc. I used Erik Temple’s real-time delays to do the fades in my demo, but either way can generate artifacts as the timer itself has nothing to do with the issue.

Code below:[spoiler][code]“Glulx Sound Demo” by Wade Clarke

Include Basic Screen Effects by Emily Short.
Include Multiple Sounds by Massimo Stella.
Include Real-Time Delays by Erik Temple.

There is a room called Don’t Plank.

To decide whether glulx sound is supported:
(- ( glk_gestalt(gestalt_Sound, 0) ) -)

Include (-

[ VolumeRaw chan val;
if (glk_gestalt(gestalt_SoundVolume,0)) {
if ((val <= 81920) && (val >=0)) {
glk_schannel_set_volume(chan, val);
}
else {
“Invalid Volume Level: please insert a value between 0 and 81920.”;
}
}
else {
“Your interpreter doesn’t support volume control.”;
}
]; -).

To set the raw foreground volume to (VOL - a number):
(- VolumeRaw(gg_foregroundchan,{VOL}); -)

Sound of StartingMusic is the file “tench_seg.ogg”.

To play_start_music: if glulx sound is supported, play the sound of StartingMusic in foreground with loop.

The infinite_loop is a number that varies. The infinite_loop is 1.

when play begins:
say “[italic type]Glulx Sound Demo - by Wade Clarke[roman type][line break]”;
try requesting the story file version;
say “and Version 2 of Multiple Sounds by Massimo Stella[paragraph break]”;
say “The sample in this tech demo is a brief passage from PS1 game Tenchu.[line break]”;
if glulx sound is supported, rock out;
otherwise switch off.

To switch off:
say “This demo only works in an interpreter with sound. This demo has stopped running.”;
stop game abruptly.

To rock out:
while the infinite_loop is 1 begin;
say “[line break]Music will be play on the foreground audio channel. Hit the number keys to do things with it. Kick off by pressing 1 to start music. Fades cannot be interrupted by keypresses![paragraph break]”;
say “1 Set volume to max then play song from beginning. Song will loop.[line break]”;
say “2 Immediately stop song dead (volume dial is untouched).[line break]”;
say “3 Immediately drop volume to OFF (song keeps playing but you can’t hear it)[line break]”;
say “4 Immediately bump volume to MAX[line break]”;
say “5 Max Volume then Fade track out with a nice-ish fade curve in steps of 25 ms (song keeps playing after fade but you won’t hear it)[line break]”;
say “6 Max Volume then Fade track out with a numerical straight line fade in steps of 25 ms (song keeps playing after fade but you won’t hear it)[line break]”;
say “7 Max Volume then Fade out with minimum possible gradient but unlimited CPU speed (fade will be so fast it will be perceived as a cut - song keeps playing after fade but you won’t hear it)[line break]”;
say “8 Max Volume then Fade out with minimum possible gradient in steps of 1 ms. This is the shallowest possible fade you can program, so bring a good book, as it will take 82 seconds to fade the track out. (song keeps playing after fade but you won’t hear it)[line break]”;
say “9 Quit this demo[paragraph break]”;
let k be 0;
while k is 0 begin;
let k be the chosen letter;
if k is 49 begin; [1]
set the foreground volume to 5;
play_start_music;
otherwise if k is 50; [2]
stop the foreground sound;
otherwise if k is 51; [3]
set the foreground volume to 0;
otherwise if k is 52; [4]
set the foreground volume to 5;
otherwise if k is 53; [5]
say “[line break]Fading out with a curve…[line break]”;
let CV be 81920;
repeat with N running from 1 to 1000 begin;
if CV is less than 300, break;
now CV is CV minus 64;
if CV is greater than 2561, now CV is CV minus 128;
if CV is greater than 5121, now CV is CV minus 256;
if CV is greater than 10241, now CV is CV minus 512;
if CV is greater than 20481, now CV is CV minus 1024;
if CV is greater than 40961, now CV is CV minus 2048;
set the raw foreground volume to CV;
wait 25 milliseconds before continuing, strictly;
end repeat;
set the foreground volume to 0;
say “Fade complete.”;
otherwise if k is 54; [6]
say “[line break]Fading out in a numerical a straight line…[line break]”;
let CV be 81920;
repeat with N running from 1 to 1000 begin;
if CV is less than 2050, break;
now CV is CV minus 2048;
set the raw foreground volume to CV;
wait 25 milliseconds before continuing, strictly;
end repeat;
set the foreground volume to 0;
say “Fade complete.”;
otherwise if k is 55; [7]
say “[line break]Fading out at max CPU speed…[line break]”;
let CV be 81920;
repeat with N running from 1 to 82000 begin;
if CV is less than 2, break;
now CV is CV minus 1;
set the raw foreground volume to CV;
end repeat;
set the foreground volume to 0;
say “Fade complete.”;
otherwise if k is 56; [8]
say “[line break]Fading out with minimum fade gradient…[line break]”;
let CV be 81920;
repeat with N running from 1 to 82000 begin;
if CV is less than 2, break;
now CV is CV minus 1;
set the raw foreground volume to CV;
wait 1 millisecond before continuing, strictly;
end repeat;
set the foreground volume to 0;
say “Fade complete.”;
otherwise if k is 57; [9]
say “You’ve quit, so now you can do whatever you want. / end of line”;
stop game abruptly;
if k is less than 49 or k is greater than 57, say “Press a number between 1 and 7, please.[line break]”;
end if;
end while;
end while.[/code][/spoiler]

Yes and no. It also depends on how often the playback engine updates the audio level. This is a multiplication. That is, at full volume we can assume the audio words are being multiplied by 1. As the volume drops, the audio stream (which is constantly changing) is multiplied by, perhaps, 0.999, 0.998, and so on. If this happens rapidly enough, the individual changes won’t be perceived. You won’t hear clicking. But if you want a fade to silence in 1 second, and the volume level is only being updated 100 times per second, each stair-step in the descending volume ramp will be rather large, resulting in clicks.

Pro audio apps perform a smoothing function during level changes, in order to prevent clicking. I don’t know how this is done, exactly, but it might work like this: If you’re only going to update the amplitude level 100 times per second, you grab 441 samples at a time (assuming the engine is playing audio at 44.1kHz) and perform a vector operation on them. That is, you don’t multiply all 441 samples by 0.99 and then spit them out. You interpolate smoothly between 1.00 and 0.99 over the range of 441 samples, and then send the resulting data block to the output buffer.

It’s doubtful that IF interpreters can do this type of operation on their own; they’re dependent on whatever audio software the OS provides. I have no idea how that works – it almost certainly depends on what hooks the OS makes available for sound manipulation.

Thanks for this. I will do some experiments with Windows Glk. As Jim notes, though, this sort of thing will always be dependent on what sort of support the interpreter gets from the operating system or framework it’s based on.

I’ve spent a bit of time playing with the example, but I can’t hear any cracking on my system. Possibly I’m insufficiently aurally sensitive, but most likely it’s dependent on the sound card hardware in your system.

I think that the only way to do a completely smooth fade is to as Jim notes, and scale down the waveform data as the sound data is sent to the operating system. Doing that really will need a new Glk call - it’s something different from changing the volume, more along the lines of “Fade this sound out over 3 seconds: I don’t mind if it takes 1/2 a second or so longer than that in order to keep it smooth”.

I don’t hear the clicks that severedhand mentioned. And I do have a really good hearing.

I don’t hear clicks in Damusix Demo (otherwise I would have corrected it) or in the test published by severedhand.

I think the problem has to do with a hardware issue of severedhand’s machine. Or maybe the problems have to do with the way he has programmed the fade effects.

The sound fades gave me a lot of work to make them function properly in all configurations.

As a rule of thumb (to avoid glk timer related clicks), never use a candence under 100ms. 100ms work correctly, at the same speed, in almost all terps.

Under 100ms, the results can begin to be random.

Damusix uses a dirty trick to make fades of less than 100ms of length. The trick has the side effect that fade may take longer than indicated (on some terps), but with the benefit that the effect occurs smoothly.

This, of course, only happens with fades of less than 100ms of length, which are exceptional cases. For fades with more than 100ms of length, Damusix uses a time based linear fade with proportional volume units increments/decrements, according to an algorithm that I wrote (do not ask how, because I’ve never been good at math and I’m not sure how I got that equation) xD

Saludos!

P.S: This message is google-translated. Thanks to anonymous friend for translation help, too.

Thanks for the info on how you did the fades Eliuk. Yes, I’m aware clicks are related to how one programs the fades.

If any of the clicking is hardware related, all Macintosh users should be able to hear it because we all have what-PC-users-would-say-is ‘the same sound card’ - which is CoreAudio.

I think the easiest place to hear clicks in my demo is by playing the song, waiting for 3 seconds (past the yelly vocal intro), then doing what I call the linear fade (option 6) - as it races through the lower half of the volume range, that’s where I often hear a few clicks, like a tiny stuttering machine gun. When it gets to the groove part of the music later, I find it almost impossible to get many clicks. The sound wave in that area is too consistent. But what I called the ‘linear fade’ in my demo is the most aggro fade which, if you do it enough and at different times, you should be able to get some clicks.

I would like to be able to share the piece of music for my game, which by its nature produces the clicks extremely easily when I fade over it, and thus caused me to look into this whole thing, but I don’t want to reveal any content from my game before I release it :slight_smile:

But here’s what I’m going to attempt to do - record my computer’s audio output while it’s fading and making what I consider to be clicks. If I can then turn this into an mp3 and I can still hear them, then I can hand it to you guys and you can listen and see if you can hear it. If that works we can agree it is not a hearing issue.

Okay, here’s an mp3 capturing the sound output of my Mac.

aeriae.com/no/edit_together.zip

First, the DaMusix demo. I use the ‘simple fade’ command, which fades in and out a few times in a row. This happens between 1 and 4 seconds.

At 8 seconds, we switch to my glulx sound demo. Here, I repeatedly restore the volume to max and perform the linear fade.

Now I should point out, these recordings are only to demonstrate that clicks can happen, not that they always do. I only included the clickingest takes. I ran both demos for a couple of minutes to capture a bunch of clicks I expect others will be able to hear. I should also point out I’m in a music studio with monitors pointed at my head and I have a big pair of headphones, and like Eliuk, good ears :slight_smile: But I’m hoping someone will say ‘oh yeah, I can hear that’. It is subtle, but it is there.

I tried the demo on my MacBook but I can’t hear the clicks. There might be some slight stuttering audible in Gargoyle with option 6, but it’s so slight that you have to strain to hear it, and even then I’m not sure if it’s just part of the song (might not be the best choice for the demo). On Spatterlight the fades take much longer but I can’t hear any anomalies. Certainly nothing like in that mp3.

I’m also on a MacBook and don’t hear those crackles and pops using Gargoyle (or Spatterlight).

–Erik