Audio issues/bugs with mobile devices - Sugarcube

Sugarcube 2.31

Hi all, I’m just testing the project across different devices and noticing that there’s an inconsistency with audio commands. Running off of the webpage where we’ve uploaded the project, something that plays normally on the desktop won’t play on my mobile phone. And this includes even very basic audio commands, eg <<audio "track" play>>

Any ideas about why this might be happening and any steps I can take to mitigate against it?

Thanks! Ben

See the audio limitations to see if any apply. My guess is #3.

Got it. So, if I understand correctly, starting playback should always be initiated by some kind of touch (ie I should embed playback of new tracks with links)…?

At the moment, the tracks that aren’t working on mobile are all in passages that follow on from a <<goto>> command. So this makes sense. I’ll take a look now.

But something like the following might also not work <<timed 3s>><<audio "track1" play>><</timed>>? (Note: it’s working on my desktop browser at the moment)

Many thanks, Ben

No, the first time the page is interacted with, audio autoplay should be “unlocked.” so only sound you’re trying to play before user interaction needs changed.

Thank you so much for helping with this. It’s amazingly helpful. But I’m not sure I follow… What do you mean by “unlocked”? And would this apply is per passage?

I’ll follow up with some examples in a moment.

Thanks! Ben

For example, at the moment this passage works on a browser, but not on a phone:
<<link[[Now — let's begin|Trans out of Prologue]]>><<run setTimeout(function () { $.wiki('<<audio "prologue" play>>'); }, 16000)>><</link>>

Only this works on a phone:
<<link[[Now — let's begin|Trans out of Prologue]]>><<audio "prologue" play>><</link>>

In this example, I thought I might be able to delay the track playing, but still “unlock it,” by adding the command into a link. (Rather than simply on page load). But that doesn’t seems to work either.

Okay, I’m trying different options, but I still can’t find something that works. I’m not sure why using the setTimeout function in conjunction with a link won’t work on mobile devices. I even just tried triggering the track at Volume 0, and then again with a delay at 100%, so that the track was “unlocked”—but that doesn’t work on mobile (everything I’m trying works on desktop, but not on mobile).

<>“link text”>><<audio “berlin” play volume 0>><<run setTimeout(function () { $.wiki(’<<audio “berlin” time 0 volume 1 play>>’); }, 7*1000)>><>

He just means that audio won’t play until the user has interacted with the page at least once first. And no, it’s not per passage, since it’s all the same page.

As for the problems you’re having playing audio on a mobile device, first note that the volume is controlled by the mobile device, not your code. Second, you should try seeing if you can play any audio in your browser, since the audio could be muted by your phone and/or the browser on your phone. (E.g. try playing any of the “Featured Tracks” at the FreePD site. Make sure you wait for the audio to load and start playing, but if you see the timer going up with no audio, then you need to adjust something in your browser or your phone.)

Let us know how that goes.

Hey, thanks so much for your help. The problem is individual tracks not playing after a “pre-wait”—even if the pre-wait is triggered by a link—on mobile devices.

The project I’m working on has a lot of audio. I’ve loaded it all on to my website, so I’m now testing everything online. On a desktop, everything plays. On a mobile, it’s much more hit and miss. Some tracks play, some don’t. What seems to be the underlying problem is tracks that have a “pre-wait” on them before playing. When I take the pre-wait off, and trigger them immediately with a link, they play fine on a mobile.

However, this solution takes away from the magic of the experience if the user is constantly triggering the sound. What works really beautifully on the desktop is that the sound works in conjunction with a reading speed variable, so sound happens in conjunction with certain words, based on an individual user. This is what I’m trying to transfer over to the mobile devices, as much as possible.

I should add that I’ve experimented with using both <<timed>> and setTimeout as ways of setting the pre-wait time.

Hi all! I tested the Free PD tracks on my phone just in case, and my phone is working fine. It’s not a question of volume, but rather something in the audio commands getting a bit tangled up on mobile phone browsers. As I mentioned, this seems mostly connected to “pre-wait” commands, where I want Twine to wait a certain amount of time before playing a track. This happens both when a passage loads, and if it’s part of a link’s execution. And it happens both with the timed macro and the setTimeout function.

The project I’m working on is online here. We’re still building it, so there’s functionality missing, it’s rough around the edges, sound hasn’t been finalised, etc, but it will give you an idea of what we’re trying to achieve, and how that breaks down when you run it on a mobile device.

Thanks again, Ben

Instead of using a timer to delay the playing of the audible part of a specific track have you considered adding a set period of “silent” to the beginning of that track. This would allow you to start playing that track as soon as the contents of the associated Passage is shown.

eg. if you want to delay the audible part of a track by 5 seconds then add 5 seconds of “silence” to the start of that track.

see: How to use Audacity to add silence to an audio file

Hi, Yes. That’s our contingency plan. The ideal, however, is that we want to trigger audio timings using the project’s in-built reading speed variable. (The subject of my first post on this forum).

Essentially, the story learns how fast you read at on-board, and then reveals the text in sentences and paragraphs at that speed (calculated as “words per second”). Part of the idea of the variable was to use it as a way of triggering sound cues, whose timing would be unique to each reader’s reading speed. (This approach is working out really well on desktops, but not so well on mobile devices).

We could definitely create an average estimate of “pre-wait silence,” but the ideal would be to incorporate the more personalised audio timings based on the “words per second” variable.

What kind of phone (i.e. Android or iPhone)? What browsers have you tried on the phone?

Try the page in a different browser, just to make sure the problem isn’t browser-specific.

Let me know, and if I can I’ll look into it.

Hey, thank you so much. I’d already worked out that Safari doesn’t support our OGG sound files.

And thanks to your suggestion, I’ve worked out the exact problem! The issue is the browser that Chrome automatically opens when you follow a link on an Android phone. It’s not the Chrome app itself, but a Chrome window (actually more of a solo-tab), which you can choose to then open in the App, or not. If you play in the Chrome “tab/window,”—problems. If you open in the app—no problems. I worked this out when I checked it in Firefox on my phone… and it worked fine! Because I copied the actual web address into the browser. So I tried the same thing in Chrome, and it also worked.

So the issue is Chrome’s automatic browser window. Any idea why that’s the culprit?

Many thanks, Ben

Intriguingly, I’ve just installed Chrome and Firefox on an iPad and no sound plays at all. Could be the OGG files don’t agree with mobile Apple devices…

Chrome on iOS is just Safari. You should be using mp3 files. If you want to use ogg files as well you can use both.

Based on what you’ve described about its behavior, it sounds like it uses a different security model. Perhaps something similar to the security model most browsers use for fullscreen mode (via. the .requestFullscreen() method), where it only works within a short timespan of a user interaction.

Also, instead of adding silence to the start of the audio file, you might want to make a 1 second “silent” audio file, and then use <<createplaylist>> and <<playlist>> to loop that a few times before playing the actual song. Hopefully that works.

Oh, and if you want to see what audio formats are compatible with which browsers, the “Can I use ___?” site is pretty handy. For example OGG vs MP3 shows that MP3 currently has far better browser support.

Hope that helps! :grinning: