Unofficial Spatterlight 0.5.5b

(Petter Sjölund) #1

A new version of Spatterlight is up at
github.com/angstsmurf/spatterlight/releases/

It runs on OS X 10.7 Lion-macOS 10.14 Mojave. All the interpreter cores are updated to Gargoyle’s versions. There is some new sorting and contextual menu functionality in the table view. I’ve tried to implement the full Glk API.

Please try it out!

EDIT: Spatterlight is the Mac-only precursor to Gargoyle by Tor Andersson. If you’re on a Mac, it has some advantages over Gargoyle, such as system native text handling, a GUI preference panel and a game library window. Please report any bugs or feature requests in this thread or on Github!

EDIT 2: 0.5.6b, which fixes scrolling, is up.

EDIT 3: And 0.5.7b, which fixes some non-displaying graphics.

EDIT 4: 0.5.8b is up, which lets you save wherever you want and does not crash Image Test.

Unofficial Spatterlight 0.5.9b released!
(Andrew Plotkin) #2

Nice! I’ll take a look when I have time.

#3

Unfortunately, by using the Gargoyle versions of the interpreters, that means this version won’t get along with the VoiceOver screen reader. I wish there was a way to make Gargoyle accessible, bbbut the Garglk library uses unorthodox printing routines, so it’s basically impossible.

(Andrew Plotkin) #4

This doesn’t use Garglk.

I did a quick test with MacOS Voiceover. It reads the opening text correctly, but then command responses aren’t read out. So it’s not fully functional; but it’s using the standard display API enough for at least some functionality, and this can probably be improved.

(Dan Fabulich) #5

The problem is that VoiceOver thinks that the Spatterlight transcript is just one big giant text area, so VoiceOver doesn’t know that it should focus on and read new text in the transcript. (In HTML terms, VoiceOver doesn’t know that the Spatterlight text window is an ARIA live region.) So VoiceOver would just keep reading and re-reading the entire transcript, with no way to focus on or skip past individual paragraphs.

Lectrote is only slightly better in this sense: it uses an HTML renderer, so it can tell VoiceOver about individual paragraphs. When you type a command, you remain focused on the command area, and you have to navigate backward through the text to find the new text and read that. But at least on Lectrote it is possible to navigate paragraph by paragraph. (Lectrote does mark the transcript window as a live region, but macOS VoiceOver doesn’t appear to notice.)

(Petter Sjölund) #6

That doesn’t sound like something that would be impossible to fix. Are there any good examples out there of Voiceover-friendly apps that work the way you’d like, preferably with source code available?

#7

I don’t think source code is available, but take a look at the Zoom interpreter by Andrew Hunter. Aside from command-line version of frotz, it’s the best Z-Machine interpreter around.
And as for gargoyle, is it possible to create a version of Gargoyle using the standard display API? Because, y’know, Windows and Linux need love too. :smiley:

(Andrew Plotkin) #8

When we say “standard display API” in this context, we’re talking about the MacOS Cocoa toolkit. Anything that uses it is, by definition, a Mac-only app.

Gargoyle achieves cross-platformness by hand-rolling a text display system. This has no connection to OS-native screen readers.

Lectote achieves cross-platformness by using HTML and a Chrome browser as a text display system. Chrome supports the HTML standard for screen readers, but this is hard to tune for the parser IF layout. (Which is something of an abuse of HTML to begin with.)

Apple has extensive documentation. I’ve never looked at the MacOS version of VoiceOver, but the iOS version was fairly straightforward.

(Petter Sjölund) #9

Just to be clear, is there any problem in particular with the current text-to-speech functionality in Gargoyle? It seems to work fine on my system in the Mac version I put up here: github.com/angstsmurf/garglk/releases. It has to be turned on in the preferences file, of course.

(bg) #10

With the old version of Spatterlight, it was reported (in the course of testing another extension) that, when running a project that used Flexible Windows, Spatterlight would become unresponsive. It may be worth testing to see if that is still the case with the updated Spatterlight. (I don’t have a machine that can run Spatterlight, and haven’t tested it myself.)

Sample code:

[code]Include Flexible Windows by Jon Ingold.

Red Window is a g-window. The main window spawns Red Window.
The background color of Red Window is “#FF0000”.

Instead of jumping:
open Red Window.

Place is a room.[/code]

With the above code, this sequence of events would crash the old Spatterlight (in a testing release):
RULES. ACTIONS. Resize the window. LOOK. (Becomes unresponsive).

There’s a screenshot here: github.com/i7/extensions/issues/31

I believe this happened when using the version of Flexible Windows that is on Github–whichever version was current in 2015, when this issue was reported. github.com/i7/extensions/blob/m … indows.i7x

(Dan Fabulich) #11

I had no idea that Gargoyle had a TTS feature. It’s not a traditional approach, but I do agree that it works better than macOS VoiceOver on any of the interpreters I tried.

The first problem I noticed was that if the first page of a game overflows the page, it will have a “-- more --” page break on screen, but the TTS engine will just keep reading on past that point. As a result, when it comes time for the player to type, input is ignored until you use the space bar to scroll down to the end. This would be very confusing to users who can’t read the screen.

The next problem is that there’s no way to go back and re-read the text. VoiceOver supports features allowing you to skip around the text, paragraph by paragraph, sentence by sentence, word by word. TTS engines often mispronounce words, and it helps to navigate to that word and have the engine spell the word aloud for you. None of that can work here, because the app is reading the text itself.

The next problem is that there’s no preference to control the word rate of the spoken text. Many (most??) VI users configure the engine to talk much faster than the normal, default speaking rate. With practice listening to a particular engine, you can get good at understanding the engine at breathtakingly fast speeds, so speed control is important. (And if you miss a bit, you’ll want to be able to rewind and replay.) Again, this all comes built-in with macOS VoiceOver.

(Petter Sjölund) #12

At least on Mac, the text-to-speech in Gargoyle will honor the speed setting of speech in the system preferences. The other things you mention will be harder to fix.

(Petter Sjölund) #13

The Flexible Windows issue seems to be gone with this version of Spatterlight. I can’t reproduce it anyway.

#14

I wonder if all that would need to be done to improve this would be to stop GlkOte from manually focusing the input element (with an option, the current code is probably best for non screenreader users.)

GlkOte issue for further discussion on this.

(Dan Fabulich) #15

Interesting. I think that would be worth a comment in the garglk.ini file. (Note that it does not honor the VoiceOver speed setting; it honors the “Speech” speed setting, which is separate, and is capped at a pretty slow speed relative to what advanced VI users would prefer.)

(Petter Sjölund) #16

I don’t know if I’m missing something, but when testing Zoom 1.1.5 on OS X 10.7 with VoiceOver, it seems to have the same issues as Spatterlight: when the game window becomes first responder all text in it will be read all the way from the start, including any scrolled off-screen. It will also speak any entered commands, but none of the new text printed after pressing enter. At least not until navigating away from the window and then back to it again, when it will once again read the text all the way from the start.

However, Zoom has custom commands such as “Speak previous move” to compensate for this, but it still seems like a bit of a kludge.

Well, Apple has plenty of documentation for its latest APIs, but if you want to write something that runs on 10.7 you are out of luck. There was an all-new accessibility API introduced in 10.10, and Apple seems to have been busy erasing any traces of the old way of doing things. Even the in-app help of Xcode 4, which purports to be the “OS X 10.8 Doc Set,” will only show the latest online docs. But I’ll keep googling I guess.

(Dan Fabulich) #17

Hey, uh, the scrollbar isn’t working in Spatterlight 0.5.5b. I can’t scroll back with trackpad, and I can’t scroll forward either (but luckily spacebar does work).

(Dan Fabulich) #18

FWIW, I have reason to think that there may not be any good online examples of making an IF-like application work with macOS VoiceOver. I ranted about this last year; I’ll copy and paste the rant here for your convenience.

The traditional IF UI is a commonplace stdio application, but stdio is totally unheard of in modern UI development.

[rant]Consider this bog standard IF transcript:

The feeling of play is conversational. You might imagine that the game is playing the role of a Dungeon Master in Dungeons and Dragons, where the player and game are taking turns speaking.

If I were coding a “conversational” game today, I’d probably take examples from chat-based apps like Slack, where there’s a fixed input box at the bottom of the screen, and a transcript of messages in the main part of the window.

But that is not at all how IF games work (not how they have ever worked historically). Instead of having a separate “input box” into which you type commands, there’s just one main window area, the “transcript.” The game prints messages into the transcript, then prints a prompt symbol (the “>” symbol) and lets the user type directly into the transcript.

To extend my Dungeons and Dragons metaphor, you might imagine the Dungeon Master writing something on a piece of paper, then handing that same piece of paper over to the player, who would write something at the bottom and then hand the paper back to the Dungeon Master, taking turns writing on the page.

This can be confusing to screen readers. One single rectangle is acting as both output and input. Inform/Quixe has handled this by marking the entire transcript area with “aria-live=polite,” meaning that any new text that appears in the window is automatically read aloud.

But the bigger problem has to do with the “focus” of the screen reader.

When it’s time for the user to type, the last line is an invisible input box, where the user types. When the user hits enter, the input box is removed and replaced with text that the user typed. Then the game says some response message, and finally a new invisible input box appears at the bottom of the screen.

Removing the currently focused UI element is very bad for screen readers. The screen reader software tries to guess where to focus next. It might try to focus the user on the top of the document (the very first line at the very beginning of the game), or it might focus on the top of the screen (probably a random moment a few turns back).

We can try to focus on the new input box, but focusing that box may interrupt the live ARIA reading of the game’s message.

An obvious thing to do here would be to abandon the idea that the user should type directly into the transcript, and instead offer the user a fixed box to type in, at the bottom of the screen. That would probably work, but it would break some existing games.

For example, a number of games like to provide the user with a custom prompt inline with the text. "Please enter the secret code: " That may not make sense when the input box appears at the bottom of the screen.

A few games even rely on the fact that the prompt is in the transcript to play games with the player. For example, “Taco Fiction” prints a fake prompt “>” into the transcript and then waits for the player to type any key. No matter what the user types, the game inserts a character of the author’s choosing, “forcing” players to type what the author wants them to type, letter by letter. I guess it’s more of a practical joke than a serious UI consideration, but needless to say that joke would not work with a fully separate input box.

(And what about menus?)[/rant]

FWIW, I just tried VoiceOver with macOS Terminal, and that works fine, but the source isn’t available, and I think it may have special-case treatment from Apple. I tried it with iTerm 2 (for which source is available) and, surprise surprise, it had the same kinds of bugs as other IF applications.

(Petter Sjölund) #19

Heh. Now when did this happen? Time for 0.5.6b, I guess.

(Dan Fabulich) #20

Better!