Screen reader not reading long responses in Parchment

Technical Development Interpreters
, ,
1

I recently wrote to Dannii Willis, concerning the fact that Parchment doesn’t contain the equivalent of a “More” Prompt, and when there are quite lengthy text descriptions, the Screen Readers I’ve tried “JAWS”, “NVDA”, and “Microsoft Narrator”, simply stop speaking, and Vital Information is lost.

The gracious reply I received, informed me that attempting to insert a “More” Prompt into the already existing code, would introduce other sorts of problems which have the potential of making the whole thing unusable.

Has anyone who uses a Screen Reader along with Parchment encountered this problem, and if so, did you find a Work-around?

3 Likes
2

So I tested this with NVDA 2022.1 and Windows 11 Narrator (I currently don’t have access to JAWS on this machine). As far as Narrator goes, it didn’t want to read anything at all in the Parchment window, no matter where I placed the focus. I had much more success with NVDA, at least when playing on IFDB. I admit that I didn’t run a particularly thorough test, but I was able to read the “Commands” and “Credits” sections of the Curses help menu without any of the errors you described. I tried it in Google Chrome, Brave, and Firefox. Things were a little glitchier for NVDA in Firefox, but it worked okay.

I ran into an issue when I tried to force the problem to happen. I wrote an Inform program to print approximately 10 screens of text, but NVDA didn’t respond the same way to my offline Parchment as it did on IFDB (perhaps they’ve modified things a little over there? I got no output from NVDA after entering any command, except for the name of the webpage. Same with Narrator.

I’ll try all of this on the other Windows machine tomorrow. If anyone can recommend a game with huge walls of text, I’ll try it with that.

3

(I split this off into a new topic just to make it a bit easier to find.)

So my understanding is that the issue Daniel is reporting is that any long response in Parchment is not being read out in full by screen readers.

You can make a very long response in any Inform story simply by doing multiple commands at once, separated by periods. l. l. l. l. l. l. l. l. l. l will look 10 times, which is probably more than once screen’s height of text. Doing something like that will make this much easier to test, as you can do it at the beginning of almost any storyfile.

I’ve never had much success trying to test Parchment with screen readers myself, so I’m hoping some of the other screen reader using members of the community might be able to share their experiences with Parchment, and whether this is a problem with all screen readers, or only some of them.

1 Like
4

My issue with a Screen Reader and Parchment
working together doesn’t have anything to do with
just a Long Response, rather it’s a problem which
occurs, when a particular Text Description
displayed by a game, is too long to fit on a single screen.

I tried this with the latest version of “NVDA”,
and “Google Chrome”, and it still happens. Try
playing Trinity, an Infocom Game, and continue
playing, until you reach the part where you
enter the White Door, which is seemingly
suspended over the Long Water. The Screen Reader
ceases speaking after typing the Command to
Enter The Door, and though one can Manually
Scroll through the Text which is displayed on the
Screen, there is a lot which has already passed
by, and that doesn’t get announced by any Screen Reader I’ve tested so far.

I can send out specific instructions as to how to
go about entering the White Door, but that would
make this Message rather lengthy, and might not
be of particular interest, to someone reading this in the future.

Thanks For Any Assistance,

Daniel

5

It’s been a minute since I played that, I’ll give it a try soon

Just so I understand, are you only having this problem in this one game? I interpreted your problem as a general one to do with multiple screens of text, but it might be something that Trinity in particular is doing?

6

https://intfiction.org/u/blueaskewPeregryn BlueAskew,

Thank you so very much for writing, and taking an interest in my problem.

To answer your question, this also happened when
I was playing Zork I, but it took some time to
crop up,which is why I didn’t go into details.

I wanted those who were willing to try this out
to encounter the problem as rapidly as I first
did. In Zork I, it also occurs, when the list of
your Treasure Collection is long enough that
other Z Code Interpreters feel it necessary to
engage a “More” prompt, in order that the in tire
List can be viewed by the Player, even though it
takes up more than one Screen, some 200 Points or so, into the game.

I Hope That This Is Helpful, I’m Sure That Other Examples Exist,

Daniel

7

Hmm, that’s odd, I’m having trouble replicating this. For multiple screens of text, NVDA lags a little, but it eventually does speak all of the text in the right order. I’ve definitely seen this kind of issue before, though, usually in apps like frotz and glulx, where the screenreader sometimes struggles to queue up speech events for text which isn’t displaying yet. A question for anyone with the all-seeing orbs, how does text appear on-screen in Parchment? Does it all come in at once, or does it smoothly scroll? Based on the way my screenreaders behave, I’d expect that it just all displays at once, but Daniel’s results give me the opposite impression. In the mean time, Daniel, you could try maximizing your browser window, or put it in full screen with F11. Not fully sure that’ll help, but it seems like it might be a focus issue, and those can sometimes be fixed by filling the entire screen with the application you want focused.

8

Visually all the text appears at once in an instant, but it does get appended one line at a time. Could this be causing a problem, that the screen reader starts reading the first new line and then 1 millisecond later gets confused by lines that come afterwards? And then once all the text has been added it scrolls so that the top of the update is at the top of the screen (if more than one screen’s length), and it possibly also focuses the text input box.

I can try to change it so that it tries to add all the new text in one operation. I’ve also just discovered there’s an aria-busy option. Maybe one of those would help with this issue. I’ll give it a try later this week.

9

Ah, I think the scrolling of the update is the problem. The screen reader starts buffering the new text, and then it moves, so now the software’s reference is off. A lot of the time this just produces garbled output, as it essentially tries to read multiple screens aloud at once. Screenreaders are supposed to be smarter about the 1 line at a time deal, but live regions can be strange beasts indeed. I think aria-busy will solve this problem. As long as all the text is still on the page, the screenreader can do its own scrolling internally to retrieve the entire update to the region.
Thanks for looking into this, Dannii.

1 Like
10

Okay, I’ve updated it to now use aria-busy, so hopefully all the updates will be caught together. Let me know if this does indeed help at all.

2 Likes
11

Since I couldn’t replicate the issue, I don’t have much constructive to say on it, except many thanks for your effort! In my experience of web development and accessibility testing, it’s a rare day when a dev confidently checks the ARIA docs without at least having ARIA explained to them first, let alone implements a solution from them as quickly as you have. Not to knock other devs (not too hard anyway), but it’s hard to show my surprise and gratitude without seeming a little… something. At any rate, your inclusiveness and competence is very appreciated. :smiley:

12

Haha, I wouldn’t say I’m confident about any of this! ARIA is still a confusing other world to me.

2 Likes
13

Ah, And We Progress, too.

I’m not sure how to explain in a technical way
what’s happening here, so I’ll just do my best,
and people can write back with questions, if I’m not doing too good a job.

First, the Progress. I played Zork I, putting
everything I could into the Trophy Case, until it contained some 26 items.

Before this latest Parchment Fix, JAWS would
begin listing the Case’s Items, but somewhere
near the bottom of the list, it would announce
“Forms Mode Off”, and cease speaking.

“Forms Mode” in JAWS is that which lets me know
if I am viewing a Web Page and if I am able to
scroll through it using the Arrow Keys, or if I
am in an Edit Field, List Box, or Combo Box, and
should type or select a given item being displayed.

Parchment is very easy to interact with, once a
game begins, because the Cursor remains in an
Edit Field, commands are typed into that Field,
and the responses are read allowed by the Screen Reader.

Once the Zork I Trophy Case contains so many
Items, for some reason, the Player gets bounced
out of the Edit Field. The reason I never
noticed this before, is that it will sometimes
occur, while browsing the Internet. You’re asked
to type in a User-name for instance, and the next
thing you know, after typing a few letters, the
“Forms Mode Off” announcement is made, you have
to find the Edit Field on the Page, and begin typing in the information again.

In the Old Version of Parchment, JAWS would just
say, “Forms Mode Off”, and it was done. With the
Fix in, now all the items are read, but the
Player still gets knocked out of the Edit
Field. This isn’t just true for JAWS, but for
NVDA as well. What’s more, I was able to
reproduce this, by finding the Edit Field, and
typing LOOK IN CASE. The same thing would happen again.

Trinity is still misbehaving, but let’s take one thing at a time.

Thanks For The Fix, It Works Better Than It Did Before,

Daniel

1 Like
14

So I think I can explain what’s going on there. When the update is small the input box is automatically focused for the player to type the next command. But when the update is longer than a screen the input is unfocused so the arrow keys will work to scroll the window rather than scrolling through the input history.

In a regular browser if you start typing and the input box is on screen but not focused then it will automatically focus the input box. Does that still work if JAWS or NVDA is enabled? Maybe there’s a way to get it to not report when the forms mode is changing? Or would that cause more issues than it solves?
Edit: Maybe there’s an ARIA role I can give the window so that when it unfocuses the input box it says something more informative than just “Forms mode off”.

I wonder if zooming out might help? If you’re zoomed out a lot then Parchment would effectively think that you have an extra tall screen. It shouldn’t really affect how it operates other than to let you have much more text on screen each update without worrying about it being taller than one screen’s worth.

15

Ah, So It’s Not A Bug, It’s A Feature!

Dannii,

The way you’ve got it working now, the “Forms
Mode” announcement is made first, and then the
Treasures are listed. It’s a surprise for
someone who has been playing for a long time with
no problems to suddenly have the input box go
unfocused, but you just find the input box by
typing the Letter “e”, press the Enter Key, and you’re back in the game again.

I’m still not sure where Trinity is going
wrong. The problem occurs when you Enter the
White Door. Part One of the game is completed
and that’s where the “More” prompt gets issued
in most Interpreters, the screen is cleared of
the Text which ends Part One, and then the Text
which begins Part Two is displayed.

The Screen Reader doesn’t announce any of this,
it just goes completely silent. You have to play
the game using a different Interpreter, in order to find out what’s going on.

That’s why playing a game like this one is so
frustrating, because there’s no way to scroll
back and listen to the Text which ends Part
One. It’s just gone. I wonder how many
other games have this sort of feature?

Anyway, Thanks For The Explanation, That’s One Mystery Solved,

Daniel

16

I wonder if it would be helpful to add some ARIA roles into Parchment. Things that could be relevant:

  • region: to generically indicate each Glk window. Or they could have specific roles:
  • log: for buffer windows?
  • status: for grid windows? It has default aria-live of polite, so maybe turn that back to off?
  • application: this would turn off all the standard screen reader key controls, so I think that’s probably a bad idea.

Other ARIA ideas:

Each window has an input box which is disabled if the window isn’t requesting input. For most games that means their status windows would never make use of the input box. But does the screen reader still tell you that there is a disabled input box there in the status window? I could make it aria-hidden so that screen readers ignore it completely.

It’s not really possible to make bold or italic text use any of the relevant html tags: b, i, strong, em. But I haven’t seen any sort of ARIA role or tag which could be used to indicate general emphasis.

17

Ahh. Yes when a window is cleared all the old text is completely removed. It’s not like a console app which might “clear” the window simply by printing a full window’s worth of blank lines, but would then allow you to scroll back to see the previous text.

But it sounds like it’s clearing the window before you’ve had the chance to read it all? It doesn’t wait for you to press a key or anything before clearing the window after part one?

I should really test Trinity myself. How long does it take to get to that point in the game? Is there a walkthrough or list of commands that you know of which I could follow?

18

Dannii,

The Screen Reader does announce the presence of a
second Edit Field, but it also says
“Unavailable”, so that the Screen Reader Operator
doesn’t try to use it, at least this one didn’t.

I wondered about why it was there, but having an
Infocom Game playing in a Browser is such a novel
idea, I ffigured that it might have been from
some Early Code Effort, and you just hadn’t bothered to remove it.

As for your “Other ARIA ideas”, I’m afraid that
your speaking over my head somewhat, so if you
want to try certain things, I can let you know
the results, but as you’ve seen, I won’t always
be able to tell you why thus and such is happening.

I know that some Interpreters announce the Status
Window with each and every new screen of
information displayed, but I’m familiar enough
with what I have tried so far, that I don’t miss
it. On the other hand, games like Beyond Zork
change the exits to a given location with each
New Play of the game, and also each time the
player moves from one room location in the game
to another. I’ll see what kind of problem this presents, and report back.

As For The Trinity Walkthrough, here goes,
below. I’ve played it so many times now, I could
probably perform the necessary actions in my sleep.

19

There’s a solution for Trinity on CASA. See :: CASA :: Trinity

1 Like
20

log might be a good thing to try for buffer text windows, with aria-busy on the window element. I also think your ideas about grid windows are good, most screen readers have a keystroke for reading elements with a status role. I suggest setting tabindex=“0” on grid windows in case the keystroke doesn’t work, although doing this should fix focus issues which may cause that error. I personally agree with setting aria-live to off for the grid windows because of the keystroke, although if there’s more than one, the screen reader may only read the first one it has buffered when the keystroke is used. I’m not sure how much control you have over that, though, since I’m pretty sure that’s game dependant.

As for the edit box in the status window, NVDA and Narrator can both see it, although they both report it as read-only, which means it shouldn’t be a focus priority for the screen reader. This might be a problem specific to JAWS, though, since I can’t reproduce it on either Windows machine. JAWS may see the status window update and be exiting forms mode to allow the user to choose which edit field to focus, even though the cursor is already positioned on a field which accepts input. Hiding it may fix this, or giving it tab-index"-1", which will keep it visible but prevent it from gaining keyboard focus, hopefully making it less of a target. I guess there’s not much harm in hiding it though if we never type in it.

I’ve been brushing up on the GLk spec and Parchment itself so I can hopefully be of more help.

1 Like