Help me fix these Counterfeit Monkey bugs

In the current version of Counterfeit Monkey, when first entering the cathedral gift shop, the player will see this:

We could ask what is worth seeing in the New Church.

>

Typing “Ask what’s worth seeing” (note the curly apostrophe) will produce this:

> ask what?s worth seeing
If you’re trying to converse with other characters, the suggestions in
the text provide possible phrasings; so if you read “I might ask about
lentils.”, you might phrase your command ASK ABOUT LENTILS.
Introducing other words or variant phrasings that weren’t part of the
suggestion may confuse the game.

Alternatively, if you just want to take an action in the game world, try
giving a direct command, such as EXAMINE THE ASP or WAVE THE
P-REMOVER AT THE PHONEY.

*** Run-time problem P39: Attempt to say a snippet value which is
currently invalid: words 5 to 4.

So we have three problems:

  1. The apostrophe is replaced by a question mark when echoing the command.
  2. There is a run-time problem.
  3. The game does not understand “what’s” as equivalent to “what is”.

The first one will not happen in interpreters that automatically replace curly apostrophes with straight ones before passing the input string to the parser, but most don’t. It also won’t happen in interpreters that don’t support input echo suppression.

In those that do, Counterfeit Monkey will suppress the automatic input echo of the interpreter and print the input string itself. The code for this is here. This is in order to avoid a superfluous line break when navigating by clicking the compass. However, it seems to convert any unicode characters in the input to question marks.

It may seem like it should be trivial to replace any curly quotes in the input with straight ones. There is Inform 6 code in the Punctuation Removal extension that replaces straight apostrophes (Latin-1 39) with spaces, for example (this code is not used by Counterfeit Monkey):

[ SingleQuoteStripping i;
	for (i = 2 : i <= (buffer->1) + 1 : i++)
	{ 
		if ((buffer->i) == 39) 
		{	buffer->i = ' ';  
		}
	}
	VM_Tokenise(buffer, parse);
];

But it seems like the curly apostrophe is already replaced with a question mark at that stage. Is this an interpreter bug?

I don’t know how to even start debugging the second one, run-time problem P39. It seems to only happen when triggering the Smarter Parser extension “asking unparseable questions rule”. It won’t happen when typing “what’s worth seeing” without the “ask” part.

The third one is more like unimplemented functionality than an actual bug. There is a rule in the Smarter Parser extension for cases like this, the standardize apostrophes rule, but even if I add a line like

	replace the regular expression "what[apostrophe]s" in reborn command with "what is";

it still won’t work, not even with a straight apostrophe.

Note that there is another rule in the Smarter Parser extension named “Smarter Parser simplify punctuation rule” which does some quotation mark removal. This is not used by Counterfeit Monkey.

The intended response, which you get if you type “ask what is worth seeing” or “what is worth seeing”, is:

“So tell me, what should I be looking for in the New Church?” we ask.

“Other than God?” he asks dryly.

3 Likes

No, it’s a parser bug, but a very deep-settled one.

The I6/I7 parser (through the latest release) uses 8-bit character buffers for all input and input processing. So it simply can’t handle curly quotes. The interpreter replaces them with question marks because it has to store some 8-bit value in the input array.

The next I7 release will fix this by replacing all those arrays with 32-bit arrays. Then all string handling will support full Unicode. Needless to say this change touches a lot of parser code, and will break a lot of extensions too. It’s a very invasive fix.

6 Likes

I’d say a well-behaved interpreter should still convert any curly quotes and apostrophes to straight ones before passing them to the parser. It is less of a problem in Gargoyle, as it will never “auto-correct” straight quotes to curly ones while typing, but the Zoom interpreter built into the Mac Inform IDE will, unless this is turned off in the macOS system settings.

EDIT: Thanks for the explanation, by the way!

1 Like

Another option is to strip out question marks from the player’s command, because you can’t know what unicode value was originally put there, and “whats” looks less objectionable than “what?s”.

2 Likes

Agreed. This is actually done already, at a later stage. Still not as nice as doing it at the interpreter level, though.

When doing 8-bit input? I can see the argument, but it’s dangerous to try to second-guess the game in ways that the game can’t work around. You can wind up with layers of undocumented interpreter cruft that will never be removed.

1 Like

As a stopgap until the full 32-bit parser comes out—or as a permanent solution if you’re not planning to put in the enormous amount of work to update the game for Inform 10—it wouldn’t be too hard to write a “for reading a command” rule that puts the player’s input into your own 32-bit buffer, then transfers it into the parser’s 8-bit buffer, with your own customized handling of any characters above 255. This wouldn’t require any parser-rewriting and would be sufficient for things like converting curly quotes to straight ones.

4 Likes

This is really weird. How is it possible to type a curly apostrophe?

Certain keyboards, or copy-paste. For me it’s Compose-<-' for ‘, and Compose->-' for ’.

I want to try this out, actually. What version of the compiler are you using for CM?

On Mac, option-[ and ] and shift-option-[ and ] are curly-quotes and curly-double-quotes. (I can never remember the order, though, and always wind up trying all four combinations to get the one I want.)

Yes, that will work.

Curly apostrophes are more likely to be used when copied from a transcript or walkthrough, where they might have been accidentally converted.

1 Like

The default on macOS is to automatically convert quotes and apostrophes to curly ones when entering text.

EDIT: On iOS as well.

1 Like

Inform 9.3 (6M62), December 2015.

I started working on this, but ran into a strange problem. After the line event comes in, the Unicode input buffer is full of zeroes. I’ll hopefully have time to debug it tomorrow, but in the meantime here’s the WIP code in case someone else can see the issue.

Unicode Input Handling.i7x (6.3 KB)

2 Likes

Is the problem in CM only because of the command-echoing involved in this case? I mean, I just tried entering curly apostrophes in commands in Spatterlight, in a game that strips apostrophes with the Punctuation Removal extension (my WIP) and it all worked okay. That is, I typed “boy’s jet” (with a curly apostrophe) and behind the scene it still became BOYS JET and was recognised by the game’s grammar lines as the jet belonging to the boy.

-Wade

This works because Spatterlight converts the curly apostrophes to straight ones before passing them on. Obviously, this highlights one potential problem with doing so: If you only test your game on Spatterlight, you won’t catch this problem, which may then cause trouble in other interpreters. It likely won’t work in the Mac Inform IDE, for example.

1 Like

But it’s not just my game’s specific strategy of stripping apostrophes that’s in trouble then, apparently.

Wouldn’t every interpreter that will accept the player typing curly quotes, and which doesn’t straighten them before they go into the game, fail to match any grammar token that includes apostrophes? Because all those apostrophes in Inform are going to be non-curly, right? So every game with possessives in object names would fail to match them when the player typed them.

Is there really an interpreter doing this? I feel I’ve never seen anyone mention this happening.

(In my own daily life, I just have smart quotes off 100% of the time. I only turned them on muck around in this topic.)

-Wade

That is correct, and that’s why it would be nice to have a straightforward way to fix it. But most players will probably never encounter this. In fact, now that I think about it, I’m not sure it is much of a problem anywhere outside of the Mac Inform IDE interpreter.

EDIT: I made a pull request that will hopefully help with this in a future release of the IDE:
https://github.com/MaddTheSane/CocoaGlk/pull/2

2 Likes

It was probably a mistake to ask what is in effect three different questions under a single topic. I’m starting a specific topic about the run-time error here:
https://intfiction.org/t/run-time-problem-p39-when-changing-the-number-of-words-in-input-in-counterfeit-monkey/65734

1 Like