Parchment transcripts from IFComp

Last month, I had the pleasure of visiting the PR-IF meeting. It was mentioned that the Parchment transcripts were stored and made available to the authors.

I was wondering if any of the authors cared to share any particularly insightful, unusual, humorous, or unexpected player commands. Also, was the transcript logging useful as a feedback tool?

It was pretty useful, but with hundreds of transcripts, a little overwhelming. Just clicked on a transcript at random. It ended:

The following transcript led to a substantial change in the code after Marcus talked himself into permanent death:

I said some of this at the meeting, but it’s worth repeating…

The transcripts were tremendously helpful. I got a Cold Iron update out late in the competition, but there’s no reason I couldn’t have gotten it into the first update cycle – I was just lazy.

It was easy to tell the experienced IF players from the newcomers. Experienced players all started out referring to the named, portable objects (in my game, the book); then looked at the named scenery (the table); and then went outside.

The newcomers weren’t that predictable. I see an “open book” here, for example, which I think most of us would never try. There was a large contingent of people who didn’t try normal compass movement – at least, not right off. “go to center of room”, “go around in room”, “walk to the door”, etc. One notable case tried “m n”, “m w”, “m sw” – I assume this is some convention from a MUD that I’ve never seen.

I don’t expect to catch the completely confused players. There was another contigent who were clearly not experts, but were typing acceptable commands: “go west”, “walk south”, etc. They didn’t get stuck. (I probably should have caught “leave house”, which was tried a lot in that group. “leave” by itself works.)

Then there was “find axe”, “get axe”, “look for axe” – commands which I really should have caught. That was the major addition in my update.

(Almost all the newcomers showed up in the first few days of judging. After the first two weeks, it was nearly all experts.)

Synonyms count. My game text said “A battered book is lying on the table”, and people typed “read the battered book”. If I’d been lazy about the “battered”, that would have been a noticeable hit. Also, there’s a reason the Inform library accepts “a” and “the” on all objects. Don’t short-circuit that when you implement new commands.

Player Experience Upgrade (or, if cherry-picking, just Smarter Parser) would catch all of these. BOOYAH!

This one isn’t caught, but I’ve now added it to Small Kindnesses (also part of Player Experience Upgrade) for the next release.

Having played a couple of IFComp games with Smarter Parser, as a player I would dearly like the option to turn it off in any given game. A fair amount of what it does seems like it captures stuff that isn’t a problem for me as a moderately experienced player (I’m not going to type “where am i” unless heavily prompted to do so), so it doesn’t do much to upgrade my personal experience.

Less selfishly, it also seems to slow things down considerably. Playing online on an admittedly tetchy laptop, referring to an unimplemented object would sometimes send Parchment into such a deep think that it would give me an “unresponsive script” error; it was a great disappointment when a long wait yielded only “You can’t see any such thing here!”* Trying it again on a non-overloaded browser just now, the delay caused by error messages wasn’t nearly as drastic, but it’s noticeable.

And I sometimes found that Smarter Parser clashed with the game in a way that made me mistrust the messages I was getting from it. The most drastic example was the message that you don’t usually need to use adverbs ending in “-ly”… in a game whose main character was named Lilly. (Which has since been revised so that error message is less likely to show, but still.) It remains true that you don’t need to use adverbs, but the message so clearly is mismatched to the details of the game that it makes me distrust the other error messages I get. So when Smarter Parser later tells me that I don’t need to refer to body parts, I don’t believe it. And I wouldn’t be surprised to find that message popping up sometime when you do need to refer to body parts.

So, I would advise caution about just dropping Smarter Parser into a project unless you know exactly what it does and can customize any messages that aren’t exactly appropriate. And, though it surely does help with some of the most common misunderstood commands that novices type in, I’m not convinced it has that much to offer the non-novice player, and it would be nice to be able to avoid its costs.

Of course I haven’t analyzed transcripts, so maybe non-novice players are making these errors too, but from zarf’s account it didn’t sound like it. (BTW, I think I recognize some of those commands from this playthrough, which was somewhat unusual in that you had a total noob and an experienced player looking over her shoulder to occasionally offer suggestions, like “try going in a compass direction.” I was impressed at how much was understood; as zarf says, accepting “a” and “the” is really helpful.)

Having complained, thank God for Small Kindnesses. I can’t stand being told “I only understood you as far as wanting to exit” when I try to exit the thing I’m in.)

*This was a game that used Smarter Parser but not Neutral Library Messages.

This is perhaps not as well-known as it should be, but you can in fact type NOVICE OFF to disable Smarter Parser.

No, it’s definitely targeted squarely at novices, who in most collections of IF transcripts I’ve studied are the majority of players. (This is usually in contexts outside of interactive fictions competitions, mind.)

This is a legitimate concern for both authors and players, I know. Progress continues on several fronts, including low-level Inform 7 optimizations and the continuing Javascript execution speed browser wars. Someone can correct me if I’m wrong, but I think regular expressions are not particularly optimized in I7 at present. I know Dannii opined at some point that this might be relatively easy code to optimise, but I don’t think that’s happened yet.

Yeah, these are both genuine concerns with Smarter Parser. On the -ly front, this was an attempt to make things run faster (natch) by switching out a list of common adjectives in a regular expression for something simpler. I don’t have a good sense of whether it’s doing more good than harm in present form, although this example is certainly a data point against.

The latter case is also problematic. Certainly if I was writing a game that did involve the use of body parts, I would probably override the Smarter Parser block related to this, or at least change its message. I do realize that not every author who can use an extension knows how to do this. But here I think there’s more of a clear win: I’d feel safe in saying there are many more games without implemented body parts than those with, so the benefit of matching the semi-common novice error of trying to interact with them outweighs the cases where it might produce a misleading message.

In general, extensions or no, authors should always customize the built-in messages for their own game. It’s not as easy as it should be to do this right now, especially for extensions in a consistent way, but it’s still true.

Oh, and also! Thanks very much for the feedback. [emote]:)[/emote] Extensions do not get the light shined on them often enough.

Disclaimer: I’m not intimately familiar with this set of extensions from an author’s perspective, so please forgive me if I’m just talking about “smarter parser” as a synonym to include some others.

I believe this extension is seriously overused right now. If you’re actually writing a game targeted specfically at newbies, fair enough, use it. If not, then think hard if you really know the implications before you drop it into your game. Out of the games which used this (or parts of it) in this year’s IF Comp, it only did harm in a good number of them. At some point, it already made me crabby if I noticed one f these “friendly” messages in a game for the first time, because I knew there would be problems.

I absolutely hate how this thing breaks the fourth wall constantly, e.g. to explain that (parahrasing) “sorry you are feeling frustrated, maybe you should save and come back later”. In response to a completely legitimate command attempt, by the way – the frustration only resulted from reading this brain-dead parser reply. Everybody here seems to be talking about immersion here, and then you’ve got parser messages like that? And what’s the deal with “That noun did not make sense in this context”? Every single time I’ve seen this message, it was in reply to a command whose noun made perfect sense in the given context – it was just the game which couldn’t make sense of it. A newbie-geared extension putting the blame on the player? Seriously?

And don’t get me started on “I understood X, trying that instead”. That is outright dangerous. These shortened implicit re-interpretations often mean something vastly different than the player intended. “throw object out of something” might be a smart idea in some situation, but dropping “object” (which is “throw object”) might not.

Of course, these are just examples of what I believe are big problems in any game whose author tries to improve his game by simply inserting this hyped extension he’s heard so much of. Don’t do that! The default Inform parser has seen years of careful tweaking and optimisation. If you mess with something this crucial, you can break many things. If you know what you’re doing, fine, but don’t think this will somehow magically improve your game.

More on this:

I think, although aimed at a wider audience, IFComp is nonetheless a Comp for IF. Meaning it should be aimed to people at least knowing what an IF is.
Many of my transcripts are 10 turns long, with them 10 being “walk here/do this/use that/sympathize with minorities/remember childhood/etc” if not straightforward profanities aimed at the parser (or so I’d like to think… ehm). Well, my game, as many of the IFComp games, isn’t targeted in that direction, and I don’t really care about what my father would experience in a game like mine.

After all, if I go looking at a football championship, I’m supposed to know what an off-side is and how it is allowed to score a goal.

So, in my mind there is no worst thing as a game too much aimed at newbies. A parser can be more or less understanding but, c’mon, I can’t foresee what happens when someone types what I know are WRONG commands in a text adventure. Let alone trying and find a solution to “f**k off author” that is more nub-friendly as the regular “Real adventurers do not use such language”.

That particular response is a default Inform library message, actually. (Your general point about overuse may of course be valid nevertheless.)

Aaron – thanks for the gracious response, and for telling me about Novice Off! I think the way to publicize it is probably in the ABOUT text of games containing Smarter Parser, which is probably not something you can automate.

About the body part thing, I’m not so sure that it’s a clear win. Very few games will have implemented body parts (though I have a WIP that implements a furniture leg!), but those games would really be harmed by this message, while the gain for the ones that don’t doesn’t seem so great.

But I think what would really help is an expansion of the Smarter Parser documentation to explain exactly what each rule is, what it does, and how careful you should be about using it. During the comp I looked around the docs to find out where these things were explained, and I couldn’t find it. Now I see that it’s in the existing rules section, but that’s kind of terse. Pretty far down we see

(which, followed by the “usually no clothing rule,” seems like something suitable for Stiffy Makane). But it’s easy not to understand exactly what it does, or what message it’ll produce.

In some ways, this means that I think Hannes is basically right – don’t plug and play this extension unless you know exactly what it’ll do!

I also agree that retrying the shorter command can be downright dangerous – maybe the default should be to retry the command if the player consents. And I think it would really be player-friendly just to sever the link between “throw the rock” and “drop the rock.” I hate that.

Actually, I think that Small Kindnesses suppresses “You can’t see any such thing” in favor of “That noun did not make sense in this context”. I’m not sure if Hannes is getting burned by this, but it is a very unfortunate bug (design choice?) in the extension.

Both the problem and a fix have been discussed on this forum: <a class=“postlink-local” href="https://intfiction.org/t/about-default-responses-using-small-kndness-extension/3006/1

–Erik

So I’m hearing the frustration, and thanks for airing it. The more feedback, the better these extensions can get.

Let me address a couple points, but first, mind that I did not invent this stuff out of whole cloth: most of the changes are in direct response to specific issues I’ve seen real people having. Some of the research I did on this is here, for anyone interested.

Also, these extensions are explicitly targeted at authors who want to make their stories more accessible to people not familiar with IF conventions. This is sort of a no-brainer for me: I would prefer to write work capable of being enjoyed by the 99.9% percent of people in the world who aren’t IF experts. But again, authors who don’t care about that crowd are free to ignore them.

(Although, if I could make a humble observation, I feel like my success bringing IF to places like IndieCade and PAX East has less to do with the quality of my work and more to do with the effort I’ve put into making that work accessible. People have a real hunger for interactive fictions; most people just can’t figure out how to play them.)

If you have specific examples of messages that you feel are putting the blame on the player, I would really like to hear them. I agree that this is bad, and my extensions have tried very hard to re-write any default library messages that do so into more neutral forms. The default response for waving, “You look ridiculous waving the noun,” is replaced by the less-insulting “You wave the noun,” for example.

This took longer to get around to than I’d intended, but thanks to Erik’s help this has been fixed for a new version of the extension pending an update to the I7 extensions page.

Perhaps. I have definitely seen a lot of newbies in a lot of different contexts refer to body parts and draw incorrect conclusions about what it meant that their command was rejected. But it could be this functionality should be subsumed by the “with my X” section. I’ll put some thought into it.

That’s probably a good idea, but are the authors who aren’t replacing the built-in messages when they should really going to wade through a dozen extra pages of extension documentation?

Isn’t any parser error message breaking the fourth wall? The ones that don’t are making assumptions that are often absolutely incorrect. This boils down to a difference in philosophy, but I’d much prefer the parser be honest with me about whether it understood me or not, rather than pretending to with a message like “Attacking the troll would have no effect.”

Can you give me an example of a completely legitimate command you typed in which Smarter Parser misinterpreted? It should only kick in for commands that the parser couldn’t understand at all.

So, scenario one: someone who doesn’t know the correct command structure tries something overly complicated and gets a flat-out refusal with no clue what the correct syntax is. They’re stuck on a guess-the-verb puzzle and end up quitting your game in frustration. Scenario two: Smarter Parser either correctly simplifies something and the game works, or mistakenly guesses and the player has to type UNDO, now understanding how that verb is supposed to work. Two seems like the clear win to me.

This is “dangerous” if the game prevents UNDO and is liable to instantly kill you, I suppose, or if the player doesn’t know about UNDO. But I suppose I would prefer moments of danger to most people never getting past a syntax problem.

Fair enough. For me, there is no worse thing than nine out of ten people who try an exciting medium I care a lot about being frustrated and turned off to it by easily correctable problems.

1 Like

I don’t know. Maybe you should be discouraging authors from dropping this in if they’re not going to replace this message when they should! But I think there’s no reason not to make it possible for authors to find out what this extension will do in their game, if they want to.

Scenario two: The thing you wanted to throw is on the floor. You’re still stuck in the guess-the-syntax puzzle if the problem was that you were using the wrong preposition (this has happened to me), and it’s obnoxious to have to pick it back up; it’s also obnoxious to have to type UNDO (do newbies even know how to do this?) because the parser grabbed your action and turned it into something different. What’s wrong with asking the player whether he wants to carry out the revised action?

Going back to the original question: Yes, I found the transcripts to be very helpful and enlightening. Of course I still haven’t updated my game yet, but I have a long list of bug tickets to fix based on the IFcomp transcripts. The biggest no-brainer for me was when I had a character say “I don’t suppose you’d trade [something the player has] to me?” but I didn’t implement the verb “trade” (and none of my beta-testers noticed). Bracket Every Notable Verb Too, I guess!

The most amusing/sad transcript is this one:

ifcomp.org/authors/transcripts.p … utput=html

In particular, they expect this command to work:

Find the shallowest point in the river and cross.

It ends in sadness, though, when they try various versions of “exit boat”, and never stumble upon just “exit” (or “south”). It might be that Small Kindnesses would have covered this case; I had in fact been using Small Kindnesses earlier in the development process, but I was running into weird/confusing messages and I didn’t have the time to fix them so I just removed the whole extension. (Sorry, Aaron, I don’t recall what they were, but when I get around to it I will look at it again and let you know.)

I did keep Neutral Library Messages, because I hate many of the I7 default messages. In general, though, I think I would rather just remove an unneeded command altogether (leaving some sort of “I don’t understand that word” reply), rather than having a different “that doesn’t do anything” message for each thing that people might type, especially when it’s a command I didn’t even realize was implemented by default in I7 (like “hug”, which results in a message about kissing). I spent a lot more time fixing messages for commands that don’t work than implementing the commands that actually do work.

Yeah, “exit [the container you’re in]” really really needs to be implemented. I always do that even though I know how it usually works. This just inspired me to install Small Kindnesses into my no-longer Speed IF, so bad things don’t happen when people try “get off the bus.”

Neutral Library Messages seems like it’s actually causing trouble here, though; if the parser had responded “I only understood you as far as wanting to exit” there’s a fighting chance they might have tried “exit” next.

I don’t have a problem with the parser trying a new command on the player’s behalf, but it should mention the option to UNDO when it redirects the command, e.g.:

–Erik

Standard I7 responses break the fourth wall constantly. The difference is that you’re more used to them.

…an incomplete and ongoing process, of which Smarter Parser is an important part.

Part of the issue there is the ‘careful’. After two decades of careful tweaking, the parser doesn’t really seem (to the casual observer) all that different from AGT. To make larger-scale improvements, occasionally it’s necessary to make bigger, riskier jumps, and piss a bunch of people off in the process. I dunno whether Smarter Parser is going to become a good answer or an instructive failure, but either way.

This is a valid point, and I don’t particularly want to see IF change so much that it can be easily played by all of the Other 99.9%. But I’d like to see it be positioned more like, say, literary poetry: not many people actually consume much of the stuff, but if you present some to any reasonably smart person they’ll probably be able to read it without getting angry and confused because the line breaks are all in the wrong places.

(IF isn’t football. IF isn’t even Australian football. IF is the Eton Wall Game. If you turned on a sports channel and they were broadcasting the Eton Wall Game, you’d probably expect the announcer to explain the rules a lot more patiently.)

1 Like

Probably the best outcome I could hope for is that I piss someone off enough that they write a better extension. [emote]:)[/emote]

I agree in principle, but at some point I’m just verbosely duplicating what authors could see for themselves by just opening up the extension source and taking a look at it-- they’re decently commented and divided into sections. But I will take a look at whether I can clarify some things without turning the extension documentation into a novel.

Maybe a better approach could be to make more use options to easily enable/disable bits of the extension based on the specific contents of your game. If you have implemented body parts, the extension docs could direct you to add “Use no body part assumptions” to your source. Something like that.

This particular example seems to be more the fault of the built-in throw = drop assumption. So far my extensions haven’t removed anything from the standard rules, just replaced or expanded on them. I’m starting to see more demand for an extension that goes farther, though (specifically, to remove all the do-nothing verbs.) Might be worth looking into, although the danger is that this breaks any other extensions that refer to those verbs.

A couple things: it makes the annoying, fourth-wall-breaking parser intrusion take two turns, not just one; using “if the player consents” breaks the skein and test commands; and certain phone/tablet/experimental IF environments are not set up to handle input in the middle of the normal parsing sequence like this. These are not insurmountable problems, but they’re the reasons I don’t use yes/no questions currently-- I did consider them.

This is a good idea-- I’ll add it. I could probably do a better job at working in more meta-instruction into the error messages. Perhaps the first time a Smarter Parser messages is shown, it could add that you can disable these messages by typing NOVICE OFF, or the first time a command is auto-completed, it lets you permanently disable that behavior.

NLM’s version of this is “I can’t understand your entire command: the first part might be an action I know, but I didn’t expect the word [x] next.” This is more verbose but seems equally likely to get the player to type “exit” next, and fixes situations where the original format produces confusing messages (like “I only understood you as far as wanting to take inventory” if you start a command with “I”.)

This is all great stuff, everyone. Didn’t mean to hijack this thread, but I’ve got a good pile of notes for future improvements now.

Point taken.

Anyway, Aaron, I was not trying to upset you. Sorry if I did. I’m just a lil bit mad at all those 400+ people who dropped my game after 6 turns cause it didn’t let them “patronize the environment” or something like that.
I think that anyone coding an extension is doing something noble, and you are one of those noblemen.

What’s eton wall game?

1 Like