(This post is a follow-up to A Matter of Heist Urgency postmortem. It’s also long.)
One aspect of the IFComp participation process that is rarely discussed, analyzed, or shared is the collection of online play transcripts. Inspired by Linus Åkesson’s article here, I’ve decided to share with you some of the fascinating lessons I learned from analyzing this valuable source of feedback.
THE STATISTICS
At the end of the judging period, I had received 160 transcripts. I don’t know if this is an average number of them; Åkesson seems to have received just about 100. Nevertheless, a large majority of the transcripts were not finished. Of these 160, only 54 made it to Part 2, 45 of those made it to Part 3, and a mere 32 of those made it to the end of the game. Around a dozen transcripts quit at the beginning of Part 3 - apparently having lost interest in fighting, as I mentioned in my postmortem.
AMoHU explicitly warns you not to play online because of the music. It is important to keep in mind that, because of this, the online players are not enjoying the game as much as desktop players. About 40 transcripts quit after seeing the warning about playing with the sound off. This could mean one of two things: either they really wanted the “full Anastasia the Power Pony experience”, or they decided that this game was too dumb and that they didn’t want to play it. It’s impossible to tell for any given transcript.
The distribution of transcripts into categories based on their end-of-game rankings is also interesting to consider. For the first fight, here is how they were distributed:
- Kung-Fu Amateur (lost fight): 5
- Kung-Fu Pony (won fight but didn’t save Ponyheart): 23
- Kung-Fu Master (won fight and saved Ponyheart): 6
(The numbers don’t exactly sum to 32 because a few transcripts reached the ending multiple times, by restarting or replaying.)
Saving Ponyheart is intentionally quite hard. It helps to use the coconut or give it to Ponyheart, to watch his targets and concentrate your attacks on the same llamas, and to use your superpowers. In Release 5, the first fight is harder than it was previously. For the second fight, the ranking is given by a table, based on two measurements: whether you won or lost and how many stunts you did. These are the results:
- Deck Swabber (lost fight with no action stunts): 5
- Buckle Swabber (lost fight with 1-2 action stunts): 7
- Stunt Actress (lost fight with 3-4 action stunts): 2
- Kung Fu Devotee (won fight with no action stunts): 3
- Buckle Swasher (won fight with 1-2 action stunts): 13
- Swashbuckler (won fight with 3-4 action stunts): 7
- Swashbuckler Supreme (won fight with 5 action stunts): 0
The two biggest categories are “Buckle Swabber” and “Buckle Swasher”. The results show clearly that the action stunts succeeded in their function and have a significant impact on your win chance: of those who performed no action stunts, 5 lost and 3 won. This ratio increases to 7 lost / 13 won for a medium number of action stunts and 2 lost / 7 won for a large number of action stunts. It’s possible to win without using them at all, but it’s unlikely.
Not one person found every action stunt - another predictable result, given my intentional choice to make it near-impossible for a casual, single-run player. Some stunts render others impossible during the same run, so the puzzle is figuring out how to perform the action stunts in the correct order. It helps to realize that performing stunts still counts even if you don’t defeat a llama with it, to avoid taking out the gun or sword llamas with stunts, and to maneuver around the ship to avoid being surrounded and defeated. The llamas are slower than you are, and only act roughly every other turn - some randomization is involved.
Interestingly, there was little correlation between those who did well in the fights and those who were the most engaged - measured by the length of their transcripts, and whether or not they were restarting and/or restoring. The most common restore point was going back to the final choice in the boss fight - whether to follow the Big, Bad Llama or take the Jewels back.
A few miscellaneous notes:
- Number of players who tried to kill Mumblebumble: 1
- Number of players who said “no” when asked if they wanted to hear about the ceremony: 11
- Number of players who gave the coconut to Ponyheart: 1
- Number of players who chose to “whap him as he stands there”: 12
- Number of players who used the verb “fight” as their primary attack command: 12
- Number of players who flew between areas on the Getaway Ship: 0
THE PLAYERS
How many transcripts were Comp judges, and how many were just casual IF enthusiasts (or their children)? At most 32 could have been judges, since they were the only ones who finished the game. Some transcripts were clearly those of players inexperienced with parser games, and the number of transcripts that got lost in Part 1 was quite surprising. Nearly all of those who never reached Part 2 were almost certainly new parser players, judging by the difficulty they seem to have had interacting with the game, and the lack of basic abbreviations like “x” for “examine”, etc.
Among those competent in parser interaction, the transcripts divided along the same line I discussed in the postmortem - a few players were engaged, replaying and exploring different possibilities (a few stretching to over 200 commands), and many were not. At first glance, the game doesn’t appear to have much replay value, since “winning” the game is guaranteed. In fact, much of the text is procedurally generated to some extent, allowing it to remain funny on repeat playthroughs. The game was designed so that saving Ponyheart and getting every action stunt in the deckhand fight is a hidden win state. It isn’t easy! This is the game’s real puzzle, and it’s one most people do not even know exists - or if they do, they don’t care to solve it.
THE VIDEO REVIEW
One particularly unique and interesting feedback source was this Twitch video transcript (starts at 1:25:09) by two people who go by the handles ferkung and jay___ram. It is a video transcript of the real-time response of real players who know nothing about, and are unaffiliated with, the game. The most interesting thing about the Twitch transcript is their gradual transition from skepticism to engagement. I have watched it several times, and it helped me greatly in the design of Release 5. It is extremely valuable, valuable in the same way that the transcripts are: both are spontaneous and undistorted by hindsight. Thank you to ferkung!
The high points of this transcript to me are:
- When ferkung first hears the music, and restarts just so jay___ram and the audience can hear it too. Later, they wait silently while Part 2’s intro music plays.
- jay___ram’s comment about the first fight being a surprisingly well-written action scene, and her later exclamation, “Okay, this game rules.” upon taking off into Part 3. (She was clearly dubious at the beginning.)
- Their continual appreciative remarks about the game’s polish level - such as when they try to wake ponyheart and “clonk” a llama with the coconut.
FINAL REMARKS
One complaint often leveled against randomized, turn-based combat in IF is the fact that one can simply >UNDO if an undesirable result happens, destroying the flow of the game and the player’s engagement. I would say AMoHU disproves (at least partially) this long-held idea; not one transcript typed >UNDO when Anastasia was hit, or even when she was knocked out. Presumably this is due to the feeling of urgency created by the writing and mentioned by many reviewers. Another point of interest is that the transition from parser-based interaction to choice-based interaction in the boss battle didn’t jar anyone, and was never even brought up in a review.
The anonymous judge feedback was interesting, but not unique - it showed the same groups and criticisms as the reviews and transcripts. It ranged the whole spectrum from those who felt they had no agency and were merely along for a confusing ride to those who thought it was extremely fun and said they would play a sequel (in fact, one person commented that they had not heard of the Anastasia games before, but that they will look for the others.)
All three of the main feedback sources - reviews, transcripts, and the video stream - were fascinating insights into the other side of the screen, and all three helped me to improve the game for Release 5. If you haven’t tried the new release already, I would encourage you to do so. In addition to several new music clips, there are two new scenes, both significantly improving the game, and only one is accessible in a given play-through. The game is still short - around 30 minutes to play.