Ways to test your walkthrough?

I have found it a struggle to test my walkthrough. I mean, it’s like healthy cereal or exercise. I know it’s good for me, but I go for more immediate fun instead.

However, I found that a listen/smell through provided me with a good way to go through my game again.

In Inform 7, this is simply:

[code]chapter lsing

lsing is an action out of world.

understand the command “ls” as something new.

understand “ls” as lsing.

ls is a truth state that varies.

carry out lsing:
if ls is false:
now ls is true;
say “Listen/smell is now on.”;
else:
now ls is false;
say “Listen/smell is now off.”;

every turn (this is the stop and hear the roses rule):
if ls is true:
say “[bold type]LISTEN:[roman type]”;
try listening;
say “[bold type]SMELL:[roman type]”;
try smelling;[/code]

Does anyone have any other variations on a straight up walkthrough that worked for them? That helped keep testing fresh? I found that tracking this helped me ask questions about my world and make me more engaged to find “stupid” bugs, too.

Also, not 100% sure this is the right subforum…but I’m wondering what ideas people have of their own, or for other languages.

I use something like this.

Test all with "test first / test second / test third / test fourth / test fifth / test sixth / test seventh / test eighth / test ninth / test tenth".

Here, the sub tests have part of the walk through allowing you go quickly get to a particular point in a game. I then also use this.

Test part with "test first / ...".

This is so that I don’t have to keep typing “test first/second/…” in order to repeatedly get to a particular point.

The great thing about this is that if “test all” doesn’t get you to the end, you know something’s broken!

Hope this helps.

Definitely, I think it’s a great spelling out of what we need to do–because it took me a while to get to where I had this, and it’s a huge help to make sure nothing’s broken for players who want to use a walkthrough.

I also have a simple extension that allows you to test which part of a chained walkthrough fails, but that’s for the I6/I7 subforum. I’m not sure if it adds anything to the existing extensions (it’s a bit different from text-reading) & I’m not sure how to document an extension adequately anyway.

But I guess I should’ve been more clear with what I was asking…once you’ve run these automated tests, you know Things Work For People Who Know What to Do.

It’s just, when you (meaning I) actually have to read, in depth, what you’ve written, you can’t rely on automation, and it’s tough to have a new perspective on things or -not- get bored of your own writing, even the stuff you’re pleased of. I’m not sure how to spice it up.

Between the false sense of security automated tests can give me and the fatigue of re-reading, I guess I consistently let cringey errors through. And I need non-obvious ways to break the dread of rereading. I bet others have fallen in this trap too.

Aren’t the skein and the transcript tabs in Inform 7 supposed to be good at this? The documentation describes the two as “a very powerful testing aid”. See “1.8. A short Skein tutorial” of The Inform 7 Documentation.

Hope this helps.

Those help a lot–I guess the thing I’m looking at is, finding a way to be able to play the game like the player without the risk of glazing over text.

Using the skein/transcript gives me the whole view, but there’s so much information I tend to gloss over it after a bit.

You can’t.

The skein is wonderful for keeping track of a ‘known good’ play through, and the transcript is fantastic for proofreading that play through (And by blessing passages you can easily tell when they change unintentionally)… but as for playing it like a new player, it’s impossible.

I know the solutions to my puzzles. I won’t ever experience them the way a new player would.

Definitely–I can’t expect to do that, but I need as many ways as possible to keep things new and to make myself pay attention to something I forgot. I really should use the skein more, though. I suppose it’s a semantic difference: like vs as (never be -as-, but I can get close by letting time pass before I revisit.)

This is less puzzle related and more poking around the world related as I am probably more concerned about stuff I never gave the player a chance to look for–stuff I implemented but did not describe adequately. Even using test extensions to make sure everything is described doesn’t help my blind spots.

I’d guess that’s what testers are for. Maybe you could find someone and enter a mutual testing relationship with them.

I don’t think this is meant as a replacement for testers, dude; it’s intended as a supplement to them, and a courtesy. There are finite testers, and they have finite time.

Particularly useful for the more mechanical aspects of testing. There are plenty of things that would be extremely boring for testers to check every aspect of, but can be very thoroughly covered by the right testing script. (Emily used lots of automated testing to find possible words for Counterfeit Monkey, for instance; hoping for testers to catch everything piecemeal would be unrealistic. Zarf uses similar techniques to test that all the solutions work in Hadean Lands.)

Automated testing lets your human testers focus on the stuff that humans are good at.

Sorry, it was a bit of an inside joke. He is currently testing my game.

Haha, yes, would’ve been difficult for him to know :slight_smile:. So, err, no harm no foul Ihope.

I think the main thing that frustrates me is that programmer-testing can’t be a replacement for tester-testing, but I don’t know if I do enough of the nontechnical stuff to be sure that something is visible. I certainly know that in a software testing environment, regression testing can be a button-pushing thing if planned right, but testing a new feature can open a lot of questions with no right answer.

So making sure everything’ there seems like it should be straightforward, but it isn’t, and that results in blind spots, or testers saying nothing because they can’t see what I wanted to show them (as opposed to because it works ok.)

It just takes so long to notice nobody noticed that thing I put in (though that takes time and effort for tracking stuff,) and I’ve seen a pattern of not revealing things I spent a good deal of time on to testers, and regular stuff like checklists etc. doesn’t help. Or I’m able to make them, but they get boring without variety & I taper off and throw something in a shiny new area.

So I think I would get a lot more out of my testers if I were able to make things clear and vivid the first time through. Because, parallel to what you mention, once a tester ‘knows’ a puzzle, he can’t unknow it–much like the author can’t un-know it.

And I’m curious how others keep their own playthroughs interesting. Because I have making sure neat things are visible as a goal, but a post-release audit shows I’m still not getting it done.

I’m probably repeating stuff you (Andrew) and I have talked about before. I guess my approach with my games is – I know that testers are finite, and also that they’re only fresh to the game once. That’s why I don’t like to give out early versions of my game in general. If I do, I think testers will spend their efforts bashing it into shape, and will have lost their freshness of the higher level stuff that will ultimately need to be done. Of course it depends on the game, and any individual’s confidence in general in dealing with the early stages themselves. I’m someone who generally feels confident in the latter.

Another approach if you do want or need people to deal with earlier versions is to preserve some testers. Only let certain people start playing the game later in the development process, though have them lined up from the start with everyone else. Then the ‘oldies’ can either continue to bash at longstanding stuff if they’re happy, or just have a rest, and the ‘newies’ can come in with full energy.

  • Wade

I pretty much set up the equivalent of that. I created a system of input recordings that go through “Tree and Star” using the playback feature in the Hugo Engine/Debugger. I have different recordings to get to different rooms, scenes, and branches.

This is a workable system, but it actually was more useful when I was originally developing the game. Now that I’ve added new features and mechanics, most of my recordings are broken, and it’s hard to discover exactly what is going wrong—even harder to discover how different mechanics and rules that I’ve changed might be conflicting. So, yeah, I’m really bad at this whole debugging thing.

I agree that you need people who haven’t seen the game, or your stuff, much before. They’re more likely to be objective. But from the testing side I enjoy being able to say, look what neat stuff this person did! Here’s stuff I don’t take for granted & it’s worth telling them this works, or to be able to say, you improved A and I bet you could improve B the same way.

I also think that there’s a half-life to being annoyed with puzzles, or remembering them…and once forgotten, they can be approached a certain way or a similar way. There’s even the possibility of the tester being able to be a useful contrarian, which, when done right, opens up other questions. That’s hard to do on the fly but I find after a month or so, I can re-attack.

Yeah. Definitely, it helps to have people tackling the whole game–and people willing to give first impressions. Also, there is general stuff like doing a walkthrough every weekend, but I’m still struggling with how to give that walkthrough enough variety that I find new things.

Though I did realize that repeated talkthroughs are valuable (new stuff to ask about.)

Maintenance of the scripts is nontrivial. I find I need to check them every 2 weeks. If you are able to take a transcript for what went wrong, when, it is a huge help. You can search the text file & I’ve found that often turns up other non-game-breakers.

You can then note whether it is a walkthrough bug or an implementation bug–and often, seeing a glitch either way can help you figure what a player might try. For instance, in the HHGG walkthrough below, ‘put gow on hook’ or ‘push red button’ would be a walkthrough bug, as would the more subtle failing to eat the peanuts, but ‘put robe on hook’ would be implementation (it’s an acceptable synonym.)

A nice side benefit of this in inform is that you can print out a walkthrough with Notepad++ or your favorite some-frills editor. HHGG walkthrough below.

test babelfish with “push green button/smell/x shadow/put towel on drain/block panel with satchel/put mail on satchel/put gown on hook/push button/put fish in ear”

becomes, replacing / with \n> (as regular expressions,)

push green button
etc.

Which can be highlighted and capitalized.

This can also be done with PERL, and I don’t know how this works for hugo.

This was invaluable for me. I had the crutch of a score so I could say, okay, 18 points means that I probably didn’t get the tape recorder working in Towers. Inform allowing you to look at objects/rules is a big help, too.

So I think a very legitimate test case is to get this working, then call for a tester just to look at transcripts of the minimal ways through.

This is the sort of thing I’m consistently able to do mechanically, but the whole making sure stuff is signposted is harder for me. So ask away with any more questions.

Another thing I do when sent a transcript is to replace “>([^>\r\n]*)\r?\n?([^>]+)(?=>)” with “\1\r\n”. This transforms the transcript into a list of commands that the player typed, which can then be loaded into the skein and run through. It’s as I always say “Using regexp alone makes it a good solution.”.

I just remembered I made this tool a while back, might still be useful to someone: nitku.net/if/beautifier/