Tales of Testing and Troubleshooting Woe

I have been scraping the inside of my skull for a week trying to figure out why a character kept coming on and doing his scenes at the wrong time. He only appears if a variable is set, so the scene mechanism is tripping him on. Let’s check the 20 different passages where that happens…

Crazy if/else stacking? No. Any place I typoed and he doesn’t get shut off? No.

Wow it’s with every character. I tracked every character watching the variables turn on and off in the debug screen as I clicked forward. The only way it would show that passage is if a specific variable got set and…

As it turns out, I had hard-set the variable in several passages so I could quick preview that the text worked correctly and forgot to take them out. Once that variable got set in one of those passages, the entire game became about that character. He’d start showing up saying his lines… like the understudy going on at the same time as the main actor. He’s doing act 3 with another version of himself also doing act 1.

Memo to me. COMMENT DEBUG COMMANDS IN THE CODE. MAKE A NEW TAG #DEBUG - #TAKETHISOUTDUMMY

Please share your most bizarre and “D’OH!” testing moments so I’m not all alone on this island!

6 Likes

You’re not alone!

I just spent the last week or so wondering why my interpreter seemed to get slower and slower with pretty minor additions to the story I’m porting. Sometimes it seemed to get slower when nothing at all had changed. Eventually it started taking close to half a second to process a single turn. I was starting to doubt the whole approach, thought about things I could optimize for a day or two, finally came up with a caching solution to get times back down around 200 milliseconds. That was good, but I still worried about things slowing to a crawl as the story grew.

Then I closed the console that had been open for a few weeks, and after opening it again, those long turn times were down around 50 milliseconds. Apparently leaving it open for a long time slows things down quite a bit. Who knew?

Not sure what the moral of that story is, but I’m sure there’s one in there somewhere.

BTW, comments like //FIXME and //TODO are so common that some text editors will give them special highlighting. You won’t be alone with those either!

2 Likes

I just had almost the exact same thing happen. One of the commands in one of my games tended to screw things up, so I left it until last to add it… and forgot to add it. The only reason it didn’t get published that way is that an editor pointed out to me today that there was no way to get to a specific chapter… because it needed that command.

It’s fixed now, but if I missed THAT then what other mistakes may still remain?

These thoughts fill me with terror.

1 Like

My sad testing/troubleshooting story involves a game with a lot of parser hacks. All my testers kept receiving a spurious “you’ll have to say which direction to go in” message every single turn. I would test the file, not find that, and mark it solved. Rinse, repeat.

What never occurred to me was that there might be a difference between the testing version of the program that I was running in the Inform app and the release version of the program that was created when I hit “release,” which was what the testers and my players were playing.

Turns out that one of the hacks would execute the first action in the library every turn. In testing versions this was a debugging action that didn’t do anything–in release versions this was the going action, without a noun, which produced the error message.

So definitely, you should run the release version of your game before you actually ship it.

3 Likes

The following anecdote isn’t about debugging IF code as such, but it could serve as a cautionary tale (read: entertainment) about programming in general.

I was a teaching assistant for an undergraduate programming course, and it was my job to stroll around the classroom and help students with their exercises and projects. A cozy job, most of the time, because when you run a group of people through the same set of exercises, they tend to get stuck in a small number of predictable places.

But there was this one person who couldn’t get their program to work, and for the life of me I couldn’t see what was wrong with it. Sure, it was a bit messy and hard to follow in places, but it was clearly supposed to reach a certain point and print a certain value, and yet somehow it didn’t.

We tried to make small alterations to the code, poking and prodding as one does, to try to pinpoint where our mental model of the program was off. Nothing seemed to help.

Now, there’s a notorious programming trap you can fall into, where you’re accidentally running a different version of the code than the one you’re editing. So you make all kinds of changes, but you keep running the old code, and none of your changes seem to have any effect. But we had already thought of that possibility, and eliminated it! We had tried to insert a dummy printout statement, and it was right there in the output, as expected.

The situation was getting increasingly absurd, and I was beginning to feel a little uncomfortable in my role as the supposed expert. But eventually, after much struggle and disbelief, we figured it out. The student was using a laptop. On this laptop, on this particular day, the ‘S’ key had begun to fail intermittently. So when the student pressed Control-S to save the source code, it would get saved roughly 50% of the time. Consequently, half of the time—completely unpredictably—we would compile and run the old code.

Moral of the story? Even a careful, determined, and systematic problem solver is powerless against a little bit of unexpected randomness. Design your puzzles wisely.

10 Likes

I almost left one of those in the final version of my game, too!

The way I’ve protected myself against it is, whenever I add a temporary thing, I put “DEBUG” in it, and then I have a to-do item “before publish, search for DEBUG” and fix them all. It was a narrow escape!

3 Likes