I realize this post may raise some people’s hackles. So before I describe what I’ve been doing, I need to insert a disclaimer.
I would never use artificial intelligence for creative work, for several reasons. First, creative work is fun! Why would I deprive myself of the fun? Second, I wouldn’t expect an AI to produce work that’s as good (or as deeply personal) as what I would produce myself. Third, AI makes mistakes, and they can be hard to catch. Fourth, large language models were trained on human writing, so there are ethical issues.
That said, there are certain tasks for which AI may be vastly more efficient than a human would be. Today I’m here to talk about one of those.
If you want to reply to this post by ranting about AI, go right ahead. I may respond by ranting about the internal combustion engine, which has had a greater environmental impact than AI ever will. But my goal here is not to stir up controversy! I just want to describe this week’s project and what I’ve learned from it, on the off-chance that a few game authors may find the information useful.
On with the show…
Back in 1999 I released my first text game, “Not Just an Ordinary Ballerina.” It was a large, complicated, and rather cruel puzzle-fest, written in Inform 6, which at the time was state-of-the-art.
At some point while switching to a new computer, I lost the source code files for Ballerina, so there’s my first tip for y’all: When switching to a new computer, make sure you keep the backup copies of creative work! Today, Ballerina exists only as the compiled ballerina102.z8 file. I know of two bugs in that version, and those bugs will never be fixed. Also, this morning Claude spotted a third bug in the original that nobody ever reported. There’s a location where LISTEN produces an endless loop.
For a few years, a guy who went by the name Radical Al (that’s short for Albert, or maybe Alvin) was one of my beta-testers. At some point Al thought it might be interesting to port Ballerina over to the new Inform 7 language. He never got very far with the project, for obvious reasons. Testing a game of any size to learn about all of its possible permutations would take months, if not years.
This week, however, it occurred to me to wonder if perhaps an AI could decompile the .z8 file for Ballerina and reassemble it as an Inform 7 game. This would not be creative work, and also it’s my own game, so I can do with it whatever I like. I asked Anthropic Claude to give it a go.
I have now learned quite a lot about the pros and cons of this attempt. If you’d like to try doing something similar with an abandoned game, what I learned may help streamline your process. Heck, you can even try it with Ballerina if you like! Maybe use GPT instead of Claude. We could compare the results.
In no particular order, here are the main things I learned.
Claude was able to get the I7 Standard Rules and compiler from Graham Nelson’s github repository. Using these tools, it was readily able to write I7 code that would compile successfully. However, it never consulted any tutorials on how to write games using I7. As a result, its code bears very little resemblance to the usual style of Inform source text. In retrospect, I should definitely have included explicit instructions about this in my prompts. I have had to hand-hold Claude to make the source text more human-readable. Nor did it take advantage of any of the I7 extensions that might have streamlined its code.
Claude’s code style is strongly influenced by traditional object-oriented coding. For instance, in one of its first attempts it included a Chapter of declarations of all of the in-game things. I had to explain to it that the Inform compiler handles this chore automatically. Explicit declarations are rarely needed. It created a wilderness of “Instead of doing something” rules, where more explicit rules such as “Instead of taking the statue” would be more efficient. I had to instruct it about how to produce blocked-exit messages for whole classes of rooms rather than iterate those messages explicitly for each room (such as the rooms in the mirror maze).
Its code declared all of the properties (both true-false properties and numerical properties) for every object in the game rather than restricting properties to kinds where they would be useful, such as the property of-color for the six balloon objects. There’s also a mess of numbered global variables, such as global-14. As we’re finishing the project, I’m going to tell it to rename all of those globals based on what they’re used for.
The only way to find issues of this sort is to read through Claude’s source code. This is not easy, both because the file is very large and because the connections among blocks of code are not organized in the way a human author would organize them.
Using the walkthrough built into the original .z8 file, Claude has been able to test whether its output for the I7 rewrite is that same as the output of the original game. However, while doing this it neglected to try most of the wrong inputs that a human beta-tester would naturally try, either maliciously or through simple confusion or inadvertence. I had to point out this omission in its internal test routine. It is now testing wrong command inputs, but whether this process will iron out all of the remaining bugs is unknown. (See the footnote at the end of this post.) I certainly wouldn’t ask human testers to work their way through a very large and very old game, so it would be fair to say that the finished I7 version will be untested, even when I release it. Given how inhuman Claude’s source text is, it’s not entirely clear that I would be able to fix a bug even if a human player reports one.
Some of the default library outputs in I6 are not the same as the default outputs in I7. I should have told Claude to list the differences for me and let me choose whether to revert to I6 style or keep the I7 outputs. In general, the I7 outputs are a bit nicer, so the AI was using extra effort to make the game slightly more old-school in places where that wasn’t necessary.
In retrospect, I should have asked Claude to produce new source code for the game in Inform 6 rather than Inform 7. This would surely have been faster and less error-prone. Speaking of “faster,” I did spend $100 for more usage bandwidth, in order to complete the project within a few days. It would have dragged on for a while if I hadn’t done that, because this is a pretty intense project for an AI to undertake.
Nonetheless, I feel the project has been worthwhile. If nothing else, the next time I write a game I will consider using an AI as a first beta-tester. And I will write better prompts. If I give it specific instructions about trying to break things by using absurd inputs, quite possibly bugs will be spotted that human testers might miss. I worry that with a large game some human testers may burn out before reaching the latter part of the game. An AI won’t burn out.
Footnote: As Claude tries to tidy up its work by doing a final run-through, it finds new bugs. Specifically, it reports, “In one run, moving southwest out of the Travel Agency sometimes lands in a different room than the original’s walkthrough.” Since that door is not randomized, a bug of this sort could almost never occur in a human-coded game. Claude is also finding that in some run-throughs the ending is never triggered because certain items are disappearing from the player’s inventory. Again, this is something that would be very unlikely to occur in a human author’s game, because there’s no reason why inventory movement would be randomized or inconsistent. I’m suspecting that Claude’s approach to coding is just not reliable. (It’s an AI. It makes mistakes.)
Questions and comments are welcome. As I said, I’m not trying to stir up controversy. I’m not going to get into any back-and-forth arguments about AI. I’m not going to try to defend my usage. If you’d like, I’ll defend my continued use of a gasoline-powered automobile. That would be just as productive a discussion.
