What are you looking for when betatesting a Game?

I have reached 100 games betatested in less than two years. There has been long and sort, amazing and simple, original and classic, polished and alpha ones. On the other hand there has been communicative, unaccesible, polite and aggresive authors.
This vivence has lend me to ask myself: What is the last reason am I testing games? So I summon you all to ANSWER the question and afterwards I will explain my own motivations.

  • Jade
13 Likes

I suppose I’m trying to be helpful to the author. Part of that’s finding bugs or infelicities. Part of it is similar to critiquing non-interactive fiction: I try to infer what’s the game the author intends this to be, and if I perceive a mismatch between their intent and execution, I explain what I thought it was trying to be and if I have anything in mind for how to bridge the gap, I’ll suggest it. And maybe I’m entirely wrong about what the author was trying to do, but it’s to be hoped it’ll still be useful to hear about how one player misunderstood things.

I don’t do a huge amount of beta testing, but all my experiences with authors have been good, including when they were strangers to me, so I can’t say it owes to uncanny judgement on my part…

12 Likes

I assumed the question means “what do authors look for” at first, but re-reading it, I think I misread.

So, as an author, I need a variety of things. I need someone who will hack through the game. And I also need someone who will poke at certain details I find hard to test. Tester styles vary and I don’t want people to restrict themselves!

I want them to do what they can, to set aside a block of time to poke at things they find most interesting, and let me know – and to understand that both subjective and objective pointers are good, and to understand sometimes my game just has environmental limitations.

So this sort of answers the question: this is the attitude I take as a tester. I try to give the same feedback I would whether or not I really like the author or what they’ve written. And before starting I nag authors to see if any specific area needs a look-over. It’s hard to ask for this–it feels like you’re taking orders. But I like knowing my time is put to a relatively good use.

I make it explicit that I work better with a walkthrough and am aggressive about disassembling a game so maybe I can say “hey, maybe this puzzle works better that way?” I think of the sort of errors I’d make and also try to poke at stuff that would seem tricky in Inform, maybe giving the author coding help if they ask for it. Through all the typos, etc., I point out, I want them to walk away with one “I didn’t consider that but it’s worth it” or “I didn’t think I was capable of that but maybe I am.”

If I’m working with someone whose work I’m familiar with, I also try to see how this work diverges from their preious ones or adds to it and try to suggest ways for them to keep it new.

But above all this I think I have two golden rules:

  1. Don’t be That Guy who says “THIS IS WRONG”
  2. try to locate what is most actionable in the least amount of time, and start with that. Or at least do so in the summary email. Recognize that the places I get stuck may not be where others get stuck and label those places to say “see what other testers say.” I am probably more active than most in encouraging authors to try something new technically, maybe something they don’t think they can do, or they see as harder than it maybe really is. They are writing games about possible new worlds. They should see these possibilities too. A lot of times I am passing on knowledge I learned from people who tested stuff for me.

After testing a while I have confidence that one-offs like typos can be easily fixed. If I get nothing more than typos, I say so … this doesn’t mean a candidate is bad or good, just, I struggled to add anything. I also like to hold a playing experience for 24 hours and follow up later. It’s a good feeling, to feel like I really thought about a creative work enough to say “Hey, it seemed to be vaguely missing something, maybe this is it?”

It’s hard for me to ask for betatesters, still – or perhaps I envision them saying “you’re experienced, you should know not to do this” or “hey, why can’t you test it, you’re more familiar with what’s going on?”

12 Likes

When I’ve been beta-testing I’ve found most authors are easy to get on with and receptive to advice and constructive criticism. I’ve never come across one who was aggressive, but I have tested for uncommunicative authors and also authors who’ve seemingly ignored all of the notes I’ve given them, including little things like typos. It’s disappointing and it’s left me feeling as though my time was wasted, but hey ho, it’s their game in the end.

As an author I really value beta-testers and I’m pretty meticulous about addressing every issue they bring up. It’s definitely made my games better as a result. Occasionally I disagree with a suggestion but I do consider them all very carefully. I like to have a mix of very proficient fellow authors and less experienced players recruited from my real-life friends and relatives. An inexperienced player will sometimes try things that a seasoned IF player would never think of, and some of them will be perfectly reasonable.

Sometimes I’ll offer to test a game that leaves me feeling way out of my depth. Linus Åkesson’s Pas De Deux was one such, and Hanon Ondricek’s The Cursèd Pickle of Shireton was another. I know nothing about music and I’ve never played a multiplayer online RPG, so my notes were probably a litany of exasperated floundering.

11 Likes

IME, often that very familiarity leaves you with enormous blindspots.

7 Likes

Nop, I am thinking about testers, their motivations and feelings more than their experiences or results, and never about authors at all. Explaining me: why do we test games, appart of being a service to the community and other obvious academic reasons? The answer will be different for each one, subjective and personal.

2 Likes

There are multiple reasons why I like playtesting games.

It gives me the chance to bring out a persona of arrogant brutishness. (“I shall eviscerate this thing you worked so hard on and see if you can re-assemble the pieces.”) This is mostly an act before I start the actual testing; my bark is worse than my bite. Still, I get a feeling of smugness when I manage to find and exploit a loophole the author missed.

I enjoy the realisation that I’m part of a select group of people who have the honour of participating in making the game as good as possible.

I love the avant-première mood of testing. Nobody has ever typed XYZZY or LICK ME in this game before! What will happen?

12 Likes

Ah–well selfishly it makes it easier for me to ask for testing help & less like “oh all I do is write games and expect people to play it.”

I also get a lot of ideas, or I say “hey, I think I could do that” or I can show off what I learned, not just to show off for a crowd but to know someone can put it to good use.

It also remembers me about the process that goes into a work. The finished project seems so polished – but no matter what someone’s experience level is, you get to explore something cool and new.

It’s also neat to see progress on something you tested 2 weeks before.

And it’s far more rewarding than playing some dumb game I played before, or visiting a website I visited before.

And oh yes I get a kick out of seeing my name in the CREDITS too :slight_smile:

9 Likes

The baseline is that I’ll look for bugs and spelling/punctuation/grammar errors, and I’ll note if I got stuck or confused anywhere. Beyond that, it depends on what the author has asked for. I’m happy to give feedback on prose, plot/structure, and characterization, but not everyone is looking for that. (Well, I might flag a sentence or two that strikes me as outstandingly clunky regardless, but I try to resist.)

I am admittedly less good at giving feedback on puzzle design—I can note what did or didn’t work for me as a player but I don’t always have great insight into what exactly could be improved and how.

If the game is for a comp and the deadline is close, I’ll also take into account what would actually be feasible to address—no one wants to hear “actually, I think your game has a fundamental structural flaw…” when it’s too late to do anything about it.

Edit: Ah, I didn’t see the comment further down clarifying that the question was supposed to be more like “why do you beta-test games?” So as for that, I guess a large part of it is that I want to be helpful and contribute to the community—as Andrew says, I rely on other people to do this for me, and it would feel weird to ask for that help if I was never willing to offer it in return.

I’m also an editor, and there’s something about helping a work become its most polished self that I find fundamentally satisfying. (It’s a competitive, poorly-paid, and often thankless job, so I feel like on some level most of us are doing it to satisfy our innate love of nitpicking. Which I suppose in my case is so great that I’ll do it for free in my spare time too! :stuck_out_tongue:)

12 Likes

Whenever I beta test, I always ask the author “what kind of feedback are you looking for?” Since I can focus in several directions.

Depending on where in the process they are, they may be working just on the gameplay and not on prose corrections, or they may have the game solid and want detailed feedback.

I kind of have three levels of playtesting:

  1. General impressions + transcript: Play the game, make a transcript, and give general impressions. This is where I play the game as a “normal” person. Sometimes it’s helpful to see how a new player interacts and plays your game with no preconceptions and much can be gleaned from a transcript like that. I then usually give general “here’s what I liked, here’s what I didn’t like, here’s what I understand about the story” feedback, only pointing out major problems that either crashed the game or some implementation that didn’t work and prevented progress.

  2. Play the game hard and try to break it. For this I’ll exploit my knowledge of how a parser system works and be the troll type of player typing every random command and synonym that comes to mind to see if the author thought of it for thoroughness. I’ll try commands like JUMP/SING/PRAY/DIE/DANCE. I’ll try to take my clothes off and see how much of the PC is implemented and if NPCs react to this. I’ll try to break puzzles and flush important items down a toilet or throw them down a well or off a cliff. I’ll try asking NPCs random questions that don’t relate to the plot or ask plot-important questions in odd ways. I’ll try to pick up and carry NPCs and kill them or throw them off the cliff or down the toilet. I’ve got a knife meant to cut this particular string, but can I also use the machete meant to kill the rabid gorilla? Can I CUT STRING/SLICE STRING/CLEAVE STRING/USE KNIFE ON STRING/SEVER STRING WITH KNIFE (usually unfair but sometimes this can be helpful). Can I stab myself? I’ll try searching for and examining every noun mentioned in a room description. This gets a thorough transcript as well, and I’ll call out egregious gaps in implementation and suggest fixes if I know how it might work better.

  3. Focus on prose/grammar/typos: For this I’ll try to focus on calling out misspellings and grammar in the test. Usually this is late-stage beta-testing for polish. I’ll also note missing descriptions for important objects or major plot holes, like an NPC letting me ask about something that hasn’t happened yet. I’ll also give style notes. Sometimes this might involve pulling sections of the transcript into a word processor and turning on revisions to show my suggested changes.

Sometimes an author will also ask for focused testing on one part of the game - like a complicated machine, or “just see if you can drive the car everywhere except where you’re not supposed to” that’s also helpful.

I find that if an author doesn’t know what they want or won’t tell me, I’ll kind of do the first one in general. Also if I’m not given direction on what to focus on, I’ll play the game to a transcript and stop as soon as I’m stuck or bored and then ask for direction what I should be doing.

12 Likes

This is a big reason why I enjoy testing games–I am also a nitpicker, and it’s very satisfying to be able to put that trait to use! Or, to put it another way: I enjoy playing IF games, but sometimes I notice errors/bugs in them. Experiencing the fun of playing a game while also having the chance to let the author know of any errors/issues I encounter, so that they can be fixed and the game can be improved, is my ideal. Same reason why I like beta reading traditional fiction for author friends. :smile:

6 Likes

When I am testing, I have two phases.

During the first phase, I am experiencing it only as a player. I provide commentary and thoughts as I experience things, and withhold suggestions. I play it as though it’s the final product and my input wouldn’t matter. I feel this is important because a game designer can actually get more useful data from the honest reactions of a player, and a dev might be able to find conclusions from my tests that I never would have realized I was feeling.

During the second phase, I look over my experiences, and write up a list of things that I would have appreciated and things I feel were done well, with the explicit disclaimer that nothing I say should be required, and should simply be taken as feedback to mull over. Just as in phase one, the things I say in phase two might reveal some underlying symptoms of the game that even I am not aware of, and the actual dev will be able to identify and address using their greater awareness of their own game.

EDIT: I know I stress the whole “the player doesn’t always know what they want” bit, which can seem like I have my head fitted so far somewhere that I can do a live reaction of my own stomach contents, but there’s a reason for it. The biggest example I can point to is the thompson being nerfed in a multiplayer shooter game, but there are many other dev stories out there. Game design is a separate art from writing the story, and there’s a reason for it. A lot of player enjoyment comes down to very subtle reactions and psychology, and game design theory aims to make sense of a lot of that, because it’s rarely obvious. For this reason, I try not to be too assertive in my feedback (though I have failed in this during one occasion before), because I need to recognize that, as a player, I do not have a complete understanding of what the game intends to do, and I am not fully aware of what systems are making me react in different ways. If given enough time, I might figure it out, but I try to deliver feedback within the same day I do testing.

6 Likes

I enjoy the feeling of editing something down and making it smoother and clearer than it was before. And I want people to test my games, so it’s only fair I test some other people’s in return!

8 Likes

After giving this some thought, I think my reasons for beta testing can be summarised as follows:

  1. I like helping people. It’s just in my nature.
  2. I’ve published quite a few games and I’ve asked people to test those games. It’s only fair that I return the favour.
  3. As we are a small community, we need all games to be as good as they possibly can be in order to draw others to our community and play those games. If I can help to improve a game, then I’m doing my bit to ensure that all games are as good as they possibly can be.
15 Likes

Interesting responses, but I have to admit my main drive is a bit more selfish; I just like to know/feel like I’m one of the first playing the game and solving its puzzles. The knowledge that there is no walkthrough available and that I’m totally on my own is exciting and motivates me. Sure, I guess the author is available and would probably give me hints, but in general I don’t want them, I prefer to approach the game as “naturally” as possible. I think this also helps the author to know what an “average” player would do in the course of a common gameplay.

6 Likes

An invitation to participate.

3 Likes

I missed the motivations bit, lol.

Mostly it comes down to wanting to be helpful/useful.

Sometimes I get this feeling that I’m a disconnected observer, like a long-dead ghost, and it makes me feel more alive when I can point at someone and say “That person’s project had a slightly different progress path because I existed!”

Even if I’m pointing at an otherwise-anonymous username on a computer screen. :grin:

8 Likes

Thanks a lot for your comments, now ahead my subjective motivations.
I have no objection when you talk about helping authors to polish and improve their games, but this is more a consequence than a motivation. Furthermore I can admit the reconforting chit chat with other people, the authors, about somethimg that both like, IF.
Even more I can agree with the idea of participate somehow in the development of a game when you lack the skills, ideas or the time needed to program a whole game by yourself.
But I agree with Joey. This feeling of ghost explorating an undiscovered new world, walking around in a land where there are no ways, no footprints, savage, virgin, unexplored can be the ultimate inner motivation.

Tello me more about this.

5 Likes

Here’s something that comes up for me … I always worry I’m repeating “that story” again. But in college, I remembered a creative writing course where we traded our works for critiquing. I remember taking it seriously and it seemed so much more fun than the scientific papers I heard I was supposed to consult on.

But I felt I really grew when I made comments, and I realized that other people had their own worlds to invite me into, and as @Bainespal said, that’s very important.

I also remember saying “gee, college is almost over, I wish I had a lot more of that! Where could I go and find and make more of that?”

I sort of found it other places on the Internet, writing reviews or guides for retro games. But I wanted even more. And I’ve found it, and it still feels worth it.

9 Likes

With IFComp coming up, I thought I’d bump this to add some general thoughts, which might not be concrete stuff to look for, but how to look for it. There also might be “what I don’t look for.” Fellow testers, steal/borrow/ignore as you please.

I realize I have one rule which is hard to quantify, but I try to keep it in mind: what will give the author the most benefit for the least effort on their part? This isn’t “help the lazy author out” but rather it’s an interesting challenge to focus on what I feel is the most important thing and potentially help the author not be blindsided with an “oh, if only I’d known.” I remember seeing this a lot in the IFComp author forums, where I show a line or two of code and the author realizes, oh, I can do a lot with that, I’ll want to for a post-comp release.

Of course you want to go through and mark off bugs you see, but you want to do and be more than just a pedantic clerk. (Some days if I don’t have a lot of energy or creativity this is a valuable mode to be in. But I want to take at least one shot doing more than that.)

I generally try to balance stuff I know (CREDITS, ABOUT, parser errors, can’t go that way errors, listing exits) and even contrarian-testing/trying to break stuff with exploration-testing. CREDITS/ABOUT is a great way to get warmed up–I suspect others have different ways to mentally warm up, and they might not seem like they work well, but they do!

I really do enjoy text extraction to see the insides of a story, or to read a chunk of text, or even to try to figure how to hit it, or even use them to see how to solve a puzzle and then evaluate whether I or the author missed something there.

I also sometimes change up what I need in the middle of a game. If something isn’t implemented, or there’s a nonfatal bug going through it, obviously I report that, but I try not to report bugs in the same category that seem to go through the same code path. I’m generally okay with one or two big things missing, with the faith that the author will fix them before release.

Obviously the closer something is to a deadline, the fewer detailed bugs I look for.

If there’s a week or so left, I look for a game-breaking bug, or if there is none, I look for something that could be fixed with a quick feature. Maybe disambiguation or better error messages with a parser. But I don’t throw too much at the author. There’ve been a few times where I say “Okay, I’d fix these two things, but I’d focus on number one for IFComp, because you want to test it and make sure it doesn’t break code elsewhere.”

I like to try to test something other testers haven’t looked at (so authors! Let testers know!) or something that is hard for a programmer to shift gears to test. From my own experience, it’s relatively easy to test the main line of my game. But I have trouble with alternate paths, paths that make sense but I can’t focus on.

So the #1 thing is, I generally look for stuff I’d bungle or avoid and then I test that pretty hard. In that vein I also ask, what part was tricky for you to program?

Again this isn’t covering for the author’s laziness–it’s just I recognize testing certain minute details would get in the way, and the author doesn’t have infinite time. It’s more useful and enlightening for someone who hasn’t seen the author’s world to poke at it.

I also like to ask the author several questions about what they feel confident works and what doesn’t. Sometimes knowing what is low value to test works well e.g. if someone has vetted the game text from the source code, I’ll worry more about how the narrative flows. If someone says someone else has tested hints, I’ll look for alternate ways through. If someone has hit the beginning or the end hard, then I’ll look for the middlegame.

(Side note: in one WIP this year in fact we had someone who extracted game text into four files: NPC dialogue, descriptions, etc. So one person could focus a bit extra on each. That worked really well.)

Sometimes I’ll just concentrate on a room that sounds cool. Or an incident I can relate to. For instance, if there is a sports or chess game as part of the narrative, I might see how authentic that feels.

But I think the most valuable things I enjoy looking for is stuff that takes time to sort out, maybe even replay.

When something doesn’t feel right, but it should. Perhaps there’s an inconsistency that I can’t explain. Or a puzzle is too easy or too hard. Or it seems like the game’s being fair to the player or giving enough hints, but I can’t put my finger on it. These are the most rewarding things for me to find and figure, and sometimes I need to play through twice. They also help me with my own game and puzzle design. I like going from “this feels like a weak spot but I don’t know why” to having something concrete to say.

Or if I’m given a walkthrough, I try a new path through the second time. It’s a way of showing myself I understand the world. There’s a balance here–spoiling everything, well, spoils things, but fighting without a walkthrough uses valuable time.

I think I understand things best when I read a walkthrough and play through again a few days later. I find it easier to evaluate 1) how well did the author clue this and 2) how rewarding is it to figure things out and 3) what tweaks could be made so that the puzzles work. And I enjoy those moments where I figure “Oh! That’s what the author means to do!” whether or not the disconnect was my fault or theirs. Obviously we’d rather have them before release than after release.

On the slightly greedy end of the spectrum, I look for two things.

  1. there have been rare times when a reviewer says “I really liked (neat feature X)!” when I, in fact, suggested X. Bragging publicly about this is wrong. And no, it hasn’t happened THAT often. And fixing a feature has nothing on creating the whole big story. But it feels good. And it gives me confidence that my testers will point out features worth implementing or tweaking. Sometimes I get a sense that the author has a good command of what they’re doing, and if they only knew this trick or method I bet they could do other things too … and I hit the mark enough that it’s worth sending them weird I6 code or whatever.
  2. I like really looking into a game to give myself ideas for my own. Obviously plagiarism is all kinds of bad, but a half-created game inspires me as full games can’t quite. I say, instead of “wow! How did they do that?” more “Wow, look what they’ve built and are building. Maybe I could build something similar, but make it my own. I’d have to change X and Y and Z, of course.” So I look for those moments, and they inspire me to look some more.

There’s a lot of golden rule/platinum rule stuff here. But I don’t think it’s too kum-ba-yah.

I think one thing to remember is that we’re not trying to be the best tester for the game, and there’s no reward for finding the most bugs, or the biggest bugs. I suppose this gets more into the general mindset for testing than what I am specifically looking for, but I like to say “Okay, what can we find today?” without being too free-form about it.

And another thing is, there have been entries where I’ve seen stuff that I know is tough to fix, and I genuinely enjoyed them despite that tweak not being in place. It is a relief to me–that I don’t have to do things perfectly to have something people will like, and they’ll accept if I miss something here or there. That’s not an excuse to deliberately miss stuff.

But it’s reduced my fear of making mistakes, whether in writing or playing games, or in real life. I can say “I can punt on X” or maybe even “I’ve been wasting time. It’s worth using my tieme better to focus on X.”

8 Likes