Finding Reliable Beta Testers

I wonder if any IF authors have suffered the same experiences as I (and my fellow ADRIFT author Finn Rosenloev) have in my efforts to find beta-testers?

As well as a request on this forum, I have posted a call on the game-testing.org website requesting beta testers for my ADRIFT game “The Fortress of Fear”. I have had more than a dozen replies to this call, I receive an email saying “Dear Larry Horsfield, I am interested in playtesting your game. Best regards, etc.”

I reply immediately to the email address given for this person, thanking them for offering to playtest my game and I ask them a couple of questions: “Do you have ADRIFT installed on your PC” (I ask this as if they do, I can send them the game as a blorb) and “Have you done any playtesting before?”. I then send the email.

What has happened for the last 7 or 8 occurances of the above (I do have two playtesters who replied to the call who have been testing the game for some time, btw) is that I hear absolutely nothing more from them whatsoever, they don’t even reply to say “Sorry, I don’t think this game is for me.” or “I don’t have time to playtest” or whatever. My first thought is whether I have offended them somehow in the questions that I have asked? I showed the email I sent them to my wife (a schoolteacher) and she said that she would not be offended by the email.

My next thought is why on earth do these people bother to answer the call and then do not contact me ever again? Why has it happened time and time again, it doesn’t make any sense to me. Are they just messing me about for the sake of it? Are they genuine playtesters or just trolls? As I mentioned above, this has also happened to a fellow ADRIFT author and he has sent his game to several “testers” and never heard another word from them.

A fellow drifter on the ADRIFT forum has argued that anyone replying to my call is not obliged to reply or do any playtesting even if I send my game to them, as I am not paying for their services. However, I would have thought that common curtesy comes into play here, surely it is not too much effort to send a quick email explaining why?

I would be very interested to learn if any other IF authors have had similar experiences? Are there any playtesters who would like to comment on this?

Thanks for reading this.

Well, hmm.

First of all, as you’re probably aware, testing can feel like a big commitment – particularly with a rather large, old-school game.

So, there’s always Life Happened, which comes up more often than you’d expect and could just be random bad luck. Likewise, you could have hit on people who haven’t done any testing before, try it out, and decide they don’t like it.

Testing can suck if you don’t develop a good working relationship with the author; I’ve faded from a few testing projects because I didn’t feel valued or listened-to or comfortable, or I didn’t trust the author to actually make the necessary improvements. (And if I’m close friends with someone ) But it seems as if you’re losing them too early in the process for this to be the reason.

I’ve also faded from testing projects when the author’s been great to work with, but the game itself has been really tough going – not necessarily terrible, but weary-making, difficult, not as rewarding as it could be. I suspect that this is a fairly common tendency among testers – games that are already pretty good will have an easier time retaining testers and getting good work out of them. So if you’re getting a lot of betatesters dropping out immediately, that might be a sign that your game doesn’t have a strong enough opening. (People fire up the game, and play for fifteen minutes; fifteen minutes doesn’t really seem like enough to produce a play report, but it’s more than enough to guess that playing on is going to be unpleasantly hard work. If you don’t really know the author yet, it can be hard to find a diplomatic way to say ‘your game is boring me to tears’, so it’s easier to say nothing.)

And the way it usually goes when I fade from a project: I always mean to get back to working on it. It’s just never a good time, but I feel like I really should work on it sometime, and then I realise that it’s kind of too late to politely tender my regrets. (Yeah, this is lame. But IF people are geeks, please remember. Our communication skills may not always be the finest.)

Hi maga, thanks for replying. What gets me is that they haven’t done any testing, I haven’t even sent my game to them, they simply don’t reply after I write back the first time. I was a bit annoyed after the first couple of times, but when it has happened several times, one after another, I start to wonder what the hell is going on here?!

Okay, I’ve got no idea about that. I suppose that your order of doing things is a bit more fiddly than normal; generally if someone agrees to test and you’ve got a testable version, the next step is to bung them the game file and whatever instructions the need; if you think that they don’t know what an interpreter is or will otherwise need help to get the game running, you include that information when you send them the game.

But that seems like a detail, really; shouldn’t be enough to explain the whole thing.

Another thing you could try is approaching specific people and asking them to test or at least give you an opinion, rather than putting out a general call. In my experience, people directly approached that way are more likely to respond in some fashion – they may say they don’t have time, but if they do commit, they’re more likely to follow through.

How do you mean, “specific people”? Where do I find these specific people? Do playtesters ever post messages offering their services?

By “specific people” I’m pretty sure she just means specific people, like a friend of yours, or a dude you know from another community who’s interested in IF, or someone who has a reputation for being a good beta-tester, etc., etc. Try to think of people you know that you’d think would be interested in your game and contact them. I’ve had good results with this in the past.

My best guess is that your second question, while not being technically offensive at all, might have had an unexpected intimidating side-effect. People might be understanding that they’re expected to be experienced testers or that this is an imperative requisite. If that’s not their case, they could feel that their feedback would be less than wellcome and might chose to silently step back.

If you’re actually targeting testers with a high level of expertise it would be wise to state it clearly in the first announcement. If not, I would suggest adding a few lines after the question explaining that you will be happy to give some guidance to newbie testers on what kind of feedback you expect. That could help in lowering your exagerated early-dropping rate.

Anyway, this is just my guess. You might be right and they might be just trolls, after all. :slight_smile:

I’ve had your experience in the right opposite proportion. My testing request received four answers (which was four times more than I expected, being a total stranger and newbie here). Only one of them disapeared without saying a word, while the support and encouragement I received from the other three (together with useful, clever feddback, opinions, bug reports, and corrections) was so awesome I could almost describe it as one of the more rewarding experiences I’ve ever had in my whole internet time (yup, my little wip went into limbo for some star conjunction of sad and wrong reasons, but it will see the light one way or another, as it would be a crime if their time was spent in a never-released thing! )

Very occasionally play-testers do post advertising their services, though it’s not super common.

But yeah, I meant “pick out some people you know and email them.” That might be friends of yours, it might be people you’ve talked to on this forum, people who have reviewed your work in the past, people for whom you have yourself beta-tested previously, whatever.

Once you’ve written and tested a couple of games, you may find that you have a few repeat testers who are an especially good fit for you: they like to play the kinds of games you like to write, or they’re extra-thorough thinking of ways to break your simulation code, or they give thoughtful commentary on your literary themes, or whatever it is you’re looking for in testers. Those people are as gold, and you want to do everything you can to make testing for you enjoyable and rewarding so that they’ll keep saying yes when you ask.

The game being in ADRIFT is an additional hurdle for testers to overcome, since Inform is more popular–still it’s not fatal. I’d think there is a simple illustrated install guide as well with how to take transcripts, etc.–if not, there should be. I know that I had trouble getting ADRIFT running and wasn’t used to its quirks when it did start running. The error messages were different, etc.

It’s difficult to know what to say when asking for testing. Maybe a few sentences like “This is what the game is about” or “this is what you might not like about the game” beyond the initial description.

Also stuff like “It’s ok if you don’t have the time or don’t like it” is good, as well as “Just let me know what parts flat out aren’t fun.” I don’t know what’s in your letter and it’s not really any of my business, but a small thing like that can go a long way.

But from your signature, I suspect one problem may be that there is too much to do (275 rooms, 30 characters as in your sig is a lot–as is a potential 4-part game. This may make people wonder if they are obliged to, say, work on part 2-4 after part 1) & people don’t know where to start & may get swamped by that many rooms–or feel silly they only got to a few of them! That on top of installing a new app can be tricky.

Now I’ve had good transcription logs and discussions after telling testers (granted, people I already know) something like: Beyond Room 3 is off limits unless you’re really having fun. Just try to get to room 3 for starters and let me know. We can do the rest later. Stop when stuck for 10 minutes because that’s valid data on the gameplay, too.

Or: Grammar bugs are good if you see them, but the narrative is still not set. So focus on what’s fun or confusing in the big picture and don’t worry if you have a lot to say that’s subjective. And don’t feel guilty hammering on a theme. If I’d have seen it, I’d have fixed it.

Or, and this is as noncommittal as their offer to you: I would be glad to return the testing favor if I have time.

Maybe there should be a general suggested boilerplate for authors seeking testers to use? And they could tweak it as needed? I mean, I’ve seen basic outlines, and you don’t want to sound like a form letter, but…just having it there would help an author know what to expect and what to ask.

Because (and this is a bit off topic) I have to admit, for my first game, I was clueless too. I found cold-calling for testers a little scary, and I feel very fortunate some nice people answered my late beta call before IFComp 2011, but that was probably more generically helping someone in need out.

ETA: misread your sig.

I am actually an experienced IF author who wrote several “text adventures” (as we knew them back then) back in the 1980’s and early 90’s. I started on the Acorn Electron using “The Quill” then graduated to a Spectrum using “P.A.W.” The arrival of the 16 bit computers and graphic adventures (plus raising a family) put a stop to my authoring, but I restarted a few years ago. “Fortress of Fear” is a game I started on the Spectrum using PAW, but never finished.

In my “heyday” I had a team of 5 playtesters (4 women and one man) who playtested every one of my games to death. Sadly, none of these people play IF any more, I did ask and one of them said she is too addicted to games on Facebook to play IF. How sad!

One more suggestion, as though a lil bit off-topic. Don’t have you RL friends betatest your work. It would be like asking mom if she did like your drawing.

1 Like

It can also be useful to email authors (particularly authors who have written things similar to what you’re doing) and ask them to recommend testers who might be appropriate to your game.

Hello,

I’ve had the opposite problem: I’ve tested some games, sent in transcripts with comments, and never heard back from the author. I’m guessing my tests were viewed as a bit harsh, but the games were really not very good, and I was only trying to improve them. Even just a “thanks” from the author would have been nice.

Neil

On behalf of authors everywhere, allow me to extend my deepest appreciation and thanks for testing, transcripting, and commenting (although based on your self-evaluation there, you might sit down with an objective third party and a couple of your reports and see if there are things you might be able to present in a more effective fashion).

Cool! Sorry about misreading your level of experience. But yeah, it’s possible to assemble a kind of amateur play-testing team even in the current community, especially if you make a few friends on this message board or through ifMUD, in-person IF meetups, etc.

I started reading this topic and was thinking, ‘Gosh, I can’t wait to start typing, I’m full of useful ideas!’

Then I got to the end of the topic and found everyone else had already dispensed my ideas.

Except perhaps for one. I’m aware that every time I see a cold call on the ‘testers wanted’ board (and I’ve answered 2-3 over time), a thought pops in my head where I’m willing the asker to directly email the author of some game(s) they like, to ask them if they’d like to test.

When you look in the credits of most recent games from authors who have made more than one game, you’ll see they’re often crammed with the names of other current authors as testers. There’s lots of reciprocation. If someone makes a game I like, I often feel compelled to talk to them in general, or inclined to be open to new projects they’re running or to offer to help with those.

If you’ve played their game to completion and you like it, and you tell them, you are giving them feedback (and/or adulation). And there were obviously qualities in the game you responded to personally in a way that might translate into the author being a good fit to test your project. You know, maybe there’s a way they did things on theirs that consciously or unconsciously you want to emulate, or are already moving towards. Obviously this is project-dependant, but I find it’s a good intuitive approach to follow in most kinds of art-making.

I’ve had people email me about a game I made and then I helped test theirs, or I’ve reviewed or commented on someone’s game and later they came to ask me for help.

I guess my suggestion is basically another extension of involvement. But it comes as a natural extension of playing some current games. In a nutshell, when you hit some you like, contact their authors to see if they might be able to help on yours, or if they know people who could.

  • Wade

Not even getting a thanks back is rough. Yeah. I’d like to think most authors can and should do that. I sometimes just give a generic thanks if I had less time than I thought I had and usually try to point out something immediately fixed with the transcript I got. Because there usually is something. Then, when I have time to look at the transcript fully, I send a follow-up mail.

It’s difficult to balance between harsh and between, well, showing the bugs, and how to do so without sounding like you’re dishing total BS. It’s also difficult to know that your tester may be trying to find that balance, too, because your own game Needs Improvement! So there needs to be some trust out there.

And the author is taking a very real risk sending things out, and a beta can be assumed to have a few things the author just didn’t see and needs someone else to help cover his blind spots with. Sometimes it may be a matter of saying “focus more on X than Y.” Maybe a puzzle is too hard, or another is too straightforward.

But I think the sort of thing that I get the most mileage from on either side is a “Did you mean this instead…” or even a mention that the author forgot to write in a detail that he probably knew. It’s like the old overused but still effective “How do you mean?” in conversation. I think a tester is there to ask questions, since a game is an open-ended thing and not necessarily (well, hopefully not) a technical manual or a mathematical proof. Asking these open-ended questions specific to the game may tip off the author to something he forgot or assumed he put in or was hoping to imagine but didn’t quite do himself.

Questions also naturally predispose people to answer them back, as opposed to pointing out what needs to be fixed, which is important and good. They can also help an author maybe talk himself out of somewhere where he may be stuck.

This post diverges from the main topic but it’s important to me on the testing and programming side. Because I think genre, size, etc., are useful information to have when saying you have a game to beta test, but even more useful is – what do I want the tester to try to break? What parts of the game do I need him to try? What would take too much time in programming-testing? What do I worry/know is rickety? What keeps getting broken? What do I need someone to focus on if he doesn’t have much time?

All this may not be in the initial call for help, but it’s useful to send to the person who’s offered help so they 1) know where to start and 2) maybe have somewhere fun to poke around if they get tired with the puzzles.

In defense of deadbeat testers… on more than one occasion I’ve signed on to test something and needed to wait a week before getting a good chunk of uninterrupted time to log a good session with it. Often I soon discover that there’s more going on than initially indicated and that more time will be needed for a deeper investigation… then while waiting a week for the next window, the game is released, me with a partial transcript full of unshared notes to the author. Oh well!

Well, the author probably should follow up after a week making sure everything is okay. And it’s also okay to send a partial transcript. I know that the authors often find stuff you won’t see, because it is their game and their world.

I know a lot of things I see in transcripts that I know I need to fix go uncommented. Usually that’s because players did something in a different order than I intended. I would find it perfectly acceptable to get an email from a tester saying “This is all I can do. Sorry I can’t comment.” Obviously the ideal sort of feedback is someone addressing the big points in an email and mentioning small stuff typos in the transcript. But even seeing what people may try is a boon to an author.

Also, while it can seem too late once a game is released, the author won’t turn on you if you still send the transcript. A lot of us fix games up once we see the first bugs & it’s possible some are still there–though it’s a bit of a courtesy to see if the stuff you broke, still is.