Beta testing is a social interaction, so has all the potential caveats. You want to trust whom you’re interacting with, but that shouldn’t stop you from reaching out to new players.
The more you do it, the more you learn whom and what to trust, and what you should take with a grain of salt. In general you don’t want feedback from just one person if possible and should not lay all your trust in any one person’s specific impression. The rule is if you get a specific subjective (non typo or bug) note from one person, that is an opinion worth considering. If you get that same note from multiple testers it probably is something important you may want to look at and consider changing.
The only major “strategy” I’ve employed against the “have as many people test as you can” rule specifically for IFComp is I try to only have non-voting testers. If I know Mathbrush isn’t entering the Comp, as valuable as his specific testing and feedback is, I would prefer his public review and vote on my game instead of knocking it out by having him test it. For newer entrants, this probably isn’t as easy and you should get as much feedback before release as you can.
Yes, it’s important to have good defenses without being defensive. I practice ambiguous stock quotes like the classic “I will give their comments the attention they deserve,” which works nicely for constructive or unconstructive criticism, or if someone gives you a combination.
90+% of people play by the rules. A small percentage cheat, and sometimes they get away with it because, as you said, people say “that’d be a weird way to cheat!” But of course cheaters rely on blind spots like these.
More on the original topic, as Robin mentioned, I’ve enjoyed trading testing with authors who finished in the general range I did. If they placed above me, okay, I wished I placed there. I know in the authors’ forums there’s often a bug sharing thread to make quick updates, and I’ve gotten PMs as well mid-comp which helped with immediate fixes or a post-comp release.
That was an old rule that change a few years back. However, the version you publish on Oct 1 will remain available the whole comp in the “big download” of all games, and some judges will only play that version.
Yes, though to check, there’s still a soft rule that it’s only for bug fixes and not adding new content. New content or features can be put into a post-comp release, which can get a bit of publicity.
It’s hard to enforce, because, for instance, is adding “H” and “HINT” as synonyms for “HELP” a feature? It’s not technically a bug. Should the author really have to wait 1 month+ for such a small change?
But it makes sense. Too many changes are likely to cause regressions, and besides, it’s fun to look at other authors’ games during the comp. But they want you to be able to fix “oh, I meant that” mistakes … or game breaking bugs your testers didn’t quite find. Or game breaking bugs introduced when you fixed your testers’ last few transcripts, because THAT couldn’t break…
Judges are not obligated to play the updated version, and in fact, some play the original. I remember some judges not being happy with the new rule, but nobody seemed to sabotage updated entries. I think the first year things got updates was 2011. Then, you had to email the organizer, and there were about 4 updates all told. Now, it’s a bit more automated.
Honestly, the only thing that’s ever galled me is when I’ve tested someone’s game before the comp, then I’ve seen it in the comp and discovered they didn’t fix 90% of the stuff I pointed out.
I’m not talking even semi-subjective stuff. I’m talking stuff like "If player takes action A, they will never be able to drop item B!’ or “Every third word in this paragraph / this game is spelled wrong!”
That shows me I wasted my time, so then I just don’t test for that person in future. (Assuming they had one.)
So I waana do some appointment.
A kind of lissting about the last 15 english betatests (last year).
3 games published without advertishing out of comp
5 without an email or pm after sending several trascripts.
5 published correctly.
2 betatest in progress.
Between all that betatests I have enjoyed with 4 of them just becouse there was a good communication with author.
I am writting down these 15 names for the record. You can suppose that there are 2 lists.
And finally I want to declare that I enjoy more betatesting than judging/ playing a game alone.
Conversely, what’s frustrating for me as a programmer is when I let one of these slip through, or a regression causes it to break. I wonder how to explain it to the tester. So if 10% of that sort of things slips through, you shouldn’t feel too bad. (Wade’s been patient with me on this before, with a bug I thought I fixed.)
I like to try to give testers a list of stuff a transcript fixed, to show I was paying attention, and I try to tell them hey, here’s something you didn’t know you uncovered–thanks for paying attention. It feels artificial at first but I think people like to know someone is listening in any context.
Yes, there’s a certain extra game angle there, isn’t there? You have to look for what’s wrong. You’re not fully told what to do. You’re reminded other people make errors, and you’re reminded you can notice them. This is particularly interesting for me if I get something in a genre I don’t like, but the author has really good in-game hinting, or provides a walkthrough. Or if they say they’re not technically inclined, so I offer source suggestions, and they take a few.
And also you are freer to concentrate on a part of the work you find more interesting instead of feeling you have to get through it. That focus helps the programmer as well …having everyone lawnmower through doesn’t give full coverage.
As a tester it’s also valuable to me to say: okay, here are the big things to fix before the upcoming comp deadline. And here are the things you can fix if you have time. It reminds me I’m not hopeless at setting my own priorities.
It’s a good feeling when you got a lot more out of an experience than you feel you should’ve. It more than makes up for the times you felt you should’ve enjoyed yourself but didn’t.