Testers for Large Game [hypothetical discussion]

Playing Beta Testing Requests and Discussion
21

There are two types of testing/feedback types that I’ve found useful to request:

  • Testing from people who write games and know the system (Inform or Twine, or whatever). If you can catch them when they have time to do extensive multiple playthroughs, and if they have insider knowledge they can often make valid suggestions how to make the game better and re-test updates based on their feedback. These relationships are sometimes hard to establish because this level of testing requires a lot of time and investment. Often these are the testing situations where you need to reciprocate testing for your tester or have done a favor for them that they can pay you back for. This is where being well-acquainted with your tester and establishing and maintaining an authoring/testing relationship comes in handy. This often works best if your tester is entering the same Comp as you are and you both are invested in mutual testing.
  • Testing from people without a major commitment, essentially asking them to “Play the game once and transcript it (or screenshot errors/give feedback for non-parser).” This is valuable from less-experienced play testers who may not write games or know the system, but can be useful to see how the general public bounces off your game or not. This feedback can be easier to attain, especially if you make it clear that you just want one play through and they aren’t going to be bashing their head against your game for an extended period. Downside is you can burn through one-time testers quickly, and often this means sacrificing a potential official score/vote/review for your game when they’ve tested the game prior to release. But if you’re good at interpreting nonspecific feedback and can make the game better based on that, it is very helpful.
1 Like
22

Aaargh!!! My eyes!

_>UNSEE! UNSEE!

Best part of parser-adventuring is mapping.

Well, not the best, but a significant portion of the fun I have with big parser games comes from mapping complicated settings.

2 Likes
23

Well, that’s just the main level. The upper and lower levels are not shown. This is an interesting point, though. My thought was that most players probably would find the mapping of such a large game layout a chore. I’m going to figure out a way to bundle the maps with the released game, but nobody will have to look at them if they don’t want to.

Also, as it says in the legend, the map is NOT guaranteed to be correct or complete, heh-heh-heh. If you rely on the map, you may miss a few things.

1 Like
24

Jumps up, looks at the ground, swivels around peering behind and under his legs…

-Huh! Did you see that? Something stepped on my toes. I don’t know what it was, but it has me feeling strangely compelled me to type this:

“I fall between the two types. I lack technical knowledge, but I sure as hell have been known to go deeper than one transcripted playthrough.”

-Toes all better now. Sorry for the disturbance. Carry on.

25

Is it possible to make a bundle of game state such that you can pass it to a tester and say, “Please explore this scenario for no more than two hours, then spend no more than one hour further to capture your impressions, disappointments and outright bugs you discovered.”

Do IF testers share game saves in this way?

26

You could do a “not for release” command “Test ruins with …” and GONEAR and PURLOIN commands, but it will only work in a test build.

1 Like
27

Can you spoiler the map? Click Edit, select the map, click the cogwheel icon and choose Blur Spoiler from the drop-down menu.

Like @rovarsson, I really enjoy the mapping, especially on big games, and I don’t want to see the author’s map until I’ve finished. Even then, it’s just a comparison to make sure I haven’t missed anything.

28

These are really useful! As is ABSTRACT ITEM TO ROOM.

There’s also no reason you can’t include this sort of functionality in a Beta build. (Standard “remove this code before comp deadline” disclaimers apply.")

However, one thing Inform doesn’t include that I like is BANISH X. Simply put, this runs the following code:

carry out banishing: now the noun is off-stage;

One other possibility, which I’ve used sparingly but could be a custom warp function, is finicky as it passes by BEFORE and AFTER rules.

It is to have a blow-by-blow list of rules that would get the player through the game.

this is the first-point rule:
  <do stuff to get the first point>

this is the second-point rule:
  <do stuff to get the second point>

let walkthrough-list be a list of rules variable. walkthrough-list is { first-point rule, second-point rule, ... }

understand "betajump [number]" as betajumping.

carry out betajumping:
    let max-points be number understood;
    let count be 0;
    while count < number of entries in walkthrough-list:
        increment count;
        process entry count in walkthough-list;
        if count is max-points, break;

This obviously focuses on a specific walkthrough to get through the game, but it has the advantage of being fast for you (nothing to type in), customizable (the player can decide how far to warp) and available in release mode, at least for the beta. The disadvantages are, it’s that much more fiddling for the programmer to make sure it works, and it breaks if the walkthrough breaks.

I think having this sort of convenience for your testers is really useful–even if they don’t use them, they know they’re there, and you’ve put thought into making things easier for them, and they can pick up where they left off. Especially important if you need to junk your old build, since save states aren’t expected to transfer to the new binary.

ETA: the above is pseudocode. It won’t compile, and I probably got some of the details of Inform wrong (I always have to look up the precise syntax for lists) but I hope it’s pretty clear what my brute-force approach is.

Naturally, it’s not so hot with a more exploratory game, though if you want to look for optional points you can have

repeat through table of walkthrough rules:
    unless LLP-flag is true or is-LLP entry is false, next;
    increment count;
    if count > number understood, break;

table of walkthrough rules
rule-to-try    is-llp
<first point rule>    false
<second point rule>    false
<first LLP rule>    true
<third point rule>    false

etc. … to pick and choose LLPs, you could probably use more advanced numbers (e.g. for LLPs 1, 3 and 6, you’d say set-llp 37(=2^0+2^2+2^5 … you could extract the binary number 100101 and see which LLPs to hit) and then betajump (however far). But that arithmetic is nontrivial, and I might be detail-mongering too much now.

2 Likes
29

Done. Good tip – thanks.

2 Likes
30

I am alright with the exposed above.

I could encourage to implement a command to enter comments as explained above.
I ussually play any test game as explained above. First gameplay as a normal player and one second as a weird one trying to crash the game. This implies random and crazy commands and some others badass: close a closed door, eat some necessary objet, drop all to reach a dead end, and so.
Afterall I try to contextualize and overview the game to improve it.

Sometimes I feel that the game argument is weak, I can’t do anything at this respect. The game and the idea is always propierty of his author.

I dislike authors that doesn’t answer my request about HIS game. As a matter of fact almost 50% of the authors didin’t follow a test.

Another appreciation is the number of testers listed. To test a game in its final steps you need 2-4 players, never 9. Then, who are going to judge and vote it?

  • Jade.
31

Jim, is there any measurable variable to give us an objective sense of how “huge” the game is? Number of actions from start to finish? Number of locations? Items?

Also, some thoughts on my part:
Splitting the game into multiple sections to assign to testers is unfortunately misleading. Beta testers usually want to get to know the whole thing anyway and not play a contextless section. In the end, free beta testers don’t see themselves as service providers, but they want to have fun in their own way.

I find the idea with the “super testers” promising. The problem is that you will probably scare away even the most motivated people if you can’t reuse savegames or if the story/puzzle sets change during the test. Therefore my personal approach would be: As long as possible, only test yourself and not immediately supply all those willing to play at once.

1 Like
32

Ah! I forgot to talk about the lenght of a game.
I try to measure the lenght of a game when I test a game. This is not an easy task.

There are some games that I have played where I had needed more than 12 hours to finish. Then I have wrote down a walkthrough that needs less than 5 minutes to complete the game.
For example the Anillo trilogy by Johan Paz. More than 12 hours each and less than 5 minutes walkthroughs.

  • Jade.
33

This presupposes that all games are made for competitions.

34

I think, play time doesn’t work well as a criteria. E.g. a game with 1 puzzle that is so stupid that you need 100 years to solve it, is not a “huge” game. Quantify objects, locs, characters works better, but still these numbers give not more than a rough idea. To be honest: I don’t expect more than that. :slight_smile:

1 Like
35

Here is my new game:

“Solve the Riemann hypothesis. Type DONE when you are finished. Then you’ve won!”

6 Likes
36

Too Leonine Nemean Riemann game, i’d say… ¬.¬

1 Like
37

>SOLVE RIEMANN HYPOTHESIS

You’ve solved the Reimann Hypothesis! Congratulations!
>DONE

that was easy! where’s my prize?

2 Likes
38

There are about 100 rooms and 70 or 75 distinct puzzles. Some of the puzzles are multi-part, some require logic and close observation, and some are simply find X/use X. Also, there are at least 8 NPCs, some of whom are very chatty – that is, you can ask or tell them about all sorts of in-game topics, and a few of their responses may give you in-game clues.

The inability to reload a saved game when you receive a new beta build is inherent in the software – it’s not something that any author can do anything about. Replaying a saved script is possible in TADS, but I’ll have to research whether Inform has an equivalent outside of the IDE.

I would hope never to change the story or the design of a puzzle (unless the puzzle is a complete disaster). At the point where a tester sees the game, the design should be completely locked down.

39

This is a good measure. 100 rooms is a good size. A puzzle density of 0.5 (i.e. an average of 1 puzzle for every two rooms) is a good baseline to aim for. Your puzzle density is 0.75, so there’s lots to do. Obviously, some puzzles will be easy and some will be hard and what constitutes a puzzle is subjective. Nevertheless, this is sounding like a very approachable game. Without seeing it, I reckon playing time would be at least 4 hours, so testing time would be something like 8 to 40 hours, depending on difficulty and level of detail.

40

The trilogy Anillo I mentioned above is based in living NPCs. You have to interact with them discovering their habits and routines. You are a ring that can be carried sometimes (or not) for some of those NPCs. This way you have to play untill you reach your destiny.
There are about ten human and animal NPCs. Some NPCs you can interact with or even command them.

The part about exploring the map and the NPCs routines are large, the map and the inventory are both rather short. My consideration is about the big amount of time you can waste exploring compared with the short time needed to finish each ending by playing the straight way.
I think this leads to rank these games with 5+ stars and I mean these games worths a try and should be studied by many authors as a reference.

-Jade.

1 Like