I just opened up a spreadsheet with the intent to keep track of the stories I read before competition end, and to find a way to normalize my own rankings. My thinking is I’ll do my ratings/reviews all at once (and that ratings I do at the beginning of the competition might be much different / need to change once I’ve read a wider scope).
I feel like I’ve read here that other people do this - so I’m looking for tips on how to structure that spreadsheet, and how others approach judging in a structured way. Thanks!
In '21 and '22 I went with a rating on a scale of 1 to 100. The score got converted to floor(x+9/10) for IFComp judging purposes. I didn’t worry too much about the actual score.
In the comments or next column over I wrote down memorable moments, themes or patterns for better or worse. (I may have used two, for technical and artistic merit.) Then I looked at things the day before IFComp ended to see if these were as important as I thought they were. And I twiddled scores. I looked for a clear gap between 2 entries to see, okay, this is where the point-jump occurs. If I didn’t find one, I didn’t force it, but I felt sort of guilty for the entry that didn’t make it.
My scores weren’t a bell curve, but they roughly resembled one without trying to force things.
I just keep a spreadsheet with all the games and authors (in the randomized order I play them), then track which I’ve played and reviewed. I typically jot down a rating as soon as I finish, but sometimes wind up tweaking that (usually after I write a review, I’ll get a clearer sense of how I feel about a game, and I usually just give out one or two tens so those are usually pretty tentative until I finish all the games).
The one thing to be aware of is that if you play a game for more than two hours, you can’t change your rating afterwards, since ratings have to be based on at most two hours of play - you need to rate it once you hit the two hour mark and can’t revisit it. So if you plan on doing that (and looks to me like there are more long games than usual this year), you can’t save those for the end, and you might want to highlight them in the spreadsheet so you don’t forget and tweak them.
What I would do is keep a ranked list of all the games. When the comp was done, I’d give the top of the list 10’s and the bottom of the list 1’s. If you’re entering things in a spreadsheet, it should be pretty easy to just insert a new row for each new game as you play it. Don’t worry too much about getting it precisely correct; it can be pretty hard to compare two very different kinds of games; and you can always just lump them together with the same score when you’re done. Just go for your general sense of what it is you appreciate about each one.
I’ve also found it helpful as far as structure to ask some general questions of each game as you go, again depending on what it is you personally like about IF. For me, I went with ‘Did the author have something to say?’ and ‘Did I have something to do?’ One advantage of doing this in a spreadsheet is that you can fill in the answers to those questions in a few words to help remind yourself of what you liked about each one when you’re handing out scores.
I just opened up a Discord channel to write down notes and observations without any systematic methodology. I plan to look up what I’ve written and then rank them by order on how much I like it. And I guess I’ll figure out if I want to do fancy statistics or just score them as-is.
The rubric provided by ifcomp is pretty good, and once I’ve played all the games I wanted to play, it’s not too hard to fit them in. A list of games and a couple notes to remind me is helpful.
Of course you have to have a spreadsheet! How can we know anything without them???
My approach is similar to @DeusIrae. sortable columns of date played, title, score and 2-5 word terse blurb that will remind me what it was. (ie “Finders Commission” was logged as “heist for cats”). Because I try to use a specific rubric documented here, I also log artistic repsonse, tech intrusiveness and bonus (usually +/- 1). I fill it out as I go with completed reviews.
Naturally, since it’s spreadsheeted, I also generate graphs from it. As inevitable as tides, really.
I suspect I’m fine, but this has come up often enough I am checking. I interpret the above guideline to mean you cannot change rating due to more gameplay. I would feel free to for example assign a new bonus/penalty point if subsequent games have shown me my initial experience was better/worse than initially thought relative to the field.
I suspect that’s within the rules, but I just personally don’t trust myself to be able to compartmentalize well enough to be able to implement that approach.
Understood, and I wouldn’t either except knowing that about myself I DO have the discipline to stop playing when the 2hr timer expires. I’ll pick up favorites again after the comp. [amends]So far I’ve had the discipline[/amends].
I changed a few of my votes by the end, but most stayed the same.