I don’t know if I feel like taking the survey so I’m just going to post my comment here. A month and a half feels like an excessively long time for the judging period. Has it always been this long?
1 Like
It has been this way since at least 2019, according to the IFWiki. Except in 2020, it was actually two whole months.
You have to take into account that there are a lot of entries and, to be able to play everything (or most of the entries), a month 1/2 is just enough sometimes.
Many other competitions have just about a month to vote. Like the SpringThing, or the ParserComp, which have significantly less entries.
10 Likes
To evaluate 67 games? Are you kidding?
It’s been six weeks since at least 1999; for much of that time the comp was like half the size it was this year.
6 Likes
Because I have a dignity to defend, I reserve the right, after cool assessment, of requesting the deletion (from the results, not the story files archive) of my 2024 and 2023 entries.
This only for the record, the issue is in evaluation, nothing decided.
Best regards from Italy,
dott. Piergiorgio.
1 Like
No I’m not kidding, though I don’t feel super strongly about it either. I’ve never played all the games in any given year, and admit I don’t really understand the people who do. I generally don’t play or rate games in genres or styles I’m not interested in, and am skeptical of people who do rate them and who think they can do so fairly. Also, a lot of the games are really short, and you aren’t supposed to play any of them more than 2 hours. Finally, it seems like the reviews really fall off a cliff the last couple weeks anyways.
1 Like
On “suggestions for next year’s comp”, since this doesn’t really fit in a one-line text box…
I would like the comp to get a bit more stringent about avoiding subjecting judges to groupthink and bias. On that note I would propose two new rules:
1a. As an enforced rule, authors who have previously placed 20th or higher in any previous competition must submit their games under a fresh pseudonym.
1b. As an accompanying guideline, while judging is in progress authors should avoid linking that pseudonym to their real name in any forum (such as this one) where judges are likely to see it, and judges should avoid posting speculation. Again, this part is a guideline and I wouldn’t seek to disqualify anyone over violations. Keeping author identities hermetically sealed isn’t practical and isn’t the goal here.
(n.b., to offset bias I currently have a personal policy of giving a slight score penalty to any game whose author I positively recognize)
2. Judges must not rate games or change existing ratings once they have read any other player’s review of that game. If you read something by accident, you can still give feedback and indicate what score you would have liked to give, but cannot enter a score. Naturally, no enforcement is practical so this would be honor-system-only, just like the two-hour rule.
2 Likes
on suggestions, I indeed have a proposal, but needs a cool-head assessment, so I’ll think on during the next day(s).
Best regards from Italy,
dott. Piergiorgio.
2 Likes
The form allows for multiple lines 
8 Likes
Based on the ratings there seem to be like 100 judges. Many of whom probably are themselves authors or play testers. I’m not sure if pseudonyms are going to work, or if group think is something you can solve without draconian measures.
1 Like
Speaking as someone who plays and reviews all the games - for the love of god please don’t shorten the judging period, I’m already sufficiently sleep deprived by the end as it is!
Admittedly before I had a kid I found the six weeks comfortable enough, but here we are - and I know there are always judges saying there are games they wished they had time to get to. Yeah, some games are short, but many are long, and by recent standards this was a low-entry year that’d still require someone playing every game to put in 10ish hours per week in - and writing reviews can easily double or triple the time commitment.
I also think one of the cool things about the Comp is that it brings together a whole lot of disparate kinds of IF - altering the judging period with the implicit idea that everyone should just be sticking to the stuff they already know they like feels like it’s in tension with that reality. Does this mean IF Comp results aren’t a perfect reflection of each game’s ability to incarnate its particular subgenre? Yeah, but I don’t think a change like this would make any difference on that front either 
22 Likes
Of course it can’t be solved; there’s no need to be that binary about it. I think simple measures such as these can significantly mitigate it, and that’s enough.
1 Like
This seems impossible to enforce without completely shutting down discussion here. Even if First Contact was entered under a pseudonym, everyone knows about dott. Piergorgio’s ongoing project that it’s a part of; very few people have been asking questions about coding in Dialog, and those questions correspond very closely to the implementation of a couple particular games in the comp; and I’ve definitely gotten help from the cover and blurb review threads.
12 Likes
Fair enough. I can’t resist a good rhetorical flourish.
2 Likes
Yeah, it’s hard for me to see how this would work in practice - most people solicit beta testers here and typically need to give at least a hint about what the game is like to recruit folks, so I don’t think it’d take long for the cat to be out of the bag. And denying players and reviewers the ability to fit a game into its authors’ broader context feels like it’d be doing the community a real disservice given that most reviewing happens during the Comp.
There has been an informal norm of Comp winners not submitting games in future years - or at least, not games calculated to win - which I think is a reasonable one for us to have to make sure there’s space for newcomers, though honestly looking at Comp winners and high-finishers over the last couple years I don’t feel like this is a big issue.
As for changing ratings upon reading reviews, I agree it’s good to avoid groupthink, but at the same time reviews are often really good at suggesting ways to look at a work that a player might not have previously considered, and I don’t think that’s a bad thing. As you say this is all unenforceable and reliant on the honor system, anyway, so feels to me like something that’s best engaged with as part of the general rule about good-faith judging.
12 Likes
Yeah, it’s hard for me to see how this would work in practice - most people solicit beta testers here and typically need to give at least a hint about what the game is like to recruit folks, so I don’t think it’d take long for the cat to be out of the bag.
I think that’s fine and still in the intended spirit of the guideline. All I mean is that authors shouldn’t deliberately go out of their way to circumvent pseudonymity. It’s a similar situation to anonymous peer review of journal articles; pretty often, it’s a small enough world that qualified reviewers will have an easy time guessing at who wrote the paper even though it’s nominally concealed. Anonymization doesn’t have to always work in order to be good practice.
I can understand why those rules seem helpful, especially to newcomers, but I also feel like they’re ultimately counterproductive. Let’s trust the reviewers to know that even though some author made some game they loved in the last competition, the current competition’s game is getting reviewed on its own merits. I don’t think that’s an unreasonable expectation.
8 Likes
That’s not what I’m saying. I’m saying you are (or at least I am) going to encounter games you fundamentally don’t like, but know other people may like them. I’m not going to give it one star because I think it sucks when I know I’m not the target audience, and I’m not going to rate it at all since I don’t trust myself to be unbiased.
Ok, now I’m definitely confused, because in that case don’t you still need to play the game to realize that, and therefore the only time being saved is the time it takes to rate the game, which is pretty negligible?
2 Likes
Honestly, I think it’s an intended part of the comp that people will rate games they don’t like, if they actually played them and feel they gave them a fair shot.
7 Likes
I get the feeling that people are dancing around what they really want to be different; we talk about not playing games we don’t like, shorter periods, anonymous authors.
What’s the bad thing that happened? What was not good this time we don’t want to have happen?
It’s like when someone’s having an affair and they come and say ‘this marriage isn’t working for me, I want a divorce’. Even if the partner wants to fix things, it won’t matter if the other person already has a partner, so just admitting it will help.
So in this case, if the problem is ‘infocom tribute games place low’ or ‘the game I like was behind a lot of people I’ve heard of’, then it would be good to say that, otherwise we could be circling around trying solutions for a long time before figuring out what people really want.
20 Likes