How do people deal with post-comp in-comp reviews?

I’m curious to understand how it works and how you people cope with it.

There are games that have a post-comp after a short or very short time after the competition has ended. Some reviews, however, especially for huge events like IFComp, arrive a month or two after the end and often talk about implementation errors, bugs or defects that may have already been resolved in the meantime.
So I’m not talking about reviews that are years old (which reflect their time and that’s ok), but about games released relatively recently that are given an average or low rating due to these shortcomings.

I know there is the option on IFDB to state “this review is for an older version”, but do you really pay attention to that?

I don’t know if it’s a problem, but if anything: how can it be solved? And, more specifically, am I making it too problematic?

4 Likes

Reviewers have that option when adding a review to the IFDB
image
(I’ve used it once or twine I think after the fact)

Other will indicate that the review was written during the competition (to indicate it may be older than its publication on the IFDB).

As a player, I personally won’t pay too much attention to it until I am done with the game or if I run into an issue (and see if other people ran into it too).

As an author, if the bug has been fixed, I’ll leave a comment on the review to indicate that the point has been fixed (I’ve seen a few reviews being changed afterwards by the reviewer), and change the version of the game on the game page.

5 Likes

It can’t be solved. You can reply to the review saying “Thanks for finding this problem! It’s now fixed.” But I don’t know if people read replies to reviews. I don’t generally even notice them.

Reviews are what they are. I have several reviews that mention bugs that are now fixed, and it pains me, but that’s the nature of the beast, and that’s how I knew to fix them. Hopefully over time better reviews of the corrected game will tip the balance. And if they don’t, oh well.

Generally I try not to fixate on reviews at all. No matter what the rating is, it’s pretty awesome that anybody took the time to play my game and write something about it, no matter what they say.

13 Likes

(This review is based on the [insert compname]-version of the game.)

That line goes at the top of all my comp reviews when I add them to IFDB. Often I’ll point out in the review itself that any bugs I mention may have been fixed in a later version, hopefully reminding people to look for the latest version on the IFDB-page.

6 Likes

I think that mine is a model case (albeit there’s no review of the post-comp 1.5 release of Creative cooking on IFDB…) of the point raised by Marco, where indeed I take in account the constructive criticism and the serious issues noted.

what can help is comparing the date of comment with the date of the post-comp release(s), but the best solution should be that the rewiever notes what version/release is reviewed.

A minor point is that I decided NOT to release bugfixes during the IFComp, accepting fully the (disastrous…) consequences of my mistakes, whose I feel is the right behaviour to keep during a competition (I don’t think that the paradigm of a team and substuite players applies to IF Comp entries, because, IMVHO, IF stories are a whole narrative and world building amalgamated together, and a bugfix release, even for typo fixing, for me is akin to an unfair “changing the whole team”, but I digress…)

Best regards from Italy,
dott. Piergiorgio.

4 Likes

Ok but you don’t change the ratings, right? That’s how it goes: bugs are forever in this case.
So people from the future will think your’s a three-star game while it is four or more.

But, ofc, this is overthinking.

3 Likes

(sorry for the Italian below, but no worries, no backstabs plotted :wink: )

Marco, ricordati che la maggioranza qua è massimamente anglosassone, ed a differenza di noi, per gli anglosassoni, persinoi quelli da questo lato dell’ Atlantico, la prima impressione non è quella che conta… quindi, sì, penso proprio che ci stai pensando troppo :wink:

Stammi bene,
Piergiorgio.

Thanks to everyone else for the patience, and
Best regards from Italy,
dott. Piergiorgio.

1 Like

As a reviewer who’s been transferring IFComp 2023 reviews to IFDB I’ve grappled with this lately.

I always check when copying my review if there is a newer version of the game out by now. If so I will add a note at the top of my review text to say I was reviewing the game version available part way through the competition. But I only add that note if I am aware of a newer version of the game on IFDB.

Ideally I would be able to give a version number for the game reviewed. But not all games have that in game or in the accompanying materials.

The alternative would be not to copy over reviews to IFDB, but there has been strong encouragement from others to do this.

9 Likes

Yeah, it’s definitely a challenge – and I think somewhat exacerbated by the current trend of how to do reviews for the big comps, which is often for folks to write up real-time reviews on the forums but then schedule them to post on IFDB a couple a day after the Comp wraps up (I do this, and my last Comp reviews should be going up over the next two days – so, a two or three month lag between when some were written and when they’re posted).

Given this approach it’s often pretty hard for reviewers to know whether there’s an updated version available, much less whether the update fixes the bugs or other issues they highlighted. And honestly I’m not always sure the “this review is for another version of the game” tag by itself accomplishes much, since as a reader of the review I don’t generally know what if anything might have shifted in between releases.

As a writer of reviews, I try to resolve this by clearly flagging up top when the review was written – usually saying that it was done during the relevant Comp – and then, where there’s a game that seems like it’s got bugs or issues that could foreseeably be addressed in a post-Comp release (versus cases where the writing is just very bad across the board, or the fundamental theme or approach doesn’t work for me), I try to say that in the text of the review, so hopefully a reader can read that language, see there’s an updated version, and assess accordingly. If an author can write a reply to the review and say “the update does deal with these things you flagged”, that’s even better, though I recognize that can be a burden on the author too.

This doesn’t deal with the ratings question, of course – really not sure how to resolve that except to shrug at the whole concept of ratings, but of course that only goes so far.

So yeah, it’s a challenge. And I guess I’d say it’s just one more reason that the advice to only enter a game into the Comp after it’s done and you’ve tested it quite thoroughly is really good!

6 Likes

I agree with you about this, and it’s what I’ve done in competitions as an author before.

As a judge/reviewer I’m also not comfortable with what seems to be an expectation from some authors that judges will play the latest bug fixed versions during a competition. I do now usually play the latest version, rather than download a huge ZIP file at the start. But I do feel the original entered version should stand on its own merits. And not have a game updated time and time again. Mid comp of course. After comp is another matter.

4 Likes

A thought:

Like with the short games showcase, there could be an end-of-year showcase for significantly corrected or expanded games. If someone wanted to run it, it could be a chance to get fresh eyes on games that we feel didn’t show their full potential in a comp. That might result in some extra reviews/ratings that reflect the corrections, and possibly give some reviewers the nudge to edit their reviews. The “Full Potential Showcase” or something like that.

6 Likes

Or maybe during the early summer and do an academic year?

2 Likes

Thanks all for the replies.

Yes @vivdunstan, it was your reviews that made me think. And I couldn’t find a decent solution other than thanking everyone who does reviews, you included, for the great work. And YES, keep posting them on IFDB!

Point is: the ratings draw people around, in the sense that one presumably goes after better rated games. What @Piergiorgio_d_errico implies is not exactly clear (and I stopped supporting the “us vs. them” theory a decade ago :slight_smile: ): what does that imply? That players apart from a handful of people in the world use to play all the games without looking at the ratings? I have a hard time believing this. Also, I fail to grasp how “judging at first or second sight” has anything to do with this.

The fact is that, yes, I more than anybody know that playing a game two times for the sake of a review is nothing one can ask (and I would advice against, for a billion reasons). And I’m not by any chance giving any fault at reviewers’s hands. But the fact stays: older ratings lower the overall rating of a game that’s NOW essentially better.

Solutions? One is unviable and one is ridicule: 1) (changing nothing) having the “Full Potential Showcase (FPS)” as older ratings would stick anyway*; 2) (creating hell and we don’t want hell) duplicating the games in IFDB with different ratings from Rel2 onward (imagine the multiverse of madness).

*One last thought came forth, tho: I’m convincing myself of overthinking if I look at my own games in IFDB. The good ones (at least, I wish they are), have 30/40 ratings taken in like 12 years. Eventually, the new deletes the old. Ubi maior, I guess. So a FPS may in fact better those ratings average.

Sorry, I didn’t mean to point fingers or hassle anyone (especially you, Viv!) and this is probably enough. Although it got some response, indeed. It’s been years since my last speculative post and I’m causing trouble again. Not my intention :slight_smile:

5 Likes

One thing I think authors don’t realize the utility of on IFDB is the “News Item” which is a link at the very bottom of the page next to the edit button:

The headlines and links to a News Item appear before reviews on the IFDB page, and with a smart link name, you can notify people of a post-comp update that might contextualize any reviews of earlier versions below that. News Items also get a featured announce on the front page until they’re pushed off by other entries or news.

10 Likes

Viv, indeed I have only played the first two RC of Never gives up her dead because in seeing the vast improvements I preferred waiting for the full release, so I understand your feeling, albeit for a major work a RC process should be the best approach for a final testing, but isn’t an workable approach for mid or small-sized work intended for a comp…

Amanda, agree on “full potential comp” (hopefully not a jam…)

Hanon, I guess eyeryone and his/her/whatever cat/dog will be grateful for your pointer on ifdb editing.

lastly, Marco Innocenti: from your name and surname both in Italian, I guess you’re either Italian or a (now very rare) first-generation Italo-American, so I have explained directly in Italian, assuming that you understand the Dolce Lingua. But seems that my very elaborate, and somewhat archaic, Italian (for a long period my grandfather, class of 1903, was the reference model…) has hit again (yes, even Italians have difficulties understanding what I say…) so, trying to clear the issue, I don’t think that US (and UK) people judge things “first look” (In Italy, the first impression matters…) so, in the long run, the merits prevails on demerits (I note that the majority of more positive reviews came from IF coders, whose have taken in account the limits of Magx, inherited from AGT)

Last thing, at the core of the issue is probably the weight of the earliest comp reviews in the decision of judging priority (Personally, I have given priority to parser-based stories, and only toward the end I have judged some choice-based stories, but when you’re facing judging between 75 and over 100 entries, is comprehensible that some prioritization criteria became necessary)

so, an half-baked, no, quarter-baked, idea: a competition rule embarging reviews, until, say, after one quarter or one third of the judging period ? I dunno if is a good or bad idea, but I’ll offer it, anyway…

Best regards from Italy,
dott. Piergiorgio.

5 Likes

I often read quite a lot of reviews. If I read these in the time of a comp, I know these reviews will be in-comp reviews. After some time, reviews are generally explained by themselves with phrases as “I played the post comp version”.

1 Like

I saw this happen recently. The changes I had made to my own game weren’t substantial enough to address the meat of the criticism, and the review itself didn’t bother me enough to complain, but the outdated parts of the review caught my attention.

6 Likes

If the game has any legs, eventually enough newer ratings will smooth out any ‘dings’ the game took from reviews based on an early, buggier version.

Like Hanon said, if the only visible reviews a game has received talk about big bugs that are now gone, the author can comment on the review, or put out a news item addressing them.

I don’t think reviewers owe authors if they initially write a review in good faith, but they might feel like being helpful anyway. Like others in the thread have said they do (similar things): I write a contemporary review in my blog during a comp, then post a version on IFDB later edited to be more for the ages.

But all authors, reviewers and people who do both are in the same boat.

In conclusion – stop putting bugs in your games, then you won’t experience any of these problems :wink:

-Wade

11 Likes