I agree. Though if you think of it as A-B-C-D-F like school grading it kind of makes more sense. I would like one more rank like a 'tier list" - So there could be ‘S-tier’ games for amazing stuff on a different level like Counterfeit Monkey and Colossal Cave (CC has flaws, but is a seminal work).
Since the averages can show half-stars, it’d be cool if you could also rate something 3.5. Though I agree that a rank from 1-10 becomes less helpful as it gets more granular - On a scale of 1-10, is there much difference between 7 and 8, really? More granularity is good for comparison, but causes stress about choosing your own rating.
Maybe a percentage score - which is even more granular but more informative as a collective rank - might be a better option. Even if you still rate 1-5 stars, the collective ranking could be a percent based on stars and weighted for number of reviews, sort of similar to how IFComp rankings where you get a bunch of 6 ratings, but there is a decimal score so you can see how all the games rated 6 compare.
I don’t agree, but it’s a thing I’ve heard several times.
The closest I’ve done is a couple times I’ve bought a graphic novel where I am not the intended audience. I didn’t care for it but since it wasn’t written “for” me and had few reviews I just marked it as read rather than giving stars that would lower the average rating.
Yeah, and I think it’s also the case that rigorously engaging with someone’s work and putting effort into good-faith criticism is actually being nice to an author – “this is great!” is nice but not nearly as rewarding to me as an author as someone digging into why and how something worked, or even didn’t work, for them.
(Though apparently I need to work on this, since @Joey crunched some more numbers from Brian’s spreadsheet and told me that I’m eight times less controversial than you are!)
When I rate books, I use a 1-10 rating for myself and convert them to 1-5 stars for Goodreads based on overall vibes. 1s are almost exclusively for DNFed books. 9s and 10s are rare, and only if the book is really good. 8 is usually my default, and then I dock points based on flaws, leaving most books around 5-6/3 stars. So a 5-star system works for me, even if it’s still too discrete.
I dunno if this is a robust data point or not, but tennish years ago I did a bunch of reviews on Goodreads of books as I got through them, and cluelessly rated things along the whole scale. I’m not active there anymore, but I occasionally get notifications that someone’s liked one of my reviews, and it’s almost invariably a more critical one (there’s this one of a Cicero biography I got kind of catty about that gets a lot of love for whatever reason).
EDIT: just checked and of my top ten most-liked reviews, there’s one 4-star one and no 5-stars.
Some communities value recommendations over constructive criticism. The idea is not stranger than its opposite simply because it is different.
I decided to stop giving out negative reviews maybe a year ago. But honestly, that’s become a function of exclusively playing games I like. I don’t rate games I haven’t played, and I don’t play games I don’t like.
I do use IFDB to find games I like. My method is relying on reviews by people I’ve found trustworthy. The numbers aren’t a big factor for me.
It’s interesting seeing all this interest in more granularity: I’ve always felt that a 5-level system is about the most granularity that’s really useful for me to make numeric ratings on, and so I’ve always rated IFComp games out of five stars and then doubled that and fine-tuned by bumping up or down a point (or not) just because the extra numbers are there…
Yeah, I’m generally of the opinion that ratings are so vague & subjective (and the philosophy of what each value means varies so much from person to person) that, if anything, 5 stars is too much granularity & gives a false sense of precision. IMO the only way to numerically aggregate reviews that actually works (if you have enough reviews) is the binary thumbs up/down, “X% positive” system. And then obviously you have in-depth written reviews, but they can’t really be aggregated into a number.
I kind of think there should be a label next to the star ratings to help standardize the subjective weight being given by reviewers.
★☆☆☆☆ Awful
★★☆☆☆ Poor
★★★☆☆ Competent
★★★★☆ Great
★★★★★ Incredible
I also believe in half-star ratings to nudge an opinion. For example, the half-star ratings could reflect: bad, lacking, good, excellent. That way you can say a game is truly excellent (4.5) without saying it’s the best thing that’s ever happened in the history of interactive fiction (5.0).
…or we could just have a scoring system from zero to Ocarina of Time and see how reviewers run with that.
OK, now that I strongly disagree with. Nope, nope, nope. IFComp has long acknowledged that different people will have different voting practices and that that’s FINE. The IFComp best practices for judges page has several paragraphs talking about this, displaying one example rubric and linking to two others, and ending with:
Ultimately, you are free to develop your own scoring system, and we indeed encourage you to do so. Score games according to your own experience, taste, and instincts. So long as you rate entries with thoughtfulness and in good faith, you can’t do it wrong.
Clearly for IFComp we need an Olympics gymnastics style scoring system, with separate scores for difficulty and execution, and fine-grained criteria for what points to deduct for which infractions. Unimplemented noun -0.1 point; Unmotivated choice -0.25 points; etc.
I think ratings are important and provide some kind of information. It’s hard to imagine a ranked competition without them. But at their base they are essentially a popularity contest and not some kind of representation of truth or goodness, and this includes the “was this review helpful” statistics.
I would seriously question anyone who claims their reviews are objective. Just considering the competition, do your reviews go down over time as you get exhausted, or maybe go up as you get exhausted with being critical, or maybe you readjust all your scores at the end to reflect a more direct comparison between the games (a method which is even more impossible on IFDB).
I’m sort of similar. In that I feel a 5-point rating system is easier for me to work with than a 10-point. Admittedly I can end up fudging, e.g. is something a 2 or a 3, or a 3 or a 4. But still dealing with the core 5 numbers to choose from is about as much complexity as I often want!
IFComp is a slightly unusual case for me, in that I have been rating it for so very many years, seemingly 30 years now assuming it’s run every year since it started, that I have a built-in intuitive feel about scoring for that one. So it’s very easy for me to do. Even if I agonise for longer over my ratings I usually come back to my initial gut reaction.
But again that’s an outlier. I am generally happier with 5-point scales, and can vote very quickly with them. That includes for example Goodreads, IFDB and Amazon reviews.
Discussions of criticism and criticism of criticism always reminds me of Hemingway getting panned by critics, then writing the thinly veiled metaphor “the old man and the sea”. A greater indictment of the critics was their embrace of that book, which is basically just Hemingway whining, than the book itself.
I find that a very strange sentiment. If someone publishes a work, they thereby offer it up for public criticism.
