Exactly.
@mathbrush this year we’ll take the results as is, but this definitively must change for the next year.
The itchio’s formula is too radical, that with a difference of 2 or 3 votes could swap the places of two games. And even, this happened for the two first places of Le Grand Guignol.
With the raw scores, without the deviation applied (raw score) Dark and Deep would win the competition in her category, but Familiar Problems got 2 more votes.
At first, I thought that the formula would just interfere in judging the non-English categories, like the Spanish one, which had a messed up ranking. I judged that 7 votes should be sufficient enough for a game to be considered “impartially judged”. But now I see it is messing even in the very first places of the ranking for English games!
As I said in other years, it doesn’t matter, the competitiveness is just an excuse for the community to be at it, playing, judging, and giving feedback. We all know that Dark and Deep and First Problems are BOTH winners, and so are all those great games we have played this year.
It seems that the formula at Itchio is designed for ranked jams that have massive participation, but the reality is that most jams there are as niche as our community. So, definitively, for next year we’ll have to prepare a proper Excel and do our math.
As I always said, itchio is not perfect, but it is convenient. I’m quite happy with the overall performance of the competition, but this aspect has to improve.
I understand that this kind of formula is needed to ensure that games are judged with a proper amount of votes and that someone with very few votes would not overpass a properly voted one. I think IF Comp has another formula for this, but at the moment of writing this I cannot find that information in the rules or website.
If this year I had more time, and more eyes, to judge all these problems, I could make better decisions, but these days Itchio was down, so it was not possible. All this happened in the course of this morning.
Thanks!