IFComp score predictions: 2018-2021

Competitions IFComp
1

So this is basically one huge procrastination/anxiety project. Yes I procrastinate by doing data science projects for fun. No I am not okay.

I’ve been trying to predict IFComp results based on the results of the previous comps. To that end, I’ve collected data from 2018-2020 in order to build a formula. This includes all the information on the review spreadsheet for those years, plus game data such as the play time and whether it’s parser or not, plus the IFDB rating at the end of the comp. I got the data by scraping the ifcomp and ifdb websites: Extract data from ifcomp/ifdb · GitHub. Then I did a multivariable linear regression on the data from 2018-2020 in order to predict the ifcomp rating from the observed variables (see the 2020+2019+2018 sheet).

The 2021_predict sheet contains the predictions, sorted in order of predicted rank. Obviously these have no relation to my personal opinions of the games.

I dunno, I just think it’s interesting :pensive: Maybe there’s other stuff in the spreadsheet that’s interesting too.

19 Likes
2

I’ll admit, I thought this sort of thing could be interesting to try, but I never got around to doing it. I really like being able to see others’ Python code, because so often I’ll say “oh, of course you can (X), why didn’t I think of that?” or “I always meant to look into this but never made the time, and here it fell in my lap.”

In this case, I haven’t worked with beautifulsoup nearly as much as I should’ve, so this is a nice crash course.

So I (and probably others) thought about it, and you went ahead and did it.

These sorts of metrics are interesting, because we know they won’t be fully accurate, but they’re probably going to be pretty good.

I know I’m curious about how things will place, but I don’t like making predictions on my own, because I’ll inevitably leave out one entry that should be a lot higher.

It’ll be interesting to see how the ratings change as more reviews roll in.

3 Likes
3

I had a good chuckle at this. It’s bonkers to think a first-time writer’s buggy parser game would place that high. Not that I think you’re bonkers, or that math is bonkers. But yeah, no.
Still, it was a nice little fantasy for a second there.

3 Likes
4

This is pretty much exactly where I expected my game to rank (I was hoping top 15, but 18 is close) so I think you’re onto something.

I’ll guess we’ll find out in a few weeks…it’s always possible that there is a bunch of reviewers who chose to play my game and will quietly vote high or low.

1 Like
5

this is really interesting! It’ll definitely be fun to compare to the final results :slight_smile:
Based purely on reviews I’ve read in this forum, I’d agree with your #1 pick. Seems like the clear winner. But you never know what can happen I suppose!

1 Like
6

Looking at the regression, it looks like parser games have a negative coefficient?

Not what I would expect, given a naive guess, but I’ve been not closely following the end results of the comp for a while.

1 Like
7

Absolutely fascinating. And highly complimentary to me, which I don’t mind one little bit. I think my game is more like 30 minutes than 60, so that will probably skew things a LOT.

1 Like
8

The results for parser vs choice varies by year; the average scores for parser and choice games in 2019 and 2018 were almost the same! In 2018, only 2 of the top 10 games were parser (out of 32% total parser games), and for 2019, it was 4/10 (out of 40% total parser). On the other hand, 2020 was a really good year for parser, with 6 out of the top 10, and an average score about 0.8 points higher than choice games. This year looks like it’ll be closer to 2020 than 2019/2018, with a very good crop of top-rated parser games.

I think that the reason that the coefficient for parser is negative is because of the IFDB ratings are taken into account. Parser games tend to have more (and often higher) IFDB ratings than choice games, so once that’s taken into account, the parser advantage disappears.

Also, I’ve been doing some cross validation testing, trying to see what the predictions for 2020 and 2019 using previous years looks like. You can compare the predicted and actual ranks to see which kinds of games were over/underrated. IFComp predictions cross-validation - Google Sheets

2 Likes
9

I’m very glad to see that Math thinks Alone is the 7th best game in the previous three years. I love you too Math.

5 Likes
10

Okay, so, these predictions are “finalized” I guess…

I’ve updated the score predictions with the current public reviews and ifdb ratings. Also, I’ve added the ifdb weighted rank to the spreadsheet, along with a new model that only uses data from 2020 as training data. The ifdb ranks are pretty similar to the algorithm’s ranks.

I’ve also added another sheet comparing various prediction methods, including one based on my own random guesses (I started with the predicted scores and added ± 0.25 to 1 point based on my vibe check of the reviews).

Does anyone else have any predictions? It would be a fun game to compare the accuracy…

5 Likes
11

I don’t have a mathematical model, but I have registered some predictions on the results.

Winner

  • And Then You Come to a House Not Unlike the Previous One: 25%
  • Fine Felines: 20%
  • The Golden Heist: 20%
  • Grandma Bethlinda’s Remarkable Egg: 5%
  • Other: 30%

Golden Banana

The standard deviation of Universal Hologram’s scores will be…

  • >2: 70%
  • The highest of all games: 40%

Personal

  • I have played and voted on the winner: 95%. For reference, I have voted on 57 out of 71 entries (80%).
  • 3 or fewer of the top 10 scoring games are in my personal top 10: 80%.
2 Likes
12

I predict Dr Horror’s House of Terror for first place and 4x4 archipelago for 2nd, followed by ‘and then you come to a house’, you are spamzapper and paradox between worlds in that order. I’m always wrong in my predictions but it’s fun to guess!

4 Likes
13

My guess at the top ten, in alphabetical order. This doesn’t necessarily reflect my own ratings, though.

4x4 Archipelago
AardVarK Versus the Hype *
And Then You Come to a House Not Unlike the Previous One *
Fine Felines
The Golden Heist
The Libonotus Cup
A Paradox Between Worlds
The Song of the Mockingbird
Sting *
What Heart Heard Of, Ghost Guessed *

On a selfish note, I enjoy being able to say I tested a game in the top 10. I have a long streak. I think the 3 other works I tested have a shot at top 10 too.

6 Likes
14

I would be thrilled to win this. Crossing my fingers :slight_smile:

6 Likes
15

I will love you forever for predicting that I had a chance at winning.

I was confident at the beginning of the comp that I’d be in the top 20, and hoping hoping hoping to be in the top 10. (As I’ve said elsewhere, I came 7th in 2015 and that game is still consistently my best seller, so I’m hoping very much to revive my flagging income by repeating that success. My “income” from writing is currently not enough to cover my web site cost, :joy:)

I’m cautiously optimistic now that I’ll place in the top ten, although I don’t THINK I’ll be in the top 5. If I do place in the top… 12 (there are more entries to the IF Comp these days), I’ll feel satisfied that I have improved as a writer in the last six years, which will mean a lot!

Of those I played, I think “The Golden Heist” is likely to place in the top 5. And from general buzz, I reckon “Sting” “Grandma Bethinda’s Remarkable Egg”, and uh… two with long names that I don’t remember as well… are also likely to be in the top 5.

3 Likes
16

It’s funny, I’m reading this thread thinking people are consistently way under-ranking their own entries – but then I’m seeing people ranking Sting in like the top ten and also going “ha, there’s no way that’s happening!” It’s really hard to self-evaluate, I guess! Anyway, here are my poorly-thought-through predictions, in binned form rather than precise order – as with others, my personal rankings are in some cases quite different:

Winner: …And then you come to a house

Rest of the top 5: 4x4 Archipelago, A Paradox Between Worlds, The Golden Heist, The Libonotus Cup

Rest of the top 10: Dr Horror’s House of Terror, Song of the Mockingbird, Aardvark vs. the Hype, At King Arthur’s Christmas Feast, Fine Felines

Rest of the top 20: What Heart Heard of, Best Man, SpamZapper, Off-Season at the Dream Factory, Grandma Bethlinda’s Remarkable Egg, Closure, D’ARKUN, Silicon and Cells, Goat Game, House on Highfield Lane

Rest of the top 30: Funicular Simulator, Ghosts Within, Mermaids of Ganymede, My Gender is a Fish, Finding Light, we the remainder, Codex Sadistica, Walking Into It, Waiting Room, Sting (probably towards the bottom)

17

I think this is a no-brainer. I’d be shocked to see something different, given the number and quality of its reviews.
I am a selfish person, so I haven’t played it yet. I wanted to play it when I wasn’t under any pressure and could just take my time with it and enjoy it, as I’m pretty sure I’m going to love it.

1 Like
18

Yeah, this is probably the prediction I’m most confident in. WHHoGG, by contrast, I’m pretty underconfident in – I actually had it in the top 10 until a last-minute wobble, which I’m now second-guessing, bumped it down slightly.

2 Likes
19

I think what’ll be interesting is the Miss Congeniality, given the amount of cross pollination in beta testers and authors (at least in parser games), and the resultant inability to vote on those games.

Maybe it’ll be whatever parser game twine/choice authors like best, maybe it’ll be whatever choice game parser authors like best.

Or maybe there’s less incestuous beta testing in the choice based world? :rofl:

3 Likes
20

Ha! I’m trying to cultivate an even more extraordinary stat.

Of all IFComp games I’ve beta-tested over the years, there were only two occasions where I’ve specifically ventured to their author, ‘I think this will do well in IFComp.’

The first of those (Andromeda Apocalypse) won IFComp.

For me to uphold my super-blessing touch, Heart Heard/Ghost Guessed needs to win this year.

So if it doesn’t… I blame Amanda for everything.

-Wade

4 Likes