IFComp 2019 Follow-Up Survey: Responses Requested by Feb 9th

One of the topics that has come up in our surveys to the community over the past few years has been the growing volume of games in the Interactive Fiction Competition. We’ve been tracking this, wondering if it was a phase, but we’ve been over 75 games for three years now.

We’d like your thoughts on whether or not you think this is an issue, and, if so, ideas on how you’d like for us to approach it.

All ideas submitted will be considered by the IFComp committee (who may do absolutely nothing, or who may make one or more changes to the competition). We encourage you to share/discuss/debate your thoughts in public forums (like here!) to help further develop your ideas. You can come back to add as many ideas as you’d like, until the survey ends (midnight Eastern on February 9th). Thanks!

4 Likes

How has the ratio of games to judges changed over the years? If that’s been more or less constant I don’t see the growth as a problem.

1 Like

I don’t think we have an official estimate of judge counts over the years. However, the subjective experience of judging is certainly different when there are 80 games (vs 12 or 20 games) – regardless of how many other judges are out there.

I filled out the survey, but I would also just like to say here that I think multi-round run-off elections are a great way to handle situations where there are more candidates than most people want to consider.

For example, we could take the top 25% of the games from the first round and have people do a second-round run-off election with just those.

I think a lot of people would prefer to participate only in the second-round vote, and that’s perfectly fine. Plus, any random game you try during the second round will probably be pretty good.

I think it’s great the organizers are dealing with this issue. First of all, let me emphasize that I’m thrilled that IF and the IF community is growing, and that IFComp is such a popular event. I very much enjoy the diversity of entries in recent years, both in terms of genre and form. I love the feeling of picking the next game from the list, and having no idea about what’s going to happen, except that it will probably be original, enjoyable, and good—possibly great. And that’s another thing: This is not a debate about quality vs. quantity, because the quality of most comp entries is actually very high.

But I think we do have a problem. If we want the IF scene (on this forum and elsewhere) to be a living, thriving community, then we have to be able to have interesting discussions about a common set of topics. We don’t have to agree on things (heaven forbid), but we need some kind of common language, a set of ideas and experiences that we can talk about. IFComp, of course, is the single biggest event on the IF calendar. Each October, people from all over the planet emerge from their hiding places and come together to join the fun. Forum activity increases, people write lengthy blog posts and reviews. But IFComp needs to be a shared experience.

With over 80 entries, most judges will only have time to look at a small subset. The official word is that you only have to play a minimum of five games to be a judge. But here’s the thing: If you play five random games, and I play five random games, then we didn’t have a shared experience that we can talk about afterwards. In fact, with 70% probability, our samples didn’t have a single game in common—let alone two, so that we could talk meaningfully about relative merits.

Here, let me plot that:

probplot

The X-axis represents the number of comp entries, and the Y-axis represents the probability that a given pair of judges—each picking five games at random—do not share the experience of even a single game. Of course, the other parameter to tweak is the recommended number of games. Five is clearly intended as a hook, to get people to start playing instead of just being overwhelmed and giving up. The unspoken hope is that people will actually play more than five games, and many presumably do. But in a competition with, say, 20 entries, I think many judges would actually set their minds on rating every single game. And most of them would follow through on that ambition.

The point I’m making here is emphatically not about fairness. A competition with non-overlapping samples can still be fair, as long as each judge is selecting entries at random, perhaps by following their personalized shuffle order. No, this is about building a community where people can participate in meaningful discussions, grounded in shared experiences.

Now, some suggestions. Like I said at the beginning, I don’t think we should place limits on the kinds of stories that people can submit. But perhaps we could place a limit on how many stories a person can (co-)author per year? Now, I know this would have excluded several excellent entries from past years, but those entries could have been released in other competitions, or saved (perhaps even refined further) for another year—for the greater cause of reducing the number of entries in IFComp.

Another idea is to introduce a separate side-competition, sort of like the back garden of Spring Thing, where the rules are a little bit different. In particular, the two-hour rule could be relaxed, the deadline could be a week later, and the judging period could be extended. If we then dedicate a fixed proportion of the colossal fund cash prize to this sub-event, then we create an incentive for authors to select the least popular category. The original 1st of October event would probably see a larger crowd, and therefore have more entries, but as a consequence it would be easier to earn a top spot in the back garden, and get a larger cash prize. And when the deadline for the October event is getting uncomfortably close and you still haven’t found that crashing bug, you’ll no longer have to choose between submitting a buggy game this year or a working game next year—you’ll be able to submit a working game next week (albeit probably to a smaller audience). And that’s a win-win for authors and judges.

3 Likes

There already is a limit of three entries per person. Reducing this to one would not make a noticeable dent in the size of IFComp.

2 Likes

Another way to put it is that we need to emphasize—more strongly than today—that this is a competition for highly-polished, finished works, and that recently, the sheer volume of entries has been a problem (if enough people end up sharing this viewpoint). That we all need to do our part to reduce the number of entries, and that we have a responsibility to think twice before submitting something. But this message needs to be communicated tactfully, in a way that is slightly holding back over-confident potential entrants, but doesn’t scare away those with a more low-key personality.

If, as you say, a restriction of one entry per person has no real impact on the comp, then I think it would be a good and safe way to send that signal.

The IFcomp organizers are asking for ideas on how to bring down the number of games, but they are also asking if we find the big number of games to be an issue at all, which is just as important. Perhaps most people don’t think it is a problem?

I personally think the IFcomp is close to perfect. The number of games seems to have stabilized about 80 games (too early to say though). Even if two judges may not play some of the same games at all, there are always a bunch of people who played the same game. In 2019 only one game (Alice Blue) had less than 10 votes, most likely because it was Linux only. If a game is not very accessible due to the format, the game will not be played very much and thus it doesn’t take time away from judges, who simply can’t play it.

I enjoyed at least seven of the games who didn’t reach the 2/3 mark to receive the colossal cash prize. I think it would be a shame to exclude games for some reason, mainly for the purpose of keeping the number of games low.

4 Likes

I’ll be another voice speaking up for keeping it “all together”…I suspect you’ll get a lot more people advocating for some kind of breakup, so let me advocate for having one large undifferentiated contest.

This is heavily colored by my own experience: I’m pretty new to this community, and most of my IF writing has been directed to IFComp in specific. I believe IFComp’s current format drove me to make my games the way I did…and I don’t think I’d have the drive to made games like this without this format.

IFComp is one of the few events out there where a newcomer can be relatively certain of submitting a work that at least one person will read, and seriously review. The practice of creating a randomized list containing all submissions, all competing for the same goal, means that a total newcomer has a chance of being noticed, and reviewed on the same terms, as an established figure. This is fun and more to the point I think it produces better content…previous posters have noted that we’re not exactly being flooded by mediocrity here…the contest is getting bigger, but the newcomers are (if I do say so myself) producing new and different stuff, of very high quality.

Part of our motivation (at least mine, anyway) is that we know we’re up against established figures who make great stuff, we know we’re going to be reviewed on the same footing as these games, and we’re participating in a tradition that goes back a long ways. This means there’s pressure both to do something new and to understand the conventions of the genre we’ve chosen to work in.

It’s hard to get this dynamic going, and breaking the contest up has a potential to ruin economies of scale that have developed.

In particular, I think it would be a bad idea to do either of these two things:

multi-round elections: This would, if anything, create more work, and break down a lot of the dynamics I discussed above (and I think any attempt to preserve one of the two would necessarily make the other effect worse). An additional concern: many games in IFComp “break late”…they get a lot of attention at the end, and are basically unnoticed in the beginning. Many games in IFComp further “fizzle out” (they get attention early on, but fail to live up to expectations). The judges in the “first round” are going to be a very different sort of person than the judges in the “second round” and that will definitely affect what games get seen, and get made.

separate categories based on “type”: the most commonly proffered demarcation is CYOA/Parser. I don’t like this, because I can never figure out which category certain games should fall into. It’s possible that I’m over-analytical but I feel like this would create a lot of ill-will and lead to weird debates about what “truly” makes a game one or the other.

7 Likes

That’s too limited a question!

The question is “What, if anything, should the organizers do about the recent trend of 80-game IFComps?” This is wide open. Maybe there are ways to make judging more comfortable in an 80-game (or 100-game, or 120-game) comp.

3 Likes

If I were to advocate a change, it would be to simply make it more clear that judges will select/play/review games based in large part on “first impressions.” (Additionally, some judges will only play games designated as parsers, some judges will only play games that are sci-fi games, some judges won’t play games with zombies in them). As the comp grows, initial presentation becomes more important. Comp. participants largely know this (in fact the reviews I appreciate the most are the comprehensive reviews of blurbs/covers) but some people seem to have been taken by surprise by it.

I don’t agree. You identified two concerns that I recognize:

  1. Ensuring at least one person will read your work
  2. A chance of being noticed and reviewed on the same terms as established figures

I don’t think either of those would be negatively impacted by multi-round elections, presuming they’re done the way we already rank IFComp entries today.

Forgive me if I’m misunderstanding you, but I think you’re under the misconception that winning IFComp requires “attention,” i.e. you need to gather the most votes, like in a political election. That’s not true for IFComp.

IFComp ranks the entries by average score. So if there were a game with an uninviting description and ugly cover art that only ran on old Commodore Amiga machines, and so nobody played it except for one judge, who rated it a 10, that game would have a perfect score, and would win the comp.

(In practice this doesn’t happen, partly because we have a few heroic completionists who review every game. Still, today, there are a number of games that are so uninviting that hardly anyone except the heroes play them.)

So a multi-round election would still ensure that at least one person would read the work. The heroes would still play your game.

And as for being noticed, certainly you’d get more attention if your game made it to the second round, but it’s not the case that a minimum amount of attention would be required to get into the second round.

If, as a newcomer, you develop a great game that nobody wants to try because they’ve never heard of you, but everybody who does try it loves it, then your game would graduate to the second round. Way more people would try it as a result.

2 Likes

I appreciate how, in recent years, more information about the entries has been added to the IFComp website–genre, estimated playing time, and so on. It helps in choosing which entries to try. I’ve never felt obligated to try to play all the entries, so if there’s enough information to help me figure out which ones I want to play, that’s good enough for me–at least, at the comp’s current size.

Do we know why IFComp has gotten so big in recent years? Is it related to the cash prizes? If IFComp continues to grow, would there be room for more than one comp during the year that offers similar cash prizes?

2 Likes

One way to improve first impressions: we could allow/require authors to upload cover art and blurbs early, and post them publicly online before the competition starts.

This would give everybody a chance to review and respond to the game metadata, giving authors a chance to improve their curb appeal before the full competition starts.

Valve requires something like this for all games that go live on Steam; you have to put your game store page online for a few weeks in “Coming Soon” mode. It helps!

3 Likes

THIS idea I really like. It would be very fun. I definitely think the best moment of IFComp for me, both years, was the initial reveal, because it was such fun to scroll through the list and see what we were “in for.” However, it is an extremely alienating experience when one realizes, many weeks into the comp, that people are just fundamentally not “getting” what you were going for with your blurb and cover (to some extent this is inevitable but it resulted in fairly jarring experiences for me both years).

A “coming soon” phase would cut down on that, and emphasize how important that first look is.

Track game that have been reviewed and suggest games for review that have not yet been reviewed or have a low number of reviews.

fos1

2 Likes

First off, I’m more on the if it ain’t broke, don’t fix it team; things seem to be working fine as is.

That said, for the sake of discussion, a few ideas:

  • Nominal entry fee
    Like the Spring Thing used to have. Make the fee tiny, 1 zorkmid say, so it shouldn’t prevent anybody from entering, just makes entering a more deliberate action (c.f. the zero price effect).
    Downsides: unequal burden on participants, (even if the price were effectively nil for everyone there would still be transaction costs that would be more onerous for some).

  • Have a cap on the number of entries

    • First come, first serve. Or a lottery to get it
      Downsides: unlikely that you’d end up with the best entries

    • Make entry by invitation only
      Downsides: unfair to previously unknown authors

      • Make a new invitational contest
        To siphon off some high profile authors
        Downsides: risk of damaging the status of IFComp
    • Screen entries and only accept the best
      Basically a multi-stage contest with a secret first stage
      Downsides: That’s just moving a ton of work onto the organizers

  • Increase the frequency of IFComp
    Have it twice a year
    Downsides: more time and effort for the organizers, probable loss of status for the comp and its winners

    • Boost the Spring Thing
      As things stand, winning the Spring Thing probably brings less prestige than being in the top 5 or so in IFComp. A higher profile Spring Thing might attract some entries away from IFComp.
      Downsides: probably, but I don’t see any right now
  • Split IFComp into categories
    I’m not going near the parser/cyoa debate with a ten foot pole, but there might be other possibly viable groupings: length, be it estimated play time (but how do you estimate it) or word count (possibly tricky to calculate for parser games); puzzly / less puzzly; humorous / serious; containing pirates / sadly devoid of pirates; …
    Downsides: Flame war bait

Having writing all this, I still think no change to IFComp is needed, but perhaps boosting the Spring Thing is in itself a worthy goal that would ancillarily help diminish the number of entries.

5 Likes

I said this in the survey, but I think that incentivizing reviewers would be effective.

Imagine every game getting multiple early reviews and discussion. That’s most author’s dream!

I think having a side-contest like Miss Congeniality but first reviewers would help. It wouldn’t even need a prize; just having “top reviewers” listed could help.

Obviously I would personally benefit from such a situation, but I have an entry this year and have two or three more planed for the future.

The votes for best reviewers could come from the authors and/or the general public (for instance, if someone’s reviews helped you pick what games to play or if you’re grateful a popular podcast gave games thoughtful discussion).

6 Likes

My suggestion in the survey was to lengthen the judging period. If there’s longer to play the games, maybe more people will get to more games. (OK, I should say that before the explosion I used to start the comp and get derailed when midterms hit. Now I barely dip into it–if the organizers want me to play a lot they’re going to have to extend the voting period approximately to late December, because that’s when I have time, and I don’t see that happening. One issue now is that lots of the games are either long or have sound, it seems.)

Idea! What about having each game “featured” some random week–it wouldn’t actually do anything but be a nudge as to games you might want to play that week. So if there were 80 games and eight weeks of judging, you’d have ten games “featured” each week. I’d suggest it could be completely random, but maybe the organizers would want to have a mix of longer and shorter games.

4 Likes

Are you envisioning this as a common, global schedule or a per-judge randomized order?