Should next year's IF Comp allow intra-comp updates?

[vivdunstan]

@matt w

Mostly this is just a pretext for complaining that the poll choices are slightly misleading. The alternative to allowing updates isn’t “let us play the original versions”; it’s “make us play the original versions.”

I don’t view those as the options as such. More the problem for me is that people are ending up judging very different games, if the games are updated. I’m thinking of one game in particular, which has changed name and changed significantly within it (including the player character). Yet other games are unchanged. is that fair, either to the judges as a whole, or the competitors?

Competition games have traditionally been revamped post-competition, and that system has worked well. I don’t see the need for changing them in the middle of the competition, and I think it introduces a lot of questionable issues, which shouldn’t be there.

[Juhana]

@vivdunstan

Competition games have traditionally been revamped post-competition, and that system has worked well.

A small correction: It has been expected for the games to make post-comp updates, but that system has not worked at all well – most games never get a single update, even those that have serious game-breaking bugs, and this has been a major point of frustration. I assume this has been one of the reasons for allowing updates during the comp and I expect that it will produce more updated games than in the previous years.

[Ghalev]

@Juhana

[…] most games never get a single update, even those that have serious game-breaking bugs, and this has been a major point of frustration.

It’d be interesting (if a little tragic … plus a bit of a grind) to assemble a list of games where the author has expressed intent to do a post-comp update but just never got around to it [emote]:([/emote]

If such a list could be accurately compiled, I don’t think it would be tiny.

[matt_weiner (matt w)]

@vivdunstan

@matt w

Mostly this is just a pretext for complaining that the poll choices are slightly misleading. The alternative to allowing updates isn’t “let us play the original versions”; it’s “make us play the original versions.”

I don’t view those as the options as such.

I’m just referring to the wording of the poll:

@Campbell

Should next year’s IF Comp allow intra-comp updates?
Yes, it’s a step forward having intra-comp updates
No, let us play the games as originally released

As I said, judges always have the option to play the games as originally released (except perhaps in the case of online play). If there were no updates, they would have to play the games that way.

@vivdunstan

More the problem for me is that people are ending up judging very different games, if the games are updated. I’m thinking of one game in particular, which has changed name and changed significantly within it (including the player character). Yet other games are unchanged. is that fair, either to the judges as a whole, or the competitors?

I just don’t see the unfairness here. Who’s being treated unequally? The judges can always play the original versions; I suppose that a judge who would’ve preferred the revision might miss out if she played the game early, but then if there were no updates she’d have missed out anyway. There’s David’s point that authors whose games are reviewed later miss out, but if judges use the randomizer then that won’t be a problem. In any case there are all sorts of unfairnesses that can arise from the order in which games are played; I suspect that judges tend to rate games more highly if they play them in their first burst of enthusiasm.

As for post-comp revisions, what Juhana and Ghalev said. Even Sarah Morayati, who cared so much about post-Comp revisions that she organized a competition for them, never finished her post-Comp revision of Broken Legs.

[Sargent]

@Juhana

@vivdunstan

Competition games have traditionally been revamped post-competition, and that system has worked well.

A small correction: It has been expected for the games to make post-comp updates, but that system has not worked at all well – most games never get a single update, even those that have serious game-breaking bugs, and this has been a major point of frustration. I assume this has been one of the reasons for allowing updates during the comp and I expect that it will produce more updated games than in the previous years.

In fact, that’s been enough of a problem that in 2009 Sarah Morayati organized a post-comp comp to encourage authors to update their games. Five games out of twenty-four were updated.

This year, twenty-five games were updated.

[Hannes]

The problem with post-comp releases, in my view, is that nobody ever plays them. At least that’s the way it seems. I entered an updated version of my own 2009 comp game into Sarah’s Post-Comp Comp and I (as well as the fresh beta testers I recruited) invested a lot of time into that update. But for what? The game’s track record still consists 100% of reviews and ratings of the original competition version. This is hardly encouraging for authors. So I like the intra-comp updates, because then, there is at least still a small incentive.

[Ghalev]

@Hannes

The problem with post-comp releases, in my view, is that nobody ever plays them. At least that’s the way it seems.

I think that’s a version of a pretty common perception. The comp is a high-energy time of attention for the games, and allowing updates while that energy is still hot can have results.

I don’t think it’s actually true that nobody plays games post-comp … obviously, some comp games have gone on to become enduring favorites (including not only “winning” games but games that received more moderate attention during the comp itself) … but I think that perception (or a less-absolute variant of it) is pretty common among the entrants, that the spotlight is Here and Now, during the comp, and then it’s gone.

I personally rarely touch a comp game until the comp is over, unless I’m a tester.

So I like the intra-comp updates, because then, there is at least still a small incentive.

Yep.

[maga (Sam Kabo Ashwell)]

@Ghalev

@Hannes

The problem with post-comp releases, in my view, is that nobody ever plays them. At least that’s the way it seems.

I don’t think it’s actually true that nobody plays games post-comp … obviously, some comp games have gone on to become enduring favorites (including not only “winning” games but games that received more moderate attention during the comp itself) … but I think that perception (or a less-absolute variant of it) is pretty common among the entrants, that the spotlight is Here and Now, during the comp, and then it’s gone.

I personally rarely touch a comp game until the comp is over, unless I’m a tester.

A further datapoint is ClubFloyd, which typically plays a good number of the higher-placed comp games in the weeks following the comp. ClubFloyd has a profile that’s a fair bit larger than its player-base might suggest, and that extends some way outside the visible community; because it maintains public transcripts, and because transcripts are easier to access than the games themselves, it’s likely to play a significant role in the perception of your work. So if you’ve placed in the top five or so, there’s definitely good reason to get a post-comp release out within a few weeks of the final results.

[bowsmand]

@Hannes

The problem with post-comp releases, in my view, is that nobody ever plays them. At least that’s the way it seems. I entered an updated version of my own 2009 comp game into Sarah’s Post-Comp Comp and I (as well as the fresh beta testers I recruited) invested a lot of time into that update. But for what? The game’s track record still consists 100% of reviews and ratings of the original competition version. This is hardly encouraging for authors. So I like the intra-comp updates, because then, there is at least still a small incentive.

Just seconding this. Same experience.

[matt_weiner (matt w)]

One thing is that, if anyone clicks on the “play on-line” button on IFDb for “Believable Adventures of an Invisible Man,” they’ll get the new version.* So that’s something.

But I agree, the post-comp comp didn’t seem to raise the visibility for the updates as much as it should have. If the rolling comp updates do that, it’d be a good thing.

*This doesn’t work for Snowquest, for some reason – I recently played through it almost all the way before I realized I was playing the same version I’d already played.

[Emerald (Emily Boegheim)]

@matt w

*This doesn’t work for Snowquest, for some reason – I recently played through it almost all the way before I realized I was playing the same version I’d already played.

I’m guessing that’s because the post-comp version of Snowquest is in a .zip file.

[matt_weiner (matt w)]

@Emerald

@matt w

*This doesn’t work for Snowquest, for some reason – I recently played through it almost all the way before I realized I was playing the same version I’d already played.

I’m guessing that’s because the post-comp version of Snowquest is in a .zip file.

But why? Why? I want my revised Snowquest online!

[Hannes]

@Ghalev

@Hannes

The problem with post-comp releases, in my view, is that nobody ever plays them. At least that’s the way it seems.

I don’t think it’s actually true that nobody plays games post-comp … obviously, some comp games have gone on to become enduring favorites (including not only “winning” games but games that received more moderate attention during the comp itself) … but I think that perception (or a less-absolute variant of it) is pretty common among the entrants, that the spotlight is Here and Now, during the comp, and then it’s gone.

I personally rarely touch a comp game until the comp is over, unless I’m a tester.

I certainly cannot judge for how many people this is true, but the issue of perception still remains. Players of the competition are likely to make themselves be heard, either in the form of reviews, short comments or at least ratings in places like IFDB. Players of later versions don’t. So anyone looking for opinions on a game will find a pack of stuff about competition release and nothing else. You might say “fair enough” and sure, it is - a release is a release. However, a newer version is also a release and larger changes in subsequent versions will remain invisible to the general public.

Another fact feeding my impression is that the people writing reviews of competition games also stop dead on November 15th sharp. Hardly anyone manages to get through all games by that and it’s not like the game aren’t there anymore, but when the judging period is over, they don’t bother anymore. I don’t understand why, but that’s the way it is.

@maga

A further datapoint is ClubFloyd, which typically plays a good number of the higher-placed comp games in the weeks following the comp. ClubFloyd has a profile that’s a fair bit larger than its player-base might suggest, and that extends some way outside the visible community; because it maintains public transcripts, and because transcripts are easier to access than the games themselves, it’s likely to play a significant role in the perception of your work. So if you’ve placed in the top five or so, there’s definitely good reason to get a post-comp release out within a few weeks of the final results.

It’s just that the top five games of the competition are usually not the ones in dire need of post-comp updates. Snowquest is one of the few such games which got an update which actually changed things (as opposed to making small technical fixes only). And even so, it obviously wouldn’t have needed it, judging from its high placement in the comp. That game was an exception, not the rule.

So I reiterate my proposal already made on the newsgroup: It would be great if the competition website would be retroactively updated to additionally link to updated versions of its entries so that people browsing the competition archives, but don’t want to research each and every game in detail using other sources can choose which version to play. Likewise, this would be great for Baf’s Guide which, IFDB or not, I’m pretty sure is still the #1 place to browse for IF for non-insiders.

[Bainespal (Paul Lee)]

Every year, I want to finish playing the games that I had to abandon because I reached the time limit; I even in some cases want to finish the games that I gave up on because I was stuck and the walkthrough didn’t work or wasn’t provided, or there was a game-stopping bug.

However, I have rarely ever made good my resolution to finish those games. Usually, after the competition finishes, my first priority is to play the games that I missed before the end of the judging period, especially if they placed highly or they sounded interesting to me. There’s a good chance that I might actually play one or two Comp games that I had not begun playing at all during the judging period – Floatpoint and Rogue of the Multiverse both fell into this category for me. But by the time I’m done playing those, I’m just too burned out to want to go back to the games that I started playing and did submit a vote for. Often I’d been playing a longer game (or had been planning to play other non-Comp games for a long time) before the judging period, and I want to go back to those game(s). Even more than that, each year I usually want to go back to my own current work-in-progress, feeling bad that I had neglected it so much during the voting period, ashamed that I once again will fail to meet my personal deadlines and show it to the people whom I had told that I would show it to. And I want to go back to my other occasional recreational interests – such as lightly playing browser-based MMORPGs and reading high fantasy novels (gotta read Sanderson’s new series some time!). I remember restoring a saved sessions on a couple Comp games that I hadn’t finished, but I simply had no motivation to try to finish those games, even though I badly wanted to. [emote]:-([/emote]

Burnout is a two-edged sword. Comp authors run out of motivation to release new versions after the judging period, and judges run out of motivation to play the same games again. This is the only aspect of the debate about the new update policy that actually makes me feel a little better about the updates. With updates, at least some of the later games I encounter may be more playable, which means I may be more likely to finish them and not have to wish that I had time to go back and play them again after the competition.

@Hannes

Players of the competition are likely to make themselves be heard, either in the form of reviews, short comments or at least ratings in places like IFDB. Players of later versions don’t. So anyone looking for opinions on a game will find a pack of stuff about competition release and nothing else.

There are SPAG reviews. If I felt so motivated by a Comp game (whether it deeply intrigued me, or just totally loved it) that I wanted to review it, I would definately make sure that I not only finished the game but saw as complete a picture of it as possible, and that would certainly include re-examining the game if a post-Comp version were released. Although I rarely write reviews, when I do, I don’t take reviewing lightly.

[Ghalev]

@Hannes

Another fact feeding my impression is that the people writing reviews of competition games also stop dead on November 15th sharp. Hardly anyone manages to get through all games by that and it’s not like the game aren’t there anymore, but when the judging period is over, they don’t bother anymore. I don’t understand why, but that’s the way it is.

I think it’s exactly the same phenomenon, really: the time of the Comp is spotlight-time, for the games and for the reviews of the games. It’s all exciting and happening and there’s a certain thrill of it being a shared, rather than solitary, experience.

[aschultz]

Now that Rule 5 doesn’t apply any more, I’ll be able to expound. Hopefully not into TLDRville.

I’d planned to update a lot more than I did, and I was worried I might be abusing the rule. I don’t feel that way any more. Because even doing minor stuff is exhausting. The stakes are high in-comp. For me, making those small tweaks paved the way to realizing the big stuff I needed to fix. I am talking about learning, understanding and implementing extensions.

Someone made the point that a person could just have the game beta-tested during the competition, which is possible…but who are the beta testers going to be? They’re all likely to be playing the games. So thorough beta-testing is not really possible. And I’d rather see a game, whether I’m competing against it or not, that is more likely to do the small things you know the writer’s talent allows instead of having moments where I’m pretty sure the author meant to mean this, or they may’ve skipped explaining something.

These sort of touch-ups are ideal, and I don’t think they were abused. And I think that while having the authors’ forums to discuss future goals helped, it was also nice to have the option to do something about it. Now it’s debatable whether the rule or the reaction to the rule spurred me trying to nail things out…but I think six weeks of helplessness can be very, very tough for an author.

And as others pointed out, nobody’s under obligation to play the latest version.

I just want to do things right. I understand my game’s subject matter may turn people off. But I think having the option to update helped. There is a lot I want to do in my game–small items or scenery that give hints, tying up the endings better, and tidying up conversations, for starters. This isn’t something that can be dashed off. I soon realized my time was better spent reading the Inform documentation that’s out there.

And I guess I have been in too many situations where people have deliberately said “We can’t STOP you doing X, but…” when of course they were trying to stop me doing it all along. There was clearly no malicious intent before the revision rule, and there is no malicious intent on the part of people who dislike it. But all the same there are rules that can’t STOP authors making needed updates when they’re most immediately motivated (e.g. right after a tough review,) but without a revision rule, there’s one less reason to go through all the steps ASAP.

I can’t say for sure whether or not I would’ve researched skeins, written up test cases, a formal test plan, or written all possible outcomes of the baseball game inside my game, or even started to look into extensions and conventions that would make my code neater and minimize testing. But knowing games could be updated was an immediate morale booster and probably spawned some of the more forward-looking discussions and posts in the authors’ forums.

[Squinky]

I just wanted to say that I as an author really appreciated the intra-comp updates as well. I didn’t feel right about making changes to the content (although I didn’t particularly mind other people doing so), but I did really appreciate the opportunity to immediately fix small errors, from typos to Internet Explorer bugginess to making the game actually playable by colourblind people. (Admittedly, being the developer of a web-based hypertext game gives me different sets of problems from most entrants, but if I were writing a parser game, I can imagine appreciating the ability to quickly fix guess-the-verb issues that my beta testers weren’t able to catch.)

[Hannes]

Hmm… this is a good point: If this rule were not in effect, how would the organisers prevent intra-comp updates of web-based games? [emote];)[/emote]

[rpatten (Robert Patten)]

If intra-comp updates are allowed next year, maybe the reviewers could post what version of a game they played.

[maga (Sam Kabo Ashwell)]

@rpatten

If intra-comp updates are allowed next year, maybe the reviewers could post what version of a game they played.

This would be okay if authors were better about updating their version info – a couple of times I checked games that I knew had been quick-updated and they still claimed to be version 1.0, so I pretty much gave that up as a lost cause. I know it’s an easy thing to miss when you’re knocking out a fast update, and that at present there’s no streamlined way for the organisers to enforce this; so I’m not sure what the solution is.