So, as I mentioned before, since you can’t moderate people voting on games they haven’t tried, it seems like the better solution would be to somehow prevent “grudge votes”. One way would be to simply ask voters to rank their top five games in order. Perhaps still allow voters to give a score to all the games if they want and display the results for posterity, but only count the top-5 rank towards actually winning.
I’d say the 2-hour limit is kind of becoming obsolete since IFComp has seems to be the major yearly competition. I don’t see a reason to limit voters that way, unless people think games will score low because they’re “too short”. Revamp the voting, and that’s also moot.
As an aside, I can understand the desire to save progress in a comp with a time limit so one can maximize their progress. Many web-based HTML games simply allow you to use the back button to rewind the story. Certain stories might prevent this ability because making choices is the only interaction and backing up to take a different branch somewhat defeats the purpose of having a choice at all. (This can also be a factor in a parser game.)
On the Storynexus platform there is no saving, but the game does remember where you are and pick up right where you left off, hence the need for a login. Storynexus as designed is actually intended for long-form narratives where you play for weeks or months at a time in short bursts as your action rate refills. I subverted that a bit for FINAL GIRL in hopes that a standard session would play from beginning to end in 30-40 minutes, so I can understand how no save in a game that actively tries to kill you off (it’s almost a simplified Rogue situation) would frustrate some people. I did build in an extra “skip” option to get the player right into the game more quickly in addition to the original shorter skip based on Emily Short’s suggestion.
I’m sure everyone has had the experience that their game feels brisk and short because they don’t read every bit of text by the 100th play through!
I understand this may sound silly, said by one who actually won the comp but… Why are we so obsessed by the ranking? Isn’t participating with a nice game enough? Is it really a matter of arriving 3rd instead of 4th? The comp, I think, has by time moved forward from the mere comp position. It’s more of an advertized venue nowadays, isn’t it?
Cool – all good design decisions with the story and the player in mind. And, you were the only one to use Story Nexus, so you took some risks no matter what. I’m going to check out Final Girl, now, that I can access the Web from home. I think being able to ‘Undo’ too easily in a game that requires tension and fear as part of the story wouldn’t work, even though it’s inconvenient to the player. I’ll save any more judgments until I’ve played it, though. [emote]:D[/emote]
I’d say this is the most important aspect of a competition. Ranking affects how people in the future will view the comp, outside of the context of any current discussions, so it is important that games are ranked according to their quality, not personal bias against how the story / game is delivered. Personally, out of the games I played, I figured Colatura had it nailed from the beginning. I didn’t try Solarium, though, and that one I definitely want to check out, after reading the reviews.
The basic deal with the comp is that the organisers do very, very little gatekeeping and no curation. There is no enforcement on what the minimum standards for releasing a game should be, or on what exactly counts as IF. And, just as there’s no standard for what you can enter, there is no standard for how you should vote.
The historical upshot of this has been that if you don’t think a game belongs in the comp at all, for whatever reason, then you give it a 1. And I think a 1 where the voter says why they handed out a 1 is more useful than one where they don’t - that way the author can, if they choose, go ‘y’know, I don’t actually care about that voter: clearly they’re not my audience, and I can feel free ignoring them’ - which is a better position to be in than being aware that a bunch of people hated your game and never said why.
Ok, well, here’s a practical idea. Include a checkbox when voting: “Is this an auto-1 vote for reasons of fundamental disagreement about something this game did?”. Save the auto1ers some time.
Then generate two lists. Make the official list the one with the auto-1s, but generate also a second list WITHOUT those so we can see that as well.
I didn’t say it was a good idea, but I’m so tired of this whole argument by now it seems the only proper thing to do. Let 'em vote as they want, we’ll still count all the votes and all, it’ll still matter, but we’ll also see what it would have been like if they didn’t auto1.
Incidently, I would be very curious as to the results we’d have THIS year if we could strip out all the 1 votes. The seriously bad games would be left either blank or with votes of 2, so no loss there. We’d lose the white noise everywhere else.
And this is to all you auto1ers out there: yes, WHITE NOISE, that’s what your protest is coming down to. You’re inflicting more pain on us than you did on the author you auto1ed.
That’s a little bit ‘you can vote however you want, but if you’re a registered Communist, place your ballot inside this special box.’ Nobody’s going to agree to having their votes potentially ignored.
(And 1-voting isn’t the only way you can protest-vote. Distinguishing a protest vote is not really something you can do without hearing the voter explain themselves. And even then you might have trouble.)
Nobody seem to agree that it’s good practice to auto1 on this scale either.
Scratch the checkbox then, and ask for justification for everyone that votes 1. A little text field. “If you’re voting 1, please say why”. Then we’d know how many are “legit”. If people really want to be asses about it they’ll start voting 2 just for spite, and by then hopefully it’ll be clear how stupid the whole thing’s gotten.
Heck, I’ll do you one better. Include a “zero” rating, reserved for people who disagree strongly with something about the game. They get their opinions in, their protest is noted. It won’t lower the game’s average but it would have a note saying “30 people rated this ZERO” with an optional field to explain why for those that do leave their reasons. They’re heard, we can sort them out, we can acknowledge their point of view, games’ ratings aren’t affected.
Here’s an even better one. We all acknowledge there’s bugger all we can do about this situation and keep going as we are because it’s the best solution really, flawed but workable, just like a Democracy ought to be.
I’m quite tired, in case you can’t tell, and it makes me snarky as all get out.
EDIT:
You edited that in after I’d read your original post, you naughty person you. But as it happens I think I addressed that as well.
I understand what you’re saying, but we seem to have a new situation with the surge of CYOA and web games, and an apparent bias by a certain faction against these. I’m all for someone who plays a game and hates it giving it a 1. I don’t agree that I should be able to go through and pick five or more games by people I don’t like or in a format I don’t think is valid without playing them and give them all 1s as a protest-troll that they shouldn’t be in the comp.
The best placed choice game (Solarium) has four 1 votes. Machine of Death only has one 1 vote. Bell Park has two 1 votes. So while I don’t even see the group hating choice games in general in the votes, let’s just be generous and say those five people who gave Angelical Understanding a 1 define this faction. Five people. Honestly, folks, is that really worth discussing? Focus on getting more people to vote on the whole and this number will be so irrelevant that we will all look back at this thread and have a good laugh.
Well, hm. I dunno. I don’t like schmups. I am never going to like schmups. This means that I’m not a very good audience for schmups, and so I don’t play them.
I certainly shouldn’t go to schmup-centred places and give all the games 1 ratings. If I did, it’d be a sensible response for the schmup-site admins to ban me and throw my ratings out.
But if a friend of mine makes a schmup, puts me in front of it, and says ‘Okay, so what do you think?’ then my correct response is going to be ‘I hate it, because it’s a schmup.’ The correct response is not for me to waffle and shrug and not give an answer; nor is it to say ‘I hate it’ and not share the detail that this is because I don’t like schmups. I shouldn’t be a dick about it, true, but if my friend keeps showing me schmups then I should probably be a little bit more forceful: I have reasons for not liking schmups, and my friend should not expect me to become a schmup afficionado.
And, awkwardly, the comp’s somewhere in between these two situations. Comp authors are putting their games in front of the voters and saying ‘what do you think?’ But at the same time we have an expectation that the results will be informed, fair and reflective. I’m not entirely sure how you reconcile those two.
If you want a comp where the voters are held to a higher standard, then I think you need to go ahead and recommend closed-panel judging (and that’d probably require more stringent standards for authors, too). If you want an event where only positive votes count - well, that’s the XYZZYs. If you can find a way to prevent nasty reviews - which is to say, a way to get people to stop being obnoxious, opinionated assholes on the Internet - then you’ll deserve a Nobel Prize.
I agree, although the fact that the majority of entries seem to take considerably less than two hours to play makes me reconsider a little.
We need the 2 hour rule because otherwise the unspoken obligation on judges/reviewers would be too great. If there were no time limit, I would feel a need to finish each game no matter how long it is, unless I reached a game-stopping bug or got stuck beyond help of hints or walkthrough. The 2 hour rule defines a maxiumum expected commitment for responsible voting. If you gave the game the 2 hours, it’s no longer your responsibility if you haven’t come to a full understanding of the game. Finishing the game is still my standard for reviewing, which is why I didn’t review the few games I voted on but did not finish playing.
(I should say that my ideology doesn’t always align with my actions. Sometimes during past Comps, I’ve given up on games before running out of time, simply because I was having a bad experience or felt too lazy or burned out to continue. I voted on those games, but I didn’t automatically give them a 1 or anything. I can think of two games that I gave up on in different years, and I think I ended up rating both of them at 6.)
Without the 2 hour rule, a Comp full of longer games would exhaust me and burn me out. The Comp starts at a fairly busy time of year and ends during an insanely busy time of year.
However, the Comp isn’t really full of longer games anymore. This year, I badly wanted to continue playing Threediopolis and Robin & Orchid after hitting the 2 hour mark, because I was having a lot of fun with them. Spending two hours on both of those games was less tedious than spending 15 minutes on some of the Twine games, which I saw value in but generally did not enjoy. I felt bad that I didn’t review either of my favorite games from the Comp, but I still feel that I can’t review something that I haven’t experienced reasonably fully.
(Actually, now that I think of it, the reason the short Twine games felt so tedious to me was probably because I was forcing myself to review them.)
If we are thinking of eliminating the EFFECT of auto1s, I have an idea. Have preferential voting and compute the winner by a condorcet method. Preferably the Schulze method. In that case, you can’t bury a game.
As someone who posted a list of reactions which referred to online-only games going to, quote, “the bottom of my list”: I went through what I managed to play mainly on a portable, non-webbed computer during my daily bus and train time. I used the randomised list, but since playing the online games was impossible during the chunks of time I wanted to primarily dedicate to the competition, they got skipped and moved to the end of that list. The intention was that I might slowly tackle them if I should manage to get through the list of games I could easily play and find the time and motivation to do some more at home. I never reached the end of the “commuter-available” list, so I didn’t get around to the online games.
I guess if we look past the comp-technical sense of the word judge, then I did to some extent ‘judge’ the delivery method: I mentioned it as an obstacle to my seeing the content. I intended no valuation other than “this is in a format which happens to be inconvenient for me”. It was never about the online games “wasting my time”, but about their demanding a type of time slot I preferred to dedicate to other purposes.
Also, would you mind not godwinning something as irrelevant as some games not receiving a score from a stranger on the internet? I do realise that in current American usage Nazi has come to mean something like ‘a person I don’t particularly like’, but like many other Europeans I’m a tad sensitive to the historical implications of the expression.
I expect Reels suffered from being a terrible game to begin with. With the kind of production values going on there in general, reviewers may have been more inclined towards thinking “this is unplayably buggy” rather than “this isn’t working on my specific setup”. There’s probably also a subjective difference between a game which never launches in the first place (“Ah well, I guess I can’t rate this then”) and one which starts and then breaks down (“This seems severely buggy and shouldn’t have been in the competition in the first place”).
But maybe web browser games are indeed held to a stricter technical standard. With the IF virtual machines we’re historically used to some degree of grief, like certain games requiring specific versions of Adrift. But whatever comes through a web browser is by default expected to work on many different setups, even if it’s not your basic web page.
If my stats are anything to go by, then it’s not particularly good advertising. If anything authors are going to lose readership because they aren’t allowed to promote their games to the outside world, where there’s a way way larger audience for interactive fiction. I don’t think that’s the end of the world though. As you said, the best part of the comp is that it gives people the opportunity to do something creative and participate in an artistic conversation, and that can happen with a small audience as well as a big one.
That said, I’d really like to see that “no talking” rule changed. Authors would be the most enthusiastic advocates for the comp you could ever hope for, and I’m sure they could bring in a lot of new players if you set them loose on twitter and whatnot.
The original point of this thread was Hanon pointing out the low quality or stupidity of some IFComp reviews. This is a thing that bothers me and I bought up Morayati’s review because I think it’s a classic poor example. And I will tell you that in the authors forum, people were pretty furious about it. I almost brought up Yoon Ha Lee’s reviews, plural, as well, but I thought ‘one example will suffice.’ David brought those up one post later himself, which doesn’t surprise me - like I say, they were on the tip of my mind. If somebody thinks Yoon Ha Lee’s blogposts about IFComp games have been illuminative or helpful to the project of criticism, or IF games (they’re certainly not helping the authors, who, again, complain about them in the authors forum) please begin to work your way through them and explain how.
I think later someone suggested these point outs were men ganging up on women. I’d be pretty incensed at that suggestion except that we’ve all promised to stop being offended every 5 seconds. I called out the worst criticism I’ve seen because I care about quality criticism. Very clearly, I do not care about the sex of the writers of it.