What I Learned From Reviewing Half the Comp: A Reviewer's Postmortem

My decision to review the 2021 comp wasn’t a sudden one. I’d rated the 2018 and 2019 comps, without publically reviewing them or otherwise making my presence known in any way. I had seriously considered posting reviews in 2019, but I was dissatisfied with the first review I wrote and never posted it. But that isn’t even the start of it. The fact is, I’ve been following this community on and off for more than half my life now - ever since I discovered the text adventure on a Wikipedia rabbit hole.

While I might have been stalking you all for a long time (not as closely as I’m making it sound, and there were long periods of disengagement) I’ve only started talking to you all recently. So what compelled me to announce myself by posting nearly a review a day for six weeks?

I think partly it was a desire to really get a clear picture of the comp as a whole - or, at least, a clearly defined cross-section of it. In 2018 and 2019, I missed some of the biggest games because I followed the shuffle and never got round to them. I wanted to do better this time. Although I didn’t express this to myself at the time, when I decided to go in reverse alphabetical order I was launching myself into a challenge: get from Z to A.

I did get from Z to A. I reviewed thirty-six games, according to the spreadsheet, plus one long post-Comp review. I’m not sure it’s given me a huge amount of insight into “the state of the community” or anything like that. But it did teach me some things about myself.

I Can Be Really Mean, As A Critic

and sometimes I can be kind! But I’m also capable of writing viciously, not all the time, just when a game tries my patience. This is a contrast to the trajectory of the community, which has on the whole become gentle in its approach, because you don’t want to discourage new authors! And that’s a noble thing, so it makes sense that the community didn’t seem inclined to encourage me in my occasional turns towards diatribe (with one exception).

I’ve regretted some of the crueler things I’ve written. For example, my Unfortunate review closed with the line “The most Unfortunate thing of all was that the author decided to release a game in this state to the IF Comp.” I stand by my opinion that Unfortunate was not ready for publication because even a single playthrough would have exposed its game-breaking bugs, that it should not have been submitted to the Comp in that state, but that line was just cheap. It was a weak reference that detracted from the review and which I only threw in at the last minute. On the whole I think no-one benefitted from that.

In other cases, though, I think my harshness was justified. I think especially of Kidney Kwest. I went in hard on that game because I felt its “pretend not to be a computer script” approach was so appalling, not just in IF terms but in general UI terms, that it needed to be denounced loudly. My review of Kidney Kwest didn’t get much engagement, but I think it was a great review and I’m still proud of it. In general I think it’s good to keep a robust element, as well as a gentle element, in the body of criticism the IF community produces.

The exception to the rule that my pissy reviews didn’t get engagement was my review of Smart Theory. That got a lot of likes, probably because I took a political position people here sympathized with. I hope I don’t get put in a box on the basis of that review. That stuff I said about wisdom isn’t just something I made up to justify left-wing priors; it’s my actual worldview. Honestly I was embarassed that my Smart Theory review got so many likes, because I didn’t think it was that good. I stand by everything in it: it’s just not well-written. I was really writing myself into a lather there.

I Felt Insecure And ‘Showed Off’

As I’ve said, my reviews this year were my first real contribution to the IF community, but I have been engaged with IF much longer than that. And I think there was a part of me that wanted to prove that I knew what I was talking about, that I belonged here. I think sometimes this made my reviews come off as ore confident, which is basically good. But it sometimes made my reviews weaker as I went out of my way to talk about “inside baseball” stuff.

For example, making it clear in my Ghosts Within review that I had read the TADS 3 Technical Manual. (Although, I think that was an insightful paragraph.) Or consider my Codex Sadistica review. Every other review of that game pointed out that it was under-implemented. I went on at length about how I knew it would be under-implemented because of the combination of its system and format. I then went on to infer that the writer deliberately targeted that format because it’s easier to do colored text in it, which was presumptuous of me. There was no call to make guesses about the author’s intent like that.

There are some other things I said which were presumptous even if they don’t fit that pattern of ‘trying to display IF systems knowledge’. My discussion of Pseudavid’s writing style (in a positive review!) probably came off as condescending, for which I apologize.

I Found The Texture (Prose, etc.) Of A Work Had A Large Effect On My Experience Of It, But Often Lacked The Critical Language To Explain Why

Nothing more to say about that, really. I kept trying to talk about prose and it came out really weak. I’ll never be Sam Kabo Ashwell but I want to get better at this.

So, How Did Maintaining That Schedule Feel?

Alright, I guess? Having to produce reviews at a fast rate was interesting. I don’t think it compromisd quality; in fact, I think it provided focus. I would occasionally have to come up with a review for something about which I really felt nothing, and I don’t feel like those reviews were particularly strong. I’m not sure whether it was a quality of the specific games or if I was just tired. In fact, it’s possible those reviews are fine and I just feel negative about them because of the circumstances.

But I wrote more words, many of which I’m happy with, than I really thought I could per day over a sustained period. It’s not that many words; I haven’t calculated the exact amount, but there were many people more prolific than me (shout-out to Mike Russo). But it suggests I can get other writing projects done with the right schedule, which is encouraging. I did have to sideline some other interests to match the schedule, though.

Do I want to do it again next year, though? …you know, that’s interesting. I was going to write about how it wasn’t always fun to do this and it sometimes felt like a joke, having to pit all these wildly different games coming from wildly different places against each other, but looking back on all the writing I did over those six weeks… mostly, what I feel is satisfaction. I’m satisfied by quantity and also quality. I think I did a decent job, all things considered. Nonetheless, I may want to give a more restrained set of reviews next year; I have not decided exactly what I plan to do or how I plan to do it.

But if I do do it again, I’ll be armed with something I got from this excercise: I have a pretty good idea of what constitutes which number scores for me now. This makes it practicable to rank a >2 hour game midway through and keep playing, which should allow for more comprehensive reviews and also lead to me engaging more deeply with 2+ hour games. I found that I tended to disconnect from the games listed as more than two hours well before my two hours were actually up, knowing that I wasn’t going to get any closure during the comp period.

Anyway, thanks to everybody who engaged with my reviews, even just by giving likes. I appreciate the kindness this community has shown me. I do think this was a strong Comp (though I find several aspects of the results baffling). Here’s hoping the world of IF has an equally good 2022!

26 Likes

You gave me my very first review (of What Heart Heard…), and I thank you for it. I found your criticisms spot-on and very useful, and that will help me grow as a writer. I had so much trepidation going into the comp and that first review of yours allowed me to breathe a little easier.

And how my game placed was indeed a baffling result. No one could be more baffled about it than I am.

I read all your reviews and found them all interesting and well-written, even if I didn’t agree with some of them. The only one I found “mean” was your not-review of The House on Highfield Lane. I was a tester for that game, and it’s a great, old-school parser game with some really fun twists and mechanics. If you don’t want to play something, I don’t think you should feel like you have to. But I think that not-review may have cost Andy some players and votes by being negative about how it appeared, not about how it actually was, since your reviews clearly had lots of views.

Of course you can do and say anything you like, but that one struck me as unfair to the author.

5 Likes

I really appreciate your reviews.
IFComp 2020: I wanna do many reviews and started for short games. I played to the end a big number, 10 or 12, in an afternoon session. This didin’t feed my brains.

IF Comp 2021: I played and reviews a few titles +2hours. I liked these games so much. As I readed almost all reviews, I finally played some other interesting short games and thankfully left out some others.

I am happy about this year’s IFComp.

When some people read a review and comment it, the forum gets richier and richier with everybody experiences.

3 Likes

It’s really good to see a reviewer postmortem. I don’t know if we’ve ever had one before. I mean, I’ve seen summaries of all the entries and generalizations of overall trends, but not enough “this is what I learned as a reviewer.”

So I think posts like this are really important for community building, and that’s not just a buzzword. We can say “Oh, that person wrote 30+ reviews, they must be an advanced expert” and leave it at that. But there are a few of us who would like to write a few good solid reviews and wonder about quality vs quantity etc. And so I hope my additional thoughts may help build the community more and don’t deviate too far from your comments.

Because the fact is, those reviews don’t happen in isolation. It’s good to know you took a year or two before jumping in. It’s also good to know you wanted, and want, to do some things better. I really do think that the formula for becoming an expert writer is, well, writing a lot and saying, I’d like to do better there.

First, anyone feeling they only wrote, say 1/3 or 1/4 or even 1/5 of the reviews? Well, back when IFComp had 25 entries, 1/3 was the complete slate. So I’d say anyone who thinks of it as a fraction, or thinks of what they haven’t done, is doing a great disservice to themselves. Set a low goal and a stretch goal.

my mini-review postmortem

I know I certainly had questions about my own process trying to review all the games this year. I don’t think I’ll do so next year, and I’ve felt guilty about which games I will dis-include and why. But I think there’s a strong case for saying up front “this entry is a bad matchup. I don’t want to auto-1 it, or to give it a review out of obligation and maybe rank it lower than I should.” I have a few such rules in place next year where I will heavily deprioritize entries. It may lead me to avoid certain higher-ranked entries for lower ones, and I am very okay with that. Reviewing should be about having something legitimate to say, and I think rejecting certain games helps maintain quality control!

I feel similarly to you that writing reviews helped give me confidence I could write more. Sometimes it feels like I wrote nothing this year, but having that topic in the authors’ forum that spanned 100kb+ of words made me realized–I most certainly wrote something! And I think writing for quantity is valuable, at least to start. But it has diminishing returns to scale.

Certainly my approach to a game, even if I viewed it unfavorably, is that I want to make my expectations clear and mention what can be improved or even what you should ignore to enjoy a game nonironically–and if a lower-placed entry does have an obvious fault you can ignore, you might really, really like it. I have to admit I’d feel bad if I reviewed 65 games and left 6 hanging. But I also think having a set of rules to sort games into buckets will help next year.

I’m gonna quibble here. “I’ll never be X” is a different statement from “X does a lot right I’d like to learn.” The first can give you an artificial ceiling. I know looking back at communities I was a part of 10+ years ago, there were writers called impressive and “you can’t write like them” and looking back, I noticed the affectations they used that puffed their writing out a bit. In those communities, Y was also enshrined as a “total legend” but it was maybe their rhetorical tricks or message-board personalities.

I don’t have a problem with “inside baseball” because even if it’s maybe not on point, it’s informative–especially if you are on the technical side as opposed to (straw man example) “Oh, I know Writer X’s weaknesses! They don’t ever really improve!” So there’s inside baseball and there’s “look at me, I know all this inside baseball you don’t!”

Yet you have to try for this inside-baseball stuff, though, because it can 1) tip things off better things for you to write about and 2) make readers say hey! I’d like to check that out some more! Some people do, in fact, write to show they are really in the know, or they remember stuff from before your time, but you never got close to that. I think it’s a great idea to be up-front about how you may concentrate on technical stuff more than most. And at any rate, just by writing reviews, you’ve probably already learned what’s worth showing and what’s just showing off.

And next year you DEFINITELY won’t have to prove you know what you’re talking about! I think it’s very natural for first-time reviewers to need to show their cred, but … I don’t think it’s emphasized enough: there are not and should not be high barriers to reviewing. A review that maybe doesn’t say a ton is better than one that says “why did you bother?” and I don’t think I read any reviews like that this year. It seems it happened a lot early in the comp. Trolling via reviews requires considerable energy, and with social media the way it is, dedicated trolls will probably use that energy elsewhere.

thoughts on harsh criticism/Unfortunate & the sports metaphor I use

You mentioned your Unfortunate review. I don’t think you were the only reviewer who made a word-play on “unfortunate.” I’d like to mention a trick I use before dropping the hammer on something, for use in general. It may not work for non-basketball fans, but it works great for me. I’m sure you’ve heard the expression “Oh, X dunked on Y.” Which, yay for X, I guess. Maybe Y allowed the easy dunk because of the writing equivalent of bad or weak defense. But if X hung on the rim or taunted Y after, well–Y didn’t make X taunt or hang on the rim, and that’s a deserved technical foul, and X should be called on it, and if X does so repeatedly, well, they foul out. So I try to picture if I am hanging on the rim or not.

(To extend this awkward sports metaphor, I really do appreciate entries that throw me a perfect leading pass so I can dunk on something that had bothered me in the past.)

pointy-head technical hat time to get your word count quickly

You could count the number of spaces a variety of ways, and that’d be close to the number of words! The quick way to scrape everything (at least on Firefox) is to hit ctrl-P then cancel. I get a text version of a topic, which can be highlighted (ctrl-a) then pasted to a text editor where the non-review posts can be weeded out.

Notepad++ (if you have windows) counts this, and it could also search-and-replace spaces and say “you replaced 40000 spaces.” Or you could have a file of all your reviews, replace spaces in a generic text editor, save a backup and see the difference in bytes.

Oh–one other thing I might recommend for you or others who get out to good quick starts? Make a 2nd post in the topic that says “like this post if you’d like a review. This will bump it up in priority, because I know the review will be read by the author.” I did so in the authors’ forum and it seemed to work well. Sometimes having a handful of things to focus on helps. I don’t know how that would conflict with your anti-alphabetical order, but I also think it’s useful for me to be able to deviate and go off on tangents and pick off a game on the side.

Finally, I know it’s tough to even write 5 reviews. It was good to see all of yours, and I appreciated having yours early for Walking Into It. I wasn’t expecting many reviews for it. And when I saw you had a bunch of reviews up early, I thought–hey, good! This sort of thing sets the pace for others to try and write a lot.

5 Likes

I really enjoyed reading this too. And I really enjoyed your reviews. I think I’ll muse on posting my own reviewer’s post-mortem. I think reviewers made a really valuable contribution though, and I felt very fortunate to take part in that this year.

1 Like

I didn’t write enough reviews to justify a postmortem, but I do have a few responses to your thoughtful words.

I’m a firm believer in writing things down: My supposedly insightful opinions on any subject are actually thin and putty-like when they’re sitting in my mind. Abstract sentiments harden when exposed to air. Writing words down is like writing a contract (even if I’m the only party in the contract). Putting words out there for others to read is like reading a contract on a stage before an audience. It forces me to knuckle down and ensure my words express what I really mean.

(In other words: I’m certain I get more out of writing a review than any reader. I don’t know if that’s true for you.)

I think this is brave to admit in a public forum. When I review, I try to remember that even if a work doesn’t seem finished, a helluva lot of time and emotional energy probably went into it. (That’s true for a lot of mediums.)

That’s well-put.

I agree, but there’s a tightrope being walked: To evaluate a work, you have to at least attempt to understand the author’s ambitions, and that involves surmising their intentions. You can’t get around it without producing a bland, surface review.

I knew a guy who worked for an early streaming music site. His job—a dream job, or so he thought—was to write hundreds of capsule reviews of heavy metal and classic rock albums. The job nearly drained his love of music entirely. But, his writing skills improved in ways he never expected.

Someone else said: “All writing is practice for your next round of writing.” I agree.

– Jim

7 Likes