AI in competitions

Competitions
82

I recognize your point! And really sorry if it wasn’t obvious, but I suggested people think about comps not needing cover art more as a thought exercise. This is more for historical background than me being obstinate (and you’re probably aware of this), but people made decisions to add in all those things to our comp sites at some point. None of it’s inherent, it’s whys all the way down. I used to just download a comp zip file and then play stuff based off reviews or even just off the title, back in like, 2002 or something. But IFComp at least has grown too large to go back to that, and we wouldn’t want to go back anyhow. I’m not remotely interested in removing all cover art, I’m just considering how games without cover art are treated and displayed. That can be changed if we want, just like the AI disclosure change.

The question about what implicit expectations we have or don’t about cover art is another topic, but hey, if some people just start thinking about it, that’s fine. I worry about there being an accessibility/barrier of entry issue if we expect cover art, but if people generally agree that not having cover art doesn’t effect entries in the long run of a competition after all, then it’s an unfounded issue.

The only change I’m going to suggest is for IFComp to look at how Spring Thing handles no-cover-art entries in their post-competition survey, and see if they can do something like it in the future. I think that’s a pretty benign ask. If anyone else thinks that’s a good idea, or hey, for anyone that disagrees, that’s what the survey’s for! As apparently this AI change might have also partly resulted from last year’s survey.

3 Likes
83

And how do human writers train?

You could say that we all consume works, learn based on what we read and then write with all the knowledge we gathered from it. But this seems like just boiling it all down to the lowest common denominator. I won’t ever be force-fed thousands upon thousands of various works – AI will. AI won’t be selective about the material it consumes because it’s been chosen for it. AI won’t ever say “I need to learn how to…”; what it learns is decided for it. We both take various works and learn from them, true, but the difference is the amount of material we’re capable of consuming in a specific timeframe and the intentionality between what we take from it.

This is why I just can’t get behind AI-generated works. There is no intention there, no agency, no careful selection of words. AI fails because it doesn’t develop its own voice and doesn’t tend to its skills. Even if the writing is grammatically immaculate and is put together in accordance to a vague idea of what “good writing” is (and it’s usually just an attempt at purple prose), it’s just a bouquet of false flowers: a pretty, dead trinket with no history.

And while I can’t always tell what’s written by an AI or not, I can still feel a certain sense of unease while I read AI generated passages. I start to wonder if this is someone’s legitimate writing style, which I often attribute to people who didn’t really find their voice yet and just put together sentences that their English teacher would approve of, or not. Because if it’s a human that writes like this, then there’s a chance that they’ll change down the road, and find their voice, be it evolving towards more intentional descriptive prose or something simpler. If it’s an AI, there’s no evolution to talk about. I think this is why we should be labelling AI usage in writing – I’d rather know if I’m reading someone who’s genuinely trying or a machine that’s incapable of trying in the first place.

8 Likes
84

I don’t think itch’s behaviour has changed, I think what you see depends on context.
If you look at the REALLY BAD IF jam’s front page, or at one of the contributors’ own pages, you see the grey star; but if you look at the jam submissions page, you see “no image :(”.
(So itch is seemingly frowning at lack of cover art specifically in the context of competitions/jams/etc.)

4 Likes
85

Disclosure avoids unsubstantiated claims that a work has or has not used AI, which can be damaging to the community. While some players might be more tuned to how AI output feels, it’s not a fail-safe method (especially when the author is ESL*). And you can’t always tell based on the blurb alone.
So it allows users to make an informed choice before playing.
* not in this Forum, but I’ve personally been accused of using ChatGPT in my work, even though I’ve never opened the website, with arguments that the style of some of my passages were too similar to ChatGPT’s (purple prose, use of “fancy” words, etc…).

This change was carefully discussed, by the way. It wasn’t just a reaction to a request made in the feedback form, but also looking at the IF community’s response to AI as a whole even before the feedback was received. As the standing of AI evolves, both in the community and legally, this rule and how the IFComp deals with AI-generated content might still change in the future.

If anyone has feedback about this rule specifically or any other aspect of the IFComp, please do fill in the feedback form shared at the end of the competition.

11 Likes
86

Regarding another competition, this is exactly what was called for above, with a harsh punishment being the method of preventing lying. While I understand the motivation and even agree with many of the arguments, I would take a more clement approach. And the idea of punishment isn’t off topic regarding IFComp either since as it has been pointed out, people can still lie about disclosure itself.

I am reading many people saying this. I’ve probably said it myself. I just don’t think it’s particularly relevant to the issue as others have argued better than I could.

1 Like
87

Just want to echo PBP’s point about usage of Derivative works and Fan Fiction. I too am grateful that the organisers allow this, since this is also an IP issue. I am currently working on a fan fiction entry. Of course, i have disclosed all my sources and given the appropriate credits - which i also do with any “free” material i use (regardless of obligation). Because this is the right thing to do. But i do notice that there is no such actual requirement in the rules.

4 Likes
88

I’ve personally been accused of using ChatGPT in my work

Yep! I’m with you on this. I have been accused of using AI art when I never have. This is going to be a problem when (and if) AI art gets better (in the sense of not making telltale errors). “Oh, this could be AI art, perhaps i won’t play it.”

I’m in favour of disclosure, but my only gripe is that i think it’s rather appalling that the organisers have slammed this new rule in now, just 3 weeks from the deadline.

1 Like
89

I disagree – this is a disclosure requirement only, over a topic that’s already caused a bunch of contention in the community. Nothing has changed in terms of what’s allowed in the comp.

We already saw a preview via ParserComp of how this would go without the disclosure requirement. As someone who wasn’t otherwise involved (participating or playing) most of the discussions that spilled into my awareness were more arguments about the use of AI. A couple games did use AI with varying levels of disclosure, and while @Joey was cool-headed enough to discuss this only after voting was over I doubt most people would be that polite in a larger comp with higher stakes. I could easily see a path for an IFComp without this requirement where the discussion focuses around accusations of AI use, merited or not, and the whole thing descends into witch-hunting and arguments.

This way, we can hopefully “live and let live” and go on to focus on the merits of the games as they are. My one hope is that the IFComp committee has a plan for dealing with any speculation about which games without a disclosure may have used AI, and that there’s some reporting mechanism that is as private as humanly possible. I say this not because I think people will violate the disclosure but because this is a sensitive topic (see also, every single AI thread on this site), and I can’t think of a worse way to deal with the new rule than having enforcement be triggered entirely by public accusations.

In conclusion, I’m in support of this rule and I think it’s more of an attempt to get people to play nice than it is to litigate what is and isn’t allowed in the comp. We’ll see how it plays out, and I’m sure there will be tweaks for next year based on how it goes.

12 Likes
90

Oh, huh, you’re right. Thanks for the correction! I guess I just haven’t spent a lot of time looking at jam submission pages.

2 Likes
91

I’m not sure what the committee could do that isn’t “more speculation”. Nobody’s going to break into an author’s house and look at their development tools.

1 Like
93

Sorry - to clarify, I mean an avenue to prevent public witch-hunting for “breaking the rules”, rather than to actually prove or disprove that someone’s using AI. I think the disclosure requirement should help that massively, but IMHO I think it should be paired with “and if you think a game is breaking a rule you should do Y action and not put the author on blast on the forums.”

Probably a reminder of the usual avenues to report rule-breaking behavior should be sufficient? Which I believe is emailing the IFTF? Recent years haven’t had too much going on in terms of rules kerfuffles so I haven’t needed to keep up. Hopefully this wouldn’t result in a deluge of emails for the comp runners, anyway.

5 Likes
94

Emailing the IFCOMP organisers :wink:

6 Likes
95

Yeah, this seems right to me, and is also/primarily something we in the community should be mindful of too - like probably worth a DM if we see somebody doing, or getting close to, speculation about AI use in a public review.

5 Likes
96

They had special mailboxes for denunciations in the Republic of Venice 400 years ago. I think we could do as much at least, in this day and age.

Seriously though, do we really believe there is a chance we can put the genie back into the bottle? As I was trying to illustrate, all attempts run the risk of creating more misery. For someone determined to break the rules and put some effort into it, there is no real risk of consequences, whereas the honest contestants have to put up with the instruments of enforcement. And for what, exactly? If someone comes out with their AI-use after the competition to make a mockery of all of us that are trying too hard, I say let them have their moment, maybe even laugh with them. If you can’t / won’t trust your fellow hobbyists to be above actual malice as opposed to the odd practical joke, it would probably be healthy to move on to some other hobby.

And on the other hand, what possible (sensible) reaction should follow if we really don’t recognize the winner of the comp as the product solely of a machine, and the fake author keeps quiet about it?

1 Like
97

I don’t think I’m the only one who would prefer to play games that the author put effort into it, and understands what’s happening in the game, because they wrote it.

Yes, admittedly, I don’t understand what’s going on in my game right now, but all of those problems were code purely written by me. I’m just a doofus using an old programming language for something that is way too complicated, and that’s my problem. I still wrote all the code.

5 Likes
98

I agree with your sentiment, but I am not prepared to participate in useless witch-hunts trying to enforce it.

This sums it up nicely. If people acknowledge their use of AI as per the current rules, I’m free to play their games or not. If they don’t, I can either notice they’ve secretly broken the rules, and again I am free to stop playing or keep playing. If I don’t notice their treachery, I’m a fool and deserve to swallow what I’m fed.

3 Likes
99

To be fair, there’s probably a chess variant not far removed from it.

100

The harshest "punishment’ available in IFComp is disqualification. They’re not saying AI disqualifies someone, only that they’re providing easy check-boxes on the submission form to notify if AI was used so judges can avoid it if they’re not interested, same as an adult-content warning.

Game disqualification has has historically been handled subtly - usually it’s a case of a game being released before the Comp, and they will post a blog entry announcing it and remove it from the voting page. It’s not as dramatic as some people are expressing and doesn’t necessarily make an author a pariah if it’s an honest mistake or mis-read of the rules.

The only thing above disqualification is being banned from IFComp, but that is mostly reserved for those who egregiously and maliciously interfere with the rules or other authors’ entries where it causes problems and isn’t something that can be settled so organizers aren’t convinced it won’t happen again.

I meant “qualitatively bad” - AI prose is most usually grammatically and structurally correct; the prose is usually “competent” even if gives the Spidey-sense you get reading AI prose. People wouldn’t use AI generated text if it was functionally “bad”.

In hopes of encouraging this issue toward conclusion, the fact that IFComp is incorporating this means they are being inclusive rather than exclusive of works that employ AI and their goal is just transparency and disclosure.

8 Likes
101

I’ve been thinking about what the difference is, too, and this is similar to where my own train of thought was going:

Humans identify specific components in certain artworks, and try to understand why the artist chose to leverage these as part of the expression and intent, and how this component might be generalized into a skill for new works, which are created for expression with intent.

An “AI” (as much as I hate this marketing term) will examine a massive body of work, and identify statistical patterns. These patterns are not the same thing as tropes or components, because the system is examining the body of work on a much more granular level, without any awareness of tropes or components.

So a human creates from studied experience and direction, which requires intent and understanding.

An “AI” simply chooses a chain of catalogued grains, according to what is most statistically-likely, and keeps going until statistics say it should stop, or it reaches some other boundary. This means it tends to replicate the average trends of the body of work it was trained on, with some deviations from the user, but none of this done with any awareness or functional structure around the idea of skills or planning or methods of expression.

You might ask ChatGPT to analyze the tropes of a story, but what it tells you will either be wrong, or will be a statistically-likely assembly of words from a range of other art critics. This does not indicate that the internal system actually has awareness or a functional structure for these ideas.

I don’t think that it’s impossible for a machine to replicate the human brain, 1-to-1. I don’t believe in any special quality about the human brain which separates it from a pure, computational space.

However, the “generative AI” of today are still nowhere near the functional class of human brains, both computationally and structurally.

They’re just really good at tricking us into thinking they are.

13 Likes
102

@manon Thanks for explaining things further. I still think my distinction between organizers’ concerns vs community sentiment holds.

This change was carefully discussed, by the way. It wasn’t just a reaction to a request made in the feedback form, but also looking at the IF community’s response to AI as a whole even before the feedback was received. As the standing of AI evolves, both in the community and legally, this rule and how the IFComp deals with AI-generated content might still change in the future.

An evolving policy based on the community’s stance is what I’m afraid of, since the community could be become very harsh very quickly.

:exclamation: I’d like IF Comp to fully describe its stance on AI use outright rather than just attributing it to direct and indirect pressure from the community.

The news post suggests the disclosure rule is just a convenience for audiences (“This information will be shared with judges, so they can make an informed choice during the voting period.”) However, based on the current and past threads — including your own comment, @manon — I’m under the impression that concerns around reputation and public accusations influenced the decision.

On top of that, if IF Comp agrees with broad ethical and ideological concerns (risk of theft/plagiarism, discriminatory content, devaluation of human labor), it should say that outright. Or, if it is neutral on AI use, and it’s meeting community demand simply because it serves the community, it should say that outright.

If there are legal or practical concerns, it could mention that, too.

I don’t know if this is the sort of thing that is done by IF Comp organizers or at the IFTF level, but that’s what I’m getting at.

@Encorm I think it should be paired with “and if you think a game is breaking a rule you should do Y action and not put the author on blast on the forums.”

@deusirae probably worth a DM if we see somebody doing, or getting close to, speculation about AI use in a public review

I support a ban on public accusations. Without it, I don’t think the disclosure rule would quell discussions. I’d rather have neither rule, but this is a needed counterbalance.

Sorry to hear you’ve been accused elsewhere @manon . I don’t know how extensive the accusations were, but even when I see it being used as a casual way to dismiss what someone says, it’s a low blow. I’ve played a few of your games and it never crossed my mind, for what it’s worth.

1 Like