Can we just ban AI content on IFComp?

I don’t think this topic fits the post-comp survey, by October 15 no one would even remember about the problem, and we really should talk about it NOW.

After last year’s competition, we expected to see a flood of feedback about AI in the post-comp survey. In reality, only about 10 percent of respondents mentioned it. Based on that, we kept the same policy for 2025. (Blog)

IFComp is big. It’s so big, there’s a special rule about not spending more than two hours on each game. Why would we want quantity over quality?

If you’re thinking “but we don’t want to be gatekeepers” or “no one will read through every game before the comp”, well it’s too late because the organizers already found resources to comply with British censorship.

18 Likes

Well, per the thread I posted this morning, at least one AI game appears to have gotten the message and is trying to do the honorable thing by banning itself.

Personally I find the quantity-over-quality point pretty compelling, though of course there are other lenses through which to look at the issue. Given the blog excerpt you quoted here, though, I do think filling out the post-Comp survey is a helpful thing to do in addition to discussing things on the forum.

10 Likes

I think it’s entirely worthwhile to ban any game that requires a live internet connection to a 3rd party api. If authors want to embed a small model within their game, I see that as no different than the AI built at some game companies, which no one here has ever complained about because some of our own prominent members have been employed at said companies.

My personal thoughts might go like this:

  • You cannot use a live API. This is a practical solution to keep games small and playable.
  • You may embed AI into your story, but you must detail its training source and/or origin. Our audience/judges have a right to know all authors, including artificial ones. A small model ethically trained by the author or community might be considered a positive development in IF authoring.
  • We can’t police the use of GenAI to write the narrative of a game, but this does seem to bend the spirit of IFComp, which is meant to showcase the author’s imagination. We highly recommend releasing such games outside IFComp.

Note: I’m on the board of IFTF and on the IFComp committee but any future rule changes are directed by Jacqueline and my response here is just my own personal opinion based on nearly three years of GenAI use. I agree more discussion is required.

18 Likes

As an author of one of this year’s IF Comp games and a player of my peers’ games, I have no qualms in saying that I have not touched a single game which has purportedly used AI for image, text or otherwise in this year’s IF Comp. My personal belief is that these generated “works” have no place in a competition like this, and I suspect this is not an isolated opinion.

I’m simply not interested in interacting with anything related to it, and would wholeheartedly support a full ban of all AI content in the following competition years. To go even further, I would seriously reconsider ever entering again if AI “works” continued to be judged alongside my own.

25 Likes

I mean, you can talk about it here all you want, but if you want a rule change, the post-comp survey is where they take feedback. They don’t particularly monitor the forums, or use forum discussions for their decision-making.

As that blog post you quoted suggests, this is what happened last year. We all complained about it here and no one filled out the survey, so they didn’t change the rule.

Edit: and I don’t believe they e-mail out the survey or anything: last year it was on the blog and social media. so if you care about it, go follow them on their Tumblr-based blog (or add that feed to your feed reader) or on Bluesky or on Mastodon (or that RSS feed) so you won’t miss the survey announcement. Set yourself a calendar reminder for October 20th (after the results livestream) to see if the survey is out yet. Post back here to remind people when it’s out. Again, we can discuss here all you want, but if you’re trying to get a rule change, that’s not going to accomplish anything. Fill out the survey when it comes out. It has free-form places for people to suggest changes. This is exactly what they have the survey for.

13 Likes

This misconception is widespread so I don’t blame anyone for having it but you can’t have a ‘small language model’ that produces the kind of highly coherent-seeming output people expect out of GPT and similar systems.

What there are, are various solutions that let you locally run a ‘base model’ trained extensively on text from the internet used without permission, and then retrain it on your own writing as another layer. People will almost invariably refer to this as ‘a model I built myself trained only on writing I own’, partly because it’s convenient to make that claim and partly out of genuinely not understanding what is going on under the hood.

As a more practical matter I don’t think ifcomp is the place for games that require a discrete graphics card and have a download size in the gigabytes.

I should also point out: at present there’s kind of a propaganda campaign to conflate any kind of ML approach or even generative approach with ‘AI’; I’ve seen people call SpeedTree ‘an AI tool’. This really muddies the waters when talking about the ethical and labor concerns surrounding LLMs and GANs, which are a qualitatively different technology and very distinct from generative or automated methods that have been used in, eg, game animation historically.

W/r/t third-party APIs: I think this ends up becoming an annoying rule with some amount of splash damage; if the goal is to ban AI, just ban AI. Eg, people making games that run on the web and use a third-party font provider like Google Fonts would be in violation of a rule here.

W/r/t games using AI-generated text: I would just make a rule against it. Yes, it’s possible for people to cheat, but the upshot of the rule would be that people wouldn’t be able to talk about or propagandize using it. I think that in itself would be a positive; a little shaming is salutary here.

W/r/t the post-comp survey: so my initial concern here was that the high prevalence of entries using AI inhibited me from judging this year, so I wasn’t sure if I’d be able to fill out a survey. If the IFTF is okay with survey responses that amount to “I didn’t participate this year, and here’s why…” that’s fine.

24 Likes

I’m not someone to “misunderstand” GenAI, so let’s set that aside. Much of the IF community are technologists, so I would never assume we’re "naive”.

There are already ethically trained models and some of them are relatively small and will be able to run on a machine or server easily.

The actual training of your own from scratch is already becoming easier, without knowing python and its libraries. This technology is changing fast. The addition of MCP and RAG and other aspects of GenAI will be in its infancy for years.

No one is suggesting an author will provide a massive game simply because it has an embedded LLM and no one is suggesting its reasonable to expect people to have an expensive GPU in their PC, but I’d also theorize that these things will converge in unusual and positive ways.

There are already improvements in model size and capability and fat GPUs will gradually move into average PCs.

I don’t like the idea of flat out banning anything (except CSAM) and there’s likely going to be an audience for AI-driven stories. Gen Alpha is already ignoring most of what we call “AI ethics”.

The IFComp has always had an “experimental” aspect and we should maintain that. If we want a secondary comp that explicitly bans any AI, I’m all for it.

5 Likes

I have mixed feelings about a total ban, but they’re “mixed” as in “I worry we may miss out on something interesting even though this would definitely be a net positive” rather than “I actually can’t decide if it would be good or bad.”

This year has seen the first AI entry I consider even remotely promising. I feel as though Penny Nichols, Troubleshooter does something interesting, even if it doesn’t seem to work very well and I’ve got the usual reservations about even touching ChatGPT etc. in the first place. For those who haven’t had (or understandably won’t give it) a go, the submission is a prompt to be pasted into an LLM, which then acts as a game master of sorts. This format just about delivers on the promise of “a parser that accepts anything you type into it,” albeit with all the jank you’d expect from a chatbot GM.

I’ll admit I found this interesting enough that I had a little go at writing such a prompt myself, and am considering submitting it next year. (I may very well be persuaded not to - Bruno Dias sums up a lot of my own concerns in this blog post.) The resulting “game” is amusing, and the effect could not be achieved by any other method.

The fact that a submission of this sort can take the form of a human-written prompt might be worth considering. Strictly speaking, I don’t think anyone submitting such a thing would be obliged to tick the “This contains AI material” box at present and it wouldn’t necessarily fall foul of a future rule preventing games making direct use of APIs. In practical terms, such a game could also be played using an ethical LLM in future (if such a thing ever comes to exist). It wouldn’t hinge upon one particular tool and could potentially even be played as a traditional TTRPG with a human GM. These are some of the reasons I’m entertaining the idea despite some pretty massive reservations.

But even while actively considering a submission that may be ruled out by an AI ban, I’d probably still support the idea on the whole. I don’t think that the convenience of typing some words into a box and getting an image that sort of matches up with what you had in mind is worth the injustice of using others’ work against their wishes. I don’t think any use of LLM text in an entry is so worthwhile that it makes up for these things having been trained on material that in some cases was blatantly pirated. In a legal environment where Disney can get away with just about anything it wants and copyright stops protecting individual creators more or less the moment it becomes convenient for a large company to ignore it, I think it’s up to us to decide what’s acceptable for the competition. Against that kind of backdrop, I don’t especially mind if my neat idea that doesn’t technically hinge upon these unpleasant elements becomes collateral damage.

14 Likes

In this case, please make exceptions that ensure that some less modern online interpreters can still run on external servers. This is currently the case for playing online games made with Adrift 5 and Quest 5. I hope Parchment will one day support those but this is currently not the case…

3 Likes

I was also really fascinated by this entry but I should note that the prompt itself wasn’t human written either. In the walkthrough accompanying the game it shows the author had a different prompt at the beginning and asked the ai to rewrite it. It’s possible that original prompt was human made but it’s interesting to see the ai-ception here (and it explains some confusing parts of the prompt like mentioning “mana” but never using it).

11 Likes

I’ve seen enough nominally technically competent people say absolute nonsense about genAI to know that’s simply not true. The entire field is rife with misinformation. The people who put the most money into this are repeatedly telling the world they’re making the computer from I Have no Mouth and I Must Scream in their basement.

There are already ethically trained models and some of them are relatively small and will be able to run on a machine or server easily.

The reality is that the well is extremely poisoned against this technology by its largest investors and boosters; it’s not really plausible to trust that a model is ‘ethically trained’, discern what exactly that means, or separate that from the models that are trained through mass-scraping without permission. More generally, I simply do not think any use of this technology is ethical.

The actual training of your own from scratch is already becoming easier, without knowing python and its libraries. This technology is changing fast. The addition of MCP and RAG and other aspects of GenAI will be in its infancy for years.

I’m sure this is not your intention but when you talk about ‘training your own from scratch’ that perpetuates the exact misconception I’m talking about here. People think they are training models exclusively on writing that they own or that they are able to ‘ethically train’ their own models from zero, which is not the reality of how this tech works. I’m not sure what the relevance of MCP and RAG are to this, either.

There are already improvements in model size and capability and fat GPUs will gradually move into average PCs.

I own a cheap bottom-of-the-line laptop that I use as a tertiary machine; it has integrated Intel graphics. This is the type of machine that might very well be the most that a lot of people can afford, and historically people who can’t afford much more than something like that have been able to judge and even submit to ifcomp. I highly doubt these machines will ever be able to reasonably run an LLM locally.

I do not believe that comparable machines will at any point in the foreseeable future come with discrete graphics or have a ‘fat GPU’, and I’d be willing to bet on it. Regardless, what future hardware can do seems beside the point.

I don’t like the idea of flat out banning anything (except CSAM) and there’s likely going to be an audience for AI-driven stories.

The IFComp has always had an “experimental” aspect and we should maintain that. If we want a secondary comp that explicitly bans any AI, I’m all for it.

By all means let’s keep an open mind but not so much that our brains fall out. What LLM usage mostly means is not ‘experimentation’, it’s low-effort slop and an overall crowding out of actual formal or thematic experimentation.

Take Penny Nichols, which people are so enamored with apparently… chatGPT is doing basically all of the heavy lifting here; I’ve read the prompt and it’s barely sensical, if anything about this entry is impressive it’s the fact that chatGPT can generate something apparently coherent at all out of it.

For me it highlights the noxious nature of this technology. A lot of what the entry’s author clearly thinks they are doing is not really what’s going on; telling chatGPT to roll dice doesn’t actually make it roll dice, it just makes it generate text claiming that dice was rolled. The prompt is largely providing role-playing game jargon for chatgpt to weave into its responses, creating the impression of game mechanics. And I’m not sure even the author understands that, let alone users.

It’s just this fundamentally dishonest machine, but the thing is that chatGPT is so adept at generating apparently coherent and extremely agreeable output that you can prompt it with nearly anything and get a result that feels like it coheres, even though there’s almost nothing authored about the resulting experience other than the vague general premise. This is toxic to the entire idea of authorship.

40 Likes

I think this discussion isn’t going away in four weeks. :)

And here! This thread in 2024.

Yes, there is an option on the survey for “I did not participate and here are my comments.”

13 Likes

I plan to write about this soon, so I really should just save what I have to say, but I would like to push back against the idea of governing via surveys.

The gravity surrounding the IF Competition is tremendous. There is nothing else like it. People we don’t see all year turn up to play and rate these games. Creators who don’t follow IF sometimes do something comp-related. Games entered in the competition get more reviews and ratings than games that do not, and this phenomenon is on average quality-agnostic. It affects IFDB ratings long after the competition ends. Which affects the algorythm. And on and on.

The comp is not something that begins and ends, it acts upon us and our work all year long. I think this LLM question is important enough that it shouldn’t just be left up to whoever feels motivated to respond to a survey. ParserComp was affected by an LLM-affiliated brigade this year. I don’t think people who care about this subject should have to whip responses here and on other platforms. I have been told so many times that if I care, I should fill out the survey. OK, I’ll do that, and I hope that people who value human endeavor do, too. But. But.

Considering the impact and influence the event has, I would really like to see leadership here. I think it’s OK to look for a bit of leadership from the committee.

It’s easy—everywhere, not specifically here—to assume that valued institutions will always be around, that nothing can ever happen to diminish them, but that isn’t true. Things falter and go away all the time. Nothing is permanent by nature. This community is old enough to know this.

I’d also like to say that I find Penny Nichols, Troubleshooter depressing. It may be interesting, but—forgive me if I toot my own horn—most of my opinions regarding Dragon Quest are interesting, too. So are nature documentaries, and long-form video essays about Resident Evil.

31 Likes

Yeah, the ParserComp situation is extremely disappointing and it highlights two of the further problems with this stuff:

  • AI often comes with a side of extreme self-promotion, general low-effort attitude, or shotgun approach to ‘using AI to enter into as many spaces as possible’. The technology and especially the community around the technology encourages bad behavior.
  • At some point any event that doesn’t explicitly ban AI just becomes a stage for people to rub their models against each other.
26 Likes

Eventually we can simplify it, so that there are only AI-bots that vote on AI created material.

8 Likes

I don’t think you’re complaining about lack of leadership here. The IFComp admins will make a decision about AI policy for 2026. (Leaving the rules the same as 2025 is one possible decision, but it’s still a decision.)

The only question is whether they know what you think when they make that decision. If they don’t, their decision will not take your opinion into account.

Are you saying that they should make a decision without considering community opinion? That’s really not in line with how IFTF operates.

Are you saying that they should be reading this thread rather than waiting for survey responses? I’m afraid that’s argument is not going to go very far. Each one of these AI discussion threads goes to dozens if not hundreds of posts, and they’re hella repetitive. I certainly don’t want to re-read them all in a month and try to extract quantitative info.

13 Likes

Penny Nichols is only interesting to me because of the implication of TTRPGs being entered in the Comp. (Feels like a lot of problems would be solved if it was just presented as a TTRPG rather than pretending to be a parser game.) Is there precedent for that?

5 Likes

This is part of the reason I was curious to try writing something like this for myself. The non-functional dice-rolling guff doesn’t appear to be at all critical. ChatGPT will act as a game master if told to in fairly straightforward terms in the prompt, and that frees up the “author” to focus on laying out the characters, setting, and broad strokes rules of the game. (Not dice-rolling or anything like that, but “The protagonist is clumsy. They should cause unintentional damage to things they interact with.”) I can see in the resulting output that these details do shape the game, but honestly the prompt I’ve been playing with is so short that it’s hard to consider the resulting game my work at all. It is ChatGPT doing all the heavy-lifting.

The fact that the author hasn’t written any of the text the reader ultimately encounters when the game is played as intended would be perfectly reasonable grounds not to allow this sort of game in the competition, I think. Still, I get the impression that two different people submitting two different prompts could end up with players experiencing very different games, and that the nature of the prompts would have a significant influence on what those games were. I’d have to see several such submissions to confirm that, though, and I don’t exactly relish the idea of IFComp being the place to consider them. The idea of some AI-free secondary comp was floated elsewhere in this thread, but actually I think it would make more sense to make IFComp AI-free and have Penny Nichols-style games entered into something else.

Somebody could set up SlopComp, where human-written text is considered an unfair advantage and participants compete for the Crown of Slop. I realise this sounds like a joke, but I personally would bung my hypothetical prompt entry into that if such a comp existed.

10 Likes

No, of course not. But the post on the IF Comp blog (about the competition generally in 2025) feels a little floaty. To me. I’d like to know what is being considered. Or what questions they have. Or something. I think there’s a position that could fit between what we have right now and “we don’t care what you think.”

I don’t think there’s any precedent that would lead me to believe that.

But the conversations will be new to the people who are visiting for the competition and may not be aware of conversations through the year. This might matter more since so much comes down to surveys.

9 Likes

That’s probably illegal in the UK

23 Likes