Right. So what I’m saying is you could use an AI service strictly as a reasoner AND have it as an optional feature of the game, and it wouldn’t be a violation of the rules.
It says “external AI or generative services”. That means, I assume, that the game cannot rely on an external AI service to run, which probably means that a game must be fully playable without connecting to any AI service. What would happen to a game which relies on player input being processed with AI when said AI goes offline? If the answer is “the game isn’t playable”, this would be a game which requires interaction with an external service as the player inputs commands which are then processed by AI which requires an external connection. Unless there’s an instance of an offline AI which gets bundled with the game, it’s an entry which requires judges or players to interact with an external AI/generative services during play.
4 Likes
So I think the hypothetical which is being tossed around here is:
Imagine a whodunnit game where you wander around collecting evidence in order to solve a murder. At the end, in true Poirot/Marple style, you gather all of the suspects together to explain the mystery, and the game provides you with a free-text entry box to type in your theory about what happened.
Also, the game has two selectable difficulty modes. In the default, easier mode, it just checks for the presence of certain keywords in your explanation to determine whether you successfully solved the mystery or not. In the hard mode, it calls out to an LLM and provides it with the real truth of the mystery plus your explanation and asks if it’s a good enough match. Then players aren’t required to interact with an AI service to play the game, because you can always leave it in the default mode.
Would this be good? Would anyone actually choose to enable the AI features to make the game harder? Would it actually be allowed in the comp? Who knows?
3 Likes
I, for one, would welcome an IFComp entry that does its utmost to violate the spirit but not the letter of the new rule, taking advantage of every feasible technicality to lean on AI as much as possible.
After all, someone has to come in last place.
15 Likes
Giving Leah Thargic some actual competition, you mean?
2 Likes
You lost me until that last sentence. Now I’m on board again 
“Welcome” is a strong word (in my case), but the point remains–even if someone tries to get past the very clear AI ban on technicalities, the community has made exceedingly clear what it thinks of AI, and they can expect to place accordingly.
5 Likes
One reason I strongly voted against AI services is that such games tend to fall apart almost immediately and are then unplayable. In the last IFComp, one game author forgot to pay for enough tokens, so the game stopped working after a day or two.
Past IFComps have served as a cultural heritage of free, permanent games that others borrowed and learned from. So while it may or may not be part of the rules, I’m strongly opposed to live-service games built on impermanent technologies. This is just my personal opinion and not a community-wide one.
19 Likes
I confess I’m having trouble seeing a use case for a completely optional AI feature to distinguish between a finite number of human-written responses that delivers any functionality or interest beyond pre-programming means to get those few responses, and isn’t just obnoxious rules lawyering. @jwalrus’s example might get the closest, but I’m not convinced it would work as presented, both because of LLM unreliability and because of the noted tendency to default to telling users “good job u are the smartest person evar.”
At a certain point, I think you’re better off taking the energy invested into pushing the rules (and people’s nerves) to their breaking point and reinvesting it into making a functional game that doesn’t rely on this sort of gimmick.
3 Likes
I’ve said before that I’m in favor of not having rules around this, however:
I’m much happier with the clarity of this year’s policy than previous years. Even if the rule was even stricter, I’d still be happy with the amount of clarity and background provided here.
This isn’t owed to us, but it does give me confidence that IFTF/IF Comp is continually looking at the rules in-depth.
Thanks @Jacqueline for putting out the feedback and background that went into making the decision.
10 Likes
Perhaps when a parser error occurs, the command and surrounding context are passed to an LLM, which guesses what was intended, and responds “maybe you wanted to SEARCH BOOKCASE instead?”
Various attempts at this haven’t impressed me so far, and Brian’s objection about archivability definitely still applies. But it is something that people have tried.
Or put that energy into running - cos someone would need to do that - an AI friendly competition.
I’m rather aghast at how much some people are proposing bending the rules for IFComp here. Which may not entirely be fully clear in all cases, but are very clear in their spirit. And reflect the overall views of the community strongly.
9 Likes
I feel like this kind of game can only exist in Hypothetical-World, because the actual development of such a game would be really cursed/backwards, and it would probably fail to rank very high in a competition.
Like…sure, congrats, you found an edge case, I guess(?). You’re welcome to try it, but it’ll probably be a pyrrhic victory. I don’t really see the point.
EDIT: An elaboration:
- You figured out an algorithm that runs without using a reasoning neural network, in which case it seems like a huge waste to optionally toss that incredible effort for some neural net gimmick.
- Your game intends to lean on the assumption that the player will opt for the reasoning neural network, in which case it’s not really an option, is it? If you turn it off, then you just have a bad game, and that will tank the ranking into the floor.
5 Likes
Okay Ben we’re all neurodivergent and can see the gaps in the rule, but this suggestion is just trolling. No one is going to build something like that (it would require the game to embed AI within its structure because the rule specifically says you can’t connect to GenAI). The level of effort far outweighs any benefit.
Alas, I expect more pokes at the edges of the rule, but the INTENT is clear. Games can only have human created text, images, and audio.
6 Likes
It’s as you say: while trying to lower the barrier to entry for players unfamiliar with, or frustrated by, parser restrictions is a worthy goal, it’s one that LLMs haven’t proven great at. (This is, of course, setting aside all the other objections I and others share.)
And as @inventor200 already pointed out, if the game is written around the ostensible improvements of an optional feature that the vast majority of the player base is uninterested in and won’t benefit from even if it is miraculously functional, this isn’t the venue for it to begin with.
3 Likes
Indeed, I got a bit carried away here—I think the new rule is great, and have no intention of skirting around it, or encouraging others to!
1 Like
Got the answer to my question.
1 Like
I can see the new rules leading to controversy if there are entries which are suspected of using AI and their authors deny it. Will the rule actually be enforced? If so, how will it be decided? The change strikes me as a pretty big hostage to fortune.
Also, given that one indicator of AI generated prose is poor quality, I worry the rule may discourage new authors who fear being wrongly suspected.
4 Likes
I don’t see that as an issue—a lot of the IFComp rules involve things that are difficult to prove. How can you ensure that judges didn’t play for more than two hours? How can you ensure that people are voting in good faith? Or that an author didn’t steal all their text from an obscure novel published last year and not widely indexed by search engines? Last year the comp required any AI use to be disclosed; how can we be sure authors didn’t lie?
But this is a small community where we can generally assume that people want to follow the rules. If someone tries to undermine the comp, the organizers can do whatever they decide is necessary—but people usually don’t.
11 Likes
I doubt it. It’s poor quality of a pretty specific type. Lots of ability but no brain, taste, judgment, style, individual character or typos. And juicy with buzzwords. We never saw it before these LLMs were around. It ain’t what new authors of any skill level (or humans 
) look like.
-Wade
4 Likes
I think that’s unduly optimistic. If there’s a game that appears to be AI generated, how will the community react if it isn’t disqualified? I suspect there will be significant acrimony. Then, on the other hand, if it is disqualified, there’s a risk that it was a false positive.
None of the other cited examples of rule breaking are as visible or as emotive. Nor do I think we can assume good faith. There was controversy in another competition not that long ago. I fear we may be expanding the scope for potential difficulties.