AI rule for Spring Thing: How to make a rule that is enforceable and fair?

Competitions Spring Thing
1

I’ve been catching up on my backlog of games recently.

I’ve been dismayed by the number of AI-generated games I’ve played, especially the parser games. In many of them, the authors leaned hard on the gas pedal to use AI liberally. Every item has multiple paragraphs of description, and there are numerous rooms and events with no purpose other than ‘have fun’.

I was on the fence with it before, but many of these games are so long that it takes up the time a lot of smaller, more heartfelt games could take up. And my favorite reason for reading and reviewing, which is feeling like a made a small connection with the author, is gone. Instead, it feels like I’m just the kid of a neglectful millionaire who asked a servant to pick up a birthday card for me. Nothing I’m reading matters to the person who asked AI to read it, so why should I?

I want others to have the best Spring Thing experience they can. And I haven’t heard anyone say they enjoy AI-written games. If you have enjoyed playing AI-written games, please respond here; this is definitely the place to make your voice heard!

Similarly, AI art games take up a lot of bandwidth. While the Spring Thing website has plenty of storage, operating requires a lot of uploads and downloads, and this year the game storage total was significantly larger due primarily to AI art embedded in games (as well as some lovely hand-made games).

So I’d like to make a rule against games that use AI art and text.

The problem is where to draw the line.

I am okay with using AI-assisted spellcheckers. It’s hard to get good spellchecking and grammar checking without AI. Similarly, AI-assisted language translation tools are, I believe, an effective use of AI in many situations and I can think of several games both I and others enjoyed that made use of AI translation.

AI coding is too difficult to detect and has a different set of problems from AI text. I don’t plan on banning it.

Some people design and code games themselves and only us AI to flesh out descriptions. I would like to include this in banned topics, as it results in the problems described earlier (large, bloated games with loads of meaningless, soulless text).

So, I’m thinking of making the rule something like:

  • Spring Thing asks authors to not enter games that use AI-generated art or which use AI-generated text. Using AI to modify hand-written text through tools like grammar-checking and translation is allowed. Games which have strong resemblance to AI-generated writing may be removed at the organizers’ discretion.

The last point may seem harsh, but Aaron Reed and, later, I, have been clear that the organizer can always remove games as they see fit, so this wouldn’t be an increase in overreach.

I wanted to get feedback on this though. What do you guys think? While I am organizing Spring Thing, I don’t run all of it according to my own thoughts; I try to stay try to Aaron Reed’s vision and try to follow the community’s feelings.

Please let me know below how you feel. I know some people don’t feel comfortable weighing in verbally so I’ll add a little poll as well.

  • I support every idea listed above without reservation.
  • I support an anti-AI policy, but this one is too lenient.
  • I support an anti-AI policy, but this one is too harsh.
  • I believe AI art should be allowed, but not AI text.
  • I believe AI text should be allowed, but not AI art.
  • I believe that AI-assisted coding should be banned.
  • I don’t have an opinion, I just want to show I was here.
0 voters
24 Likes
2

Yeah, this seems like the right approach to me; I’ve had the same experience playing games with AI art and especially text, and it just completely deflates me motivation to review or engage. The Grammarly/translation stuff feels like an enforceable bright line - I’m sure there will be some people who might try to use it to slippery-slope their way around the main prohibition, but that’s why the savings clause at the end is important IMO.

15 Likes
3

I’ll say that I wouldn’t care if AI-written games had good prose (they don’t). From a competition or exhibition point of view, there’s no parity between spending hundreds of hours writing something and… not. It doesn’t make sense to include computer-generated content in an event that celebrates the unique creativity of the individual.

Perhaps someone could organize an event to recognize AI-generated content, if there’s a demand for it.

17 Likes
4

Very supportive of an anti-AI policy!

I picked the “too lenient” option in the poll just because I think the second sentence here could be clearer. I don’t know how AI-powered grammar checkers work—do they completely rewrite text, as opposed to (for example) MS Word underlining things and making suggestions that you can choose to accept or ignore? I’d think writing your own text and then feeding it to ChatGPT and saying “make this grammatically correct” would be outside the bounds of what Spring Thing would want, but that second sentence feels like it leaves some ambiguity. I think it’s the word “modify” that’s really getting me.

12 Likes
5

If you’re feeling summary removal is too harsh, maybe if you determine a game is AI generated, perhaps you could leave it up but move it into a “Machine Generated” back-corner category that isn’t eligible for votes.

6 Likes
6

i agree with the ban on AI art in a work that’s largely a graphic work. but i’m a horrible artist and have, in the past, used AI to generate the cover art for games. i feel like this is a pretty minimal use and has nothing to do with the content of the game itself and, perhaps, should be allowed?

8 Likes
7

I also support no gen AI, but think this is lenient. Spell checkers have existed for decades without generative AI, so I don’t think there should be an allowance for that. Regarding translation, a writer can recruit an additional writer who knows whatever language to collaborate with them.

10 Likes
8

Lots of suggestions in this thread for creating cover art without being an artist or using genAI!

7 Likes
9

I never enter into the AI debate, but the Spring thing is something I care, so:

of course people who actually known the text of Isekai understand that for me AI generated room & object description = massive & unfair cheating. so I agree on the text ban, with extreme prejudice.

And, Tabitha, I think that for grammar checking AI is an huge overshot, reading the ßtest proofreading report, I can place 95-99% of my instances of bad english in few well-defined category, and my assessment is that a well-tailored expert system can largely suffice (ex. a good chunk of typo are lowercase T and Y where should be uppercase; this stems from a fact, my hand is small, and on top of it, my left pinky is slower than the other four fingers, so the index has closed the contact under the keys in question when the left shift keytop, driven by the slow pinky, has NOT closed the contact. not a great intelligence, natural or otherwise, needed for getting that, for an example, a lowercase t or y after a period and space IS a typo. But I’m digressing…

Cygni, spellcheckers aren’t AI, and neither expert system: is simply an indexed database of words, and warn the typer if it don’t found the word in the database (I kept the spellchecker in all my editors off because I’m tired of adding ThisorThatMethodorRoutine name in a custom sub-database every few lines…)

Personally, I think that that AI-generated feelie should be allowed (e.g. the manual of Moka Espresso and Breakfast in the Dolomites are an excellent example of wise use of AI graphics); here I agree with improvmonster.

AI assisted coding: here again we must draw a line between generative AI and expert system; an expert system capable of scanning lines of code for well-known, if not classical errors, like carryable doors or detachable bodyparts…

… Detachable bodyparts ??? oh, wow… now I touch in my hand the greatness of Asimov… no spoiler for The Naked Sun, 1956 vintage, but his foresight, in huge advance of his times, of computer and AI issues is literally palpable here !

[after 20 minutes in comparing back and forth 1950s and 2020s thru the Materia Asimoviana…]

Anyway, if we recognise expert systems as line in the sand, if we recognise expert system in dealing with spelling and grammar in the text part of an IF, why we don’t recognise expert system also in linting the code part of an IF ?

in other words, summing up my tentative position on the critical issue, the real matter is that IMVHO where line is to be drawn in AI, is in generative AI, but not in assistive or predictive AI…

and don’t draw me more into the AI debate. Sinclair vs. Commodore, Emacs vs. Vi, OS vs. commercial software/o.s. is enough for me !

Best regards frm Italy,
dott. Piergiorgio.

2 Likes
10

“AI-powered” grammar checkers like DeepL Write will currently look at the original text and then output a suggested way to rewrite it. If you recall how machine translation websites like Google Translate are laid out, the original text will be slotted into the left and the new lines will be on the right.

My experience with them is that they’ll catch your collocation mistakes, a bit like an “AI proofreader”. But while you can’t tell these checkers to turn these sentences into Dickensian prose with the snap of a finger, your attempts to render it in text may be caught by these checkers and will “clean up” any “mistakes” you may have made.

So, I think it’s somewhere between “MS Word telling you what mistakes you’ve made” and “ChatGPT, please turn my writing into Lovecraft”. You still have to put in the effort to make it work, and you can see these apps as helping you garner more polish for your work. In the hands of most efficient writers, it can be an advantage.

Whether that is fair or unfair, it’s not for me to say. I still don’t know what to make of it. At the moment, I find it difficult to find any grammar checker that isn’t influenced in any way by LLMs. Not including humans, anyway.

As for me, I more or less agree with this. I do think a disclosure on what tools you’ve used (regardless of LLMs) will be nice. It’s always amusing to learn that one of my favorite games, Void Stranger, had its art done on MSPaint. Not exactly pertinent to the discussion, but I think it’s in the spirit of the event to show your craftsmanship whenever possible.

11 Likes
11

Spellchecker vs. AI:

Grammarly now has AI features (beyond what it’s always had), but they are relegated to a chatbot. You can highlight sentences and say, for instance, “shorten” or “make more impactful” or whatever with mixed results. Beyond that, Grammarly has AI (but not LLM)-assisted features including synoynm suggestions, grammar/punctuation/spelling checker, and a passive voice detector. With the paid version there is also in-line suggestions for making your writing clearer or less wordy, although I think these have been around since before the rise of LLMs.

4 Likes
12

To a degree, yes. But I think part of the confusion is that LLMs may also be supplementing these features. It isn’t clear in the marketing, and I honestly don’t see why Grammarly wouldn’t implement some of the tech into their basic features. Synonym suggestions would be more “efficient” with LLMs for example.

On a more practical standpoint, I think it may be better to think about how people may use it instead of figuring out how these black boxes might work. Tabitha’s question points to I think the most salient part of this discussion: are we using these services to replace the text or are they meant to be suggestion tools? If the former, it will be no different from asking ChatGPT to write me a story. If the latter, then it will be us crossreferencing the original text and the suggestions (whether it be underlined MS Word-style or what I’ve explained with DeepL Write) and deciding on which improvements to incorporate.

Allowing the latter may be more pragmatic. Earlier AI in IF discussions are always concerned about witchhunting, and I am sympathetic to this because I kinda don’t want to question my peers all the time. I’m just not a fan of writing reviews where I have to lay out my suspicions if this is something that relied too heavily on LLMs. Even if I think many of us are in agreement that generating text for the sake of it just sucks for a hobbyist festival like this, I feel like rules discussions like this are about making this enforceable and fair for everyone.

And I don’t really expect anyone to be in the know for everything LLM-related. Someone who doesn’t know English that well but wants to make a parser game may accidentally use a tool that is suspected of unethically using LLMs. Especially since MathBrush approves of machine translation for usage in Spring Thing, I feel like it will be unfair to people who just don’t know any better. DeepL Translate and DeepL Write are on the same website, a link away. I can imagine someone using those services without any distinction.

There’s a part of me that wants the rules to be stricter and clearer, but I’m not sure what to add or remove. This suggested rule is a nice compromise. I just want something to point to and move on with my day.

11 Likes
13

@mathbrush points out the issue of bandwidth though. It’s very easy for someone to decide they want to make a “feelie” consisting of hundreds of MB of low-effort AI images and the Spring Thing website is stuck with hosting it, with all the associated costs.

4 Likes
14

This thread began with Brian discussing the legitimate harm that dumping AI text and art into the pool does to IF in general. This seems like a legitimate reason to ban outright the use of AI to generate the bulk of your ST entry. When we talk about the finer points of what spell checker should be allowed, though, we lose sight of the original objective – to preserve the integrity of ST and IF. I find it hard to believe that using DeepL Translate (which I admittedly am unfamiliar with), or for that matter GitHub Copilot, will create the kind of pollution that Brian and Mike talked about above.

11 Likes
15

I suggest a form field where entrants are required to disclose the extent in which AI is used in grammar checking and translation. This need not be a public disclosure- just something for the organizer to make an assessment in the event of disputes.

Otherwise, looks ok to me.

If let’s say it’s an honest mistake- say, someone took a stock image from a public domain website and sincerely didn’t know it was AI-generated. How would that be handled?

6 Likes
16

Two issues with that: (a) you can’t always be sure what algorithm is being used under the hood of a particular tool, and (b) even if you do know, there is usually no way to get the old algorithm back.

E.g., for English, macOS has been using a transformer-based spellcheck and next word suggestion system since the 2023 release. The only way around that is to turn off spellcheck entirely, downgrade macOS, or buy a different computer.

Edit: also, trying to ban a particular spellcheck implementation sounds like it would fail the enforceability criterion. Like, are we going to ask people to submit an account of all the software they used and if any of them are found to use AI, the entry is out?

Also, disclaimer I guess: this post was written on a Mac with transformer-based spellcheck. Would you have noticed?

Edit 2: A look at Apple’s new Transformer-powered predictive text model (blog post from 2023)

5 Likes
17

An anti-AI policy is something I can totally support. (Talking about parser-games here:) The bloated descriptions full of unrecognised nouns that serve only to whip up the “atmosphere” of the game completely deflate any deep engagement with the text for me. It’s also plain bad writing.

I chose too lenient, because I don’t agree with this:

For spellchecking: get a dictionary and do the work.
For grammar, I’d rather see a bunch of awkwardly twisted sentences that come from the author’s brain than grammatically correct AI-assisted porridge. Often, those skewed grammar constructions can even add to the feel of the game.

9 Likes
18

Were there any games in Spring Thing that actually used AI?

4 Likes
19

The use of AI doesn’t bother me when it’s implemented with care. When there’s (to borrow a line from Roger Ebert) “authorial control” being exercised. When it’s edited for human consumption. When it’s used as a tool.

When creating IF, if the act of generating content provides more satisfaction for the author than the substance of the content itself, then we have a problem.

I think an author should just disclose if they used AI and how it was used. Then players (or judges) can decide if that’s okay with their sensibilities or not.

@mathbrush I think how you’ve described handling the issue is quite reasonable. :+1:

3 Likes
20

I’m not good at picking up AI writing, but if you scroll the Spring Thing contest page, there is some cover art which is clearly AI generated. (Wrong number of fingers, illegible background text.)