I definitely think it could be valuable to have a more experimental “anything goes” type contest so that there’s a clear venue to steer people towards when they want to do things that fall outside the rules/norms of the existing competitions. There are definitely some people on this forum who are interested in exploring the use of LLMs for IF; I wonder if any of them would be interested in organising such a thing?
I’m curious about your mention of LLMs with “less plagiarism”. Are there any publicly-available general-purpose LLMs which are transparent about their training sets?
Thanks for continuing the discussion. Yes, in my opinion, interactive fiction offers the scope for novel, innovative features, which increase the immersion and makes playing these stories more exciting. Including AI but I would like to emphasize that the site I am developing is not focussed on AI. It is more like a side tool you can use or not to make interesting stories.
You can use LLMs if you want in dialogues with other characters. The idea is that conversations might feel more lively this way. Without LLMs it might be difficult (not to say impossible) to implement conversations into the story which go beyond the classical options selection.
The other point LLMs are included is as a writing assistant. You can type in any question for continuing or paraphrasing your text and the LLM will do it.
Another point an LLM is included (but you don’t have to) is if you select “instruction following” in the story nodes. Then you can include instructions for the model (information about the scene, characters, what the protagonist should do or anything else) and the LLM will formulate it in play time with background information about previous happenings.
But you can simply write down the story plain text in these nodes as well.
If there’s no visible mention of it, I suspect that means that you can’t get good enough results without the indiscriminate scraping-the-whole-Internet approach.
It’s possible, but I think the classifier-based approach has a few big advantages here:
Classifiers generally need a lot less training data than generative models, because their task is a lot easier (instead of predicting the next word, they’re predicting an overall category; a hundred options instead of a million)
People are generally less concerned about their data being used for classifiers than generative models, which is why AI training was never a big issue legally until the creation of generative models—speaking personally, I don’t care if my social media posts are used to help a system learn “is this a real user or a bot” (which basically all social media sites have been doing for ages), but I do care if my social media posts are used to help a system learn to imitate them and impersonate me
Most of the problems with the modern AI craze go back to trying to use generative models for everything, when often they’re just the wrong tool for the job. There’s no need for a generative model in a spellchecker, for example; older systems did that job perfectly well without dragging in all the baggage of ChatGPT.
Anyway, the big issue I see with an “anything goes” contest is getting people to judge it. Spring Thing was previously the “anything goes” contest, but now it’s incorporating an anti-AI rule specifically because judges were losing interest and motivation. I’m not sure how much interest there would be in a competition that’ll be 90% dominated by AI.
Well, in which case being able to say “we tried running a contest for AI-assisted IF but no-one wanted to play any of the games” at least gives a clear message to anyone trying to sell the community on their latest groundbreaking idea.
My other worry here is that if IFcomp is the sole holdout in terms of allowing AI-assisted entries, we’ll have the same loss-of-engagement issue there if that’s where all the AI games end up funnelled to.
Many people who make IF without AI don’t seem to want to play, judge, or compete with games made with AI. Does it also go the other way? Do people who make IF with AI have any interest in competing with human-made IF? If not, maybe those people should just make their own competition strictly for AI games?
And if they do want to compete with human games: Why? To find out if they’re as good, as a sort of Turing test?
I was discussing this with a (visual) artist this morning. She was pushing the idea that there’s lots of space to explore with small, narrowly-trained models that do a specific thing.
The “LLMs for everything” approach is really an artifact of hundreds of billions of dollars flying around. The Facebook/Google/MS corporate gang require the answer to be “come to Facebook/Google/MS for all your needs”. Every solution that doesn’t scrape the entire Internet has been left in the dust, because it doesn’t put all power in the hands of megacorporations.
The problem (“problem”) is that to make a narrowly-trained model which does exactly what you need, you have to understand enough about AI models to create it! Shoving prompts into ChatGPT simply will not get you there. You have to approach the problem the same way we tell people to approach IF tools: it’s programming, and you’re going to have to learn it, and you’re also going to have to hold onto your artistic vision and figure out how to mold the tool to your needs.
…but we as a community aren’t building those skills, because the Facebook/Google/MS axis has such repellent business practices that they’ve turned most of us off.
(I very much include myself there.)
Anyhow, to get back to the original suggestion: a contest is only helpful if there are enough interesting experiments in the air to fill out a contest slate.
Possibly the upcoming IFComp will answer that question.
To stop an “anything goes” contest from being just AI content, I could see it as an expo for formats that are generally a hard sell to audiences, such as AI games, desktop EXEs, social media-based games, ARGs, etc.
The question is whether there’s an audience for this. The least accessible games in any event usually get a few players and/or reviews … but is this due to completionists or people who really crave stuff outside what is now mostly the parser+choice paradigm?
I submitted an entry to IFComp that uses AI during NPC interactions. Since IFComp only requires disclosure, and I was willing to disclose the use of AI in the blurb, I didn’t feel that this competition restricted me from submitting the game I wanted to make.
The reason why I wanted to make a text adventure game that uses AI is because I wanted to create an experience that allows the player to engage in realistic role playing. You can talk to NPCs using natural language and they will respond conversationally. You can ‘be yourself’ or you can play a character of your own choosing. I am not interested in using AI to do what a human can do. That’s why none of the narrative sections in this game are AI generated. I am only interested in using AI to do what cannot possibly be done without it. It is impossible to anticipate every possible player input. Therefore AI allows us to do what has not previously been done in role playing games, allowing the player to decide what to say without having to choose amongst a set of pre-written options. When the player is put into situations and required to make a decision without looking at a list of options, I think this is a better critical thinking role playing experience because it requires more active reading on the part of the player. It also requires more creativity to solve the puzzles. At least this was my ambition in starting this project.
As you’re probably gathering from the surrounding discussion, this community is pretty staunchly against the use of AI in any capacity. Personally, I think it’s worth having a venue to experiment in, because even if there are very serious issues affecting current systems like ChatGPT, experimentation is the only way we’ll find out if there are other uses and ways around those issues. As zarf put it:
…but, given that people are saying they don’t want to participate in comps that allow AI use, or even have the existence of such comps mentioned on the forum, this community may not be the place to do it. I’m actually a little surprised by the level of vitriol; I knew people here generally didn’t like AI, but hadn’t seen it to this degree before.
It may just be that quite a lot of us have been musing quietly on the side lines and not talking about our views so publicly.
I am extremely uncomfortable about the use of AI in a creative context, especially a competitive context.
I’ve also had enough of trying games with AI written description that I can’t interact with properly (e.g. unimplemented referenced scenery). Plus over flowery descriptions.
I am angry at the use of eg LLM of copyright material without author/creator consent. Including my own writings (not IF).
And the environmental cost and impact of AI is significant, and worries me probably more than just about anything else.
I am totally in favour of a contest open to and welcoming AI IF entries. But I very much doubt that I would be playing, judging or reviewing the games.
Yeah, speaking personally the vitriol coexisted with a sort of abstract sense that some LLM use in some contexts could be interesting despite the issues with the big platforms (I think I’ve mentioned this game, which uses a more custom system, as reasonably successful). And allowing AI games to enter events, with transparency, and reviewing them on an equal footing, seemed like a reasonable compromise to allow for experiments even if actually playing and reviewing them was often painful.
I feel like the reasonableness of that compromise has been eroded by the steady drumbeat of news about AI companies behaving unethically and the flaws in their systems causing real harm to people, the failure to actually see any worthwhile experiments even years after the broad adoption of these technologies (that game I linked to above is from 2023), and the dreary recurrence of announcements of new platforms and games with big promises, annoying promotion strategies, and little substance. And then that erosion was drastically hurried along by the recent events around ParserComp – if I step back, I can still see the upsides of compromise and experimentation, but right now I have such a bad taste in my mouth around LLMs and the people who use them that it’s hard to set my feelings aside even if I wanted to.