Consuming vs producing AI

Playing
1

Continuing the discussion from AI in competitions:

I’ve seen tons of interest from people who want to create games with AI.

I’ve seen very little interest from people who want to consume games with AI. The main things I’ve seen people enjoy doing with AI are interfacing directly with ChatGPT with a prompt (like AI dungeon or character ai).

I’m wondering, can anyone point to a game that used AI for text creation that they enjoyed? Or a game that used AI for art creation where they enjoyed the art? Especially something that you enjoyed so much that you thought, ‘I’d like to find more games that use AI for text or art?’

For transparency, I don’t like AI and I’m trying to prove a point by showing that games made with AI aren’t fun. In the interest of fairness, I’ll point out the game that used AI that I enjoyed the most: Midsummer’s Eve, which used Ai to help generate pixel art. I’m not aware of any ai-generated text that was a pleasure for me to read.

17 Likes
2

I thought Robert Goodwin’s games Thanatophobia and Cheree: Remembering my Murder wound up effectively leveraging of the chatbot approach - from my understanding these were specifically-trained models rather than implementations of the big, ethically-dubious and hallucination-prone ones though.

Edit: I am also on record as thinking the original AI cover art for Galaxy Jones was good.

11 Likes
3

Yeah, I was going to say, I liked Thanatophobia but I’m not sure if that counts. I’ve definitely never seen ChatGPT or any similar general-purpose model produce anything I thought was worth reading.

7 Likes
4

I’ve done extensive experiments with GPT novel writing. It doesn’t get any better. I solved the long term consistency problem pretty well for the more advanced OpenAI models, but beyond its lackluster prose, it can’t write an extended passage to save its life.

10 Likes
5

The Garden of Earthly Regrets used AI art for little background images, which actually looked nice most of the time; they have this crosshatched, monochrome style that was distinct from usual games without being distracting. The only image I thought looked off was the statue, which had the usual AI problems of trying to generate humans.

I do admit that I was hesitant to play it based on the “AI art” tag, but the game itself was interesting and the art is easy to block out to focus on the prose if you don’t want to see it.

7 Likes
6

I think in every game I’ve played that included AI-generated art, I found the art distracting/detrimental to my enjoyment. The only exception is The Kuolema, in which I believe the AI-produced imagery was heavily edited, as I don’t remember noticing any of the usual AI artifacts/oddities. But the art didn’t really impact my opinion of the game either way. (For transparency, I don’t usually care too much about art in IF, so that statement is likely to be true for games with non-AI art too.)

7 Likes
7

I’m kind of addicted to playing around with LLMs, currently mostly using dolphin-llama3 via Ollama since the online models have annoying amounts of censorship, but yeah, the output is mostlypulp quality stuff that’s only worth it because an uncensored model will cater to niche interests that can be hard to find human written material for.

Can’t comment on AI-generated graphics because blind, and the last time I tried any kind of image description AI, it involved the hassle of using a browse dialog in a clunky web interface, was highly censored, and didn’t seem to work that well to begin with, but it was also a proof of concept version that’s practically a fossil with how AI has been advancing… Wish I could find a image description solution where I could just cd to a directory of images on my computer and run something like ai-describe *.jpg *.png and let it run overnight to wake up to a .txt for every image file including everything the AI got from analyzing the images, or which would let me jump to an image on a web page, hit the context menu button on my keyboard, and there be a describe image option, but as far as I know, most of the existing solutions either require uploading images to a website or are smartphone apps that use the phone’s camera and can’t analyze images already on the phone.

No one else has mentioned it, but where AI has wowed me the most outside of using LLMs to generate pulpy prose and chat with fictional characters has been in the realm of audio.

I use espeak-ng as my voice synthesizer on my computer, both with Orca in the GUI and espeakup in the console, partly because it’s the default on most Linux distros, but also because, while it has an obviously synthetic sound, I much prefer it to most of the more natural sounding dumb text-to-speech I’ve heard as I feel like most natural sounding TTS falls into the uncanny valley the moment it encounters something that makes it try to intone, but some of the AI TTS I’ve heard is actually climbing the far side of the uncanny valley. I’d still prefer human voice actors in a game with recorded voice lines, a human reader for an audiobook, or a human narrator for a descriptive audio track, but I wouldn’t mind playing a game that uses AI voice acting, listening to an AI read audiobook, or a fandub of an anime with no official dub that uses AI voice acting and wish it were possible to run AI TTS locally and in real time or that I could run an AI TTS on a .txt file on my computer and it would spit out a .wav file. I’m also impressed with some of the AI covers of songs and AI remixes/extensions of instrumental tracks I’ve heard.

8 Likes
8

Interesting! I haven’t really heard much about this area, and I don’t think most people I know have, either; I did a survey of my students on their AI use and no one mentioned AI audio as something they’d tried. Thanks for sharing this info!

5 Likes
9

I’ll mention The Roottrees Are Dead, which started as a free Itch game with AI-generated artwork.

I thought the AI artwork worked fine. This is one of those “solve a puzzle by searching through documents” games, so the point of the illustrations isn’t to be good. The point is to convey a bunch of information: this is a photo, it was taken in 1925, it shows two men and a woman, one of the men is carrying a wrench. Ok, here’s a 1970s disco album cover showing a different guy who might be the first guy’s grandson. Etc.

The game needs a lot of this stuff, in lots of different styles. AI is a reasonable way to generate that. Or at least start generating it – I believe the important clue-details (e.g. the wrench) were hand-edited on top of the AI output.

To be sure, the designer is now in the process of remaking the game with illustrations from a human artist. That’s also a reasonable choice. (The remake will be paid, so it has a budget! Which the original free version did not.)

What I’m getting at is, there’s a lot of work that goes into a game which is not there to be a stunning achievement of human creativity. Sometimes you just want variety in a given range. This goes for text too.

14 Likes
10

I’ll second @alyshkalia and say I thoroughly enjoyed The Kuolema. Setting aside the quality of the artwork—which was a helluva lot better than anything I could produce—the graphics were integral to the gameplay. Sure, I could imagine a text-only version of the game, but they really contributed to the gestalt.

There’s an old Google Forms test where you can try to distinguish between celebrity and AI-generated voice:

Same. Of course, I’ve read plenty of lackluster human-generated prose too.

5 Likes
11

It’s a big deal in music, with the attendant copyright wars etc, and the anticipated studio aim of owning the audio data of famous voices so they can crank and make money from them with or without the artist.

The joke is that people prefer AI Kendrick Lamar to real Kendrick Lamar (prejudicial spoiler: I think his music sux anyway, so maybe it’s unsurprising that AI generated some stimulation or variety where there was almost none to begin with :smile:)

The race is on amongst social media attention-getters to produce (read - produce by prompt, mostly) AI versions of famous songs with other famous, or dead and famous, singers singing them.

I am always annoyed by Rick Beato’s clickbait video captions and headlines (I suspect someone makes them for him, targeting the algorithm) but if you ignore what’s on the box, you will find the contents of the box solid, and divorced from the cover art. He gives a good and active summation of a lot of these current issues in music amongst his never-giving-up but disappointed appraisal of most current pop music and production. (https://www.youtube.com/channel/UCJquYOG5EL82sKTfH9aMA9Q)

-Wade

6 Likes
12

I actually disliked the AI artwork in this game—I found it detrimental to matching faces across photos, for example, because the people didn’t always actually look much alike, and I found the very fake-looking “historical” photos distracting.

6 Likes
13

I once played a short fantasy choicescript game, on the CoG forums, which used AI art pictures which were modified and enhanced by hand. The pictures were really good and fit the setting, and I liked them… until I spotted one sign that the pictures were AI generated.

It was previously quite easy for me to spot AI art, but playing that game really revealed how much harder it is to tell nowadays.

As for how this story ended:

I reminded the writer that Hosted Games would not accept AI art, even if it had been modified by hand. He agreed to take down all AI art pictures, so I assumed that all was well with the cosmos. The game was expected to be submitted in one month and published in three months.

Three months later, I got a surprising mailer in my email from CoG informing me that the game was withdrawn from the store for the use of AI art and text. I was like… what? Didn’t I already remind you that you’re not supposed to submit AI art!?

12 Likes
14

In all fairness, most of the discussion about AI audio I’ve encountered was on the Audio Games forum, which flips the usual blind versus sighted user ratio on its head from most online spaces, so naturally, no one there really cares about AI image generation and audio stuff gets fixated on more than most places.

Plus, a lot of AI covers swap out the original singer’s voice for an AI-generated singer while leaving the instrumentation alone, so they can feel even more derivative than human made covers.

That said, AI Hank Hill is pretty good at singing country music and there are some covers with him I’d swear were from a lost episode of King of the Hill if I didn’t know they were AI made.

Also, I get the impression that making an AI cover is more involved than text or image generation and requires more computing power, so there are fewer ways to play around with the tech for free.

Still, I get the impression that all the pieces are there to take the Japanese audio of an anime, subtitle scripts that include tags for who speaks each line, and have an AI do a passable job of producing a dub track that makes it sound like the original cast reprised their roles, it’s just no one has brought the pieces together and it would currently require a cost prohibitive amount of compute for a fan to do.

1 Like
15

AI song covers really died once the novelty of them wore off. Not to mention that they were so easy to make, a lot of people uploaded them with little fine-tuning, to the point that it barely sounded like the character in question.

Voice acting is difficult; VAs are trained just like any other profession, and we often take good voice work for granted. When a work has bad voice acting, you notice it, so it’s easy to forget just how much goes into doing it right. Not only do you have to put on another character’s voice, you have to speak in certain tones and emotions and make it sound convincing. Emphasis is also a big deal: try saying “I never said she stole my money” multiple times, but stress a different word and see how the meaning changes. Also, the voice actor will ideally be given direction by someone else, or have other actors in the studio to play off of.

AI-generated voice acting is either copying a line or song someone else has already done, or using chopped-up samples to try to imitate words. The latter sounds particularly unnatural. It doesn’t have the context, emotion, or guidance that a human would, and I wouldn’t say it’s feasible for any long-term works.

5 Likes
16

I played that game recently, and I thought the art did its job, although some clarfication issues arose for some level of wonkiness and lack of character art consistency. Also, everyone had that shiny, uncanny, “too pretty” look. I think the only games that work with AI art is if the rest of the game is otherwise well made, and even then it is going to be a detriment in my mind.

5 Likes
17

I can’t point to a game that has used Generative Tech art or text, but to note on the more general question of ‘consuming vs producing AI…’

I’ve come up with a general rule: the most invested and interested person in a the generative tech process is the person writing the prompt. Every other person is some fraction of interest below that point.

There may be some particular novelty or uniqueness to the image or text that strikes a downstream consumer, but the very nature of the process is that it creates such a volume of content that even a spark of novelty gets drowned out. (That’s pages of text, or many dozens of similar images on a theme.)

Now, this isn’t just being said as an anti-AI polemic; but to offer that perhaps the most interesting implementations of this tech isn’t in ‘producing’ content to be ‘consumed’ in a very linear process: prompt to game-text at a descriptive level, or prompt to game-image as a main image or illustration.

But if I wanted to generate descriptions of many, many, similar-but-not-the-same things, like room descriptions for example, where the whole point is to have slight variation, well, then it could come in very handy, though would still probably take some editing.

For example, let’s say I’m making an IF game where you are wandering through a modern office building. I want there to be hundreds of offices you could poke into for verisimilitude, but the scale itself is texture, not narrative direction. Now getting ten pages of paragraphs describing individual modern offices might be perfect. Though of course, I’d want to make sure the game design makes it clear that the player isn’t meant to go into every room. Kind of like how Dwarf Fortress creates a whole world, but it isn’t really meant to be explored fully, the same way one might try to find everything in Skyrim.

Or if it’s a visual game, not generating the main art, but generating some of those things that would help fill the art but aren’t part of the overall intent. If you can go room-to-room, maybe it’s generating some of the vague art on the walls or the pictures on the dresser, that kind of thing. Stuff the player will not focus on, but by having a little extra, adds more texture.

(Those examples are vague and not fully thought out, but hopefully the point still makes sense.)

4 Likes
18

I think this is probably right, but does frame up the question of what game designs benefit from having a hundred possible slightly-different office descriptions, vs. like half a dozen hand-crafted ones – I think in IF specifically the balance is much more strongly towards the latter, whereas the use case for something like a giant AAA open-world game is much clearer.

8 Likes
19

I currently have the lowest-attention span for anything currently, but I’ve checked out a couple of “play the AI” games and they’re kind of an oddity that I look at then put back on the shelf.

There was the AI Dungeon experiment a while back that apparently scraped a bunch of data from the Choose Your Story community. It often seemed like it gave a scenario and let you sort of improvise a reaction or action, but follow ups were so non-sequitur-ish that there was little point in actually playing further than a couple of moves.

I believe that’s because from what I’ve read, AI tends to have a short term "memory’ but no long term memory that would be necessary to build and develop a plot. So it’s literally just a computer-aided “keep the story going” improv game. Or like a DM who has nothing planned or written down and who is literally riffing an adventure on the fly that by nature is going to make no sense.

I did think Colossal Cursehead was kind of neat and the mashup of those texts presented some heady juxtapositions, but that was likely random, and other than an interesting side-show curiosity, it wasn’t coherent enough to play more than a few turns since it’s just spitting randomized text at you that was perhaps obliquely related to what the player inputs.

I think this is the one kind of useful thing AI might serve as a credited “2nd unit co-author”. If I have a game with a broken desk lamp and don’t just want the “It’s an ordinary broken desk lamp.” message if someone decides they are very invested in an unimportant object - which is usually something you don’t want to do to distract the player from the actual plot, you could use AI assertively but curate and edit the responses. I wouldn’t want the AI to make up stuff in real time and I wouldn’t want important things described as you can see it’s affected an old-school Zorkian-Infocom flavor.

Me: Can you write me five adventure game descriptions of the same desk lamp which has a broken switch and thus is not operational?

Chat-GPT: Sure, here are five different adventure game descriptions for a desk lamp with a broken switch:

  1. The Dormant Beacon
    In the dimly lit study, a desk lamp stands solemnly on an oak desk. Its polished brass base hints at a once-grand design, but the switch is now a mere stub, devoid of function. The lamp’s shade, a faded relic of past splendor, leans slightly to one side. As you approach, you realize this beacon of light is no more than a dormant artifact, its brilliance extinguished by time. To proceed, you must find a way to restore its lost glow or uncover its secrets through other means.

  2. The Flickering Phantom
    Amidst the cluttered desk, an old desk lamp rests with an air of melancholy. Its once-sturdy switch is now broken, rendering it incapable of casting light. The lamp’s brass frame and tattered shade hint at a bygone era, but its current state is that of a forgotten relic. As you examine it, you can almost feel the faint remnants of its past flickers, a ghostly reminder of what once was. Discover its history and perhaps you will find a way to bring its light back to life.

  3. The Shadowed Sentinel
    On the cluttered desk, a desk lamp stands like a silent sentinel over the scattered papers and forgotten trinkets. Its switch is broken, leaving it in perpetual darkness. The lamp’s once-vibrant shade is now a faded memory, its metal base tarnished with age. Though it cannot illuminate the room, it holds the potential for secrets hidden in the shadows. Investigate the lamp’s surroundings and uncover the story behind its long-forgotten light.

  4. The Unlit Lantern
    Resting on the corner of the desk, the desk lamp remains a relic of a bygone era, its switch broken and its light extinguished. The lamp’s ornate design is now overshadowed by its inability to function, casting only shadows where there once might have been a warm glow. As you delve into the mysteries of the desk, you must decide if you will seek to repair the lamp or if its true value lies in the clues it might offer about the past.

  5. The Inactive Illumination
    A desk lamp of distinguished craftsmanship sits idly on the desk, its once-brilliant light now a distant memory. The switch, a critical component of its design, is broken, leaving the lamp forever inactive. Its elegant base and aged shade suggest a history of elegance now overshadowed by its current state of disrepair. Investigate the lamp to unlock its potential, or explore the desk’s other items to find a solution to reignite its lost brilliance.

Feel free to adapt these descriptions as needed for your adventure game!

Here’s how I would curate this realistically:

A desk is a supporter in Study. A lamp is a device on desk. "On the cluttered desk, a desk lamp stands like a silent sentinel over the scattered papers and forgotten trinkets." Understand "desk lamp" and "light" as lamp. Understand "scattered/papers/forgotten/trinkets" as the lamp. After switching on lamp, say: "Its switch is broken, leaving it in perpetual darkness." The description is "The lamp’s ornate design is now overshadowed by its inability to function, casting only shadows where there once might have been a warm glow."

It’s decent enough filler text for an object that otherwise wouldn’t get much authorial attention. But notice all the stuff it suggested that doesn’t fit, which is why you don’t just want to turn AI loose on a story.

5 Likes
20

@Jason: or having the AI generate dozens of bark, leaf, stone, wood, etc. textures to give natural environments and structures built with natural materials variation instead of copy pasting the same pattern a million times… granted, some of this can probably be automated just as well with dumb algorithms.

Though another thought that comes to mind is Generative AI versus Programmer Art. Games tend to be collaborative projects because it’s rare for someone to be good at all of the creative skills that go into making games, but in a sense, programmers are the most important since without a programmer, you don’t have a game, so if a solo creative produces a minimal viable product of a game, they’re probably a programmer and are likely in need of a graphics artist to make the game look good, and in the absence of an artist friend as a collaborator or the funds to commission a third party artist, I suspect there’s a lot of programmers out there for whom AI art is a better placeholder than whatever they can wip up in whatever freeware paint program they have access to, and considering how a game looks is the thing most people notice first about a game, better placeholder art might make the difference between a game never earning enough to cover the cost of commissioning an artist and attracting enough to commission proper art and turn a profit for all involved.

3 Likes