Is AI Generated IF Discussion OK Here?

Site Feedback
161

Hi,

  1. In that example, nowhere, it was just a light-hearted example to show what I meant about natural language. You can make those decisions as you always have, and edit the first draft of code yourself or add that detail in via the prompt. I haven’t been using it as a one-hit prompt thing, I’ve asked for a base of code and then worked on the first draft to add detail in, then repeat. To your followup about pulling ideas from IFDB, nothing is stopping you, you can still do this.

  2. I don’t know because I’ve never used Inform myself (there are different systems available), for ZIL it’s 90% there and improving fast.

Thanks

Adam

162

I’d agree with that to be fair, I did preface it by saying it was just a supposition with no real evidence.

But with respect you’re doing that somewhat yourself, by stating “we wouldn’t use this hypothetical Inform 8”, I accept that you wouldn’t, but others may, neither of us know this for sure.

Not everyone uses Inform either, so it may (or may not) be used by another design app, not at all.

Thanks

Adam

1 Like
163

Ok, I’m going to wave a white flag at this point! I’ll go back to quietly tinkering with ChatGPT-ZIL and occasionally sharing what I’m finding, what’s working, what isn’t, and any interested parties can watch and others can ignore, and we can each do our own thing; as it should be.

Thanks

Adam

164

This is a crux for me as well. If I make a game using Inform 7, and I implement an NPC, I know, without a doubt, what will happen if I ask that NPC what their opinion is on, say, Oreo’s. Every time, all the time, I can depend on players receiving the same deterministic response (most likely an error or an I dunno in this case).

Now, if I were to use this hypothetical “Inform 8” with ChatGPT under the hood, I may get a plausible Oreo response, but I won’t get the same plausible Oreo response, nor a predictable one. I’ve just introduced a black box into this process. Not only don’t I know what the NPC will say in regards to an Oreo inquiry, but I can’t possibly know, and I can’t confirm that this oreo response will even be the same oreo response for each player, or even the same player if they were to ask twice in one playthrough. Likely it won’t be the same, as the black box inside is constantly being tweaked and is essentially a live service.

My game is no longer entirely deterministic, and the lines between what remains deterministic in the game and what is fueled by an infinite pool of plausible sounding results is not visible to the player. This would make both bug reporting and bug fixing a nightmare. It also introduces a great way to mislead the player, who typically assumes everything visible in a game is specifically curated to be there. If the NPC says they would do anything for an oreo, a plausible sounding result, the player could easily take that as an in-context hint to find an oreo to get the NPC to do something for them. This is just one example of a potentially infinite number of ways a player could be accidentally led astray by this “Inform 8”.

Also, I really chafe on the idea of having that built in because of the assumptions it makes. You can’t use Chat-GPT offline, it’s a live service right? You also can’t guarantee it will not change or even go down altogether in the future. What happens when your game is integrated hooks into that server? We already have issues with that regarding games that were built with anti-pirating mechanisms that check for a live copy-protection server before allowing the player to install. The problem is that some of these companies no longer exist, and these servers are long defunct, meaning people that had legitimately bought a physical copy of the game can no longer install or play it, even with period hardware, without breaking the terms of service and basically pirating it themselves. (The PC version of Fable 3 comes to mind, buy there are others.)

I can play Collosal Cave Adventure today, no problem, with no internet connection once I download it. I don’t like assuming all potential players have consistent reliable internet connections. I don’t like assuming the for-profit entity maintaining the very energy intensive LLM under “Inform 8’s” hood is going to be there forever. I don’t like the idea of baking in a live service requirement into an engine used by tens of thousands (at least) worldwide. I want to know that, with understandable effort to emulate both compatible hardware and software, that it will at least be possible to play a game I write in Inform 7.

I know specifically that the code assistance with ZIL is produced using ChatGPT, but the product of that doesn’t require the player use ChatGPT, and I understand and appreciate the distinction. I’m just responding to the concept of an LLM infused IF engine, like the hypothetical “Inform 8.” If you want to see me really get up in arms about the topic, go have Inform 11 bake in an LLM into the software. I think the response to such a hypothetical “Inform 8,” assuming it worked via an active LLM and not witchcraft or a quantum link to a Boltzmann brain, would be far more mixed than some expect.

tl;dr: Not a fan of an LLM infused IF engine for reasons (see above).

7 Likes
165

To clarify here, because you took what I was saying in a much different direction than I intended, I was specifically responding to the idea of having a hypothetical inform 8 that takes natural language instructions from you and then produces code for you under the hood, ostensibly to do what you described to it in natural language, in a sort of semi-analogous way to how inform 7 takes its cobal keyword soup and turns it into inform 6 or VM instructions, I forget which. My problem would be the application of that non-deterministic black box to the process of programming itself. I agree with your problems with using it as the game engine itself, but that isn’t what Adam is doing, so that’s not what I was responding to.

3 Likes
166

That’s fair, and not as egregious, although I would still have issues with requiring writers be online to use “Inform 8.” In general not fond of software that should run locally in theory but still requires you have an active Internet connection to run it anyway. Also, sorry for numerous typos above. Apparently you can’t fix them with a topic in slow-mo. TIL.

1 Like
167

I think I was amalgamating my thoughts on the periodic individual showing up to the forum with a virulently enthusiastic LLM-Infinite-IF engine proposal into the more specific discussion here. You know, the “ChatGPT is the parser now!” proposals.

Normally, I would have edited my above response to include this thought, but Slow-Mo.

3 Likes
168

Pedantic nitpick: It may be unreliable and unpredictable, but a given AI with identical training data and query should always be deterministic. Introducing randomness would in all likelihood harm the output more than it would improve it.

169

LLMs to produce large-scale formal artifacts (like “Inform 8” or an original IF writer) are hopeless. As “code completion on steroids” they’re a godsend.

1 Like
170

This actually isn’t true. Different people get different results for identical prompts on the same day from the same Ai service, and the same person can even get different results for the same prompt on the same day with the same AI, this is why it is possible to ask chat GPT to regenerate it’s answer to a prompt in the hopes of getting a better answer. This is because most AI models introduce some level of randomness into the system, called a heat level I believe. You can turn that all the way down to zero, but even that doesn’t appear to make these things deterministic, and it also makes them significantly less creative and useful.

8 Likes
171

Yeah, especially for language models, it’s important to introduce a bit of randomness to keep it from getting too repetitive. To dramatically oversimplify things, a “language model” (the LM part of LLM) generally involves assigning probabilities to different sequences of words, which lets it incorporate some randomness for variety while still keeping things overall not too improbable.

2 Likes
172

I know I said I was exiting stage left, but as I was writing my video narration it turned into more of a blog so I thought I would share it here also; hopefully this does a better job of explaining how i’ve been using ChatGPT with ZIL.

1 Like
173

I stand corrected. Simpler ML models are definitely deterministic. Ugh, I don’t see how adding randomness would improve anything, and it makes understanding the relationship between input and output that much more inscrutable.

3 Likes
174

This is a huge reason why so-called prompt engineering tends to resemble Blackmagic or cargo culting more than anything remotely sane. Although it isn’t the only reason, since these are black boxes we don’t really understand in any case.

At least in my experience it does seem to improve things to be fair. Essentially it’s a way of slightly jolting the generative model out of being a pure regression to the mean, and actually doing something interesting once in a while. It’s actually very fun to turn the randomness factor all the way up to the max and watch it do a very interesting and complex form of word association that often produces fairly evocative (if nonsensical) poetry.

4 Likes
175

Actual randomness seems completely unnecessary to achieve this. A fully deterministic input adjustment value would accomplish the same thing while keeping reproducibility. I really truly dislike seeing randomness injected into things that don’t need it - and very few things outside of cryptography actually need it.

3 Likes
176

All I can say is:

Good luck! I’m not following in those footsteps, and although I don’t like it, I won’t chase after you. I do me, you do you. But just make sure you credit ChatGPT as a co-author if you enter it in a competition…

5 Likes
177

It’s a statistical model. By definition it uses randomness.

Probabilistic algorithms have their utility, even outside of machine learning and cryptography.

4 Likes
178

There’s no need for actual randomness. Without an actual hardware RNG, all “randomness” in code is really pseudo-random generated from a seed and algorithm. Being able to reproduce the seed and state of the PRNG gives you full deterministic behavior, while maintaining unpredictability and able to meet any statistical test you need. Maybe that’s what they use behind the scenes, but not exposing that level of state and control to the end user is…disappointing and unnecessarily limiting. As an aside, I find OpenAI’s name hilariously ironic. They’re about as transparent as a brick wall.

2 Likes
179

Before GenAI, there were many programs for generating images, such as Evolvotron. These programs tended to use standard mathematical functions, although the resulting images could be quite “creative”. What about them? Should they be credited when publishing?

As for the use of generated images. I don’t see much difference between a generated image and an image taken by a photographer. Why doesn’t anyone argue that a photographer has a copyright on an image (e.g. of nature)? Even though he didn’t create the composition, he just clicked a button and selected the best shots.

2 Likes
180

You seriously going to make that argument? Aside from taking a photo of a gray card, almost all photos took artistic skill (to varying extents…) Even a landscape photographer does far more than just push a button and choosing the best shots: they choose the location, framing, camera format, time of day, exact timing to include or exclude moving elements (clouds, birds, cars, etc), exposure, shutter speed, as well as any post-processing steps. Of course nature and landscape photographers have copyright on their photos!

7 Likes