To be fair, I don’t think it’s possible to do much of anything in linguistics without running into Chomsky’s influence, like it or not. Even if I think he’s fundamentally wrong about universal grammar, basically every part of the field has been touched by that idea now!
3 Likes
Using an LLM to make a parser accept arbitrary input seems like using an LLM to make a crossword accept arbitrary input. Is it really more satisfying to solve the puzzle of engineering a prompt that will make the AI generate a valid input than to come up with that input yourself?
2 Likes
Well, it depends. Sometimes you’ve solved the in-universe puzzle, but get stuck trying to express that to the parser, and that usually isn’t the fun part. (Think about the classic complaints of “guess the verb”, “guess the noun”, and in especially awful cases “guess the adverb”.)
For example, there’s one problem in Adventureland that Graham Nelson famously never solved. You have some bees in a jar that will suffocate if you wait too many turns, so how do you get them all the way across the map? This is a two-word parser, no prepositions allowed in most cases.
The answer is POKE HOLE. I imagine you thought of that as a potential solution in real life right away, but it’s in no way obvious how to express that to a two-word parser, since it doesn’t refer to any nouns actually visible in the world.
1 Like
Hanon has concerns
I suppose it’s good that you generously offer to immediately stop using 9:05 for this public experiment if Adam responds to you. This kind of rings of “Instead of asking permission, ask for forgiveness…” I understand you likely were wanting to use a small size game and didn’t think much of it, and in most cases the output of games is public domain but “remixing” of original code is sometimes a reserved right hinging on whether the use is for monetary gain or not.
In the future I’d recommend getting explicit permission from an author to use their work instead of assuming implicit permission for this type of experiment. That’s one of the things that really puts people off AI in general - it needs fodder for its model, and so many people just consider that everything on the internet is free for use.
There are enough people participating here in real time that you likely would be successful if you requested someone offer up a game for experimentation.
Adam Cadre has been away from the IF scene so he may or may not even be aware you are doing this unless he’s lurking here incognito or without logging in, which I can’t be sure about.
13 Likes
Sure, but putting an AI in between doesn’t mean you’ll have an easier time getting “poke hole” typed in. It just changes the puzzle from typing “poke hole” to getting the AI to type “poke hole”, which actually sounds harder, unless the AI has specific knowledge of that game.
But I meant the comparison more generally. The point of the crossword form is that it’s constrained. If you modify it to translate “dog” into “hound”, it becomes less fun, because you skipped the whole procedure of following the clues. Parser games are a bit waffly over whether they want to be guess the word puzzles, and as an ideal, they usually want specific actions rather than specific phrasings. But ultimately, a game has implemented certain actions, and what those are matters aesthetically. I don’t see how bringing the player out of contact with the actual functioning of a puzzle can enhance their appreciation of it.
4 Likes
I got into a bit of a loop. I keep on trying to put on clean clothing, but the LLM converts it into “wear clothes” and that creates a disambiguation loop.
Thanks for your concerns, Hanon. We just heard back from Adam and we’re all good. Love the IF community
2 Likes
Thank you for taking the time @NegSec – on it – will push a fix by EOD. Thank you
Michael
@Draconis that is exactly the problem we are trying to solve – and hope LLMs can help with. though part of me wonders if the best path in drafting LLMs to IF would be to start with a clean sheet of paper, and reimagine gameplay. For example, LLM’s can categorize sentiment pretty well – so movement through a game could be determined not only by WHAT you say – but HOW you say it. . berate a game character with profanity and you will not receive anything but insults right back – ask in the obsequious, gracious manner and you get a clue. . .
Or maybe it’s best suited for an even older form of puzzle/riddle – I remember reading these old Anglo-Saxon poems where the speaker or object describes themselves in a series of statements from their perspective and you have to guess what/who is talking. Could be an interesting puzzle that relies on an LLM’s vast knowledge of synonyms and double meanings. (here’s one :
When I am alive I do not speak.
Anyone who wants to takes me captive and cuts off my head.
They bite my bare body
I do no harm to anyone unless they cut me first.
Then I soon make them cry.
Answer. . an Onion. (Gotta love those Anglo Saxons – they had a LOT of time to kill sitting around the campfires waiting for the Danes to invade.)
1 Like
So, I just took a stab at this, and the first thing I notice is that it seems to think commands in Inform games are separated by commas (they aren’t).
For example, “Swipe the keys and wallet” becomes “take keys, take wallet”. The underlying game, I think, then looks for an item called “keys, take wallet” and chokes. Similarly “Get out of bed and go into the bathroom, quickly” becomes “Get bed”, “Go bathroom” (yes, with quotes, too). “Get bed” is itself a weird result.
For reference, in Inform games, commands can be inputted multiple at a time but separated as sentences with periods/full stops. So Lava Ghost’s command should be TAKE KEYS. TAKE WALLET or TAKE KEYS AND WALLET.
1 Like