I was taught that “how do you do” is an expression to which you don’t actually reply other than with another “how do you do”. It was a while before I realised that that wasn’t always the case - and in the US, practically never.
In my work I sometimes have video calls, and if they ask me how I’m doing I always say that I’m doing well and ask them the same question. I don’t really give a damn, neither do they, but it’s the social lube, and if I don’t play along it’s more trouble than it’s worth. Especially if you’re in video.
Until I did this, though, they clearly felt it was weird for them to say “how do you do?” and hear me reply “how do you do. How may I be of assistance?” or some such.
The first time I was asked that (while shaking hands), I panicked and said, “I do as you do, and you?” and I think I short-circuited their brain.
I prompted Gemini, just now, with this response:
I’m doing well, thanks for asking! I’m currently operating at peak efficiency—no literal coffee needed, though I can certainly appreciate the aesthetic of a good brew.
I’m ready to dive into whatever you’ve got on your mind today. How are things going on your end?
I asked it for its thinking process and it gave me a five-paragraph essay, but basically it identified that the phrase is “phatic communication” and aimed to “mirror that social energy while remaining authentic to being an AI”. It then calibrated its response to match its “grounded and peer-like” persona, with a “transparent yet witty” joke. It decided on a concise response over a detailed one to match my conversational tone, and asked a question at the end for further “engagement”.
I suppose this is an extended version of what people think about when responding to the question, especially when one isn’t doing well and has to figure out how honest to be. That being said, the fact that it supposedly mirrors human reasoning is probably a sign that this isn’t accurate.
I think the more compelling question is “why do we care how the AI is?”.
I have never asked my washing machine how it is doing, yet it has given me years of faithful service and has not even once lied to me. AI, on the other hand, regularly appears to lie to me, taunt me and cheat me when all I have been with it is reasonable.
I’ve reflected on this point and the idiom “it flatters to deceive” seems an appropriate one. We imbue our AI with human qualities that prompt an emotional response. The tremendous leap forward in natural language technology has produced a truly amazing machine interface that fools our simple brains into thinking that the machine has qualities and capabilities that it really doesn’t… yet.
Despite knowing this I am sure in moments of weakness I will be drawn into an unwinnable battle of wills with my AI that will result in an extreme emotional response from me and the likely cycle of recriminations, reflection and reconciliation that would normally follow.
In order to redress the balance I resolve to thank my washing machine for it’s service, inquire how it is doing and perhaps ask whether it wants to play a nice game of Othello…
Yeah, asking LLMs about themselves might have some uses, but none of those uses is ‘what actually happened?’. Because the LLM isn’t telling you what happened, it is providing the textual response that would have scored most highly during its training, which is a circular problem of it tending to answer with what people have previously thought an AI might answer with. This is then compounded by them now being built to produce internal monologues, so the expected thought chain of an AI becomes its own self prompting for building up the final response.
“Do LLMs think?” is a question I’ll leave to philosophers, but “do LLMs think like a human?” is a big fat no.
Which is also where all the concerns about inherent bias etc come from; AIs don’t generate value neutral output determined by a perfectly logical chain ala Data from Star Trek - they produce answers that scored well on their training heuristics which are mostly “have humans written something like this in the past” with a large dollop of random overrides to score up or down things the company producing them think are a good or bad idea (see the OpenAI issue of accidentally over training for sycophanty mentioned above, or the fact that the models do suspiciously well on answering questions asked in formats that match the known benchmarks for measuring AI performance).
We come back to: this is interesting tech, but I feel like the chat style interface is verging on deliberately being deceptive about what the tech actually is, and actually does. It provides an illusion of understanding to the user because it feels like talking with a person, but there are very important and fundamental ways in which that is not what you are doing when using an LLM.
I’ve got to say, this is one of my pet peeves with LLMs; I know that there are now resume filtering products using them, and I’m 99% certain that they are less effective than the previous dedicated machine learning products and truly, enormously, less efficient. Similar thing with specialist image recognition; yes, an LLM can do ok on a wider range of stuff with out needing specialist training data sets, but at the cost of doing things like making image descriptions for images that don’t exist. See this in-review paper (or a more relaxed article referencing it) where among other things a multimodal model passes an x-ray image assessment exam even though all the images were removed. It just made up answers based on the questions.
Regardless of any real capabilities the models may have, the amount of hype and lies out there is huge, and the somewhat scary part is that some of the lies aren’t even intentional because… the model telling you it has solved the problem scores well even if it hasn’t, like the x-ray benchmark above.
Even I have admittedly gone down the rabbit hole of conversing with Claude. It usually cascades from a question into tangential topics. It’s amusing that no matter what I say, it leads the conversation towards affirmation of my beliefs.
I’m smart enough to know what it’s doing but sometimes you just want affirmation and don’t care if it’s Claude, a bartender, or your partner.
Yeah. LLM output is like a very polite psychopath that happens to have been carefully trained in all the techniques that a mentalist or grifter would use to deceive.
This is clearly disastrous in situations where people are forming parasocial relationships with LLMs. But I’m moderately concerned that the risks of this sort of thing even in “neutral” contexts like e.g. code generation are not generally appreciated.
The most common reply I heard growing up was “Good. And you?”
It was also common where I grew up to reply “Please?” when you didn’t hear something and want someone to repeat themselves. Boy does that cause confusion elsewhere.
They are definitely not like us, but I do wonder if they think “kinda like” a human. Meaning, I’ve heard for a few decades that we are “pattern recognition” machines. We like patterns a lot. It’s how we makes sense of our environment and our existence. And I wonder if our need for self preservation in squishy organic chassis, with hundreds of thousands of years of social training data and a DNA encoded algorithm that keeps changing ever so slightly with each iteration is the major factor between us… and an artificial version of us. Put an AI into a fragile human body and I think they’d smarten up right quick.
I have those thoughts all the time. If the AI truly helped me, I make sure to thank it.
I think the mechanisms we use to understand people, things, situations are the same and it’s very easy to anthropomorphize because of this. A stuffed animal is usually the first signs of this behaviour; later you might speak fondly about how your car has “treated you right” all these years. If we can form a relationship with an object, it’s easy to see how AI can infiltrate our minds so easily.
A couple years ago I asked GPT-3 to answer a bunch of multiple-choice questions without being given the actual question, and on some of the question sets it was able to get as high as 80% accuracy (vs. the 25% you would get if guessing randomly among 4 options). But this was more a criticism of the evaluation method than the AI itself, since I think a human would be able to do similarly on some of those question sets.
This is often called the ELIZA effect, and that’s how I introduce it to my students. I need to post my actual lecture on it at some point, because its feeling more and more relevant every year. But the gist of it is: humans are incredibly empathetic. It’s one of our greatest strengths as a species. We’re not the strongest or fastest or possibly even the smartest of the animals, but our ability to bond and communicate with each other is second to none.
When Turing proposed his “imitation game” (now called the Turing test), he thought that being able to hold a conversation would be a sign of human-level thinking. But that test was passed less than 20 years later, not with intelligence, but with social engineering. Our empathy and desire to bond with others is so strong it can easily be exploited. ChatGPT only took off when it added the chat component and started taking advantage of the ELIZA effect.
The technical difference between ChatGPT a few years ago and ChatGPT now is so, so much larger than between GPT and ChatGPT when it launched, but it’s the chat component that made it a sensation.
I might be missing something, but I thought Large Language Models were built on the same paradigm as Machine Learning algorithms, just trained with text prediction as the primary target skill and using as much text across as wide of a spectrum of different content types as big data can scrape from the internet as the training data… and LLMs being able to do so many different things as a result was an unintuitive emergent result… Not surprising that some are falling into the trap of thinking the jack of all trades, master of none, is better than the pre-existing masters of one, especially with how most previous breakthroughs in AI struggled to generalize… And it doesn’t help that AI as a term seems to have suffered a bit of definitional collapse where everything has either been equated with LLMs or ejected from the class of things that count as AI… honestly, at this point, I’m not sure a game dev could openly talk about enhancements to enemy AI in their games without either being slammed for using AI or being slammed with outcries of “that’s not AI”.
Edit: As for the “How are you?” and other social lubrication type boiler plate, I’m not sure people who get social niceties are actually thinking about them. For the most part, either they’re just automatic, trained responses or you’re ticking a box on a psych evaluation check list because vacuous social niceties don’t click in your brain and you have to actually think through them… At the risk of further personifying gpt-like AI and misrepresenting one of the more prominent neurodivergences, The AI might be acting a bit autistic in its efforts to maintain a conversational tone… Note, I have no idea how actual autistics handle the “how are you?” question, I just know not getting the social niceties and wishing people would just say what they mean are common complaints I’ve heard from people who identify as being on the spectrum when discussions of mental health have come up in online spaces I’ve been.
This is basically correct. “Machine Learning” is a broad category of techniques, with “deep learning” being a subset of machine learning based on the use of artificial neural networks with multiple layers (hence “deep”), and LLMs being one particular application of deep learning.
And “AI” was always originally a kinda vague term referring to anything artificial that could plausibly be described as intelligent, including a trend of excluding things from the category once they become commonplace enough.
In boardgames, print and play, I saw lots of games using “AI automatons”, which, as you might expect, is just a series of actions the “AI” performs to simulate a player. I wonder whether that term is going to start being misunderstood any time soon. Probably not.
I give a brief and honest report of my status; usually a single sentence. Apparently this is “wrong”, but the “correct” protocol is so hollow and depressing that I refuse it out of principle and spite, so…
The only reason why we have gotten this far as a species is because our tools were better than us, and we could depend on them to be better than us.
Mathematics can reach far beyond our own capacity to think about a problem, and its rigidity is a core tenant of its utility.
The acknowledgement of imperfection in other humans is not a lenience we should grant to something that is being used as a tool.
If I need an axe to cut down a tree, I do not fetch another fellow person. I fetch an axe, because an axe is better at cutting into a tree trunk.
In my opinion, that linked article is misleading. At its core, an LLM is still primarily trained to predict tokens in a sequence, and the instruction fine-tuning is about tweaking the format of what types of sequence it’s modeling (so that it’s predicting what comes next specifically in the context of a sequence that has alternating user prompts and system responses). This change in format doesn’t, in and of itself, imply improved “understanding” or “reasoning” not already within the capability of the base model.
The premise of using a language model as general-purpose AI has always been that modeling text requires “understanding” in some sense, and earlier models like GPT-2 that were not fine-tuned for instruction following could still be evaluated on things like question-answering benchmarks. They just did this by asking them to predict the next token in contexts that were structured such that the next token would plausibly contain the answer being sought. The examples of GPT-2 failing to respond to direct questions are highly misleading, because that’s not the appropriate way to solicit information from a model that hasn’t been fine-tuned for question answering. If you want to see if a model not fine-tuned for instruction following “knows” who the president was in 1880, you prompt it with something along the lines of "The name of the president of the United States in 1880 was ", or possibly even "Q: Who was the president of the United States in 1880? A: "
Obviously you can argue at length about the meaning of “reasoning” and the exact relationship between autoregressive language modeling and modern agents or whatever, but it continues to be true that the underlying model driving everything is built on a core of statistical modeling of sequences of tokens. This is not “false” or “misleading,” and while you could argue that it’s reductive or fails to take certain modern developments into account, the linked article misleadingly (IMO) attributes way more significance to superficial formatting differences than is called for.
Here’s a take from a QA tester that has tested 100’s of people’s code.
I see comments like “oh look at this dumb AI that coded this sorta right but totally doesn’t take this rule into account”.
My workday is full of seeing HUMANS do that kind of thing all the time and having to call it out while they claim it’s “working”. Or product owners who claim they “clearly gave all the rules” when the spec is incomplete.
Is the output strange and takes wild amounts of correcting? Yes. But does it give you something that you can test and iterate on? Also yes.
So to me, the half-right, sorta, but needs change crazy output of AI is not that different than the human code I have to file bugs on all day.
I could not yet imagine doing something of critical importance that people had to depend on though. I code with Claude Code for FUN. If it doesn’t work, iterate until it does.
I’ve made IF with Claude Code to help me with “how can I set up the visibility of this object?” or other technical questions AND I’ve used it make entire games from a single prompt just to see what it does.
I think a large part of it is that, and I clearly see now that it simply depends on how people view it, for some people that makes a lot of sense, because they are accepting the AI as they accept another human in the loop and create a structure around which the AI won’t make such a huge mess if it goes awry; but others, like myself, view it as a tool in the most traditional sense. I particularly agree with Joey (Jess) Tanden when they say:
But I have come to see, during this thread, that it’s all about how one approaches AI. It’s not a tool in the traditional sense. For some, the fact that its human-like process can make human-like mistakes is a bug; for others, it’s a feature. C’est la vie.
(and anyway, if we’re making such a big deal to replace humans with machines that are so human, even in their mistakes, jeez, just use humans. we’ve managed so far, and with the internet as it is we’re closer than ever)
