Hallucination, Orwell and dying metaphors

AI can be biased and inaccurate but it does not hallucinate!

Dear Readers,

Welcome to another edition of the S.A.D. newsletter, where we explore the world of software, algorithm, and data. Today we'll cover topics such as hallucination, language, and, of course, chatGPT. It seems like chatGPT is everywhere these days — at work, in casual conversations in the grocery store aisles. People often mention that chatGPT hallucinates, and opinions on its usage range from fascination to caution.

Does your refrigerator hallucinate? No. So chatGPT or AI also do not!

I find myself cringing more and more at these statements. Let me make it clear: chatGPT does not hallucinate! $%#@&!!!$%#@&!!! It is simply a tool (a powerful tool, no doubt) that applies statistical methods. That’s it. It is just an applied statistical software. Let’s keep it that way and stop mentioning the word hallucination! And of course, this does not mean what chatGPT or other model generates is all good and dandy, correct, or reflects reality. That is another question. AI can be wrong or provide inaccurate information. But it is not a matter of hallucination!

However, it's understandable that technology can be anthropomorphised. Perhaps the random stranger I encountered is not to blame; social media, pundits, and academics are leveraging these metaphors to ride the hype (including me here!), and we are simply responding to that.

The following advice from the newsletter

AI Snake Oil

is spot on. They mention that even though anthropomorphising might be helpful sometime to get the idea across, developers should steer clear of behaviours that encourage the anthropomorphisation of AI:

Developers should avoid behaviors that make it easy to anthropomorphize these tools, except in specific cases such as companion chatbots. Journalists should avoid clickbait headlines and articles that exacerbate this problem. Research on human-chatbot interaction is urgently needed. Finally, experts need to come up with a more nuanced message than “don’t anthropomorphize AI”. Perhaps the term anthropomorphize is so broad and vague that it has lost its usefulness when it comes to generative AI.

Before we proceed, let's clarify a what we mean by AI here. When we refer to generative AI, we're discussing a subset of AI that focuses on creating new content, such as images or texts. It uses machine learning techniques by analysing data distributions and the probability of specific samples. Large Language Models (LLMs) are one example of generative AI models that use — “large” language data for training. When given a prompt, they can predict the next sequence (in the case of text-based examples). Clear so far? Great. So, let's remember: chatGPT and AI do not hallucinate, alright?

One way we anthropomorphise is by using metaphors. But we need to get our metaphors right. George Orwell wrote the following in 1946:

Dying metaphors. A newly invented metaphor assists thought by evoking a visual image, while on the other hand a metaphor which is technically ‘dead’ (e. g. iron resolution) has in effect reverted to being an ordinary word and can generally be used without loss of vividness. But in between these two classes there is a huge dump of worn-out metaphors which have lost all evocative power and are merely used because they save people the trouble of inventing phrases for themselves. Examples are: Ring the changes on, take up the cudgels for, toe the line, ride roughshod over, stand shoulder to shoulder with, play into the hands of, no axe to grind, grist to the mill, fishing in troubled waters, on the order of the day, Achilles’ heel, swan song, hotbed. Many of these are used without knowledge of their meaning (what is a ‘rift’, for instance?), and incompatible metaphors are frequently mixed, a sure sign that the writer is not interested in what he is saying. Some metaphors now current have been twisted out of their original meaning without those who use them even being aware of the fact. For example, toe the line is sometimes written as tow the line. Another example is the hammer and the anvil, now always used with the implication that the anvil gets the worst of it. In real life it is always the anvil that breaks the hammer, never the other way about: a writer who stopped to think what he was saying would avoid perverting the original phrase.

George Orwell by Mike Scott. Image source: https://pixels.com/featured/george-orwell-mike-scott.html

In a previous issue, I highlighted the absurdity of comparing big data to oil, as it only “fuelled” (see what I did there!!) the hype surrounding big data without providing meaningful insights. Similarly, we must now refrain from using the term "hallucination" (and other dying metaphors) in reference about generative AI, as it lacks coherence and, more significantly, conceals the underlying power and structure of the generative engine, as George Orwell astutely suggested.

I was pleasantly surprised to observe that even mainstream media outlets, typically not focused on AI and technology, expressed similar cautions. But we keep still hearing the term hallucination!!! Rachel Metz succinctly captured this sentiment in her April 2023 Bloomberg article, stating:

As a rapidly growing number of people access these chatbots, the language used when referring to them matters. The discussions about how they work are no longer exclusive to academics or computer scientists in research labs. It has seeped into everyday life, informing our expectations of how these AI systems perform and what they’re capable of.

Similarly, The Economist, even though more on the neoliberal and hawkish, side, surprisingly is on the same wavelength as Orwell, talking about how anthropomorphisation is natural but can be dangerous.

But ai is too important for loose language. If entirely avoiding human-like metaphors is all but impossible, writers should offset them, early, with some suitably bloodless phrasing. “An llm is designed to produce text that reflects patterns found in its vast training data,” or some such explanation, will help readers take any later imagery with due scepticism. Humans have evolved to spot ghosts in machines. Writers should avoid ushering them into that trap. Better to lead them out of it.

[ W ] Kim Whanki - Untitled, CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=52231680

Our use of language, particularly employing dying metaphors, creates a slippery slope that tends to attribute agency to AI applications where none exists. While these metaphors can sometimes be useful, they can also be misleading. AI does not hallucinate; it is humans who do. AI learns from data and models provided to it, generating outputs based on that information. Interfaces like chatGPT may give it a human-like appearance, allowing it to respond and converse, but fundamentally it is a powerful and sophisticated statistical machine—an application of automation and applied statistics. That’s it. While the scale and impact of AI will undoubtedly be profound in our society, let's refrain from associating hallucination with it.

And don’t forget, hallucination has a far more complicated historical and cultural significance. When talking about hallucinating I am thinking about unusual sensory experiences. I am thinking about Moses and his burning bush, Paul on the road to Damascus. I am thinking about psychoactive chemicals. I am thinking about the Beatles and Lucy in the Sky with Diamonds!

Speaking of the Beatles, the recent story about an AI and Beatles song highlights how poorly we often explain and comprehend AI. The headlines might make it appear as if something magical occurred, suggesting that John Lennon has returned from beyond with the aid of AI:

“The Beatles will release a final record, using John Lennon's voice via an AI assist” — NPR

New Music From The Beatles? Thank AI - Time Magazine

Paul McCartney Says A.I. Helped Complete ‘Last’ Beatles Song - The New York Times.

Again, a bit of agency here. However, this is not the case. The reality involves a combination of statistical analysis, computational speed, and machine learning techniques. AI is not creating a hallucination of Lennon. This underscores the issue of misused metaphors, language traps, and the general lack of public understanding surrounding AI. According to Sean Ono Lennon this is what actually happend:

All we did was clean the noise from the vocal track. People are completely misunderstanding what occurred. There have always been ways of ‘de-noising’ tracks but AI just does it better because it learns what the vocal is and is able to very precisely remove everything that is not the vocal.

Verity Harding, a Researcher at Cambridge has a summary of the Beatles AI episode that aligns nicely with the language trap idea:

The fevered reporting around the Beatles’ use of AI illustrates well the serious issue that a lack of public understanding of AI systems and capabilities affords a disproportionate voice to those who claim to be expert, even if that does not reliably relay to the wider populace what is actually realistic. The reason that the AI extinction letter, and the ‘six-month moratorium’ letter before it (we seem only to communicate in open letters in the AI industry these days) received so much coverage is that they were signed by people with real technical expertise. But just because someone is extremely technically adept, does not mean that they understand much else about how the world works. At a recent appearance in Cambridge, the former Google computer scientist Geoff Hinton confessed that he knew a lot about machines but not much about people, power, and politics. This hasn’t prevented Hinton from being listened to and courted by politicians across the world about how they should handle AI. 

While discussing AI we should be cautious of using human-like metaphors. We should aim to instead provide clear explanations of how AI works to avoid misleading readers into attributing human like qualities to machines. So let’s not say anymore that AI hallucinates!