Spirits in the machine

Before LLMs, we could happily assume that texts are not given. They do not exist of their own accord, grow wild like trees or drop from the sky like rain. They are written by a person. That person has made countless decisions, some of them reflexive, while others are deeply considered. Close reading is how we unearth what an author wanted to convey by these choices. Why this word, instead of that? Why is the comma right at this spot and why use a particular metaphor? We can ask all these questions and more of the text, because the work was written with intention.

An explainer of close reading from the University of York suggests that one of the first questions one might want to ask when close reading is: who is speaking? With LLMs, the answer can only be all the millions of us on whose material the LLM was trained on. There is no intender. There is no one trying to communicate something vital to us. There is just word guessing. A lack of intender disinvites close reading of LLM text, which is prolix and bloodless. As the academic Henry Farrell says in his blog, Programmable Mutter, "LLM art is not the product of any individual artist (even when it is prompted to copy an individual artist’s style, it tends to create a flavored composite of a broader genre). Like the ouija board, LLMs are a technology that transforms collective inputs - the enormous corpuses of words and human created content that they has (sic) been fed on - into apparently quite specific outputs that are not the intended creation of any conscious mind."

Human beings are hard wired to explore and try to understand intention. The science fiction writer and activist Cory Doctorow (whom Providence preserve) observes, "Figuring out intentions is our minds' reflexive preoccupation. It's what keeps us from dying in traffic, it's what lets us win at poker. It's the key to love and parenting, to effective management and "managing up." It's why we get mad at people, and why we forgive them. It's why our hearts race in sympathy with the characters in a book or on a screen. It's the automatic and irresistible starting point for every encounter with a poem, a song, a book or a sculpture. It's the most fundamental difference between a sculpture of a tree and a tree: the tree grew all on its own, while the sculpture was made, on purpose, by another person with another mind."

To Doctorow's point, the purpose or why another person made a thing, is part of how we delight in art. But unless you have a specific interest in doing so, parsing LLM output feels empty and dissatisfactory. There's no meaning or anyone there to seize your mind and heart in the embrace of human closeness. This lack of meaning is one reason I find LLM output so boring. The result is a feeling of unease when interacting with LLMs, something Cal Newport the writer, podcaster and academic, has experienced, "Like many people, I feel a general sense of unease when holding a fluent conversation with a chatbot. I understand that this lexical fluidity is merely an illusion – the result of endless matrix multiplications, autoregressively generating one token after another – but it still creates a discomforting sense of transgression."

LLMs can feel as creepy as conversing with a disembodied voice echoing in a room. Here's an answer, a voice from the void, but there is no face behind the mask. It's eerie in the sense that Mark Fisher described "there is something present where there should be nothing, or there is nothing present when there should be something."

Farrell explains that "LLM art sometimes seems to communicate a message, as art does, but it is unclear where that message comes from, or what it means. If it has any meaning at all, it is a meaning that does not stem from organizing intention. To adapt someone else’s phrase, we can no longer speak of intelligence; only the products of its decay remain. It is unsurprising that many people find these products to be disturbing... LLM art is so disturbing because it is culture that has been drained of all direct intentionality."

This feeling of eeriness perhaps speaks to why the discourse on AI is undergirded by mysticism. In some, LLMs conjure Hadean visions. One blogger referred to an LLM as a "Cheery demon" whilst a writer characterised Google Gemini's invitation to 'Help Me Write' as a "Satanic offer." To yet others they seem godlike. Churches have been founded to worship AI divinities. To wit, Way of the Future, founded by dacamillionaire and ex-Google AI engineer Anthony Levandowski and at one point registered as a tax-exempt religious organisation with the IRS. See also Roko's Basilisk, an AI thought experiment which comes with AI hell included for failing to contribute to its advancement.

Conjuring deities has a history as long as human existence. We developed this skill to help us cope with life's vicissitudes and myriad anxieties, such as fear of death, uncertainty about the future and concern for loved ones. Gods give us the illusion and comfort of controlling the uncontrollable.

But LLMs are neither angels, demons, nor gods. What they are is normal technology, and we should treat them as such. Whenever we feel the urge to ask the void, "Who is there? Who is speaking?" the answer is not omniscient gods. It's us, the aggregate human output used to train that particular LLM.

⚘