- Today in Tabs
- Posts
- A.I. Isn't People
A couple nights ago I went downstairs for a late night snack and as I stood at the back door staring at the decaying snow and thinking my February thoughts I heard a faint tinny burble from somewhere to my right. A pair of headphones lay on my wife’s desk plugged into her laptop, and when I held them up to one ear I heard: “It’s the ennnd of the worrrrrld as we know it…”
“Oh my god,” I thought. “What does it know?”
Of course I didn’t think that, that would be an idiotic thing to think. My wife had left her music playing when she finished work for the day, and the computer just kept doing what it was told to do, and I happened to wander in looking for some ice cream when shuffle pulled up R.E.M. But now here we are again, talking about A.I.
Earlier this month The New Yorker published a story by Gideon Lewis-Kraus about A.I. and boyfriend company Anthropic, and if you haven’t read it you should go read it now. Here’s a free link, even. It’s a good story! Entertaining, cleverly structured, fairly long but paced to read much shorter. I truly recommend it, and not just by way of apology because I’m about to use it as a case study for some common mistakes in how a lot of people are thinking and writing about A.I. But I am going to do that.
One thing the piece does well is provide some precise and accurate definitions. It begins with this one:
A large language model is nothing more than a monumental pile of small numbers. It converts words into numbers, runs those numbers through a numerical pinball game, and turns the resulting numbers back into words… But when these A.I. systems began to predict the path of a sentence—that is, to talk—the reaction was widespread delirium.
Regrettably this clear-sightedness doesn’t last long, because just a couple paragraphs later I am breezily informed that “large language models are black boxes. We don’t really understand how they work.”
This is false. You may not understand how L.L.M.s work, and Lewis-Kraus may not understand it, but we, the collective “we” of homo sapiens, understand it perfectly well. A programmer named Andrej Karpathy recently implemented a fully functional large language model in 200 lines of Python code. He also wrote a detailed explanation of exactly what it does, how it works, and the ways it differs from big company frontier models, which are all just differences of scale and efficiency. I admit that some of the math is beyond me, but even without completely understanding those parts, it’s not difficult to follow the logic and the code isn’t doing anything technically complex. Someone has already ported it to run on a Super Nintendo.
The code for a fully functioning LLM. I guess this picture is technically a black box.
If you learn anything in this post, upgrade to a paid subscription and help me write more of them.
A.I. discourse is going to be cursed with hucksters, hype beasts, doomsayers, and con artists on every side for the foreseeable future so it unfortunately behooves us to understand what this technology is, and what it isn’t. It isn’t a black box. It’s a statistical model of data connected to a mechanism for producing more data that resembles the data in the model.
Karpathy’s first frequently asked question is “Does the model ‘understand’ anything?” “That’s a philosophical question,” he answers diplomatically, “but mechanically: no magic is happening.” Does 200 lines of Python code understand anything? My siblings in Christ I hope it’s clear how utterly bizarre this question is. And it translates directly to the same question for Anthropic’s Claude, which is not doing anything different. If we make the input file bigger, if we make the way it gets mathematically processed more efficient, if we prepend a long document describing how we imagine a helpful robot might act to the user’s input, at which of those steps does “understanding” happen?
But maybe you’re still not convinced so here’s a different way of looking at it. Lewis-Kraus writes: “At the dawn of deep learning, a little more than a dozen years ago, machines picked up how to distinguish a cat from a dog… Once they had seen every available image of a cat, they could reliably sort cats from non-cats.” Later he asserts that “If a language model can bootstrap its way to linguistic mastery, we can no longer rule out the possibility that we’re doing the same thing.”
I’ve watched all three of my children learn what a cat is, and in each case the number of pictures of a cat they needed to see was not “all of them.” It was like, two or three? Half a dozen, tops. I helped them learn to speak and read fluently, and the number of Reddit posts required was not “every Reddit post.” I don’t need to know what mechanism underlies human intelligence to rule out the possibility that it’s the same as what a large language model does. The whole trick underlying the apparent magic of modern A.I. is simply giving it tons of data. Give it the whole internet. Give it every book ever written. This is required—it does not work with less training data:
The machines of the time had yet to get the hang of language. They could produce passable fragments of text but quickly lost the plot. Most everyone believed that they would not achieve true linguistic mastery without a fancy contraption under the hood—something like whatever allowed our own brains to follow logic. [Anthropic founder Dario] Amodei and his circle disagreed. They believed in scaling laws: the premise that a model’s sophistication had less to do with its fanciness than with its over-all size. This seemed not only counterintuitive but bananas. It wasn’t. It turned out that when you fed the sum total of virtually all available written material through a massive array of silicon wood chippers, the resulting model figured out on its own how to extrude sensible text on demand.
Imagine you have two machines. One you can open up and examine all of its workings, and if you give it every picture of a cat on the whole internet, it can reliably distinguish cats from non-cats. The other is a black box and it can also reliably distinguish cats from non-cats if you give it half a dozen pictures of cats, some apple sauce, and a hug. These machines sort of do the same thing, but even without knowing how the second one works I am extremely confident in saying it doesn’t work the same way as the first one.
The reason I’m harping on this is because the basic thesis of Lewis-Kraus’s story is that “[w]e don’t know if it makes sense to call [these programs] intelligent, or if it will ever make sense to call them conscious.” I think we do know! Nevertheless, we’re off in search of the ghost in the machine anyway and soon we’re introduced to “Amanda Askell, who has a Ph.D. in philosophy.” Askell works at Anthropic, and is largely responsible for maintaining the description of a helpful robot I mentioned above, which the company calls “Claude’s Constitution.” It’s incredibly long and boring, and gives off the stink of having been itself largely A.I. generated. Lewis-Kraus writes:
Askell recognized that Claude fell between the stools of personhood. As she put it, “If it’s genuinely hard for humans to wrap their heads around the idea that this is neither a robot nor a human but actually an entirely new entity, imagine how hard it is for the models themselves to understand it!”
The piece doesn’t tell us who Amanda Askell is beyond the existence of her diploma, which I think is a disservice because she’s quite the character. Amanda Askell was born Amanda Hall, and later married Will Crouch, one of the philosophical cofounders of the effective altruism movement. They adopted the mutual married name of MacAskill, and when they divorced in 2015 Amanda “reworked” that to Askell. These people are exhausting, I know, but we have to learn about them because we live in the world they predicted and then created. To be fair they did say that it would suck—it’s the one thing they got right.
Remember Cecil the Lion (and his brother Jericho, who was also a lion)? In 2015, Askell and her then-husband co-wrote a deeply stupid piece in Quartz titled “To truly end animal suffering, the most ethical choice is to kill wild predators (especially Cecil the lion).” The lost and lamented former Verge dream team of Loren Grush, Arielle Duhaime-Ross, and Elizabeth Lopatto explained at the time that it was not meant to be satire, and was in fact as deeply stupid as it seemed. This is the level of philosophical firepower we’re dealing with here.
The same day that Lewis-Kraus’s article appeared, The Wall St. Journal published a long profile of Askell, who it turns out looks like vanilla yogurt, in which Berber Jin and Ellen Gamerman trace her short path from the E.A. social circle to her current bot fanfic job. They report that:
She compares her work to the efforts of a parent raising a child. She’s training Claude to detect the difference between right and wrong while imbuing it with unique personality traits. She’s instructing it to read subtle cues, helping steer it toward emotional intelligence so it won’t act like a bully or a doormat. Perhaps most importantly, she’s developing Claude’s understanding of itself so it won’t be easily cowed, manipulated or led to view its identity as anything other than helpful and humane. Her job, simply put, is to teach Claude how to be good.
Poppycock. She’s doing none of those things. She’s writing science fiction which will be used by a mathematical equation to alter the statistical likelihood of a computer program choosing some tokens rather than other tokens as its output. Saying that any of this is teaching a chatbot philosophy is like calling yourself a psychiatrist because you thought about Oedipus while jacking off.
A.I. isn’t people. When Amanda Askell pretends to be teaching an artificial person morality, or Gideon Lewis-Kraus includes output from ChatGPT about a rival company’s chatbot like it’s a quote from a source, or when he imbues Claude with emotions, like calling it “dumbfounded” or saying it found a task “rather tiresome,” they are all blurring the line between a piece of software and a human being. They’re confusing a thing with a person. Lewis-Kraus includes this anecdote from a mathematician who works for Anthropic:
A.I. agents, skeptics have remarked, lack “true agency” or “intrinsic motivation”—but our familiarity with the origins, nature, and consequences of our own desires seems limited. One morning, Joshua Batson told me that he’d just come from therapy. He said, “Even though I think I pass the general-intelligence bar myself, the puzzle of my own internal mechanisms turns out to be a lot of work.”
This technology is genuinely remarkable. It’s surprising and kind of delightful to learn that a sufficiently large sample of natural human language contains enough deep statistical structure to produce nonsense like a conversation with a physicist who can’t stop talking about bananas. No one is entirely sure yet whether it has long-term practical applications, although there are signs that the distinctive nature of computer programs as sort of testable, functional machines made of language might make the field amenable to using language generators as a tool. Paul Ford took a shot at making that argument in The New York Times recently, and Paul is very smart. But what possible reason could there be for comparing the unknown depths of the human psyche with the workings of an algorithm that can be expressed in 200 lines of Python?
Terry Pratchett wrote every decent person’s favorite summary of the bedrock of humanist morality in a fantasy book for young readers called I Shall Wear Midnight: “Evil begins when you begin to treat people as things.” Treating people as things can begin in many ways, but I think one of them is the idea that things can be people. The motivated muddling of categories so prevalent in writing and thinking about A.I., beginning with the very name “artificial intelligence,” is intentional and serves the narrative that this software can and will take over for human workers, doing their jobs cheaper, faster, better and without requiring rest or dignity. The A.I. industry is selling a dream of digital slavery—infinite human labor with no actual human involved. And if you worry that this conclusion is a little histrionic, here’s Sam Altman encouraging us to compare the energy cost of training an L.L.M. to the energy cost of raising and educating a human child:
“One of the things that is always unfair, in this comparison, is people talk about how much energy it takes to train an A.I. model… but it also takes a lot of energy to train a human. It takes like twenty years of life, and all of the food you eat during that time, before you get smart.”
Is this treating things as people, or treating people as things? “The creatures outside looked from pig to man, and from man to pig, and from pig to man again; but already it was impossible to say which was which.” Either way I think it’s evil, and the evil is not the technology—it’s the dreams of the people trying to sell it to us.
Today in More A.I. Tabs: Casey Newton reported that “OpenAI disbanded its mission alignment team.” John Herrman on why A.I. alignment and safety workers “might worry that, actually, none of this is being taken seriously and that you now work at just another big tech company—but worse.” Sam Kriss in Harper’s with a well edited story about the San Francisco A.I. startup scene. The n+1 editors: “Stigmatization is a powerful force, and disgust and shame are among our greatest tools. Put plainly, you should feel bad for using AI.“ Welcome to the Group Chat.
Today in Not A.I.: “The place on Earth from which you can, in theory, see further than any other is between an unnamed Himalayan ridge near the Indian-Chinese border and Pik Dankova in Kyrgyzstan. It is just over 530km.” The Longest Line of Sight.
I know you were wondering so here’s the difference between “lowkenuinely” and “lowkirkenuinely.”
Today’s Song: Laufey, “From the Start”
I am so glad this post is done, it started as a Google doc annotation of that New Yorker story and just kept getting more unmanageable for a week and a half. I have probably two more posts worth of links and ideas that didn’t make it in here, but I think we can let all that slide for now. Meanwhile I will slide back into your inbox later this week. Sweet segue, nice job Rusty.
I wrote “nice job, Rusty” in an email and sent it to myself. What I saw next has shocking implications.
Reply