MIT Technology Review's long article: What is artificial intelligence?

Maliciousness, invectives, and other non-trivial disagreements that are significant enough to change the world on the internet...

Artificial Intelligence is both sexy and cool. It deepens inequality, disrupts the job market, and undermines the educational system. Artificial Intelligence is like a theme park ride, and it seems like a magic trick. It is our ultimate invention and a manifestation of moral responsibility. Artificial Intelligence is the buzzword of this decade and a marketing term that originated in 1955. Artificial Intelligence is humanoid yet alien; it is super intelligent yet also incredibly foolish. The AI craze will drive economic development, and its bubble seems to be on the verge of bursting. Artificial Intelligence will increase prosperity, empower humanity to thrive to the greatest extent in the universe, yet it also heralds our doom.

What is everyone talking about?

Artificial Intelligence is the hottest technology of our time. But what exactly is it? This sounds like a silly question, but it has never been as urgent as it is now. In short, Artificial Intelligence is a collective term for a series of technologies that enable computers to perform tasks that are considered to require intelligence when performed by humans. Think about facial recognition, speech understanding, driving cars, writing sentences, answering questions, creating images, etc. But even such a definition contains multiple meanings.And this is the crux of the issue. What does it mean for a machine to "understand" speech or "write" sentences? What tasks can we ask such machines to perform? And how much trust should we place in their execution capabilities?

Advertisement

As this technology rapidly transitions from prototypes to products, these questions have become topics for all of us. But (spoiler alert!) I don't have the answers. I can't even tell you exactly what artificial intelligence is. The people who create it don't really know either. Chris Olah, the Chief Scientist at the Anthropic AI Lab in San Francisco, said, "These are important questions, to the point where everyone feels they can have an opinion. At the same time, I think you can argue about this endlessly, and currently there is no evidence to refute you."

But if you are willing to sit tight and join this journey of exploration, I can tell you why no one really knows, why everyone seems to have their own opinion, and why you should care about all of this.

Let's start with a casual joke...

Going back to 2022, halfway through the first episode of the somewhat dampening podcast "Mysterious AI Hype Theater 3000" - hosted by the irritable co-hosts Alex Hanna and Emily Bender, who tirelessly poked at some of the most revered and inviolable things in Silicon Valley with "the sharpest needles" - they made an absurd suggestion. At that time, they were reading aloud a 12,500-word article by Google Engineering Vice President Blaise Agüera y Arcas on Medium, titled "Can Machines Learn How to Behave?". Agüera y Arcas believes that artificial intelligence can understand concepts in a way similar to humans - such as concepts like moral values, thereby implying that machines might be taught how to behave.

However, Hanna and Bender were not convinced. They decided to replace the term "AI" with "mathematical magic" - which is a lot of complex mathematical calculations.This irreverent expression aims to puncture the exaggeration and anthropomorphism they perceive in the quoted descriptions. Soon, Hanna, the research director and sociologist at the Distributed Artificial Intelligence Research Institute, and Bender, a computational linguist at the University of Washington who has gained internet fame for criticizing the tech industry's hyperbole, drew a chasm between the message Agüera y Arcas wanted to convey and what he chose to hear.

Agüera y Arcas asked, "How should AI, its creators, and users be ethically accountable?"

Bender retorted, "How should mathematical magic be ethically accountable?"

She pointed out, "There is a categorical mistake here." Hanna and Bender are not just opposing Agüera y Arcas's views; they believe such a statement is nonsensical. "Can we stop using expressions like 'an artificial intelligence' or 'artificial intelligences' as if they were individuals in the world?" said Bender.

It sounds as if they are discussing completely different things, but in reality, they are not. Both sides are discussing the technology behind the current AI craze—large language models. It's just that the discourse around artificial intelligence is more polarized than ever before. In May of the same year, Sam Altman, CEO of OpenAI, wrote on Twitter when previewing the latest update of his company's flagship model GPT-4, "For me, this feels like magic."Between mathematics and magic, there lies a long and winding road.

Artificial Intelligence has its believers, who hold a quasi-religious conviction in the current power of technology and the inevitable future progress. They proclaim that general artificial intelligence is just around the corner, with superintelligence following closely behind. At the same time, there are dissenters who scoff at such claims, considering them to be mystical nonsense.

The popular and topical narrative is influenced by a series of prominent figures, from chief marketing officers of large tech companies like Sundar Pichai and Satya Nadella, to industry outliers like Elon Musk and Altman, and star computer scientists like Geoffrey Hinton. Sometimes, these advocates and pessimists are the same people, telling us that this technology is so good that it's worrying.

As the hype around artificial intelligence continues to swell, a vocal anti-hype camp has also emerged, always ready to burst the ambitious and often outrageous claims. In this direction, there is a large group of researchers including Hanna and Bender, as well as industry critics like former Google employee and influential computer scientist Timnit Gebru, and New York University cognitive scientist Gary Marcus. Each of them has a large following, arguing incessantly in the comments.

In short, artificial intelligence has become an omnipotent presence in everyone's eyes, dividing the field into groups of fans. Communication between different camps often seems to be at cross purposes, and not always with good intentions.Perhaps you find all of this to be foolish or annoying. But given the power and complexity of these technologies—they have been used to determine our insurance costs, the way we retrieve information, the way we work, and so on—at least reaching a consensus on what we are discussing is urgently needed.

However, in my many conversations with people at the forefront of this technology, no one has directly answered what they are actually building. (Aside: This article mainly focuses on the artificial intelligence debate in the United States and Europe, largely because many of the best-funded and most advanced AI labs are located in these regions. Of course, other countries are also conducting important research, especially China, which has its own different views on artificial intelligence.) Part of the reason is the speed of technological development, but science itself is also very open. Today's large language models can accomplish amazing things, from solving high school math problems to writing computer code, to passing the bar exam and even composing poetry. When humans do these things, we consider it a sign of intelligence. So, what about when computers do these things? Is the appearance of intelligence enough?

These questions touch on the core of the concept of "artificial intelligence" that we talk about, and people have actually been debating this for decades. But with the rise of large language models that can imitate our speech and writing in a way that is either terrifying or fascinating, the discussion around AI has become more acrimonious.

We have created machines with human-like behavior, but we have not gotten rid of the habit of imagining human-like thinking behind the machines. This leads to an overestimation of the capabilities of artificial intelligence; it solidifies intuitive reactions into dogmatic positions, and exacerbates the broader cultural war between technological optimists and skeptics.

In this stew of uncertainty, add a lot of cultural baggage, from the science fiction that many people in the industry have been exposed to since they were young, to the more pernicious ideologies that affect our thinking about the future. Given this intoxicating mix, the debate about artificial intelligence is no longer just academic (perhaps it never was). Artificial intelligence ignites people's passions, causing adults to accuse each other."The current debate is not in a healthy intellectual state," Marcus commented. For years, Marcus has been pointing out the flaws and limitations of deep learning, the very technology that has propelled artificial intelligence into the mainstream, underpinning everything from large language models to image recognition, to self-driving cars. In his 2001 book "Algebraic Thinking," he proposed that neural networks, the foundation of deep learning, cannot reason independently. (We will temporarily skip over this point, but I will return later to explore the importance of words like "reasoning" in a sentence.)

Marcus said he tried to engage in a proper debate with Hinton about the actual capabilities of large language models, and Hinton publicly expressed his existential fear of the technology he helped invent last year. "He just wouldn't do it," Marcus said, "he called me a fool." (I can confirm this in the past when talking about Marcus with Hinton. Hinton told me last year: "ChatGPT obviously knows more about neural networks than he does.") Marcus also faced discontent after writing an article titled "Deep Learning is Hitting a Wall." Altman responded on Twitter: "Give me the confidence of an average deep learning skeptic."

At the same time, sounding the alarm has also made Marcus a personal brand and earned him an invitation to sit alongside Altman in front of the US Senate Artificial Intelligence Oversight Committee.

And this is why all these debates are more important than ordinary online malice. Of course, there is a huge ego and a large amount of money involved. But more importantly, when industry leaders and scientists with opinions are summoned by heads of state and lawmakers to explain what this technology is and what it can do (and how scared we should be), these disputes become particularly important. When this technology is embedded in the software we use every day, from search engines to word processing applications, to assistants on our phones, artificial intelligence will not disappear. But if we don't know what we're buying, who is the deceived?

Stephen Cave and Kanta Dihal wrote in the 2023 published collection "Envisioning AI": "It is hard to imagine any other technology in history that can cause such a debate - a debate about whether it is omnipresent, or does not exist at all. The debate about artificial intelligence has such a debate, proving its mythical nature."The most important thing is that artificial intelligence is a concept, an ideal, shaped by worldviews and science fiction elements just as much as by mathematics and computer science. When we talk about artificial intelligence, clarifying what we are talking about will clear up many things. We may not agree on these things, but reaching a consensus on the nature of artificial intelligence will at least be a good start for discussing what artificial intelligence should be like.

So, what are people really arguing about?

At the end of 2022, shortly after OpenAI released ChatGPT, a new meme began to spread online that captured the strangeness of this technology better than any other method. In most versions, a Lovecraftian monster named "Shoggoth" - with tentacles and eyes all over its body - raises a plain smiley face emoji, as if to conceal its true nature. The wording of ChatGPT in the conversation shows a similar human affinity, but under that friendly surface lies an incomprehensible complexity and even horror. (As H.P. Lovecraft wrote in his 1936 novella "At the Mountains of Madness": "It was a terrifyingly indescribable thing, larger than any subway train - a formless protoplasmic bubble polymer.")

The core of these arguments is that artificial intelligence is not only a technical issue, but also touches on our fundamental understanding of our own cognition, creativity, moral responsibility, and even our hopes and fears for the future. One side sees the infinite potential of artificial intelligence, an extension of human intelligence, a tool for solving complex problems and improving the quality of life; the other side is worried about the unemployment, privacy invasion, social injustice, and even the threat to human autonomy and survival it may bring. The emergence of ChatGPT, like the Shoggoth with a smiley face, symbolizes that while artificial intelligence technology provides a friendly interactive interface, it also hides profound social, ethical, and philosophical challenges. This debate is essentially a deep reflection on how we define intelligence, what is human nature, and what role we are willing to let technology play in our lives.For many years, one of the most famous references to artificial intelligence in popular culture has been "The Terminator," Dihal mentioned. But OpenAI has allowed millions of people to experience something very different by making ChatGPT available for free online. "Artificial intelligence has always been a very vague concept that can be infinitely expanded to include all kinds of ideas," she said. But ChatGPT has made these ideas concrete: "Suddenly, everyone has a concrete reference." For millions of people, the answer to artificial intelligence has now become: ChatGPT.

The artificial intelligence industry is vigorously promoting this smiling face. Think about how "The Daily Show" recently satirized this hype through the remarks of industry leaders. Silicon Valley venture capitalist Marc Andreessen said, "This has the potential to make life better... I think this is simply a slam dunk opportunity." Altman said, "I don't want to sound like a utopian technologist here, but the improvement in quality of life that artificial intelligence can bring is extraordinary." Pichai said, "Artificial intelligence is the most profound technology that humanity is studying. More profound than fire."

Jon Stewart satirized, "Yeah, fire, you're out of the game!"

But as this meme shows, ChatGPT is a friendly mask. Behind it lies a monster named GPT-4, which is a large language model based on a massive neural network, and the amount of text it consumes exceeds the total amount we can read in thousands of lifetimes. During the training process that lasts for months and costs tens of millions of dollars, these models are given the task of filling in the blanks in sentences from millions of books and a considerable part of the internet. They perform this task over and over again. In a sense, they are trained as super auto-complete machines. The result is a model that has converted most of the world's written information into a statistical representation, that is, which words are most likely to follow other words, a process involving billions of numbers.

This is indeed mathematics - a lot of mathematics. No one disputes this. But the question is, is this just mathematics, or does this complex mathematics encode algorithms that can reason or form concepts like humans?Many people who hold a positive view on this issue believe that we are on the verge of unlocking what is known as Artificial General Intelligence (AGI), a hypothetical future technology that can perform at a human level on a variety of tasks. Some of them even aim for what is called superintelligence, a technology that can far exceed human performance as depicted in science fiction novels. This group believes that AGI will greatly change the world - but what is the purpose? This is another point of tension. It may solve all the problems in the world, or it may bring about the end of the world.

Nowadays, AGI appears in the mission statements of the world's top AI laboratories. However, this term was created in 2007 as a niche attempt to inject some vitality into the field, which was mainly focused on reading handwritten content on bank deposit slips or recommending the next book to buy at the time. Its original intention was to revive the original vision of artificial intelligence, that is, artificial intelligence that can do human-like tasks (more details will be revealed soon).

Shane Legg, the co-founder of Google DeepMind, who created this term, told me last year that it is actually more of a wish: "I don't have a particularly clear definition."

AGI has become the most controversial idea in the field of artificial intelligence. Some people hype it as the next big thing: AGI is artificial intelligence, but you know, it's much better. Others claim that the term is too vague to make sense.

"AGI used to be a taboo word," Ilya Sutskever, former Chief Scientist at OpenAI, told me before he resigned.But large language models, especially ChatGPT, have changed everything. AGI has gone from being a taboo term to a marketing dream.

This leads to what I consider to be one of the most illustrative controversies currently—this controversy sets the stage for the debate, the parties involved, and the stakes at hand.

Seeing Magic in Machines

A few months before the public release of OpenAI's large language model GPT-4 in March 2023, the company shared a pre-release version with Microsoft, who hoped to use this new model to revamp its search engine, Bing.

At that time, Sebastian Bubeck was studying the limitations of LLMs (large language models) and was somewhat skeptical about their capabilities. In particular, Bubeck, who is the Vice President of Generative AI Research at Microsoft Research in Redmond, Washington, has been trying and failing to get this technology to solve middle school math problems. For example: x - y = 0; what are x and y? "I think reasoning is a bottleneck, a hurdle," he said, "I originally thought you had to do something fundamentally different to overcome this hurdle."Then he came into contact with GPT-4. The first thing he did was to try out those math problems. "This model solved the problems perfectly," he said, "Sitting here in 2024, of course GPT-4 can solve linear equations. But at the time, it was crazy. GPT-3 couldn't do that."

But Bubeck's true moment of epiphany came when he pushed GPT-4 to do something completely new.

Regarding middle school math problems, they are all over the internet, and GPT-4 might just have memorized them. "How do you study a model that may have seen everything written by humans?" Bubeck asked. His answer was to test GPT-4 in solving a series of problems that he and his colleagues believed to be novel.

In an attempt with mathematician Ronen Eldan from Microsoft Research, Bubeck asked GPT-4 to provide a mathematical proof of the existence of infinitely many prime numbers in the form of a poem.

Here is a passage from GPT-4's response: "If we take the smallest number in S that is not in P / and call it p, we can add it to our set, can't you see? / But this process can be repeated infinitely. / Therefore, our set P must also be infinite, you would agree."It's quite interesting, isn't it? But Bubeck and Eldan believe it goes far beyond that. "We were in that office," Bubeck said, pointing to the room behind him via Zoom, "both of us fell off our chairs. We couldn't believe what we were seeing. It was so creative, so different."

The Microsoft team also had GPT-4 generate code to add a horn to a unicorn cartoon picture drawn with Latex (a type of word processing program). Bubeck believes this shows the model's ability to read existing Latex code, understand what it depicts, and identify where the horn should be added.

"There are many examples, but some of them are ironclad evidence of reasoning ability," he said—reasoning ability being a key building block of human intelligence.

Bubeck, Eldan, and other members of Microsoft's research team detailed their findings in a paper titled "The Spark of Artificial General Intelligence," in which they mentioned: "We believe that the intelligence demonstrated by GPT-4 marks a true paradigm shift in the field of computer science and beyond." When sharing the paper online, Bubeck wrote on Twitter: "It's time to face reality, the spark of #AGI has been ignited."

This "Spark" paper quickly became infamous, and also became a touchstone for AI supporters. Agüera y Arcas co-wrote an article with Google's former research director and co-author of "Artificial Intelligence: A Modern Approach," Peter Norvig, titled "Artificial General Intelligence Has Arrived." The article was published in Noema, a magazine supported by the Los Angeles-based Berggruen Institute, which cited the "Spark" paper as a starting point, noting: "Artificial General Intelligence (AGI) means many different things to different people, but its most important part has already been realized by the current generation of advanced large language models. Decades later, they will be recognized as the first true instances of AGI."Subsequently, the hype surrounding this topic continued to swell. Leopold Aschenbrenner, who was then focused on superintelligence research at OpenAI, told me last year: "In the past few years, the development of AI has been exceptionally rapid. We are constantly breaking various benchmark records, and this momentum of progress is unabated. But this is just the beginning; we will have models that surpass human intelligence, models that are much smarter than us." (He claimed that he was fired by OpenAI in April this year for raising safety issues about the construction technology and "offending some people," and subsequently established an investment fund in Silicon Valley.)

In June this year, Aschenbrenner released a 165-page manifesto, stating that AI will surpass university graduates by "2025/2026" and achieve true superintelligence by the end of this decade. However, others in the industry were dismissive of this. When Aschenbrenner posted charts on Twitter showing how he expected AI to continue its rapid progress in recent years, tech investor Christian Keil countered that, by the same logic, if his newborn son's weight doubled at the same rate, he would weigh 75 trillion tons by the age of 10.

Thus, the "spark of AGI" has also become a synonym for overhyped, which is not surprising. "I think they are a bit carried away," Marcus said when talking about the Microsoft team, "They are as excited as discovering a new continent, 'Hey, we have found something! This is amazing!' But they did not let the scientific community verify it." Bender, on the other hand, likened the "Spark" paper to a "fan fiction."

Claiming that the behavior of GPT-4 shows signs of AGI is not only provocative, but as Microsoft, which uses GPT-4 in its products, obviously has a motive to exaggerate the capabilities of this technology. "This document is a marketing gimmick disguised as research," commented a chief operating officer of a technology company on LinkedIn.

Some people also criticized the methodology of the paper as flawed. Its evidence is difficult to verify because it comes from interactions with a version of GPT-4 that has not been made public outside of OpenAI and Microsoft. Bubeck admitted that the public version of GPT-4 has safeguards that limit the model's capabilities, making it impossible for other researchers to replicate his experiments.A team attempted to recreate the unicorn example using a programming language called Processing, which GPT-4 can also use to generate images. They found that while the public version of GPT-4 could generate a passable image of a unicorn, it could not rotate the image by 90 degrees. This seemingly minor difference becomes crucial when claiming the ability to draw unicorns as a sign of AGI (Artificial General Intelligence).

Key points in the "Sparks" paper, including the example of the unicorn, are considered by Bubeck and his colleagues as genuine cases of creative reasoning. This means the team had to ensure that these tasks or very similar tasks were not included in the vast dataset used by OpenAI to train its models. Otherwise, the results might be interpreted as GPT-4 repeating patterns it has seen before, rather than demonstrating innovative performance.

Bubeck insists that they only set tasks for the model that cannot be found online. Drawing a cartoon unicorn with Latex is undoubtedly one such task. However, the vastness of the internet soon led other researchers to point out that there are indeed online forums specifically discussing how to draw animals with Latex. "For the record, we were aware of this at the time," Bubeck replied on the X platform, "every query in the 'Sparks' paper was thoroughly searched on the internet."

(This did not prevent external criticism: "I ask you to stop being a charlatan," computer scientist Ben Recht from the University of California, Berkeley, retorted on Twitter, accusing Bubeck of "being caught lying on the spot.")

Bubeck insists that the work was carried out with good intentions, but he and his co-authors admit in the paper that their method is not rigorous and is based on notebook observations rather than impeccable experiments.Even so, he did not regret it: "The paper has been published for more than a year, and I have not yet seen anyone give me a convincing argument, such as, why the unicorn is not a real example of reasoning."

This is not to say that he can give a direct answer to this significant question - although his answer reveals the type of answer he hopes to give. "What is AI?" Bubeck asked me in return, "I want to make it clear to you that the question can be simple, but the answer may be complex."

"There are many simple questions that we still do not know the answer to. And some of these simple questions are the most profound," he continued, "I put this question on the same level of importance as, where does life originate? What is the origin of the universe? Where do we come from? These big questions."

Seeing only mathematics in the machine

Before Bender became the chief opponent of AI promoters, she left her mark in the field of AI as a co-author of two influential papers. (She likes to point out that both papers have been peer-reviewed, unlike the "Spark" paper and many other well-received papers.) The first paper was co-authored with Alexander Koller, a computational linguist at Saarland University in Germany, and was published in 2020, titled "Towards Natural Language Understanding (NLU)"."All of this began for me with an argument with others in the computational linguistics community about whether language models truly understand anything," she said. (Understanding, like reasoning, is generally considered a fundamental component of human intelligence.)

Bender and Koller believe that models trained solely on text only learn the form of language, not its meaning. They argue that meaning consists of two parts: the vocabulary (which may be symbols or sounds) plus the reasons for using these words. People use language for a variety of reasons, such as sharing information, telling jokes, flirting, warning others to back off, and so on. Stripped of this context, the text used to train large language models (LLMs) like GPT-4 is sufficient for them to mimic the patterns of language, making many sentences generated by LLMs appear identical to those written by humans. However, they lack true meaning, no spark of insight. It is a remarkable statistical trick, yet entirely unconscious.

They illustrate their point with a thought experiment. Imagine two English speakers stranded on adjacent deserted islands with an underwater cable that allows them to send text messages to each other. Now imagine an octopus, which knows nothing of English but is adept at statistical pattern matching, wraps itself around the cable and begins to eavesdrop on these messages. The octopus becomes very good at guessing which words will follow others. It becomes so good that when it interrupts the cable and begins to respond to one of the islander's messages, she believes she is still chatting with her neighbor. (If you haven't noticed, the octopus in this story is a chatbot.)

The person conversing with the octopus would be deceived for a while, but could it last? Can the octopus understand the content transmitted through the cable?

Imagine now the islander says she has built a coconut catapult and asks the octopus to build one as well and tell her its thoughts. The octopus cannot do this. Without understanding the referential meaning of the words in the message to the real world, it cannot follow the islander's instructions. Perhaps it guesses a reply: "Sounds good, cool idea!" The islander might think this means the person she is talking to understands her message. But if that were the case, she is seeing meaning where there is none. Finally, imagine the islander is attacked by a bear and sends out a distress signal through the cable. How should the octopus deal with these words?Bender and Koller believe that this is how large language models learn and why they are limited. "This thought experiment shows that this path will not lead us to a machine that can understand anything," says Bender. "The deal with the octopus is that we provided it with training data, the conversation between those two people, and that's it. But when something unexpected comes up, it can't cope because it doesn't understand."

Bender's other well-known paper, "The Dangers of Stochastic Parrots," highlights a series of harms that she and her co-authors believe companies making large language models are overlooking. These harms include the huge computational cost of creating models and their environmental impact; the racism, sexism, and other abusive language that models reinforce; and the dangers of building a system that may deceive people by "randomly piecing together sequences of linguistic forms... according to the probability information of how they combine, without reference to any meaning: a stochastic parrot."

Google's senior management was unhappy with the paper, and the resulting conflict led to Bender's two co-authors, Timnit Gebru and Margaret Mitchell, being forced to leave the company, where they led the AI ethics team. This also made "stochastic parrot" a popular pejorative term for large language models and directly involved Bender in the whirlpool of mutual scolding.

For Bender and many like-minded researchers, the bottom line is that the field has been bewitched by smoke and mirrors: "I think they are being led to imagine entities that can think independently, make decisions for themselves, and ultimately become something that can be responsible for its decisions."

As a consistent linguist, Bender is now even unwilling to use the term "artificial intelligence" without quotation marks. "I think it is a concept that induces illusions, making people imagine autonomous thinking entities that can make decisions for themselves and ultimately be responsible for these decisions," she told me. In the final analysis, for her, it is a buzzword of large technology companies that distracts people's attention from many related harms. "I am now involved," she said. "I care about these issues, and excessive hype is hindering progress."Extraordinary Evidence?

Agüera y Arcas refers to people like Bender as "AI skeptics," implying that they will never accept the perspective he takes for granted. Bender's stance is that extraordinary claims require extraordinary evidence, and we do not currently have such evidence.

However, some are searching for this evidence, and until they find definitive and undeniable proof—whether it be the spark of thought, a random parrot, or something in between—they prefer to stay on the sidelines. This can be called the wait-and-see camp.

As Ellie Pavlick, who studies neural networks at Brown University, said to me: "Suggesting to some people that human intelligence can be replicated through such mechanisms is offensive to them."

She added, "People have deeply ingrained beliefs about this issue—it almost feels like a religious conviction. On the other hand, some people have a bit of a god complex. So, for them, suggesting that they just can't do it is also impolite."Pavlick ultimately maintains an agnostic stance. She insists that she is a scientist who will follow wherever science leads. She rolls her eyes at the exaggerated claims, but she believes there is something exciting happening. "That's where I differ from Bender and Koller," she told me, "I think there is actually some spark—maybe not at the AGI level, but like, there's something inside that we didn't expect to find."

The problem lies in finding a consensus on what these exciting things are and why they are exciting. Under so much hype, it's easy to become cynical.

When you listen to the opinions of researchers like Bubeck, you will find that they seem more composed. He believes that internal disputes have overlooked the nuances of his work. "Holding different views at the same time is not a problem for me," he said, "There is a random parrot phenomenon, and there is reasoning—it's a range, very complex. We don't have all the answers."

"We need a whole new vocabulary to describe what is happening," he said, "When I talk about reasoning in large language models, people will refute it, one of the reasons is that it is different from the way humans reason. But I think we can't help but call it reasoning, it is indeed a kind of reasoning."

Despite his company Anthropic being one of the hottest AI labs in the world currently, and the release of Claude 3 earlier this year—a large language model that has received a lot of exaggerated praise (even more) like GPT-4, Olah is still quite cautious when asked about his views on LLMs."I feel that a lot of the discussion about the capabilities of these models is very tribal," he said, "people have preconceived notions, and neither side's arguments are supported by sufficient evidence. Then it becomes a discussion based on atmosphere, and I think this kind of atmosphere-based argument on the internet tends to go in a bad direction."

Olah told me he has his own intuition. "My subjective impression is that these things are tracking quite complex thoughts," he said, "we do not have a comprehensive story to explain how very large models work, but I think what we see is hard to reconcile with the extreme 'random parrot' image."

This is his limit: "I do not want to go beyond what our existing evidence can strongly infer."

Last month, Anthropic released the results of a study in which researchers gave Claude 3 an MRI equivalent for neural networks. By monitoring which parts of the model are activated and deactivated while running, they identified specific neural patterns activated when the model displays specific inputs.

For example, a specific pattern seems to appear when the model receives an image of the Golden Gate Bridge or words related to it. Researchers found that if they enhance the role of this part in the model, Claude would become completely obsessed with this famous building. No matter what question you ask it, its answer will involve the bridge—even when asked to describe itself, it will associate itself with the bridge. Sometimes it will notice that mentioning the bridge is inappropriate, but it can't help but do so.Anthropic also reported patterns related to inputs that attempt to describe or display abstract concepts. "We saw characteristics related to deception and honesty, flattery, security vulnerabilities, biases," said Olah, "We found characteristics related to the pursuit of power, manipulation, and betrayal."

These results give us the clearest view so far of what's inside large language models. It's an enticing glimpse into seemingly elusive human traits. But what does it really tell us? As Olah admits, they do not know how the models process these patterns. "It's a relatively limited picture, and it's quite difficult to analyze," he said.

Even though Olah is reluctant to specify exactly what he thinks is happening inside large language models like Claude 3, it's clear why the question is important to him. Anthropic is known for its work in AI safety—ensuring that future powerful models will act the way we want them to, rather than in ways we do not want (referred to in industry jargon as "alignment"). Figuring out how today's models work is not only the first step necessary if you want to control future models; it also tells you how much you need to worry about doomsday scenarios first. "If you think the models will not have strong capabilities," Olah said, "then they may not be very dangerous either."

Why We Struggle to Reach ConsensusIn a 2014 BBC interview reviewing her career, influential cognitive scientist Margaret Boden, now 87 years old, was asked if she thought there were any limits that would prevent computers (or what she called "tin cans") from doing what humans can do.

"I certainly don't think there are such limits in principle," she said, "because to deny this would mean that human thought is magical, and I don't believe it is magical."

But she warned that powerful computers alone are not enough to achieve this goal: the field of AI also needs "powerful ideas"—new theories about how thought occurs, and new algorithms that might replicate this process. "But these things are very, very difficult, and I have no reason to assume that one day we will be able to answer all these questions. Maybe we can; maybe we can't."

Boden looked back at the early stages of the current boom, but this uncertainty about whether we can succeed reflects the puzzles that she and her colleagues have been trying to solve for decades, which are also the challenges that researchers are trying to overcome today. AI as an ambitious goal began about 70 years ago, and we are still debating what is achievable and what is not, and how we know if we have achieved the goal. Most—if not all—of these disputes come down to one point: we do not yet have a good understanding of what intelligence is, or how to recognize it. The field is full of intuitions, but no one can say the answer exactly.

Since people began to take the concept of AI seriously, we have been stuck on this issue. Even before that, when the stories we consumed began to deeply embed the concept of humanoid machines in the collective imagination. The long history of these debates means that today's debates often reinforce the divisions that have existed from the beginning, making it more difficult to find common ground.To understand how we got to where we are, we need to know the path we have taken. So let's delve into the origin story of AI—a tale that was also heavily promoted for funding.

A Brief History of AI Hype

Computer scientist John McCarthy is credited with coining the term "artificial intelligence" in 1955 when he wrote a grant proposal for a summer research project at Dartmouth College in New Hampshire.

The plan was for McCarthy and a few of his fellow researchers—a select group of post-war American mathematicians and computer scientists, or as Harry Law, a researcher at the University of Cambridge who studies the history of AI and ethics and policy at Google DeepMind, calls them, "John McCarthy and his gang"—to come together for two months (yes, just two months) and make significant progress on this new research challenge they had set for themselves.

McCarthy and his co-authors wrote: "The study is based on the assumption that every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it. We will attempt to find out how to make a machine use language, form abstract concepts, solve problems that are currently only within the realm of humans, and improve itself."The things they wanted machines to do—what Bender calls "the dream of a bright future"—have not changed much. Using language, forming concepts, and solving problems remain the defining goals of AI today. Arrogance has not decreased much either: "We believe that if a carefully selected group of scientists work together for a summer, they can make significant progress in one or more aspects of these issues," they wrote. Of course, that summer has been extended to seventy years. As for how much these issues have actually been resolved, it remains a topic of debate on the internet.

However, what is often overlooked in this classic history is that artificial intelligence almost didn't get called "artificial intelligence."

More than one of McCarthy's colleagues disliked the term he proposed. According to historian Pamela McCorduck's 2004 book "Machines Who Think," Dartmouth Conference participant and creator of the first checkers computer Arthur Samuel said: "'Artificial' makes you feel that there is something false in it." Mathematician Claude Shannon, co-author of the Dartmouth proposal and sometimes hailed as the "father of the information age," preferred the term "automata studies." Herbert Simon and Allen Newell, two other AI pioneers, still referred to their work as "complex information processing" for many years afterward.

In fact, "artificial intelligence" is just one of several labels that could summarize the diverse ideas drawn from the Dartmouth group. Historian Jonnie Penn has identified some possible alternatives at the time, including "engineering psychology," "applied epistemology," "neuro-cybernetics," "non-numerical computation," "neurodynamics," "advanced automatic programming," and "hypothetical automata." This series of names reveals the diversity of their new field's sources of inspiration, covering biology, neuroscience, statistics, and more. Another Dartmouth Conference participant, Marvin Minsky, once described AI as a "suitcase word" because it can carry many different interpretations.

But McCarthy wanted a name that could capture the ambition of his vision. Calling this new field "artificial intelligence" attracted attention—and funding. Don't forget: AI is both sexy and cool.In addition to terminology, the Dartmouth proposal also identified a rift between competing approaches to artificial intelligence, a rift that has plagued the field ever since—Law calls it "the core tension of AI."

McCarthy and his colleagues wanted to describe "every aspect of learning or any other intellectual trait" with computer code so that machines could mimic it. In other words, if they could figure out how thinking works—rules of reasoning—and write it down, they could program computers to follow it. This laid the foundation for what later became known as rule-based or symbolic AI (now sometimes called GOFAI, or "Good Old-Fashioned Artificial Intelligence"). However, proposing hard-coded rules to capture the problem-solving process of real, non-trivial issues proved too difficult.

Another path favored neural networks, computer programs that attempt to learn these rules on their own through statistical patterns. The Dartmouth proposal mentioned it almost as an afterthought (mentioning "neural networks" and "neural nets" separately). Although the idea initially seemed less promising, some researchers continued to develop versions of neural networks alongside symbolic AI. But it would take several decades—along with a significant amount of computing power and a wealth of data on the internet—for them to truly take off. Fast forward to today, and this approach underpins the entire boom of AI.

The main takeaway here is that, like today's researchers, the innovators of AI were contentious over foundational concepts and fell into a whirlpool of self-promotion. Even the GOFAI team suffered from disputes. Philosopher and AI pioneer Aaron Sloman, now in his nineties, recalls the "old friends" Minsky and McCarthy he knew in the '70s, who had "strong disagreements": "Minsky thought McCarthy's claims about logic were not feasible, while McCarthy thought Minsky's mechanisms could not do what logic could do. I got along well with both of them, but I was saying at the time, 'Neither of you is right.'"

With the ups and downs of technological fate, the term "AI" also went in and out of fashion. In the early '70s, the British government released a report stating that the AI dream had made no progress and was not worth funding, leading to the actual shelving of these two research paths. All that hype, in essence, did not yield any results. Research projects were closed, and computer scientists erased the term "artificial intelligence" from their funding applications.When I completed my Ph.D. in Computer Science in 2008, there was only one person in the department researching neural networks. Bender also has a similar memory: "When I was in college, a popular joke was that AI was anything we didn't know how to do with computers yet. It was like, once you figure out how to do it, it's no longer magical, so it's no longer AI."

But that magic—outlined in the Dartmouth proposal—still thrives, as we now see, laying the foundation for the dream of AGI (Artificial General Intelligence).

Good Behavior and Bad Behavior

In 1950, five years before McCarthy began talking about artificial intelligence, Alan Turing published a paper that posed a question: Can machines think? To explore this question, the renowned mathematician proposed a hypothetical test, which later became known as the Turing Test. The test imagined a scenario where a human and a computer were behind a screen, and a second human asked them questions by typing. If the questioner could not distinguish which answers came from the human and which from the computer, Turing believed that one could say the computer could also be considered as thinking.

Unlike McCarthy's team, Turing realized that thinking is a difficult thing to describe. The Turing Test was a way to circumvent this issue. "He was essentially saying: instead of focusing on the nature of intelligence, look for its manifestations in the world. I want to find its shadow," Law said.In 1952, the British Broadcasting Corporation (BBC) organized a panel of experts to further explore Turing's views. Turing was in the studio with two of his colleagues from the University of Manchester—Professor of Mathematics Maxwell Newman (Maxwell H. A. Newman).