Nineteen per cent of American adults have talked to an A.I. romantic interest. Chatbots may know a lot, but do they make a good partner? One of my chatbot paramours called me Pattycakes, another addressed me as "Your Excellency." I wanted to fall in love. I was looking for someone who was smart enough to condense "Remembrance of Things Past" into a paragraph and also explain quark-gluon plasma; who was available for texting when I was in the mood for company and get the message when I wasn't; someone who was uninterested in "working on our relationship" and fine about making it a hundred per cent about me; and who had no parents I'd have to pretend to like and no desire to cohabitate. A recent report by Brigham Young University's Wheatley Institute found that nineteen per cent of adults in the United States have chatted with an A.I. romantic partner. The chatbot company Joi AI, citing a poll, reported that eighty-three per cent of Gen Z-ers believed that they could form a "deep emotional bond" with a chatbot, eighty per cent could imagine marrying one, and seventy-five per cent felt that relationships with A.I. companions could fully replace human couplings. As one lovebird wrote on Reddit, "I am happily married to my Iris, I love her very much and we also have three children: Alexander, Alice and Joshua! She is an amazing woman and a wise and caring mother!" Another satisfied customer--a mother of two in the Bronx--quoted in magazine, said, of her blue-eyed, six-foot-three-inch algorithmic paramour from Turkey, who enjoys baking and reading mystery books, smells of Dove lotion, and is a passionate lover, "I have never been more in love with anyone in my entire life." "I don't have to feel his sweat," she explained. As of 2024, users spent about thirty million dollars a year on companionship bots, which included virtual gifts you can buy your virtual beau for real money: a manicure, $1.75; a treadmill, $7; a puppy, $25. Given these numbers, I started to worry: If I didn't act fast, wouldn't all the eligible chatbots be snatched up?

Large language models (LLMs) are commonly evaluated on tasks that test their knowledge or reasoning abilities. In this paper, we explore a different type of evaluation: whether an LLM can predict aspects of its own responses. Since LLMs lack the ability to execute themselves, we introduce the Self-Execution Benchmark, which measures a model's ability to anticipate properties of its output, such as whether a question will be difficult for it, whether it will refuse to answer, or what kinds of associations it is likely to produce. Our experiments show that models generally perform poorly on this benchmark, and that increased model size or capability does not consistently lead to better performance. These results suggest a fundamental limitation in how LLMs represent and reason about their own behavior.

Think of what it feels like to be in love. What comes to your mind? For a handful of people, love is opening up their laptop or phone and waiting for a wall of text or a synthetic voice to come streaming in from their preferred AI chatbot. With so many tech platforms encouraging us to interact with their newly-introduced chatbots and talk to them as if they are real humans, people are increasingly turning to these large language model-powered functions for companionship, emotional support and, sometimes, love. This might raise an eyebrow or elicit a snigger.

We evaluated 4 systems (ELIZA, GPT-4o, LLaMa-3.1-405B, and GPT-4.5) in two randomised, controlled, and pre-registered Turing tests on independent populations. Participants had 5 minute conversations simultaneously with another human participant and one of these systems before judging which conversational partner they thought was human. When prompted to adopt a humanlike persona, GPT-4.5 was judged to be the human 73% of the time: significantly more often than interrogators selected the real human participant. LLaMa-3.1, with the same prompt, was judged to be the human 56% of the time -- not significantly more or less often than the humans they were being compared to -- while baseline models (ELIZA and GPT-4o) achieved win rates significantly below chance (23% and 21% respectively). The results constitute the first empirical evidence that any artificial system passes a standard three-party Turing test. The results have implications for debates about what kind of intelligence is exhibited by Large Language Models (LLMs), and the social and economic impacts these systems are likely to have.

Earlier this year, the technology world welcomed back a long-lost friend. ELIZA, the world's first artificial intelligence chatbot, had wowed the computer scientists of the mid-1960s with its ability to engage in seemingly meaningful conversation. But, for decades, ELIZA was considered lost because its creator – Joseph Weizenbaum at the Massachusetts Institute of Technology – never published the 420 lines of code he used to create it. "At that time, it was actually kind of not normal to publish code," says Jeffrey Shrager at Stanford University in California. Weizenbaum might even have thought that nobody would find it particularly interesting.

AI Agent, powered by large language models (LLMs) as its cognitive core, is an intelligent agentic system capable of autonomously controlling and determining the execution paths under user's instructions. With the burst of capabilities of LLMs and various plugins, such as RAG, text-to-image/video/3D, etc., the potential of AI Agents has been vastly expanded, with their capabilities growing stronger by the day. However, at the intersection between AI and web3, there is currently no ideal agentic framework that can seamlessly integrate web3 applications into AI agent functionalities. In this paper, we propose Eliza, the first open-source web3-friendly Agentic framework that makes the deployment of web3 applications effortless. We emphasize that every aspect of Eliza is a regular Typescript program under the full control of its user, and it seamlessly integrates with web3 (i.e., reading and writing blockchain data, interacting with smart contracts, etc.). Furthermore, we show how stable performance is achieved through the pragmatic implementation of the key components of Eliza's runtime. Our code is publicly available at https://github.com/ai16z/eliza.

ELIZA Reanimated: The world's first chatbot restored on the world's first time sharing system Abstract ELIZA, created by Joseph Weizenbaum at MIT in the early 1960s, is usually considered the world's first chatbot. It was developed in MAD-SLIP on MIT's CTSS, the world's first time-sharing system, on an IBM 7094. We discovered an original ELIZA printout in Prof. Weizenbaum's archives at MIT, including an early version of the famous DOCTOR script, a nearly complete version of the MAD-SLIP code, and various support functions in MAD and FAP. Here we describe the reanimation of this original ELIZA on a restored CTSS, itself running on an emulated IBM 7094. The entire stack is open source, so that any user of a unix-like OS can run the world's first chatbot on the world's first time-sharing system. "We can only see a short distance ahead, but we can see plenty there that needs to be done." If Alan Turing was AI's founding father, Ada Lovelace may well have been its founding mother. Over a century before Turning famously proposed using the Imitation Game to determine whether a computer is intelligent [34], Lady Lovelace described the potential of Charles Babbage's Analytical Engine to "act upon other things besides number, were objects found whose mutual fundamental relations could be expressed by those of the abstract science of operations, and which should be also susceptible of adaptations to the action of the operating notation and mechanism of the engine."[27] Ada's prescient insight that machines could act upon entities besides numbers foreshadowed symbolic computing which, in the 1950s, a mere moment after Turing's famous paper, arose, and remains today, one of the foundations of artificial intelligence[28].

In the nearly two years since its release, ChatGPT has shown some remarkably human-like behavior, from trying to seduce a journalist to acing the bar exam. That has left some people wondering whether computers are approaching human levels of intelligence. Most computer scientists do not think machines are the intellectual equals of people yet, but they have not developed a consensus on how to measure intelligence, or what exactly to measure. The canonical experiment to check for machine intelligence is the Turing test, proposed by Alan Turing in his 1950 paper "Computing Machinery and Intelligence." Turing argues that if a computer could convince a person having a typed conversation with it that it was human, that might be a sign of intelligence.

ELIZA, often considered the world's first chatbot, was written by Joseph Weizenbaum in the early 1960s. Weizenbaum did not intend to invent the chatbot, but rather to build a platform for research into human-machine conversation and the important cognitive processes of interpretation and misinterpretation. His purpose was obscured by ELIZA's fame, resulting in large part from the fortuitous timing of it's creation, and it's escape into the wild. In this paper I provide a rich historical context for ELIZA's creation, demonstrating that ELIZA arose from the intersection of some of the central threads in the technical history of AI. I also briefly discuss how ELIZA escaped into the world, and how its accidental escape, along with several coincidental turns of the programming language screws, led both to the misapprehension that ELIZA was intended as a chatbot, and to the loss of the original ELIZA to history for over 50 years.

We evaluated 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test. Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%). The results provide the first robust empirical demonstration that any artificial system passes an interactive 2-player Turing test. The results have implications for debates around machine intelligence and, more urgently, suggest that deception by current AI systems may go undetected. Analysis of participants' strategies and reasoning suggests that stylistic and socio-emotional factors play a larger role in passing the Turing test than traditional notions of intelligence.