On a side street outside the headquarters of the Department of Science, Innovation and Technology in the center of London on Monday, 20 or so protesters are getting their chants in order. When do we want it?" These protesters are part of Pause AI, a group of activists petitioning for companies to pause development of large AI models which they fear could pose a risk to the future of humanity. Other PauseAI protests are taking place across the globe: In San Francisco, New York, Berlin, Rome, Ottawa, and a handful of other cities. Their aim is to grab the attention of voters and politicians ahead of the AI Seoul Summit--a follow-up to the AI Safety Summit held in the UK in November 2023. But the loosely organized group of protesters itself is still figuring out exactly the best way to communicate its message. "The Summit didn't actually lead to meaningful regulations," says Joep Meindertsma, the founder of PauseAI. The attendees at the conference agreed to the "Bletchley Declaration," but that agreement doesn't mean much, Meindertsma says. "It's only a small first step, and what we need are binding international treaties." The group's main demand is for a pause on the training of AI systems more powerful than GPT-4--it's calling for all countries to implement this measure, but specifically calls out the United States as the home of most leading AI labs. The group also wants all UN member states to sign a treaty that sets up an international AI safety agency with responsibility for granting new deployments of AI systems and training runs of large models. Their protests are taking place on the same day as OpenAI announced a new version of ChatGPT to make the chatbot act more like a human. "We have banned technology internationally before," says Meindertsma, pointing to the Montreal Protocol, a global agreement finalized in 1987 that saw the phaseout of CFCs and other chemicals known to deplete the ozone layer. "We've got treaties that ban blinding laser weapons.

The UK government's AI Safety Summit, which claims to be "bringing together the world to discuss frontier AI", has come under fire for a lack of diverse perspectives among its delegates, with critics dismissing it as little more than a photo opportunity. Prime minister Rishi Sunak is hosting a group of 100 representatives from business, politics and academia at Bletchley Park on 1 and 2 November to discuss the risks of so-called "frontier" AI models – defined as "highly capable general-purpose AI models that can perform a wide variety of tasks and match or exceed the capabilities present in today's most advanced models". But the government has not publicly revealed the list of delegates, and campaigners have criticised the narrow range of voices in attendance. Mark Lee at the University of Birmingham says the summit is a stage-managed "photo opportunity" rather than a chance for open discourse, and that it is focusing on the wrong problems. "We need an open debate," he says.

The first time we speak, Joep Meindertsma is not in a good place. He tears up as he describes a conversation in which he warned his niece about the risk of artificial intelligence causing societal collapse. Afterward, she had a panic attack. "I cry every other day," he says, speaking over Zoom from his home in the Dutch city of Utrecht. "Every time I say goodbye to my parents or friends, it feels like it could be the last time."