Why So Many AI Researchers Think the Machines Could Kill Everyone

A combination of rapid advances, recursive self-improvement, and agentic swarms are genuinely "spooking people" inside big labs. Earlier this year, Rishub Jain left his position as an artificial intelligence researcher at Google DeepMind after a revelation. As he worked on new models, he came to believe that he and everyone else on AI's frontier were ceding control. By using AI's coding skills to accelerate work on the next generation of models, he was removing himself from the equation. AI labs hope to evolve this approach to the point that AI will improve itself indefinitely, a process known as recursive self-improvement.