Yann LeCun's new venture is a contrarian bet against large language models In an exclusive interview, the AI pioneer shares his plans for his new Paris-based company, AMI Labs. Yann LeCun is a Turing Award recipient and a top AI researcher, but he has long been a contrarian figure in the tech world. He believes that the industry's current obsession with large language models is wrong-headed and will ultimately fail to solve many pressing problems. Instead, he thinks we should be betting on world models--a different type of AI that accurately reflects the dynamics of the real world. He is also a staunch advocate for open-source AI and criticizes the closed approach of frontier labs like OpenAI and Anthropic. Perhaps it's no surprise, then, that he recently left Meta, where he had served as chief scientist for FAIR (Fundamental AI Research), the company's influential research lab that he founded. Meta has struggled to gain much traction with its open-source AI model Llama and has seen internal shake-ups, including the controversial acquisition of ScaleAI. LeCun sat down with in an exclusive online interview from his Paris apartment to discuss his new venture, life after Meta, the future of artificial intelligence, and why he thinks the industry is chasing the wrong ideas.

Why an AI'godfather' is quitting Meta after 12 years Just a couple of weeks ago, one of the godfathers of artificial intelligence was in St James's Palace being handed an award from King Charles for his work in artificial intelligence (AI). Professor Yann LeCun was being honoured along with six other recipients for his contributions to the field, which have been credited as advancing deep learning. But Mr LeCun is at odds with some of the AI world over the future of the generation-defining technology. And now he is going all-in on his idea of advanced machine intelligence after announcing he is leaving his role as Meta's chief AI scientist to start a new firm. During his 12 years at the company, Prof LeCun won the prestigious Turing Award and witnessed several flurries of excitement around AI - not least the most recent boom in generative AI accelerated by rival OpenAI's launch of ChatGPT in late 2022.

Upon its release earlier this month, OpenAI's Sora 2 model took the Internet by storm, thanks to its ability to generate realistic videos from just a text prompt. But Sora is about more than just capturing eyeballs with viral content. "On the surface, Sora, for example, does not look like it is AGI-relevant," OpenAI CEO Sam Altman said on a podcast earlier this month. "But I would bet that if we can build really great world models, that will be much more important to AGI than people think." Altman was speaking to a growing belief inside the AI industry at large: that if you can simulate the world with enough accuracy, you could drop AI agents into those simulations. There, they could learn more skills than they currently can from just text, photos, and videos--because they could interact with a simulated world. That form of training could be highly efficient, in part because simulated time can be accelerated, and because many simulations can be run in parallel.

The stop gradient and exponential moving average iterative procedures are commonly used in non-contrastive approaches to self-supervised learning to avoid representation collapse, with excellent performance in downstream applications in practice. This presentation investigates these procedures from the dual viewpoints of optimization and dynamical systems. We show that, in general, although they do not optimize the original objective, or any other smooth function, they do avoid collapse Following Tian et al. (2021), but without any of the extra assumptions used in their proofs, we then show using a dynamical system perspective that, in the linear case, minimizing the original objective function without the use of a stop gradient or exponential moving average always leads to collapse. Conversely, we characterize explicitly the equilibria of the dynamical systems associated with these two procedures in this linear setting as algebraic varieties in their parameter space, and show that they are, in general, asymptotically stable . Our theoretical findings are illustrated by empirical experiments with real and synthetic data. Self-supervised learning (or SSL) is an approach to representation learning that exploits the internal consistency of training data without requiring expensive annotations. However, non-contrastive approaches to SSL (Assran et al., 2023; Bardes et al., 2022) that take as input different views of the same data samples and learn to predict one view from the other, are susceptible to representational collapse where a constant embedding is learned for all data points (LeCun, 2022). We use in this presentation the dual viewpoints of optimization and dynamical systems to study theoretically and empirically the well-known stop gradient (Chen and He, 2021) and exponential moving average (Grill et al., 2020) training procedures that are specifically designed to avoid this problem. Here C is the global minimum of E (θ,ψ) (shown as negative instead of zero for readibility) associated with a collapse of the training process; B is a nontrivial local minimum one may reach using an appropriate regularization to avoid collapse; and A is a limit point of the stop gradient (SG) training procedure associated with parameters θ and ψ at convergence. In general, it is not a minimum of E and thus does not correspond to a collapse of the training process, but it is a minimum with respect to ψ of E ( θ,ψ).

'I have to do it': Why one of the world's most brilliant AI scientists left the US for China In 2020, after spending half his life in the US, Song-Chun Zhu took a one-way ticket to China. By the time Song-Chun Zhu was six years old, he had encountered death more times than he could count. This was the early 1970s, the waning years of the Cultural Revolution, and his father ran a village supply store in rural China . There was little to do beyond till the fields and study Mao Zedong at home, and so the shop became a refuge where people could rest, recharge and share tales. Zhu grew up in that shop, absorbing a lifetime's worth of tragedies: a family friend lost in a car crash, a relative from an untreated illness, stories of suicide or starvation. "That was really tough," Zhu recalled recently. The young Zhu became obsessed with what people left behind after they died. One day, he came across a book that contained his family genealogy. When he asked the bookkeeper why it included his ancestors' dates of birth and death but nothing about their lives, the man told him matter of factly that they were peasants, so there was nothing worth recording. He resolved that his fate would be different. Today, at 56, Zhu is one of the world's leading authorities in artificial intelligence. In 1992, he left China for the US to pursue a PhD in computer science at Harvard. Later, at University of California, Los Angeles (UCLA), he led one of the most prolific AI research centres in the world, won numerous major awards, and attracted prestigious research grants from the Pentagon and the National Science Foundation. He was celebrated for his pioneering research into how machines can spot patterns in data, which helped lay the groundwork for modern AI systems such as ChatGPT and DeepSeek. He and his wife, and their two US-born daughters, lived in a hilltop home on Los Angeles's Mulholland Drive. He thought he would never leave. But in August 2020, after 28 years in the US, Zhu astonished his colleagues and friends by suddenly moving back to China, where he took up professorships at two top Beijing universities and a directorship in a state-sponsored AI institute.

Abstract: With the advent of Joint Embedding Predictive Architectures (JEPAs), which appear to be more capable than reconstruction-based methods, this paper introduces a novel technique for creating world models using continuous-time dynamic systems from arbitrary observation data. The proposed method integrates sequence embeddings with neural ordinary differential equations (neural ODEs). It employs loss functions that enforce contractive embeddings and Lipschitz constants in state transitions to construct a well-organized latent state space. The approach's effectiveness is demonstrated through the generation of structured latent state-space models for a simple pendulum system using only image data. This opens up a new technique for developing more general control algorithms and estimation techniques with broad applications in robotics.

Fortunately, our Theorem 1 provides a clean positive answer.

One of the "godfathers" of modern artificial intelligence has predicted a further revolution in the technology by the end of the decade, and says current systems are too limited to create domestic robots and fully automated cars. Yann LeCun, the chief AI scientist at Mark Zuckerberg's Meta, said new breakthroughs are needed in order for the systems to understand and interact with the physical world. LeCun spoke as one of seven engineers who were awarded the 500,000 Queen Elizabeth prize for engineering on Tuesday for their contributions to machine learning, a cornerstone of AI. Recent breakthroughs in the sector, led by the launch of OpenAI's ChatGPT chatbot, have heightened expectations – and fears – of systems gaining human levels of intelligence. However, LeCun said there was some way to go before AIs matched humans or animals, with the current cutting-edge technology excelling at "manipulating language" but not at understanding the physical world.