SIMA 2: A Generalist Embodied Agent for Virtual Worlds

SIMA team, null, Bolton, Adrian, Lerchner, Alexander, Cordell, Alexandra, Moufarek, Alexandre, Bolt, Andrew, Lampinen, Andrew, Mitenkova, Anna, Hallingstad, Arne Olav, Vujatovic, Bojan, Li, Bonnie, Lu, Cong, Wierstra, Daan, Sawyer, Daniel P., Slater, Daniel, Reichert, David, Vercelli, Davide, Hassabis, Demis, Hudson, Drew A., Williams, Duncan, Hirst, Ed, Pardo, Fabio, Hill, Felix, Besse, Frederic, Openshaw, Hannah, Chan, Harris, Soyer, Hubert, Wang, Jane X., Clune, Jeff, Agapiou, John, Reid, John, Marino, Joseph, Kim, Junkyung, Gregor, Karol, Sridhar, Kaustubh, McKinney, Kay, Kampis, Laura, Zhang, Lei M., Matthey, Loic, Wang, Luyu, Raad, Maria Abi, Loks-Thompson, Maria, Engelcke, Martin, Kecman, Matija, Jackson, Matthew, Gazeau, Maxime, Purkiss, Ollie, Knagg, Oscar, Stys, Peter, Mendolicchio, Piermaria, Hadsell, Raia, Ke, Rosemary, Faulkner, Ryan, Chakera, Sarah, Baveja, Satinder Singh, Legg, Shane, Kashem, Sheleem, Terzi, Tayfun, Keck, Thomas, Harley, Tim, Scholtes, Tim, Roberts, Tyson, Mnih, Volodymyr, Liu, Yulan, Wang, Zhengdong, Ghahramani, Zoubin

arXiv.org Artificial Intelligence 

We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a significant step toward active, goal-directed interaction within an embodied environment. Unlike prior work (e.g., SIMA 1) limited to simple language commands, SIMA 2 acts as an interactive partner, capable of reasoning about high-level goals, conversing with the user, and handling complex instructions given through language and images. Across a diverse portfolio of games, SIMA 2 substantially closes the gap with human performance and demonstrates robust generalization to previously unseen environments, all while retaining the base model's core reasoning capabilities. Furthermore, we demonstrate a capacity for open-ended self-improvement: by leveraging Gemini to generate tasks and provide rewards, SIMA 2 can autonomously learn new skills from scratch in a new environment. This work validates a path toward creating versatile and continuously learning agents for both virtual and, eventually, physical worlds.

Duplicate Docs Excel Report

Title
None found

Similar Docs  Excel Report  more

TitleSimilaritySource
None found