It is hard to make robots (including telerobots) that are useful, and harder still to make autonomous and collaborative robots that are robust and general. Current smart robots are created using manual programming, mathematical models, planning frameworks, and reinforcement learning. These methods do not lead to the leaps in performance and generality seen with deep learning, generative AI, and foundation models (FMs). Today's robots do not learn to provide home care, to be nursing assistants, or to do household chores nearly as well as people do. Addressing the aspirational goals of service robots requires improving how they are created. The high cost of bipedal multi-sensory robots ("bodies") is a significant obstacle for both research and deployment. A deeper issue is that mainstream FMs ("minds") are not created by agents sensing, acting, and learning in context in the real world. They do not lead to robots that communicate well or collaborate. They do not lead to robots that learn by experimenting, by asking others, and by imitation learning as appropriate. In short, they do not lead to robots that are ready to be deployed widely in human service applications. This paper focuses on what human-compatible service robots need to know. It recommends developing "robotic" FMs based on diverse experiential data.

We present BEHAVIOR-1K, a comprehensive simulation benchmark for human-centered robotics. BEHAVIOR-1K includes two components, guided and motivated by the results of an extensive survey on "what do you want robots to do for you?". The first is the definition of 1,000 everyday activities, grounded in 50 scenes (houses, gardens, restaurants, offices, etc.) with more than 9,000 objects annotated with rich physical and semantic properties. The second is OMNIGIBSON, a novel simulation environment that supports these activities via realistic physics simulation and rendering of rigid bodies, deformable bodies, and liquids. Our experiments indicate that the activities in BEHAVIOR-1K are long-horizon and dependent on complex manipulation skills, both of which remain a challenge for even state-of-the-art robot learning solutions. To calibrate the simulation-to-reality gap of BEHAVIOR-1K, we provide an initial study on transferring solutions learned with a mobile manipulator in a simulated apartment to its real-world counterpart. We hope that BEHAVIOR-1K's human-grounded nature, diversity, and realism make it valuable for embodied AI and robot learning research. Project website: https://behavior.stanford.edu.