Llontop, Edith
GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
NVIDIA, null, Bjorck, Johan, Castañeda, Fernando, Cherniadev, Nikita, Da, Xingye, Ding, Runyu, Fan, Linxi "Jim", Fang, Yu, Fox, Dieter, Hu, Fengyuan, Huang, Spencer, Jang, Joel, Jiang, Zhenyu, Kautz, Jan, Kundalia, Kaushil, Lao, Lawrence, Li, Zhiqi, Lin, Zongyu, Lin, Kevin, Liu, Guilin, Llontop, Edith, Magne, Loic, Mandlekar, Ajay, Narayan, Avnish, Nasiriany, Soroush, Reed, Scott, Tan, You Liang, Wang, Guanzhi, Wang, Zu, Wang, Jing, Wang, Qi, Xiang, Jiannan, Xie, Yuqi, Xu, Yinzhen, Xu, Zhenjia, Ye, Seonghyeon, Yu, Zhiding, Zhang, Ao, Zhang, Hao, Zhao, Yizhou, Zheng, Ruijie, Zhu, Yuke
General-purpose robots need a versatile body and an intelligent mind. Recent advancements in humanoid robots have shown great promise as a hardware platform for building generalist autonomy in the human world. A robot foundation model, trained on massive and diverse data sources, is essential for enabling the robots to reason about novel situations, robustly handle real-world variability, and rapidly learn new tasks. To this end, we introduce GR00T N1, an open foundation model for humanoid robots. GR00T N1 is a Vision-Language-Action (VLA) model with a dual-system architecture. The vision-language module (System 2) interprets the environment through vision and language instructions. The subsequent diffusion transformer module (System 1) generates fluid motor actions in real time. Both modules are tightly coupled and jointly trained end-to-end. We train GR00T N1 with a heterogeneous mixture of real-robot trajectories, human videos, and synthetically generated datasets. We show that our generalist robot model GR00T N1 outperforms the state-of-the-art imitation learning baselines on standard simulation benchmarks across multiple robot embodiments. Furthermore, we deploy our model on the Fourier GR-1 humanoid robot for language-conditioned bimanual manipulation tasks, achieving strong performance with high data efficiency.
The Teenager's Problem: Efficient Garment Decluttering as Probabilistic Set Cover
Adler, Aviv, Ahmad, Ayah, Qiu, Yulei, Wang, Shengyin, Agboh, Wisdom C., Llontop, Edith, Qiu, Tianshuang, Ichnowski, Jeffrey, Kollar, Thomas, Cheng, Richard, Dogar, Mehmet, Goldberg, Ken
This paper addresses the "Teenager's Problem": efficiently removing scattered garments from a planar surface into a basket. As grasping and transporting individual garments is highly inefficient, we propose policies to select grasp locations for multiple garments using an overhead camera. Our core approach is segment-based, which uses segmentation on the overhead RGB image of the scene. We propose a Probabilistic Set Cover formulation of the problem, aiming to minimize the number of grasps that clear all garments off the surface. Grasp efficiency is measured by Objects per Transport (OpT), which denotes the average number of objects removed per trip to the laundry basket. Additionally, we explore several depth-based methods, which use overhead depth data to find efficient grasps. Experiments suggest that our segment-based method increases OpT by $50\%$ over a random baseline, whereas combined hybrid methods yield improvements of $33\%$. Finally, a method employing consolidation (with segmentation) is considered, which locally moves the garments on the work surface to increase OpT, when the distance to the basket is much greater than the local motion distances. This yields an improvement of $81\%$ over the baseline.