Inner Monologue: Embodied Reasoning through Planning with Language Models - Technology Org

Large language models (LLMs) have rich internalized knowledge about the world and are able to carry out some degree of deduction and respond to questions requiring reasoning and inference. An example of ViLD object detection segmentation mask and bounding box predictions. The Inner Monologue system is created to chain together these components in a shared language prompt. As a result, the system can accomplish complex, long-horizon, and unseen tasks in simulation as well as on real-world robotic platforms. Recent works have shown how the reasoning capabilities of Large Language Models (LLMs) can be applied to domains beyond natural language processing, such as planning and interaction for robots.