ChatGPT for Robotics: Design Principles and Model Abilities

The rapid advancement in natural language processing (NLP) has led to the development of large language models (LLMs), such as BERT [2], GPT-3 [3], and Codex [4], that are revolutionizing a wide range of applications. These models have achieved remarkable results in various tasks such as text generation, machine translation, and code synthesis, among others. A recent addition to this collection of models was the OpenAI ChatGPT [1], a pretrained generative text model which was finetuned using human feedback. Unlike previous models which operate mostly upon a single prompt, ChatGPT provides particularly impressive interaction skills through dialog, combining text generation with code synthesis. Our goal in this paper is to investigate if and how the abilities of ChatGPT can generalize to the domain of robotics. Robotics systems, unlike text-only applications, require a deep understanding of real-world physics, environmental context, and the ability to perform physical actions. A generative robotics model needs to have a robust commonsense knowledge and a sophisticated world model, and the ability to interact with users to interpret and execute commands in ways that are physically possible and that makes sense in the real world. These challenges fall beyond the original scope of language models, as they must not only understand the meaning of a given text, but also translate the intent into a logical sequence of physical actions. In recent years there have been different attempts to incorporate language into robotics systems.