Inspired by modern datadriven through diverse environments to distant goals? This remains approaches, the lower level of our system design is an open challenge due to the complexity and difficulty of a neural network-based controller that maps observations designing a robot that can generalise over environments, directly to velocity commands, and which is learned end-toend tolerate significant mapping and positioning inaccuracies from real world experience. Neural networks have the and recover from inevitable navigation errors. While many flexibility to accept a wide variety of input types, and we works tackle robot navigation, few systems capable of find that design space for the signals used by the system's long-range, kilometre-scale navigation exist. Classical robot upper level to guide the lower level is large. We exploit systems capable of long-range navigation like Montemerlo this property to design several different types of guidance et al. (2008); Kümmerle et al. (2013) use e xplicit signals, which we call intentions. We find that designing maps and find paths over them using classical planning the appropriate intention imbues the navigation system with algorithms (Siegwart et al. 2011), allowing them to reach specific abilities, such as the ability to tolerate significant arbitrarily distant goals in principle.

How can we build robots for open-world semantic navigation tasks, like searching for target objects in novel scenes? While foundation models have the rich knowledge and generalisation needed for these tasks, a suitable scene representation is needed to connect them into a complete robot system. We address this with Open Scene Graphs (OSGs), a topo-semantic representation that retains and organises open-set scene information for these models, and has a structure that can be configured for different environment types. We integrate foundation models and OSGs into the OpenSearch system for Open World Object-Goal Navigation, which is capable of searching for open-set objects specified in natural language, while generalising zero-shot across diverse environments and embodiments. Our OSGs enhance reasoning with Large Language Models (LLM), enabling robust object-goal navigation outperforming existing LLM approaches. Through simulation and real-world experiments, we validate OpenSearch's generalisation across varied environments, robots and novel instructions.

Humans are remarkable in their ability to navigate without metric information. We can read abstract 2D maps, such as floor-plans or hand-drawn sketches, and use them to navigate in unseen rich 3D environments, without requiring prior traversals to map out these scenes in detail. We posit that this is enabled by the ability to represent the environment abstractly as interconnected navigational behaviours, e.g., "follow the corridor" or "turn right", while avoiding detailed, accurate spatial information at the metric level. We introduce the Scene Action Map (SAM), a behavioural topological graph, and propose a learnable map-reading method, which parses a variety of 2D maps into SAMs. Map-reading extracts salient information about navigational behaviours from the overlooked wealth of pre-existing, abstract and inaccurate maps, ranging from floor-plans to sketches. We evaluate the performance of SAMs for navigation, by building and deploying a behavioural navigation stack on a quadrupedal robot. Videos and more information is available at: https://scene-action-maps.github.io.