The Foreseeable Future: Self-Supervised Learning to Predict Dynamic Scenes for Indoor Navigation

Abstract--We present a method for generating, predicting, and using Spatiotemporal Occupancy Grid Maps (SOGM), which embed future semantic information of real dynamic scenes. We present an auto-labeling process that creates SOGMs from noisy real navigation data. We use a 3D-2D feedforward architecture, trained to predict the future time steps of SOGMs, given 3D lidar frames as input. Our pipeline is entirely self-supervised, thus enabling lifelong learning for real robots. The network is composed of a 3D back-end that extracts rich features and enables the semantic segmentation of the lidar frames, and a 2D front-end that predicts the future information embedded in the SOGM representation, potentially capturing the complexities and uncertainties of real-world multi-agent, multi-future interactions. We also design a navigation system that uses these predicted SOGMs within planning, after they have been transformed into Spatiotemporal Risk Maps (SRMs). We verify our navigation system's abilities in simulation, validate it on a real robot, study SOGM predictions on real data in various circumstances, and Time is represented as a color, from red (now) to yellow (future). REDICTING the future has always fascinated humanity. In this paper, we provide a detailed curiosity for the unknown has never faded. But we tend to description of the collection of algorithms required for these forget that we already predict the future constantly in our daily various tasks, for a complete view of the overall approach, as lives, only it is for a short horizon. Walking in the street, illustrated in Figure 2. catching a falling object, or driving a car, all these actions Some of the algorithms we use have already been introduced require a certain level of anticipation. In the first one [1], we described can become quite good at predicting what might happen for how to automatically annotate 3D lidar points, and train a the next few seconds in many situations; what about robots? In the second one We study this question in the context of a concrete example: [2], our system learned to predict the future of dynamic a robot learning on its own to navigate among humans or scenes as SOGMs. Until now, we only evaluated results in dynamic objects in an indoor space. Our approach allows the a simulated environment.