Automatic Rule Learning for Autonomous Driving Using Semantic Memory

Abstract-- This paper presents a novel approach for automatic rule learning applicable to an autonomous driving system using real driving data. We represent the actions of other agents (provided by sensors) in the scene via temporal sequences called "episodes". The proposed method adaptively creates new rules automatically by extracting and segmenting valuable information about other agents and their interactions. These rules, which take the form of a "spatiotemporal grammar" or "episodic memory" are stored in a "semantic memory" module for later use. During the testing phase, the system segments constantly changing situations, finds the corresponding parse tree for the current state of the self-car and other agents, and applies the rules stored in semantic memory to stop, yield, continue driving, etc. The method also allows for continues online training during agent driving. Unlike traditional deep driving and machine learning methods that require significant amount of training data to achieve desired quality, the proposed method demonstrates good results with just a few training examples.