Sony's AI Racer Wins By Being Nice

The difference was due to adding a new dimension of driving skills. In addition to Race Car Control, the essential physics for car dynamics, racing lines and precision maneuvers and Racing Tactics, the split-second decision-making for passing and overtaking, training was provided in the Racing Etiquette deemed essential for fair play. To achieve this not only did the team have to find ways to encode the written and unwritten rules of racing into a complex reward function they also had to choose training examples for reinforcement learning that provided the correct balance of aggression and timidity.