Inequity aversion reduces travel time in the traffic light control problem

The problem of traffic light control is to coordinate between intersections by controlling their traffic lights to improve traffic flow. This problem remains as one of the greatest challenges in the 21 st century (Qadri, Gökçe, & Öner, 2020). To tackle this challenge, researchers have taken various approaches such as the coordinated method modifying the start time of the green lights between the consecutive intersections (Koonce & Rodegerdts, 2008), the optimization technique minimizing the vehicles' travel time under certain traffic flow assumptions (Diakaki, Papageorgiou, & Aboudolas, 2002), and the models applying perimeter control to handle transferring flows between regions of a city (Kouvelas, Saeedmanesh, & Geroliminis, 2015, 2017). In addition to conventional approaches, the problem was recently tackled with Reinforcement Learning (RL) methods (Qadri et al., 2020). RL is a promising machinelearning framework where an agent interacts within a given environment by applying actions and receiving signals, which are interpreted as rewards and punishments. Via the interactions, the agents learn an optimal policy, a probability distribution over the available actions that maximizes the total obtained rewards for each visited environment state (Alamiyan-Harandi, Derhami, & Jamshidi, 2018; Rasheed, Yau, Noor, Wu, & Low, 2020; Sutton, Barto, et al., 1998). Encompassing several intersections, the traffic light control problem requires several actions to be executed at the same time. Hence, often the Multi-Agent (MA) extension of RL, i.e., MARL, is used for this problem.