Semi Centralized Training Decentralized Execution Architecture for Multi Agent Deep Reinforcement Learning in Traffic Signal Control

Traffic congestion is a major and complex challenge for cities worldwide with the rapid growth of urbanization and vehicle ownership. Longer commute times, excessive fuel consumption, and elevated air pollution levels are direct consequences of over-saturated roads. For instance, according to the 2024 INRIX Global Traffic Scorecard, individual commuters in Istanbul, New York City, and Chicago experienced total annual delay of about 105, 102, and 102 hours, respectively, underscoring the magnitude of intersection-driven delays in major metros (INRIX). Within urban networks, signalized intersections are the dominant bottlenecks: the policies implemented at these intersections allocate scarce space-time among competing traffic streams and therefore largely determine corridor-level delay, queues, and emissions. Reinforcement learning (RL) has become a standard practice for adaptive traffic signal control (ATSC), controlling phase selection and timing as a sequential decision problem that optimizes long-horizon objectives such as delay, throughput, and emissions under nonstationary demand (Yau et al., 2017). Deep RL (DRL) extends this by using function approximation to digest rich state representations--from detector queues to trajectories and graph-structured networks--enabling policies that generalize across varying traffic flows and topologies (Zhao et al., 2024). Collectively, this body of work motivates moving beyond single-intersection controllers toward coordinated, network-level solutions and setting the stage for multi-agent formulations.