Action Space Reduction Strategies for Reinforcement Learning in Autonomous Driving

--Reinforcement Learning (RL) offers a promising framework for autonomous driving by enabling agents to learn control policies through interaction with environments. However, large and high-dimensional action spaces--often used to support fine-grained control--can impede training efficiency and increase exploration costs. In this study, we introduce and evaluate two novel structured action space modification strategies for RL in autonomous driving: dynamic masking and relative action space reduction. These approaches are systematically compared against fixed reduction schemes and full action space baselines to assess their impact on policy learning and performance. Our framework leverages a multimodal Proximal Policy Optimization agent that processes both semantic image sequences and scalar vehicle states. The proposed dynamic and relative strategies incorporate real-time action masking based on context and state transitions, preserving action consistency while eliminating invalid or subop-timal choices. Through comprehensive experiments across diverse driving routes, we show that action space reduction significantly improves training stability and policy performance. The dynamic and relative schemes, in particular, achieve a favorable balance between learning speed, control precision, and generalization. The development of Autonomous V ehicles (A Vs) has accelerated in recent years, offering the potential to improve road safety, reduce traffic congestion, and enhance mobility. However, building reliable and efficient self-driving systems remains a formidable challenge due to the complexity of real-world driving. These environments involve dynamic interactions with multiple agents, unpredictable traffic behaviors, and rare but critical edge cases that demand robust decision-making.