Scalable Deep Reinforcement Learning for Routing and Spectrum Access in Physical Layer