Smart Scheduling based on Deep Reinforcement Learning for Cellular Networks

To improve the system performance towards the Shannon limit, advanced radio resource management mechanisms play a fundamental role. In particular, scheduling should receive much attention, because it allocates radio resources among different users in terms of their channel conditions and QoS requirements. The difficulties of scheduling algorithms are the tradeoffs need to be made among multiple objectives, such as throughput, fairness and packet drop rate. We propose a smart scheduling scheme based on deep reinforcement learning (DRL). We not only verify the performance gain achieved, but also provide implementation-friend designs, i.e., a scalable neural network design for the agent and a virtual environment training framework. With the scalable neural network design, the DRL agent can easily handle the cases when the number of active users is time-varying without the need to redesign and retrain the DRL agent. Training the DRL agent in a virtual environment offline first and using it as the initial version in the practical usage helps to prevent the system from suffering from performance and robustness degradation due to the time-consuming training. Through both simulations and field tests, we show that the DRL-based smart scheduling outperforms the conventional scheduling method and can be adopted in practical systems. The wireless communication industry has been keeping a fast growing and updating speed for several decades. About every ten years, new generations of mobile communication system were standardized with lots of new features and supported scenarios. Thanks to the evolution of wireless communications technologies, we are now enjoying diverse services and applications conveniently. It is well known that the fifth generation (5G) mobile communications system supports three major categories of services, i.e., enhanced mobile broadband (eMBB), ultrareliable and low-latency communications (uRLLC) and massive machine-type communications (mMTC). Meanwhile, new applications and scenarios have never stopped coming up, which sets up new requirements including even higher throughput, more connected devices, faster access with lower latency and higher efficiency for wireless communication systems. With all these requirements in mind, designing a new generation of mobile communications system becomes a quite challenging work.