We consider a dynamic millimeter-wave network with integrated access and backhaul, where mobile relay nodes move to auto-reconfigure the wireless backhaul. Specifically, we focus on in-band relaying networks, which conduct access and backhaul links on the same frequency band with severe constraints on co-channel interference. In this context, we jointly study the complex problem of dynamic relay node positioning, user association, and backhaul capacity allocation. To address this problem, with limited complexity, we adopt a hierarchical multi-agent reinforcement with a two-level structure. A high-level policy dynamically coordinates mobile relay nodes, defining the backhaul configuration for a low-level policy, which jointly assigns user equipment to each relay and allocates the backhaul capacity accordingly. The resulting solution automatically adapts the access and backhaul network to changes in the number of users, the traffic distribution, and the variations of the channels. Numerical results show the effectiveness of our proposed solution in terms of convergence of the hierarchical learning procedure. It also provides a significant backhaul capacity and network sum-rate increase (up to 3.5x) compared to baseline approaches.

Millimeter Wave (MmWave) communication is one of the key technology of the fifth generation (5G) wireless systems to achieve the expected 1000x data rate. With large bandwidth at mmWave band, the link capacity between users and base stations (BS) can be much higher compared to sub-6GHz wireless systems. Meanwhile, due to the high cost of infrastructure upgrade, it would be difficult for operators to drastically enhance the capacity of backhaul links between mmWave BSs and the core network. As a result, the data rate provided by backhaul may not be sufficient to support all mmWave links, the backhaul connection becomes the new bottleneck that limits the system performance. On the other hand, as mmWave channels are subject to random blockage, the data rates of mmWave users significantly vary over time. With limited backhaul capacity and highly dynamic data rates of users, how to allocate backhaul resource to each user remains a challenge for mmWave systems. In this article, we present a deep reinforcement learning (DRL) approach to address this challenge. By learning the blockage pattern, the system dynamics can be captured and predicted, resulting in efficient utilization of backhaul resource. We begin with a discussion on DRL and its application in wireless systems. We then investigate the problem backhaul resource allocation and present the DRL based solution. Finally, we discuss open problems for future research and conclude this article.