A Reinforcement Learning Approach to Age of Information in Multi-User Networks

We consider a source node that communicates the most up-to-date status packets to multiple users (see Figure 1). We are interested in the average age of information (AoI) [1]-[3] at the users, for a system in which the source node samples an underlying timevarying process and schedules the transmission of the sample values over imperfect links. The AoI at each user at any point in time can simply be defined as the amount of time elapsed since the most recent status update at that user was generated. Most of the earlier work on AoI consider queue-based models, in which the status updates arrive at the source node randomly following a memoryless Poisson process, and are stored in a buffer before being transmitted to the destination [2], [3]. Instead, in the so-called generate-at-will model [1], [4]-[7], also considered in this paper, the status updates of the underlying process of interest can be generated at any time by the source node. AoI in multi-user networks has been studied in [6]- [11]. It is shown in [8] that the scheduling problem for the age minimization is NPhard in general. Scheduling transmissions to multiple receivers is investigated in [7], focusing on a perfect transmission medium, and the optimal scheduling algorithm is shown to be threshold-type. Average AoI has also been studied when status updates over unreliable multi-access channels [10] and multi-cast networks [11] are considered.