s-ppo
Deep Reinforcement Learning-Based User Association in Hybrid LiFi/WiFi Indoor Networks
--Hybrid light fidelity (LiFi) and wireless fidelity (WiFi) indoor networks has been envisioned as a promising technology to alleviate radio frequency spectrum crunch to accommodate the ever-increasing data rate demand in indoor scenarios. The hybrid LiFi/WiFi indoor networks can leverage the advantages of fast data transmission from LiFi and wider coverage of WiFi, thus complementing well with each other and further improving the network performance compared with the standalone networks. However, to leverage the co-existence, several challenges should be addressed, including but not limited to user association, mobility support, and efficient resource allocation. Therefore, the objective of the paper is to design a new user-access point association algorithm to maximize the sum throughput of the hybrid networks. We first mathematically formulate the sum data rate maximization problem by determining the AP selection for each user in indoor networks with consideration of user mobility and practical capacity limitations, which is a nonconvex binary integer programming problem. T o solve this problem, we then propose a sequential-proximal policy optimization (S-PPO) based deep reinforcement learning method. Extensive simulations are conducted to evaluate the proposed method by comparing it with exhaustive search (ES), signal strength strategy (SSS), and trust region policy optimization (TRPO) methods. Comprehensive simulation results demonstrate that our solution algorithm can outperform SSS by about 32.25% of the sum throughput and 19.09% of the fairness on average, and outperform TRPO by about 10.34% and 10.23%, respectively. Over the past few years, the usage of the internet has been continuously increasing. According to the latest data, people spend an average of 6 hours and 58 minutes daily on screens connected to the internet [1]. Moreover, an increasing number of applications require high-speed support, such as video calls, VR gaming, streaming media, and so on. However, we are facing a global digital divide, i.e., internet speeds in urban areas are often much faster than in rural areas, due to the generally less developed internet infrastructure in rural locations. Visible light communication (VLC), where light-emitting diodes (LEDs) can be used to transmit data by optical spectrum, has been envisioned as a promising solution for last-mile access because of its high bandwidth, enhanced security, electromagnetic interference-free nature, and easy integration with existing infrastructure [2]-[7].
Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agents
Sun, Chung-En, Gao, Sicun, Weng, Tsui-Wei
Robustness remains a paramount concern in deep reinforcement learning (DRL), with randomized smoothing emerging as a key technique for enhancing this attribute. However, a notable gap exists in the performance of current smoothed DRL agents, often characterized by significantly low clean rewards and weak robustness. In response to this challenge, our study introduces innovative algorithms aimed at training effective smoothed robust DRL agents. We propose S-DQN and S-PPO, novel approaches that demonstrate remarkable improvements in clean rewards, empirical robustness, and robustness guarantee across standard RL benchmarks. Notably, our S-DQN and S-PPO agents not only significantly outperform existing smoothed agents by an average factor of $2.16\times$ under the strongest attack, but also surpass previous robustly-trained agents by an average factor of $2.13\times$. This represents a significant leap forward in the field. Furthermore, we introduce Smoothed Attack, which is $1.89\times$ more effective in decreasing the rewards of smoothed agents than existing adversarial attacks.