Markov Models
Hardness in Markov Decision Processes: Theory and Practice
Finally, we benchmark five tabular agents in our newly proposed benchmark.
Contrastive Active Inference
In contrast, reinforcement learning requires human-designed rewards to accomplish any desired outcome.
Provably Efficient Reinforcement Learning with Linear Function Approximation under Adaptivity Constraints
Real-world reinforcement learning (RL) applications often come with possibly infinite state and action space, and in such a situation classical RL algorithms developed in the tabular setting are not applicable anymore. A popular approach to overcoming this issue is by applying function approximation techniques to the underlying structures of the Markov decision processes (MDPs).
Bayesian Optimistic Optimization: Optimistic
In this paper, we consider the RL in Markov decision processes (MDPs), where the agent observes the state of the environment at each timestep and makes decisions accordingly.
A Compositional Atlas of Tractable Circuit Operations for Probabilistic Inference
Circuit representations are becoming the lingua franca to express and reason about tractable generative and discriminative models.
Restless-UCB, an Efficient and Low-complexity Algorithm for Online Restless Bandits
As a result, our analysis technique can also be adopted to tighten the regret bounds of existing algorithms.
Restless-UCB, an Efficient and Low-complexity Algorithm for Online Restless Bandits
As a result, our analysis technique can also be adopted to tighten the regret bounds of existing algorithms.
Non-Stationary Restless Multi-Armed Bandits with Provable Guarantee
Hung, Yu-Heng, Hsieh, Ping-Chun, Wang, Kai
Online restless multi-armed bandits (RMABs) typically assume that each arm follows a stationary Markov Decision Process (MDP) with fixed state transitions and rewards. However, in real-world applications like healthcare and recommendation systems, these assumptions often break due to non-stationary dynamics, posing significant challenges for traditional RMAB algorithms. In this work, we specifically consider $N$-armd RMAB with non-stationary transition constrained by bounded variation budgets $B$. Our proposed \rmab\; algorithm integrates sliding window reinforcement learning (RL) with an upper confidence bound (UCB) mechanism to simultaneously learn transition dynamics and their variations. We further establish that \rmab\; achieves $\widetilde{\mathcal{O}}(N^2 B^{\frac{1}{4}} T^{\frac{3}{4}})$ regret bound by leveraging a relaxed definition of regret, providing a foundational theoretical framework for non-stationary RMAB problems for the first time.