Markov Models
Beyond Optimism: Exploration With Partially Observable Rewards
Exploration in reinforcement learning (RL) remains an open challenge.
A Separation in Heavy-Tailed Sampling: Gaussian vs. Stable Oracles for Proximal Samplers
This problem is particularly challenging when using gradient-based Markov Chain Monte Carlo (MCMC) algorithms due to diminishing gradients, which occurs when the tails of the target density decay at a slow (e.g.
Learning Distinguishable Trajectory Representation with Contrastive Loss
We implement CTR on top of QMIX and evaluate its performance in various cooperative multi-agent tasks.
The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains
Work done while visiting Harvard University.
benchmarks (Freeman et al., 2021) show that T A
However, various systems are inherently continuous in time, making discrete-time MDPs an inexact modeling choice. In many applications, such as greenhouse control or medical treatments, each interaction (measurement or switching of action) involves manual intervention and thus is inherently costly.
Unscrambling disease progression at scale: fast inference of event permutations with optimal transport
Disease progression models infer group-level temporal trajectories of change in patients' features as a chronic degenerative condition plays out.
Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy Cameron Allen UC Berkeley Aaron Kirtland Brown University Ruo Y u Tao
These authors contributed equally and are ordered alphabetically.
Induced Model Matching: Restricted Models Help Train Full-Featured Models
This restricted model may be thought of as "side-information", and can come
Expectation Alignment: Handling Reward Misspecification in the Presence of Expectation Mismatch
Detecting and handling misspecified objectives, such as reward functions, has been widely recognized as one of the central challenges within the domain of Artificial Intelligence (AI) safety research.
Periodic agent-state based Q-learning for POMDPs
The standard approach for Partially Observable Markov Decision Processes (POMDPs) is to convert them to a fully observed belief-state MDP . However, the belief state depends on the system model and is therefore not viable in reinforcement learning (RL) settings. A widely used alternative is to use an agent state, which is a model-free, recursively updateable function of the observation history. Examples include frame stacking and recurrent neural networks. Since the agent state is model-free, it is used to adapt standard RL algorithms to POMDPs. However, standard RL algorithms like Q-learning learn a stationary policy.