Markov Models
Large-Scale Stochastic Sampling from the Probability Simplex
Jack Baker, Paul Fearnhead, Emily Fox, Christopher Nemeth
Neural Information Processing Systems http://nips.cc/
Modeling Dynamic Missingness of Implicit Feedback for Recommendation
Menghan Wang, Mingming Gong, Xiaolin Zheng, Kun Zhang
Collaborative filtering methods based on implicit feedback (e.g., purchase records and browsing history) are widely used in recommender systems. Compared to explicit feedback (e.g., 1-5 star ratings), implicit feedback is more abundant and accessible in real-world applications. However, the missing data of implicit feedback also brings two challenges.
Policy Regret in Repeated Games
Raman Arora, Michael Dinitz, Teodor Vanislavov Marinov, Mehryar Mohri
Neural Information Processing Systems http://nips.cc/
Combining Generative and Discriminative Models for Hybrid Inference
Victor Garcia Satorras, Zeynep Akata, Max Welling
Neural Information Processing Systems http://nips.cc/
GroundedAnswersforMulti-agentDecision-making ProblemthroughGenerativeWorldModel
Theempirical results demonstrate that this framework can improve the answers for multi-agent decision-making problems by showing superior performance on the training and unseen tasks of the StarCraft Multi-Agent Challenge benchmark.
BehaveNet: nonlinear embedding and Bayesian neural decoding of behavioral videos
Eleanor Batty, Matthew Whiteway, Shreya Saxena, Dan Biderman, Taiga Abe, Simon Musall, Winthrop Gillis, Jeffrey Markowitz, Anne Churchland, John P. Cunningham, Sandeep R. Datta, Scott Linderman, Liam Paninski
Neural Information Processing Systems http://nips.cc/
Reconciling ฮป-Returns with Experience Replay
Brett Daley, Christopher Amato
A unique benefit to this approach is that each transition's TD error can be
Learning to Play With Intrinsically-Motivated, Self-Aware Agents
Nick Haber, Damian Mrowca, Stephanie Wang, Li F. Fei-Fei, Daniel L. Yamins
Neural Information Processing Systems http://nips.cc/
State Regularized Policy Optimization on Data with Dynamics Shift
We then demonstrate a lower-bound performance guarantee on policies regularized by the stationary state distribution. In practice, SRPO can be an add-on module to context-based algorithms in both online and offline RL settings.
Provable Offline Reinforcement Learning for Structured Cyclic MDPs
Lee, Kyungbok, Sarteau, Angelica Cristello, Kosorok, Michael R.
We introduce a novel cyclic Markov decision process (MDP) framework for multi-step decision problems with heterogeneous stage-specific dynamics, transitions, and discount factors across the cycle. In this setting, offline learning is challenging: optimizing a policy at any stage shifts the state distributions of subsequent stages, propagating mismatch across the cycle. To address this, we propose a modular structural framework that decomposes the cyclic process into stage-wise sub-problems. While generally applicable, we instantiate this principle as CycleFQI, an extension of fitted Q-iteration enabling theoretical analysis and interpretation. It uses a vector of stage-specific Q-functions, tailored to each stage, to capture within-stage sequences and transitions between stages. This modular design enables partial control, allowing some stages to be optimized while others follow predefined policies. We establish finite-sample suboptimality error bounds and derive global convergence rates under Besov regularity, demonstrating that CycleFQI mitigates the curse of dimensionality compared to monolithic baselines. Additionally, we propose a sieve-based method for asymptotic inference of optimal policy values under a margin condition. Experiments on simulated and real-world Type 1 Diabetes data sets demonstrate CycleFQI's effectiveness.