Markov Models
A Hardware-oriented Approach for Efficient Active Inference Computation and Deployment
Piลพurica, Nikola, Miloviฤ, Nikola, Jovanฤeviฤ, Igor, Heins, Conor, de Prado, Miguel
Active Inference (AIF) offers a robust framework for decision-making, yet its computational and memory demands pose challenges for deployment, especially in resource-constrained environments. This work presents a methodology that facilitates AIF's deployment by integrating pymdp's flexibility and efficiency with a unified, sparse, computational graph tailored for hardware-efficient execution. Our approach reduces latency by over 2x and memory by up to 35%, advancing the deployment of efficient AIF agents for real-time and embedded applications.
Estimating Convergence of Markov chains with L-Lag Couplings
Niloy Biswas, Pierre E. Jacob, Paul Vanetti
Neural Information Processing Systems http://nips.cc/
Almost Horizon-Free Structure-Aware Best Policy Identification with a Generative Model
Andrea Zanette, Mykel J. Kochenderfer, Emma Brunskill
Markov Decision Process (MDP) provided that we can access the reward and transition function through a generative model.
Combining Generative and Discriminative Models for Hybrid Inference
Victor Garcia Satorras, Zeynep Akata, Max Welling
A graphical model is a structured representation of the data generating process. The traditional method to reason over random variables is to perform inference in this graphical model. However, in many cases the generating process is only a poor approximation of the much more complex true data generating process, leading to suboptimal estimations.
Stability and Generalization for Markov Chain Stochastic Gradient Methods
There are mainly two notable types of SGMs which are inherent for different learning problems.
Learning Structure from the Ground up--Hierarchical Representation Learning by Chunking
Sequential data in our everyday life is often hierarchically structured.
Giving Feedback on Interactive Student Programs with Meta-Exploration
One approach toward automatic grading is to learn an agent that interacts with a student's program and explores states indicative of errors via reinforcement learning. However, existing work on this approach only provides binary feedback of whether a program is correct or not, while students require finer-grained feedback on the specific errors in their programs to understand their mistakes. In this work, we show that exploring to discover errors can be cast as a meta-exploration problem.
Reinforcement Learning with a Terminator Guy T ennenholtz
We present the problem of reinforcement learning with exogenous termination. We define the Termination Markov Decision Process (TerMDP), an extension of the MDP framework, in which episodes may be interrupted by an external non-Markovian observer.
Recursive Reinforcement Learning
These RL algorithms are designed with a flat Markovian view of the environment in the form of a "state, action, reward, and next state" interface [
Learning to Attack Federated Learning: A Model-based Reinforcement Learning Attack Framework
Markov processes over denumerable products of spaces, describing large systems of automata.