Markov Models
Practical Two-Step Lookahead Bayesian Optimization
Neural Information Processing Systems http://nips.cc/
Deep Temporal Sigmoid Belief Networks for Sequence Modeling
Zhe Gan, Chunyuan Li, Ricardo Henao, David E. Carlson, Lawrence Carin
Deep dynamic generative models are developed to learn sequential dependencies in time-series data. The multi-layered model is designed by constructing a hierarchy of temporal sigmoid belief networks (TSBNs), defined as a sequential stack of sigmoid belief networks (SBNs). Each SBN has a contextual hidden state, inherited from the previous SBNs in the sequence, and is used to regulate its hidden bias. Scalable learning and inference algorithms are derived by introducing a recognition model that yields fast sampling from the variational posterior. This recognition model is trained jointly with the generative model, by maximizing its variational lower bound on the log-likelihood. Experimental results on bouncing balls, polyphonic music, motion capture, and text streams show that the proposed approach achieves state-of-the-art predictive performance, and has the capacity to synthesize various sequences.
Integrating Markov processes with structural causal modeling enables counterfactual inference in complex systems
Robert Ness, Kaushal Paneri, Olga Vitek
An important aspect of studying these systems is predicting the equilibrium behavior of the system upon an intervention, and selecting high-value interventions.
Compact task representations as a normative model for higher-order brain activity
Higher-order brain areas such as the frontal cortices are considered essential for the flexible solution of tasks. However, the precise computational role of these areas is still debated.
Learning Graph Structure With A Finite-State Automaton Layer
As a motivating example, consider a dataset of source code samples.
Learning Dynamic Belief Graphs to Generalize on Text-Based Games
Playing text-based games requires skills in processing natural language and sequential decision making.
Toward the Fundamental Limits of Imitation Learning
Imitation learning (IL) aims to mimic the behavior of an expert policy in a sequential decision-making problem given only demonstrations.
Mixing Time Estimation in Reversible Markov Chains from a Single Sample Path
Daniel J. Hsu, Aryeh Kontorovich, Csaba Szepesvari
The interval is computed from a single finite-length sample path from the Markov chain, and does not require the knowledge of any parameters of the chain. This stands in contrast to previous approaches, which either only provide point estimates, or require a reset mechanism, or additional prior knowledge.