Markov Models
Re-examination of the Role of Latent Variables in Sequence Modeling
Guokun Lai, Zihang Dai, Yiming Yang, Shinjae Yoo
Neural Information Processing Systems http://nips.cc/
A Meta-MDP Approach to Exploration for Lifelong Reinforcement Learning
Francisco Garcia, Philip S. Thomas
Neural Information Processing Systems http://nips.cc/
Learning low-dimensional state embeddings and metastable clusters from time series data
Yifan Sun, Yaqi Duan, Hao Gong, Mengdi Wang
Markov state trajectories, where the transition kernel has a small intrinsic rank. In the spirit of diffusion map, we propose an efficient method for learning a low-dimensional state embedding and capturing the process's dynamics.
Machine Teaching of Active Sequential Learners
Tomi Peltola, Mustafa Mert รelikok, Pedram Daee, Samuel Kaski
Neural Information Processing Systems http://nips.cc/
Expressive power of tensor-network factorizations for probabilistic modeling
Ivan Glasser, Ryan Sweke, Nicola Pancotti, Jens Eisert, Ignacio Cirac
Many problems in diverse areas of computer science and physics involve constructing efficient representations of high-dimensional functions. Neural networks are a particular example of such representations that have enjoyed great empirical success, and much effort has been dedicated to understanding their expressive power - i.e. the set of functions that they can efficiently represent. Analogously, tensor networks are a class of powerful representations of high-dimensional arrays (tensors), for which a variety of algorithms and methods have been developed.
Variational Temporal Abstraction
Taesup Kim, Sungjin Ahn, Yoshua Bengio
We introduce a variational approach to learning and inference of temporally hierarchical structure and representation for sequential data.
Decoding Communications with Partial Information
Machine language acquisition is often presented as a problem of imitation learning: there exists a community of language users from which a learner observes speech acts and attempts to decode the mappings between utterances and situations. However, an interesting consideration that is typically unaddressed is partial observability, i.e. the learner is assumed to see all relevant information. This paper explores relaxing this assumption, thereby posing a more challenging setting where such information needs to be inferred from knowledge of the environment, the actions taken, and messages sent. We see several motivating examples of this problem, demonstrate how they can be solved in a toy setting, and formally explore challenges that arise in more general settings. A learning-based algorithm is then presented to perform the decoding of private information to facilitate language acquisition.
A Hardware-oriented Approach for Efficient Active Inference Computation and Deployment
Piลพurica, Nikola, Miloviฤ, Nikola, Jovanฤeviฤ, Igor, Heins, Conor, de Prado, Miguel
Active Inference (AIF) offers a robust framework for decision-making, yet its computational and memory demands pose challenges for deployment, especially in resource-constrained environments. This work presents a methodology that facilitates AIF's deployment by integrating pymdp's flexibility and efficiency with a unified, sparse, computational graph tailored for hardware-efficient execution. Our approach reduces latency by over 2x and memory by up to 35%, advancing the deployment of efficient AIF agents for real-time and embedded applications.
Estimating Convergence of Markov chains with L-Lag Couplings
Niloy Biswas, Pierre E. Jacob, Paul Vanetti
Neural Information Processing Systems http://nips.cc/
Almost Horizon-Free Structure-Aware Best Policy Identification with a Generative Model
Andrea Zanette, Mykel J. Kochenderfer, Emma Brunskill
Markov Decision Process (MDP) provided that we can access the reward and transition function through a generative model.