Beta DVBF: Learning State-Space Models for Control from High Dimensional Observations

Philip Becker-Ehmck philip.becker-ehmck@argmax.ai Patrick van der Smagt Disclosure: Parts of this work have been submitted in form of a Master's Thesis towards partial fulfillment of the requirements for a Masters program at the Technical University of Munich [4] Abstract Learning a model of dynamics from high-dimensional images can be a core ingredient for success in many applications across different domains, especially in sequential decision making. However, currently prevailing methods based on latent-variable models are limited to working with low resolution images only. In this work, we show that some of the issues with using high-dimensional observations arise from the discrepancy between the dimensionality of the latent and observable space, and propose solutions to overcome them. 1 Introduction Learning a probabilistic model for sequential data is a key step towards solving a lot of interesting problems, including analysis and deconstruction of auditory sequences [18], predicting the next piece of information given previously recorded data such as video frames [9] and text [20], and controlling an agent to perform specific tasks (model-based reinforcement learning [5]). While in this paper, we consider probabilistic models especially tuned for the needs of the last, i.e with control inputs, the approaches we discuss can be applied to control-less environments as well. A key feature required in control-based models is that they should be able to generate a feasible trajectory distribution given a control policy. To this end, one of the more successful solutions proposed in the past for modeling dynamical systems is Deep V ariational Bayes Filter ( DVBF) [10] - a framework for learning a State-Space Model given sequential observations from the environment in an unsupervised manner.