learning low-dimensional state
Learning low-dimensional state embeddings and metastable clusters from time series data
Yifan Sun, Yaqi Duan, Hao Gong, Mengdi Wang
Neural Information Processing Systems http://nips.cc/
Learning low-dimensional state embeddings and metastable clusters from time series data
This paper studies how to find compact state embeddings from high-dimensional Markov state trajectories, where the transition kernel has a small intrinsic rank. In the spirit of diffusion map, we propose an efficient method for learning a low-dimensional state embedding and capturing the process's dynamics. This idea also leads to a kernel reshaping method for more accurate nonparametric estimation of the transition function. State embedding can be used to cluster states into metastable sets, thereby identifying the slow dynamics. Sharp statistical error bounds and misclassification rate are proved. Experiment on a simulated dynamical system shows that the state clustering method indeed reveals metastable structures. We also experiment with time series generated by layers of a Deep-Q-Network when playing an Atari game. The embedding method identifies game states to be similar if they share similar future events, even though their raw data are far different.
Learning low-dimensional state embeddings and metastable clusters from time series data
Yifan Sun, Yaqi Duan, Hao Gong, Mengdi Wang
Markov state trajectories, where the transition kernel has a small intrinsic rank. In the spirit of diffusion map, we propose an efficient method for learning a low-dimensional state embedding and capturing the process's dynamics.
Reviews: Learning low-dimensional state embeddings and metastable clusters from time series data
This appears to be a detailed, carefully written, and theoretically/empirically supported piece of work. The results are based on RKHS theory and appear to extend the state-of-the-art in a number of ways, including learning KMEs while dealing with dependent samples (e.g. from a Markov chain), low dimensional/rank approximations with state-of-the-art error bounds, and a variational approach for clustering, as well as a robust set of experiments. On the whole, I am satisfied that this clearly meets the acceptance criteria for NeurIPS and I am recommending it for acceptance. I have a few comments below on aspects of the paper, where things were either unclear, or might benefit from a revisit. It is unclear how strong assumption 2 is.
Learning low-dimensional state embeddings and metastable clusters from time series data
This paper studies how to find compact state embeddings from high-dimensional Markov state trajectories, where the transition kernel has a small intrinsic rank. In the spirit of diffusion map, we propose an efficient method for learning a low-dimensional state embedding and capturing the process's dynamics. This idea also leads to a kernel reshaping method for more accurate nonparametric estimation of the transition function. State embedding can be used to cluster states into metastable sets, thereby identifying the slow dynamics. Sharp statistical error bounds and misclassification rate are proved.
Learning low-dimensional state embeddings and metastable clusters from time series data
Sun, Yifan, Duan, Yaqi, Gong, Hao, Wang, Mengdi
This paper studies how to find compact state embeddings from high-dimensional Markov state trajectories, where the transition kernel has a small intrinsic rank. In the spirit of diffusion map, we propose an efficient method for learning a low-dimensional state embedding and capturing the process's dynamics. This idea also leads to a kernel reshaping method for more accurate nonparametric estimation of the transition function. State embedding can be used to cluster states into metastable sets, thereby identifying the slow dynamics. Sharp statistical error bounds and misclassification rate are proved.