Reviews: Re-examination of the Role of Latent Variables in Sequence Modeling

The authors discuss the role of latent variable models in sequence models where multiple observations of the time series are modeled at once using a factorized form which assumes conditional independence. This assumption is almost surely violated in practice, thus limiting the performance of such models. When the sequence model is provided with latent variables it is possible to account for the correlation structure of the likely correlated observations within a time window, thus resulting in better performance compared to models without latent variables. Results on multiple datasets demonstrate this intuition. Though the analysis presented by the authors is clear, well motivated and justified, the paper seems to downplay the importance and motivation of sequence models that consider multiple observations at once in a windowed manner, and how sequence models with stochastic (latent) variables by their ability to capture correlation structure alleviate some of the issues associated with windowing, i.e., the conditional independence assumption.