Linear pretraining in recurrent mixture density networks

Financial time series are known for their rich dynamics, which incorporates heteroskedasticity, skewness and fat tails, among other stylized facts. For such reason, distributional forecasting of stock returns is a highly non-trivial task. Multiple models have been developed to model heteroskedastic time series. Pioneering works in such strand of literature include the development of the Autoregressive Conditional Heteroskedasticity (ARCH) model [Engle, 1982] and the Generalized Autoregressive Conditional Heteroskedasticity (GARCH) model [Bollerslev, 1986]. These simple models are able to capture the persistence in the evolution of volatility present in financial time series, and to produce associated volatility clusters. The use of Hidden Markov Models (HMM), also referred to as regime-switching models, also gained traction in the literature since the seminal work of Hamilton [1989]. Such an approach involves modeling return distributions for which parameters are modulated through a Markov chain, which therefore produces mixtures of distributions. Other papers combined the two aforementioned approaches and developed regime-switching GARCH models, see for instance Gray [1996] and Klaassen [2002], which allow for time-varying parameters in the mixture distributions, and thus for more volatility persistence. Financial support from Mitacs, Quantolio and NSERC (Godin: RGPIN-2017-06837) is gratefully acknowledged.