Autoregressive Quantile Flows for Predictive Uncertainty Estimation

Numerous applications of machine learning involve representing probability distributions over high-dimensional data. We propose autoregressive quantile flows, a flexible class of normalizing flow models trained using a novel objective based on proper scoring rules. Our objective does not require calculating computationally expensive determinants of Jacobians during training and supports new types of neural architectures, such as neural autoregressive flows from which it is easy to sample. We leverage these models in quantile flow regression, an approach that parameterizes predictive conditional distributions with flows, resulting in improved probabilistic predictions on tasks such as time series forecasting and object detection. Our novel objective functions and neural flow parameterizations also yield improvements on popular generation and density estimation tasks, and represent a step beyond maximum likelihood learning of flows. Reasoning about uncertainty via the language of probability is important in many application domains of machine learning, including medicine (Saria, 2018), robotics (Chua et al., 2018; Buckman et al., 2018), and operations research (Van Roy et al., 1997). Especially important is the estimation of predictive uncertainties (e.g., confidence intervals around forecasts) in tasks such as clinical diagnosis (Jiang et al., 2012) or decision support systems (Werling et al., 2015; Kuleshov and Liang, 2015). Normalizing flows (Rezende and Mohamed, 2016; Papamakarios et al., 2019; Kingma et al., 2016) are a popular framework for defining probabilistic models, and can be used for density estimation (Papamakarios et al., 2017), out-of-distribution detection (Nalisnick et al., 2019), content generation (Kingma and Dhariwal, 2018), and more. Flows feature tractable posterior inference and maximum likelihood estimation; however, maximum likelihood estimation of flows requires carefully designing a family of bijective functions that are simultaneously expressive and whose Jacobian has a tractable determinant.