Probabilistic linear solvers (PLSs) return probability distributions that quantify uncertainty due to limited computation in the solution of linear systems. The literature has traditionally distinguished between Bayesian PLSs, which condition a prior on information obtained from projections of the linear system, and probabilistic iterative methods (PIMs), which lift classical iterative solvers to probability space. In this work we show this dichotomy to be false: Bayesian PLSs are a special case of non-stationary affine PIMs. In addition, we prove that any realistic affine PIM is calibrated. These results motivate a focus on (non-stationary) affine PIMs, but their practical adoption has been limited by the significant manual effort required to implement them. To address this, we introduce affine tracing, an algorithmic framework that automatically constructs a PIM from a standard implementation of an affine iterative method by passing symbolic tracers through the computation to build an affine computational graph. We show how this graph can be transformed to compute posterior covariances, and how equality saturation can be used to perform algebraic simplifications required for computation under specific prior choices. We demonstrate the framework by automatically generating a probabilistic multigrid solver and evaluate its performance in the context of Gaussian process approximation.

Neural Information Processing Systems http://nips.cc/

For3 now, we report total running times on the cross-validated computational graphs, for a diverse selection of datasets.4 We will augment this description with a detailed description in the12 supplementary. Missing values: We selected k-NN imputation because it arguably provides a stronger baseline than simple mean19 imputation (while being computationally more demanding). However, using EM as an inner loop within a structure search would be computationally quite21 demanding. Determining the computational graph isfarsimpler,and can be tackled with cross-validation30 (asinthispaper), orassuggested bythereviewer using AutoML techniques orneural structural search (NAS).

However, current systems focus on specific use-cases (e.g.