We explore a new research direction in Bayesian variational inference with discrete latent variable priors where we exploit Kronecker matrix algebra for efficient and exact computations of the evidence lower bound (ELBO). The proposed DIRECT approach has several advantages over its predecessors; (i) it can exactly compute ELBO gradients (i.e.

Update: read the author feedback and all reviews and still agree the paper should be accepted. This paper addresses the problem of performing Bayesian inference on mobile hardware (e.g., self-driving car, phone) efficiently. As one would imagine, approaches that operate with discrete values have an advantage in hardware. Variational inference, a method for approximate Bayesian inference, often involves continuous latent variables and continuous variational parameters. This paper's contribution is to cast everything in the discrete space with an approximating discrete prior.

We explore a new research direction in Bayesian variational inference with discrete latent variable priors where we exploit Kronecker matrix algebra for efficient and exact computations of the evidence lower bound (ELBO). The proposed "DIRECT" approach has several advantages over its predecessors; (i) it can exactly compute ELBO gradients (i.e. In addition, our DIRECT models can exactly compute statistical moments of the parameterized predictive posterior without relying on Monte Carlo sampling. The DIRECT approach is not practical for all likelihoods, however, we identify a popular model structure which is practical, and demonstrate accurate inference using latent variables discretized as extremely low-precision 4-bit quantized integers. While the ELBO computations considered in the numerical studies require over 10 2352 log-likelihood evaluations, we train on datasets with over two-million points in just seconds.