Uncertainty in Deep Learning (PhD Thesis) Yarin Gal - Blog Cambridge Machine Learning Group

Some of the work in the thesis was previously presented in [Gal, 2015; Gal and Ghahramani, 2015a,b,c,d; Gal et al., 2016], but the thesis contains many new pieces of work as well. There are two factors at play when visualising uncertainty in dropout Bayesian neural networks: the dropout masks and the dropout probability of the first layer. Uncertainty depictions in my previous blog posts drew new dropout masks for each test point--which is equivalent to drawing a new prediction from the predictive distribution for each test point -2 \leq \x \leq 2 . More specifically, for each test point \x_i we drew a set of network parameters from the dropout approximate posterior \boh_{i} \sim q_\theta(\bo), and conditioned on these parameters we drew a prediction from the likelihood \y_i \sim p(\y \x_i, \boh_{i}) . Another important factor affecting visualisation is the dropout probability of the first layer.