This is the appendix of the paper " Quantification of Uncertainty with Adversarial Models ". It consists of three sections.

Quantifying uncertainty is important for actionable predictions in real-world applications.

Despite its theoretical appeal, this viewpoint lacks a crucial ingredient of deep learning in finite DNNs, laying at the heart of their success --feature learning. Here we consider DNNs trained with noisy gradient descent on a large training set and derive a self-consistent Gaussian Process theory accounting forstrongfinite-DNN and feature learning effects.

We introduce a semidefinite programming hierarchy to estimate the global and local Lipschitz constant of a multiple layer deep neural network.

Several machine learning applications involve the optimization of higher-order derivatives(e.g., gradients ofgradients) during training, which can beexpensive with respect to memory and computation even with automatic differentiation.