A Appendix

The majority of the appendix is devoted to a faithful listing of hyperparameters, datasets, and training settings for all of our experiments. A.1 Addendum on Proposition 3 and Choice of Activation Function We begin with a proof of Proposition 3, which we rewrite here for convenience: Proposition 3: We then calculate f ( P)= k ( P 0 . The third claim is proven. Although Proposition 3 was proven for a specific family of activation functions (i.e. Then the following must be true: 1. E [ L We note that the variance claims in both Proposition 3 and Corollary 3.1 are relatively simple extensions of the intuitive result that the variance of a random variable that can take on only two values is maximized when the two values each have a probability weight of As described in Section 3.2, it is necessary to develop a version of GradDrop that operates nontrivially The issue we need to resolve is that these gradients are dependent on their batch's input values, so just summing gradients across the batch dimension is not an option.