RAINING IN L

Formally, this constraint also applies to special cases of Natural Evolution Strategies [37, 3]. Similar estimators can be obtained for other symmetric distributions with finite second moment. B.1 Convergence behaviour of random bases training Figure B.6 provides the validation curves of different the random subspace methods for the baseline dimensionality d = 250. B.2 Approximation with growing dimensionality As discussed in Section 4.5, the performance of random bases descent improves when using a larger number of basis vectors. Figure B.7 quantifies the approximation quality in terms of achieved accuracy as well as correlation with the SGD gradient for an increasing number of base dimensions.