Existing approaches can be categorized into two types. The first type involves leveraging labeled source data for target-domain model selection [9,14-16]. The second type designs unsupervised metrics based on priors of the learned target-domain structure and utilizes the metrics for model selection[17,19,18,20].

Formally,this constraint also applies to special cases of Natural Evolution Strategies [37, 3]. Similar estimators can be obtained for other symmetric distributions with finite second moment. Moreover,theadditionalhyperparameter σ that determines the magnitude of the perturbation needs to be carefully chosen [33]. Figure B.7: Validation accuracy after 100 epochs and mean gradient correlation with SGD plotted against increasing subspace dimensionality d on the CIFAR-10 CNN (average of three runs). As expected, the mean cosine similarity across 100 pairs of random vectors decreases with growing dimensionality.