Appendix: On the Modularity of Hypernetworks

As an additional experiment, we repeated the same experiment (i.e., varying the number of layers of The experiments with Type I functions are presented in the main text. In all of the experiments, the weights of y are set using the He uniform initialization [11]. Figure 1: (a-b) The error obtained by hypernetworks and the embedding method with varying number of layers (x-axis). For the purpose of comparison, we considered the following setting. In the rotations prediction experiment in Sec. 5, we did not apply any regularization or normalization We compared the two models in the configuration of Sec. 5 when fixing the MNIST classification with a varying number of hidden neurons.