Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Prediction Appendix

In the following, we provide the Appendix as part of the supplementary material to the main paper. Section C contains additional content about the model zoos. We also provide visualizations of some of the properties of our model zoo for better intuition. Consider a common, fully-connected feed-forward neural network (FFN). Training of neural networks is defined as an optimization against a objective function on a given dataset, i.e. their weights and biases are chosen to minimize a cost function, usually called loss, denoted by Subsequent earlier layer's error are computed with δ L, (6) where β is a positive learning rate.