A Appendix

A.1 ImageNet Model Details We use the following models from Torchvision (details at https://pytorch.org/ From the CLIP family [37], we use 33 models formed by weight-space interpolation between zero-shot and finetuned ViT-B/16, ViT-B/32, and ViT-L/14 (11 models from each architecture). A.2 Pruning methods In Section 4.1 we present results considering three categories of model pruning used in Diffenderfer et al. [8]: traditional (fine-tuning [16], gradual magnitude pruning [57]), rewinding lottery-tickets (weight-rewinding [12], learning-rate rewinding [42]), and initialization lottery-tickets (edgepopup [39], biprop [7]). Here we provide a brief description of each pruning method. As the name suggests, fine-tuning prunes a model at the end of the regular training period by removing p% of the weights with the smallest magnitude, then fine tunes the remaining weights using the learning rate at the end of the regular training period.