Towards Data-Agnostic Pruning At Initialization: What Makes a Good Sparse Mask?

PaI methods manage to find trainable subnetworks that outperform random pruning, their performance in terms of both accuracy and computational reduction is far from satisfactory compared to post-training pruning and the understanding of PaI is missing.