Advancing Model Pruning via Bi-level Optimization

As illustrated by the Lottery Ticket Hypothesis (L TH), pruning also has the potential of improving their generalization ability. At the core of L TH, iterative magnitude pruning (IMP) is the predominant pruning method to successfully find'winning tickets'. Y et, the computation cost of IMP grows prohibitively as the targeted pruning ratio increases. To reduce the computation overhead, various efficient'one-shot' pruning methods have been developed but these schemes are usually