Whilemuchwork has focused on different weight pruning criteria, the overallsparsifiabilityofthe network, i.e., its capacity to be pruned without quality loss, has often been overlooked.

Classical methods, such as weighted cross-entropy, fail when training deep nets to the terminal phase of training (TPT), that is training beyond zero training error.