0 2 0 3 1 3 1 2 drop 0 2 0 3 1 3 1 2 grow 0 2 2 0 1 2 3 0 1 2 3 shuffle 0 2 1 3 0 1 2 3 blocking 0 2 0 2 1 3 1 2 Before After active zero removed added

Aug-19-2025, 21:23:45 GMT–Neural Information Processing Systems 

Sparse training is a popular technique to reduce the overhead of training large models.