Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Rerouting