A Appendix

This is simple to see as the ranks in the uneven depthwise are computed per input and the merging is done by output. The proposed RED method is summarized in algorithm 1. Note that we didn't describe the Strategy % removed parameters linear descending 77.90 constant 78.69 linear ascending 80.35 block 84.52 The constant strategy provides the best results. Following the study from Section 5.2, we want to empirically validate that hashing a DNN RED appears to be robust to dropout.