Appendices

And, for each of them, the second (final) stripe has 44 options. It could seem that small improvements in efficacy may have only a minor effect on final network accuracy, especially considering the noisiness inherent in large-scale training. Better thanreducing themagnitude oflostweights, though, iscompletely eliminating it - by using the zeros already present in the unstructured sparse weight matrix, it may be possible to find a permutation that does notloseanymagnitude after applying theN:M constraint.