DominoSearch: Find layer-wise fine-grained N: M sparse schemes from dense neural networks - Supplementary Material

Section 2: Experimental study of a different policy with fixed N and flexible M. Section 3: Sensitivity of hyper-parameter β In the main paper, we assume a policy with fixed M and flexible N. Furthermore, we also use a design space with N equal to a power-of-two. This is achieved by transforming the schemes of fixed M. For instance, 8:16, 4:16, 2:16 and 1:16 will be transformed as 1:2, 1:4, 1:8 and 1:16 with fixed N (1) and flexible M (2,4,8,16). Results are shown in Table 3. Figure 1 and 2 illustrate the differences between 1:2 and 2:4 with the same dense weight matrix and sparsity (i.e. Details can be found in Section 3.4 of the main paper. It consists of more than 1.2 million training images and Each image is labelled as one of 1K classes.