Thus, Eq.(7) in the main paper can be written as: e Thus, the transformation in Eq.(7) in the main paper can be expressed as a linear combination of In this section, more experimental results of PKKD are conducted. A T [5] on ResNet-20 using CIFAR-10 dataset as shown in Tab. 1. Table 1: Compared with other methods on ResNet-20 using CIFAR-10 dataset.PKKD ANN + dropout Snapshot-KD [3] SP-KD [2] Gift-KD [4] A T [5] 92.96% 92.20% 92.33% 92.38% 92.22% 92.27% Then, we show the superiority of the proposed methods on the traditional CNN distillation. The results are shown in Tab. 2. Table 2: PKKD and KD in CNN distillation. NP' stands for using progressive or fixed teacher.