Supplementary Material: Progressive Kernel Based Knowledge Distillation for Adder Neural Networks

Given two input vector x and f, the result of convolutional operation on a specific point of the image is the dot product of two vectors. Thus, Eq.(7) in the main paper can be written as: Thus, the transformation in Eq.(7) in the main paper can be expressed as a linear combination of infinite kernel functions, which means the output space is mapped to an infinite dimensional space. Also note that when n, L also goes to infinity, which means that the input space is mapped to an infinite dimensional space. In this section, more experimental results of PKKD are conducted. We compared the proposed method with other methods, such as ANN+dropout, Snapshot-KD [3], SP-KD [2], Gift-KD [4] and AT [5] on ResNet-20 using CIFAR-10 dataset as shown in Tab. 1. Table 1: Compared with other methods on ResNet-20 using CIFAR-10 dataset.