This section presents the implementation code of DIST, as shown in Figure 4. The purpose of these methods is to learn the similarity relationships between instances from teacher, e.g., the semantic spaces of instances with the same KD methods, which help us to achieve better performance especially when the student is trained with a stronger teacher. Here we conduct experiments to investigate the efficacy of our method with cosine similarity. As discussed in our main text, the matching functions such as KL divergence and MSE are used to match the outputs between student and teacher in KD.

We empirically find that the discrepancy of predictions between the student and a stronger teacher may tend to be fairly severer.

Our framework employs traditional regret minimizers as black-box components.

Our framework employs traditional regret minimizers as black-box components.

Despite the prevalence of recent success in learning from static graphs, learning from time-evolving graphs remains an open challenge.

In practice, to effectively acquire auxiliary knowledge from view images, we develop a teacher-student framework and formulate the cross-modal learning as a knowledge distillation problem. PointCMT eliminates the distribution discrepancy between different modalities through novel feature and classifier enhancement criteria and avoids potential negative transfer effectively.