Goto

Collaborating Authors

 Technology





ImprovedFeature

Neural Information Processing Systems

In this section, we further investigate the effectiveness of the proposed method when the feature dimensions of the student and teacher are different. In our experiments, we find that simply initializing different projectors with different seeds and the default initialization method of linear layer in Pytorch is sufficient to yield good performance. Therefore, we stick to this strategy to make the proposed method as simple as possible. Experimentalresults showthatmixing differentinitialization methods hasaslightimpact ontheperformance and is a potential way to further improve the distillation performance. We can see that the training times and memory usages of our method will slightly increase with theincrease ofthenumber ofprojectors.


4ec0b6648bdf487a2f1c815924339022-Paper-Conference.pdf

Neural Information Processing Systems

In knowledge distillation, previous feature distillation methods mainly focus on the design of loss functions and the selection of the distilled layers, while the effectofthefeatureprojector between thestudent andtheteacher remains underexplored.


Japan to revise economic security law to support projects abroad

The Japan Times

The government plans to submit a bill to revise the economic security promotion law during the current session of parliament that began on Wednesday. The Japanese government plans to revise the economic security promotion law to support companies with economic security-linked projects overseas. This will be the first revision of the law, established in 2022. The move comes amid a rapidly changing international environment, as the Ukraine-Russia war drags on and China continues to flex its economic muscle. Competition is also intensifying in the development of artificial intelligence and other cutting-edge technologies.