Education
Structured Energy Network as a Loss Function Jay-Y oon Lee
Belanger & McCallum (2016) and Gygli et al. (2017) have shown that energy In this work, we propose Structured Energy As Loss (SEAL) to take advantage of the expressivity of energy networks without incurring the high inference cost. This raises a question: Can energy networks be used in a way that is as expressive as SPENs, as efficient at inference as feedforward approaches, and also easy to train?
Supplementary Materials for: Online Training Through Time for Spiking Neural Networks
A.3 Proof of Theorem 1 In this subsection, we prove Theorem 1 with Assumption 1. Assumption 1. l = 1,, N, t = 1,, T, diag null As described in Sections 4.1 and 4.2, for gradients of OTTT, we have Remark 2. The above conclusion mainly focuses on the gradients for connection weights Remark 3. Note that the gradients based on spike representation may also include small errors since A.4 Proof of Theorem 2 In this subsection, we prove Theorem 2. Theorem 2. If Assumption 1 holds, As described in Sections 4.1 and 4.2 and similar to the proof of Theorem 1, let Remark 4. The above conclusion considers the single-layer condition. It can be generalized to the multi-layer condition. Therefore, the conclusion can be directly generalized to these conditions as well. L} based on the gradient-based optimizer. For VGG network structures, we directly impose sWS on all weights. For more illustrations and other details, please directly refer to [4].