Neural Information Processing Systems http://nips.cc/

In this paper we propose an implementation of backpropagation using functions with callbacks, where the forward pass is executed as a sequence of function calls, and the backward pass as a corresponding sequence of function returns. A key realization is that this technique of chaining callbacks is well known in the programming languages community as continuation-passing style (CPS) .

The supplementary material is organized as follows: A: more details on the efficient implementation; B: additional graphical illustration; C: more performance comparisons; D: additional visualization results; E: discussions. Since there are no loops in the tree, the shortest path between any two vertices is unique. To facilitate a better comprehension, we provide a detailed graphical illustration in Fig. A1 to describe In the implementation, it is unnecessary to compute as it explicitly. Figure A1: The graphical illustration of the detailed process of global affinity propagation. The experimental results are shown in Table A1.

Broadly, we expect slowdowns to be attributed to one of the following categories: 1.

Ethical questions are thus not sufficiently prominent in this work to warrant a dedicated discussion section. In general, we believe, this work will have an overall positive impact asitcan help shed light into theblack-box that isdeep learning.

Two other important techniques are mixed precision training [36] and in-place activated BatchNorm [53]. Mixed precision training involves training using both 32-bit and 16-bit IEEE floating point numbers depending onthenumerical sensitivityofdifferent layers [36].