Appendix 475 A Extended Related Work and Discussion

Although these methods are "parameter-efficient", they actually cannot reduce the The solution to the above convex problem is the distribution defined in Equation (3). Var[f (j) |j D\C ]. (16) By combining the above two inequality, we have Algorithm 2. For the ease of illustration, we ignore the sequential length. Cache is used for saving the norm of output gradient Z. end procedure F E.2 More Experimental Speed Analysis In Table 3, "Fwd", "Bwd", and "F-B" are the time of forward pass, the time of backward Latency (ms) of Forward and Backward pass. We give the detailed hyper-parameter setting in this section. The computational infrastructure information is given in Table 4.