The upper bound of gradient's Frobenius norm for spectrally-normalized discriminators follows directly. As lw(x) is a linear transformation, we have lcw(x) = c lw(x), and lw(cx) = c lw(x). Moreover, since ReLU and leaky ReLU is linear in R+ and R region, we have ai(cx) = c ai(x). In this section we discuss the gradients with respect the actual parameter wi. From Eq. (12) in [30] we know wtDθ(x) = A, we know that w0tDθ(x) F, otl(x)Dθ(x), and kotl (x)k have upper bounds. From Theorem 1.1 in [44] we know that if wt is initialized with i.i.d random variables from uniform or Gaussian distribution, E kwtkspis lower bounded away from zero at initialization. So k wtDθ(x)kF is upper bounded at initialization. Moreover, we observe empirically that kwtksp is usually increasing during training. Therefore, k wtDθ(x)kF is typically upper bounded during training as well. The following proposition states that spectral normalization also gives an upper bound on kHwi(Dθ)(x)ksp for networks with ReLU or leaky ReLU internal activations.

Graduating university represents the culmination of years of hard work for most, but for Canberra man and paraplegic Paul Jenkins it marked something much more. Mr Jenkins has spent the past six months training in an exoskeleton device and on Tuesday realised his dream of walking on stage at Parliament House and accepting his two bachelor degrees from the University of Canberra. Paul Jenkins, left paralysed by a motorcycle accident, is the first person in Australia approved under the NDIS to use a robotic walker that allows him to walk with the help of crutches. Counter-terrorism police are investigating links to terrorism in the stabbing death of a man in Queanbeyan, New South Wales. Police hunt for two teenagers after a service station worker was stabbed to death near the ACT border on Thursday night.