In many learning problems, the domain scientist is often interested in discovering thegroups offeatures that areredundant and areimportant forclassification.

Gradient NormPreserving (GNP) architectures where each layer preserves the gradient norm during backpropagation. For 1-Lipschitz Convolutional Neural Networks (CNNs), this involves using orthogonal convolutions (convolution layers with an orthogonal Jacobian matrix) [Li et al., 2019b, Trockman and Kolter,

With the top-ranked PTM candidates, we further learn to enrich task repr.