TrAct: Making First-layer Pre-Activations Trainable

We consider the learning of first-layer embeddings / pre-activations in vision models, and in particular learning the weights with which the input images are transformed in order to obtain these embeddings.