The proposed layers lead toamore data efficient representation and areduction in computation by exploiting symmetry. We evaluate chirality nets on the task ofhuman poseregression, which naturally exploits theleft/right mirroring ofthe human body.

Models are trained with Stochastic Gradient Descent with momentum equal to 0.9 [ We use a learning rate annealing scheme, decreasing the learning rate by a factor of 0.1 every 30 epochs. We train all models for 150 epochs. Then, we select the best learning rate and weight decay for each method and run 5 different seeds to report mean and standard deviation. We use the validation set of ImageNet to perform cross-validation and report performance on it. In section G we train the Augerino method on top of the Resnet-18 architecture.

Graph contrastivelearning (GCL), by training GNNs to maximize the correspondence between the representations of the same graph in its different augmented forms, may yield robust and transferable GNNs even without using labels.

Typically, DNNs suffer from drastic performance degradation of previously learned tasksafterlearning newknowledge, which isawell-documented phenomenon, knownascatastrophic forgetting [8,9,10].

To Reviewer 1: [Intuition of benefits of advanced data augmentation] In line 198, we explained the theoretical3 connection between advanced data augmentation and better semi-supervised learning performance. We stated that4 "Importantly, the number of components is actually decided by the quality of the augmentation operation: an ideal5 augmentation should be able to reach all other examples of the same category given a starting instance.