Review for NeurIPS paper: Robust Pre-Training by Adversarial Contrastive Learning

Strengths: The paper's main idea is easy to follow: extending a recently successful contrastive learning framework SimCLR [2] to adversarial training. While SimCLR is already popular for a number of tasks, exploring its usage for adversarial defense appears to be new and original. The authors explained why SimCLR might be particularly suitable for the goal of adversarial robustness: one cause of adversarial fragility is the lack of feature invariance to small input perturbations, and SimCLR learns representations by maximizing feature invariance under differently augmented views. That makes this paper well motivated and grounded. The main technical part of this paper explores options to formulate the contrastive task.