Review for NeurIPS paper: Improving Auto-Augment via Augmentation-Wise Weight Sharing

Summary and Contributions: POST REBUTTAL: After reading the authors' response and discussing with other reviewers, I decided to keep my score. In this paper, the authors propose Augmentation-Wise Weight Sharing (AWS), a weight sharing strategy to efficiently search for data augmentation operations in image classification. AWS is based on a simple observation: data augmentation is more effective later in the training process, rather than earlier. Thus, AWS trains a single model for a while, and then only performs data augmentation search for a few last epochs, all starting from the trained shared weights. I think this is such a simple and elegant observation.