Review for NeurIPS paper: What Makes for Good Views for Contrastive Learning?

Weaknesses: • In line 121, the optimal z* is derived based on the fact that the downstream task is known (Line 145-146). However, for a pretrained model that can served for multiple downstream tasks, the optimal z* for one of the tasks might not be closed to optimal for another task. In other word, the z* selected from the proposed method might have worse generalizability than the z learning by standard contrastive learning. A clear example of this is the result from Table 1. To demonstrate the fact that optimal views depend on the downstream task, the author constructs a toy experiments as shown in Table 1. This experiment exposes the fact that when only selecting specific views for specific downstream task, the learned representation cannot generalize well to other tasks.