The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning