Empirically, we find that a variety of neural networks successfully learn sparse parities, with discontinuous phase transitions in the training curves.

Extensive experiments across 13 CNN search spaces and one NLP space demonstrate the remarkable efficiency of GenNAS using regression, in terms of both evaluating the neural architectures (quantified by the ranking correlation Spearman's

We also find that efficient ways to improve model inference ( e.g ., via adapters, lightweight networks that transform FM embeddings) do not consistently improve and can

Recent work on automated data augmentation strategies has led to state-of-the-art results in image classification and object detection. An obstacle to a large-scale adoption of these methods is that they require a separate and expensive search phase. A common way to overcome the expense of the search phase was to use a smaller proxy task.