Addressing these challenges requires scale. To that end we introduce ProteinGym, a large-scale and holistic set of benchmarks specifically designed for protein fitness prediction and design.

Recent neural architecture search (NAS) frameworks have been successful in finding optimal architectures for given conditions (e.g., performance or latency).

LetS bethe independently {1,2,...,}, whereN is Thensimilar S is ˆgS( (t)) =r (t) ˆR( (t)) = 1 |S| X International [12] K. He, Confer [13] K. He, confer, 2016.

The goal of this paper is to further understand this phenomenon. In particular, we study the extent to whichtheseminal domain adaptation theory ofBen-Davidetal.(2007)explains

Figure 1: Fourteen queries used in the experiments. They do not contain personally identifiable information or offensive content. All the models are implemented in Pytorch [5] and based on the official implementation of BETAE [6]2 for a fair comparison. Forall the modules using multi-layer perceptron (MLP), we use a three-layer MLP with 1600 hidden neurons and ReLU activation. We apply dropout to the min function inCardMin and search the dropout rate in{0.05,0.10,0.15,0.20}.