A Extraction methods

ESM-1v is pre-trained to output the probability for each possible amino acid at a masked position. At each position, we introduce a mask token and record the model's predicted In all cases, we assume an additive model when multiple mutations are present in a sequence. For example, if mutations are introduced at positions 3 and 6, then M = {3, 6}. This method performs best among the four. ESM-1v and MSA Transformer amortize compute cost into a single expensive pre-training run.