The full modeling code, written in Mesh TensorFlow and designed to be run on TPUs. Optimizer states, which allow you to continue training the model from where EleutherAI left off. A Google Colab notebook that shows you how to use the code base to train, fine-tune, and sample from a model." Their notebook requires a Google storage bucket to access their data since TPUs can't be read from local file systems. You can set up a free-trial fairly easily, they provide a link in the notebook.
Mar-29-2021, 07:55:12 GMT