Step-by-step guide on how to train GPT-2 on books using Google Colab

We will use Google Drive to save our checkpoints (a checkpoint is our last saved trained model). Once our trained model is saved we can load it whenever we want to generate both conditional and unconditional texts. Now that you have your Google Drive connected let's create a checkpoints folder: Now let's clone the GPT-2 repository that we will use, which is forked from nnsheperd's awesome repository (which is forked from OpenAI's but with the awesome addition of train.py), I have added a conditional_model() method which will let us pass multiple sentences at once and return a dictionary with the relevant model output samples. It also lets us avoid using bash-code.