From TensorFlow to PyTorch

In this post, you'll learn the main recipe to convert a pretrained TensorFlow model in a pretrained PyTorch model, in just a few hours. We'll take the example of a simple architecture like OpenAI GPT-2 Doing such a conversion assumes a good familiarity with both TensorFlow and PyTorch but it's also one of the best ways to get to know better both frameworks! The first step is to retrieve the TensorFlow code and a pretrained checkpoint. Let's get them from OpenAI GPT-2 official repository: TensorFlow checkpoints are usually composed of three files named XXX.ckpt.data-YYY A trained NLP model should also be provided with a vocabulary to associate the tokens to the embeddings indices (here encoder.json