The Annotated GPT-2

Welcome to "The Annotated GPT-2". One of the most brilliant and well-explained articles I have ever read is The Annotated Transformer. It introduced Attention like no other post ever written. The simple idea was to present an "annotated" version of the paper Attention is all you need along with code. Something I have come to realize with my little experience in Machine Learning, when you write things in code, the implementation and the secrets become clearer. It is not magic anymore.