How to compress a neural network

Modern state-of-the-art neural network architectures are HUGE. For instance, you have probably heard about GPT-3, OpenAI's newest revolutionary NLP model, capable of writing poetry and interactive storytelling. Well, GPT-3 has around 175 billion parameters. To give you a perspective about how large this number is, consider the following. A $100 bill is approximately 6.14 inches wide.