The power of constrained language models.

Pre-trained generative language models (such as OpenAI's GPT2 and GPT3) or seq2seq models (such as T5 or the recently released T0) generate free-flowing natural language. This entails that their output sentences can have any shape. To get the most value out of these models, we sometimes want the outputs to follow a certain structure -- this is called a constrained language model. After reading this post, you will be able to gain more value out of your language model by controlling the shape of its outputs. This post relates to my previous post, where I explain how to unlock the true potential of GPT3 by manipulating the shape of its inputs (referred to prompt engineering).