ThinkBig, TeachSmall: DoLanguageModelsDistilOccam'sRazor?

Large language models have recently shown a remarkable ability for few-shot learning, including patterns of algorithmic nature. However, it is still an open question to determine what kind of patterns these models can capture and how manyexamples theyneedintheirprompts.