OpenAI Codex shows the limits of large language models

This article is part of our reviews of AI research papers, a series of posts that explore the latest findings in artificial intelligence. In a new paper, researchers at OpenAI have revealed details about Codex, a deep learning model that generates software source code. Codex powers Copilot, an "AI pair programmer" tool developed jointly by OpenAI and GitHub. Copilot is currently available in beta test mode to a limited number of users. The paper is a fascinating read that explains the process through which the scientists at OpenAI managed to repurpose their flagship language model GPT-3 to create Codex.