Researchers open-source benchmarks measuring quality of AI-generated code

The applications of computer programming are vast in scope. And as computers become ubiquitous, the demand for quality code draws an ever-growing number of aspiring programmers to the profession. After years of study to become proficient at coding, experts learn to convert abstracts into concrete, executable programs. But what if AI could do the same? In recent years, large-scale AI language models have shown promise in generalizing to tasks including writing code, implying that humans' work may be one day supplemented by AI systems.