AI-assisted coding: Experiments with GPT-4

Recent developments in artificial intelligence, particularly through large language models, have enabled the automated generation of computer code (Chen et al. 2021; Bubeck et al. 2023). In particular, GPT-4 has enabled human-level performance on a set of coding challenges that are outside of the training set of the model (Bubeck et al. 2023). In addition, automated coding assistants (particularly Github Copilot) have become integrated into commmon devlopment environments and are widely used, with some evidence that they can signficantly improve coding productivity. The performance of these models is also raising important questions regarding coding education, given that the current models can easily complete most coding problem sets using in introductory programming courses (Finnie-Ansley et al. 2022). In the present paper we explore some of the implications of AI-assisted coding using GPT-4, in a more qualitative way than previous benchmarking assessments. First we examine the experience of interactive coding and debugging using the ChatGPT interface to GPT-4 on a set of data science coding problems.