Collaborating Authors


ChatGPT can write code. Now researchers say it's good at fixing bugs too


OpenAI's ChatGPT chatbot can fix software bugs very well, but its key advantage over other methods and AI models is its unique ability for dialogue with humans that allows it to improve the correctness of an answer. Researchers from Johannes Gutenberg University Mainz and University College London pitted OpenAI's ChatGPT against "standard automated program repair techniques" and two deep learning approaches to program repairs: CoCoNut, from researchers at the University of Waterloo, Canada; and Codex, OpenAI's GPT-3 based model that underpins GitHub's Copilot paired programming auto code completion service. "We find that ChatGPT's bug fixing performance is competitive to the common deep learning approaches CoCoNut and Codex and notably better than the results reported for the standard program repair approaches," the researchers write in a new arXiv paper, first spotted by New Scientist. That ChatGPT can solve coding problems isn't new, but the researchers highlight that its unique capacity for dialogue with humans gives it a potential edge over other approaches and models. The researchers tested ChatGPT's performance using the QuixBugs bug fixing benchmark.