Human-In-The-Loop Automatic Program Repair

--We introduce L EARN2 FIX, the first human-in-the-loop, semiautomatic repair technique when no bug oracle-except for the user who is reporting the bug-is available. Our approach negotiates with the user the condition under which the bug is observed. Only when a budget of queries to the user is exhausted, it attempts to repair the bug. A query can be thought of as the following question: "When executing this alternative test input, the program produces the following output; is the bug observed"? Through systematic queries, L EARN2 FIX trains an automatic bug oracle that becomes increasingly more accurate in predicting the user's response. Our key challenge is to maximize the oracle's accuracy in predicting which tests are bug-revealing given a small budget of queries. From the alternative tests that were labeled by the user, test-driven automatic repair produces the patch. Our experiments demonstrate that L EARN2 FIX learns a sufficiently accurate automatic oracle with a reasonably low labeling effort (lt. Given L EARN2 FIX's test suite, the GenProg test-driven repair tool produces a higher-quality patch (i.e., passing a larger proportion of validation tests) than using manual test suites provided with the repair benchmark. I NTRODUCTION Automatic program repair (APR) [1], [2] holds the promise of automating the tedious, manual task of patching bugs. In their seminal paper, Le Goues and colleagues [3] demonstrated that APR is both feasible and cost-effective even at the scale of several million lines of code. Given a failing test suite, APR changes the buggy program such that all test cases pass. However, what if no such test suite is available? Suppose, a user reports a bug and provides a test input to reproduce the bug. We envision a semiautomatic approach that keeps the human-in-the-loop and negotiates the condition under which the bug is observed before repairing the bug. Strategically, the user is asked: " F or this other input, the program produces that output; is the bug observed "? While the user might not have the expertise to understand the source code or to produce a patch, it seems reasonable to ask to distinguish expected from unexpected program behavior. Iteratively, an automatic bug oracle is trained to predict the user's responses with increasing accuracy. Using the trained oracle, the user can be asked more strategically.