Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learning

Due to the promising future of Automated Program Repair (APR), researchers have proposed various APR techniques, including heuristic-based, template-based, and constraint-based techniques. Among such classic APR techniques, template-based techniques have been widely recognized as state of the art. However, such template-based techniques require predefined templates to perform repair, and their effectiveness is thus limited. To this end, researchers leveraged the recent advances in Deep Learning to further improve APR. Such learning-based techniques view APR as a Neural Machine Translation problem, using the buggy/fixed code snippets as the source/target languages for translation. In this way, such techniques heavily rely on large numbers of high-quality bug-fixing commits, which can be extremely costly and challenging to construct. Furthermore, the edit variety of these learning-based techniques are limited to the available bug-fixes within their training datasets. Therefore, in this paper, we aim to revisit the learning-based APR problem, and propose AlphaRepair, to leverage zero-shot learning directly using large pre-trained code models for APR. Our main insight is instead of modeling what a repair edit should look like, we can directly predict what the correct code is based on the context information. We have implemented AlphaRepair as a practical multilingual APR tool based on the recent CodeBERT model. Our results on the widely used Defects4J benchmark show that AlphaRepair can substantially outperform state-of-the-art APR tools. We also studied the impact of different design choices and show that AlphaRepair performs even better on a newer version of Defects4J (2.0) with 3.3X more fixes than best performing baseline, indicating that AlphaRepair can potentially avoid the dataset-overfitting issue of existing learning-based techniques.