Evaluating Representation Learning of Code Changes for Predicting Patch Correctness in Program Repair
A large body of the literature of automated program repair develops approaches where patches are generated to be validated against an oracle (e.g., a test suite). Because such an oracle can be imperfect, the generated patches, although validated by the oracle, may actually be incorrect. While the state of the art explore research directions that require dynamic information or that rely on manually-crafted heuristics, we study the benefit of learning code representations in order to learn deep features that may encode the properties of patch correctness. Our empirical work mainly investigates different representation learning approaches for code changes to derive embeddings that are amenable to similarity computations. We report on findings based on embeddings produced by pre-trained and re-trained neural networks. Experimental results demonstrate the potential of embeddings to empower learning algorithms in reasoning about patch correctness: a machine learning predictor with BERT transformer-based embeddings associated with logistic regression yielded an AUC value of about 0.8 in the prediction of patch correctness on a deduplicated dataset of 1000 labeled patches. Our investigations show that learned representations can lead to reasonable performance when comparing against the state-of-the-art, PATCH-SIM, which relies on dynamic information. These representations may further be complementary to features that were carefully (manually) engineered in the literature.
Thu 24 Sep Times are displayed in time zone: (UTC) Coordinated Universal Time
08:00 - 09:00: Bugs and Automated RepairResearch Papers at Kangaroo Chair(s): Jifeng XuanWuhan University | |||
08:00 - 08:20 Talk | No Strings Attached: An Empirical Study of String-related Software Bugs Research Papers Pre-print File Attached | ||
08:20 - 08:40 Research paper | Automated Patch Correctness Assessment: How Far are We? Research Papers Shangwen WangNational University of Defense Technology, Ming WenHuazhong University of Science and Technology, China, Bo LinNational University of Defense Technology, Hongjun WuNational University of Defense Technology, Yihao QinNational University of Defense Technology, Deqing ZouHuazhong University of Science and Technology, Xiaoguang MaoNational University of Defense Technology, Hai JinHuazhong University of Science and Technology DOI Pre-print Media Attached | ||
08:40 - 09:00 Research paper | Evaluating Representation Learning of Code Changes for Predicting Patch Correctness in Program Repair Research Papers Haoye TianUniversity of Luxembourg, Kui LiuUniversity of Luxembourg, Luxembourg, Abdoul Kader KaboréUniversity of Luxembourg, Anil KoyuncuUniversity of Luxembourg, Luxembourg, Li LiMonash University, Australia, Jacques KleinUniversity of Luxembourg, Luxembourg, Tegawendé F. BissyandéUniversity of Luxembourg, Luxembourg |