A deep learning (DL) model is inherently imprecise. To fix this problem, existing techniques retrain a given DL model over a larger training dataset or with the help of fault injected models or using the insight of failing test cases in the given DL model. In this paper, we present Apricot, a novel weight-adaptation approach to fixing DL models iteratively. Our key observation is that if the deep learning architecture of a DL model is trained over a subset of the original training dataset, the weights in the resultant reduced DL model (rDLM) can provide insights on the adjustment direction and magnitude of the weights in the original DL model to handle the test cases that the original DL model misclassify. Apricot generates a set of such reduced DL models from the original DL model. In each iteration, for each weight of the input DL model (iDLM) of that iteration, Apricot adjusts the weight of this iDLM toward the average weight of these rDLMs correctly classifying the failing test cases experienced by this iDLM and/or away from these rDLMs misclassifying the same failing test cases, followed by training the weight-adjusted iDLM over the original training dataset to generate a new iDLM for the next iteration. The experiment using five state-of-the-art DLMs shows that Apricot can increase the accuracy of these DL models by 0.35%-2.81% with an average of 1.45%. The experiment also reveals the complementary nature of these rDLMs in Apricot.

Hao Zhang

City University of Hong Kong

Wing-Kwong Chan

City University of Hong Kong, Hong Kong

China

Automated program repair has been used to provide feedback for incorrect student programming assignments, since program repair captures the code modification needed to make a given buggy program pass a given test-suite. Existing student feedback generation techniques are limited because they either require manual effort in the form of providing an error model, or require a large number of correct student submissions to learn from, or suffer from lack of scalability and accuracy.

In this work, we propose a fully automated approach for generating student program repairs in real-time. This is achieved by first re-factoring all available correct solutions to semantically equivalent solutions. Given an incorrect program, we match the program with the closest matching refactored program based on its control flow structure. Subsequently, we infer the input-output specifications of the incorrect program’s basic blocks from the executions of the correct program’s aligned basic blocks. Finally, these specifications are used to modify the blocks of the incorrect program via search-based synthesis.

Our dataset consists of almost 1,800 real-life incorrect Python program submissions from 361 students for an introductory programming course at a large public university. Our experimental results suggest that our method is more effective and efficient than recently proposed feedback generation approaches. About 30% of the patches produced by our tool Refactory are smaller than those produced by the state-of-art tool Clara, and can be produced given fewer correct solutions (often a single correct solution) and in a shorter time. We opine that our method is applicable not only to programming assignments, and could be seen as a general-purpose program repair method that can achieve good results with just a single correct reference solution.

Yang Hu

The University of Texas at Austin

Umair Z. Ahmed

National University of Singapore

Singapore

Sergey Mechtaev

University College London

Ben Leong

National University of Singapore

Abhik Roychoudhury

National University of Singapore

Singapore

This paper presents InFix, a technique for automatically fixing erroneous program inputs for novice programmers. Unlike comparable existing approaches for automatic debugging and maintenance tasks, InFix repairs input data rather than source code, does not require test cases, and does not require special annotations. Instead, we take advantage of patterns commonly used by novice programmers to automatically create helpful, high quality input repairs. InFix iteratively applies error-message based templates and random mutations based on insights about the debugging behavior of novices. This paper presents an implementation of InFix for Python. We evaluate on 29,995 unique scenarios with input-related errors collected from four years of data from Python Tutor, a free online programming tutoring environment. Our results generalize and scale; compared to previous work, we consider an order of magnitude more unique programs. Overall, InFix is able to repair 94.5% of deterministic input errors. We also present the results of a human study with 97 participants. Surprisingly, this simple approach produces high quality repairs; humans judged the output of InFix to be equally helpful and within 4% of the quality of human-generated repairs.

Madeline Endres

University of Michigan

Georgios Sakkas

University of California, San Diego

Benjamin Cosman

University of California at San Diego, USA

Ranjit Jhala

University of California, San Diego

United States

Westley Weimer

University of Michigan

United States

This article contributes to defining the design space of program repair. Repair approaches can be loosely characterized according to the main design philosophy, in particular “generate-and-validate” and synthesis-based approaches. Each of those repair approaches is a point in the design space of program repair. Our goal is to facilitate the design, development and evaluation of repair approaches by providing a framework that: a) contains components commonly present in most approaches, b) provides built-in implementations of existing repair approaches. This paper presents a Java framework named Astor that focuses on the design space of generate-and-validate repair approaches. The key novelty of Astor is to provide explicit extension points to explore the design space of program repair. Thanks to those extension points, researchers can both reuse existing program repair components and implement new ones. Astor includes 6 unique implementations of repair approaches in Java, including GenProg for Java called jGenProg. Researchers have already defined new approaches over Astor. The implementations of program repair approaches built already available in Astor are capable of repairing, in total, 98 real bugs from 5 large Java programs. Astor code is publicly available on Github: https://github.com/SpoonLabs/astor.

Matias Martinez

Université Polytechnique Hauts-de-France

Martin Monperrus

KTH Royal Institute of Technology

Sweden

Automated program repair (APR) is one of the recent advances in automated software engineering aiming for reducing the burden of debugging by suggesting high-quality patches that either directly fix the bugs, or help the programmers in the course of manual debugging. We believe scalability, applicability, and accurate patch validation are the main design objectives for a practical APR technique. In this paper, we present PraPR, our implementation of a practical APR technique that operates at the level of JVM bytecode. We discuss design decisions made in the development of PraPR, and argue that the technique is a viable baseline toward attaining aforementioned objectives. PraPR is publicly available at https://github.com/prapr/prapr.

Ali Ghanbari

The University of Texas at Dallas

United States

Lingming Zhang

The University of Texas at Dallas

United States

Developer trust is a major barrier to the deployment of automatically-generated patches. Understanding the effect of a patch is a key element of that trust. We find that differences in sets of formal invariants characterize patch differences and that implication-based distances in invariant space characterize patch similarities. When one patch is similar to another it often contains the same changes as well as additional behavior; this pattern is well-captured by logical implication. We can measure differences using a theorem prover to verify implications between invariants implied by separate programs. Although effective, theorem provers are computationally intensive, but we find that string distance is an efficient heuristic for our implication-based distance measurements. We propose to use distances between patches to construct a hierarchy highlighting patch similarities. We evaluate our approach on over 300 patches and find that it correctly categorizes programs into semantically similar clusters. Clustering programs reduces human effort by reducing the number of semantically distinct patches that must be considered by over 50%, thus reducing the time required to establish trust in automatically generated repairs.

Padraic Cashin

Arizona State University

Cari Martinez

University of New Mexico

Stephanie Forrest

Arizona State University

Westley Weimer

University of Michigan

United States