A deep learning (DL) model is inherently imprecise. To fix this problem, existing techniques retrain a given DL model over a larger training dataset or with the help of fault injected models or using the insight of failing test cases in the given DL model. In this paper, we present Apricot, a novel weight-adaptation approach to fixing DL models iteratively. Our key observation is that if the deep learning architecture of a DL model is trained over a subset of the original training dataset, the weights in the resultant reduced DL model (rDLM) can provide insights on the adjustment direction and magnitude of the weights in the original DL model to handle the test cases that the original DL model misclassify. Apricot generates a set of such reduced DL models from the original DL model. In each iteration, for each weight of the input DL model (iDLM) of that iteration, Apricot adjusts the weight of this iDLM toward the average weight of these rDLMs correctly classifying the failing test cases experienced by this iDLM and/or away from these rDLMs misclassifying the same failing test cases, followed by training the weight-adjusted iDLM over the original training dataset to generate a new iDLM for the next iteration. The experiment using five state-of-the-art DLMs shows that Apricot can increase the accuracy of these DL models by 0.35%-2.81% with an average of 1.45%. The experiment also reveals the complementary nature of these rDLMs in Apricot.

Hao Zhang

City University of Hong Kong

Wing-Kwong Chan

City University of Hong Kong, Hong Kong

China

Automated program repair has been used to provide feedback for incorrect student programming assignments, since program repair captures the code modification needed to make a given buggy program pass a given test-suite. Existing student feedback generation techniques are limited because they either require manual effort in the form of providing an error model, or require a large number of correct student submissions to learn from, or suffer from lack of scalability and accuracy.

In this work, we propose a fully automated approach for generating student program repairs in real-time. This is achieved by first re-factoring all available correct solutions to semantically equivalent solutions. Given an incorrect program, we match the program with the closest matching refactored program based on its control flow structure. Subsequently, we infer the input-output specifications of the incorrect program’s basic blocks from the executions of the correct program’s aligned basic blocks. Finally, these specifications are used to modify the blocks of the incorrect program via search-based synthesis.

Our dataset consists of almost 1,800 real-life incorrect Python program submissions from 361 students for an introductory programming course at a large public university. Our experimental results suggest that our method is more effective and efficient than recently proposed feedback generation approaches. About 30% of the patches produced by our tool Refactory are smaller than those produced by the state-of-art tool Clara, and can be produced given fewer correct solutions (often a single correct solution) and in a shorter time. We opine that our method is applicable not only to programming assignments, and could be seen as a general-purpose program repair method that can achieve good results with just a single correct reference solution.

Yang Hu

The University of Texas at Austin

Umair Z. Ahmed

National University of Singapore

Singapore

Sergey Mechtaev

University College London

Ben Leong

National University of Singapore

Abhik Roychoudhury

National University of Singapore

Singapore

This paper presents InFix, a technique for automatically fixing erroneous program inputs for novice programmers. Unlike comparable existing approaches for automatic debugging and maintenance tasks, InFix repairs input data rather than source code, does not require test cases, and does not require special annotations. Instead, we take advantage of patterns commonly used by novice programmers to automatically create helpful, high quality input repairs. InFix iteratively applies error-message based templates and random mutations based on insights about the debugging behavior of novices. This paper presents an implementation of InFix for Python. We evaluate on 29,995 unique scenarios with input-related errors collected from four years of data from Python Tutor, a free online programming tutoring environment. Our results generalize and scale; compared to previous work, we consider an order of magnitude more unique programs. Overall, InFix is able to repair 94.5% of deterministic input errors. We also present the results of a human study with 97 participants. Surprisingly, this simple approach produces high quality repairs; humans judged the output of InFix to be equally helpful and within 4% of the quality of human-generated repairs.

Madeline Endres

University of Michigan

Georgios Sakkas

University of California, San Diego

Benjamin Cosman

University of California at San Diego, USA

Ranjit Jhala

University of California, San Diego

United States

Westley Weimer

University of Michigan

United States

This article contributes to defining the design space of program repair. Repair approaches can be loosely characterized according to the main design philosophy, in particular “generate-and-validate” and synthesis-based approaches. Each of those repair approaches is a point in the design space of program repair. Our goal is to facilitate the design, development and evaluation of repair approaches by providing a framework that: a) contains components commonly present in most approaches, b) provides built-in implementations of existing repair approaches. This paper presents a Java framework named Astor that focuses on the design space of generate-and-validate repair approaches. The key novelty of Astor is to provide explicit extension points to explore the design space of program repair. Thanks to those extension points, researchers can both reuse existing program repair components and implement new ones. Astor includes 6 unique implementations of repair approaches in Java, including GenProg for Java called jGenProg. Researchers have already defined new approaches over Astor. The implementations of program repair approaches built already available in Astor are capable of repairing, in total, 98 real bugs from 5 large Java programs. Astor code is publicly available on Github: https://github.com/SpoonLabs/astor.

Matias Martinez

Université Polytechnique Hauts-de-France

Martin Monperrus

KTH Royal Institute of Technology

Sweden

Automated program repair (APR) is one of the recent advances in automated software engineering aiming for reducing the burden of debugging by suggesting high-quality patches that either directly fix the bugs, or help the programmers in the course of manual debugging. We believe scalability, applicability, and accurate patch validation are the main design objectives for a practical APR technique. In this paper, we present PraPR, our implementation of a practical APR technique that operates at the level of JVM bytecode. We discuss design decisions made in the development of PraPR, and argue that the technique is a viable baseline toward attaining aforementioned objectives. PraPR is publicly available at https://github.com/prapr/prapr.

Ali Ghanbari

The University of Texas at Dallas

United States

Lingming Zhang

The University of Texas at Dallas

United States

Developer trust is a major barrier to the deployment of automatically-generated patches. Understanding the effect of a patch is a key element of that trust. We find that differences in sets of formal invariants characterize patch differences and that implication-based distances in invariant space characterize patch similarities. When one patch is similar to another it often contains the same changes as well as additional behavior; this pattern is well-captured by logical implication. We can measure differences using a theorem prover to verify implications between invariants implied by separate programs. Although effective, theorem provers are computationally intensive, but we find that string distance is an efficient heuristic for our implication-based distance measurements. We propose to use distances between patches to construct a hierarchy highlighting patch similarities. We evaluate our approach on over 300 patches and find that it correctly categorizes programs into semantically similar clusters. Clustering programs reduces human effort by reducing the number of semantically distinct patches that must be considered by over 50%, thus reducing the time required to establish trust in automatically generated repairs.

Padraic Cashin

Arizona State University

Cari Martinez

University of New Mexico

Stephanie Forrest

Arizona State University

Westley Weimer

University of Michigan

United States

Recent trends in Web development demonstrate an increased interest in serverless applications, i.e. applications that utilize computational resources provided by cloud services on demand instead of requiring traditional server management. This approach enables better resource management while being scalable, reliable, and cost-effective. However, it comes with a number of organizational and technical difficulties which stem from the interaction between the application and the cloud infrastructure, for example, having to set up a recurring task of reuploading updated files. In this paper, we present Kotless – a Kotlin Serverless Framework. Kotless is a cloud-agnostic toolkit that solves these problems by interweaving the deployed application into the cloud infrastructure and automatically generating the necessary deployment code. This relieves developers from having to spend their time integrating and managing their applications instead of developing them. Kotless has proven its capabilities and has been used to develop several serverless applications already in production. Its source code is available at https://github.com/JetBrains/kotless, a tool demo can be found at https://www.youtube.com/watch?v=IMSakPNl3TY.

Vladislav Tankov

JetBrains, ITMO University

Yaroslav Golubev

JetBrains Research, ITMO University

Timofey Bryksin

JetBrains Research, Saint-Petersburg State University

We present PeASS (Performance Analysis of Software System versions), a tool for detecting performance changes at source code level that occur between different code versions. By using PeASS, it is possible to identify performance regressions that happened in the past to fix them.

PeASS measures the performance of unit tests in different source code versions. To achieve statistic rigor, measurements are repeated and analyzed using an agnostic t-test. To execute a minimal amount of tests, PeASS uses a regression test selection.

We evaluate PeASS on a selection of Apache Commons projects and show that 81% of all unit test covered performance changes can be found by PeASS. A video presentation is available at https://www.youtube.com/watch?v=RORFEGSCh6Y and PeASS can be downloaded from https://github.com/DaGeRe/peass.

David Georg Reichelt

Universität Leipzig

Germany

Stefan Kühne

Universität Leipzig

Wilhelm Hasselbring

Kiel University

Germany

The amount of android applications is having a tremendous increasing trend, exerting pressure over practitioners and researchers around application quality, frequent releases, and quick fixing of bugs. This pressure leads practitioners to make use of automated approaches. However, most of those use source-code as input. Nevertheless, third-party services are not able to use this approaches due to privacy factors. In this paper we present MutAPK, an open source mutation testing tool that enables the usage of APKs as input for this task. MutAPK generates mutants without the need of having access to source code, because the mutations are done in an intermediate representation of the code (i.e., SMALI) that does not require compilation. MutAPK is publicly available at GitHub https://bit.ly/2KYvgP9 VIDEO: https://bit.ly/2WOjiyy

Camilo Escobar-Velásquez

Universidad de los Andes

Colombia

Michael Osorio-Riaño

Universidad de los Andes

Mario Linares-Vásquez

Systems and Computing Engineering Department , Universidad de los Andes , Bogotá, Colombia

Colombia

Coding convention plays an important role in guaranteeing software quality. However, coding conventions are usually informally presented and inconvenient for programmers to use. In this paper, we present CocoQa, a system that answers programmer’s questions about coding conventions. CocoQa answers questions by querying a knowledge graph for coding conventions. It employs 1) a subgraph matching algorithm that parses the question into a SPARQL query, and 2) a machine comprehension algorithm that uses an end-to-end neural network to detect answers from searched paragraphs. We have imple- mented CocoQa, and evaluated it on a coding convention QA dataset. The results show that CocoQa can answer questions about coding conventions precisely. In particular, CocoQa can achieve a precision of 82.92% and a recall of 91.10%. Repository: https://github.com/14dtj/CocoQa/ Video: https://youtu.be/VQaXi1WydAU

Tianjiao Du

Shanghai JiaoTong University

Junming Cao

Shanghai JiaoTong University

Qinyue Wu

Shanghai JiaoTong University

Wei Li

Shanghai JiaoTong University

Beijun Shen

School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University

Yuting Chen

Shanghai Jiao Tong University

China

Automated input generators must constantly choose which UI element to interact with and how to interact with it, in order to achieve high coverage with a limited time budget. Currently, most black-box input generators adopt pseudo-random or brute-force searching strategies, which may take very long to find the correct combination of inputs that can drive the app into new and important states. We propose Humanoid, a deep learning-based approach to automated black-box Android app testing, which can explore the app more efficiently. The key technique behind Humanoid is a deep neural network model that can learn how human users choose actions based on an app’s GUI from human interaction traces . The learned model can be used to guide test input generation to achieve higher coverage. Experiments on both open-source apps and market apps demonstrate that Humanoid is able to reach higher coverage, and faster as well, than the state-of-the-art test input generators. Humanoid is open-sourced at https://github.com/yzygitzh/Humanoid and a demo video can be found at https://youtu.be/PDRxDrkyORs.

Yuanchun Li

Peking University

Ziyue Yang

Peking University

Yao Guo

Peking University

China

Xiangqun Chen

Peking University

Evidence shows that developer reputation is extremely important when accepting pull requests or resolving reported issues. It is particularly salient in Free/Libre Open Source Software since the developers are distributed around the world, do not work for the same organization and, in most cases, never meet face to face. The existing solutions to expose developer reputation tend to be forge specific (GitHub), focus on activity instead of impact, do not leverage social or technical networks, and do not correct often misspelled developer identities. We aim to remedy this by amalgamating data from all public Git repositories, measuring the impact of developer work, expose developer’s collaborators, and correct notoriously problematic developer identity data. We leverage World of Code (WoC), a collection of an almost complete (and continuously updated) set of Git repositories by first allowing developers to select which of the 34 million(M) Git commit author IDs belong to them and then generating their profiles by treating the selected collection of IDs as that single developer. As a side-effect, these selections serve as a training set for a supervised learning algorithm that merges multiple identity strings belonging to a single individual. As we evaluate the tool and the proposed impact measure, we expect to build on these findings to develop reputation badges that could be associated with pull requests and commits so developers could easier trust and prioritize them.

Sadika Amreen

University of Tennessee Knoxville

Andrey Karnauch

University of Tennessee Knoxville

Audris Mockus

University of Tennessee - Knoxville

United States

Deep Neural Network(DNN) techniques have been prevalent in software engineering. They are employed to facilitate various software engineering tasks and embedded into many software applications. However, because DNNs are built upon a rich data-driven programming paradigm that employs plenty of labeled data to train a set of neurons to construct the internal system logic, analyzing and understanding their behaviors becomes a difficult task for software engineers. In this paper, we present an instance-based visualization tool for DNN, namely NeuralVis, to support software engineers in visualizing and interpreting deep learning models. NeuralVis is designed for: 1). visualizing the structure of DNN models, i.e., neurons, layers, as well as connections; 2). visualizing the data transformation process; 3). integrating existing adversarial attack algorithms for test input generation; 4). comparing intermediate outputs of different instances. To demonstrate the effectiveness of NeuralVis, we design a task-based user study involving ten participants on two classic DNN models, i.e., LeNet and VGG-12. The result shows NeuralVis can assist engineers in identifying critical features that determine the prediction results. Tool is available at http://118.178.18.181:18080. Video: https://youtu.be/hRxCovrOZFI.

Xufan Zhang

State Key Laboratory for Novel Software Technology Nanjing University, Nanjing, China

Ziyue Yin

State Key Laboratory for Novel Software Technology Nanjing University, Nanjing, China

Yang Feng

University of California, Irvine

United States

Qingkai Shi

Hong Kong University of Science and Technology

China

Jia Liu

State Key Laboratory for Novel Software Technology Nanjing University, Nanjing, China

Zhenyu Chen

Nanjing University

China

Analyzing executions of concurrent software is very difficult. Even if a trace is available, such traces are very hard to read and interpret. A textual trace contains a lot of data, most of which is not relevant to the issue at hand. Past visualization attempts either do not show concurrent behavior, or result in a view that is overwhelming for the user.

We provide a visual analytics tool, VA4JVM, for error traces produced by either the Java Virtual Machine, or by Java Pathfinder. Its key features are a layout that spatially associates events with threads, a zoom function, and the ability to filter event data in various ways. We show in examples how filtering and zooming in can highlight a problem without having to read lengthy textual data.

Cyrille Artho

KTH Royal Institute of Technology, Sweden

Monali Pande

KTH Royal Institute of Technology

Qiyi Tang

University of Oxford

United Kingdom

This paper presents Sip4J, a fully automated, scalable and effective tool to automatically generate access permission contracts for a sequential Java program. The access permission contracts, which represent the dependency of code blocks, have been frequently used to enable concurrent execution of sequential programs. Those permission contracts, unfortunately, need to be manually created by programmers, which is known to be time- consuming, laborious and error-prone. To mitigate those manual efforts, Sip4J performs inter-procedural static analysis of Java source code to automatically extract the implicit dependencies in the program and subsequently leverages them to automatically generate access permission contracts, following the Design by Contract principle. The inferred specifications are then used to identify the concurrent (immutable) methods in the program. Experimental results further show that Sip4J is useful and effective towards generating access permission contracts for sequential Java programs.

Ayesha Sadiq

Monash University

Li Li

Monash University, Australia

Australia

Yuan-Fang Li

Monash University

Australia

Ijaz Ahmed

University of Lahore

Sea Ling

Monash University

To detect specific types of bugs and vulnerabilities, static analysis tools must be correctly configured with security-relevant methods (SRM), e.g., sources, sinks, sanitizers and authentication methods—usually a very labour-intensive and error-prone process. This work presents the semi-automated tool SWAN_ASSIST, which aids the configuration with an IntelliJ plugin based on active machine learning. It integrates our novel automated machine-learning approach SWAN, which identifies and classifies Java SRM. SWAN_ASSIST further integrates user feedback through iterative learning. SWAN_ASSIST aids developers by asking them to classify at each point in time exactly those methods whose classification best impact the classification result. Our experiments show that SWAN_ASSIST classifies SRM with a high precision, and requires a relatively low effort from the user. A video demo of SWAN_ASSIST can be found at https://youtu.be/fSyD3V6EQOY. The source code is available at https://github.com/secure-software-engineering/swan.

Goran Piskachev

Fraunhofer IEM

Germany

Lisa Nguyen Quang Do

Google

Switzerland

Oshando Johnson

Fraunhofer IEM

Eric Bodden

Heinz Nixdorf Institut, Paderborn University and Fraunhofer IEM

Fuzzing is widely used for vulnerability detection. One of the challenges for an efficient fuzzing is covering code guarded by constraints such as the magic number and nested conditions. Recently, academia has partially addressed the challenge via whitebox methods. However, high-level constraints such as array sorts, virtual function invocations, and tree set queries are yet to be handled.

To meet this end, we present VisFuzz, an interactive tool for better understanding and intervening fuzzing process via real-time visualization. It extracts call graph and control flow graph from source code, maps each function and basic block to the line of source code and tracks real-time execution statistics with detail constraint contexts. With VisFuzz, test engineers first locate blocking constraints and then learn its semantic context, which helps to craft targeted inputs or update test drivers. Preliminary evaluations are conducted on four real-world programs in Google fuzzer-test-suite. Given additional 15 minutes to understand and intervene the state of fuzzing, the intervened fuzzing outperform the original pure AFL fuzzing, and the path coverage improvements range from 10.84% to 150.58%, equally fuzzed by for 12 hours.

Chijin Zhou

Tsinghua University

Mingzhe Wang

Tsinghua University

Jie Liang

Tsinghua University

Zhe Liu

Nanjing University of Aeronautics and Astronautics

Chengnian Sun

Waterloo University

Yu Jiang

Tsinghua University