Automated test case generation tools have been successfully pro- posed to reduce the amount of human and infrastructure resources required to write and run test cases. However, recent studies demonstrate that the readability of generated tests is very limited due to (i) uninformative identifiers and (ii) lack of proper documentation. Prior studies proposed techniques to improve test readability by either generating natural language summaries or meaningful methods names. While these approaches are shown to improve test readability, they are also affected by two limitations: (1) generated summaries are often perceived as too verbose and redundant by developers, and (2) readable tests require both proper method names but also meaningful identifiers (within-method readability). In this work, we combine template based methods and Deep Learning (DL) approaches to automatically generate test case scenarios (elicited from natural language patterns of test case statements) as well as to train DL models on path-based representations of source code to generate meaningful identifier names. Our ap- proach, called DeepTC-Enhancer , recommends documentation and identifier names with the ultimate goal of enhancing readability of automatically generated test cases. An empirical evaluation with 36 external and internal developers shows that (1) DeepTC-Enhancer outperforms significantly the baseline approach for generating summaries and performs equally with the baseline approach for test case renaming, (2) the transformation proposed by DeepTC-Enhancer result in a significant increase in readability of automatically generated test cases, and (3) there is a significant difference in the feature preferences between external and internal developers.

Devjeet Roy

Washington State University

Ziyi Zhang

Washington State University

Maggie Ma

Washington State University

Venera Arnaoudova

Washington State University

United States

Annibale Panichella

Delft University of Technology

Netherlands

Sebastiano Panichella

Zurich University of Applied Sciences

Switzerland

Danielle Gonzalez

Rochester Institute of Technology, USA

United States

Mehdi Mirakhorli

Rochester Institute of Technology

United States

Specification mining, in general, and inferring behavior model of a running system, in particular, are quite useful for several automated software engineering tasks, such as program comprehension, anomaly detection, and testing. Most existing dynamic model inference techniques are white-box, i.e., they require source code to be instrumented to get run-time traces. However, in many systems, access to source code is not possible for parts of the program that use third-party binaries and off-the-shelf-components. One useful scenario for automated black-box behaviour inference is in software control units (such as autopilots), where the software system’s reactions over time changes based on the inputs. Run-time state models of such systems are very powerful means for anomaly detection and debugging. Unfortunately, most black-box techniques that detect state changes over time are either uni-variate (which is limiting the application in real-world systems) or are weak with respect to learning from past behaviour. Therefore, in this paper, we propose a hybrid deep neural network that accepts as input a set of time series, one per input signal of the system, and applies a set of convolution and recurrent layers to both learn the non-linear correlations between signals and the patterns over time. We have applied our approach to a real UAV auto-pilot solution from our industry partner with half a million lines of C code. We ran 888 random recent test cases of the system and inferred states over time. We compared our results with several traditional time series change point detection techniques and showed that our approach can improve their performance 88% to 102%, in terms of finding state change points, measured by F1 score. We also showed that our state classification algorithm provides on average 90.45% F1 score, which improves traditional classification algorithms 7% to 17%.

Mohammad Jafar Mashhadi

University of Calgary

Canada

Hadi Hemmati

University of Calgary

Canada

This paper investigates the problem of classifying Android applications into malicious and benign. We analyze the performance of a popular malware detection tool, Drebin, on malware datasets commonly used in an academic setup and show that the high detection accuracy often stems from learning benign rather than malicious indicators. That, effectively, turns the malware detection tools into benign app detectors. Yet, in practice, malware samples are often larger and can exhibit many behaviors similar to those of benign apps. Under such a challenging setup, looking for benign indicators becomes ineffective and the ability of the tools to detect malware degrades substantially.

In this paper, we propose an approach for identifying malicious portions of an app in the presence of numerous benign features, effectively eliminating “noise” and focusing the detection on truly malicious indicators.We also propose a novel metric estimating the “reasons” for correct malware classification, i.e., whether it is based on the presence of malicious indicators or the absence of benign ones. We show that our proposed approach is effective in both increasing the “standard” classification accuracy and in making more “justifiable” classification decisions.

Michael Cao

The University of British Columbia

Sahar Badihi

University of British Columbia, Canada

Khaled Ahmed

The University of British Columbia

Peiyu Xiong

The University of British Columbia

Julia Rubin

University of British Columbia, Canada

Canada