In unit testing, mocking is popularly used to ease test effort, reduce test flakiness, and increase test coverage by replacing the actual dependencies with simple implementations. However, there are no clear criteria to determine which dependencies in a unit test should be mocked. Inappropriate mocking can have undesirable consequences: under-mocking could result in the inability to isolate the class under test (CUT) from its dependencies while over-mocking increases the developers’ burden on maintaining the mocked objects and may lead to spurious test failures. According to existing work, various factors can determine whether a dependency should be mocked. As a result, mocking decisions are often difficult to make in practice. Studies on the evolution of mocked objects also showed that developers tend to change their mocking decisions: 17% of the studied mocked objects were introduced sometime after the test scripts were created and another 13% of the originally mocked objects eventually became unmocked. In this work, we are motivated to develop an automated technique to make mocking recommendations to facilitate unit testing. We studied 10,846 test scripts in four actively maintained open-source projects that use mocked objects, aiming to characterize the dependencies that are mocked in unit testing. Based on our observations on mocking practices, we designed and implemented a tool, MockSniffer, to identify and recommend mocks for unit tests. The tool is fully automated and requires only the CUT and its dependencies as input. It leverages machine learning techniques to make mocking recommendations by holistically considering multiple factors that can affect developers’ mocking decisions. Our evaluation of Mock- Sniffer on ten open-source projects showed that it outperformed three baseline approaches, and achieved good performance in two potential application scenarios.

Hengcheng Zhu

Southern University of Science and Technology

China

Lili Wei

The Hong Kong University of Science and Technology

Ming Wen

Huazhong University of Science and Technology, China

China

Yepang Liu

Southern University of Science and Technology

Shing-Chi Cheung

Hong Kong University of Science and Technology, China

China

Qin Sheng

WeBank Co Ltd

Cui Zhou

WeBank Co Ltd

Today, most automated test generators, such as search-based software testing (SBST) techniques focus on achieving high code coverage. However, high code coverage is not sufficient to maximise the number of bugs found, especially when given a limited testing budget. In this paper, we propose an automated test generation technique that is also guided by the estimated degree of defectiveness of the source code. Parts of the code that are likely to be more defective receive more testing budget than the less defective parts. To measure the degree of defectiveness, we leverage Schwa, a notable defect prediction technique.

We implement our approach into EvoSuite, a state of the art SBST tool for Java. Our experiments on the Defects4J benchmark demonstrate the improved efficiency of defect prediction guided test generation and confirm our hypothesis that spending more time budget on likely defective parts increases the number of bugs found in the same time budget.

Anjana Perera

Monash University

Australia

Aldeida Aleti

Monash University

Australia

Marcel Böhme

Monash University, Australia

Australia

Burak Turhan

Monash University

Australia

Crowdsourced mobile testing has been widely used due to its convenience and high efficiency. Crowdsourced workers complete testing tasks and record results in test reports. However, the problem of duplicate reports has prevented the efficiency of crowdsourced mobile testing from further improving. Existing crowdsourced testing report analysis techniques usually leverage screenshots and text descriptions independently, but fail to recognize the link between these two types of information. In this paper, we present a crowdsourced mobile testing report selection tool, namely STIFA, to extract image and text feature information in reports and establish an image-text-fusion bug context. Based on text and image fusion analysis results, STIFA performs cluster analysis and report selection. To evaluate, we employed STIFA to analyze 150 reports from 2 apps. The results show that STIFA can extract, on average, 95.23% text feature information and 84.15% image feature information. Besides, STIFA reaches an accuracy of 87.64% in detecting duplicate reports. The demo can be found at https://github.com/ZhenfeiCao/STIFA.

Zhenfei Cao

Nanjing University

Xu Wang

Nanjing University

Shengcheng Yu

Nanjing University, China

China

Yexiao Yun

Nanjing University

Chunrong Fang

Nanjing University, China

China

In object-oriented programming, a method is pure if calling the method does not change object states that exist in the pre-states of the method call. Pure methods are widely-used in automatic techniques, including test generation, complier optimization, and program repair. Due to the source code dependency, it is infeasible to completely and accurately identify all pure methods. Instead, existing techniques such as ReImInfer are designed to identify a subset of accurate results of pure method and mark the other methods as unknown ones. In this paper, we designed and implemented MetPurity , a learning-based tool of pure method identification. Given all methods in a project, MetPurity labels a training set via automatic program analysis and builds a binary classifier (implemented with the random forest classifier) based on the training set. This classifier is used to predict the purity for all the other methods (i.e., unknown ones) in the same project. Preliminary evaluation on four open-source Java projects shows that MetPurity can provide a list of identified pure methods with a low error rate. Applying MetPurity to EvoSuite can increase the number of killed mutants in the test generation of EvoSuite. A demo video of this tool can be found at https://youtu.be/ Ac3cmjn4CCs/; the prototype and evaluation data are available at http://cstar.whu.edu.cn/p/metpurity/.

Runze Yu

Wuhan University

Youzhe Zhang

Wuhan University

Jifeng Xuan

Wuhan University

China

As a common IT infrastructure, APM (Application PerformanceManagement) systems have been widely adopted to monitor call request to an on-line service. Usually, each request may contain multi-dimensional attributes(e.g., City, ISP, Platform, etc. ), which may become the reason for a certain anomaly regarding DSR (Declining Success Rate) of service calls either solely or as a combination. Moreover, each attribute may also have multiple values (e.g., ISP could be T-Mobile, Vodafone, CMCC, etc.), rendering intricate root causes and huge challenges to identify the root causes. In this paper, we propose a prototype tool, Imp APTr(Imp act Analysis based on Pruning Tree), to identify the combination of dimensional attributes as the clues leading to the root causes of anomalies regarding DSR of a service call in a timely manner. Imp APTr have been evaluated in MeiTuan, one of the biggest on-line service providers. Performance regarding the accuracy outperforms several previous tools in the same field.

hao wang

Guoping Rong

Nanjing University

Yangchen Xu

Nanjing University

Yong You

Meituan-Dianping Group

Software performance testing is an essential quality assurance mechanism that can identify optimization opportunities. Automating this process requires strong tool support, especially in the case of Continuous Integration (CI) where tests need to run completely automatically and it is desirable to provide developers with actionable feedback. A lack of existing tools means that performance testing is normally left out of the scope of CI. In this paper, we propose a toolchain - PerfCI - to pave the way for developers to easily set up and carry out automated performance testing under CI. Our toolchain is based on allowing users to (1) specify performance testing tasks, (2) analyze unit tests on a variety of python projects ranging from scripts to full-blown flask-based web services, by extending a performance analysis framework (VyPR) and (3) evaluate performance data to get feedback on the code. We demonstrate the feasibility of our toolchain by using it on a web service running at the Compact Muon Solenoid (CMS) experiment at the world’s largest particle physics laboratory — CERN.

Omar Javed

Università della Svizzera italiana

Joshua Heneage Dawes

University of Manchester and CERN

Marta Han

University of Zagreb and CERN

Giovanni Franzoni

CERN

Andreas Pfeiffer

CERN

Giles Reger

University of Manchester

United Kingdom

Walter Binder

University of Lugano, Switzerland

Approaches for automatic crash reproduction aim to generate test cases that reproduce crashes starting from the crash stack traces. These tests help developers during their debugging practices. One of the most promising techniques in this research field leverages search-based software testing techniques for generating crash reproducing test cases. In this paper, we introduce Botsing, an open-source search-based crash reproduction framework for Java. Botsing implements state-of-the-art and novel approaches for crash reproduction. The well-documented architecture of Botsing makes it an easy-to-extend framework, and can hence be used for implementing new approaches to improve crash reproduction. We have applied Botsing to a wide range of crashes collected from open source systems. Furthermore, we conducted a qualitative assessment of the crash-reproducing test cases with our industrial partners. In both cases, Botsing could reproduce a notable amount of the given stack traces.

Pouria Derakhshanfar

Delft University of Technology

Xavier Devroey

Delft University of Technology

Netherlands

Annibale Panichella

Delft University of Technology

Netherlands

Andy Zaidman

Delft University of Technology

Netherlands

Arie van Deursen

Delft University of Technology, Netherlands

Netherlands

When generating GUI tests for Android apps, it typically is a separate test computer that generates interactions, which are then executed on an actual Android device. While this approach is efficient in the sense that apps and interactions execute quickly, the communication overhead between test computer and device slows down testing considerably. In this work, we present DD-2, a test generator for Android that tests other apps on the device using Android accessibility services. In our experiments, DD-2 has shown to be 3.2 times faster than its computer-device counterpart, while sharing the same source code.

Nataniel Borges Jr.

CISPA Helmholtz Center for Information Security

Germany

Jenny Rau

CISPA Helmholtz Center for Information Security

Andreas Zeller

CISPA, Germany

Germany