Network partitions are inevitable in large-scale cloud systems. Despite developers’ efforts in handling network partitions throughout designing, implementing and testing cloud systems, bugs caused by network partitions, i.e., partition bugs, still exist and cause severe failures in production clusters. It is challenging to expose these partition bugs because they often require network partitions to start and stop at specific timings.

In this paper, we propose Consistency-Guided Fault Injection (CoFI), a novel technique that smartly injects network partitions to effectively expose partition bugs. We observe that, network partitions can leave cloud systems at inconsistent states, where partition bugs are more likely to occur. Based on this observation, CoFI first infers invariants (i.e., consistent states) among different nodes in a cloud system. Once observing a violation to the inferred invariants (i.e., inconsistent states) while running the cloud system, CoFI injects network partitions to prevent the cloud system from recovering back to consistent states, and thoroughly tests whether the cloud system still proceeds correctly at inconsistent states.We have applied CoFI to three widely-deployed cloud systems, i.e., Cassandra, HDFS, and YARN. CoFI has detected 7 previously-unknown bugs, and three of them have been confirmed by developers.

Haicheng Chen

The Ohio State University, USA

United States

Wensheng Dou

Institute of Software, Chinese Academy of Sciences

China

Dong Wang

Institute of software, Chinese academy of sciences

China

Feng Qin

Ohio State University, USA

In recent years the use of non-traditional computing mechanisms has grown rapidly. One paradigm uses chemical reaction networks (CRNs) to compute via chemical interactions. CRNs are used to prototype molecular devices at the nanoscale such as intelligent drug therapeutics. In practice, these programs are first written and simulated in environments such has MatLab and later compiled into physical molecules such as DNA strands. However, techniques for testing the correctness of CRNs are lacking. Current methods of validating CRNs include model checking and theorem proving, but these are limited in scalability. In this paper we present an automated testing framework, ChemTest. In contrast to model checking, ChemTest evaluates test oracles on individual simulation traces and supports functional, metamorphic, internal and hyper test cases. It also allows for flakiness and programs that are probabilistic. We performed a large case study demonstrating that ChemTest can find seeded faults and scales beyond model checking. Of our tests, 21% are inherently flaky, suggesting that systematic support for this paradigm is needed. On average functional tests find 66.5% of the faults, while metamorphic tests find 80.4%, showing the benefit of using metamorphic relationships in our test framework. In addition, we show how time of evaluation impacts fault detection.

Michael C. Gerten

Iowa State University

United States

James I. Lathrop

Iowa State University

Myra Cohen

Iowa State University

United States

Titus H. Klinge

Drake University

United States

As a common IT infrastructure, APM (Application PerformanceManagement) systems have been widely adopted to monitor call request to an on-line service. Usually, each request may contain multi-dimensional attributes(e.g., City, ISP, Platform, etc. ), which may become the reason for a certain anomaly regarding DSR (Declining Success Rate) of service calls either solely or as a combination. Moreover, each attribute may also have multiple values (e.g., ISP could be T-Mobile, Vodafone, CMCC, etc.), rendering intricate root causes and huge challenges to identify the root causes. In this paper, we propose a prototype tool, Imp APTr(Imp act Analysis based on Pruning Tree), to identify the combination of dimensional attributes as the clues leading to the root causes of anomalies regarding DSR of a service call in a timely manner. Imp APTr have been evaluated in MeiTuan, one of the biggest on-line service providers. Performance regarding the accuracy outperforms several previous tools in the same field.

hao wang

Guoping Rong

Nanjing University

Yangchen Xu

Nanjing University

Yong You

Meituan-Dianping Group

In object-oriented programming, a method is pure if calling the method does not change object states that exist in the pre-states of the method call. Pure methods are widely-used in automatic techniques, including test generation, complier optimization, and program repair. Due to the source code dependency, it is infeasible to completely and accurately identify all pure methods. Instead, existing techniques such as ReImInfer are designed to identify a subset of accurate results of pure method and mark the other methods as unknown ones. In this paper, we designed and implemented MetPurity , a learning-based tool of pure method identification. Given all methods in a project, MetPurity labels a training set via automatic program analysis and builds a binary classifier (implemented with the random forest classifier) based on the training set. This classifier is used to predict the purity for all the other methods (i.e., unknown ones) in the same project. Preliminary evaluation on four open-source Java projects shows that MetPurity can provide a list of identified pure methods with a low error rate. Applying MetPurity to EvoSuite can increase the number of killed mutants in the test generation of EvoSuite. A demo video of this tool can be found at https://youtu.be/ Ac3cmjn4CCs/; the prototype and evaluation data are available at http://cstar.whu.edu.cn/p/metpurity/.

Runze Yu

Wuhan University

Youzhe Zhang

Wuhan University

Jifeng Xuan

Wuhan University

China

Crowdsourced mobile testing has been widely used due to its convenience and high efficiency. Crowdsourced workers complete testing tasks and record results in test reports. However, the problem of duplicate reports has prevented the efficiency of crowdsourced mobile testing from further improving. Existing crowdsourced testing report analysis techniques usually leverage screenshots and text descriptions independently, but fail to recognize the link between these two types of information. In this paper, we present a crowdsourced mobile testing report selection tool, namely STIFA, to extract image and text feature information in reports and establish an image-text-fusion bug context. Based on text and image fusion analysis results, STIFA performs cluster analysis and report selection. To evaluate, we employed STIFA to analyze 150 reports from 2 apps. The results show that STIFA can extract, on average, 95.23% text feature information and 84.15% image feature information. Besides, STIFA reaches an accuracy of 87.64% in detecting duplicate reports. The demo can be found at https://github.com/ZhenfeiCao/STIFA.

Zhenfei Cao

Nanjing University

Xu Wang

Nanjing University

Shengcheng Yu

Nanjing University, China

China

Yexiao Yun

Nanjing University

Chunrong Fang

Nanjing University, China

China

Software performance testing is an essential quality assurance mechanism that can identify optimization opportunities. Automating this process requires strong tool support, especially in the case of Continuous Integration (CI) where tests need to run completely automatically and it is desirable to provide developers with actionable feedback. A lack of existing tools means that performance testing is normally left out of the scope of CI. In this paper, we propose a toolchain - PerfCI - to pave the way for developers to easily set up and carry out automated performance testing under CI. Our toolchain is based on allowing users to (1) specify performance testing tasks, (2) analyze unit tests on a variety of python projects ranging from scripts to full-blown flask-based web services, by extending a performance analysis framework (VyPR) and (3) evaluate performance data to get feedback on the code. We demonstrate the feasibility of our toolchain by using it on a web service running at the Compact Muon Solenoid (CMS) experiment at the world’s largest particle physics laboratory — CERN.

Omar Javed

Università della Svizzera italiana

Joshua Heneage Dawes

University of Manchester and CERN

Marta Han

University of Zagreb and CERN

Giovanni Franzoni

CERN

Andreas Pfeiffer

CERN

Giles Reger

University of Manchester

United Kingdom

Walter Binder

University of Lugano, Switzerland

Approaches for automatic crash reproduction aim to generate test cases that reproduce crashes starting from the crash stack traces. These tests help developers during their debugging practices. One of the most promising techniques in this research field leverages search-based software testing techniques for generating crash reproducing test cases. In this paper, we introduce Botsing, an open-source search-based crash reproduction framework for Java. Botsing implements state-of-the-art and novel approaches for crash reproduction. The well-documented architecture of Botsing makes it an easy-to-extend framework, and can hence be used for implementing new approaches to improve crash reproduction. We have applied Botsing to a wide range of crashes collected from open source systems. Furthermore, we conducted a qualitative assessment of the crash-reproducing test cases with our industrial partners. In both cases, Botsing could reproduce a notable amount of the given stack traces.

Pouria Derakhshanfar

Delft University of Technology

Xavier Devroey

Delft University of Technology

Netherlands

Annibale Panichella

Delft University of Technology

Netherlands

Andy Zaidman

Delft University of Technology

Netherlands

Arie van Deursen

Delft University of Technology, Netherlands

Netherlands

When generating GUI tests for Android apps, it typically is a separate test computer that generates interactions, which are then executed on an actual Android device. While this approach is efficient in the sense that apps and interactions execute quickly, the communication overhead between test computer and device slows down testing considerably. In this work, we present DD-2, a test generator for Android that tests other apps on the device using Android accessibility services. In our experiments, DD-2 has shown to be 3.2 times faster than its computer-device counterpart, while sharing the same source code.

Nataniel Borges Jr.

CISPA Helmholtz Center for Information Security

Germany

Jenny Rau

CISPA Helmholtz Center for Information Security

Andreas Zeller

CISPA, Germany

Germany