Pinpointing the location where a given unit of functionality—or feature—was implemented is a demanding and time-consuming task, yet prevalent in most software maintenance or evolution efforts. To that extent, we present PANGOLIN, an Eclipse plugin that helps developers identifying features among the source code. It borrows Spectrum-based Fault Localization techniques from the software diagnosis research field by framing feature localization as a diagnostic problem. PANGOLIN prompts users to label system executions based on feature involvement, and subsequently presents its spectrum-based feature localization analysis to users with the aid of a color-coded, hierarchic, and navigable visualization which was shown to be effective at conveying diagnostic information to users. Our evaluation shows that PANGOLIN accurately pinpoints feature implementations and is resilient to misclassifications by users.

Bruno Miguel Sotto-Mayor de Castro Machado

IST, University of Lisbon

Alexandre Perez

Palo Alto Research Center

Rui Abreu

Instituto Superior Técnico, U. Lisboa & INESC-ID

Portugal

Recurrent neural network (RNN) has achieved great success in processing sequential inputs for applications such as automatic speech recognition, natural language processing and machine translation. However, quality and reliability issues of RNNs make them vulnerable to adversarial attacks and hinder their deployment in real-world applications. In this paper, we propose a quantitative analysis framework — DeepStellar — to pave the way for effective security and quality analysis of software systems powered by RNNs. DeepStellar is generic to handle various RNN architectures, including LSTM and GRU, scalable to work on industrial-grade RNN models, and extensible to develop customized analyzers and tools. We demonstrated that, with DeepStellar, users are able to design efficient test generation tools, and develop effective adversarial sample detectors. We evaluated the developed applications on three real RNN models, including speech recognition and image classification.
DeepStellar outperforms existing approaches three hundred times in generating defect-triggering tests and achieves 97% accuracy in detecting adversarial attacks. A video demonstration which shows the main features of DeepStellar is available at https://sites.google.com/view/deepstellar/home/tool-demo.

Xiaoning Du

Nanyang Technological University

Xiaofei Xie

Nanyang Technological University

Yi Li

Nanyang Technological University

Singapore

Lei Ma

Kyushu University

Japan

Yang Liu

Nanyang Technological University, Singapore

Singapore

Jianjun Zhao

Kyushu University

Japan

Misuse of APIs happens frequently due to misunderstanding of API semantics and lack of documentation. An important category of API-related defects is the error handling defects, which may result in security and reliability flaws. These defects can be detected with the help of static program analysis, provided that error specifications are known. The error specification of an API function indicates how the function can fail. Writing error specifications manually is time-consuming and tedious. Therefore, automatic inferring the error specification from API usage code is preferred. In this paper, we present Ares, a tool for automatically inferring error specifications for C code through static analysis. we employ multiple heuristics to identify error handling blocks and infer error specifications by analyzing the corresponding condition logic. Ares is evaluated on 19 real world projects, and the results reveal that Ares outperforms the state-of-the-art tool APEx by 37% in precision. Ares can also identify more error specifications than APEx. Moreover, the specifications inferred from Ares help find dozens of API-related bugs in well-known projects such as OpenSSL, among them 10 bugs are confirmed by developers. Video: https://youtu.be/nf1QnFAmu8Q. Repository: https://github.com/lc3412/Ares.

Li Chi

Tsinghua University

Zuxing Gu

School of Software, Tsinghua University

China

Min Zhou

Tsinghua University

Ming Gu

Tsinghua University

Hongyu Zhang

The University of Newcastle

Australia

Modern software often exists in many different, yet similar versions and/or variants, usually derived from a common code base (e.g., via clone-and-own). In the context of product- line engineering, family-based-analysis has shown very promising potential for improving efficiency in applying quality-assurance techniques to variant-rich software, as compared to a variant- by-variant approach. Unfortunately, these strategies rely on an product-line representation superimposing all program variants in a syntactically well-formed, semantically sound and variant- preserving manner, which is manually hard to obtain in practice. We demonstrate the SiMPOSE methodology for automatically generating superimpositions of N given program versions and/or variants facilitating family-based analysis of variant-rich soft- ware. SiMPOSE is based on a novel N-way model-merging tech- nique operating at the level of control-flow automata (CFA) rep- resentations of C programs, CFAs constitute a unified program abstraction utilized by many recent software-analysis tools. We illustrate different merging strategies supported by SiMPOSE, namely variant-by-variant, N-way merging, incremental 2-way merging, and partition-based N/2-way merging, and demonstrate how SiMPOSE can be used to systematically compare their impact on efficiency and effectiveness of family-based unit-test generation. The SiMPOSE tool, the demonstration of its usage as well as related artifacts and documentation can be found at http://pi.informatik.uni-siegen.de/projects/variance/simpose.

Dennis Reuling

Software Engineering Group, University of Siegen

Germany

Udo Kelter

Software Engineering Group, University of Siegen

Sebastian Ruland

TU Darmstadt, Real-time Systems Lab

Malte Lochau

TU Darmstadt

Verification of programs continues to be a challenge and no single technique succeeds on all programs. In this paper we present VeriAbs, a reachability C program verifier that incorporates a portfolio of techniques implemented as four strategies, where a strategy is a set of techniques applied in a specific sequence. It selects a strategy based on the kind of loops in the program. We analysed the effectiveness of implemented strategies on the 3831 verification tasks from the ReachSafety category of the 8th International Competition on Software Verification (SV-COMP) 2019 and found that although simple classic techniques - explicit state model checking and bounded model checking, succeed on a majority of the programs, a wide range of further techniques are required to analyse the rest. A screencast of the tool is available at https://youtu.be/YHxB2V4U0gQ.

Mohammad Afzal

Tata Cosultancy Services

A Asia

Tata Cosultancy Services

Avriti Chauhan

Tata Cosultancy Services

Bharti Chimdyalwar

Tata Consultancy Services

Priyanka Darke

Tata Consultancy Services

Advaita Datar

Tata Consultancy Services Ltd

Shrawan Kumar

Tata Cosultancy Services

R Venkatesh

Tata Research Development and Design Centre

Deep neural network (DNN) has been widely applied to safety-critical scenarios such as autonomous vehicle, security surveillance, and cyber-physical control systems. Yet, the incorrect behaviors of DNNs can lead to severe accidents and tremendous losses due to hidden defects. In this paper, we present DeepHunter, a general-purpose fuzzing framework for detecting defects of DNNs. DeepHunter is inspired by traditional grey-box fuzzing and aims to increase the overall test coverage by applying adaptive heuristics according to runtime feedback. Specifically, DeepHunter provides a series of seed selection strategies, metamorphic mutation strategies, and testing criteria customized to DNN testing; all these components support multiple built-in configurations which are easy to extend. We evaluated DeepHunter on two popular datasets and the results demonstrate the effectiveness of DeepHunter in achieving coverage increase and detecting real defects. A video demonstration which showcases the main features of DeepHunter can be found at https://youtu.be/s5DfLErcgrc.

Xiaofei Xie

Nanyang Technological University

Hongxu Chen

Nanyang Technological University

Yi Li

Nanyang Technological University

Singapore

Lei Ma

Kyushu University

Japan

Yang Liu

Nanyang Technological University, Singapore

Singapore

Jianjun Zhao

Kyushu University

Japan

Smart pointers are widely used to prevent memory errors in modern C++ code. However, improper usage of smart pointers may also lead to common memory errors, which makes the code not as safe as expected. To avoid smart pointer errors as early as possible, we present a coding style checker to detect possible bad smart pointer usages during compile time, and notify programmers about bug-prone behaviors. The evaluation indicates that the currently available state-of-the-art static code checkers can only detect 25 out of 116 manually inserted errors, while our tool can detect all these errors. And we also found 521 bugs among 8 open source projects with only 4 false positives.

Xutong Ma

Institute of Software, Chinese Academy of Sciences

Jiwei Yan

Institute of Software, Chinese Academy of Sciences

Yaqi Li

Institute of Software, Chinese Academy of Sciences

Jun Yan

Institute of Software, Chinese Academy of Sciences

Jian Zhang

Institute of Software, Chinese Academy of Sciences

The fragmentation issue spreads over multiple mobile platforms such as Android, iOS, mobile web and even WeChat, which hinders test scripts from running across platforms. To reduce the cost of adapting scripts for various platforms, some existing tools apply conventional computer vision techniques to replay the same script on multiple platforms automatically. However, because these solutions can hardly identify dynamic or similar widgets, it becomes difficult for engineers to apply them in practice. In this paper, we present an image-driven tool, namely LIRAT, to record and replay test scripts cross platforms, solving the problem of test script cross-platform replay for the first time. LIRAT records screenshots and layouts of the widgets, and leverages image understanding techniques to locate them in the replay process. Based on accurate widget localization, LIRAT supports replaying test scripts across devices and platforms. We employed LIRAT to replay 25 scripts from 5 application across 8 Android devices and 2 iOS devices. The results show that LIRAT can replay 88% scripts on Android platforms and 60% on iOS platforms.

Shengcheng Yu

Nanjing University, China

China

Chunrong Fang

Nanjing University

Yang Feng

University of California, Irvine

United States

Wenyuan Zhao

Nanjing University

Zhenyu Chen

Nanjing University

China

Workflow underlies most process automation software, such as those for product lines, business processes, and scientific computing. However, current Cloud Computing based workflow systems cannot support real-time applications due to network latency, which limits their application in many IoT systems such as smart healthcare and smart traffic. Fog Computing extends the Cloud by providing virtualized computing resources close to the End Devices so that the response time of accessing computing resources can be reduced significantly. However, how to most effectively manage heterogeneous resources and different computing tasks in the Fog is a big challenge. In this paper, we introduce “FogWorkflowSim” an efficient and extensible toolkit for automatically evaluating resource and task management strategies in Fog Computing with simulated user-defined workflow applications. Specifically, FogWorkflowSim is able to: 1) automatically set up a simulated Fog Computing environment for workflow applications; 2) automatically execute user submitted workflow applications; 3) automatically evaluate and compare the performance of different computation offloading and task scheduling strategies with three basic performance metrics, including time, energy and cost. FogWorkflowSim can serve as an effective experimental platform for researchers in Fog based workflow systems as well as practitioners interested in adopting Fog Computing and workflow systems for their new software projects.

Xiao Liu

School of Information Technology, Deakin University

Australia

Lingmin Fan

School of Computer Science and Technology, Anhui University

Jia Xu

School of Computer Science and Technology, Anhui University

Xuejun Li

School of Computer Science and Technology, Anhui University

Lina Gong

School of Computer Science and Technology, Anhui University

John Grundy

Monash University

Australia

Yun Yang

Swinburne University of Technology

Spreadsheets are widely used but subject to various defects. In this paper, we present SGUARD to effectively detect spreadsheet defects. SGUARD learns spreadsheet features to cluster cells with similar computational semantics, and then refine these clusters to recognize anomalous cells as defects. SGUARD well balances the trade-off between the precision (87.8%) and recall rate (71.9%) in the defect detection, and achieves an F-measure of 0.79, exceeding existing spreadsheet defect detection techniques. We introduce the SGUARD implementation and its usage by a video presentation (https://youtu.be/gNPmMvQVf5Q), and provide its download repository for public access (https://github.com/sheetguard/sguard).

Da Li

State Key Lab. for Novel Software Tech. and Dept. of Comp. Sci. and Tech., Nanjing University, Nanjing, China

Huiyan Wang

State Key Lab. for Novel Software Tech. and Dept. of Comp. Sci. and Tech., Nanjing University, Nanjing, China

Chang Xu

Nanjing University

China

Ruiqing Zhang

Search Tech. Center Asia, Microsoft, Suzhou, China

Shing-Chi Cheung

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology

China

Xiaoxing Ma

State Key Lab. for Novel Software Tech. and Dept. of Comp. Sci. and Tech., Nanjing University

Floating-point arithmetic is widely used in applications from several fields including scientific computing, machine learning, graphics, and finance. Many of these applications are rapidly adopting the use of GPUs to speedup computations. GPUs, however, have limited support to detect floating-point exceptions, making it difficult to software developers to detect unexpected exceptional cases in such applications. We present FPChecker, the first tool to automatically detect floating-point exceptions in GPU applications. FPChecker uses the clang/LLVM compiler to instrument GPU kernels and to detect floating-point exceptions in GPUs at runtime. Once an exception is detected, it reports to the programmer the code location of the exception as well as other useful information. The programmer can then use this information to avoid the exception, e.g., by modifying the application algorithm or changing its input. We present the design of FPChecker, an evaluation of the overhead of the tool, and a real-world case scenario on which the tool is used to identify a hidden exception. The slowdown of FPChecker is moderate ($1.5\times$ on average) and the code is publicly available as open source.

Ignacio Laguna

Lawrence Livermore National Laboratory

This paper presents PMExec, a tool that supports the execution of partial UML-RT models. To this end, the tool implements the following steps: static analysis, automatic refinement, and input-driven execution. The static analysis respects the execution semantics of UML-RT models is used to detect problematic model elements, i.e., elements that cause problems during execution due to the partiality. Then, the models are refined automatically using model transformation techniques, which mostly add decision points where missing information can be supplied. Third, the refined models are executed, and when the execution reaches the decision points, input required to continue the execution is obtained either interactively or from a script that captures how to deal with partial elements. We have evaluated PMExec using several use-cases that show that the static analysis, refinement, and application of user input can be carried out with reasonable performance, and that the overhead of approach, due to the refinement step, is manageable. (https://youtu.be/BRKsselcMnc)

Mojtaba Bagherzadeh

Queen's University

Canada

Karim Jahed

Queen's University

Nafiseh Kahani

Queen's University

Juergen Dingel

Queen's University, Kingston, Ontario

Precisely modeling complex systems like cyber-physical systems is challenging, which often render model-based system verification techniques like model checking infeasible. To overcome this challenge, we propose a method called LAR to automatically ‘verify’ such complex systems through a combination of learning, abstraction and refinement from a set of system log traces. We assume that log traces and sampling frequency are adequate to capture ‘enough’ behaviour of the system. Given a safety property and the concrete system log traces as input, LAR automatically learns and refines system models, and produces two kinds of outputs. One is a counterexample with a bounded probability of being spurious. The other is a probabilistic model based on which the given property is ‘verified’. The model can be viewed as a proof obligation, i.e., the property is verified if the model is correct. It can also be used for subsequent system analysis activities like runtime monitoring or model-based testing. Our method has been implemented as a self-contained software toolkit. The evaluation on multiple benchmark systems as well as a real-world water treatment system shows promising results.

Link to Publication: https://ieeexplore.ieee.org/abstract/document/8576657

Jingyi Wang

National University of Singapore, Singapore

Singapore

Jun Sun

Singapore Management University, Singapore

Singapore

Shengchao Qin

University of Teesside

United Kingdom

Cyrille Jegourel

ISTD, Singapore University of Technology and Design

Context Specification mining techniques are typically used to extract the specification of a software in the absence of (up-to-date) specification documents. This is useful for program comprehension, testing, and anomaly detection. However, specification mining can also potentially be used for debugging, where a faulty behavior is abstracted to give developers a context about the bug and help them locating it.

Objective In this project, we investigate this idea in an industrial setting. We propose a very basic semi-automated specification mining approach for debugging and apply that on real reported issues from an AutoPilot software system from our industry partner, MicroPilot Inc. The objective is to assess the feasibility and usefulness of the approach in a real-world setting.

Method The approach is developed as a prototype tool, working on C code, which accept a set of relevant state fields and functions, per issue, and generates an extended finite state machine that represents the faulty behavior, abstracted with respect to the relevant context (the selected fields and functions).

Results We qualitatively evaluate the approach by a set of interviews (including observational studies) with the company’s developers on their real-world reported bugs. The results show that (a) our approach is feasible, (b) it can be automated to some extent, and (c) brings advantages over only using their code-level debugging tools. We also compared this approach with traditional fully automated state-merging algorithms and reported several issues when applying those techniques on a real-world debugging context.

Conclusion The main conclusion of this study is that the idea of an “interactive” specification mining rather than a fully automated mining tool is NOT impractical and indeed is useful for the debugging use case.

Link to Publication: https://www.sciencedirect.com/science/article/pii/S0950584919301004

Mohammad Jafar Mashhadi

University of Calgary

Canada

Taha R. Siddiqui

InfoMagnetics Technologies Corp

Canada

Hadi Hemmati

University of Calgary

Canada

Howard W. Loewen

Department of Electrical & Computer Engineering, University of Calgary

Canada

Modern software systems are increasingly dependent on third-party libraries. It is widely recognized that reusing functionality provided by mature and well-tested third-party libraries can improve developers’ productivity, reduce time-to-market, and produce more reliable software. Today’s open-source repositories provide a wide range of libraries that can be freely downloaded and used. However, as software libraries are documented separately but intended to be used together, developers are unlikely to fully take advantage of these reuse opportunities. In this paper, we present a novel approach to automatically identify third-party library usage patterns, i.e., collections of libraries that are commonly used together by developers. Our approach employs a hierarchical clustering technique to group together software libraries based on external client usage. Our approach is based on the analysis of the joint versus separate use of the libraries. The pattern’s libraries are distributed on different usage cohesion levels/layers. Each layer reflects the co-usage frequency between a set of libraries, while the distribution on the different levels demonstrates the graduation in the degree of co-usage frequency. To evaluate our approach, we mined a large set of over 6000 popular libraries from Maven Central Repository and investigated their usage by over 38,000 client systems from the GitHub repository. Our experiments show that our technique is able to detect the majority (77%) of highly consistent and cohesive library usage patterns across a considerable number of client systems.

Link to Publication: https://www.sciencedirect.com/science/article/pii/S0164121218301699

Mohamed Aymen Saied

Concordia University

Ali Ouni

ETS Montreal, University of Quebec

Canada

Houari Sahraoui

Université de Montréal

Canada

Raula Gaikovina Kula

NAIST

Japan

Katsuro Inoue

Osaka University

Japan

David Lo

Singapore Management University

Singapore

Software systems are often released without formal specifications. To deal with the problem of lack of and outdated specifications, rule-based specification mining approaches have been proposed. These approaches analyze execution traces of a system to infer the rules that characterize the protocols, typically of a library, that its clients must obey. Rule-based specification mining approaches work by exploring the search space of all possible rules and use interestingness measures to differentiate specifications from false positives. Previous rule-based specification mining approaches often rely on one or two interestingness measures, while the potential benefit of combining multiple available interestingness measures is not yet investigated. In this work, we propose a learning to rank based approach that automatically learns a good combination of 38 interestingness measures. Our experiments show that the learning to rank based approach outperforms the best performing approach leveraging single interestingness measure by up to 66%.

Link to Publication: https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=4990&context=sis_research

Zherui Cao

Zhejiang University

China

Yuan Tian

Queens University, Kingston, Canada

Canada

Tien-Duy B. Le

School of Information Systems, Singapore Management University

David Lo

Singapore Management University

Singapore

Precise pointer analysis is desired since it is a core technique to find memory defects. There are several dimensions of pointer analysis precision, flow sensitivity, context sensitivity, field sensitivity and path sensitivity. For static analysis tools utilizing pointer analysis, considering all dimensions is difficult because the trade-off between precision and efficiency should be balanced. This paper presents TsmartGP, a static analysis tool for finding memory defects in C programs with a precise and efficient pointer analysis. The pointer analysis algorithm is flow, context, field, and quasi path sensitive. Control flow automatons are the key structures for our analysis to be flow sensitive. Function summaries are applied to get context information and elements of aggregate structures are handled to improve precision. Path conditions are used to filter unreachable paths. For efficiency, a multi-entry mechanism is proposed. Utilizing the pointer analysis algorithm, we implement a checker in TsmartGP to find uninitialized pointer errors in 13 real-world applications. Cppcheck and Clang Static Analyzer are chosen for comparison. The experimental results show that TsmartGP can find more errors while its accuracy rate is 100%, much higher than 3.1% for Cppcheck and 16.7% for Clang Static Analyzer. The demo video is available at https://youtu.be/IQlshemk6OA and TsmartGP can be downloaded from https://github.com/laoyaolandq/TsmartGP.

Yuexing Wang

Tsinghua University

Guang Chen

Tsinghua University

Min Zhou

Tsinghua University

Ming Gu

Tsinghua University

Jiaguang Sun

Tsinghua University