In modern programming languages, exception handling is an effective mechanism to avoid unexpected runtime errors. Thus, failing to catch and handle exceptions could lead to serious issues like system crashing, resource leaking, or negative end-user experiences. However, writing correct exception handling code is often challenging in mobile app development due to the fast-changing nature of API libraries for mobile apps and the insufficiency of their documentation and source code examples. Our prior study shows that in practice mobile app developers cause many exception-related bugs and still use bad exception handling practices (e.g. catch an exception and do nothing). To address such problems, in this paper, we introduce two novel techniques for recommending correct exception handling code. One technique, XRank, recommends code to catch an exception likely occurring in a code snippet. The other, XHand, recommends correction code for such an occurring exception. We have developed ExAssist, a code recommendation tool for exception handling using XRank and XHand. The empirical evaluation shows that our techniques are highly effective. For example, XRank has top-1 accuracy of 70% and top-3 accuracy of 87%. XHand’s results are 89% and 96%, respectively.

Tam Nguyen

Phong Minh Vu

Tung Thanh Nguyen

To detect large-variance code clones (i.e. clones with relatively more differences) in large-scale code repositories is difficult because most current tools can only detect almost identical or very similar clones. It will make promotion and changes to some software applications such as bug detection, code completion, software analysis, etc. Recently, CCAligner made an attempt to detect clones with relatively concentrated modifications called large-gap clones. Our contribution is to develop a novel and effective detection approach of large-variance clones to more general cases for not only the concentrated code modifications but also the scattered code modifications. A detector named LVMapper is proposed, borrowing and changing the approach of sequencing alignment in bioinformatics which can find two similar sequences with more differences. The ability of LVMapper was tested on both self-synthetic datasets and real cases, and the results show substantial improvement in detecting large-variance clones compared with other state-of-the-art tools including CCAligner. Furthermore, our new tool also presents good recall and precision for general Type-1, Type-2 and Type-3 clones on the widely used benchmarking dataset, BigCloneBench.

Ming Wu

Pengcheng Wang

University of Science and Technology of China

China

Kangqi Yin

Haoyu Cheng

Yun Xu

University of Science and Technology of China

China

Chanchal K. Roy

University of Saskatchewan

Canada

The correctness of compilers is instrumental in the safety and reliability of other software systems, as bugs in compilers can produce programs that do not reflect the intents of programmers. Compilers are complex software systems due to the complexity of optimization. GCC is an optimizing C compiler that has been used in building operating systems and many other system software. In this paper, we describe K-CONFIG, an approach that uses the bugs reported in the GCC repository to generate new test inputs. Our main insight is that the features appearing in the bug reports are likely to reappear in the future bugs, as the bugfixes can be incomplete or those features may be inherently challenging to implement hence more prone to errors. Our approach first clusters the failing test input extracted from the bug reports into clusters of similar test inputs. It then uses these clusters to create configurations for Csmith, the most popular test generator for C compilers. In our experiments on two versions of GCC, our approach could trigger up to 36 miscompilation failures, and 179 crashes, while Csmith with the default configuration did not trigger any failures. This work signifies the benefits of analyzing and using the reported bugs in the generation of new test inputs.

Md Rafiqul Islam Rabin

University of Houston

United States

Mohammad Amin Alipour

Developer forums are one of the most popular and useful Q&A websites on API usages. The analysis of API forums can be a critical resource for the automated question and answer approaches. In this paper, we empirically study three API forums including Twitter, eBay, and AdWords, to investigate the characteristics of question-answering process. We observe that +60% of the posts on all three forums were answered by providing API method names or documentation. +85% of the questions were answered by API development teams and the answers from API development teams drew fewer follow-up questions. Our results provide empirical evidences for us in a future work to build automated solutions to answer developer questions on API forums.

Yi Li

Nanyang Technological University

Singapore

Shaohua Wang

New Jersey Institute of Technology, USA

United States

Tien N. Nguyen

University of Texas at Dallas

United States

Son Nguyen

The University of Texas at Dallas

United States

Xinyue Ye

Yan Wang

Deep neural networks have been increasingly used in software engineering and program analysis tasks. They usually take a program and make some prediction about it, e.g., bug prediction. We call these models neural programs. The reliability of neural programs can impact the reliability of the encompassing analyses. In this paper, we describe our ongoing efforts in developing effective techniques to test neural programs. We discuss the challenges in developing such tools and our future plans. In our preliminary experiment on a recent neural model proposed in the literature, we found that the model is very brittle and simple perturbations in the input can cause the model to make a mistake in its prediction.

Md Rafiqul Islam Rabin

University of Houston

United States

Ke Wang

Visa Research

United States

Mohammad Amin Alipour

Automatically generating code from a textual description of method invocation confronts challenges. There were two current research directions for this problem. One direction focuses on considering a textual description of method invocations as a separate Natural Language query and do not consider the surrounding context of the code. Another direction takes advantage of a practical large scale code corpus for providing a Machine Translation model to generate code. However, this direction got very low accuracy. In this work, we tried to improve these drawbacks by proposing MethodInfoToCode, an approach that embeds context information and optimizes the ability of learning of original Phrase-based Statistical Machine Translation (PBMT) in NLP to infer implementation of method invocation given method name and other context information. We conduct an expression prediction models learned from 2.86 million method invocations from the practical data of high qualities corpus on Github that used 6 popular libraries: JDK, Android, GWT, Joda-Time, Hibernate, and Xstream. By the evaluation, we show that if the developers only write the method name of a method invocation in a body of a method, MethodInfoToCode can predict the generated expression correctly at 73% in F1 score.

Hung Phan

Deep learning (DL) techniques have demonstrated satisfactory performance in many tasks, even in safety-critical applications. Reliability is hence a critical consideration to DL-based systems. However, the statistical nature of DL makes it quite vulnerable to invalid inputs, i.e., those cases that are not considered in the training phase of a DL model. This paper proposes to perform data sanity check to identify invalid inputs, so as to enhance the reliability of DL-based systems. To this end, we design and implement a tool to detect behavior deviation of a DL model when processing an input case, and considers it the symptom of invalid input cases. Via a light, automatic instrumentation to the target DL model, this tool extracts the data flow footprints and conducts an assertion-based validation mechanism.

Haochuan Lu

Fudan University

Huanlin Xu

Nana Liu

Yangfan Zhou

Fudan University

Xin Wang

We assert that it is the ethical duty of software engineers to strive to reduce software discrimination. This paper discusses how that might be done. This is an important topic since machine learning software is increasingly being used to make decisions that affect people’s lives. Potentially, the application of that software will result in fairer decisions because (unlike humans) machine learning software is not biased. However, recent results show that the software within many data mining packages exhibits “group discrimination”; i.e. their decisions are inappropriately affected by “protected attributes” (e.g., race, gender, age, etc.). There has been much prior work on validating the fairness of machine-learning models (by recognizing when such software discrimination exists). But after detection, comes mitigation. What steps can ethical software engineers take to reduce discrimination in the software they produce? This paper shows that making fairness as a goal during hyperparameter optimization can (a) preserve the predictive power of a model learned from a data miner while also (b) generates fairer results. To the best of our knowledge, this is the first application of hyperparameter optimization as a tool for software engineers to generate fairer software.

Joymallya Chakraborty

North Carolina State University

United States

Tianpei Xia

Fahmid M. Fahid

Tim Menzies

North Carolina State University

United States

Modern software development relies heavily on Application Programming Interface (API) libraries. However, there are often certain constraints on using API elements in such libraries. Failing to follow such constraints (API misuse) could lead to serious programming errors. Many approaches have been proposed to detect API misuses, but they still have low accuracy and cannot repair the detected misuses. In this paper, we propose SAM, a novel approach to detect and repair API misuses automatically. SAM uses statistical models to describe five factors involving in any API method call: related method calls, exceptions, pre-conditions, post-conditions, and values of arguments. These statistical models are trained from a large repository of high-quality production code. Then, given a piece of code, SAM verifies each of its method calls with the trained statistical models. If a factor has a sufficiently low probability, the corresponding call is considered as an API misuse. SAM performs an optimal search for editing operations to apply on the code until it has no API issue.

Tam Nguyen

Phong Minh Vu

Tung Thanh Nguyen

The goal of automated program repair is to automate patch generation for buggy programs to reduce the manual effort by developers. A generate-and-validate method, such as GenProg, is a kind of typical repair methods that continuously generate potential patches and then validate the patches with a given test suite. A generate-and-validate method can accumulate patches when the execution time of repair methods increases. However, how many buggy programs can be newly patched when the time increase? In this paper, we conducted an exploratory study of repairing 2980 small-scale buggy programs from the CODEFLAWS benchmark with three repair methods GENPROG, SPR, and PROPHET. The aim of this study is to understand the execution time of repair methods via investigating four research questions. Experimental results show that the time of patch generation correlates with the number of executable lines of code and the Cyclomatic complexity. That is, a complex program is difficult to be repaired. This motivates us to explore a new repair method that can weaken such correlation with the lines of code and the complexity. We designed VANFIX, a simple and effective repair method for small-scale C programs. VANFIX leverages the probability of exploring the search space to conduct a variable search neighborhood for potential patches, rather than patching suspicious statements one by one. The comparison among repair methods shows that VANFIX can generate patches for 653 buggy programs, which contains 408 correctly patched buggy programs. This makes VANFIX achieve 24% to 30% better precision than GENPROG, SPR, and PROPHET.

Chuanqi Xu

Yisen Xu

Yingqi Zhang

Jifeng Xuan

Wuhan University

China

Modern software projects include automated tests written to check the programs’ functionality. The set of functions invoked by a test is called the trace of the test, and the action of obtaining a trace is called tracing. There are many tracing tools since traces are useful for a variety of software engineering tasks such as test generation, fault localization, and test execution planning. A major drawback in using test traces is that obtaining them, i.e., tracing, can be costly in terms of computational resources and runtime. Prior work attempted to address this in various ways, e.g., by selectively tracing only some of the software components or compressing the trace on-the-fly. However, all these approaches still require building the project and executing the test in order to get its (partial, possibly compressed) trace. This is still very costly in many cases. In this work, we propose a method to predict the trace of each test without executing it, based only on static properties of the test and the tested program, as well as past experience on different tests. This prediction is done by applying supervised learning to learn the relation between various static features of test and function and the likelihood that one will include the other in its trace. Then, we show how to use the predicted traces in a recent automated troubleshooting paradigm called Learn Diagnose and plan (LDP), instead of the actual, costly-to-obtain, test traces. In a preliminary evaluation on real-world open-source projects, we observe that our prediction quality is reasonable. In addition, using our trace predictions in LDP yields almost the same results comparing to when using real traces, while requiring less overhead.

Eyal Hadad

Roni Stern

Developers today use significant amounts of open source code, surfacing the need for ways to automatically audit and upgrade library dependencies and leading to the emergence of Software Composition Analysis (SCA). SCA products are concerned with three tasks: discovering dependencies, checking the reachability of vulnerable code for false positive elimination, and automated remediation. The latter two tasks rely on call graphs of library and application code to check whether vulnerable methods found in the open source components are called by applications. However, statically-constructed call graphs introduce both false positives and false negatives on real-world projects. In this paper, we develop a novel, modular means of combining statically- and dynamically-constructed call graphs via instrumentation to improve the performance of false positive elimination. Our experiments indicate significant performance improvements, but that instrumentation-based call graphs are less readily applicable in practice.

Darius Foo

Jason Yeo

Hao Xiao

Asankhaya Sharma

The development of complex and dependable systems like autonomous vehicles relies increasingly on the use of systems modeling language (SysML). In fact, SysML has become a de facto standard for systems engineering. With model-driven engineering, a SysML model serves as a reference for the early defect detection of the system under design: the earlier the errors are detected, the less is the cost of handling the errors. Mutation testing is a fault-based technique that has recently seen its applications to SysML behavioral models (e.g., state machine diagrams). Specifically, a system’s state-transition design can be fed to a model checker where mutants are automatically generated and then killed against the desired design specifications (e.g., safety properties). In this paper, we present a novel approach based on process mining to improve the effectiveness and efficiency of the SysML mutation testing based on model checking. In our approach, the mutation operators are applied directly to the state machine diagram. These mutants are then fed as traces into a process mining tool and checked according to the event logs. Our initial results indicates that the process mining approach kills more mutants faster than the model checking method.

Mounifah Alenazi

Nan Niu

University of Cincinnati

United States

Juha Savolainen

Danfoss

For teams using distributed version control systems, the right collaborative development workflows can help maintaining the long-term quality of project repositories and improving work efficiency. Despite the fact that the workflows are important, empirical evidence on how they are used and what impact they make on the project repositories is scarce. Most suggestions on the use of workflows are merely expert opinions. Often, there is only some anecdotal evidence that underpins those advices. As a result, we do not know what workflows are used, how the developers are using them, and what impact do they have on the project repositories. This work investigates the various collaborative development workflows in eight major open-source projects, identifies and analyses the workflows together with the structures of their project repositories, discusses their similarities and differences, and lays out the future works.

panuchart bunyakiati

kasetsart university

Thailand

Usa Sammapun

kasetsart university

Thailand

Deep neural networks (DNNs) are shown to be promising solutions in many challenging artificial intelligence tasks, including object recognition, natural language processing, and even unmanned driving. A DNN model, generally based on statistical summarization of in-house training data, aims to predict correct output given an input encountered in the wild. In general, 100% precision is therefore impossible due to its probabilistic nature. For DNN practitioners, it is very hard, if not impossible, to figure out whether the low precision of a DNN model is an inevitable result, or caused by defects such as bad network design or improper training process. This paper aims at addressing this challenging problem. We approach with a careful categorization of the root causes of low precision. We find that the internal data flow footprints of a DNN model can provide insights to locate the root cause effectively. We then develop a tool, namely, DeepMorph (DNN Tomography) to analyze the root cause, which can instantly guide a DNN developer to improve the model. Case studies on four popular datasets show the effectiveness of DeepMorph.

Jiazhen Gu

Huanlin Xu

Yangfan Zhou

Fudan University

Xin Wang

Hui Xu

Michael Lyu

The Chinese University of Hong Kong

Recent studies showed that the dialogs between app developers and app users on app stores are important to increase user satisfaction and app’s overall ratings. However, the large volume of reviews and the limitation of resources discourage app developers from engaging with customers through this channel. One solution to this problem is to develop an Automated Responding System for developers to respond to app reviews in a manner that is most similar to a human response. Toward designing such system, we have conducted an empirical study of the characteristics of mobile apps’ reviews and their human-written responses. We found that an app reviews can have multiple fragments at sentence level with different topics and intentions. Similarly, a response also can be divided into multiple fragments with unique intentions to answer certain parts of their review (e.g., complaints, requests, or information seeking). We have also identified several characteristics of review (rating, topics, intentions, quantitative text feature) that can be used to rank review by their priority of need for response. In addition, we identified the degree of re-usability of past responses is based on their context (single app, apps of the same category, and their common features). Last but not least, a responses can be reused in another review if some parts of it can be replaced by a placeholder that is either a named-entity or a hyperlink. Based on those findings, we discuss the implications of developing an Automated Responding System to help mobile apps’ developers write the responses for users reviews more effectively.

Phong Minh Vu

Tam Nguyen

Tung Thanh Nguyen