During code reviews, software developers often raise security concerns if they find any. Ignoring such concerns can bring a severe impact on the performance of a software product. This risk can be reduced if we can automatically identify such code reviews that trigger security concerns so that we can perform additional scrutiny from the security experts. Therefore, the objective of this study is to develop an automated tool to identify code reviews that trigger security concerns.

With this goal, I developed an approach named ASTOR, where I combine two separate deep learning-based classifiers– (i) using code review comments and (ii) using the corresponding code context, and make an ensemble using Logistic Regression. Based on stratified ten-fold cross-validation, the best ensemble model achieves the F1-score of 79.7% with an accuracy of 88.4% to automatically identify code reviews that raise security concerns.

Rajshakhar Paul

Wayne State University

United States

Since toxicity during developers’ interactions in open source software (OSS) projects show negative impacts on developers’ relation, a toxicity detector for the Software Engineering (SE) domain is a need. However, prior studies found that contemporary toxicity detection tools failed their reliability with the SE texts by achieving a poor performance. To encounter this challenge, I have developed ToxiCR, a SE-specific toxicity detector that is evaluated with manually labeled 19,571 code review comments. I evaluate ToxiCR with different combinations of ten supervised learning models, five text vectorizers, and eight preprocessing techniques (two of them are SE domain-specific). After applying all possible combinations, I have found that ToxiCR significantly outperformed existing toxicity classifiers with accuracy of 95.8% and an 𝐹1_1 score of 88.9%

Jaydeb Sarker

Department of Computer Science, Wayne State University

United States

Data science libraries are updated frequently, and new version releases commonly include breaking changes. However, developers often use older versions of libraries because it is challenging to update the source code of large projects. We propose CombyInferPy, a new tool for analyzing and fixing breaking changes in library APIs. CombyInferPy infers rules in the form of Comby templates, a structural code search and replace tool that can automatically update source code. Preliminary results show that CombyInferPy can update the pandas library’s Python code. Using the Comby rules inferred by CombyInferPy, we can automatically fix several failing tests and deprecation warnings. This shows that this approach is promising and can help developers update their libraries.

Hailie Mitchell

Carnegie Mellon University

Path coverage is the process of measuring the fraction of execution paths that are taken during run-time in a software by a given set of inputs. It is commonly used to assess the stability, security, and functionality of an application; therefore, this is closely associated with software testing. Path coverage requires knowledge of the software’s source code (white-box testing), specifically the software’s potential execution paths; however, the problem becomes more challenging when the source code is not available and path coverage must be done using only the software’s binary code. This can occur if the software is a product, the software is a legacy system, or the source code is not available (e.g.~contracted software or permission-less).

This paper investigates how black-box \textit{path detection and discovery} can be achieved using \textit{execution fingerprints} that are a concise frequency representation of a software’s executed assembly instructions. Execution fingerprints can be used to identify which inputs exercised different sections of code, thus revealing execution paths. Experimental results show that clustering execution fingerprints can be used to differentiate the execution paths of software and provide a method to detect these different paths all inside a true black-box testing environment

Frank Whitworth

Wake Forest University

Embedded systems are widely used for implementing diverse Internet-of-Things (IoT) applications. These applications often deal with secret/sensitive data and encryption keys which can potentially be leaked through timing side-channel analysis. Runtime-based timing side-channel attacks are performed by measuring the time a code takes to execute and using that information to extract sensitive data. Effectively detecting such vulnerabilities with high precision and low false positives is a challenging task due to the runtime dependence of software code on the underlying hardware. Effectively fixing such vulnerabilities with low overhead is also non-trivial due to the diverse nature of embedded systems. In this article, we propose an automatic runtime side channel vulnerability detection and mitigation framework that not only considers the software code but also use the underlying hardware architecture information to tune the framework for more accurate vulnerability detection and system-specific tailored mitigation.

Prabuddha Chakraborty

University of Florida

The AAA pattern, i.e. the Arrangement, Action, and Assertion, is a common and nature layout to create a test case. Following this pattern in test cases may benefit comprehension, debugging, and maintenance. The AAA structure of real-life test cases may not be explicit due to its high complexity. Manually labeling AAA statements in test cases is tedious. Thus, an automated approach for labeling AAA statements in existing test cases could benefit new developers and projects that practice collective code ownership and test driven development.

This study contributes an automatic approach based on machine learning models. The ``secret sauce" of this approach is a set of three learning features that are based on the semantic, syntax, and context information in test cases, derived from the manual tagging process. Thus, our approach mimics how developers may manually tag the AAA pattern of a test case. We assess the precision, recall, and F-1 score of our approach based on 449 test cases, containing about 16,612 statements, across 4 Apache open source projects. For achieving the best performance in our approach, we explore the usage of six machine learning models; the contribution of the SMOTE data balancing technique; the comparison of the three learning features; and the comparison of five different methods for calculating the semantic feature. The results show our approach is able to identify Arrangement, Action, and Assertion statements with a precision upwards of 92%, and recall up to 74%. Our experiments also provide empirical insights regarding how to best leverage machine learning for software engineering tasks.

Chenhao Wei

Stevens Institute of Technology

Lu Xiao

Stevens Institute of Technology

United States

Tingting Yu

University of Cincinnati

United States

Xinyu Chen

HSBC Software Development (Guangdong) Limited

Xiao Wang

Stevens Institute of Technology

United States

Sunny Wong

Envestnet

United States

Abigail Clune

AGI

Smart contracts are self-governed computer programs that run on blockchain to facilitate asset transfer between users within a trustless environment. The absence of contract specifications hinders routine tasks, such as program understanding, debugging, testing, and verification of smart contracts. In this work, we propose a unified specification mining framework to infer specification models from past transaction histories. These include access control models describing high-level authorization rules, program invariants capturing low-level program semantics, and behavior models characterizing interaction patterns allowed by contract implementations. The extracted specification models can be used to perform conformance checking on smart contracts, with the goal of eliminating unforeseen contract quality issues.

Ye Liu

Nanyang Technological University

Being extremely dominated by men, software development organizations lack diversity. People from other groups often encounter sexist, misogynistic, and discriminatory (SMD) speech during communication. To identify SMD contents, I aim to build an automatic misogyny identification (AMI) tool for the domain of software developers. On this goal, I built a dataset of 10,138 pull request comments mined from Github based on a keyword-based selection, followed by manual validation. Using ten-fold cross-validation, I evaluated ten machine learning algorithms for automatic identification. The best performing model achieved 67.7% precision, 55.9% recall, 60.9% f-score, and 96.2% accuracy for the best performing model.

Sayma Sultana

Wayne State University

United States

Developers use exceptions guarded by conditions to abort the execution when a program reaches an unexpected state. However, sometimes the condition and the raised exception do not imply the same stopping reason, in which case, we call them inconsistent if-condition-raise statements. The inconsistency can originate from a mistake in the condition or the message of the exception. This paper presents IICR-Finder, a deep learning-based approach to detect inconsistent if-condition-raise statements. The approach reasons both about the logic of the condition and the natural language of the exception message and raises a warning in case of an inconsistency. We present six techniques to automatically generate large numbers of inconsistent statements for training. Moreover, we implement two neural models, based on binary classification and triplet loss, to learn inconsistency detection. We apply the approach to 210K if-condition-raise statements extracted from 42 million lines of Python code. It achieves a precision of 72% at a recall of 60% on a dataset of past bug fixes. Running IICR-Finder on open-source projects reveals 30 previously unknown bugs, ten of which we reported, with eight confirmed by the developers.

Islem BOUZENIA

Software Lab, University of Stuttgart

Many techniques have been proposed to mine knowledge from software artefacts and solve software evolution management tasks. To promote effective reusing of those knowledge, we propose a unified format, differential facts, to represent software changes across versions as well as various relations within each version, such as call graphs. Based on queryable formats, differential facts can be manipulated to implement complex evolution management tasks. Since facts once extracted can be shared among different tasks, the reusability brings improvements to oveall performance. We validate the technique and show its benefits of being efficient, flexible, and easy to implement, with several applications, including semantic history slicing, regression test selection, documentation error detection and client-specific usage patterns discovery.

Xiuheng Wu

Nanyang Technological University, Singapore

Singapore

RESTful APIs have been applied to provide cloud services by various notable companies. The quality assurance of RESTful API is critical. Several automatic RESTful API testing techniques have been proposed to tame this issue. By analyzing crashes caused by each test case, developers can find potential bugs in cloud services. However, automatic tools usually generate a massive number of failed test cases. Since it is labor-intensive for developers to validate each test case, automated crash clustering is one promising solution to help debug cloud services.

In this paper, we propose RESTCluster, a two-phase crash clustering approach. The preliminary evaluation result shows that RESTCluster can achieve 100% precision in different sizes of subjects with a high recall.

YI LIU

Nanyang Technological University