Static Program Analysis (SPA) has long been established as an important technique for gaining insights into software systems. Over the last years, analysis designers increasingly produced analyses that are compositional, collaborative, or incremental in nature - thus relying on some form of existing results to increase performance or even precision. However, systematic result reuse is still rare in this field even though the analyzed software is mainly composed of reusable software components.
For this work, we study 40 state-of-the-art SPA implementations and find that there is a tremendous potential for reusing analysis results. We attribute this to the fact that there is no systematic process in place for persisting and sharing analysis results and propose such a process here to fill this gap. In this paper, we present SPARRI, a prototype implementation providing an HTTP API to publish, search, and reuse SPA results. Our evaluation shows that reusing existing results with SPARRI can improve analysis performance by up to 92%. Furthermore, we see potential in applying it to other research areas like empirical software studies, benchmark creation, and artifact evaluation.
Neng Zhang School of Software Engineering, Sun Yat-sen University, ChenQinde School of Software Engineering, Sun Yat-sen University, Zibin Zheng Sun Yat-sen University, Ying Zou Queen's University, Kingston, Ontario
Code Difference Guided Adversarial Example Generation for Deep Code Models
Adversarial examples are important to test and enhance the robustness of deep code models. As source code is discrete and has to strictly stick to complex grammar and semantics constraints, the adversarial example generation techniques in other domains are hardly applicable. Moreover, the adversarial example generation techniques specific to deep code models still suffer from unsatisfactory effectiveness due to the enormous ingredient search space. In this work, we propose a novel adversarial example generation technique (i.e., CODA) for testing deep code models. Its key idea is to use code differences between the target input (i.e., a given code snippet as the model input) and reference inputs (i.e., the inputs that have small code differences but different prediction results with the target input) to guide the generation of adversarial examples. It considers both structure differences and identifier differences to preserve the original semantics. Hence, the ingredient search space can be largely reduced as the one constituted by the two kinds of code differences, and thus the testing process can be improved by designing and guiding corresponding equivalent structure transformations and identifier renaming transformations. Our experiments on 15 deep code models demonstrate the effectiveness and efficiency of CODA, the naturalness of its generated examples, and its capability of enhancing model robustness after adversarial fine-tuning. For example, CODA reveals 88.05% and 72.51% more faults in models than the state-of-the-art techniques (i.e., CARROT and ALERT) on average, respectively.
DiffSearch: A Scalable and Precise Search Engine for Code Changes
The source code of successful projects is evolving all the time, resulting in hundreds of thousands of code changes stored in source code repositories. This wealth of data can be useful, e.g., to find changes similar to a planned code change or examples of recurring code improvements. This paper presents DiffSearch, a search engine that, given a query that describes a code change, returns a set of changes that match the query. The approach is enabled by three key contributions. First, we present a query language that extends the underlying programming language with wildcards and placeholders, providing an intuitive way of formulating queries that is easy to adapt to different programming languages. Second, to ensure scalability, the approach indexes code changes in a one-time preprocessing step, mapping them into a feature space, and then performs an efficient search in the feature space for each query. Third, to guarantee precision, i.e., that any returned code change indeed matches the given query, we present a tree-based matching algorithm that checks whether a query can be expanded to a concrete code change. We present implementations for Java, JavaScript, and Python, and show that the approach responds within seconds to queries across one million code changes, has a recall of 80.7% for Java, 89.6% for Python, and 90.4% for JavaScript, enables users to find relevant code changes more effectively than a regular expression-based search and GitHub’s search feature, and is helpful for gathering a large-scale dataset of real-world bug fixes.
Persisting and Reusing Results of Static Program Analyses on a Large Scale
Static Program Analysis (SPA) has long been established as an important technique for gaining insights into software systems. Over the last years, analysis designers increasingly produced analyses that are compositional, collaborative, or incremental in nature - thus relying on some form of existing results to increase performance or even precision. However, systematic result reuse is still rare in this field even though the analyzed software is mainly composed of reusable software components.
For this work, we study 40 state-of-the-art SPA implementations and find that there is a tremendous potential for reusing analysis results. We attribute this to the fact that there is no systematic process in place for persisting and sharing analysis results and propose such a process here to fill this gap. In this paper, we present SPARRI, a prototype implementation providing an HTTP API to publish, search, and reuse SPA results. Our evaluation shows that reusing existing results with SPARRI can improve analysis performance by up to 92%. Furthermore, we see potential in applying it to other research areas like empirical software studies, benchmark creation, and artifact evaluation.
