Developers often want to find out how to use a certain API (e.g., FileReader.read in JDK library). API usage examples are very helpful in this regard. Over the years, many automated methods have been proposed to generate code examples by clustering and summarizing relevant code snippets extracted from a code corpus. These approaches simplify source code as method invocation sequences or feature vectors. Such simplifications only model partial aspects of the code and tend to produce inaccurate examples.

We propose CodeKernel, a graph kernel based approach to the selection of API usage examples. Instead of approximating source code as method invocation sequences or feature vectors, CodeKernel represents source code as object usage graphs. Then, it clusters graphs by embedding them into a continuous space using a graph kernel. Finally, it outputs code examples by selecting a representative graph from each cluster using designed ranking metrics. Our empirical evaluation shows that CodeKernel selects more accurate code examples than MUSE and eXoaDocs. A user study involving 25 developers in a multinational company also confirms the usefulness of CodeKernel in selecting API usage examples.

Xiaodong Gu

The Hong Kong University of Science and Technology

China

Hongyu Zhang

The University of Newcastle

Australia

Sunghun Kim

Hong Kong University of Science and Technology

Hong Kong

High quality method names are critical for the readability and maintainability of programs. However, constructing concise and consistent method names is often challenging, especially for inexperienced developers. To this end, advanced machine learning techniques have been recently leveraged to recommend method names automatically for given method bodies/implementation. Recent large-scale evaluations also suggest that such approaches are accurate. However, little is known about where and why such approaches work or don’t work. To figure out the state of the art as well as the rationale for the success/failure, in this paper we conduct an empirical study on the state-of-the-art approach code2vec. We assess code2vec on a new dataset with more realistic settings. Our evaluation results suggest that although switching to new dataset does not significantly influence the performance, more realistic settings do significantly reduce the performance of code2vec. Further analysis on the successfully recommended method names also reveals the following findings: 1) around half (48.3%) of the accepted recommendations are made on getter/setter methods; 2) a large portion (19.2%) of the successfully recommended method names could be copied from the given bodies. To further validate its usefulness, we ask developers to manually score the difficulty in naming methods they developed. Code2vec is then applied to such manually scored methods to evaluate how often it works in need. Our evaluation results suggest that code2vec rarely works when it is really needed. Finally, to intuitively reveal the state of the art and to investigate the possibility of designing simple and straightforward alternative approaches, we propose a heuristics based approach to recommending method names. Evaluation results on large-scale dataset suggest that this simple heuristics-based approach significantly outperforms the state-of-the-art machine learning based approach, improving precision and recall by 65.25% and 22.45%, respectively. The comparison suggests that machine learning based recommendation of method names still has a long way to go.

Link to Publication: https://liuhuigmail.github.io/2019ASE.pdf

Lin Jiang

beijing university of posts and telecommunication

Hui Liu

Beijing Institute of Technology

China

He Jiang

School of Software, Dalian University of Technology

China

Designing usable APIs is critical to developers’ productivity and software quality, but is quite difficult. Understanding how the API is used by analyzing client code at scale to discover usability issues with the APIs is even harder. In this paper, we focus on “boilerplate” code, which a number of experts in API design have said can be an indicator of API usability problems. We investigate what properties make code count as boilerplate, the reasons for boilerplate, and how programmers can reduce the need for it. We propose MARBLE, a novel approach to automatically mine boilerplate code from a large set of client code. MARBLE adapts existing techniques, including an API usage mining algorithm, an AST comparison algorithm, and a graph partitioning algorithm. We evaluate MARBLE with 13 Java APIs, and show that our algorithm successfully identifies both already-identified and new boilerplate code instances.

Daye Nam

Carnegie Mellon University

United States

Amber Horvath

Carnegie Mellon University

Andrew Macvean

Google, Inc.

Brad Myers

Carnegie Mellon University

United States

Bogdan Vasilescu

Carnegie Mellon University

United States

The decompiler is one of the most common tools for examining binaries without corresponding source code. It transforms binaries into high-level code, reversing the compilation process. However, compilation loses information contained within the original source code (e.g., structure, type information, and variable names). Semantically meaningful variable names are known to increase code understandability, but they generally cannot be recovered by decompilers. We propose the Decompiled Identifier Renaming Engine (DIRE), a novel probabilistic technique for variable name recovery that uses both lexical and structural information. We also present a technique for generating corpora suitable for training and evaluating models of decompiled code renaming, which we use to create a corpus of 164,632 unique x86-64 binaries generated from C projects mined from GitHub. Our results show that on this corpus DIRE can predict variable names identical to the names in the original source code up to 74.3% of the time.

Jeremy Lacomis

Carnegie Mellon University

United States

Pengcheng Yin

Carnegie Mellon University

Edward J. Schwartz

Carnegie Mellon University Software Engineering Institute

Miltiadis Allamanis

Microsoft Research, Cambridge

United Kingdom

Claire Le Goues

Carnegie Mellon University

United States

Graham Neubig

Carnegie Mellon University

Bogdan Vasilescu

Carnegie Mellon University

United States

Application programming interfaces (APIs) continually evolve to meet ever-changing user needs, and documentation provides an authoritative reference for their usage. However, API documentation is commonly outdated because nearly all of the associated updates are performed manually. Such outdated documentation, especially with regard to API names, causes major software development issues. In this paper, we propose a method for automatically updating outdated API names in API documentation. Our insight is that API updates in documentation can be derived from API implementation changes between code revisions. To evaluate the proposed method, we applied it to four open source projects. Our evaluation results show that our method, FreshDoc, detects outdated API names in API documentation with 48% higher accuracy than the existing state-of-the-art methods do. Moreover, when we checked the updates suggested by FreshDoc against the developers? manual updates in the revised documentation, FreshDoc addressed 82% of the outdated names. When we reported 40 outdated API names found by FreshDoc via issue tracking systems, developers accepted 75% of the suggestions. These evaluation results indicate that FreshDoc can be used as a practical method for the detection and updating of API names in the associated documentation.

Link to Publication: https://ieeexplore.ieee.org/document/8651318

Seonah Lee

Gyeongsang National University

Rongxin Wu

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology

China

Shing-Chi Cheung

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology

China

Sungwon Kang

Korea Advanced Institute of Science and Technology