Data science pipelines to train and evaluate models with machine learning may contain bugs just like any other code. Leakage between training and test data can lead to overestimating the model’s accuracy during offline evaluations, possibly leading to deployment of low-quality models in production. Such leakage can happen easily by mistake or by following poor practices but may be tedious and challenging to detect manually. We develop a static analysis approach to detect common forms of data leakage in data science code. Our evaluation shows that our analysis accurately detects data leakage and that such leakage is pervasive among over 100,000 analyzed public notebooks. We discuss how our static analysis approach can help both practitioners and educators, and how leakage prevention can be designed into the development process.

Chenyang Yang

Rachel A Brower-Sinning

Carnegie Mellon Software Engineering Institute

Grace Lewis

Carnegie Mellon Software Engineering Institute

United States

Christian Kästner

Carnegie Mellon University

Infrastructure as Code (IaC) is the process of managing IT infrastructure via programmable configuration files (also called IaC scripts). Like other software artefacts, IaC scripts may contain security smells, which are coding patterns that can result in security weaknesses. Automated analysis tools to detect security smells in IaC scripts exist, but they focus on specific technologies such as Puppet, Ansible, or Chef. This means that when the detection of a new smell is implemented in one of the tools, it is not immediately available for the technologies supported by the other tools — the only option is to duplicate the effort.

This paper presents GLITCH, a new technology-agnostic framework that enables automated polyglot smell detection by transforming IaC scripts into an intermediate representation, on which different security smell detectors can be defined. GLITCH currently supports the detection of nine different security smells in scripts written in Puppet, Ansible, or Chef. We compare GLITCH with state-of-the-art security smell detectors. The results obtained not only show that GLITCH can reduce the effort of writing security smell analyses for multiple IaC technologies, but also that it has higher precision and recall than the current state-of-the-art tools.

Nuno Saavedra

INESC-ID and IST, University of Lisbon

João F. Ferreira

INESC-ID and IST, University of Lisbon

Portugal

no description available

Mohan Cui

Fudan University

Chengjun Chen

Fudan University

Hui Xu

Fudan University

China

Yangfan Zhou

Fudan University

China

Web applications are attractive attack targets given their popularity and large number of vulnerabilities. To mitigate the threat of web vulnerabilities, an important piece of information is their affected versions. However, it is non-trivial to build accurate affected version information because confirming a version as affected or unaffected requires security expertise and huge efforts, while there are usually hundreds of versions to examine. As a result, such information is maintained in a low-quality manner in almost every public vulnerability database. Therefore, it is extremely useful to have a tool that can automatically and precisely examine a large part (even if not all) of the software versions as affected or unaffected.

To this end, this paper proposes a vulnerability-centric approach for precise (un)affected version analysis for web vulnerabilities. The key idea is to extract the vulnerability logic from a patch and directly use the vulnerability logic to check whether a version is (un)affected or not. Compared with existing works, our vulnerability-centric approach helps to tolerate the code changes across different software versions. We construct a high-quality dataset with 34 CVEs and 299 software versions to evaluate our approach. The results show that our approach achieves a precision of 98.15% and a recall of 85.01% in identifying (un)affected versions and significantly outperforms existing tools (e.g., V-SZZ, ReDebug, V0Finder).

ShiYoukun

Fudan University

Yuan Zhang

Fudan University

China

Tianhan Luo

Fudan University

Xiangyu Mao

Fudan University

Min Yang

Fudan University

Infrastructure-as-Code (IaC) is a technology that enables the managing, provisioning, and distributing of infrastructure through code instead of manual processes. As with any piece of code, IaC scripts are not immune to defects. A recent Cloud Threat Report from Palo Alto Network’s Unit 42 announced the discovery of over 199K vulnerable IaC templates. This highlights the importance of tools to prevent vulnerabilities from reaching production and shift security left in the development pipeline. Unfortunately, we observed through a comprehensive study that security linters for IaC scripts can be very imprecise. Our approach to address this problem was to leverage community expertize to improve the precision of these tools. More precisely, we interviewed professional developers of Puppet scripts to collect their feedback on the root causes of imprecision of the state-of-the-art security linter for Puppet. From that feedback, we developed a new linter adjusting 7 rules of the original linter ruleset and adding 3 new rules. We conducted a new study with 131 professional developers, showing an increase in precision from 8% to 83%. The main message of this paper is that obtaining professional feedback is feasible and highly effective and that feedback is key to the creation of high precision rulesets, which is critical for the usefulness and adoption of IaC security linters.

Sofia Reis

Instituto Superior Técnico, U. Lisboa & INESC-ID

Portugal

Rui Abreu

Faculty of Engineering, University of Porto, Portugal

Portugal

Marcelo d'Amorim

Federal University of Pernambuco

Brazil

Daniel Fortunato

INESC-ID, University of Porto

Vulnerabilities, referred to as CLV issues, are induced by cross-language invocations of vulnerable libraries. Such issues greatly increase the attack surface of Python/Java projects due to their pervasive use of C libraries. Since existing Python/Java build tools in PyPI and Maven ecosystems fail to report vulnerable libraries written in other languages such as C, CLV issues are easily missed by developers. In this paper, we conduct the first empirical study on the status quo of CLV issues in PyPI and Maven ecosystems. It is found that 82,951 projects in these ecosystems are directly or indirectly dependent on libraries compiled from the C project versions that are identified to be vulnerable in CVE reports. Our study arouses the awareness of CLV issues in popular ecosystems and presents related analysis results.

The study also leads to the development of the first automated tool, \textsc{Insight}, which provides a turn-key solution to the identification of CLV issues in PyPI and Maven projects based on published CVE reports of vulnerable C projects. \textsc{Insight} automatically identifies if a PyPI or Maven project is using a C library compiled from vulnerable C project versions in published CVE reports. It also deduces the vulnerable APIs involved by analyzing the usage of various foreign function interfaces such as \emph{CFFI} and \emph{JNI} in the concerned PyPI or Maven project. \textsc{Insight} achieves a high detection rate of 88.4% on a popular CLV issue benchmark. Contributing to the open-source community, we report 226 CLV issues detected in the actively maintained PyPI and Maven projects that are directly dependent on vulnerable C library versions. Our reports are well received and appreciated by developers with queries on the availability of \textsc{Insight}. 127 reported issues (56.2%) were quickly confirmed by developers and 74.8% of them were fixed/under fixing by popular projects, such as {\mycode Mongodb}~\cite{Mongodb} and {\mycode Eclipse/Sumo}~\cite{Eclipse/Sumo}.

Meiqiu Xu

Northeastern University, China

China

Ying Wang

Northeastern University, China

Shing-Chi Cheung

Hong Kong University of Science and Technology

China

Hai Yu

Northeastern University, China

Zhiliang Zhu

Northeastern University, China