Today, a plethora of different software verification tools exist. When having a concrete verification task at hand, software developers thus face the problem of algorithm selection. Existing algorithm selectors for software verification typically use handpicked program features together with (1) either manually designed selection heuristics or (2) machine learned strategies. While the first approach suffers from not being transferable to other selection problems, the second approach lacks interpretability, i.e., insights into reasons for choosing particular tools.

In this paper, we propose a novel approach to algorithm selection for software verification. Our approach employs representation learning together with an attention mechanism. Representation learning circumvents feature engineering, i.e., avoids the handpicking of program features. Attention permits a form of interpretability of the learned selectors. We have implemented our approach and have experimentally evaluated and compared it with existing approaches. The evaluation shows that representation learning does not only outperform manual feature engineering, but also enables transferability of the learning model to other selection tasks.

Cedric Richter

Paderborn University, Germany

Germany

Heike Wehrheim

Paderborn University

Germany

In this paper we present lightweight model-based testing of privacy and authorization concepts of national portal for electronic health services in Norway (which has over a million of visits per month). We have developed test models for creating and updating privacy levels and authorization categories using finite state machine notation. Our models emphasize not only positive but also negative behavioral aspects of the system. Using edge and edge-pair coverage as an acceptance criteria we identify and systematically derive abstract test cases (high level user scenario) from the models. Abstract test cases are further refined and transformed into concrete test cases with detailed test steps and concrete test data. Although derivation of abstract test cases and their transformation into concrete test cases are manual, execution of concrete test cases and generation of test report are automated. In total, we extracted 85 abstract test cases which resulted in about 80 concrete test cases with over 550 iterations. Automated execution of all test iterations takes about one hour, while manual test of one iteration takes about five minutes (over 40 times speedup). Model-based testing contributed to shift the focus of our intellectual work effort into model design rather than test case design, thus making derivation of test scenarios systematic and (relatively) straight forward. In addition, applying model-based testing augmented and extended our traditional quality assurance techniques by facilitating better comprehension of new privacy and authorization concepts. Graphical models helped on improved understanding of textual specifications.

Davrondzhon Gafurov

Norsk Helsenett SF

Margrete Sunde Grovan

Norsk Helsenett SF

When generating GUI tests for Android apps, it typically is a separate test computer that generates interactions, which are then executed on an actual Android device. While this approach is efficient in the sense that apps and interactions execute quickly, the communication overhead between test computer and device slows down testing considerably. In this work, we present DD-2, a test generator for Android that tests other apps on the device using Android accessibility services. In our experiments, DD-2 has shown to be 3.2 times faster than its computer-device counterpart, while sharing the same source code.

Nataniel Borges Jr.

CISPA Helmholtz Center for Information Security

Germany

Jenny Rau

CISPA Helmholtz Center for Information Security

Andreas Zeller

CISPA, Germany

Germany

Software performance testing is an essential quality assurance mechanism that can identify optimization opportunities. Automating this process requires strong tool support, especially in the case of Continuous Integration (CI) where tests need to run completely automatically and it is desirable to provide developers with actionable feedback. A lack of existing tools means that performance testing is normally left out of the scope of CI. In this paper, we propose a toolchain - PerfCI - to pave the way for developers to easily set up and carry out automated performance testing under CI. Our toolchain is based on allowing users to (1) specify performance testing tasks, (2) analyze unit tests on a variety of python projects ranging from scripts to full-blown flask-based web services, by extending a performance analysis framework (VyPR) and (3) evaluate performance data to get feedback on the code. We demonstrate the feasibility of our toolchain by using it on a web service running at the Compact Muon Solenoid (CMS) experiment at the world’s largest particle physics laboratory — CERN.

Omar Javed

Università della Svizzera italiana

Joshua Heneage Dawes

University of Manchester and CERN

Marta Han

University of Zagreb and CERN

Giovanni Franzoni

CERN

Andreas Pfeiffer

CERN

Giles Reger

University of Manchester

United Kingdom

Walter Binder

University of Lugano, Switzerland