Identifying Software Performance Changes Across Variants and Versions
We address the problem of identifying performance changes in the evolution of configurable software systems. Finding optimal configurations and configuration options that influence performance is already difficult, but in light of software evolution, configuration-dependent performance changes may lurk in a potentially large number of different versions of the system.
In this work, we combine two perspectives – variability and time – and propose a novel approach to identify configuration-dependent performance changes. In a nutshell, we iteratively sample pairs of configurations and versions and measure the respective performance that help us update a model of likelihoods for performance changes. Pursuing a search strategy with the goal of measuring selectively and incrementally further pairs, we increase the accuracy of identified change points related to configuration options and interactions.
We have conducted a number of experiments both on controlled synthetic datasets as well as in real-world scenarios with different software systems. Our evaluation demonstrates that we can pinpoint performance shifts to configuration options and interactions as well as commits introducing change points with high accuracy and at scale. Our experiments on three real-world systems confirm the effectiveness and practicality of our approach.
Wed 23 Sep Times are displayed in time zone: (UTC) Coordinated Universal Time
09:10 - 10:10: Configuration Management (1)Research Papers at Koala Chair(s): Carmine VassalloUniversity of Zurich, Switzerland | |||
09:10 - 09:30 Talk | Automated Implementation of Windows-related Security-Configuration Guides Research Papers Patrick StöckleTechnical University of Munich, Bernd GrobauerSiemens AG, Alexander PretschnerTechnical University of Munich DOI Pre-print | ||
09:30 - 09:50 Talk | Identifying Software Performance Changes Across Variants and Versions Research Papers Stefan MühlbauerLeipzig University, Sven ApelSaarland University, Germany, Norbert SiegmundLeipzig University DOI Pre-print | ||
09:50 - 10:10 Talk | CP-Detector: Using Configuration-related Performance Properties to Expose Performance Bugs Research Papers Haochen HeNational University of Defense Technology, Zhouyang JiaNational University of Defense Technology, Shanshan LiNational University of Defense Technology, China, Erci XuNational University of Defense Technology, Tingting YuUniversity of Kentucky, Yue YuCollege of Computer, National University of Defense Technology, Changsha 410073, China, Ji WangNational University of Defense Technology, Xiangke LiaoNational University of Defense Technology, China DOI Pre-print |