Family-based Performance Measurement


Description

Most contemporary programs are customizable. They provide many features that give rise to millions of program variants. Determining which feature selection yields an optimal performance is challenging, because of the exponential number of variants. Predicting the performance of a variant based on previous measurements has proved successful, but induces a trade-off between the measurement effort and prediction accuracy. We propose the alternative approach of family-based performance measurement, to reduce the number of measurements required for identifying feature interactions and for obtaining accurate predictions. The key idea is to create a variant simulator (by translating compile-time to run-time variability) that can simulate the behavior of all program variants. We use it to measure performance of individual methods, trace methods to features, and infer feature interactions based on the call graph. We evaluate our approach by means of five feature-oriented programs. On average, we achieve accuracy of 98%, with only a single measurement per customizable program.




Supplemental Material for GPCE 2013 submission: Family-based Performance Measurement


Download all case studies here and all results here.
ProjektCodeRaw MeasurementsResults
AJStats clickclickclick
Elevatorclickclickclick
Emailclickclickclick
ZipMeclickclickclick
MBenchclickclickclick



To download the latest version of FeatureHouse, including the variant-simulator generator, please visit: FeatureHouse HQ

Contact

This web site is maintained by Norbert Siegmund at the University of Passau.