Performance models have been used in the past to understand the performance characteristics of software systems. However, the identification of performance criticalities is still an open challenge, since there might be several system components contributing to the overall system performance. This work combines two different areas of research to improve the process of interpreting model-based performance analysis results: (i) software performance engineering that provides the ground for the evaluation of the system’s performance; (ii) mutation-based techniques that nicely supports the experimentation of changes in performance models and contribute to a more systematic assessment of performance indices. We propose mutation operators for specific performance models, i.e., queueing networks, that resemble changes commonly made by designers when exploring the properties of a system’s performance. Our approach consists in introducing a mutation-based approach that generates a set of mutated queueing network models. The performance of these mutated networks is compared to that of the original network to better understand the effect of variations in the different components of the system. A set of benchmarks is adopted to show how the technique can be used to get a deeper understanding of the performance characteristics of software systems.

Thomas Laurent

Lero & University College Dublin

Ireland

Paolo Arcaini

National Institute of Informatics

Japan

Catia Trubiani

Gran Sasso Science Institute

Italy

Anthony Ventresque

University College Dublin & Lero, Ireland

Ireland

NOTE: SUBMITTED A RESEARCH PAPER TO ASE RESEARCH TRACK Application development for the modern Web involves sophisticated engineering workflows which include user interface aspects. Those involve Web elements typically created with HTML/CSS markup and JavaScript-like languages, yielding Web documents. WebMonitor leverages requirements formally specified in a logic able to capture both the layout of visual components as well as how they change over time, as a user interacts with them. Then, requirements are verified upon arbitrary web pages, allowing for automated support for a wide set of use cases in interaction testing and simulation. We position WebMonitor within a developer workflow, where in case of a negative result, a visual counterexample is returned. The monitoring framework we present follows a black-box approach, and as such is independent of the underlying technologies a Web application may be developed with, as well as the browser and operating system used.

Ennio Visconti

TU Wien

Christos Tsigkanos

University of Bern, Switzerland

Laura Nenzi

University of Trieste

Italy

Communication nondeterminism is one of the main reasons for the intractability of verification of message passing concurrency. In many practical message passing programs, the non-deterministic communication structure is symmetric and decomposed into epochs to obtain efficiency. Thus, symmetries and epoch structure can be exploited to reduce verification complexity. In this paper, we present a dynamic-symbolic runtime verification technique for single-path MPI programs, which (i) exploits communication symmetries by way of specifying symmetry breaking predicates (SBP) and (ii) performs compositional verification based on epochs. On the one hand, SBPs prevent the symbolic decision procedure from exploring isomorphic parts of the search space, and on the other hand, epochs restrict the size of a program needed to be analyzed at a point in time. We show that our analysis is sound and complete for single-path MPI programs on a given input. We further demonstrate that our approach leads to (i) a significant reduction in verification times and (ii) scaling up to larger benchmark sizes compared to prior trace verifiers.

Rishabh Ranjan

IIT Delhi

Ishita Agrawal

IIT Delhi

Subodh Sharma

IIT Delhi

India

In industrial environments it is critical to find out the capacity of a system and plan for a deployment layout that meets the production traffic demands. The system capacity is influenced by both the performance of the system’s constituting components and the physical environment setup. In a large system, the configuration parameters of individual components give the flexibility to developers and load test engineers to tune system performance without changing the source code. However, due to the large search space, estimating the capacity of the system given different configuration values is a challenging and costly process. In this paper, we propose an approach, called MLASP, that uses machine learning models to predict the system key performance indicators (i.e., KPIs), such as throughput, given a set of features made off configuration parameter values, including server cluster setup, to help engineers in capacity planning for production environments. Under the same load, we evaluate MLASP on two large-scale mission-critical enterprise systems developed by Ericsson and on one open-source system. We find that: 1) MLASP can predict the system throughput with a very high accuracy. The difference between the predicted and the actual throughput is less than 1%; and 2) By using only a small subset of the training data (e.g., 3% of the entire data for the open-source system), MLASP can still predict the throughput accurately. We also document our experience of successfully integrating the approach into an industrial setting. In summary, this paper highlights the benefits and potential of using machine learning models to assist load test engineers in capacity planning.

Arthur Vitui

Concordia University

Canada

Tse-Hsun (Peter) Chen

Concordia University

Canada

With the ever increasing scale and complexity of online systems, incidents are gradually becoming commonplace. Without appropriate handling, they can seriously harm the system availability. However, in large-scale online systems, these incidents are usually drowning in a slew of issues (i.e., something abnormal, while not necessarily an incident), rendering them difficult to handle. Typically, these issues will result in a cascading effect across the system, and a proper management of the incidents depends heavily on a thorough analysis of this effect. Therefore, in this paper, we propose a method to automatically analyze the cascading effect of availability issues in online systems and extract the corresponding graph based issue representations incorporating both of the issue symptoms and affected service attributes. With the extracted representations, we train and utilize a graph neural networks based model to perform incident detection. Then, for the detected incident, we leverage the PageRank algorithm with a flexible transition matrix design to locate its root cause. We evaluate our approach using real-world data collected from a very large instant messaging company. The results confirm the effectiveness of our approach. Moreover, our approach is successfully deployed in the company and eases the burden of operators in the face of a flood of issues and related alert signals.

Zilong He

Sun Yat-Sen University

Pengfei Chen

Sun Yat-Sen University

Yu Luo

Tencent Inc.

Qiuyu Yan

Tencent Inc.

Hongyang Chen

School of Computer Science and Engineering, Sun Yat-sen University

Guangba  Yu

Sun Yat-Sen University

Fangyuan Li

Tencent Inc.

Sustainable software engineering has received a lot of attention in recent times, as we witness an ever-growing slice of energy use, for example, at data centers, as software systems utilize the underlying infrastructure. Characterizing servers for their energy use accurately without being intrusive, is therefore important to make sustainable software deployment choices. In this paper, we introduce ESAVE which is a machine learning-based approach that leverages a small set of hardware attributes to characterize any server or virtual machine’s energy use across different levels of utilization. This is based upon an extensive exploration of multiple ML approaches, with a focus on a minimal set of required attributes, while showcasing good accuracy. Early validations show that ESAVE has only around 12% average prediction error. This approach is non-intrusive and therefore can enable many sustainable software engineering use cases, promoting greener DevOps.

Priyavanshi Pathania

Accenture Labs

Rohit Mehra

Accenture Labs

India

Vibhu Saujanya Sharma

Accenture Labs

Vikrant Kaulgud

Accenture Labs, India

India

Sanjay Podder

Accenture

India

Adam P. Burden

Accenture

Deploying applications on hybrid clouds with computational artifacts distributed over public backends and private edges involve several constraints. Designing such deployment requires application architects to solve several challenges, spanning over hard regulatory policy constraints as well as business policy constraints such as enablement of privacy by on-prem processing of data to the extent the business wants, backend support of privacy enabling technologies (PET), sustainability in terms of green energy utilization, latency sensitivity of the application. In this paper, we propose to optimize hybrid cloud application architectures, while taking all those factors into consideration, and empirically demonstrate the effectiveness of our approach. To the best of our knowledge, this work is the first of its kind.

Manish Ahuja

Accenture Labs

Narendranath Sukhavasi

Accenture Labs

Swapnajeet Choudhury

Accenture Labs

Kaushik Amar Das

Accenture Labs

Kapil Singi

Accenture

Kuntal Dey

Accenture Labs, India

Vikrant Kaulgud

Accenture Labs, India

India