Keynote

Neural networks are powerful tools for automated decision-making, seeing increased application in safety-critical domains, such as autonomous driving. Due to their black-box nature and large scale, reasoning about their behavior is challenging. Statistical analysis is often used to infer probabilistic properties of a network, such as its robustness to noise and inaccurate inputs. While scalable, statistical methods can only provide probabilistic guarantees on the quality of their results and may underestimate the impact of low probability inputs leading to undesired behavior of the network.

We investigate here the use of symbolic analysis and constraint solution space quantification to precisely quantify probabilistic properties in neural networks. We demonstrate the potential of the proposed technique in a case study involving the analysis of ACAS-Xu, a collision avoidance system for unmanned aircraft control.

Corina S Pasareanu

Carnegie Mellon University Silicon Valley, NASA Ames Research Center

Technical

Control theoretical techniques have been successfully adopted as methods for self-adaptive systems design to provide formal guarantees about the effectiveness and robustness of adaptation mechanisms. However, the computational effort to obtain guarantees poses severe constraints when it comes to dynamic adaptation. In order to solve these limitations, in this paper, we propose a hybrid approach combining software engineering, control theory, and AI to design for software self-adaptation. Our solution proposes a hierarchical and dynamic system manager with performance tuning. Due to the gap between high-level requirements specification and the internal knob behavior of the managed system, a hierarchically composed components architecture seek the separation of concerns towards a dynamic solution. Therefore, a two-layered adaptive manager was designed to satisfy the software requirements with parameters optimization through regression analysis and evolutionary meta-heuristic. The optimization relies on the collection and processing of performance, effectiveness, and robustness metrics w.r.t control theoretical metrics at the offline and online stages. We evaluate our work with a prototype of the Body Sensor Network (BSN) in the healthcare domain, which is largely used as a demonstrator by the community. The BSN was implemented under the Robot Operating System (ROS) architecture, and concerns about the system dependability are taken as adaptation goals. Our results reinforce the necessity of performing well on such a safety-critical domain and contribute with substantial evidence on how hybrid approaches that combine control and AI-based techniques for engineering self-adaptive systems can provide effective adaptation.

Ricardo Caldas

Chalmers

Arthur Rodrigues

University of Brası́lia

Eric Bernd Gil

University of Brasilia

Brazil

Genaína Nunes Rodrigues

University of Brasília

Thomas Vogel

Humboldt-Universität zu Berlin

Germany

Patrizio Pelliccione

University of L'Aquila and Chalmers | University of Gothenburg

Italy

Technical

Jeroen Van Der Donckt

KU Leuven

Belgium

Danny Weyns

KU Leuven

Belgium

Federico Quin

Katholieke Universiteit Leuven

Belgium

Jonas Van Der Donckt

Ghent University

Sam Michiels

Katholieke Universiteit Leuven

NIER

Max Scheerer

FZI Research Center for Information Technology, Germany

Jonas Klamroth

FZI Research Center for Information Technology, Germany

Ralf Reussner

Karlsruhe Institute of Technology (KIT) and FZI - Research Center for Information Technology (FZI)

Germany

Bernhard Beckert

Karlsruhe Institute of Technology

NIER

Yuning He

NASA Ames

United States

Johann Schumann

NASA Ames

NIER

Recent approaches in testing autonomous driving systems (ADS) are able to generate a scenario in which the autonomous car collides , and a different ADS configuration that avoids the collision. However, such test information is too low level to be used by engineers to improve the ADS. In this paper, we consider a path planner component provided by our industry partner, that can be configured through some weights. We propose a technique to automatically re-engineer the path planner in terms of a self-adaptive path planner (SAPP) following the MAPE loop reference architecture. The Knowledge Base (KB) of SAPP contains descriptions of collision scenarios discovered with testing, and the corresponding alternative weights that avoid the collisions. We forecast two main usages of SAPP. First of all, designers are provided with a prototype that should facilitate the re-implementation of the path planner. As second usage, SAPP can be useful for improving the diversity of testing, as performing test case generation on SAPP will guarantee to find dangerous situations different from those used to build SAPP. Preliminary experiments indicate that SAPP can effectively adapt on the base of the solutions stored in KB.

Kun Liu

Peking University, China

Xiaoyi Zhang

National Institute of Informatics, Japan

Paolo Arcaini

National Institute of Informatics

Japan

Fuyuki Ishikawa

National Institute of Informatics

Japan

Wenpin Jiao

Peking University, China

Experience

Kenji Mori

Japan Aerospace Exploration Agency, Japan

Naoko Okubo

Japan Aerospace Exploration Agency, Japan

Yasushi Ueda

Japan Aerospace Exploration Agency, Japan

Masafumi Katahira

Japan Aerospace Exploration Agency

Japan

Toshiyuki Amagasa

University of Tsukuba, Japan

TechnicalBest Student Paper

The rapidly changing workload of service-based systems can easily cause under-/over-utilization on the component services, which can consequently affect the overall Quality of Service (QoS), such as latency. Self-adaptive services composition rectifies this problem, but poses several challenges: (i) the effectiveness of adaptation can deteriorate due to over-optimistic assumptions on the latency and utilization constraints, at both local and global levels; and (ii) the benefits brought by each composition plan is often short term and is not often designed for long-term benefits – a natural prerequisite for sustaining the system. To tackle these issues, we propose a two levels constraint reasoning framework for sustainable self-adaptive services composition, called DATESSO. In particular, DATESSO consists of a re ned formulation that differentiates the “strictness” for latency/utilization constraints in two levels. To strive for long-term benefits, DATESSO leverages the concept of technical debt and time-series prediction to model the utility contribution of the component services in the composition. The approach embeds a debt-aware two level constraint reasoning algorithm in DATESSO to improve the efficiency, effectiveness and sustainability of self-adaptive service composition. We evaluate DATESSO on a service-based system with real-world WS-DREAM dataset and comparing it with other state-of-the-art approaches. The results demonstrate the superiority of DATESSO over the others on the utilization, latency and running time whilst likely to be more sustainable.

Satish Kumar

University of Birmingham, United Kingdom

United Kingdom

Tao Chen

Loughborough University

United Kingdom

Rami Bahsoon

University of Birmingham

United Kingdom

Rajkumar Buyya

University of Melbourne, Australia

NIER

Two of the main paradigms used to build adaptive software employ different types of properties to capture relevant aspects of the system’s run-time behavior. On the one hand, control systems consider properties that concern static aspects like stability, as well as dynamic properties that capture the transient evolution of variables such as settling time. On the other hand, self-adaptive systems consider mostly non-functional properties that capture concerns such as performance, reliability, and cost. In general, it is not easy to reconcile these two types of properties or identify under which conditions they constitute a good fit to provide run-time guarantees. There is a need of identifying the key properties in the areas of control and self-adaptation, as well as of characterizing and mapping them to better understand how they relate and possibly complement each other. In this paper, we take a first step to tackle this problem by: (1) identifying a set of key properties in control theory, (2) illustrating the formalization of some of these properties employing temporal logic languages commonly used to engineer self-adaptive software systems, and (3) illustrating how to map key properties that characterize self-adaptive software systems into control properties, leveraging their formalization in temporal logics. We illustrate the different steps of the mapping on an exemplar case in the cloud computing domain and conclude with identifying open challenges in the area.

Javier Camara

University of York

United Kingdom

Alessandro Vittorio Papadopoulos

Mälardalen University

Sweden

Thomas Vogel

Humboldt-Universität zu Berlin

Germany

Danny Weyns

KU Leuven

Belgium

David Garlan

Carnegie Mellon University

United States

Shihong Huang

Florida Atlantic University

United States

Kenji Tei

Waseda University / National Institute of Informatics, Japan

Japan

NIER

Many self-adaptive systems benefit from human involvement and oversight, where a human operator can provide expertise not available to the system and can detect problems that the system is unaware of. One way of achieving this is by placing the human operator on the loop – i.e., providing supervisory oversight and intervening in the case of questionable adaptation decisions. To make such interaction effective, explanation is sometimes helpful to allow the human to understand why the system is making certain decisions and calibrate confidence from the human perspective. However, explanations come with costs in terms of delayed actions and the possibility that a human may make a bad judgement. Hence, it is not always obvious whether explanations will improve overall utility and, if so, what kinds of explanation to provide to the operator. In this work, we define a formal framework for reasoning about explanations of adaptive system behaviors and the conditions under which they are warranted. Specifically, we characterize explanations in terms of explanation content, effect, and cost. We then present a dynamic adaptation approach that leverages a probabilistic reasoning technique to determine when the explanation should be used in order to improve overall system utility.

NIANYU LI

Peking University, China

China

Sridhar Adepu

Singapore University of Technology and Design, Singapore

Eunsuk Kang

Carnegie Mellon University

United States

David Garlan

Carnegie Mellon University

United States

Technical

A distributed system’s functionality must continuously evolve, especially when environmental context changes. Such required evolution imposes unbearable complexity on system development. An alternative is to make systems able to self-adapt by opportunistically composing at runtime to generate systems of systems (SoSs) that offer value-added functionality. The success of such an approach calls for abstracting the heterogeneity of systems and enabling the programmatic construction of SoSs with minimal developer intervention. We propose a general ontology-based approach to describe distributed systems, seeking to achieve abstraction and enable runtime reasoning between systems. We also propose an architecture for systems that utilize such ontologies to enable systems to discover and ‘understand’ each other, and potentially compose, all at runtime. We detail features of the ontology and the architecture through two contrasting case studies: one on controlling multiple systems in smart home environment, and another on the management of dynamic computing clusters. We also quantitatively evaluate the scalability and validity of our approach through experiments and simulations. Our approach enables system developers to focus on high-level SoS composition without being constrained by deployment-specific implementation details

Abdessalam Elhabbash

Lancaster University

United Kingdom

Vatsala Nundloll

Lancaster University

United Kingdom

Yehia Elkhatib

Lancaster University

United Kingdom

Gordon Blair

Lancaster University

United Kingdom

Vicent Sanz Marco

Lancaster University

Demo

Self-Adaptive Systems (SASs) reflect on both their state and on the environment and change their behavior to satisfy the expected objectives. Cloud systems are self-adaptive by nature, especially considering the resources used in a pay-as-you-go manner. Satisfying trustworthiness (worthiness of a service based on evidences of its trust) properties also demands self-adaptation capabilities. Unfortunately, developers lack an easy-to-use platform to support the assessment of such properties and to execute the required adaptions. This paper presents TMA, a platform that implements a MAPE-K control loop for cloud systems, supported by a distributed monitoring system based on probes. Quality Models are used to express trustworthiness properties, resulting in scores, which are used to plan adaptations through evaluation rules. These plans are executed by actuators. A demo shows the scaling up/down of the number of containers in a cloud application of a set of web services from TPC Benchmarks, as a result of changes observed in the environment.

José D'Abruzzo Pereira

University of Coimbra

Portugal

Rui Silva

University of Coimbra

Portugal

Nuno Antunes

Universidade de Coimbra

Portugal

Jorge Luiz Machado da Silva

Unicamp

Brazil

Breno de França

UNICAMP

Brazil

Regina Moraes

UNICAMP

Brazil

Marco Vieira

University of Coimbra

Portugal

NIER

Self-adaptive systems continuously adapt to internal and external changes in their execution environment. In context-based self-adaptation, adaptations take place in response to the characteristics of the execution environment, captured as a context. However, in large-scale adaptive systems operating in dynamic environments, multiple contexts are often active at the same time, requiring simultaneous execution of multiple adaptations. Complex interactions between such adaptations might not have been foreseen or accounted for at design time. For example, adaptations can partially overlap, requiring only partial execution of each, or they can be conflicting, requiring some of the adaptations not to be executed at all, in order to preserve system execution. To ensure a correct composition of adaptations, we propose ComInA, a novel reinforcement learning based approach, which autonomously learns interactions between adaptations as well as the most appropriate adaptation composition for each combination of active contexts, as they arise. We present an initial evaluation of ComInA in an urban public transport network simulation, where multiple adaptations to buses, routes, and stations are required. Early results show that ComInA correctly identifies whether adaptations are compatible or conflicting and learns to execute adaptations which maximize system performance. However, further investigation is needed into how best to utilize such identified relationships to optimize a wider range of metrics and utilize more complex composition strategies.

Nicolás Cardozo

Universidad de los Andes

Colombia

Ivana Dusparic

Trinity College Dublin, Ireland

Doctoral

Self-adaptive Systems (SASs) are one way to address the ever growing complexity of software systems by allowing the system to react on changes in its operating environment. In today’s systems, self-adaptation is typically realized with a control loop, from which the MAPE-K feedback loop is a prominent example. Research uses the notion of patterns to describe the distribution and decentralization of individual control loop components or control loops and their underlying managed subsystems. While there are some well accepted standards about which components an managed subsystem has to implement so that it can interact with the control loop, research still lacks best practices for communication within and across control loops. This paper aims to identify several research challenges which exist currently in this domain. Furthermore, ideas on upcoming research to create distributed SASs on the basis of roles, distributed benchmarks for SASs and inter- and intra-communication for control loops will be presented. The relation between communication in control loops and distributed benchmarking will be introduced and ongoing work on a self-adaptive distributed benchmarking application will be discussed. Next, an evaluation strategy will be presented to provide evidence for viable results to the community. Finally, a research roadmap to tackle the discussed challenges is presented.

Ilja Shmelkin

Technische Universität Dresden, Germany

Doctoral

Tim Kluge

TU Dresden, Germany

Keynote

This keynote discusses a radically unconventional approach to developing robots: using evolution. Artificial evolution has been successful in optimization and design in the field of Evolutionary Computing; applying it to developing robots implies huge challenges and great promises, [1], [2], [3]. The long-term vision foresees robots that reproduce and evolve in real-time and real space. Possible application scenarios can be divided into two categories depending on the extent of human involvement. First, the “breeding farm scenario”, where humans steer and accelerate evolution through influencing selection (and possibly also reproduction) until a good robot design emerges in the environment of the “breeding farm”. This design can then be validated, produced and employed on a large scale in the real application environment, e.g., cave systems, deep seas, rainforests. Second, the “other planet scenario”, where the robot population evolves and adapts autonomously to the unknown environment without the need for direct human oversight. In this case, evolution is not merely a design method that stops when the outcome is satisfactory, but a permanent force that continually improves and adapts the population to the given circumstances. While at the current level of technology this may sound more fiction than science, the quick development of 3D-printing and autonomous assembly can make this a feasible option in the near future. As of today, the overall system architecture has been designed [4], and the first proofs-of concept with automated robot (re)production have been published [5], [6]. To illustrate specific aspects I will reflect on an ongoing EPSRC project that is developing the first evolutionary robot system where robots evolve in a breeding farm scenario [7].

A.E. Eiben

Vrije Universiteit Amsterdam, Netherlands

Technical

Self-adaptive systems increasingly need to reason about and adapt both structural and behavioral system aspects, such as in mobile service robots, which must reason about missions that they need to achieve and the architecture of the software executing them. Deciding how to best adapt these systems to run time changes is challenging because it entails considering mutual dependencies between the software architecture that the system is running and the outcome of plans for completing tasks, while also considering multiple trade-offs and uncertainties. Considering all these aspects in planning for adaptation often yields large solution spaces which cannot be adequately explored at run time. We address this challenge by proposing a planning approach able to consider the impact of mutual dependencies between software architecture and task planning on the satisfaction of mission goals. The approach is able to reason quantitatively about the outcome of adaptation decisions handling both the reconfiguration of the system’s architecture and adaptation of task plans under uncertainty and in a rich trade-off space. Our results show: (i) feasibility of run-time decision-making for self-adaptation in an otherwise intractable solution space by dividing-and-conquering adaptation into architecture reconfiguration and task planning sub-problems, and (ii) improved quality of adaptation decisions with respect to decision making that does not consider dependencies between architecture and task planning.

Javier Camara

University of York

United Kingdom

Bradley Schmerl

Carnegie Mellon University, USA

United States

David Garlan

Carnegie Mellon University

United States

TechnicalBest Paper

Seung Yeob Shin

SnT Centre/University of Luxembourg

Shiva Nejati

University of Ottawa

Canada

Mehrdad Sabetzadeh

University of Ottawa

Canada

Lionel Briand

SnT Centre/University of Luxembourg

Luxembourg

Chetan Arora

School of Information Technology, Deakin University

Australia

Frank Zimmer

SES Techcom

NIER

Ana Petrovska

Technical University of Munich, Germany

Sergio Quijano

Technical University of Munich, Germany

Germany

Ilias Gerostathopoulos

Vrije Universiteit Amsterdam

Netherlands

Alexander Pretschner

Demo

Petr Hnetynka

Charles University

Tomas Bures

Charles University, Czech Republic

Czech Republic

Ilias Gerostathopoulos

Vrije Universiteit Amsterdam

Netherlands

Jan Pacovský

Charles University, Czechia

Technical

In the last four years, the number of distinct autonomous vehicles platforms deployed in the streets of California increased 6-fold, while the reported accidents increased 12-fold. This can become a trend with no signs of subsiding as it is fueled by a constant stream of innovations in hardware sensors and machine learning software. Meanwhile, if we expect the public and regulators to trust the autonomous vehicle platforms, we need to find better ways to solve the problem of adding technological complexity without increasing the risk of accidents. We studied this problem from the perspective of reliability engineering in which a given risk of an accident has an associated severity and probability of occurring. Timely information on accidents is important for engineers to anticipate and reuse previous failures to approximate the risk of accidents in a new city. However, this is challenging in the context of autonomous vehicles because of the sparse nature of data on the operational scenarios (driving trajectories in a new city). Our approach was to mitigate data sparsity by reducing the state space through the monitoring of multiple-vehicles operations. We then minimized the risk of accidents by determining proper allocation of tests for each equivalence class. Our contributions comprise (1) a set of strategies to monitor the operational data of multiple autonomous vehicles, (2) a Bayesian model that estimates changes in the risk of accidents, and (3) a feedback control-loop that minimizes these risks by reallocating test effort. Our results are promising in a sense that we were able to measure and control risk for a diversity of changes in the operational scenarios. We evaluated our models with data from two real cities with distinct traffic patterns and made the data available for the community.

Joachim Haensel

Hasso Plattner Institute, University of Potsdam, Germany

Christian Medeiros Adriano

Hasso-Plattner-Institute, Potsdam

Germany

Johannes Dyck

Hasso Plattner Institute for Software Systems Engineering, Germany

Holger Giese

Hasso Plattner Institute, University of Potsdam

Germany

NIER

Assaf Marron

Weizmann Institute of Science, Israel

Lior Limonad

IBM Corporation, Israel

Sarah Pollack

Weizmann Institute of Science, Israel

David Harel

Weizmann Institute of Science, Israel

NIER

Attacks against business logic rules occur when the attacker exploits the domain rules in a malicious way. Such logic attacks have not received sufficient attention in research so far. In this paper, we propose a novel self-protecting approach that defends a system against the exploitation of business logic vulnerabilities. The approach empowers a system with a self-protecting layer to protect it against attacks that misusing business logic rules. The appraoch maintains up-to-date domain knowledge that is analyzed using runtime verification (RV) to detect logical attacks. When attacks are discovered they are dynamically mitigated by applying proper system reconfigurations. We evaluate the approach using a case in the domain of hotel booking systems.

Silvan Zeller

Omegapoint AB, Sweden

Narges Khakpour

Linnaeus University

Danny Weyns

KU Leuven

Belgium

Daniel Deogun

Omegapoint AB, Sweden

NIER

Advanced systems such as IoT comprise many heterogeneous, interconnected, and autonomous entities operating in often highly dynamic environments. Due to their large scale and complexity, large volumes of monitoring data are generated and need to be stored, retrieved, and mined in a time- and resource-efficient manner. Architectural self-adaptation automates the control, orchestration, and operation of such systems. This can only be achieved via sophisticated decision-making schemes supported by monitoring data that fully captures the system behavior and its history.

Employing model-driven engineering techniques we propose a highly scalable, history-aware approach to store and retrieve monitoring data in form of enriched runtime models. We take advantage of rule-based adaptation where change events in the system trigger adaptation rules. We first present a scheme to incrementally check model queries in the form of temporal logic formulas which represent the conditions of adaptation rules against a runtime model with history. Then we enhance the model to retain only information that is temporally relevant to the queries, therefore reducing the accumulation of information to a required minimum. Finally, we demonstrate the feasibility and scalability of our approach via experiments on a simulated smart healthcare system employing a real-world medical guideline.

Lucas Sakizloglou

Hasso Plattner Institute, University of Potsdam

Germany

Sona Ghahremani

Hasso Plattner Institute, University of Potsdam

Germany

Thomas Brand

Matthias Barkowsky

Hasso Plattner Institute, University of Potsdam, Germany

Holger Giese

Hasso Plattner Institute, University of Potsdam

Germany

Ethical

Danny Weyns

KU Leuven

Belgium

Ethical

The main goal of any feedback control system is essentially to remove humans from the loop. This has always been the goal in the engineering of control systems. The MAPE-K loop is the embodiment of a feedback control loop in self-adaptive software systems, but the complete removal of humans from the control loop has not been thoroughly debated. One of the reasons is that, software systems are social-technical systems, and as such, humans need to be considered right from the inception of such systems, otherwise their deployment is bound to fail. However, as software self-adaptation progresses, enabling to place higher assurances on the deployment of these systems to the point humans become dispensable, some ethical questions need to be raised. Similar questions have been raised in past when the first automatic systems became intrinsic to the industrial fabric. The difference between then and now is that then the impact was confined to portions of the society, but now the implications are much wider, if we consider, in particular, software systems that are able to change themselves. If humans are not aware of those changes, and their implications, humans cease to be in tune with the system they are operating, and inevitably accidents will ensue. The point of no return in self-adaptive software systems refers to the moment in their technical maturity when any human involvement with the operation of a system is perceived to create more harm than benefit. Confronted with this situation, software engineers need start asking themselves some basic ethical questions. Do we really need to consider humans as an integral part of self-adaptive software systems? If humans are removed from the control loop, what kind of assurances will be needed for society to accept such systems?

Rogério de Lemos

University of Kent, UK

United Kingdom

MIP

Shang-Wen Cheng

Uber Advanced Technologies Group

United States

David Garlan

Carnegie Mellon University

United States

Bradley Schmerl

Carnegie Mellon University, USA

United States

MIP

Our SEAMS 2007 paper introduced sTile, an automated, fault-tolerant method for distributing computation in a way that preserves data confidentiality. Over the years, this work has inspired research in a diverse set of disciplines, from manufacturing systems, to vehicle network authentication, to smart grids that use blockchains. Our own work was influenced too, building secure, private, and fault-tolerant distributed and cloud systems.

Link to Publication: http://dx.doi.org/10.1109/SEAMS.2007.4

Yuriy Brun

University of Massachusetts Amherst

United States

Nenad Medvidović

University of Southern California

United States