ITEQS 2020
Sat 24 Oct 2020 Porto, Portugal
co-located with ICST 2020

4th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems (ITEQS) Co-located with the 13th IEEE International Conference on Software Testing, Verification and Validation (ICST 2020), Porto - Portugal.

All information about this workshop can be found at the workshop’s official website:

ITEQS 2020 is held as a virtual workshop via Zoom. Contact for the details.

You're viewing the program in a time zone which is different from your device's time zone -

Sat 24 Oct
Times are displayed in time zone: Greenwich Mean Time : Lisbon

09:15 - 10:30: Session IITEQS 2020 at São João
Chair(s): Mehrdad SaadatmandRISE Research Institutes of Sweden

ITEQS 2020 is held as a virtual workshop via Zoom. Contact for the details.

09:15 - 09:30
Day opening
Welcome and opening message
ITEQS 2020
09:30 - 10:00
Automatic exploratory performance testing using a discriminator neural network
ITEQS 2020
Ivan PorresÅbo Akademi University, Tanwir AhmadÅbo Akademi University, Hergys RexhaÅbo Akademi University, Sebastien LafondÅbo Akademi University, Dragos TruscanÅbo Akademi University
Link to publication DOI
10:00 - 10:30
Performance Evaluation of Container's Shared Volumes
ITEQS 2020
Naylor GarciaUniversity of São Paulo - USP, Paulo Sergio Lopes de SouzaUniversity of São Paulo - USP, Sarita M. BruschiUniversity of São Paulo - USP, Simone do Rocio Senger de SouzaUniversity of São Paulo - USP
Link to publication DOI
11:00 - 12:30: Session IIITEQS 2020 at São João
Chair(s): Eduard Paul EnoiuMälardalen University

ITEQS 2020 is held as a virtual workshop via Zoom. Contact for the details.

11:00 - 11:30
Prioritizing Scenarios based on STAMP/STPA Using Statistical Model Checking
ITEQS 2020
Mitsuaki TsujiNara Institute of Science and Technology, Toshinori TakaiNara Institute of Science and Technology, Kazuki KakimotoNara Institute of Science and Technology, Naoki IshihamaJapan Aerospace Exploration Agency, Masafumi KatahiraJapan Aerospace Exploration Agency, Hajimu IidaNara Institute of Science and Technology
Link to publication DOI
11:30 - 12:00
Generating Tests for the Discovery of Security Flaws in Product Variants
ITEQS 2020
Francisco AraújoLASIGE, Faculdade de Ciências da Universidade de Lisboa, Ibéria MedeirosLaSIGE, Faculdade de Ciências da Universidade de Lisboa, Nuno NevesDI FC UL
Link to publication DOI
12:00 - 12:30
Test Data Generation for False Data Injection Attack Testing in Air Traffic Surveillance *Best Paper*
ITEQS 2020
Aymeric CretinFemto-ST Institute, Alexandre VernotteFemto-ST Institute, Antoine ChevrotFemto-ST Institute, Fabien PeureuxInstitut FEMTO-ST (UMR CNRS 6174) and Smartesting S&S, Bruno LegeardFEMTO-ST / DISC
Link to publication DOI
14:00 - 15:30: Session IIIITEQS 2020 at São João
Chair(s): Dragos TruscanÅbo Akademi University

ITEQS 2020 is held as a virtual workshop via Zoom. Contact for the details.

14:00 - 15:00
Security Testing using Artificial Intelligence Methods and Techniques
ITEQS 2020
Franz WotawaTechnische Universitaet Graz
15:00 - 15:30
Keywords-based test categorization for Extra-Functional Properties
ITEQS 2020
Muhammad AbbasRISE Research Institutes of Sweden, Abdul RaufRISE Research Institutes of Sweden, Mehrdad SaadatmandRISE Research Institutes of Sweden, Eduard Paul EnoiuMälardalen University, Daniel SundmarkMälardalen University
Link to publication DOI
15:30 - 15:40
Day closing
Best Paper Award & Closing
ITEQS 2020

Call for Submissions

The rapid development towards increased integration of software with the social and physical world that we see today means that quality aspects such as performance, safety, security, and robustness become more important in an increasing number of the systems and devices, which we use and depend on. In this context, the success of a software product may not only depend on the logical correctness of its functions, but also on the system quality characteristics. Such system characteristics, which are referred to and captured as Extra-Functional Properties (EFPs) or Non-Functional Properties, are particularly important in resource constrained systems such as in the domains of real-time embedded and cyber-physical systems. Therefore, such systems need to be tested with a special attention to the EFPs. Testing EFPs is challenging and often requires different approaches compared to testing normal functionality. ITEQS provides a focused forum with the goal of bringing together researchers and practitioners to share ideas, identify challenges, propose solutions and techniques, and in general expand the state of the art in testing EFPs and quality characteristics of software systems and services. The workshop endorses contributions in a wide range of topics related to testing of EFPs in the form of full papers and short yet solid work-in-progress/position papers.

Note: The workshop does not accept papers that focus purely on functional testing!


  • Model-based testing of EFPs; e.g., choice of modeling languages to capture EFPs and their role on testability, model-based test case generation, etc.
  • Performance, Robustness, and Security Testing
  • Mutation-based testing for EFPs; e.g., application of mutation techniques for testing of EFPs particularly introduction of EFP-specific mutation operators
  • Testing of AI and Machine Learning based systems with respect to quality attributes; e.g., such as safety, security, and ethical concerns
  • Testing quality in software product lines and variant-intensive system
  • Testing quality characteristics of distributed, mobile, and cloud applications
  • Search-based testing techniques for EFPs
  • Testability, observability, controllability and the role of the platform; e.g., how the choice of operating system can impact testability of EFPs, for instance, a real-time operating system, introducing testability mechanisms into a platform, designing - middlewares for testing of EFPs
  • Empirical studies and experience reports; e.g., on the importance of testing EFPs, evaluation of testing methods, case-study and reports on project failures due to EFPs, comparison of methods and techniques
  • Quality assurance, standards, and their impact on testing EFPs
  • Requirements and testing EFPs; e.g., identification and generation of test oracles for EFPs from requirements, requirements for testability, traceability
  • Coverage criteria in testing EFPs
  • Processes and their role in testing EFPs; e.g., agile and TDD
  • Fault localization for EFPs and debugging
  • Formal methods, model-checking, and reasoning about EFPs
  • Parallelism, Concurrency, and Testing of multicore applications
  • Testing real-time, embedded, and cyber-physical systems, and their challenges

Submission Guidelines

Paper format:

  1. full papers 6-10 pages
  2. solid work-in-progress and position papers 4 pages; in IEEE double column format.

Submission site: Accepted papers will be published in IEEE Digital Library.