Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems
Modern software systems, such as Cyber-Physical Systems (CPSs), often require interacting with environment or human. As environment and human are unpredictable or non-determinate, uncertainty is inevitable in such systems. In this paper, we proposed UncerTest to test such systems (i.e., CPSs in our context) with an explicit consideration of uncertainty, through test design, test generation, test optimization and test reporting. UncerTest is a model-based and search-based approach that rely on belief models — a test ready model with explicit subjective uncertainties. In UncerTest, we extended an uncertainty-wise test modeling framework (UncerTum) for designing and enabling of introduction of indeterminacy sources in the environment during test execution. To obtain tests, we first proposed an uncertainty-wise test generation with two strategies, designed for different coverage criteria of models, for enabling an automated test generations with uncertainties. In addition, we developed an uncertainty-wise test minimization to optimize the generated tests based on uncertainty related property using multi-objective search. We conducted an extensive empirical study with two phases to evaluate UncerTest with five use cases of two industrial CPS case studies. In the first phase, eight commonly used multi-objective search algorithms were selected for studying the best algorithm for each of the four minimization strategies. Next, the test cases obtained with UncerTest strategies (i.e., two generation strategies combined with four minimization strategies with the best algorithm) were executed on the two real CPSs for studying the performance of each UncerTest strategies. With the best strategy, we managed to observe 51% more uncertainties due to unknown indeterminate behaviors of the physical environments of the CPSs as compared to the other test strategies. In addition, with the same strategy, we managed to observe 118% more unknown uncertainties that are not specified in the test ready models.
Thu 14 Nov
16:00 - 17:40: Papers - Emerging Domains at Cortez 1 Chair(s): Joshua GarciaUniversity of California, Irvine | ||||||||||||||||||||||||||||||||||||||||||
16:00 - 16:20 Talk | Improving the Decision-Making Process of Self-Adaptive Systems by Accounting for Tactic Volatility Jeffrey PalmerinoRochester Institute of Technology, Qi YuRochester Institute of Technology, Travis Desell University of North Dakota, Daniel KrutzRochester Institute of Technology Pre-print | |||||||||||||||||||||||||||||||||||||||||
16:20 - 16:40 Talk | Learning-Guided Network Fuzzing for Testing Cyber-Physical System Defences Yuqi ChenSingapore University of Technology and Design, Singapore, Chris PoskittSingapore University of Technology and Design, Jun SunSingapore Management University, Singapore, Sridhar AdepuSingapore University of Technology and Design, Singapore, Fan ZhangZhejiang University, Zhejiang Lab, and Alibaba-Zhejiang University Joint Institute of Frontier Technologies, China Pre-print File Attached | |||||||||||||||||||||||||||||||||||||||||
16:40 - 17:00 Talk | Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems Man ZhangKristiania University, Shaukat AliSimula Research Lab, Tao YueNanjing University of Aeronautics and Astronautics & Simula Research Laboratory Link to publication | |||||||||||||||||||||||||||||||||||||||||
17:00 - 17:20 Talk | Finding Trends in Software Research George MathewDepartment of Computer Science, North Carolina State University, Amritanshu AgrawalWayfair, Tim MenziesNorth Carolina State University Link to publication | |||||||||||||||||||||||||||||||||||||||||
17:20 - 17:30 Demonstration | XRaSE: Towards Virtually Tangible Software using Augmented Reality Rohit MehraAccenture Labs, India, Vibhu Saujanya SharmaAccenture Labs, Vikrant KaulgudAccenture Labs, India, Sanjay PodderAccenture | |||||||||||||||||||||||||||||||||||||||||
17:30 - 17:40 Demonstration | MuSC: A Tool for Mutation Testing of Ethereum Smart Contract Zixin LiNanjing University, Haoran WuState Key Laboratory for Novel Software Technology, Nanjing University, Jiehui XuNanjing University, Xingya WangState Key Laboratory for Novel Software Technology, Nanjing University, Lingming ZhangThe University of Texas at Dallas, Zhenyu ChenNanjing University |