Hybrid Deep Neural Networks to Infer State Models of Black-Box Systems
Specification mining, in general, and inferring behavior model of a running system, in particular, are quite useful for several automated software engineering tasks, such as program comprehension, anomaly detection, and testing. Most existing dynamic model inference techniques are white-box, i.e., they require source code to be instrumented to get run-time traces. However, in many systems, access to source code is not possible for parts of the program that use third-party binaries and off-the-shelf-components. One useful scenario for automated black-box behaviour inference is in software control units (such as autopilots), where the software system’s reactions over time changes based on the inputs. Run-time state models of such systems are very powerful means for anomaly detection and debugging. Unfortunately, most black-box techniques that detect state changes over time are either uni-variate (which is limiting the application in real-world systems) or are weak with respect to learning from past behaviour. Therefore, in this paper, we propose a hybrid deep neural network that accepts as input a set of time series, one per input signal of the system, and applies a set of convolution and recurrent layers to both learn the non-linear correlations between signals and the patterns over time. We have applied our approach to a real UAV auto-pilot solution from our industry partner with half a million lines of C code. We ran 888 random recent test cases of the system and inferred states over time. We compared our results with several traditional time series change point detection techniques and showed that our approach can improve their performance 88% to 102%, in terms of finding state change points, measured by F1 score. We also showed that our state classification algorithm provides on average 90.45% F1 score, which improves traditional classification algorithms 7% to 17%.
Tue 22 Sep Times are displayed in time zone: (UTC) Coordinated Universal Time
17:10 - 18:10: AI for Software Engineering (1)Research Papers / NIER track at Koala Chair(s): Tingting YuUniversity of Kentucky | |||
17:10 - 17:30 Talk | DeepTC-Enhancer: Improving the Readability of Automatically Generated Tests Research Papers Devjeet RoyWashington State University, Ziyi ZhangWashington State University, Maggie MaWashington State University, Venera ArnaoudovaWashington State University, Annibale PanichellaDelft University of Technology, Sebastiano PanichellaZurich University of Applied Sciences, Danielle GonzalezRochester Institute of Technology, USA, Mehdi MirakhorliRochester Institute of Technology | ||
17:30 - 17:50 Talk | Hybrid Deep Neural Networks to Infer State Models of Black-Box Systems Research Papers Pre-print | ||
17:50 - 18:00 Talk | On Benign Features in Malware Detection NIER track Michael CaoThe University of British Columbia, Sahar BadihiUniversity of British Columbia, Canada, Khaled AhmedThe University of British Columbia, Peiyu XiongThe University of British Columbia, Julia RubinUniversity of British Columbia, Canada |