Automated online recognition of unexpected conditions is an indispensable component of autonomous vehicles to ensure safety even in unknown and uncertain situations. In this paper we propose a runtime monitoring technique rooted in the attention maps computed by explainable artificial intelligence techniques. Our approach, implemented in a tool called ThirdEye, turns attention maps into confidence scores that are used to discriminate safe from unsafe driving behaviours. The intuition is that uncommon attention maps are associated with unexpected runtime conditions.

In our empirical study, we evaluated the effectiveness of different configurations of ThirdEye at predicting simulation-based injected failures induced by both unknown conditions (adverse weather and lighting) and unsafe/uncertain conditions created with mutation testing. Results show that, overall, ThirdEye can predict 98% misbehaviours, up to three seconds in advance, outperforming a state-of-the-art failure predictor for autonomous vehicles.

Andrea Stocco

Università della Svizzera italiana (USI)

Switzerland

Paulo J. Nunes

Federal University of Pernambuco

Marcelo d'Amorim

Federal University of Pernambuco

Brazil

Paolo Tonella

USI Lugano

Switzerland

Self-driving cars are perceived as one of the most impactful applications of machine learning in society. At the core of these driverless vehicles are AI computing infrastructures built on deep neural networks that should be able to make thousands of reliable predictions in real time. This talk focuses on the peculiarities that concern the testing of deep neural networks for autonomous driving including topics such as offline and online testing, simulated vs real-world testing, and virtual and physical testing. We review the main research contributions in these areas and discuss some of the challenges and open problems to be addressed.

Andrea Stocco

Università della Svizzera italiana (USI)

Switzerland

My interests concern devising techniques for testing web- and AI-based software systems. Over the years, I focused on solving problems related to different aspects of regression testing, such as test suite minimization, or improving the robustness and maintainability of test suites for web applications.