Autonomous Driving Systems (ADSs) are safety-critical, and must be fully tested before being deployed on real-world roads. To comprehensively evaluate the performance of ADSs, it is essential to generate various safety-critical scenarios. Most of existing studies assess ADSs either by searching high dimensional input space, or using simple and pre-defined test scenarios, which are not efficient or not adequate. To better test ADSs, this paper proposes to automatically generate safety-critical test scenarios for ADSs by influential behavior patterns, which are mined from real traffic trajectories. Based on influential behavior patterns, a novel scenario generation technique, CRISCO, is presented to generate safety-critical scenarios for ADSs testing. CRISCO assigns participants to perform influential behavior patterns to challenge the ADS. It generates diverse test scenarios by solving a group of trajectory constraints, and improves the challenge of those non-critical scenarios by adding participants’ behavior from influential behavior patterns incrementally. We demonstrate CRISCO on an industrial-grade full-stack ADS platform, Baidu Apollo. The experiment results show that our approach can effectively and efficiently generate safety-critical scenarios to crash ADS, and it exposes 13 distinct types of safety violations in 12 hours. It also outperforms two state-of-art ADS testing techniques by exposing more 5 distinct types of safety violations on the same road.

Haoxiang Tian

Institute of Software, Chinese Academy of Sciences

China

Guoquan Wu

Institute of Software at Chinese Academy of Sciences, China

Jiren Yan

Institute of Software, Chinese Academy of Sciences

Yan Jiang

Institute of Software, Chinese Academy of Sciences

Jun Wei

Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences

China

Wei Chen

Institute of Software at Chinese Academy of Sciences, China

Shuo Li

Institute of Software, Chinese Academy of Sciences

Dan Ye

Institute of Software, Chinese Academy of Sciences