BigFuzz: Efficient Fuzz Testing for Data Analytics using Framework Abstraction
As big data analytics become increasingly popular, data-intensive scalable computing (DISC) systems help address the scalability issue of handling large data. However, there exists a lack of automated testing techniques to test such data-centric applications, because data is often incomplete, continuously evolving, and hard to know a priori. Fuzz testing has been proven to be highly effective in other domains such as security; however, it is nontrivial to apply such traditional fuzzing to big data analytics directly for three reasons: (1) the long latency of DISC systems prohibits the applicability of fuzzing: naïve fuzzing would spend 98% of the time in setting up a test environment; (2) conventional branch coverage is unlikely to scale to DISC applications because most binary code comes from the framework implementation such as Apache Spark; and (3) random bit or byte-level mutations can hardly generate meaningful data, which fails to reveal real-world application bugs.
We propose a novel coverage-guided fuzz testing tool for big data analytics, called BigFuzz. The key essence of our approach is that: (a) we focus on exercising application logic as opposed to increasing framework code coverage by abstracting the DISC frame-work using specifications. BigFuzz performs automated source to source transformations to construct an equivalent DISC application suitable for fast test generation, and (b) we design schema-aware data mutation operators based on our in-depth study of DISC application error types. BigFuzz speeds up the fuzzing time by 78X-1477X compared to random fuzzing, improves application code coverage by 20%-271%, and achieves 33%-157% improvement in detecting application errors. When compared to the state of the art that uses symbolic execution to test big data analytics, BigFuzz is applicable to twice more programs and can find 80.6% more bugs.
Wed 23 Sep Times are displayed in time zone: (UTC) Coordinated Universal Time
16:00 - 17:00: Testing (2)Research Papers at Kangaroo Chair(s): Alex GroceNorthern Arizona University | |||
16:00 - 16:20 Talk | TestMC: Testing Model Counters using Differential and Metamorphic TestingExperience Research Papers Muhammad UsmanUniversity of Texas at Austin, USA, Wenxi WangUniversity of Texas at Austin, USA, Sarfraz KhurshidUniversity of Texas at Austin, USA | ||
16:20 - 16:40 Talk | BigFuzz: Efficient Fuzz Testing for Data Analytics using Framework Abstraction Research Papers Qian ZhangUniversity of California, Los Angeles, Jiyuan WangUniversity of California, Los Angeles, Muhammad Ali GulzarUniversity of California at Los Angeles, USA, Rohan PadhyeCarnegie Mellon University, Miryung KimUniversity of California at Los Angeles, USA | ||
16:40 - 17:00 Talk | Scaling Client-Specific Equivalence Checking via Impact Boundary Search Research Papers Nick FengUniversity of Toronto, Vincent HuiUniversity of Toronto, Federico MoraUniversity of California, Berkeley, Marsha ChechikUniversity of Toronto |