SEGATE: Unveiling Semantic Inconsistencies between Code and Specification of String Inputs
Automated testing techniques are often assessed on coverage based metrics. However, despite giving good coverage, the test cases may miss the gap between functional specification and the code implementation. This gap may be subtle in nature, arising due to the absence of logical checks, either in the implementation or in the specification, resulting in inconsistencies in the input definition. The inconsistencies may be prevalent especially for structured inputs, commonly specified using string-based data types. Our study on defects reported over popular libraries reveals that such gaps may not be limited to input validation checks. We propose a test generation technique for structured string inputs where we infer inconsistencies in input definition to expose semantic gaps in the method under test and the method specification. We assess this technique using our tool SEGATE, Semantic Gap Tester. SEGATE uses static analysis and automaton modeling to infer the gap and generate test cases. On our benchmark dataset, comprising of defects reported in 15 popular open-source libraries, written in Java, SEGATE was able to generate tests to expose 80% of the defects.
Tue 12 Nov
13:40 - 15:20: Papers - Testing and Verification at Cortez 1 Chair(s): Weihang WangUniversity at Buffalo, SUNY | ||||||||||||||||||||||||||||||||||||||||||
13:40 - 14:00 Talk | Systematically Covering Input Structure Nikolas HavrikovCISPA Helmholtz Center for Information Security, Andreas ZellerCISPA Helmholtz Center for Information Security Pre-print | |||||||||||||||||||||||||||||||||||||||||
14:00 - 14:20 Talk | SEGATE: Unveiling Semantic Inconsistencies between Code and Specification of String Inputs Pre-print | |||||||||||||||||||||||||||||||||||||||||
14:20 - 14:40 Talk | Detecting Error-Handling Bugs without Error Specification Input Zhouyang JiaNational University of Defense Technology, Shanshan LiNational University of Defense Technology, Tingting YuUniversity of Kentucky, Xiangke LiaoNational University of Defense Technology, China, Ji WangNational University of Defense Technology, Xiaodong LiuNational University of Defense Technology, Yunhuai LiuPeking University | |||||||||||||||||||||||||||||||||||||||||
14:40 - 15:00 Talk | Test Automation and its Limitations Ahyoung SungSamsung Electronics, Yangsu KimSamsung Electronics, Sangjun KimSamsung Electronics, Jongin KimSamsung Electronics, Neo JangSamsung Electronics | |||||||||||||||||||||||||||||||||||||||||
15:00 - 15:10 Talk | Grading-Based Test Suite Augmentation Jonathan Osei-OwusuUniversity of Illinois at Urbana-Champaign, Angello AstorgaUniversity of Illinois at Urbana-Champaign, Liia ButlerUniversity of Illinois at Urbana-Champaign, Tao XiePeking University, Geoffrey ChallenUniversity of Illinois at Urbana-Champaign | |||||||||||||||||||||||||||||||||||||||||
15:10 - 15:20 Demonstration | MutAPK: Source-Codeless Mutant Generation for Android Apps Camilo Escobar-VelásquezUniversidad de los Andes, Michael Osorio-RiañoUniversidad de los Andes, Mario Linares-VásquezSystems and Computing Engineering Department , Universidad de los Andes , Bogotá, Colombia |