Abstract: Large Language Models (LLMs), such as ChatGPT, have shown impressive performance in various downstream tasks spanning diverse fields. In this talk, I will present our recent work on leveraging LLMs for improving software quality, covering techniques for breaking, fixing, and synthesizing software systems. More specifically, I will first talk about our TitanFuzz work, the first approach demonstrating that LLMs can be directly applied for both generation- and mutation-based fuzz testing studied for decades, while being fully automated, generalizable, and applicable to challenging application domains (such as ML systems). Next, I will talk about our AlphaRepair work, which reformulates the Automated Program Repair (APR) problem as an infilling (or cloze) task and demonstrates that LLMs can outperform all prior APR techniques studied for over a decade. Lastly, I will briefly talk about our recent EvalPlus work, which shows that the evaluation of almost all recent LLMs on program synthesis can be largely affected by the weak test suites in existing datasets. Furthermore, I will also briefly talk about our other work along the covered directions.

Lingming Zhang is an Associate Professor at the Department of Computer Science in University of Illinois Urbana-Champaign. His main research interests lie in Software Engineering, and its synergy with Machine Learning, Programming Languages, and Formal Methods. He has published over 80 research papers, winning the ACM SIGSOFT Early Career Researcher Award, four ACM SIGSOFT Distinguished Paper Awards, and one Best Industry Paper Award. His research has helped detect hundreds of bugs for open-source projects from Apache and GitHub, as well as software systems from eBay, eMetric, Google, Meta/Facebook, Microsoft, NVIDIA, OctoML, Oracle, and Yahoo!. His work on regression testing optimization has been run day-to-day in Google, while his work on automated program repair and unified debugging has been successfully deployed to the Alipay system with million lines of code and over 1 billion global users.

Lingming Zhang

University of Illinois at Urbana-Champaign