Abstract: When embracing new technology like LLMs, are we “throwing the baby out with the bath water”? What are we forgetting, from past research, that it is still relevant and useful? For example, I firmly believe that deep learning and generative AI methods such as ``chain of thought’ will dramatically change the nature of science (in general) and SE (in particular). But moving forward away from generative tasks to classification, regression, and optimization tasks, my experimental results strongly suggest that other non-neural methods can be just as effective, particularly when combined with hyperparameter optimization. This is an important point since the non-neural methods can yield the succinct symbolic models that humans need to review and understand a model.

Timothy Menzies (IEEE Fellow, Ph.D., UNSW, 1995) is a full Professor in CS at North Carolina State University where he teaches software engineering, automated software engineering, and foundations of software science. He is the directory of the RAISE lab (real world AI for SE) and the author of over 280 referred publications. In his career, he has been a lead researcher on projects for NSF, NIJ, DoD, NASA, USDA (funding totalling over 13 million dollars) as well as joint research work with private companies. Prof. Menzies is the editor-in-chief of the Automated Software Engineering journal and associate editor of IEEE Transactions on Software Engineering (and other leading SE journals).

Tim Menzies

North Carolina State University

United States

Full prof, ex-nurse,rocketman,taxi-driver,journalist (it all made sense at the time).

Software developed on public platform is a source of data that can be used to make predictions about those projects. While the individual developing activity may be random and hard to predict, the developing behavior on project level can be predicted with good accuracy when large groups of developers work together on software projects.

To demonstrate this, we use 64,181 months of data from 1,159 GitHub projects to make various predictions about the recent status of those projects (as of April 2020). We find that traditional estimation algorithms make many mistakes. Algorithms like k-nearest neighbors (KNN), support vector regression (SVR), random forest (RFT), linear regression (LNR), and regression trees (CART) have high error rates. But that error rate can be greatly reduced using hyperparameter optimization.

To the best of our knowledge, this is the largest study yet conducted, using recent data for predicting multiple health indicators of open-source projects.

Tianpei Xia

North Carolina State University

United States

Wei Fu

North Carolina State University

Rui Shu

North Carolina State University

United States

Rishabh Agrawal

North Carolina State University

Tim Menzies

North Carolina State University

United States

no description available

Tim Menzies

North Carolina State University

United States

Full prof, ex-nurse,rocketman,taxi-driver,journalist (it all made sense at the time).

Automatically generated static code warnings suffer from a large number of false alarms. Hence, developers only take action on a small percent of those warnings. To better predict which static code warnings should not be ignored, we suggest that analysts need to look deeper into their algorithms to find choices that better improve the particulars of their specific problem. Specifically, we show here that effective predictors of such warnings can be created by methods that locally adjust the decision boundary (between actionable warnings and others). These methods yield a new high water-mark for recognizing actionable static code warnings. For eight open-source Java projects (cassandra, jmeter, commons, lucene-solr, maven, ant, tomcat, derby) we achieve perfect test results on 4/8 datasets and, overall, a median AUC (area under the true negatives, true positives curve) of 92%.

Rahul Yedida

Hong Jin Kang

UCLA

United States

Huy Tu

North Carolina State University, USA

United States

Xueqi Yang

NCSU

United States

David Lo

Singapore Management University

Singapore

Tim Menzies

North Carolina State University

United States