Link to join: https://eu01web.zoom.us/j/61841934182?pwd=ZkpFVFhpTnZKcXhNbU4ydlUxRWl1Zz09
Research
Wed 15 Jun 2022 14:30 - 15:00 at Wednesday Track 2 session 2 (Tesla and online) - Research Track Virtual 2 Session 2 Chair(s): Aleksander Fabijan Microsoft, Rebekka Wohlrab Carnegie Mellon Universityno description available
Link to join: https://eu01web.zoom.us/j/61841934182?pwd=ZkpFVFhpTnZKcXhNbU4ydlUxRWl1Zz09
Research
Wed 15 Jun 2022 15:00 - 15:30 at Wednesday Track 2 session 2 (Tesla and online) - Research Track Virtual 2 Session 2 Chair(s): Aleksander Fabijan Microsoft, Rebekka Wohlrab Carnegie Mellon Universityno description available
Link to join: https://eu01web.zoom.us/j/61841934182?pwd=ZkpFVFhpTnZKcXhNbU4ydlUxRWl1Zz09
Research
Wed 15 Jun 2022 15:30 - 16:00 at Wednesday Track 2 session 2 (Tesla and online) - Research Track Virtual 2 Session 2 Chair(s): Aleksander Fabijan Microsoft, Rebekka Wohlrab Carnegie Mellon UniversityThe tremendous success of Deep Learning (DL) has significantly boosted the number of open-sourced DL frameworks hosted on GitHub. Among others, performance and accuracy bugs are critical factors that affect the reputation of these DL frameworks, therefore understanding the practice of discovering and investigating them for DL is important. In this paper, we conduct an exploratory study on the nature of reporting performance and accuracy bugs bugs for DL frameworks, aiming to improve our knowledge on this topic. Our study covers 10 most popular open-sourced DL frameworks on GitHub (e.g., TensorFlow, Keras, and PyTorch), based on which we sample 664 representative performance and accuracy bugs bug reports out of a total population of 22,522. Through systematic analysis of these samples, our key findings are: (1) low speed is the primary reason that a performance bug related report is submitted but we see no consistent pattern for accuracy related ones; (2) most of the reports are about issues encountered in the training stage; (3) only a small proportion of the reports provide insufficient information to investigate; (4) the majority of the performance and accuracy bugs bug reports (from 69% to 100%) are not related to the actual bug or regarded as unclassified; (5) around 50% of the performance and accuracy bug reports, which indeed reveal bugs, are not resolved by direct patches. Deriving from the above, we discuss a set of actionable implications to the researchers, maintainers, and report submitters on this subject. To promote open science, the labeled dataset has been made publicly available at https://zenodo.org/record/6371676.
Pre-printLink to join: https://eu01web.zoom.us/j/61841934182?pwd=ZkpFVFhpTnZKcXhNbU4ydlUxRWl1Zz09
Research
Wed 15 Jun 2022 16:30 - 17:00 at Wednesday Track 2 session 2 (Tesla and online) - Research Track Virtual 2 Session 3 Chair(s): Aleksander Fabijan Microsoft, Rebekka Wohlrab Carnegie Mellon UniversityContext: Citations are a key measure of scientific performance in most fields, including software engineering. However, there is limited research that studies which characteristics of articles’ metadata (title, abstract, keywords, and author list) are driving citations in this field. Objective: In this study, we propose a simple theoretical model for how citations come to be with respect to article metadata, we hypothesize theoretical linkages between metadata characteristics and citations of articles, and we empirically test these hypotheses. Method: We use multiple regression analyses to examine a data set comprising the titles, abstracts, keywords, and authors of 16,131 software engineering articles published between 1990 and 2020 in 20 highly influential software engineering venues. Results: We find that number of authors, number of keywords, number of question marks and dividers in the title, number of acronyms, abstract length, abstract propositional idea density, and corresponding authors in the core Anglosphere are significantly related to citations. Conclusion: Various characteristics of articles’ metadata are linked to the frequency with which the corresponding articles are cited. These results partially confirm and partially go counter to prior findings in software engineering and other disciplines.
DOI Pre-printLink to join: https://eu01web.zoom.us/j/61841934182?pwd=ZkpFVFhpTnZKcXhNbU4ydlUxRWl1Zz09
Research
Wed 15 Jun 2022 17:00 - 17:30 at Wednesday Track 2 session 2 (Tesla and online) - Research Track Virtual 2 Session 3 Chair(s): Aleksander Fabijan Microsoft, Rebekka Wohlrab Carnegie Mellon Universityno description available
Link to join: https://eu01web.zoom.us/j/61841934182?pwd=ZkpFVFhpTnZKcXhNbU4ydlUxRWl1Zz09
Industrial Track
Wed 15 Jun 2022 17:30 - 17:45 at Wednesday Track 2 session 2 (Tesla and online) - Industrial track session 3 Chair(s): Aleksander Fabijan Microsoft, Rebekka Wohlrab Carnegie Mellon UniversitySatisfactory software performance is essential for the adoption and the success of a product. In organizations that follow traditional software development models (e.g., waterfall), Software Performance Engineering (SPE) involves time-consuming experimental modeling and performance testing outside the actual production environment. Such existing SPE methods, however, are not optimized for environments utilizing Continuous Integration (CI) and Continuous Delivery (CD) that result in high frequency and high volume of code changes. We present a summary of lessons learned and propose improvements to the SPE process in the context of CI/CD. Our findings are based on SPE work on two products conducted over 5 years at a major online services company. We find that (a) SPE has mainly become a post hoc activity based on data from the production environment, (b) successful application of SPE techniques require frequent re-evaluation of priorities, and (c) engineers working on SPE require a broader skill set than one traditionally possessed by engineers working on performance.
Link to publication DOI Pre-printLink to join: https://eu01web.zoom.us/j/61841934182?pwd=ZkpFVFhpTnZKcXhNbU4ydlUxRWl1Zz09
Industrial Track
Wed 15 Jun 2022 17:45 - 18:00 at Wednesday Track 2 session 2 (Tesla and online) - Industrial track session 3 Chair(s): Aleksander Fabijan Microsoft, Rebekka Wohlrab Carnegie Mellon Universityno description available
Link to join: https://eu01web.zoom.us/j/61841934182?pwd=ZkpFVFhpTnZKcXhNbU4ydlUxRWl1Zz09
