Kubernetes is a free, open-source container orchestration system for deploying and managing Docker containers that host microservices. Kubernetes cluster logs help in determining the reason for the failure. However, as systems become more complex, identifying failure reasons manually becomes more difficult and time-consuming. This study aims to identify effective and efficient classification algorithms to automatically determine the failure reason. We compare five classification algorithms, Support Vector Machines, K-Nearest Neighbors, Random Forest, Gradient Boosting Classifier, and Multilayer Perceptron. Our results indicate that Random Forest produces good accuracy while requiring fewer computational resources than other algorithms.
Analyzing Bug Life Cycles to Derive Practical Insights
Bug tracking systems define bug life cycles that outline their bug tracking process. In this study, we assess bug life cycles to identify bottlenecks in the bug tracking processes, and examine the effectiveness of bug tracking tracking system usage practices linked to bug states and state transitions. To achieve this, we examined the bug life cycles of three open-source software projects which use Bugzilla as their bug tracking system. In total, we have analyzed 106.196 bugs gathered from these projects. We started by looking at the temporal and quantitative aspects of these projects’ bug life cycles. After that, we collected data about how bug life cycles differ over time. Finally, we inspected the frequency of reopened and state-looping bugs in these projects. After our analysis, we have deduced that the presented temporal and quantitative analysis of bug life cycles is useful for finding bottlenecks and undesired behaviors in the bug tracking processes. We also inferred that examining the changes in bug life cycles over time can provide insights into how bug tracking practices changed throughout the project’s lifetime, and it can be used as a parameter to assess whether the bug tracking system usage has improved. Lastly, we deducted that analyzing undesired state trails’ frequency provides insights into the performance of bug tracking processes. Based on the insights gained from analyzing bug life cycles with the presented methods, we believe that decision makers can improve their workflow by introducing or removing new states to the bug life cycle and adding new rules and restrictions to their bug tracking process.
Automating Microservices Test Failure Analysis using Kubernetes Cluster Logs
Industry Experience Report
Kubernetes is a free, open-source container orchestration system for deploying and managing Docker containers that host microservices. Kubernetes cluster logs help in determining the reason for the failure. However, as systems become more complex, identifying failure reasons manually becomes more difficult and time-consuming. This study aims to identify effective and efficient classification algorithms to automatically determine the failure reason. We compare five classification algorithms, Support Vector Machines, K-Nearest Neighbors, Random Forest, Gradient Boosting Classifier, and Multilayer Perceptron. Our results indicate that Random Forest produces good accuracy while requiring fewer computational resources than other algorithms.
Many advanced program analysis and verification methods are based on solving systems of Constrained Horn Clauses (CHC). Testing CHC solvers is very important, as correctness of their work determines whether bugs in the analyzed programs are detected or missed. One of the well-established and efficient methods of automated software testing is fuzzing: analyzing the reactions of programs to random input data. Currently, there are no fuzzers for CHC solvers, and fuzzers for SMT solvers are not efficient in CHC solver testing, since they do not consider CHC specifics. In this paper, we present HornFuzz, a mutation-based gray-box fuzzing technique for detecting bugs in CHC solvers based on the idea of metamorphic testing. We evaluated our fuzzer on one of the highest performing CHC solvers, Spacer, and found a handful of bugs in Spacer. In particular, some discovered problems are so serious that they require fixes with significant changes to the solver.
Distributed state model inference for scriptless GUI testing
Abstract: State model inference of software applications through the Graphical User Interface (GUI) is a technique that identifies GUI states and transitions, and maps them into a model. Scriptless GUI testing tools can benefit substantially from the availability of these state models, for example, to improve the exploration, or have sophisticated test oracles. However, inferring models for large systems requires a long execution time. Our goal is to improve the speed of the state model inference process. To achieve this goal, this paper presents a distributed state model inference approach with an open source scriptless GUI testing tool. Moreover, in order to be able to infer a suitable model, we design a set of strategies to deal with abstraction challenges and to distinguish GUI states and transitions in the model. To validate it, we conduct an experiment with two open source web applications that have been tested with the distributed architecture using one to six Docker containers sharing the same state model. With the obtained results, we can conclude that it is feasible to infer a model with a distributed approach and that using the distributed approach reduces the time required for inferring a state model.
Requirements validation is an important aspect for ensuring high quality software. During requirements validation not the software product is validated but the requirements themselves. Therefore, much effort is spent on manual validation activities. Commonly used are requirements inspections, where the specification is read from different persons assuming different roles or applying different reading techniques partly accompanied by checklists. Actual defect detection with requirements inspection is costly and defect detection rates must be considered low. Therefore, repeated validation is used or validation with multiple inspection groups - known as N-fold inspections. However, this does not only yield more defects found but also more false positives. In this paper, we investigate how defect aggregation can be used to improve the overall quality of validation. Therefore, we conducted an experiment with 22 N-fold inspection groups consisting of four to five reviewers each. Results show that simple aggregation of all results leads to a number of false positives that can actually negatively impact the validation task, while the use of more tailored aggregation strategies can considerably improve the validation of requirements with N-fold inspections.
U Owns the Code That Changes and How Marginal Owners Resolve Issues Slower in Low-Quality Source Code
Markus Borg, Adam Tornhill and Enys Mones
[Context] Accurate time estimation is a critical aspect of predictable software engineering. Previous work shows that low source code quality increases the uncertainty in issue resolution times. [Objective] Our goal is to evaluate how developers’ project experience and file ownership are related to issue resolution times. [Method] We mine 40 proprietary software repositories and conduct an observational study. Using CodeScene, we measure source code quality and active development time connected to Jira issues. [Results] Most source code changes are made by either a marginal or dominant code owner. Also, most changes to low-quality source code are made by developers with low levels of ownership. In low-quality source code, marginal owners need 45% more time for small changes, and 93% more time for large changes. [Conclusions] Collective code ownership is a popular target, but industry practice results in many dominant and marginal owners. Marginal owners are particularly hampered when working with low-quality source code, which leads to productivity losses. In codebases plagued by technical debt, newly onboarded developers will require more time to complete tasks.
Assessing the Impact of File Ordering Strategies on Code Review Process
Industry Perspective Paper
Popular modern code review tools (e.g. Gerrit and GitHub) sort files in a code review in alphabetical order. A prior study (on open-source projects) shows that the changed files’ positions in the code review affect the review process. Their results show that files placed lower in the order have less chance of receiving reviewing efforts than the other files. Hence, there is a higher chance of missing defects in these files. This paper explores the impact of file order in the code review of the well-known industrial project IntelliJ IDEA. First, we verify the results of the prior study on a big proprietary software project. Then, we explore an alternative to the default Alphabetical order: ordering changed files according to their code diff. Our results confirm the observations of the previous study. We discover that reviewers leave more comments on the files shown higher in the code review. Moreover, these results show that, even with the data skewed toward Alphabetical order, ordering changed files according to their code diff performs better than standard Alphabetical order regarding placing problematic files, which needs more reviewing effort, in the code review. These results confirm that exploring various ordering strategies for code review needs more exploration.
