Experience

[experience paper] Although tremendous efforts have been devoted to the quality assurance of online service systems, in reality, these systems still come across many incidents (i.e., unplanned interruptions and outages), which can decrease user satisfaction or cause economic loss. To better understand the characteristics of incidents and improve the incident management process, we perform the first large-scale empirical analysis of incidents collected from 18 real-world online service systems in a multinational technology company M. Surprisingly, we find that although a large number of incidents could occur over a short period of time, many of them actually do not matter, i.e., engineers will not fix them with a high priority after manually diagnosing their root cause. We call these incidents incidental incidents. Our qualitative and quantitative analyses show that incidental incidents are significant in terms of both number and cost. Therefore, it is important to prioritize incidents by identifying incidental incidents in advance to optimize incident management efforts. In particular, we propose an approach, called DeepIP (Deep learning based Incident Prioritization), to prioritizing incidents based on a large amount of historical incident data. More specifically, we design an attention-based CNN (Convolutional Neural Network) model to learn a prediction model to identify incidental incidents. We then prioritize all incidents by ranking the predicted probabilities of incidents being incidental. We evaluate the performance of DeepIP using real-world incident data. The experimental results show that DeepIP effectively prioritizes incidents by identifying incidental incidents and significantly outperforms all the compared approaches. For example, the AUC of DeepIP achieves 0.808, while that of the best compared approach is only 0.624 onaverage. Also, we share our experience and lessons learned from practice.

Junjie Chen

Tianjin University, China

China

Shu Zhang

Microsoft Research, Beijing

Xiaoting He

Microsoft

Qingwei Lin

Microsoft Research, China

China

Hongyu Zhang

University of Newcastle, Australia

Australia

Dan Hao

Peking University, China

China

Yu Kang

Microsoft Research, China

China

Feng Gao

Microsoft, China

Zhangwei Xu

Microsoft, China

Yingnong Dang

Microsoft, USA

United States

Dongmei Zhang

Microsoft Research, China

China

Bug reports in a repository are generally organized line by line in a list-view, with their titles and other meta-data displayed. In this list-view, a concise and precise title plays an important role that enables project practitioners to quickly and correctly digest the core idea of the bug, without carefully reading the corresponding details. However, the quality of bug report titles varies in open-source communities, which may be due to the limited time and unprofessionalism of authors. To help report authors efficiently draft good-quality titles, we propose a method, named iTAPE, to automatically generate titles for their bug reports. iTAPE formulates title generation into a one-sentence summarization task. By properly tackling two domain-specific challenges (i.e. lacking off-the-shelf dataset and handling the low-frequency human-named tokens), iTAPE then generates titles using a Seq2Seq-based model. A comprehensive experimental study shows that iTAPE can obtain fairly satisfactory results, in terms of the comparison with three latest one-sentence summarization works, as well as the feedback from human evaluation.

Songqiang Chen

School of Computer Science, Wuhan University

China

Xiaoyuan Xie

School of Computer Science, Wuhan University, China

Bangguo Yin

School of Computer Science, Wuhan University

Yuanxiang Ji

School of Computer Science, Wuhan University

Lin Chen

Nanjing University

China

Baowen Xu

State Key Laboratory for Novel Software Technology, Nanjing University

Graphical User Interface (GUI) provides a visual bridge between a software application and end users, through which they can interact with each other. With the development of technology and aesthetics, the visual effects of the GUI are more and more attracting. However, such GUI complexity posts a great challenge to the GUI implementation. According to our pilot study of crowdtesting bug reports, display issues such as text overlap, blurred screen, missing image always occur during GUI rendering on difference devices due to the software or hardware compatibility. They negatively influence the app usability, resulting in poor user experience. To detect these issues, we propose a novel approach, OwlEye, based on deep learning for modelling visual information of the GUI screenshot. Therefore, OwlEye can detect GUIs with display issues and also locate the detailed region of the issue in the given GUI for guiding developers to fix the bug. We manually construct a large-scale labelled dataset with 4,470 GUI screenshots with UI display issues and develop a heuristics-based data augmentation method for boosting the performance of our OwlEye. The evaluation demonstrates that our OwlEye can achieve 85% precision and 84% recall in detecting UI display issues, and 90% accuracy in localizing these issues. We also evaluate OwlEye with popular Android apps on Google Play and F-droid, and successfully uncover 57 previously-undetected UI display issues with 26 of them being confirmed or fixed so far.

Zhe Liu

Laboratory for Internet Software Technologies, Institute of Software Chinese Academy of Sciences, University of Chinese Academy of Sciences

China

Chunyang Chen

Monash University, Australia

Australia

Junjie Wang

Institute of Software, Chinese Academy of Sciences

Yuekai Huang

Institute of Software, Chinese Academy of Sciences

Jun Hu

Institute of Software, Chinese Academy of Sciences

Qing Wang

Institute of Software, Chinese Academy of Sciences

China