Localising software faults by learning patterns of failing executions

Abstract

Many Fault Localisation (FL) techniques have been developed to alleviate the developers debugging cost. Spectrum-Based FL (SBFL) is known as the most effective family as a standalone FL technique. However, summarising program coverage information into program spectrum often results in huge loss of information such as 'which element is covered by which test?' or 'which elements are always executed together?'. In this work, we propose to preserve the original information as much as possible and let a classifier model learn the coverage pattern of failing tests. Once the training is done, we extract the suspiciousness scores from the trained classifier. Pursuing more effective learning from the coverage data, we use the random over-sampling method to mitigate the class imbalance problem and also try to reflect the execution frequency information in training set by using a new regularisation method. As a result, we found that our method can significantly increase the FL performance, especially when using an Artificial Neural Network classifier. On the Defect4J benchmark, some of our best-performing models can localise about 20 more faults at the top place than the state-of-the-art SBFL formulae such as Ochiai and Op2. We also found that hybridising our technique with well-performing SBFL formulae would further improve the overall FL performance than using each of them individually.