Test case prioritization based on information retrieval concepts

Abstract

Test case prioritization is one of the techniques used in regression testing. This technique determines an execution order of test cases in test suites so that the scheduled test cases can achieve certain goals such as maximizing code coverage or increasing fault detection rate. Traditionally, code coverage based prioritization techniques such as statement and method coverage have been used. These techniques are effective because a test case covering many lines is likely to execute many faulty lines. Therefore, there has been much research on coverage based prioritization techniques. However, there has been less research focusing on less tested code. Faults in the code might not be detected fast when coverage-based techniques are used. Also, there are faults that even a test case covering many lines cannot detect. These issues can be limitations of the existing coverage based prioritization techniques. This work, involving research overcoming the limitation of the coverage based techniques, utilizes Term Frequency (TF) and Inverted Document Frequency (IDF), which are document modeling concepts in the Information Retrieval (IR) field. In this work, an effective prioritization technique is suggested with the TF/IDF scores. This work explains how TF/IDF can fit into test case prioritization. The proposed approach gives TF and IDF scores to each covered element such as a line or branch. Then, each test case that covers the elements has TF/IDF scores. Next, the proposed approach schedules the test cases by using a combination of the total TF/IDF scores and the coverage scores of the test cases. To determine appropriate weights between TF/IDF and coverage scores, a set of mutants are used as training data. In our study, five open source applications are evaluated. The proposed approach is compared to baseline approaches such as a random ordering, statement, branch and additional statement techniques. The results demonstrate that the proposed approach is generally mo...