Clustering test cases to achieve effective & scalable prioritisation incorporating expert knowledge

Abstract

Pair-wise com arison has been successfully utilised in order to rioritise test cases by ex loiting the rich, valuable and unique knowledge of the tester. However, the rohibitively large cost of the airwise com arison method revents it from being a lied to large test suites. In this a er, we introduce a cluster-based test case rioritisation technique. By clustering test cases, based on their dynamic runtime behaviour, we can reduce the required number of air-wise com arisons significantly. The a roach is evaluated on seven test suites ranging in size from 154 to 1,061 test cases. We resent an em irical study that shows that the resulting rioritisation is more effective than existing coverage-based rioritisation techniques in terms of rate of fault detection. Perha s sur risingly, the a er also demonstrates that clustering (even without human in ut) can out erform unclustered coverage-based technologies, and discusses an automated rocess that can be used to determine whether the a lication of the ro osed a roach would yield im rovement.