Practical Combinatorial Interaction Testing: Empirical Findings on Efficiency and Early Fault Detection

Cited 57 time in webofscience Cited 74 time in scopus
  • Hit : 571
  • Download : 0
Combinatorial interaction testing (CIT) is important because it tests the interactions between the many features and parameters that make up the configuration space of software systems. Simulated Annealing (SA) and Greedy Algorithms have been widely used to find CIT test suites. From the literature, there is a widely-held belief that SA is slower, but produces more effective tests suites than Greedy and that SA cannot scale to higher strength coverage. We evaluated both algorithms on seven real-world subjects for the well-studied two-way up to the rarely-studied six-way interaction strengths. Our findings present evidence to challenge this current orthodoxy: real-world constraints allow SA to achieve higher strengths. Furthermore, there was no evidence that Greedy was less effective (in terms of time to fault revelation) compared to SA; the results for the greedy algorithm are actually slightly superior. However, the results are critically dependent on the approach adopted to constraint handling. Moreover, we have also evaluated a genetic algorithm for constrained CIT test suite generation. This is the first time strengths higher than 3 and constraint handling have been used to evaluate GA. Our results show that GA is competitive only for pairwise testing for subjects with a small number of constraints.
Publisher
IEEE COMPUTER SOC
Issue Date
2015-09
Language
English
Article Type
Article
Keywords

TEST SUITES; CONSTRAINTS; PRIORITIZATION

Citation

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, v.41, no.9, pp.901 - 924

ISSN
0098-5589
DOI
10.1109/TSE.2015.2421279
URI
http://hdl.handle.net/10203/203908
Appears in Collection
CS-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.
This item is cited by other documents in WoS
⊙ Detail Information in WoSⓡ Click to see webofscience_button
⊙ Cited 57 items in WoS Click to see citing articles in records_button

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0