Improving software reliability prediction through multi-criteria based dynamic model selection and combination

Abstract

In spite of much research efforts to develop software reliability models, there is no single model which is appropriate in all circumstances. Accordingly, some recent studies on software reliability have attempted to use existing models more effectively in practice (e.g., model selection and combination). However, it is not easy to identify which model is likely to make the most trustworthy predictions and to assign appropriate weights to models for the combination. The improper model selection or weight assignment often causes unsuccessful software reliability prediction in practice, which leads to cost/schedule overrun. In this paper, we propose a systematic reliability prediction framework which dynamically selects and combines multiple software reliability models based on the decision trees learning of multi-criteria. For the model selection, the proposed approach uses the empirical patterns of multi-criteria derived from models. Reduced error pruning decision tree identifies the models with the best predictive patterns and automatically assign a weight to each model. Then, the identified models fall into two groups according to the likelihood of over- or under-prediction, and the competitive models from each group are combined based on their given weights. From the evaluation results, our approach outperformed existing methods on average prediction accuracy.