Research of automated document classification deals with three areas: expression of documents, creating a classifier and analyzing the classifier. The purpose of this paper is to produce an efficient classifier. More specifically, it is to suggest and test the performance of a specific method to generate classifiers which employ the multiple model machine learning schemes to improve the performance and reliability of the document classification. While current machine schemes rely on a single model generated from a training set, the multiple model scheme aim to improve the reliability of the predicted value by generating multiple learning models from the same training set, and combining the categorization of each model. This paper studies the various concepts of multiple model learning and the characteristics of each type while suggesting a hybrid multiple model scheme which can implement and improve the existing multiple model learning scheme. In addition, it will also test the performance of the classifier derived from the hybrid multiple model scheme via a categorization experiment of document sets.