Comparisons of classification methods in the original and pattern spaces and development of new pattern selection approaches for the logical analysis of data

Abstract

The logical analysis of data (LAD) is one of promising data mining and machine learning techniques to extract knowledge from data. The LAD was developed based on the concepts from combinatorics, optimization, and Boolean functions. The main steps of the LAD are composed of data binarization, support set construction, pattern generation and selection, and theory formulation. The key feature of the LAD is the capability of detecting hidden patterns in the data. Patterns are basically combinations of certain attributes and they are used to build a decision boundary for classification in the LAD. The patterns can provide important information to distinguish observations in one class from those in the other class. The use of patterns may result in more stable performance for the classification of both positive and negative classes due to their robustness to measurement errors. In addition, the patterns are interpretable and can serve as an essential tool for understanding the problem. Desirable properties of the patterns generated from the LAD motivate the use of the LAD patterns as input variables to other classification techniques to achieve more stable and accurate performance. In the first part of this thesis, the patterns generated from the LAD are used as the input variables to the decision tree and k-nearest neighbor classification methods. The applicability and usefulness of the LAD patterns for classification are investigated by experimental study. The classification results for different classifiers in the original and pattern spaces are compared using several public data sets in terms of classification accuracy and sensitivity. Comparisons of the LAD and other classification methods in the pattern space are also made using the same data sets to examine the effect of the LAD after the completion of the pattern generation step. The experimental results show that classifications in the pattern space can yield better performance than in the original space...