Optimization of Classifiers using Genetic Programming

Abstract

The success of pattern classification system depends on the improvement of its classification stage. The work of thesis has investigated the potential of Genetic Programming (GP) search space to optimize the performance of various classification models. In this thesis, two GP approaches are proposed. In the first approach, GP is used to optimize the performance of individual classifiers. The performance of linear classifiers and nearest neighbor classifiers is improved during GP evolution to develop a high performance numeric classifier. In second approach, component classifiers are trained on the input data and their predictions are extracted. GP search space is then used to combine the predictions of component classifiers to develop an optimal composite classifier (OCC). This composite classifier extracts useful information from its component classifiers during evolution process. In this way, the decision space of composite classifier is more informative and discriminant. Effectiveness of GP combination technique is investigated for four different types of classification models including linear classifiers, support vector machines (SVMs) classifiers, statistical classifiers and instance based nearest neighbor classifiers. The successfulness of such composite classifiers is demonstrated by performing various experiments, while using Receiver Operating Characteristics (ROC) curve as the performance measure. It is evident from the experimental results that OCC outperforms its component classifiers. It attains high margin of improvement at small feature sets. Further, it is concluded that classification models developed by heterogeneous combination of classifiers have more promising results than their homogenous combination. GP optimization technique automatically caters the selection of suitable component classifiers and model selection. Two main objectives are achieved, while using GP optimization. First, objective achieved is the development of more optimal classification models. The second one is the enhancement in the GP search strategy itself.