One -class classification: Concept learning in the absence of counter-examples

This thesis treats the problem of one-class classification. It starts with an introduction of the problem of conventional, multi-class classification. Next, it explains the problem of one-class classification, where it is the goal to distinguish between objects from one class and all other possible objects. It is assumed that only examples of one of the classes, the target class, are available. The fact that no examples not belonging to the target class (outliers) are available, complicates the training of a one-class classifier. It is not enough to minimize the number of errors the classifier makes on the target set, but it is also required to minimize in some way the chance that it makes an error on the outlier data. One way to minimize the chance of error on the outlier data, is to minimize the volume what the one-class classifier covers in the feature space.

In the second chapter a new type of one-class classifier is presented, the support vector data description. It models the boundary of the target data by a hypersphere with minimal volume around the data. The boundary is described by a few training objects, the support vectors. Analogous to the support vector classifier, the support vector data description has the ability to replace normal inner products by kernel functions to obtain more flexible data descriptions. Furthermore, the fact that only the boundary of the data is modelled, makes the support vector classifier less dependent on the quality of the sampling of the target class. The SVDD can cope with situations in the exact density of the target class is unknown, but where just the area in the feature space can be estimated.

In chapters three and four several other one-class classifiers are investigated. Three types of one-class classifiers are considered, the density estimators, the boundary methods and the reconstruction methods. On several artificial datasets, each with their own distribution characteristics, the performance of these methods have been evaluated. Finally the one-class classifiers are applied to some real world problems, to investigate which data characteristics can be identified.

Inspired by the gains in performance which have be obtained when normal classifiers are combined, we looked at the possibilities of combining one-class classifiers in the last chapter. Due to the different characteristics of one-class classifiers with respect to conventional two-class classifiers, other results for combining classifiers is expected. One concern is that some one-class classifiers output not a posterior probability for the target class, but output a distance to a, model. This distance should be mapped to a posterior probability before the outputs of several one-class classifiers can be combined. The results of experiments show, that combining one-class classifiers often improve results, especially when classifiers trained on different feature sets are combined.