An outlier map for support vector machine classification. (English) Zbl 1185.62112

Summary: Support vector machines are a widely used classification technique. They are computationally efficient and provide excellent predictions even for high-dimensional data. Moreover, support vector machines are very flexible due to the incorporation of kernel functions. The latter allow to model nonlinearity, but also to deal with non-numerical data such as protein strings. However, support vector machines can suffer a lot from unclean data containing, for example, outliers or mislabeled observations. Although several outlier detection schemes have been proposed in the literature, the selection of outliers versus non-outliers is often rather ad hoc and does not provide much insight in the data. In robust multivariate statistics outlier maps are quite popular tools to assess the quality of data under consideration. They provide a visual representation of the data depicting several types of outliers. This paper proposes an outlier map designed for support vector machine classification. The Stahel-Donoho outlyingness measure from multivariate statistics is extended to an arbitrary kernel space. A trimmed version of support vector machines is defined trimming the part of the samples with largest outlyingness. Based on this classifier, an outlier map is constructed visualizing data in any type of high-dimensional kernel spaces. The outlier map is illustrated on 4 biological examples showing its use in exploratory data analysis.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
68T05 Learning and adaptive systems in artificial intelligence
62P10 Applications of statistics to biology and medical sciences; meta analysis


Full Text: DOI arXiv


[1] Alon, U., Barkai, N., Notterman, D. A., Gish, K., Ybarra, S., Mack, D. and Levine, A. J. (1999). Broad patterns of gene expression revealed by clustering of tumor and normal colon tissues probed by oligonucleotide arrays. Proc. Natl. Acad. Sci. 96 6475-6750.
[2] Chiaretti, S., Li, X., Gentleman, R., Vitale, A., Vignetti, M., Mandelli, F., Ritz, J. and Foa, R. (2004). Gene expression profile of adult T-cell acute lymphocytic leukemia identifies distinct subsets of patients with different response to therapy and survival. Blood 103 2771-2778.
[3] Christmann, A. and Steinwart, I. (2004). On robustness properties of convex risk minimization methods for pattern recognition. J. Mach. Learn. Res. 5 1007-1034. · Zbl 1222.68348
[4] Donoho, D. L. (1982). Breakdown properties of multivariate location estimators. Qualifying paper. Harvard Univ.
[5] Furey, T. S., Cristianini, N., Duffy, D., Bednarski, W., Schummer, M. and Haussler, D. (2000). Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics 16 906-914.
[6] Jaakkola, T., Diekhans, M. and Haussler, D. (2000). A discriminative framework for detecting remote protein homologies. J. Comput. Biol. 7 95-114.
[7] Guyon, I., Weston, J., Barnhill, S. and Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Mach. Learn. 46 389-422. · Zbl 0998.68111
[8] Hubert, M. and Engelen, S. (2004). Robust PCA and classification in biosciences. Bioinformatics 20 1728-1736.
[9] Hubert, M., Rousseeuw, P. J. and Vanden Branden, K. (2005). ROBPCA: A new approach to robust principal component analysis. Technometrics 47 64-79.
[10] Kadota, K., Tominaga, D., Akiyama, Y. and Takahashi, K. (2003). Detecting outlying samples in microarray data: A critical assessment of the effect of outliers on sample classification. Chem-Bio. Inform. J. 3 30-45.
[11] Leslie, C., Eskin, E. and Noble, W. S. (2002). The spectrum kernel: A string kernel for svm protein classification. In Proceedings of the Pacific Symposium on Biocomputing 2002 (R. B. Altman, A. K. Dunker, L. Hunter, K. Lauerdale and T. E. Klein, eds.) 564-575. World Scientific, Hackensack, NJ.
[12] Leslie, C., Eskin, E., Weston, J. and Noble, W. S. (2003). Mismatch string kernels for svm protein classification. In Advances in Neural Information Processing Systems (S. Becker, S. Thrun and K. Obermayer, eds.) 15 1441-1448. MIT Press, Cambridge, MA.
[13] Li, L., Darden, T. A., Weinberg, C. R., Levine, A. J. and Pedersen, L. G. (2001). Gene assessment and sample classification for gene expression data using a genetic algorithm/k-nearest neighbor method. Comb. Chem. High Throughput Screen . 4 727-739.
[14] Liao, L. and Noble, W. S. (2002). Combining pairwise sequence similarity and support vector machines for remote protein homology detection. In Proceedings of the Sixth International Conference on Computational Molecular Biology (T. Lengauer, ed.) 225-232. ACM Press, New York.
[15] Malossini, A., Blanzieri, E. and Ng, R. T. (2006). Detecting potential labeling errors in microarrays by data perturbation. Bioinformatics 22 2114-2121.
[16] Maronna, R. and Yohai, V. (1995). The behavior of the Stahel-Donoho robust multivariate estimator. J. Amer. Statist. Assoc. 90 330-341. · Zbl 0820.62050
[17] Pochet, N., De Smet, F., Suykens, J. A. K. and De Moor, B. (2004). Systematic benchmarking of microarray data classification: Assessing the role of nonlinearity and dimensionality reduction. Bioinformatics 20 3185-3195.
[18] Pollack, J. D., Li, Q. and Pearl, D. K. (2005). Taxonomic utility of a phylogenetic analysis of phosphoglycerate kinase proteins of Archaea, Bacteria and Eukaryota: Insights by Bayesian analyses. Mol. Phylogenet. Evol. 35 420-430.
[19] Rousseeuw, P. J. and Van Zomeren, B. C. (1990). Unmasking multivariate outliers and leverage points. J. Amer. Statist. Assoc. 85 633-639.
[20] Saigo, H., Vert, J., Ueda, N. and Akutsul, T. (2004). Protein homology detection using string alignment kernels. Bioinformatics 20 1682-1689.
[21] Schölkopf, B. and Smola, A. (2002). Learning with Kernels . MIT Press, Cambridge, MA. · Zbl 1019.68094
[22] Stahel, W. A. (1981). Robuste Schätzungen: Infinitesimale optimalität und schätzungen von kovarianzmatrizen. Ph.D. thesis, ETH Zürich. · Zbl 0531.62036
[23] Steinwart, I. and Christmann, A. (2008). Support Vector Machines . Springer, New York. · Zbl 1203.68171
[24] Vapnik, V. (1998). Statistical Learning Theory . Wiley, New York. · Zbl 0935.62007
[25] West, M., Blanchette, C., Dressman, H., Huang, E., Ishida, S., Spang, R., Zuzan, H., Marks, J. R. and Nevins, J. R. (2001). Predicting the clinical status of human breast cancer by using gene expression profiles. Proc. Natl. Acad. Sci. 98 11462-11467.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.