×

Polychotomous kernel Fisher discriminant via top-down induction of binary tree. (English) Zbl 1201.62082

Summary: In spite of the popularity of Fisher discriminant analysis in the realm of feature extraction and pattern classification, it is beyond the capability of Fisher discriminant analysis to extract nonlinear structures from the data. That is where the kernel Fisher discriminant algorithm sets in the scenario of supervised learning. In this article, a new trail is blazed in developing innovative and effective algorithms for polychotomous kernel Fisher discriminants with the capability of estimating the posterior probabilities, which is exceedingly necessary and significant in solving complex nonlinear pattern recognition problems arising from the real world.
Different from the conventional ‘divide-and-combine’ approaches to polychotomous classification problems, such as pairwise and one-versus-others, the method proposed herein synthesizes the multi-category classifier via the induction of top-to-down binary trees by means of a kernelized group clustering algorithm. The deficiencies inherited in the conventional multi-category kernel Fisher discriminant are surmounted and the simulations on a benchmark image data set demonstrate the superiority of the proposed approach.

MSC:

62H30 Classification and discrimination; cluster analysis (statistical aspects)
68T05 Learning and adaptive systems in artificial intelligence

Software:

STPRTool; PRMLT
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Schölkopf, B.; Smola, A. J., Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (2002), MIT Press: MIT Press Cambridge, MA
[2] Campbell, C., Kernel methods: a survey of current techniques, Neurocomputing, 48, 63-84 (2002) · Zbl 1006.68770
[3] S. Mika, G. Rätsch, J. Weston, B. Schölkopf, K. Müller, Fisher discriminant analysis with kernel, in: Proc. IEEE Int’l Workshop Neural Networks for Signal Processing IX, August 1999, pp. 41-48.; S. Mika, G. Rätsch, J. Weston, B. Schölkopf, K. Müller, Fisher discriminant analysis with kernel, in: Proc. IEEE Int’l Workshop Neural Networks for Signal Processing IX, August 1999, pp. 41-48.
[4] Cristianini, N.; Shawe-Taylor, J., An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods (2000), Cambridge University Press
[5] Kecman, V., Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models (2001), MIT Press · Zbl 0994.68109
[6] Schölkopf, B.; Smola, A.; Müller, K. R., Nonlinear component analysis as a kernel eigenvalue problem, Neural Computation, 10, 5, 1299-1319 (1998)
[7] Bishop, C., Pattern Recognition and Machine Learning (2006), Springer · Zbl 1107.68072
[8] Loog, M.; Duin, R. P.W.; Haeb-Umbach, R., Multiclass linear dimension reduction by weighted pairwise Fisher criteria, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 7, 762-766 (2001)
[9] Lu, Z.; Lin, F.; Ying, H., Design of decision tree via kernelized hierarchical clustering for multiclass support vector machines, Cybernetics and Systems, 38, 2, 187-202 (2007) · Zbl 1178.68438
[10] Pekalsak, E.; Paclik, P.; Duin, R. P.W., A generalized kernel approach to dissimilarity-based classification, Journal of Machine Learning Research, 2, 175-211 (2001) · Zbl 1037.68127
[11] Schölkopf, B., The kernel trick for distances, (Leen, T. K.; Diettrich, T. G.; Tresp, V., Advances in Neural Information Processing Systems, vol. 13 (2001), MIT Press), 301-307
[12] Eiter, T.; Mannila, H., Distance measures for point sets and their computation, Acta Informatica, 34, 109-133 (1997) · Zbl 0865.51011
[13] King, R.; Feng, C.; Shutherland, A., Statlog: comparison of classification algorithms on large real-world problems, Applied Artificial Intelligence, 9, 289-333 (1995)
[14] Vaclav Hlavac, Vojtech Franc, Statistical pattern recognition toolbox for MATLAB, Center for Machine Perception, Czech Technical University, Prague, Czech.; Vaclav Hlavac, Vojtech Franc, Statistical pattern recognition toolbox for MATLAB, Center for Machine Perception, Czech Technical University, Prague, Czech. · Zbl 1383.62175
[15] J. Weston, C. Watkins, Support vector machines for multi-class pattern recognition, in: Proceedings of the 7th European Symposium on Artificial Neural Networks, Bruges, Belgium, 1999.; J. Weston, C. Watkins, Support vector machines for multi-class pattern recognition, in: Proceedings of the 7th European Symposium on Artificial Neural Networks, Bruges, Belgium, 1999.
[16] N. Lawrence, B. Schölkopf, Estimating a kernel Fisher discriminant in the presence of label noise, in: Proceedings of the 18th International Conference on Machine Learning, San Francisco, 2001.; N. Lawrence, B. Schölkopf, Estimating a kernel Fisher discriminant in the presence of label noise, in: Proceedings of the 18th International Conference on Machine Learning, San Francisco, 2001.
[17] Lee, K. C.; Ou, J. S., Radar target recognition by using linear discriminant algorithm on angular-diversity RCS, Journal of Electromagnetic Waves and Applications, 21, 14, 2033-2048 (2007)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.