×

zbMATH — the first resource for mathematics

Nonparallel plane proximal classifier. (English) Zbl 1157.68442
Summary: We observed that the two costly optimization problems of twin support vector machine (TWSVM) classifier can be avoided by introducing a technique as used in proximal support vector machine (PSVM) classifier. With this modus operandi we formulate a much simpler nonparallel plane proximal classifier (NPPC) for speeding up the training of it by reducing significant computational burden over TWSVM. The formulation of NPPC for binary data classification is based on two identical mean square error (MSE) optimization problems which lead to solving two small systems of linear equations in input space. Thus it eliminates the need of any specialized software for solving the quadratic programming problems (QPPs). The formulation is also extended for nonlinear kernel classifier. Our computations show that a MATLAB implementation of NPPC can be trained with a data set of 3 million points with 10 attributes in less than 3 s. Computational results on synthetic as well as on several bench mark data sets indicate the advantages of the proposed classifier in both computational time and test accuracy. The experimental results also indicate that performances of classifiers obtained by MSE approach are sufficient in many cases than the classifiers obtained by standard SVM approach.

MSC:
68T10 Pattern recognition, speech recognition
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Cortes, C.; Vapnik, V. N.: Support vector networks, Machine learning 20, No. 3, 273-297 (1995) · Zbl 0831.68098
[2] Vapnik, V.: The nature of statistical learning theory, (1995) · Zbl 0833.62008
[3] Cristianini, N.; Shawe-Taylor, J.: An introduction to support vector machines, An introduction to support vector machines 3 (2000) · Zbl 0994.68074
[4] Burges, C. J. C.: A tutorial on support vector machines for pattern recognition, Data mining knowledge discovery 2, No. 2, 121-167 (1998)
[5] S. Lee, A. Verri, Pattern recognition with support vector machines, in: First International Workshop, SVM 2002, Springer, Niagara Falls, Canada, 2002. · Zbl 0997.68682
[6] T. Joachims, C. Ndellec, C. Rouveriol, Text categorization with support vector machines: learning with many relevant features, in: Proceedings of the European Conference on Machine Learning (ECML), Berlin, 1998, pp. 137 – 142.
[7] Lin, D.; Cristianini, N.; Sugne, C.; Furey, T.; Ares, M.; Brown, M.; Grundy, W.; Haussler, D.: Knowledge-base analysis of microarray gene expression data by using support vector machines, PNAS, Knowledge-base analysis of microarray gene expression data by using support vector machines, PNAS 97 (2000)
[8] Noble, W. S.: Kernel methods in computational biology, support vector machine applications in computational biology, (2004)
[9] Ebrahimi, T.; Garcia, G. N.; Vesin, J. M.: Joint time-frequency-space classification of EEG in a brain – computer interface application, J. appl. Signal process. 1, No. 7, 713-729 (2003) · Zbl 1052.92036
[10] Lal, T. N.; Schroder, M.; Hinterberger, T.; Weston, J.; Bogdan, M.; Birbaumer, N.; Scholkopf, B.: Support vector channel selection in BCI, IEEE trans. Biomed. eng. 51, No. 6, 1003-1010 (2004)
[11] H. Ince, T.B. Trafalis, Support vector machine for regression and applications to financial forecasting, in: International Joint Conference on Neural Networks (IJCNN’02), Como, Italy, IEEE-INNS-ENNS, 2002.
[12] Hsu, C. J.; Chen, W. H.; Wuc, S.; Huang, Z.; Chen, H.: Credit rating analysis with support vector machines and neural networks: a market comparative study, Decision support systems 37, 543-558 (2004)
[13] Cristianini, N.; Taylor, J. Shawe: Kernel methods for pattern analysis, (2004) · Zbl 0994.68074
[14] Joachims, T.: Making large-scale support vector machine learning practical, Advances in kernel methods — support vector learning, 169-184 (1999)
[15] S. Haykin, Neural Networks — A Comprehensive Foundation, second ed., Pearson Education, 2006, Chapter 4, pp. 235 – 240.
[16] Suykens, J. A. K.; Vandewalle, J.: Least squares support vector machine classifiers, Neural process. Lett. 9, No. 3, 293-300 (1999) · Zbl 0958.93042
[17] Suykens, J. A. K.; Van Gestel, T.; De Brabanter, J.; De Moor, B.; Vandewalle, J.: Least squares support vector machines, (2002) · Zbl 1017.93004
[18] G. Fung, O.L. Mangasarian, Proximal support vector machine classifiers, in: 7th International Proceedings on Knowledge Discovery and Data Mining, 2001, pp. 77 – 86. · Zbl 1101.68758
[19] Evgeniou, T.; Pontil, M.; Poggio, T.: Regularization networks and support vector machines, Advances comput. Math. 1, No. 13, 1-50 (2000) · Zbl 0939.68098
[20] Evgeniou, T.; Pontil, M.; Poggio, T.: Regularization networks and support vector machines, Advances in large margin classifiers, 171-203 (2000) · Zbl 0939.68098
[21] Mangasarian, O. L.; Wild, E. W.: Multisurface proximal support vector classification via generalized eigenvalues, IEEE trans. Pattern anal. Machine intell. 28, No. 1, 69-74 (2006)
[22] Jayadeva; Khemchandani, R.; Chandra, S.: Twin support vector machines for pattern classification, IEEE trans. Pattern anal. Machine intell. 29, No. 5, 905-910 (2007) · Zbl 1329.68226
[23] Jain, A. K.; Duin, R. P. W.; Mao, J.: Statistical pattern recognition: a review, IEEE trans. Pattern anal. Machine intell. 22, No. 1, 4-37 (2000)
[24] Boyd, S.; Vandenberghe, L.: Convex optimization, (2002) · Zbl 1058.90049
[25] Nocedal, J.; Wright, S.: Numerical optimization, (2006) · Zbl 1104.65059
[26] Chambers, J.; Avlonities, A.: A robust mixed-norm adaptive filter algorithm, IEEE signal process. Lett. 4, No. 2, 46-48 (1997)
[27] Bazarra, M. S.; Sherali, H. D.; Shetty, C. M.: Nonlinear programming — theory and algorithms, (2004)
[28] Tikhonov, A. N.; Arsenin, V. Y.: Solutions of ill-posed problems, (1977) · Zbl 0354.65028
[29] Golub, G. H.; Van Loan, C. F.: Matrix computations, (1996) · Zbl 0865.65009
[30] Chua, K. S.: Efficient computations for large least square support vector machine classifiers, Pattern recognition lett. 24, 75-80 (2003) · Zbl 1053.68082
[31] W. Pao, L. Lan, D. Yang, The mixed norm proximal support vector classifier, Department of Electronics Engineering, National Yunlin University of Science & Technology, Taiwan.
[32] Y.-J. Lee, O.L. Mangasarian, RSVM: reduced support vector machines, Technical Report 00-07, Data Mining Institute, Computer Science Department, University Wisconsin, Madison, WI, USA, July 2000, Available from: \langle ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/00-07.ps\rangle .
[33] Checker data set \langle ftp://ftp.cs.wisc.edu/math-prog/cpo-dataset/machine-learn/checker\rangle .
[34] C.L. Blake, C.J. Merz, UCI Repository for Machine Learning Databases, Department of Information and Computer Sciences, University of California, Irvine, 1998 \langle http://www.ics.uci.edu/ mlearn/MLRepository.html\rangle .
[35] MATLAB, User’s Guide, The MathWorks, Inc., 1994 – 2001 \langle http://www.mathworks.com\rangle .
[36] G. Fung, O.L. Mangasarian, SVM toolbox home page \langle http://www.cs.wisc.edu/dmi/svm/psvm\rangle .
[37] S.R. Gunn, Support vector machine Matlab toolbox, 1998 \langle http://www.isis.ecs.soton.ac.uk/resources/svminfo/\rangle .
[38] LS-SVM toolbox, version-1.5 advanced \langle http://www.esat.kuleuven.ac.be/sista/lssvmlab/\rangle .
[39] Mitchell, T. M.: Machine learning, (1997) · Zbl 0913.68167
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.