×

zbMATH — the first resource for mathematics

Least squares recursive projection twin support vector machine for classification. (English) Zbl 1234.68347
Summary: In this paper we formulate a least squares version of the recently proposed projection twin support vector machine (PTSVM) for binary classification. This formulation leads to extremely simple and fast algorithm, called least squares projection twin support vector machine (LSPTSVM) for generating binary classifiers. Different from PTSVM, we add a regularization term, ensuring the optimization problems in our LSPTSVM are positive definite and resulting better generalization ability. Instead of usually solving two dual problems, we solve two modified primal problems by solving two systems of linear equations whereas PTSVM need to solve two quadratic programming problems along with two systems of linear equations. Our experiments on publicly available datasets indicate that our LSPTSVM has comparable classification accuracy to that of PTSVM but with remarkably less computational time.

MSC:
68T05 Learning and adaptive systems in artificial intelligence
68T10 Pattern recognition, speech recognition
Software:
LIBLINEAR; UCI-ml; LIBSVM
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Cortes, C.; Vapnik, V.N., Support vector networks, Machine learning, 20, 273-297, (1995) · Zbl 0831.68098
[2] Burges, C., A tutorial on support vector machines for pattern recognition, Data mining and knowledge discovery, 2, 121-167, (1998)
[3] Vapnik, V.N., Statistical learning theory, (1998), Wiley New York · Zbl 0935.62007
[4] Cristianini, N.; Shawe-Taylor, J., An introduction to support vector machines: and other kernel-based learning methods, (2000), Cambridge University Press Cambridge, New York
[5] N.Y. Deng, Y.J. Tian, C.H. Zhang, Support Vector Machines: Theory, Algorithms and Extensions, CRC Press, 2012. · Zbl 1278.68011
[6] Noble, W.S., Kernel methods in computational biology, (), 71-92
[7] S. Lee, A. Verri, Pattern recognition with support vector machines, in: First International Workshop, Springer, Niagara Falls, Canada, 2002. · Zbl 0997.68682
[8] Ince, H.; Trafalis, T.B., Support vector machine for regression and applications to financial forecasting, ()
[9] Cortes, C.; Vapnik, V.N., Support vector networks, Machine learning, 20, 273-297, (1995) · Zbl 0831.68098
[10] Platt, J., Fast training of support vector machines using sequential minimal optimization, (), 185-208
[11] Joachims, T., Making large-scale support vector machine learning practical, advances in kernel methods: support vector learning, (1999), MIT Press Cambridge, MA
[12] C. Chang, C. Lin, LIBSVM: A Library for Support Vector Machines, Technical Report, Department of Computer Science and Information Engineering, National Taiwan University, 2001.
[13] Fan, R.E.; Chang, K.W.; Hsieh, C.J.; Wang, X.R.; Lin, C.J., LIBLINEAR: a library for large linear classification, Journal of machine learning research, 9, 1871-1874, (2008) · Zbl 1225.68175
[14] Suykens, J.A.K.; Lukas, L.; vanDooren, P.; DeMoor, B.; Vandewalle, J., Least squares support vector machine classifiers: a large scale algorithm, (), 839-842
[15] Suykens, J.A.K.; Vandewalle, J., Least squares support vector machine classifiers, Neural processing letter, 9, 3, 293-300, (1999)
[16] Fung, G.; Mangasarian, O., Multicategory proximal support vector machine classifiers, Machine learning, 59, 77-97, (2005) · Zbl 1101.68758
[17] Mangasarian, O.; Wild, E., Multisurface proximal support vector classification via generalize eigenvalues, IEEE transactions on pattern analysis and machine intelligence, 28, 1, 69-74, (2006)
[18] Jayadeva; Khemchandani, R.; Chandra, S., Twin support vector machines for pattern classification, IEEE transactions on pattern analysis and machine intelligence, 29, 5, 905-910, (2007)
[19] Shao, Y.H.; Zhang, C.H.; Wang, X.B.; Deng, N.Y., Improvements on twin support vector machines, IEEE transactions on neural networks, 22, 6, 962-968, (2011)
[20] Kumar, M.A.; Gopal, M., Least squares twin support vector machines for pattern classification, Expert systems with applications, 36, 4, 7535-7543, (2009)
[21] Ye, Q.; Zhao, C.; Ye, N.; Chen, Y., Multi-weight vector projection support vector machines, Pattern recognition letters, 31, 13, 2006-2011, (2010)
[22] Chen, X.; Yang, J.; Ye, Q.; Liang, J., Recursive projection twin support vector machine via within-class variance minimization, Pattern recognition, (2011), doi: 10.1016/j.patcog.2011.03.001 · Zbl 1218.68119
[23] Parlett, B., The symmetric eigenvalue problem, (1998), Society for Industrial Mathematics · Zbl 0885.65039
[24] Bishop, C., Pattern recognition and machine learning, (2006), Springer New York · Zbl 1107.68072
[25] Tao, Y.; Yang, J., Quotient vs. difference: comparison between the two discriminant criteria, Neurocomputing, 73, 10-12, 1808-1817, (2010)
[26] Li, H.; Jiang, T.; Zhang, K., Efficient and robust feature extraction by maximum margin criterion, IEEE transactions on neural networks, 17, 1, 157-165, (2006)
[27] Belhumeur, P.; Hespanha, J.; Kriegman, D., Eigenfaces vs. fisherfaces: recognition using class specific linear projection, (), 43-58
[28] Yang, J., Why can LDA be performed in PCA transformed space?, Pattern recognition, 36, 563-566, (2003)
[29] Kotsia, I.; Zafeiriou, S.; Pitas, I., Novel multiclass classifiers based on the minimization of the within-class variance, IEEE transactions on neural networks, 20, 14-34, (2009)
[30] Wang, X.; Chung, F.; Wang, S., On minimum class locality preserving variance support vector machine, Pattern recognition, 43, 8, 2753-2762, (2010) · Zbl 1213.68510
[31] Tao, Q.; Chu, D.; Wang, J., Recursive support vector machines for dimensionality reduction, IEEE transactions on neural networks, 19, 189-193, (2008)
[32] Xiang, C.; Fan, X.; Lee, T., Face recognition using recursive Fisher linear discriminant, IEEE transactions on image processing, 15, 8, 2097-2105, (2006)
[33] Mangasarian, O.; Musicant, D., Lagrangian support vector machines, Journal of machine learning research, 1, 161-177, (2001) · Zbl 0997.68108
[34] Golub, G.H.; Van Loan, C.F., Matrix computations, (1996), The John Hopkins University Press, p. 50 · Zbl 0865.65009
[35] C. Blake, C. Merz, UCI Repository of Machine Learning Databases, Department of Information and Computer Sciences, University of California, Irvine, \(\langle\)http://www.ics.uci.edu/∼mlearn/MLRepository.html〉, 1998.
[36] D.R. Musicant, NDC: Normally Distributed Clustered Datasets, 1998 \(\langle\)www.cs.wisc.edu/dmi/svm/ndc/〉.
[37] \(\langle\)http://www.mathworks.com〉, 2007.
[38] Duda, R.O.; Hart, P.E.; Stork, D.G., Pattern classification, (2001), John Wiley and Sons · Zbl 0968.68140
[39] Li, Y.J.; Liu, B.; Yang, X.W.; Fu, Y.Z.; Li, H.J., Multiconlitron: a general piecewise linear classifier, IEEE transactions on neural networks, 22, 2, 276-289, (2011)
[40] Ye, Q.L.; Zhao, C.X.; Zhang, H.F.; Ye, N., Distance difference and linear programming nonparallel plane classifier, Expert systems with applications, 38, 9425-9433, (2011)
[41] Jin, Z.; Yang, J.; Hu, Z.; Lou, Z., Face recognition based on the uncorrelated discriminant transformation, Pattern recognition, 34, 1405-1416, (2001) · Zbl 0978.68118
[42] Yang, J.; Yang, J.; Zhang, D., What’s wrong with the Fisher criterion?, Pattern recognition, 35, 11, 2665-2668, (2002) · Zbl 1006.68924
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.