Consensus proximal support vector machine for classification problems with sparse solutions.

*(English)*Zbl 1308.90105Summary: Classification problem is the central problem in machine learning. Support vector machines (SVMs) are supervised learning models with associated learning algorithms and are used for classification in machine learning. In this paper, we establish two consensus proximal support vector machines (PSVMs) models, based on methods for binary classification. The first one is to separate the objective functions into individual convex functions by using the number of the sample points of the training set. The constraints contain two types of the equations with global variables and local variables corresponding to the consensus points and sample points, respectively. To get more sparse solutions, the second one is \(l_1\)-\(l_2\) consensus PSVMs in which the objective function contains an \(\ell_1\)-norm term and an \(\ell_2\)-norm term which is responsible for the good classification performance while \(\ell_1\)-norm term plays an important role in finding the sparse solutions. Two consensus PSVMs are solved by the alternating direction method of multipliers. Furthermore, they are implemented by the real-world data taken from the University of California, Irvine Machine Learning Repository (UCI Repository) and are compared with the existed models such as \(\ell_1\)-PSVM, \(\ell_p\)-PSVM, GEPSVM, PSVM, and SVM-light. Numerical results show that our models outperform others with the classification accuracy and the sparse solutions.

##### MSC:

90C10 | Integer programming |

90C20 | Quadratic programming |

49M20 | Numerical methods of relaxation type |

65K05 | Numerical mathematical programming methods |

##### Keywords:

classification problems; support vector machine; proximal support vector machine; consensus; alternating direction method of multipliers
PDF
BibTeX
XML
Cite

\textit{Y.-Q. Bai} et al., J. Oper. Res. Soc. China 2, No. 1, 57--74 (2014; Zbl 1308.90105)

Full Text:
DOI

##### References:

[1] | Bagarinao, E.; Kurita, T.; Higashikubo, M.; Inayoshi, H., Adapting SVM image classifiers to changes in imaging conditions using incremental SVM: an application to car detection, Computer Vision, 5996, 363-372, (2010) |

[2] | Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and distributed computation: numerical methods. Prentice Hall, Englewood Cliffs (1989) · Zbl 0743.65107 |

[3] | Bishop, C.M.: Pattern recognition and machine learning. Springer, Heidelberg (2007) · Zbl 1107.68072 |

[4] | Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J., Distributed optimization and statistical learning via the alternating direction method of multipliers, Found. Trends Mach. Learn., 3, 1-122, (2010) · Zbl 1229.90122 |

[5] | Chakrabarty, B.; Li, B.G.; Nguyen, V.; Van Ness, R.A., Trade classification algorithms for electronic communications network trades, J. Bank. Financ., 31, 3806-3821, (2007) |

[6] | Chandrasekar, V.; Keränen, R.; Lim, S.; Moisseev, D., Recent advances in classification of observations from dual polarization weather radars, Atmos. Res., 119, 97-111, (2013) |

[7] | Chen, W.J.; Tian, Y.J., \(l\)_{\(p\)}-norm proximal support vector machine and its applications, Proc. Comput. Sci., 1, 2417-2423, (2012) |

[8] | Chew, H.G., Crisp, D., Bogner, R.E. et al.: Target detection in radar imagery using support vector machines with training size biasing. In: Proceedings of the sixth international conference on control, Automation, Robotics and Vision, Singapore (2000) |

[9] | Cortes, C.; Vapnik, V., Support-vector networks, Mach. Learn., 20, 273-297, (1995) · Zbl 0831.68098 |

[10] | Forero, P.A.; Cano, A.; Giannakis, G.B., Consensus-based distributed support vector machines, Journal of Machine Learning Research, 11, 1663-1707, (2010) · Zbl 1242.68222 |

[11] | Fung, G.M., Mangasarian, O.L.: Proximal support vector machine classifiers. Proc. Knowledge Discovery and Data Mining, Asscociation for Computing Machinery, NewYork:77-86 (2001) · Zbl 1101.68758 |

[12] | Han, J.W., Kamber, M., Pei, J.: Data mining: Concepts and techniques, third edition. Morgan Kaufmann, Burlington (2011) · Zbl 1445.68004 |

[13] | He, B.S.; Tao, M.; Yuan, X.M., Alternating direction method with Gaussian back substitution for separable convex programming, SIAM J. Optim., 22, 313-340, (2011) · Zbl 1273.90152 |

[14] | He, B.S.; Zhou, J., A modified alternating direction method for convex minimization problems, Appl. Math. Lett., 13, 123-130, (2000) · Zbl 0988.90020 |

[15] | Hernandez, J.C.H.; Duval, B.; Hao, J.K., SVM-based local search for gene selection and classification of microarray data, Bioinform. Res. Dev., 13, 499-508, (2008) |

[16] | Imam, T.; Ting, K.M.; Kamruzzaman, J., Z-SVM: an SVM for improved classification of imbalanced data, Adv. Artif. Intell., 4304, 264-273, (2006) |

[17] | Jayadeva, Khemchandani, R., Chandra, S.: Twin support vector machines for pattern classification. IEEE Trans. Pattern Anal. Mach. Intell. 29, 905-910 (2007) · Zbl 1329.68226 |

[18] | Lee, Y.J., Mangasarian, O.L.: RSVM: reduced support vector machines. In Proceedings of the First SIAM International Conference on Data Mining, CD-ROM Proceedings, Chicago (2001) |

[19] | Li, Y.K.; Shao, X.G.; Cai, W.S., A consensus least squares support vector regression(LS-SVR) for analysis of near-infrared spectra of plant samples, Tanlan, 72, 217-222, (2007) |

[20] | Lu, C.J.; Shao, Y.E.; Chang, C.L., Applying ICA and SVM to mixture control chart patterns recognition in a process, Advances in Neural Networks, 6676, 278-287, (2011) |

[21] | Mangasarian, O.L.; Wild, E.W., Multisurface proximal support vector machine classification via generalized eigenvalues, IEEE Trans. Pattern Anal. Mach. Intell. , 28, 69-74, (2006) |

[22] | Schölkopf, B.; Platt, J.; Shawe, T.J.; etal., Estimating the support of a high-dimensional distribution, Neural Comput., 13, 1443-1471, (2001) · Zbl 1009.62029 |

[23] | Schölkopf, B.; Smola, A.J.; Bartlett, P., New support vector algorithms, Neural Comput., 12, 1207-1245, (2000) |

[24] | Suykens, J.A.K.; Vandewalle, J., Least squares support vector machine classifiers, Neural Process. Lett., 9, 293-300, (1999) |

[25] | Tax, D.; Duin, R., Support vector domain description, Pattern Recognit. Lett., 20, 1191-1199, (1999) |

[26] | Üstün, T.B.: International classification systems for health. International Encyclopedia of Public Health:660-668 (2008) |

[27] | Vapnik, V.: Statistical learning theory. Wiley, New York (1998) · Zbl 0935.62007 |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.