×

zbMATH — the first resource for mathematics

Spatially adaptive binary classifier using B-splines and total variation penalty. (English) Zbl 1434.62111
For classifier problems the computation of decision boundaries is considered, where for a theoretical analysis, the problem is restricted to a two-dimensional predictor space, hence, the computation of a decision curve \(g\). Based on certain training data, and using a spline approximation, the problem is reduced to the solution of a parameter optimization problem for the coefficients of the spline approximation. Here, the objective function is defined by the mean hinge loss with respect to given training data and an additive weighted norm term depending of the vector of spline-coefficients. The resulting optimization problem is numerically solved by a coordinate decent algorithm. Theoretical properties of the method are discussed, and numerical studies are given.
MSC:
62H30 Classification and discrimination; cluster analysis (statistical aspects)
65D07 Numerical computation using splines
62C25 Compound decision problems in statistical decision theory
Software:
e1071
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Abu-Mostafa, Y. S.; Magdon-Ismail, M.; Lin, H. T., Learning from data, (2012), New York, NY: AMLBook, New York, NY
[2] 1998‘Feature selection via concave minimization and support vector machines’, in Proceedings of the Fifteenth International Conference on Machine Learning, ICML ’98, San Francisco, CA. Morgan Kaufmann Publishers Inc., pp. 82-90
[3] Cristianini, N.; Shawe-Taylor, J., An introduction to support vector machines and other kernel-based learning methods, (2000), Cambridge: Cambridge University Press, Cambridge
[4] Dembo, A.; Zeitouni, O., Large deviations techniques and applications, (1998), New York: Springer, New York · Zbl 0896.60013
[5] Devore, R. A.; Popov, V. A., Interpolation of Besov spaces, Transactions of the American Mathematical Society, 305, 397-414, (1988) · Zbl 0646.46030
[6] Friedman, J.; Hastie, T.; Höfling, H.; Tibshirani, R., Pathwise coordinate optimization, The Annals of Applied Statistics, 1, 302-332, (2007) · Zbl 1378.90064
[7] Hastie, T.; Tibshirani, R.; Friedman, J., The elements of statistical learning, (2009), New York: Springer, New York
[8] Jhong, J. H.; Koo, J. Y.; Lee, S. W., Penalized B-spline estimator for regression functions using total variation penalty, Journal of Statistical Planning and Inference, 184, 77-93, (2017) · Zbl 1395.62080
[9] Koo, J. Y.; Lee, Y.; Kim, Y.; Park, C., A bahadur representation of the linear support vector machine, Journal of Machine Learning Research, 9, 1343-1368, (2008) · Zbl 1225.68191
[10] 1990‘Handwritten digit recognition with a back-propagation network’, in Advances in neural information processing systems, pp. 396-404
[11] Lee, Y.; Kim, Y.; Lee, S.; Koo, J. Y., Structured multicategory support vector machines with analysis of variance decomposition, Biometrika, 93, 555-571, (2006) · Zbl 1108.62059
[12] Mammen, S.; Van De Geer, E., Locally adaptive regression splines, The Annals of Statistics, 25, 387-413, (1997) · Zbl 0871.62040
[13] Massart, P., About the constants in Talagrand’s concentration inequalities for empirical processes, The Annals of Probability, 28, 863-884, (2000) · Zbl 1140.60310
[14] 2017e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien, . R package version 1.6-8
[15] Scholkopf, B.; Smola, A. J., Learning with kernels: support vector machines, regularization, optimization, and beyond, (2001), Cambridge: MIT Press, Cambridge
[16] Steinwart, I.; Christmann, A., Support vector machines, (2008), New York: Springer, New York
[17] Tarigan, B.; Van De Geer, S. A., Classifiers of support vector machine type with \(ℓ_1\) complexity regularization, Bernoulli, 12, 1045-1076, (2006) · Zbl 1118.62067
[18] Tsybakov, A. B., Introduction to Nonparametric Estimation, (2009), New York: Springer, New York · Zbl 1176.62032
[19] Weston, J.; Elisseeff, A.; Schölkopf, B.; Tipping, M., Use of the zero-norm with linear models and kernel methods, Journal of machine learning research, 3, 1439-1461, (2003) · Zbl 1102.68605
[20] Zhang, H. H.; Ahn, J.; Lin, X.; Park, C., Gene selection using support vector machines with non-convex penalty, Bioinformatics, 22, 88-95, (2005)
[21] 2004‘1-norm support vector machines’, in Advances in Neural Information Processing Systems, pp. 49-56
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.