zbMATH — the first resource for mathematics

Examples
Geometry Search for the term Geometry in any field. Queries are case-independent.
Funct* Wildcard queries are specified by * (e.g. functions, functorial, etc.). Otherwise the search is exact.
"Topological group" Phrases (multi-words) should be set in "straight quotation marks".
au: Bourbaki & ti: Algebra Search for author and title. The and-operator & is default and can be omitted.
Chebyshev | Tschebyscheff The or-operator | allows to search for Chebyshev or Tschebyscheff.
"Quasi* map*" py: 1989 The resulting documents have publication year 1989.
so: Eur* J* Mat* Soc* cc: 14 Search for publications in a particular source with a Mathematics Subject Classification code (cc) in 14.
"Partial diff* eq*" ! elliptic The not-operator ! eliminates all results containing the word elliptic.
dt: b & au: Hilbert The document type is set to books; alternatively: j for journal articles, a for book articles.
py: 2000-2015 cc: (94A | 11T) Number ranges are accepted. Terms can be grouped within (parentheses).
la: chinese Find documents in a given language. ISO 639-1 language codes can also be used.

Operators
a & b logic and
a | b logic or
!ab logic not
abc* right wildcard
"ab c" phrase
(ab c) parentheses
Fields
any anywhere an internal document identifier
au author, editor ai internal author identifier
ti title la language
so source ab review, abstract
py publication year rv reviewer
cc MSC code ut uncontrolled term
dt document type (j: journal article; b: book; a: book article)
Practical selection of SVM parameters and noise estimation for SVM regression. (English) Zbl 1075.68632
Summary: We investigate practical selection of hyper-parameters for support vector machines (SVM) regression (that is, $\epsilon$-insensitive zone and regularization parameter $C$). The proposed methodology advocates analytic parameter selection directly from the training data, rather than re-sampling approaches commonly used in SVM applications. In particular, we describe a new analytical prescription for setting the value of insensitive zone $\epsilon$, as a function of training sample size. Good generalization performance of the proposed parameter selection is demonstrated empirically using several low- and high-dimensional regression problems. Further, we point out the importance of Vapnik’s $\epsilon$-insensitive loss for regression problems with finite samples. To this end, we compare generalization performance of SVM regression (using proposed selection of $\epsilon$-values) with regression using `least-modulus’ loss ($\epsilon=0$) and standard squared loss. These comparisons indicate superior generalization performance of SVM regression under sparse sample settings, for various types of additive noise.

MSC:
68T05Learning and adaptive systems
Software:
ElemStatLearn
WorldCat.org
Full Text: DOI
References:
[1] Chapelle, O.; Vapnik, V.: Model selection for support vector machines. Advances in neural information processing systems 12 (1999)
[2] Cherkassky, V.; Ma, Y.: Comparison of model selection for regression. Neural computation 15, 1691-1714 (2003) · Zbl 1046.62001
[3] Cherkassky, V.; Mulier, F.: Learning from data: concepts, theory, and methods. (1998) · Zbl 0960.62002
[4] Cherkassky, V.; Shao, X.; Mulier, F.; Vapnik, V.: Model complexity control for regression using VC generalization bounds. IEEE transaction on neural networks 10, No. 5, 1075-1089 (1999)
[5] Drucker, H.; Burges, C.; Kaufman, L.; Smola, A.; Vapnik, V.: Support vector regression machines. Neural information processing systems 9, 155-161 (1997)
[6] Hastie, T.; Tibshirani, R.; Friedman, J.: The elements of statistical learning: data mining, inference and prediction. (2001) · Zbl 0973.62007
[7] Huber, P.: Robust estimation of a location parameter. Annals of mathematical statistics 35, 73-101 (1964) · Zbl 0136.39805
[8] Kwok, J. T.: Linear dependency between ${\epsilon}$ and the input noise in ${\epsilon}$-support vector regression. Lncs 2130, 405-410 (2001) · Zbl 1001.68888
[9] Mattera, D.; Haykin, S.: Support vector machines for dynamic reconstruction of a chaotic system. Advances in kernel methods: support vector machine (1999)
[10] Muller, K.; Smola, A.; Ratsch, G.; Schölkopf, B.; Kohlmorgen, J.; Vapnik, V.: Using support vector machines for time series prediction. Advances in kernel methods: support vector machine (1999)
[11] Schölkopf, B.; Bartlett, P.; Smola, A.; Williamson, R.: Support vector regression with automatic accuracy control. Perspectives in neural computing, 111-116 (1998)
[12] Schölkopf, B.; Burges, J.; Smola, A.: Advances in kernel methods: support vector machine. (1999) · Zbl 0935.68084
[13] Schölkopf, B.; Smola, A.: Learning with kernels: support vector machines, regularization, and beyond. (2002)
[14] Smola, A.; Murata, N.; Schölkopf, B.; Muller, K.: Asymptotically optimal choice of ${\epsilon}$-loss for support vector machines. Proceedings of ICANN 1998 (1998)
[15] Smola, A., & Schölkopf, B (1998). A tutorial on support vector regression. NeuroCOLT Technical Report NC-TR-98-030, Royal Holloway College, University of London, UK. · Zbl 0910.68189
[16] Vapnik, V.: Statistical learning theory. (1998) · Zbl 0935.62007
[17] Vapnik, V.: The nature of statistical learning theory. (1999) · Zbl 0928.68093
[18] Vapnik, V (2001). Personal communication.