zbMATH — the first resource for mathematics

Nonparametric regression function estimation using interaction least squares splines and complexity regularization. (English) Zbl 1093.62528
Summary: Let \((X,Y)\) be a pair of random variables with \(\text{supp}(X)\subseteq [0,1]^l\) and \(EY^2<\infty\), \(l\geq 1\). Let \(m^*\) be the best approximation of the regression function of \((X, Y)\) by sums of functions of at most \(d\) variables (\(1\leq d\leq l)\). Estimation of \(m^*\) from i.i.d. data is considered.
For the estimation interaction least squares splines, which are defined as sums of polynomial tensor product splines of at most d variables, are used. The knot sequences of the tensor product splines are chosen equidistant. Complexity regularization is used to choose the number of the knots and the degree of the splines automatically using only the given data.
Without any additional condition on the contribution of \((X, Y)\) the weak and strong \(L^2\)-consistency of the estimate is shown. Furthermore, for even \(p \geq 1\) and every distribution of \((X, Y)\) with \(\text{supp}(X)\subseteq [0,1]^l\), \(Y\) bounded and \(m^*\) \(p\)-smooth, the integrated squared error of the estimate achieves up to a logarithmic factor the (optimal) rate \(n[(2p)/(2p+d)]\).

62G08 Nonparametric regression and quantile regression
62G20 Asymptotic properties of nonparametric inference
Full Text: DOI EuDML
[1] Barron AR, Birgé L, Massart P (1995) Risk bounds for model selection via penalization. Technical Report No. 95.54, Université Paris Sud · Zbl 0946.62036
[2] Chen Z (1991) Interaction spline models and their convergence rates. Ann. Statist. 19:1855–1868 · Zbl 0738.62065
[3] de Boor C (1978) A practical guide to splines. Springer, New York · Zbl 0406.41003
[4] Devroye L, Györfi L, Lugosi G (1996) A probabilistic theory of pattern recognition. Springer, Berlin · Zbl 0853.68150
[5] Dudley R (1978) Central limit theorems for empirical measures. Annals of Probability 6:899–929 · Zbl 0404.60016
[6] Haussler D (1992) Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation 100:78–150 · Zbl 0762.68050
[7] Kohler M (1996) Universally consistent regression function estimation using hierarchical B-splines. Preprint 96-11, Mathematisches Institut A, Universität Stuttgart. Submitted to Journal of Multivariate Analysis · Zbl 0954.62051
[8] Kohler M (1997) On the universal consistency of a least squares spline regression estimator. Math. Methods of Statistics 6:349–364 · Zbl 0884.62044
[9] Krzyzak A, Linder T (1996) Radial basis function networks and complexity regularization in function learning. To appear in IEEE Transaction on Neural Networks 9, 1998
[10] Lee WS, Bartlett PL, Williamson RC (1996) Efficient agnostic learning of neural networks with bounded fan-in. IEEE Trans. Inform. Theory 42:2118–2132 · Zbl 0858.94010
[11] Lugosi G, Zeger K (1995) Nonparametric estimation via empirical risk minimization. IEEE Trans. Inform. Theory 41:677–687 · Zbl 0818.62041
[12] Pollard D (1984) Convergence of stochastic processes. Springer-Verlag, New York · Zbl 0544.60045
[13] Schumaker L (1981) Spline functions: Basic theory. Wiley, New York · Zbl 0449.41004
[14] Stone CJ (1982) Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10:1040–1053 · Zbl 0511.62048
[15] Stone CJ (1985) Additive regression and other nonparametric models. Ann. Statist. 13:689–705 · Zbl 0605.62065
[16] Stone CJ (1986) The dimensionality reduction principle for generalized additive models. Ann. Statist. 14:590–606 · Zbl 0603.62050
[17] Stone CJ (1994) The use of polynomial splines and their tensor products in multivariate function estimation. Ann. Statist. 22:118–184 · Zbl 0827.62038
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.