Simultaneous estimation of the mean and the variance in heteroscedastic Gaussian regression. (English) Zbl 1320.62092

Gendre, Xavier

Electron. J. Stat. 2, 1345-1372 (2008).

Summary: Let \(Y\) be a Gaussian vector of \(\mathbb R^{n}\) of mean \(s\) and diagonal covariance matrix \(\Gamma\). Our aim is to estimate both \(s\) and the entries \(\sigma _{i}=\Gamma _{i,i}\), for \(i=1,\dots ,n\), on the basis of the observation of two independent copies of \(Y\). Our approach is free of any prior assumption on \(s\) but requires that we know some upper bound \(\gamma\) on the ratio max \(_{i}\sigma _{i}/\min_{i}\sigma_{i}\). For example, the choice \(\gamma =1\) corresponds to the homoscedastic case where the components of \(Y\) are assumed to have common (unknown) variance. In the opposite, the choice \(\gamma >1\) corresponds to the heteroscedastic case where the variances of the components of \(Y\) are allowed to vary within some range. Our estimation strategy is based on model selection. We consider a family \(\{S_{m}\times \Sigma _{m}, m\in \mathcal M\}\) of parameter sets where \(S_{m}\) and \(\Sigma _{m}\) are linear spaces. To each \(m\in \mathcal M\), we associate a pair of estimators \((\hat s_{m},\hat{\sigma}_{m})\) of \((s,\sigma )\) with values in \(S_{m}\times \Sigma _{m}\). Then we design a model selection procedure in view of selecting some \(\hat m\) among \(\mathcal M\) in such a way that the Kullback risk of \((\hat s_{\hat {m}}, \hat{\sigma}_{\hat{m}})\) is as close as possible to the minimum of the Kullback risks among the family of estimators \(\{(\hat s_m, \hat{\sigma}_m), m\in \mathcal M\}\). Then we derive uniform rates of convergence for the estimator \((\hat s_{\hat{m}}, \hat{\sigma}_{\hat{m}})\) over Hölderian balls. Finally, we carry out a simulation study in order to illustrate the performances of our estimators in practice.

MSC:

62G08

Nonparametric regression and quantile regression

Keywords:

Gaussian regression; heteroscedasticity; model selection; Kullback risk; convergence rate

Software:

R

References:

[1]	Akaike, H. (1969). Statistical predictor identification., Annals Inst. Statist. Math. . · Zbl 0198.51201 · doi:10.1007/BF02506337
[2]	Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In, Proceedings 2nd International Symposium on Information Theory , P. Petrov and F. Csaki, Eds. Akademia Kiado, Budapest, 267-281. · Zbl 0283.62006
[3]	Akaike, H. (1974). A new look at the statistical model identification., IEEE Trans. on Automatic Control . · Zbl 0314.62039 · doi:10.1109/TAC.1974.1100705
[4]	Arlot, S. (2007). Rééchantillonnage et sélection de modèles. Ph.D. thesis, Université Paris, 11.
[5]	Baraud, Y. (2000). Model selection for regression on a fixed design., Probab. Theory Related Fields 117 , 467-493. · Zbl 0997.62027 · doi:10.1007/s004400000058
[6]	Baraud, Y., Giraud, C., and Huet, S. (2006). Gaussian model selection with an unknown variance. To appear in Annals of, Statistics. · Zbl 1162.62051 · doi:10.1214/07-AOS573
[7]	Barron, A., Birgé, L., and Massart, P. (1999). Risk bounds for model selection via penalization., Probab. Theory Related Fields 113 , 301-413. · Zbl 0946.62036 · doi:10.1007/s004400050210
[8]	Birgé, L. and Massart, P. (1997). From model selection to adaptive estimation., Festschrift for Lucien Lecam: Research Papers in Probability and Statistics . · Zbl 0920.62042
[9]	Birgé, L. and Massart, P. (2001a). Gaussian model selection., Journal of the European Mathematical Society 3 , 3, 203-268. · Zbl 1037.62001 · doi:10.1007/s100970100031
[10]	Birgé, L. and Massart, P. (2001b). A generalized, c p criterion for gaussian model selection. Prépublication 647, Universités de Paris 6 and Paris 7 . · Zbl 1037.62001
[11]	Boucheron, S., Bousquet, O., Lugosi, G., and Massart, P. (2005). Moment inequalities for functions of independent random variables., Annals of Probability 33 , 2, 514-560. · Zbl 1074.60018 · doi:10.1214/009117904000000856
[12]	Comte, F. and Rozenholc, Y. (2002). Adaptive estimation of mean and volatility functions in (auto-)regressive models., Stochastic Processes and their Applications 97 , 1, 111-145. · Zbl 1064.62046 · doi:10.1016/S0304-4149(01)00128-4
[13]	DeVore, R. and Lorentz, G. (1993)., Constructive approximation . Vol. 303 . Springer-Verlag. · Zbl 0797.41016
[14]	Galtchouk, L. and Pergamenshchikov, S. (2005). Efficient adaptive nonparametric estimation in heteroscedastic regression models. Université Louis Pasteur, IRMA, · Zbl 1081.62058
[15]	Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic functional by model selection., Annals of Statistics 28 , 5, 1302-1338. · Zbl 1105.62328 · doi:10.1214/aos/1015957395
[16]	Mallows, C. (1973). Some comments on, c p . Technometrics 15 , 661-675. · Zbl 0269.62061 · doi:10.2307/1267380
[17]	McQuarrie, A. and Tsai, C. (1998)., Regression and times series model selection . World Scientific Publishing Co, Inc. · Zbl 0907.62095
[18]	R Development Core Team. (2007)., R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, http://www.R-project.org.
[19]	Wang, L. and, al. (2008). Effect of mean on variance function estimation in nonparametric regression. The Annals of Statistics 36 , 2, 646-664. · Zbl 1133.62033 · doi:10.1214/009053607000000901