×

A Robbins-Monro procedure for estimation in semiparametric regression models. (English) Zbl 1273.62065

Summary: This paper is devoted to the parametric estimation of a shift together with the nonparametric estimation of a regression function in a semiparametric regression model. We implement a very efficient and easy to handle Robbins-Monro procedure. On the one hand, we propose a stochastic algorithm similar to that of Robbins-Monro in order to estimate the shift parameter. A preliminary evaluation of the regression function is not necessary to estimate the shift parameter. On the other hand, we make use of a recursive Nadaraya-Watson estimator for the estimation of the regression function. This kernel estimator takes into account the previous estimation of the shift parameter. We establish the almost sure convergence for both Robbins-Monro and Nadaraya-Watson estimators. The asymptotic normality of our estimates is also provided. Finally, we illustrate our semiparametric estimation procedure on simulated and real data.

MSC:

62G05 Nonparametric estimation
62G20 Asymptotic properties of nonparametric inference
PDF BibTeX XML Cite
Full Text: DOI arXiv Euclid

References:

[1] Bercu, B. and Portier, B. (2008). Kernel density estimation and goodness-of-fit test in adaptive tracking. SIAM J. Control Optim. 47 2440-2457. · Zbl 1171.93034
[2] Bickel, P. J., Klaassen, C. A. J., Ritov, Y. and Wellner, J. A. (1998). Efficient and Adaptive Estimation for Semiparametric Models . Springer, New York. · Zbl 0894.62005
[3] Castillo, I. and Loubes, J. M. (2009). Estimation of the distribution of random shifts deformation. Math. Methods Statist. 18 21-42. · Zbl 1282.62069
[4] Chen, H. F., Lei, G. and Gao, A. J. (1988). Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds. Stochastic Process. Appl. 27 217-231. · Zbl 0632.62082
[5] Choi, E., Hall, P. and Rousson, V. (2000). Data sharpening methods for bias reduction in nonparametric regression. Ann. Statist. 28 1339-1355. · Zbl 1105.62336
[6] Clifford, G. D., Azuaje, F. and McSharry, P. (2006). Advanced Methods and Tools for ECG Data Analysis . Artech House, Boston.
[7] Dalalyan, A. S., Golubev, G. K. and Tsybakov, A. B. (2006). Penalized maximum likelihood and semiparametric second-order efficiency. Ann. Statist. 34 169-201. · Zbl 1091.62020
[8] Devroye, L. and Lugosi, G. (2001). Combinatorial Methods in Density Estimation . Springer, New York. · Zbl 0964.62025
[9] Duflo, M. (1997). Random Iterative Models. Applications of Mathematics ( New York ) 34 . Springer, Berlin. · Zbl 0868.62069
[10] Fabian, V. (1973). Asymptotically efficient stochastic approximation; the RM case. Ann. Statist. 1 486-495. · Zbl 0258.62048
[11] Gamboa, F., Loubes, J.-M. and Maza, E. (2007). Semi-parametric estimation of shifts. Electron. J. Stat. 1 616-640. · Zbl 1141.62313
[12] Gapoškin, V. F. and Krasulina, T. P. (1974). The law of the iterated logarithm in stochastic approximation processes. Teor. Verojatnost. i Primenen. 19 879-886. · Zbl 0333.62053
[13] Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application . Academic Press [Harcourt Brace Jovanovich Publishers], New York. · Zbl 0462.60045
[14] Hall, P. and Huang, L.-S. (2001). Nonparametric kernel regression subject to monotonicity constraints. Ann. Statist. 29 624-647. · Zbl 1012.62030
[15] Härdle, W. (1984). A law of the iterated logarithm for nonparametric regression function estimators. Ann. Statist. 12 624-635. · Zbl 0591.62030
[16] Härdle, W., Janssen, P. and Serfling, R. (1988). Strong uniform consistency rates for estimators of conditional functionals. Ann. Statist. 16 1428-1449. · Zbl 0672.62050
[17] Härdle, W. and Marron, J. S. (1990). Semiparametric comparison of regression curves. Ann. Statist. 18 63-89. · Zbl 0703.62053
[18] Härdle, W. and Tsybakov, A. B. (1988). Robust nonparametric regression with simultaneous scale curve estimation. Ann. Statist. 16 120-135. · Zbl 0668.62025
[19] Hürtgen, H. and Gervini, D. (2009). Semiparametric shape-invariant models for periodic data. J. Appl. Stat. 36 1055-1065.
[20] Kneip, A. and Engel, J. (1995). Model estimation in nonlinear regression under shape invariance. Ann. Statist. 23 551-570. · Zbl 0828.62052
[21] Kneip, A. and Gasser, T. (1988). Convergence and consistency results for self-modeling nonlinear regression. Ann. Statist. 16 82-112. · Zbl 0725.62060
[22] Kushner, H. J. and Yin, G. G. (2003). Stochastic Approximation and Recursive Algorithms and Applications. Applications of Mathematics ( New York ) 35 . Springer, New York. · Zbl 1026.62084
[23] Lassen, K. and Friis-Christensen, E. (1995). Variability of the solar cycle length during the past five centuries and the apparent association with terrestrial climate. J. Atmospheric and Terrestrial Physics 57 835-845.
[24] Lawton, W. H., Sylvestre, E. A. and Maggio, M. S. (1972). Self modeling nonlinear regression. Technometrics 14 513-532. · Zbl 0239.62045
[25] Lelong, J. (2008). Almost sure convergence for randomly truncated stochastic algorithms under verifiable conditions. Statist. Probab. Lett. 78 2632-2636. · Zbl 1147.62072
[26] McDonald, J. A. (1986). Periodic smoothing of time series. SIAM J. Sci. Statist. Comput. 7 665-688. · Zbl 0628.65148
[27] Mokkadem, A. and Pelletier, M. (2007). A companion for the Kiefer-Wolfowitz-Blum stochastic approximation algorithm. Ann. Statist. 35 1749-1772. · Zbl 1209.62191
[28] Nadaraja, È. A. (1964). On a regression estimate. Teor. Verojatnost. i Primenen. 9 157-159.
[29] Noda, K. (1976). Estimation of a regression function by the Parzen kernel-type density estimators. Ann. Inst. Statist. Math. 28 221-234. · Zbl 0369.62068
[30] Parzen, E. (1962). On estimation of a probability density function and mode. Ann. Math. Statist. 33 1065-1076. · Zbl 0116.11302
[31] Pelletier, M. (1998). On the almost sure asymptotic behaviour of stochastic algorithms. Stochastic Process. Appl. 78 217-244. · Zbl 0926.62072
[32] Robbins, H. and Monro, S. (1951). A stochastic approximation method. Ann. Math. Statist. 22 400-407. · Zbl 0054.05901
[33] Robbins, H. and Siegmund, D. (1971). A convergence theorem for non negative almost supermartingales and some applications. In Optimizing Methods in Statistics ( Proc. Sympos. , Ohio State Univ. , Columbus , Ohio , 1971) 233-257. Academic Press, New York. · Zbl 0286.60025
[34] Schuster, E. F. (1972). Joint asymptotic distribution of the estimated regression function at a finite number of distinct points. Ann. Math. Statist. 43 84-88. · Zbl 0248.62027
[35] Stone, C. J. (1975). Adaptive maximum likelihood estimators of a location parameter. Ann. Statist. 3 267-284. · Zbl 0303.62026
[36] Trigano, T., Isserles, U. and Ritov, Y. (2011). Semiparametric curve alignment and shift density estimation for biological data. IEEE Trans. Signal Process. 59 1970-1984. · Zbl 1392.94491
[37] Tsybakov, A. B. (2004). Introduction à L’estimation Non-Paramétrique. Mathématiques and Applications ( Berlin ) 41 . Springer, Berlin. · Zbl 1029.62034
[38] Vimond, M. (2010). Efficient estimation for a subclass of shape invariant models. Ann. Statist. 38 1885-1912. · Zbl 1189.62057
[39] Wang, Y. and Brown, M. B. (1996). A flexible model for human circadian rhythms. Biometrics 52 588-596. · Zbl 0875.62525
[40] Watson, G. S. (1964). Smooth regression analysis. Sankhyā Ser. A 26 359-372. · Zbl 0137.13002
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.