zbMATH — the first resource for mathematics

Donsker-type theorems for nonparametric maximum likelihood estimators. (English) Zbl 1113.60028
Summary: Let \({\mathcal{P}}\) be a nonparametric probability model consisting of smooth probability densities and let \({\widehat{p}_{n}}\) be the corresponding maximum likelihood estimator based on \(n\) independent observations each distributed according to the law \({\mathbb{P}}\). With \(\widehat{\mathbb{P}}_{n}\) denoting the measure induced by the density \({\widehat{p}_{n}}\), define the stochastic process \({\widehat{\nu}}_{n}: f\mapsto \sqrt{n} \int fd({\widehat{\mathbb{P}}}_{n} -\mathbb{P})\) where \(f\) ranges over some function class \({\mathcal{F}}\). We give a general condition for Donsker classes \({\mathcal{F}}\) implying that the stochastic process \(\widehat{\nu}_{n}\) is asymptotically equivalent to the empirical process in the space \({\ell^{\infty}(\mathcal{F})}\) of bounded functions on \({\mathcal{F}}\). This implies in particular that \(\widehat{\nu}_{n}\) converges in law in \({\ell^{\infty}(\mathcal{F})}\) to a mean zero Gaussian process. We verify the general condition for a large family of Donsker classes \({\mathcal{ F}}\). We give a number of applications: convergence of the probability measure \({\widehat{\mathbb{P}}_{n}}\) to \({\mathbb{P}}\) at rate \({\sqrt{n}}\) in certain metrics metrizing the topology of weak(-star) convergence; a unified treatment of convergence rates of the MLE in a continuous scale of Sobolev-norms; \({\sqrt{n}}\) -efficient estimation of nonlinear functionals defined on \({\mathcal{P}}\); limit theorems at rate \({\sqrt{n}}\) for the maximum likelihood estimator of the convolution product \({\mathbb P\ast \mathbb P}\).

60F05 Central limit and other weak theorems
62G07 Density estimation
62F12 Asymptotic properties of parametric estimators
46F05 Topological linear spaces of test functions, distributions and ultradistributions
Full Text: DOI
[1] Adams R.A., Fournier J.F. (2003) Sobolev spaces, 2nd edn. Academic, New York · Zbl 1098.46001
[2] Bickel J.P., Ritov Y. (1988) Estimating integrated squared density derivatives: sharp best order of convergence estimates. Sankhya Ser. A 50, 381–393 · Zbl 0676.62037
[3] Bickel J.P., Ritov Y. (2003) Nonparametric estimators which can be ’plugged-in’. Ann. Stat. 31, 1033–1053 · Zbl 1058.62031
[4] Birgé L., Massart P. (1993) Rates of convergence of minimum contrast estimators. Probab. Theory Relat. Fields 97, 113–150 · Zbl 0805.62037
[5] Birgé L., Massart P. (1995) Estimation of integral functionals of a density. Ann. Stat. 23, 11–29 · Zbl 0848.62022
[6] Dieudonné J. (1960) Foundations of Modern Analysis. Academic, New York · Zbl 0100.04201
[7] Donoho, D.L., Liu R.C.: Geometrizing rates of convergence II, III. Ann. Stat. 19, 633–667, 668–701 (1991) · Zbl 0754.62028
[8] Dudley R.M. (1999) Uniform Central Limit Theorems. Cambridge University Press, Cambridge · Zbl 0951.60033
[9] Dudley R.M. (2002) Real Analysis and Probability. Cambridge University Press, Cambridge · Zbl 1023.60001
[10] Dunford N., Schwartz J.T. (1966) Linear Operators. Part I: General Theory. Interscience, New York · Zbl 0146.12601
[11] Frees E.W. (1994) Estimating densities of functions of observations. J. Am. Stat. Assoc. 89, 517–525 · Zbl 0798.62051
[12] Giné E. (1975) Invariant tests for uniformity on compact Riemannian manifolds based on Sobolev-norms. Ann. Stat. 3, 1243–1266 · Zbl 0322.62058
[13] Giné, E., Mason, D.M.: On local U-statistic processes and the estimation of densities of functions of several variables. Ann. Stat. (in press) (2006) · Zbl 1175.60017
[14] Giné E., Zinn J. (1986) Empirical processes indexed by Lipschitz functions. Ann. Probab. 14, 1329–1338 · Zbl 0611.60029
[15] Hall P., Marron J.S. (1987) Estimation of integrated squared density derivatives. Stat. Probab. Lett. 6, 109–115 · Zbl 0628.62029
[16] Kiefer J., Wolfowitz J. (1976) Asymptotically minimax estimation of concave and convex distribution functions. Z. Wahrscheinlichkeitstheorie verw. Gebiete 34, 73–85 · Zbl 0354.62035
[17] Laurent B. (1996) Efficient estimation of integral functionals of a density. Ann. Stat. 24, 659–681 · Zbl 0859.62038
[18] Laurent B. (1997) Estimation of integral functionals of a density and its derivatives. Bernoulli 3, 181–211 · Zbl 0872.62044
[19] Leeb H., Pötscher B.M. (2006) Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk-bound results. Econom. Theory 22, 69–97 · Zbl 1083.62060
[20] Lions J.L., Magenes E. (1972) Non-Homogeneous Boundary Value Problems and Applications I. Springer, Berlin Heidelberg New York · Zbl 0223.35039
[21] Nickl, R.: Empirical and Gaussian processes on Besov classes. In: Giné, E., Kolchinskii, V., Li, W., Zinn, J. (eds.) High Dimensional Probability IV, IMS Lecture Notes (in press) (2006a) · Zbl 1156.60020
[22] Nickl, R.: On convergence and convolutions of random signed measures (preprint) (2006b) · Zbl 1156.60020
[23] Nickl, R.: Uniform central limit theorems for density estimators (preprint) (2006c)
[24] Nickl, R., Pötscher, B.M.: Bracketing metric entropy rates and empirical central limit theorems for function classes of Besov-and Sobolev-type. J. Theor. Probab. (in press) (2005) · Zbl 1130.46020
[25] Pötscher B.M., Prucha I.R. (1997) Dynamic Nonlinear Econometric. Models Asymptotic Theory. Springer, Berlin Heidelberg New York · Zbl 0923.62121
[26] Radulovic, D., Wegkamp, M.: Weak convergence of smoothed empirical processes. Beyond Donsker classes. In: Giné, E., Mason, D.M., Wellner, J.A. (eds.) High Dimensional Probability II, Progr. Probab. 47, pp. 89–105 Birkhäuser, Boston (2000) · Zbl 0973.60025
[27] Radulovic D., Wegkamp M. (2003) Necessary and sufficient conditions for weak convergence of smoothed empirical processes. Stat. Probab. Lett.61, 321–336 · Zbl 1048.62051
[28] Rost, D.: Limit theorems for smoothed empirical processes. In: Giné, E., Mason, D.M., Wellner, J.A. (eds.) High dimensional probability II, Progr. Probab. 47, pp. 107–113 Birkhäuser, Boston (2000) · Zbl 0968.60023
[29] Rufibach, K., Dümbgen, L.: Maximum likelihood estimation of a log-concave density. Basic properties and consistency (preprint) (2004) · Zbl 1200.62030
[30] Schick A., Wefelmeyer W. (2004) Root n consistent density estimators for sums of independent random variables. J. Nonparametr. Stat. 16, 925–935 · Zbl 1062.62065
[31] Schmeisser H.-J., Triebel H. (1987) Topics in Fourier Analysis and Function Spaces. Wiley, New York · Zbl 0661.46024
[32] Stone C.J. (1980) Optimal rates of convergence for nonparametric estimators. Ann. Stat. 8, 1348–1360 · Zbl 0451.62033
[33] Strassen V., Dudley R.M. (1969) The central limit theorem and \(\varepsilon \) -entropy. Probability and information theory. Lect. Notes Math. 1247, 224–231 · Zbl 0196.21101
[34] Triebel H. (1983) Theory of Function Spaces. Birkhäuser, Basel · Zbl 0546.46028
[35] van de Geer S. (1993) Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Stat. 21, 14–44 · Zbl 0779.62033
[36] van de Geer S. (2000) Empirical Processes in M-estimation. Cambridge University Press, Cambridge · Zbl 0953.62049
[37] van der Vaart A.W. (1994) Weak convergence of smoothed empirical processes. Scand. J. Stat. 21, 501–504 · Zbl 0809.62040
[38] van der Vaart A.W., Wellner J.A. (1996) Weak Convergence and Empirical Processes. Springer, Berlin Heidelberg New York · Zbl 0862.60002
[39] von Mises R. (1947) On the asymptotic distribution of differentiable statistical functions. Ann. Math. Stat. 20, 309–348 · Zbl 0037.08401
[40] Wong W.H., Severini T.A. (1991) On maximum likelihood estimation in infinite dimensional parameter spaces. Ann. Stat. 19, 603–632 · Zbl 0732.62026
[41] Wong W.H., Shen X. (1995) Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Stat. 23, 339–362 · Zbl 0829.62002
[42] Yukich J.E. (1992) Weak convergence of smoothed empirical processes. Scand. J. Stat. 19, 271–279 · Zbl 0756.60026
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.