# zbMATH — the first resource for mathematics

Donsker-type theorems for nonparametric maximum likelihood estimators. (English) Zbl 1113.60028
Summary: Let $${\mathcal{P}}$$ be a nonparametric probability model consisting of smooth probability densities and let $${\widehat{p}_{n}}$$ be the corresponding maximum likelihood estimator based on $$n$$ independent observations each distributed according to the law $${\mathbb{P}}$$. With $$\widehat{\mathbb{P}}_{n}$$ denoting the measure induced by the density $${\widehat{p}_{n}}$$, define the stochastic process $${\widehat{\nu}}_{n}: f\mapsto \sqrt{n} \int fd({\widehat{\mathbb{P}}}_{n} -\mathbb{P})$$ where $$f$$ ranges over some function class $${\mathcal{F}}$$. We give a general condition for Donsker classes $${\mathcal{F}}$$ implying that the stochastic process $$\widehat{\nu}_{n}$$ is asymptotically equivalent to the empirical process in the space $${\ell^{\infty}(\mathcal{F})}$$ of bounded functions on $${\mathcal{F}}$$. This implies in particular that $$\widehat{\nu}_{n}$$ converges in law in $${\ell^{\infty}(\mathcal{F})}$$ to a mean zero Gaussian process. We verify the general condition for a large family of Donsker classes $${\mathcal{ F}}$$. We give a number of applications: convergence of the probability measure $${\widehat{\mathbb{P}}_{n}}$$ to $${\mathbb{P}}$$ at rate $${\sqrt{n}}$$ in certain metrics metrizing the topology of weak(-star) convergence; a unified treatment of convergence rates of the MLE in a continuous scale of Sobolev-norms; $${\sqrt{n}}$$ -efficient estimation of nonlinear functionals defined on $${\mathcal{P}}$$; limit theorems at rate $${\sqrt{n}}$$ for the maximum likelihood estimator of the convolution product $${\mathbb P\ast \mathbb P}$$.

##### MSC:
 60F05 Central limit and other weak theorems 62G07 Density estimation 62F12 Asymptotic properties of parametric estimators 46F05 Topological linear spaces of test functions, distributions and ultradistributions
Full Text:
##### References:
  Adams R.A., Fournier J.F. (2003) Sobolev spaces, 2nd edn. Academic, New York · Zbl 1098.46001  Bickel J.P., Ritov Y. (1988) Estimating integrated squared density derivatives: sharp best order of convergence estimates. Sankhya Ser. A 50, 381–393 · Zbl 0676.62037  Bickel J.P., Ritov Y. (2003) Nonparametric estimators which can be ’plugged-in’. Ann. Stat. 31, 1033–1053 · Zbl 1058.62031  Birgé L., Massart P. (1993) Rates of convergence of minimum contrast estimators. Probab. Theory Relat. Fields 97, 113–150 · Zbl 0805.62037  Birgé L., Massart P. (1995) Estimation of integral functionals of a density. Ann. Stat. 23, 11–29 · Zbl 0848.62022  Dieudonné J. (1960) Foundations of Modern Analysis. Academic, New York · Zbl 0100.04201  Donoho, D.L., Liu R.C.: Geometrizing rates of convergence II, III. Ann. Stat. 19, 633–667, 668–701 (1991) · Zbl 0754.62028  Dudley R.M. (1999) Uniform Central Limit Theorems. Cambridge University Press, Cambridge · Zbl 0951.60033  Dudley R.M. (2002) Real Analysis and Probability. Cambridge University Press, Cambridge · Zbl 1023.60001  Dunford N., Schwartz J.T. (1966) Linear Operators. Part I: General Theory. Interscience, New York · Zbl 0146.12601  Frees E.W. (1994) Estimating densities of functions of observations. J. Am. Stat. Assoc. 89, 517–525 · Zbl 0798.62051  Giné E. (1975) Invariant tests for uniformity on compact Riemannian manifolds based on Sobolev-norms. Ann. Stat. 3, 1243–1266 · Zbl 0322.62058  Giné, E., Mason, D.M.: On local U-statistic processes and the estimation of densities of functions of several variables. Ann. Stat. (in press) (2006) · Zbl 1175.60017  Giné E., Zinn J. (1986) Empirical processes indexed by Lipschitz functions. Ann. Probab. 14, 1329–1338 · Zbl 0611.60029  Hall P., Marron J.S. (1987) Estimation of integrated squared density derivatives. Stat. Probab. Lett. 6, 109–115 · Zbl 0628.62029  Kiefer J., Wolfowitz J. (1976) Asymptotically minimax estimation of concave and convex distribution functions. Z. Wahrscheinlichkeitstheorie verw. Gebiete 34, 73–85 · Zbl 0354.62035  Laurent B. (1996) Efficient estimation of integral functionals of a density. Ann. Stat. 24, 659–681 · Zbl 0859.62038  Laurent B. (1997) Estimation of integral functionals of a density and its derivatives. Bernoulli 3, 181–211 · Zbl 0872.62044  Leeb H., Pötscher B.M. (2006) Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk-bound results. Econom. Theory 22, 69–97 · Zbl 1083.62060  Lions J.L., Magenes E. (1972) Non-Homogeneous Boundary Value Problems and Applications I. Springer, Berlin Heidelberg New York · Zbl 0223.35039  Nickl, R.: Empirical and Gaussian processes on Besov classes. In: Giné, E., Kolchinskii, V., Li, W., Zinn, J. (eds.) High Dimensional Probability IV, IMS Lecture Notes (in press) (2006a) · Zbl 1156.60020  Nickl, R.: On convergence and convolutions of random signed measures (preprint) (2006b) · Zbl 1156.60020  Nickl, R.: Uniform central limit theorems for density estimators (preprint) (2006c)  Nickl, R., Pötscher, B.M.: Bracketing metric entropy rates and empirical central limit theorems for function classes of Besov-and Sobolev-type. J. Theor. Probab. (in press) (2005) · Zbl 1130.46020  Pötscher B.M., Prucha I.R. (1997) Dynamic Nonlinear Econometric. Models Asymptotic Theory. Springer, Berlin Heidelberg New York · Zbl 0923.62121  Radulovic, D., Wegkamp, M.: Weak convergence of smoothed empirical processes. Beyond Donsker classes. In: Giné, E., Mason, D.M., Wellner, J.A. (eds.) High Dimensional Probability II, Progr. Probab. 47, pp. 89–105 Birkhäuser, Boston (2000) · Zbl 0973.60025  Radulovic D., Wegkamp M. (2003) Necessary and sufficient conditions for weak convergence of smoothed empirical processes. Stat. Probab. Lett.61, 321–336 · Zbl 1048.62051  Rost, D.: Limit theorems for smoothed empirical processes. In: Giné, E., Mason, D.M., Wellner, J.A. (eds.) High dimensional probability II, Progr. Probab. 47, pp. 107–113 Birkhäuser, Boston (2000) · Zbl 0968.60023  Rufibach, K., Dümbgen, L.: Maximum likelihood estimation of a log-concave density. Basic properties and consistency (preprint) (2004) · Zbl 1200.62030  Schick A., Wefelmeyer W. (2004) Root n consistent density estimators for sums of independent random variables. J. Nonparametr. Stat. 16, 925–935 · Zbl 1062.62065  Schmeisser H.-J., Triebel H. (1987) Topics in Fourier Analysis and Function Spaces. Wiley, New York · Zbl 0661.46024  Stone C.J. (1980) Optimal rates of convergence for nonparametric estimators. Ann. Stat. 8, 1348–1360 · Zbl 0451.62033  Strassen V., Dudley R.M. (1969) The central limit theorem and $$\varepsilon$$ -entropy. Probability and information theory. Lect. Notes Math. 1247, 224–231 · Zbl 0196.21101  Triebel H. (1983) Theory of Function Spaces. Birkhäuser, Basel · Zbl 0546.46028  van de Geer S. (1993) Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Stat. 21, 14–44 · Zbl 0779.62033  van de Geer S. (2000) Empirical Processes in M-estimation. Cambridge University Press, Cambridge · Zbl 0953.62049  van der Vaart A.W. (1994) Weak convergence of smoothed empirical processes. Scand. J. Stat. 21, 501–504 · Zbl 0809.62040  van der Vaart A.W., Wellner J.A. (1996) Weak Convergence and Empirical Processes. Springer, Berlin Heidelberg New York · Zbl 0862.60002  von Mises R. (1947) On the asymptotic distribution of differentiable statistical functions. Ann. Math. Stat. 20, 309–348 · Zbl 0037.08401  Wong W.H., Severini T.A. (1991) On maximum likelihood estimation in infinite dimensional parameter spaces. Ann. Stat. 19, 603–632 · Zbl 0732.62026  Wong W.H., Shen X. (1995) Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Stat. 23, 339–362 · Zbl 0829.62002  Yukich J.E. (1992) Weak convergence of smoothed empirical processes. Scand. J. Stat. 19, 271–279 · Zbl 0756.60026
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.