Convex models, MLE and misspecification. (English) Zbl 1029.62020

Summary: We analyze the asymptotic behavior of maximum likelihood estimators (MLE) in convex dominated models when the true distribution generating the independent data does not necessarily belong to the model. Inspired by the Hellinger distance and its properties, we introduce a family of divergences (contrast functions) which allow a unified treatment of well- and misspecified convex models. Convergence and rates of convergence of the MLE with respect to our divergences are obtained from inequalities satisfied by these divergences and results from empirical process theory (uniform laws of large numbers and maximal inequalities).
As a particular case we recover existing results for Hellinger convergence of MLEs in well-specified convex models. Four examples are considered: mixtures of discrete distributions, monotone densities, decreasing failure rate distributions and a finite-dimensional parametric model.


62F12 Asymptotic properties of parametric estimators
62G20 Asymptotic properties of nonparametric inference
62G30 Order statistics; empirical distribution functions
Full Text: DOI


[1] Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference under Order Restrictions. Wiley, New York. · Zbl 0246.62038
[2] Barlow, R. E, Marshall, A. and Prochan, F. (1963). Properties of probability distributions with monotone hazard rates. Ann. Math. Statist. 34 375-389. · Zbl 0249.60006 · doi:10.1214/aoms/1177704147
[3] Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York. · Zbl 0172.21201
[4] Birgé, L. (1989). The Grenander estimator: a nonasymptotic approach. Ann. Statist. 17 1532-1549. · Zbl 0703.62042 · doi:10.1214/aos/1176347380
[5] Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113-150. · Zbl 0805.62037 · doi:10.1007/BF01199316
[7] Dahlhaus, R. and Wefelmeyer, W. (1996). Asymptotically optimal estimation in misspecified time series models. Ann. Statist. 24 952-974. · Zbl 0865.62063 · doi:10.1214/aos/1032526951
[8] Grenander, U. (1956). On the theory of mortality measurement, part II. Scand. Actuar. J. 39 125-153. · Zbl 0077.33715
[9] Huber, P. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proc. Fifth Berkeley Symp. Math. Statist. Probab. 1 221-233. Univ. California Press, Berkeley. · Zbl 0212.21504
[10] Lambert, D. and Tierney, L. (1984). Asymptotic properties of the maximum likelihood estimates in mixed Poisson model. Ann. Statist. 12 1388-1399. · Zbl 0562.62038 · doi:10.1214/aos/1176346799
[11] Lindsay B. (1983). The geometry of mixture likelihood: a general theory. Ann. Statist. 11 86-94. · Zbl 0512.62005 · doi:10.1214/aos/1176346059
[12] Koltchinskii, V. I. (1997). M-estimation, convexity and quantiles. Ann. Statist. 25 435-477. · Zbl 0878.62037 · doi:10.1214/aos/1031833659
[13] L üxmann-Ellinghaus, U. (1987). On the identifiability of mixture of infinitely divisible power series distributions. Statist. Probab. Lett. 5 375-379. · Zbl 0631.60013 · doi:10.1016/0167-7152(87)90014-9
[14] Marshall, A. W. (1970). Discussion of Barlow and van Zwet’s paper ”Asymptotic properties of isotonic estimators for the generalized failure rate function: strong consistency.” In Nonparametric Techniques in Statistical Inference (M. L. Puri, ed.) 174-176. Cambridge Univ. Press.
[15] Marshall, A. W. and Proschan, F. (1965). Maximum likelihood estimation for distribution with monotone failure rate. Ann. Math. Statist. 36 69-77. · Zbl 0128.38506 · doi:10.1214/aoms/1177700271
[16] Noack, A. (1950). A class of random variables with discrete distribution Ann. Math. Statist. 21 127-132. · Zbl 0036.08601 · doi:10.1214/aoms/1177729894
[17] Milhaud, X. and Mounime, S. (1996). A modified maximum likelihood estimator for infinite mixtures. Preprint, Univ. P. Sabatier, Toulouse.
[18] Patilea, V. (1997). Convex models, NPLME and misspecification, Ph.D dissertation, part I. Institute of Statistics, Univ. catholique de Louvain.
[19] Pfanzagl J. (1969). On the measurability and consistency of minimum contrast estimates. Metrika 249-272. · Zbl 0181.45501 · doi:10.1007/BF02613654
[20] Pfanzagl J. (1988). Consistency of maximum likelihood estimators for certain nonparametric families, in particular: Mixtures. J. Statist. Plann. Inference 19 137-158. · Zbl 0656.62044 · doi:10.1016/0378-3758(88)90069-9
[21] Pfanzagl, J. (1990). Large deviations probabilities for certain nonparametric maximum likelihood estimators. Ann. Statist. 18 1868-1877. · Zbl 0721.62048 · doi:10.1214/aos/1176347884
[22] Pollard, D. (1984). Convergence of Stochastic Processes. Springer, New York. · Zbl 0544.60045
[23] Rolin, J.-M. (1992). Personal notes.
[24] Roberts, W. and Varberg, D. (1973). Convex Functions. Academic Press, New York. · Zbl 0271.26009
[25] Simar, L. (1976). Maximum likelihood estimation of a compound Poisson process. Ann. Statist. 4 1200-1209. · Zbl 0362.62095 · doi:10.1214/aos/1176343651
[26] van de Geer, S. (1993). Hellinger consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14-44. · Zbl 0779.62033 · doi:10.1214/aos/1176349013
[27] van de Geer, S. (2000). Empirical Processes in M-estimation. Cambridge Univ. Press. · Zbl 0953.62049
[28] van der Vaart, A. and Wellner, J. (1996). Weak Convergence and Empirical Process. Springer, New York. · Zbl 0862.60002
[29] Wang, J.-L. (1985). Strong consistency of approximate maximum likelihood estimators with applications to nonparametrics. Ann. Statist. 13 932-946. · Zbl 0598.62034 · doi:10.1214/aos/1176349647
[30] Wong, W. H. and Shen, X. (1995). Probabilities inequalities for likelihood ratios and convergence rates of sieve MLE’s. Ann. Statist. 23 339-362. · Zbl 0829.62002 · doi:10.1214/aos/1176324524
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.