×

On methods of sieves and penalization. (English) Zbl 0895.62041

Summary: We develop a general theory which provides a unified treatment for the asymptotic normality and efficiency of the maximum likelihood estimates (MLE’s) in parametric, semiparametric and nonparametric models. We find that the asymptotic behavior of substitution estimates for estimating smooth functionals is essentially governed by two indices: the degree of smoothness of the functional and the local size of the underlying parameter space. We show that when the local size of the parameter space is not very large, the substitution standard (nonsieve), substitution sieve and substitution penalized MLE’s are asymptotically efficient in the Fisher sense, under certain stochastic equicontinuity conditions of the log-likelihood.
Moreover, when the convergence rate of the estimate is slow, the degree of smoothness of the functional needs to compensate for the slowness of the rate in order to achieve efficiency. When the size of the parameter space is very large, the standard and penalized maximum likelihood procedures may be inefficient, whereas the method of sieves may be able to overcome this difficulty. This phenomenon is particularly manifested when the functional of interest is very smooth, especially in the semiparametric case.

MSC:

62G05 Nonparametric estimation
62F12 Asymptotic properties of parametric estimators
62A01 Foundations and philosophical topics in statistics
62G20 Asymptotic properties of nonparametric inference
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Bahadur, R. R. (1964). On Fisher’s bound for asy mptotic variances. Ann. Math. Statist. 35 1545- 1552. · Zbl 0218.62034 · doi:10.1214/aoms/1177700378
[2] Bahadur, R. R. (1967). Rate of convergence of estimates and test statistics. Ann. Math. Statist. 38 303-324. · Zbl 0201.52106 · doi:10.1214/aoms/1177698949
[3] Begun, J., Hall, W., Huang, W. and Wellner, J. (1983). Information and asy mptotic efficiency in parametric-nonparametric models. Ann. Statist. 11 432-452. · Zbl 0526.62045 · doi:10.1214/aos/1176346151
[4] Bickel, P. J. (1982). On adaptive estimation. Ann. Statist. 10 647-671. · Zbl 0489.62033 · doi:10.1214/aos/1176345863
[5] Bickel, P. J., Klassen, C. A. J., Ritov, Y. and Wellner, J. A. (1994). Efficient and Adaptive Inference in Semi-parametric Models. Johns Hopkins Univ.
[6] Bickel, P. J. and Ritov, Y. (1988). Estimating integrated squared density derivatives: sharp best order of convergence estimates. Sankhy\?a Ser. A 50 381-393. · Zbl 0676.62037
[7] Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113-150. · Zbl 0805.62037 · doi:10.1007/BF01199316
[8] Birgé, L. and Massart, P. (1994). Minimum contrast estimators on sieves. Unpublished manuscript. · Zbl 0954.62033
[9] Cramér, H. (1946). Mathematical Methods of Statistics. Princeton Univ. Press. · Zbl 0063.01014
[10] Devore, R. and Lorentz, G. (1991). Constructive Approximation. Springer, New York. · Zbl 0797.41016
[11] Gabushin, V. N. (1967). Inequalities for norms of functions and their derivatives in the Lp metric. Mat. Zametki 1 291-298. · Zbl 0164.15301 · doi:10.1007/BF01098882
[12] Grenander, U. (1981). Abstract Inference. Wiley, New York. · Zbl 0505.62069
[13] Hájek, J. (1970). A characterisation of limiting distributions of regular estimates. Z. Wahrsch. Verw. Gebiete 14 323-330. · Zbl 0193.18001 · doi:10.1007/BF00533669
[14] Ibragimov, I. A. and Has’minskii, R. Z. (1981). Statistical Estimation. Springer, New York. · Zbl 0389.62024
[15] Ibragimov, I. A. and Has’minskii, R. Z. (1991). Asy mptotically normal families of distributions and efficient estimation. Ann. Statist. 19 1681-1724. · Zbl 0760.62043 · doi:10.1214/aos/1176348367
[16] Koenker, R. and Bassett, F. (1978). Regression quantiles. Econometrica 46 33-50. JSTOR: · Zbl 0373.62038 · doi:10.2307/1913643
[17] Kolmogorov, A. N. and Tihomirov, V. M. (1961). -entropy and -capacity of sets in function spaces. Amer. Math. Soc. Transl. 2 227-304. · Zbl 0133.06703
[18] Le Cam, L. (1960). Local asy mptotically normal families of distributions. Univ. California Publ. Statist. 3 37-98.
[19] Levit, B. (1974). On optimality of some statistical estimates. In Proceedings of the Prague Sy mposium on Asy mptotic Statistics (J. Hajek, ed.) 2 215-238. Univ. Karlova, Prague. · Zbl 0351.62023
[20] Levit, B. (1978). Infinite dimensional informational inequalities. Theory Probab. Appl. 23 371- 377. · Zbl 0404.62053
[21] Lindsay, B. G. (1980). Nuisance parameters, mixture models and the efficiency of partial likelihood estimators. Philos. Trans. Roy. Soc. London Ser. A 296 639-665. JSTOR: · Zbl 0434.62028 · doi:10.1098/rsta.1980.0197
[22] Lorentz, G. (1966). Approximation of Functions. Holt, Reinehart and Winston, New York. · Zbl 0153.38901
[23] Ossiander, M. (1987). A central limit theorem under metric entropy with L2 bracketing. Ann. Probab. 15 897-919. · Zbl 0665.60036 · doi:10.1214/aop/1176992072
[24] Parzen, M. and Harrington, D. (1993). Proportional odds regression with right-censored data using adaptive integrated splines. Technical report, Dept. Biostatistics, Harvard Univ.
[25] Pfanzagl, J. (1982). Contribution to a General Asy mptotic Statistical Theory. Springer, New York. · Zbl 0512.62001
[26] Pollard, D. (1984). Convergence of Stochastic Processes. Springer, New York. · Zbl 0544.60045
[27] Ritov, Y. and Bickel, P. J. (1990). Achieving information bounds in non and semiparametric models. Ann. Statist. 18 925-938. · Zbl 0722.62025 · doi:10.1214/aos/1176347633
[28] Schoenberg, I. J. (1964). Spline functions and the problem of graduation. Proc. Nat. Acad. Sci. U.S.A. 52 947-950. JSTOR: · Zbl 0147.32102 · doi:10.1073/pnas.52.4.947
[29] Severini, T. A. and Wong, W. H. (1992). Profile likelihood and conditionally parametric models. Ann. Statist. 20 1768-1862. Shen, X. (1997a). On the method of penalization. Statist. Sinica. To appear. Shen, X. (1997b). Proportional odds regression and sieve maximum likelihood estimation. Biometrika. · Zbl 0768.62015 · doi:10.1214/aos/1176348889
[30] Shen, X. and Wong, W. H. (1994). Convergence rate of sieve estimates. Ann. Statist. 22 580-615. · Zbl 0805.62008 · doi:10.1214/aos/1176325486
[31] Stone, C. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040-1053. · Zbl 0511.62048 · doi:10.1214/aos/1176345969
[32] Tikhonov, A. (1963). Solution of incorrectly formulated problems and the regularization method. Soviet. Math. Dokl. 5 1035-1038. · Zbl 0141.11001
[33] Triebel, H. (1983). Theory of Function Spaces. Birkhäuser, Boston. · Zbl 0546.46027
[34] von Mises, R. (1947). On the asy mptotic distribution of differentiable statistical functions. Ann. Math. Statist. 18 309-348. · Zbl 0037.08401 · doi:10.1214/aoms/1177730385
[35] Wahba, G. (1990). Spline Models for Observational Data. IMS, Hay ward, CA. · Zbl 0813.62001
[36] Whittaker, E. (1923). On new method of graduation. Proc. Edinburgh Math. Soc. 2 41.
[37] Wong, W. H. (1992). On asy mptotic efficiency in estimation theory. Statist. Sinica 2 47-68. · Zbl 0820.62025
[38] Wong, W. H. and Severini, T. A. (1991). On maximum likelihood estimation in infinite dimensional parameter space. Ann. Statist. 19 603-632. · Zbl 0732.62026 · doi:10.1214/aos/1176348113
[39] Wong, W. H. and Shen, X. (1995). A probability inequality for the likelihood surface and convergence rate of the maximum likelihood estimate. Ann. Statist. 23 339-362. · Zbl 0829.62002 · doi:10.1214/aos/1176324524
[40] Zeidler, E. (1990). Nonlinear Functional Analy sis and Its Applications II/A. Springer, New York. · Zbl 0684.47029
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.