×

Law of iterated logarithm and model selection consistency for generalized linear models with independent and dependent responses. (English) Zbl 1478.62210

Summary: We study the law of the iterated logarithm (LIL) for the maximum likelihood estimation of the parameters (as a convex optimization problem) in the generalized linear models with independent or weakly dependent \(\rho\)-mixing) responses under mild conditions. The LIL is useful to derive the asymptotic bounds for the discrepancy between the empirical process of the log-likelihood function and the true log-likelihood. The strong consistency of some penalized likelihood-based model selection criteria can be shown as an application of the LIL. Under some regularity conditions, the model selection criterion will be helpful to select the simplest correct model almost surely when the penalty term increases with the model dimension, and the penalty term has an order higher than \(O(\log \log n)\) but lower than \(O(n)\). Simulation studies are implemented to verify the selection consistency of Bayesian information criterion.

MSC:

62J12 Generalized linear models (logistic models)
62F12 Asymptotic properties of parametric estimators

Software:

Fahrmeir; catdata
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Ai, M. Y.; Wang, F.; Yu, J.; Zhang, H. M., Optimal subsampling for large-scale quantile regression, J Complexity, 62, 101512 (2021) · Zbl 1460.62049 · doi:10.1016/j.jco.2020.101512
[2] Ai M Y, Yu J, Zhang H M, Wang H Y. Optimal subsampling algorithms for big data regressions. Statist Sinica, 2021, 31(2) (to appear) · Zbl 1469.62422
[3] Akaike H. Information theory and an extension of the maximum likelihood principle. In: Second International Symposium on Information Theory. 1973, 267-281 · Zbl 0283.62006
[4] Bosq, D., Nonparametric Statistics for Stochastic Processes: Estimation and Prediction (1998), Berlin: Springer, Berlin · Zbl 0902.62099 · doi:10.1007/978-1-4612-1718-3
[5] Brown, L. D., Fundamentals of Statistical Exponential Families: with Applications in Statistical Decision Theory (1986), Hayward: Inst Math Stat, Hayward · Zbl 0685.62002
[6] Chen, X. R., Quasi Likelihood Method for Generalized Linear Model (2011), Hefei: Press of University of Science and Technology of China, Hefei
[7] Czado, C.; Munk, A., Noncanonical links in generalized linear models when is the effort justified?, J Statist Plann Inference, 87, 2, 317-345 (2000) · Zbl 0969.62048 · doi:10.1016/S0378-3758(99)00195-0
[8] Efron, B.; Hastie, T. C., Computer Age Statistical Inference: Algorithms, Evidence, and Data Science (2016), Cambridge: Cambridge Univ Press, Cambridge · Zbl 1377.62004 · doi:10.1017/CBO9781316576533
[9] Fahrmeir, L.; Kaufmann, H., Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models, Ann Statist, 13, 1, 342-368 (1985) · Zbl 0594.62058 · doi:10.1214/aos/1176346597
[10] Fahrmeir, L.; Tutz, G., Multivariate Statistical Modelling Based on Generalized Linear Models (2001), New York: Springer, New York · Zbl 0980.62052 · doi:10.1007/978-1-4757-3454-6
[11] Fan, J. Q.; Qi, L.; Tong, X., Penalized least squares estimation with weakly dependent data, Sci China Math, 59, 12, 2335-2354 (2016) · Zbl 1367.62211 · doi:10.1007/s11425-016-0098-x
[12] Fang, X. Z., Laws of the iterated logarithm for maximum likelihood estimates of parameter vectors in nonhomogeneous Poisson processes, Acta Sci Natur Univ Pekinensis, 34, 5, 563-573 (1998) · Zbl 1092.62509
[13] Hansen B. Econometrics. Version: Jan 2018. 2018
[14] He, X. M.; Wang, G., Law of the iterated logarithm and invariance principle for M-estimators, Proc Amer Math Soc, 123, 2, 563-573 (1995) · Zbl 0855.62017
[15] Kim, Y. D.; Jeon, J. J., Consistent model selection criteria for quadratically supported risks, Ann Statist, 44, 6, 2467-2496 (2016) · Zbl 1365.60030
[16] Kroll, M., Non-parametric Poisson regression from independent and weakly dependent observations by model selection, J Statist Plann Inference, 199, 249-270 (2019) · Zbl 1418.62119 · doi:10.1016/j.jspi.2018.07.003
[17] Lai, T. L.; Wei, C. Z., A law of the iterated logarithm for double arrays of independent random variables with applications to regression and time series models, Ann Probab, 10, 2, 320-335 (1982) · Zbl 0485.60031 · doi:10.1214/aop/1176993860
[18] Lin, Z. Y.; Lu, C. R., Limit Theory for Mixing Dependent Random Variables, Mathematics and Its Applications (1997), Beijing/Dordrecht: Science Press/Kluwer Academic Publishers, Beijing/Dordrecht
[19] Mahoney, M. W.; Duchi, J. C.; Gilbert, A. C., The Mathematics of Data (2018), Providence: Amer Math Soc, Providence · Zbl 1404.68012 · doi:10.1090/pcms/025
[20] Markatou, M.; Basu, A.; Lindsay, B. G., Weighted likelihood equations with bootstrap root search, J Amer Statist Assoc, 93, 442, 740-750 (1998) · Zbl 0918.62046 · doi:10.1080/01621459.1998.10473726
[21] McCullagh, P.; Nelder, J. A., Generalized Linear Models (1989), London: Chapman and Hall, London · Zbl 0588.62104 · doi:10.1007/978-1-4899-3242-6
[22] Miao, Y.; Yang, G. Y., The loglog law for LS estimator in simple linear EV regression models, Statistics, 45, 2, 155-162 (2011) · Zbl 1283.62047 · doi:10.1080/02331880903450576
[23] Nelder, J. A.; Wedderburn, R. W M., Generalized linear models, J R Statist Soc Ser A, 135, 3, 370-384 (1972) · doi:10.2307/2344614
[24] Qian, G. Q.; Wu, Y. H., Strong limit theorems on model selection in generalized linear regression with binomial responses, Statist Sinica, 16, 4, 1335-1365 (2006) · Zbl 1109.62063
[25] Rao, C. R.; Wu, Y. H., A strongly consistent procedure for model selection in a regression problem, Biometrika, 76, 2, 369-374 (1989) · Zbl 0669.62051 · doi:10.1093/biomet/76.2.369
[26] Rao, C. R.; Zhao, L. C., Linear representation of M-estimates in linear models, Canad J Statist, 20, 4, 359-368 (1992) · Zbl 0767.62056 · doi:10.2307/3315607
[27] Rigollet, P., Kullback-Leibler aggregation and misspecified generalized linear models, Ann Statist, 40, 2, 639-665 (2012) · Zbl 1274.62298 · doi:10.1214/11-AOS961
[28] Rissanen, J., Stochastic Complexity in Statistical Inquiry (1989), Singapore: World Scientific, Singapore · Zbl 0800.68508
[29] Schwarz, G., Estimating the dimension of a model, Ann Statist, 6, 2, 461-464 (1978) · Zbl 0379.62005 · doi:10.1214/aos/1176344136
[30] Shao, J., Mathematical Statistics (2003), New York: Springer, New York · Zbl 1018.62001 · doi:10.1007/b97553
[31] Stout, W. F., Almost Sure Convergence (1974), New York: Academic Press, New York · Zbl 0321.60022
[32] Tutz, G., Regression for Categorical Data (2011), Cambridge: Cambridge Univ Press, Cambridge · Zbl 1304.62021 · doi:10.1017/CBO9780511842061
[33] van der Vaart, A. W., Asymptotic Statistics (1998), Cambridge: Cambridge Univ Press, Cambridge · Zbl 0943.62002 · doi:10.1017/CBO9780511802256
[34] Wu, Y.; Zen, M. M., A strongly consistent information criterion for linear model selection based on M-estimation, Probab Theory Related Fields, 113, 4, 599-625 (1999) · Zbl 0973.62050 · doi:10.1007/s004400050219
[35] Yin, C. C.; Zhao, L. C.; Wei, C. D., Asymptotic normality and strong consistency of maximum quasi-likelihood estimates in generalized linear models, Sci China Ser A, 49, 2, 145-157 (2006) · Zbl 1112.62070 · doi:10.1007/s11425-004-5169-x
[36] Zhang H, Jia J. Elastic-net regularized high-dimensional negative binomial regression: consistency and weak signals detection. Statist Sinica (to appear)
[37] Zhang, H. M.; Tan, K.; Li, B., COM-negative binomial distribution: modeling overdispersion and ultrahigh zero-inflated count data, Front Math China, 13, 4, 967-998 (2018) · Zbl 1401.60022 · doi:10.1007/s11464-018-0714-z
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.