×
Chang, Jinyuan; Tang, Cheng Yong; Wu, Tong Tong

A new scope of penalized empirical likelihood with high-dimensional estimating equations. (English) Zbl 1408.62053

Ann. Stat. 46, No. 6B, 3185-3216 (2018).
Summary: Statistical methods with empirical likelihood (EL) are appealing and effective especially in conjunction with estimating equations for flexibly and adaptively incorporating data information. It is known that EL approaches encounter difficulties when dealing with high-dimensional problems. To overcome the challenges, we begin our study with investigating high-dimensional EL from a new scope targeting at high-dimensional sparse model parameters. We show that the new scope provides an opportunity for relaxing the stringent requirement on the dimensionality of the model parameters. Motivated by the new scope, we then propose a new penalized EL by applying two penalty functions respectively regularizing the model parameters and the associated Lagrange multiplier in the optimizations of EL. By penalizing the Lagrange multiplier to encourage its sparsity, a drastic dimension reduction in the number of estimating equations can be achieved. Most attractively, such a reduction in dimensionality of estimating equations can be viewed as a selection among those high-dimensional estimating equations, resulting in a highly parsimonious and effective device for estimating high-dimensional sparse model parameters. Allowing both the dimensionalities of model parameters and estimating equations growing exponentially with the sample size, our theory demonstrates that our new penalized EL estimator is sparse and consistent with asymptotically normally distributed nonzero components. Numerical simulations and a real data analysis show that the proposed penalized EL works promisingly.

Cited in 23 Documents

MSC:

62G05 Nonparametric estimation
62G99 Nonparametric inference
62G20 Asymptotic properties of nonparametric inference

Keywords:

empirical likelihood; estimating equations; high-dimensional statistical methods; moment selection; penalized likelihood
Cite Review PDF
Full Text: DOI arXiv Euclid

References:

[1] Bartolucci, F. (2007). A penalized version of the empirical likelihood ratio for the population mean. Statist. Probab. Lett.77 104–110. · Zbl 1106.62050 · doi:10.1016/j.spl.2006.05.016
[2] Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when \(p\) is much larger than \(n\). Ann. Statist.35 2313–2351. · Zbl 1139.62019 · doi:10.1214/009053607000000442
[3] Chang, J., Chen, S. X. and Chen, X. (2015). High dimensional generalized empirical likelihood for moment restrictions with dependent data. J. Econometrics185 283–304. · Zbl 1331.62188 · doi:10.1016/j.jeconom.2014.10.011
[4] Chang, J., Tang, C. Y. and Wu, Y. (2013). Marginal empirical likelihood and sure independence feature screening. Ann. Statist.41 2123–2148. · Zbl 1277.62109 · doi:10.1214/13-AOS1139
[5] Chang, J., Tang, C. Y. and Wu, Y. (2016). Local independence feature screening for nonparametric and semiparametric models by marginal empirical likelihood. Ann. Statist.44 515–539. · Zbl 1486.62082 · doi:10.1214/15-AOS1374
[6] Chang, J., Tang, C. Y. and Wu, T. T. (2018). Supplement to “A new scope of penalized empirical likelihood with high-dimensional estimating equations.” DOI:10.1214/17-AOS1655SUPP.
[7] Chen, X. (2007). Large sample sieve estimation of semi-nonparametric models. In The Handbook of Econometrics, 6B (J. J. Heckman and E. Leamer, eds.). North- Holland, Amsterdam.
[8] Chen, J. and Chen, Z. (2008). Extended Bayesian information criterion for model selection with large model space. Biometrika95 759–771. · Zbl 1437.62415 · doi:10.1093/biomet/asn034
[9] Chen, S. X. and Cui, H. (2006). On Bartlett correction of empirical likelihood in the presence of nuisance parameters. Biometrika93 215–220. · Zbl 1152.62325 · doi:10.1093/biomet/93.1.215
[10] Chen, S. X. and Cui, H. (2007). On the second-order properties of empirical likelihood with moment restrictions. J. Econometrics141 492–516. · Zbl 1407.62157 · doi:10.1016/j.jeconom.2006.10.006
[11] Chen, S. X., Peng, L. and Qin, Y. L. (2009). Effects of data dimension on empirical likelihood. Biometrika96 711–722. · Zbl 1170.62023 · doi:10.1093/biomet/asp037
[12] Chen, X. and Pouzo (2012). Sieve quasi likelihood ratio inference on semi/nonparametric conditional moment models. Econometrica80 277–321. · Zbl 1274.62232 · doi:10.3982/ECTA7888
[13] Cheng, X. and Liao, Z. (2015). Select the valid and relevant moments: An information-based LASSO for GMM with many moments. J. Econometrics186 443–464. · Zbl 1331.62459 · doi:10.1016/j.jeconom.2015.02.019
[14] Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc.96 1348–1360. · Zbl 1073.62547 · doi:10.1198/016214501753382273
[15] Friedman, J., Hastie, T., Hoefling, H. and Tibshirani, R. (2007). Pathwise coordinate optimization. Ann. Appl. Stat.2 302–332. · Zbl 1378.90064 · doi:10.1214/07-AOAS131
[16] Gautier, E. and Tsybakov, A. B. (2014). High-dimensional instrumental variables regression and confidence sets. Manuscript, arXiv:1105.2454v4.
[17] Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators. Econometrica50 1029–1054. · Zbl 0502.62098 · doi:10.2307/1912775
[18] Hjort, N. L., McKeague, I. W. and Van Keilegom, I. (2009). Extending the scope of empirical likelihood. Ann. Statist.37 1079–1111. · Zbl 1160.62029 · doi:10.1214/07-AOS555
[19] Lahiri, S. N. and Mukhopadhyay, S. (2012). A penalized empirical likelihood method in high dimensions. Ann. Statist.40 2511–2540. · Zbl 1373.62132 · doi:10.1214/12-AOS1040
[20] Leng, C. and Tang, C. Y. (2012). Penalized empirical likelihood and growing dimensional general estimating equations. Biometrika99 703–716. · Zbl 1437.62522 · doi:10.1093/biomet/ass014
[21] Liang, K. Y. and Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika73 13–22. · Zbl 0595.62110 · doi:10.1093/biomet/73.1.13
[22] Lv, J. and Fan, Y. (2009). A unified approach to model selection and sparse recovery using regularized least squares. Ann. Statist.37 3498–3528. · Zbl 1369.62156 · doi:10.1214/09-AOS683
[23] Newey, W. K. and Smith, R. J. (2004). Higher order properties of GMM and generalized empirical likelihood estimators. Econometrica72 219–255. · Zbl 1151.62313 · doi:10.1111/j.1468-0262.2004.00482.x
[24] Owen, A. (1988). Empirical likelihood ratio confidence intervals for a single functional. Biometrika75 237–249. · Zbl 0641.62032 · doi:10.1093/biomet/75.2.237
[25] Owen, A. (1990). Empirical likelihood ratio confidence regions. Ann. Statist.18 90–120. · Zbl 0712.62040 · doi:10.1214/aos/1176347494
[26] Owen, A. (2001). Empirical Likelihood. Chapman & Hall-CRC, New York. · Zbl 0989.62019
[27] Petrov, V. V. (1995). Limit Theorems of Probability Theory: Sequences of Independent Random Variables. Oxford Univ. Press, Oxford. · Zbl 0826.60001
[28] Qin, J. and Lawless, J. (1994). Empirical likelihood and general estimating equations. Ann. Statist.22 300–325. · Zbl 0799.62049 · doi:10.1214/aos/1176325370
[29] Qu, A., Lindsay, B. G. and Li, B. (2000). Improving estimating equations using quadratic inference functions. Biometrika87 823–836. · Zbl 1028.62045 · doi:10.1093/biomet/87.4.823
[30] Rudin, W. (1976). Principles of Mathematical Analysis. McGraw-Hill, New York. · Zbl 0346.26002
[31] Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist.6 461–464. · Zbl 0379.62005 · doi:10.1214/aos/1176344136
[32] Shi, Z. (2016). Econometric estimation with high-dimensional moment equalities. J. Econometrics195 104–119. · Zbl 1443.62506 · doi:10.1016/j.jeconom.2016.07.004
[33] Tang, C. Y. and Leng, C. (2010). Penalized high dimensional empirical likelihood. Biometrika97 905–920. · Zbl 1204.62050 · doi:10.1093/biomet/asq057
[34] Tang, C. Y. and Wu, T. T. (2014). Nested coordinate descent algorithms for empirical likelihood. J. Stat. Comput. Simul.84 1917–1930. · Zbl 1453.62436
[35] Tsao, M. (2004). Bounds on coverage probabilities of the empirical likelihood ratio confidence regions. Ann. Statist.32 1215–1221. · Zbl 1091.62040 · doi:10.1214/009053604000000337
[36] Tsao, M. and Wu, F. (2013). Empirical likelihood on the full parameter space. Ann. Statist.41 2176–2196. · Zbl 1360.62140 · doi:10.1214/13-AOS1143
[37] Tsao, M. and Wu, F. (2014). Extended empirical likelihood for estimating equations. Biometrika101 703–710. · Zbl 1334.62044 · doi:10.1093/biomet/asu014
[38] Wang, H., Li, B. and Leng, C. (2009). Shrinkage tuning parameter selection with a diverging number of parameters. J. R. Stat. Soc. Ser. B. Stat. Methodol.71 671–683. · Zbl 1250.62036 · doi:10.1111/j.1467-9868.2008.00693.x
[39] Wu, T. T. and Lange, K. (2008). Coordinate descent algorithms for lasso penalized regression. Ann. Appl. Stat.2 224–244. · Zbl 1137.62045 · doi:10.1214/07-AOAS147
[40] Zhang, C. H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist.38 894–942. · Zbl 1183.62120 · doi:10.1214/09-AOS729
[41] Zhao, P. and Yu, B. (2006). On model selection consistency of Lasso. J. Mach. Learn. Res.7 2541–2563. · Zbl 1222.62008
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.