zbMATH — the first resource for mathematics

An Akaike information criterion for model selection in the presence of incomplete data. (English) Zbl 1067.62504
Summary: We derive and investigate a variant of AIC, the Akaike information criterion, for model selection in settings where the observed data is incomplete. Our variant is based on the motivation provided for the PDIO (”predictive divergence for incomplete observation models”) criterion of H. Shimodaira [in: Selecting Models from Data: Artificial Intelligence and Statistics IV, Lect. Notes Stat. 89, 21–29 (1994; Zbl 0828.62004)]. However, our variant differs from PDIO in its ”goodness-of-fit” term. Unlike AIC and PDIO, which require the computation of the observed-data empirical log-likelihood, our criterion can be evaluated using only complete-data tools, readily available through the EM algorithm and the SEM (”supplemented” EM) algorithm of X. Meng and D. Rubin [J. Am. Stat. Assoc. 86, 899–909 (1991)]. We compare the performance of our AIC variant to that of both AIC and PDIO in simulations where the data being modeled contains missing values. The results indicate that our criterion is less prone to overfitting than AIC and less prone to underfitting than PDIO.

62B10 Statistical aspects of information-theoretic topics
PDF BibTeX Cite
Full Text: DOI
[1] Akaike, H., Information theory and an extension of the maximum likelihood principle, (), 267-281 · Zbl 0283.62006
[2] Akaike, H., A new look at the statistical model identification, IEEE trans. automat. control AC-19, 716-723, (1974) · Zbl 0314.62039
[3] Bedrick, E.J.; Tsai, C.L., Model selection for multivariate regression in small samples, Biometrics, 50, 226-231, (1994) · Zbl 0825.62564
[4] Bhansali, R.J., Order selection for linear time series models: A review, (), 50-66 · Zbl 0883.62095
[5] Dempster, A.P.; Laird, N.M.; Rubin, D.B., Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion), J. roy. statist. soc. ser. B, 39, 1-38, (1977) · Zbl 0364.62022
[6] Goodman, L.A., Exploratory latent structure models using both identifiable and unidentifiable models, Biometrika, 61, 315-331, (1974) · Zbl 0281.62057
[7] Hurvich, C.M.; Shumway, R.H.; Tsai, C.L., Improved estimators of Kullback-Leibler information for autoregressive model selection in small samples, Biometrika, 77, 709-719, (1990)
[8] Hurvich, C.M.; Tsai, C.L., Regression and time series model selection in small samples, Biometrika, 76, 297-307, (1989) · Zbl 0669.62085
[9] Kullback, S., Information theory and statistics, (1968), Dover New York · Zbl 0149.37901
[10] Linhart, H.; Zucchini, W., Model selection, (1986), Wiley New York · Zbl 0665.62003
[11] Little, R.J.A., Maximum likelihood inference for multiple regression with missing values: A simulation study, J. roy. statist. soc. ser. B, 41, 76-87, (1979) · Zbl 0399.62018
[12] Little, R.J.A.; Rubin, D.B., Statistical analysis with missing data, (1987), Wiley New York
[13] Meng, X.L.; Rubin, D.B., Using EM to obtain asymptotic variance-covariance matrices: the SEM algorithm, J. amer. statist. assoc., 86, 899-909, (1991)
[14] Rubin, D.B., Noniterative least squares estimates, standard errors and F-tests for any analysis of variance with missing data, J. roy. statist. soc. ser. B, 38, 270-274, (1976) · Zbl 0344.62062
[15] Shibata, R., Asymptotically efficient selection of the order of the model for estimating parameters of a linear process, Ann. statist., 80, 147-164, (1980) · Zbl 0425.62069
[16] Shimodaira, H., A new criterion for selecting models from partially observed data, (), 21-29 · Zbl 0828.62004
[17] Shumway, R.H.; Stoffer, D.S., An approach to time series smoothing and forecasting using the EM algorithm, J. time series anal., 3, 253-264, (1982) · Zbl 0502.62085
[18] Titterington, D.M.; Smith, A.F.M.; Makov, U.E., Statistical analysis of finite mixture distributions, (1985), Wiley New York · Zbl 0646.62013
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.