The influence of violations of assumptions on multilevel parameter estimates and their standard errors. (English) Zbl 1429.62085

Summary: A crucial problem in the statistical analysis of hierarchically structured data is the dependence of the observations at the lower levels. Multilevel modeling programs account for this dependence and in recent years these programs have been widely accepted. One of the major assumptions of the tests of significance used in the multilevel programs is normality of the error distributions involved. Simulations were used to assess how important this assumption is for the accuracy of multilevel parameter estimates and their standard errors. Simulations varied the number of groups, the group size, and the intraclass correlation, with the second level residual errors following one of three non-normal distributions. In addition asymptotic maximum likelihood standard errors are compared to robust (Huber/White) standard errors.The results show that non-normal residuals at the second level of the model have little or no effect on the parameter estimates. For the fixed parameters, both the maximum likelihood-based standard errors and the robust standard errors are accurate. For the parameters in the random part of the model, the maximum likelihood-based standard errors at the lowest level are accurate, while the robust standard errors are often overcorrected. The standard errors of the variances of the level-two random effects are highly inaccurate, although the robust errors do perform better than the maximum likelihood errors. For good accuracy, robust standard errors need at least 100 groups. Thus, using robust standard errors as a diagnostic tool seems to be preferable to simply relying on them to solve the problem.


62F10 Point estimation


Full Text: DOI Link


[1] Afshartous, D., 1995. Determination of sample size for multilevel model design. Unpublished paper. Annual Meeting of the American Educational Research Association, San Francisco, CA.
[2] Browne, W.J., 1998. Applying MCMC methods to multilevel models. Unpublished Ph.D. Thesis, University of Bath, Bath, UK.
[3] Browne, W.J.; Draper, D., Implementation and performance issues in the Bayesian and likelihood Fitting of multilevel models, Comput. statist., 15, 391-420, (2000) · Zbl 1037.62013
[4] Bryk, A.S.; Raudenbush, S.W., Hierarchical linear models, (1992), Sage Newbury Park, CA
[5] Bryk, A.S.; Raudenbush, S.W.; Congdon, R.T., HLM. hierarchical linear and nonlinear modeling with the HLM/2L and HLM/3L programs, (1996), Scientific Software International Chicago
[6] Busing, F., 1993. Distribution characteristics of variance estimates in two-level models. Unpublished manuscript. Department of Psychometrics and Research Methodology, Leiden University, Leiden.
[7] Carpenter, J., Goldstein, H., Rasbash, J., 1999. A non-parametric bootstrap for multilevel models. Multilevel Modelling Newslett. 11, 1, 2-5.
[8] Cohen, J., Statistical power analysis for the behavioral sciences, (1988), Lawrence Erlbaum Associates Mahwah, NJ · Zbl 0747.62110
[9] Eliason, S.R., Maximum likelihood estimation, (1993), Sage Newbury Park, CA
[10] Evans, M.; Hastings, N.; Peacock, B., Statistical distributions, (1993), Wiley New York
[11] Goldstein, H., 1995. Multilevel Statistical Models. Edward Arnold, London; Halsted, New York. · Zbl 1014.62126
[12] Gulliford, M.C.; Ukoumunne, O.C.; Chinn, S., Components of variance and intraclass correlations for the design of community-based surveys and intervention studies, Amer. J. epidemiol., 149, 876-883, (1999)
[13] Hox, J.J., Multilevel modeling: when and why, (), 147-154
[14] Hox, J.J., Multilevel analysis, techniques and applications, (2002), Lawrence Erlbaum Associates Mahwah, NJ · Zbl 1226.62001
[15] Hox, J.J.; Maas, C.J.M., The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples, Struct. equation modeling, 8, 157-174, (2001)
[16] Huber, P.J., 1967. The behavior of maximum likelihood estimates under non-standard conditions. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, Berkeley, pp. 221-233.
[17] Kish, L., Survey sampling, (1965), Wiley New York · Zbl 0151.23403
[18] Kreft, I.; de Leeuw, J., Introducing multilevel modeling, (1998), Sage Newbury Park, CA
[19] Leeuw, J.de.; Kreft, I.G.G., Random coefficient models for multilevel analysis, J. ed. statist., 11, 57-85, (1986)
[20] Longford, N.T., Random coefficient models, (1993), Clarendon Press Oxford · Zbl 0859.62064
[21] Maas, C.J.M., Hox, J.J., 2003. Sufficient sample sizes for multilevel modeling. Department of Methodology and Statistics, Utrecht University, NL, submitted for publication.
[22] Muthén, B.; Satorra, A., Complex sample data in structural equation modeling, (), 267-316
[23] Rasbash, J.; Browne, W.; Goldstein, H.; Yang, M.; Plewis, I.; Healy, M.; Woodhouse, G.; Draper, D.; Langford, I.; Lewis, T., A User’s guide to mlwin. multilevel models project, (2000), University of London London
[24] Raudenbush, S.W.; Bryk, A.S., Hierarchical linear models, (2002), Sage Thousand Oaks, CA
[25] Raudenbush, S.; Bryk, A.; Cheong, Y.F.; Congdon, R., HLM 5. hierarchical linear and nonlinear modeling, (2000), Scientific Software International Chicago
[26] Scott Long, J.; Ervin, L.H., Using heteroscedasticity consistent standard errors in the linear regression model, Amer. statist., 54, 217-224, (2000)
[27] Snijders, T.A.B.; Bosker, R., Multilevel analysis. an introduction to basic and advanced multilevel modeling, (1999), Sage Thousand Oaks, CA · Zbl 0953.62127
[28] Tabachnick, B.G.; Fidell, L.S., Using multivariate statistics, (1996), HarperCollins Publishers Inc New York
[29] Van der Leeden, R., Busing, F., 1994. First iteration versus IGLS/RIGLS estimates in two-level models: a Monte Carlo study with ML3. Unpublished manuscript. Department of Psychometrics and Research Methodology, Leiden University, Leiden.
[30] Van der Leeden, R., Busing, F., Meijer, E., 1997. Applications of bootstrap methods for two-level models. Unpublished paper. Multilevel Conference, Amsterdam, April 1-2.
[31] White, H., Maximum likelihood estimation of misspecified models, Econometrica, 50, 1-25, (1982) · Zbl 0478.62088
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.