Bayes factor testing of multiple intraclass correlations. (English) Zbl 1416.62353

Summary: The intraclass correlation plays a central role in modeling hierarchically structured data, such as educational data, panel data, or group-randomized trial data. It represents relevant information concerning the between-group and within-group variation. Methods for Bayesian hypothesis tests concerning the intraclass correlation are proposed to improve decision making in hierarchical data analysis and to assess the grouping effect across different group categories. Estimation and testing methods for the intraclass correlation coefficient are proposed under a marginal modeling framework where the random effects are integrated out. A class of stretched beta priors is proposed on the intraclass correlations, which is equivalent to shifted \(F\) priors for the between groups variances. Through a parameter expansion it is shown that this prior is conditionally conjugate under the marginal model yielding efficient posterior computation. A special improper case results in accurate coverage rates of the credible intervals even for minimal sample size and when the true intraclass correlation equals zero. Bayes factor tests are proposed for testing multiple precise and order hypotheses on intraclass correlations. These tests can be used when prior information about the intraclass correlations is available or absent. For the noninformative case, a generalized fractional Bayes approach is developed. The method enables testing the presence and strength of grouped data structures without introducing random effects. The methodology is applied to a large-scale survey study on international mathematics achievement at fourth grade to test the heterogeneity in the clustering of students in schools across countries and assessment cycles.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
62P25 Applications of statistics to social sciences
62H20 Measures of association (correlation, canonical correlation, etc.)
62F15 Bayesian inference


Full Text: DOI Euclid


[1] Albert, J. H. and Chib, S. (1993). “Bayesian Analysis of Binary and Polychotomous Response Data.” Journal of the American Statistical Association, 88(422): 669–679. URL http://www.tandfonline.com/doi/abs/10.1080/01621459.1993.10476321 · Zbl 0774.62031
[2] Armagan, A., Dunson, D. B., and Clyde, M. (2011). “Generalized Beta Mixtures of Gaussians.” In Shawe-Taylor, J., Zemel, R., Bartlett, P., Pereira, F., and Weinberger, K. (eds.), Advances in Neural Information Processing Systems 24, 523–531.
[3] Bayarri, M. J., Berger, J. O., Forte, A., and García-Donato, G. (2012). “Criteria for Bayesian model choice with application to variable selection.” The Annals of Statistics, 40: 1550–1577. · Zbl 1257.62023
[4] Berger, J. O. (1994). “An overview of robust Bayesian analysis (with discussion).” Test, 3: 5–124. · Zbl 0827.62026
[5] Berger, J. O. (2006). “The case for objective Bbayesian analysis.” Bayesian Analysis, 1: 385–402. · Zbl 1331.62042
[6] Berger, J. O. and Bernardo, J. (1992). Reference priors in a variance component’s problem, 323–340. London: Oxford University Press.
[7] Berger, J. O., De Oliveira, V., and Sansó, B. (2006). “Objective Bayesian analysis of spatially correlated data.” Journal of the American Statistical Association, 96: 1361–1374. · Zbl 1051.62095
[8] Berger, J. O., Liseo, B., and Wolpert, R. L. (1999). “Integrated likelihood methods for eliminating nuisance parameters.” Statistical Science, 14(1): 1–22. URL http://www.jstor.org/stable/2676641 · Zbl 1059.62521
[9] Berger, J. O. and Mortera, J. (1995). “Discussion to fractional Bayes factors for model comparison (by O’Hagan).” Journal of the Royal Statistical Society Series B, 56: 130.
[10] Berger, J. O. and Pericchi, L. (2001). “Objective Bayesian methods for model selection: Introduction and comparison (with discussion).” In Lahiri, P. (ed.), Model Selection, volume 38 of Monograph Series, 135–207. Beachwood Ohio, institute of mathematical statistics lecture notes edition.
[11] Berger, J. O. and Pericchi, L. R. (1996). “The intrinsic Bayes factor for model selection and prediction.” Journal of the American Statistical Association, 91: 109–122. · Zbl 0870.62021
[12] Berger, J. O. and Sun, D. (2008). “Objective priors for the bivariate normal model.” The Annals of Statistics, 36: 963–982. · Zbl 1133.62014
[13] Böing-Messing, F., van Assen, M. A. L. M., Hofman, A., Hoijtink, H., and Mulder, J. “Bayesian evaluation of constrained hypotheses on variances of multiple independent groups.” Psychological Methods, 22: 262–287.
[14] Box, G. E. P. and Tiao, G. C. (1973). Bayesian Inference in Statistical Snalysis. Reading, MA: Addison-Wesley. · Zbl 0271.62044
[15] Braeken, J., Mulder, J., and Wood, S. (2015). “Relative effects at work: Bayes factors for order hypotheses.” Journal of Management, 41: 544–573.
[16] Chung, Y. and Dey, D. K. (1998). “Bayesian approach to estimation of intraclass correlation using reference prior.” Communications in Statistics, 26: 2241–2255. · Zbl 0917.62019
[17] Conigliani, C. and O’Hagan, A. (2000). “Sensitivity of the fractional Bayes factor to prior distributions.” The Canadian Journal of Statistics, 28: 343–352. · Zbl 0962.62022
[18] Consonni, G. and Paroli, R. (2017). “Objective Bayesian Comparison of Constrained Analysis of Variance Models.” Psychometrika, 82: 589–609. · Zbl 1402.62040
[19] De Santis, F. and Spezzaferri, F. (2001). “Consistent fractional Bayes factor for nested normal linear models.” Journal of Statistical Planning and Inference, 97: 305–321. · Zbl 1015.62021
[20] Donner, A. and Zou, G. (2002). “Testing the equality of dependent intraclass correlation coefficients.” Journal of the Royal Statistical Society Series D, 51: 379–379.
[21] Fox, J.-P. (2010). Bayesian Item Response Modeling: Theory and Applications. New York: Springer. · Zbl 1271.62012
[22] Fox, J.-P., Mulder, J., and Sinharay, S. (2017). “Bayes factor covariance testing in item response models.” Psychometrika, 82: 979–1006. · Zbl 1402.62308
[23] García-Donato, G. and Sun, D. (2007). “Objective priors for hypothesis testing in one-way random effects models.” Canadian Journal of Statistics, 35: 302–320.
[24] Gelman, A. (2006). “Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper).” Bayesian Analysis, 1: 515–534. · Zbl 1331.62139
[25] Ghosh, J. K. and Mukerjee, R. (1992). Non-informative priors, 195–210. Oxford University Press.
[26] Gilks, W. R. (1995). “Discussion to fractional Bayes factors for model comparison (by O’Hagan).” Journal of the Royal Statistical Society Series B, 56: 118–120.
[27] Gu, X., Mulder, J., and Hoijtink, H. (2014). “Bayesian evaluation of inequality constrained hypotheses.” Psychological Methods, 9(4): 511-527.
[28] Gu, X., Mulder, J., Decovíc, M., and Hoijtink, H. (2017). “Approximated adjusted fractional Bayes factors: A general method for testing informative hypotheses.” British Journal of Mathematical and Statistical Psychology. · Zbl 1460.62180
[29] Haldane, J. B. S. (1932). “A note on inverse probability.” Mathematical Proceedings of the Cambridge Philosophical Society, 28: 55–61. · Zbl 0003.35602
[30] Harris, J. A. (1913). “On the calculation of intra-class and inter-class coefficients of correlation from class moments when the number of possible combinations is large.” Biometrika, 9: 446–472.
[31] Hedges, L. V. and Hedberg, E. C. (2007). “Intraclass Correlation Values for Planning Group-Randomized Trials in Education.” Educational Evaluation and Policy Analysis, 29(1): 60–87. URL http://www.jstor.org/stable/30128045
[32] Hoijtink, H. (2011). Informative Hypotheses: Theory and Practice for Behavioral and Social Scientists. New York: Chapman & Hall/CRC.
[33] Hoijtink, H., Gu, X., and Mulder, J. (2018). “Bayesian evaluation of informative hypotheses for multiple populations.” British Journal of Mathematical and Statistical Psychology. · Zbl 1460.62180
[34] Jeffreys, H. (1961). Theory of Probability-3rd ed. New York: Oxford University Press. · Zbl 0116.34904
[35] Johnson, V. E. and Rossell, D. (2010). “On the use of non-local prior densities in Bayesian hypothesis tests.” Journal of the Royal Statistical Society Series B, 72: 143–170.
[36] Kass, R. E. and Vaidyanathan, S. K. (1992). “Approximate Bayes Factors and Orthogonal Parameters, with Application to Testing Equality of Two Binomial Proportions.” Journal of the Royal Statistical Society, Series B, 54: 129–144. · Zbl 0777.62032
[37] Kass, R. E. and Wasserman, L. (1995). “A Reference Bayesian Test for Nested Hypotheses and Its Relationship to the Schwarz Criterion.” Journal of the American Statistical Association, 90: 928–934. · Zbl 0851.62020
[38] Kinney, S. and Dunson, D. B. (2007). “Fixed and Random Effects Selection in Linear and Logistic Models.” Biometrics, 63: 690–698. · Zbl 1147.62022
[39] Kish, L. (1965). Survey sampling. Chichester: Wiley. · Zbl 0151.23403
[40] Konishi, S. and Gupta, K. A. (1989). “Testing the equality of several intraclass correlation coefficients.” Journal of Statistical Planning and Inference, (1): 93–105. · Zbl 0666.62057
[41] Lancaster, H. O. (1965). “The Helmert matrices.” The American Mathematical Monthly, 72: 4–12. · Zbl 0124.01102
[42] Liang, F., Paulo, R., Molina, G., Clyde, M. A., and Berger, J. O. (2008). “Mixtures of \(g\) priors for Bayesian variable selection.” Journal of American Statistical Association, 103(481): 410–423. · Zbl 1335.62026
[43] Liang, K.-Y. and Zeger, S. L. (1986). “Longitudinal data analysis using generalized linear models.” Biometrika, 73(1): 13. · Zbl 0595.62110
[44] Ly, A., Verhagen, J., and Wagenmakers, E.-J. (2016). “An evaluation of alternative methods for testing hypotheses, from the perspective of Harold Jeffreys.” Journal of Mathematical Psychology, 72: 43–55. · Zbl 1357.62118
[45] Mulder, J. (2014). “Prior adjusted default Bayes factors for testing (in)equality constrained hypotheses.” Computational Statistics and Data Analysis, 71: 448–463. · Zbl 1471.62145
[46] Mulder, J. (2016). “Bayes factors for testing order-constrained hypotheses on correlations.” Journal of Mathematical Psychology, 72: 104–115. · Zbl 1357.62122
[47] Mulder, J. and Fox, J.-P. (2013). “Bayesian tests on components of the compound symmetry covariance matrix.” Statistics and Computing, 23: 109–122. · Zbl 1322.62111
[48] Mulder, J. and Fox, J.-P. (2018). “Supplemetary material for “Bayes factor testing of multiple intraclass correlations”.” Bayesian Analysis.
[49] Mulder, J., Klugkist, I., van de Schoot, A., Meeus, W., Selfhout, M., and Hoijtink, H. (2009). “Bayesian Model Selection of Informative Hypotheses for Repeated Measurements.” Journal of Mathematical Psychology, 53: 530–546. · Zbl 1181.62026
[50] Mulder, J. and Pericchi, L. R. (2018). “The matrix-\(F\) prior for estimating and testing covariance matrices.” Bayesian Analysis. · Zbl 1407.62080
[51] Mullis, I., Martin, M. O., Foy, P., and Arora, A. (2011). TIMSS 2011 International Results in Mathematics. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College Chestnut Hill, MA, USA and International Association for the Evaluation of Educational Achievement (IEA) IEA Secretariat, Amsterdam, the Netherlands, first edition.
[52] O’Hagan, A. (1995). “Fractional Bayes Factors for Model Comparison (with discussion).” Journal of the Royal Statistical Society Series B, 57: 99–138.
[53] O’Hagan, A. (1997). “Properties of intrinsic and fractional Bayes factors.” Test, 6: 101–118. · Zbl 0891.62001
[54] Olson, J. F., Martin, M. O., and Mullis, I. V. S. (2008). TIMSS 2007 Technical Report. Chestnut Hill, MA: TIMSS and PIRLS International Study Center, Boston College, first edition.
[55] Pauler, D. K., Wakefield, J. C., and Kass, R. E. (1999). “Bayes Factors and Approximations for Variance Component Models.” Journal of the American Statistical Association, 94(448): 1242–1253. · Zbl 0998.62017
[56] Pérez, M. E., Pericchi, L. R., and Ramirez, I. C. (2017). “The Scaled Beta2 Distribution as a Robust Prior for Scales.” Bayesian Analysis, 12: 615. · Zbl 1384.62048
[57] Pericchi, L. R. (2005). “Model selection and hypothesis testing based on objective probabilities and Bayes Factors.” In D. K. Dey, C. R. Rao (eds.), Bayesian Thinking: Modeling and Computation, 115–149. Elsevier, Amsterdam, The Netherlands.
[58] Pericchi, L. R., Liu, G., and Torres, D. (2008). “Objective Bayes factors for informative hypotheses: “Completing” the informative hypothesis and “splitting” the Bayes factors.” In H. Hoijtink, I. Klugkist, and P. Boelen (eds.), Bayesian Evaluation of Informative Hypotheses, 131–154. New York: Springer.
[59] Polson, N. G. and Scott, J. G. (2012). “On the Half-Cauchy Prior for a Global Scale Parameter.” Bayesian Analysis, 7. · Zbl 1330.62148
[60] Raftery, A. E. (1995). “Bayesian model selection in social research.” Sociological Methodology, 25: 111–163.
[61] Raudenbush, S. W. (1997). “Statistical analysis and optimal design for cluster randomized trials.” Psychological Methods, (2): 173–185. URL http://dx.doi.org/10.1037/1082-989X.2.2.173
[62] Schwarz, G. E. (1978). “Estimating the dimension of a model.” Annals of Statistics, 6: 461–464. · Zbl 0379.62005
[63] Scott, J. G. and Berger, J. O. (2006). “An exploration of aspects of Bayesian multiple testing.” Journal of Statistical Planning and Inference, 136: 2144–2162. · Zbl 1087.62039
[64] Searle, S. R., Casella, G., and McCulloch, C. E. (1992). Variance Components. John Wiley & Sons, Inc., Hoboken, NJ, USA, first edition. · Zbl 1108.62064
[65] Sellke, T., Bayarri, M. J., and Berger, J. O. (2001). “Calibration of \(p\) Values for Testing Precise Null Hypotheses.” The American Statistician, 55: 62–71. · Zbl 1182.62053
[66] Severini, T. A., Mukerjee, R., and Ghosh, M. (2002). “On an exact probability matching property of right-invariant priors.” Biometrika, 89: 952–957. · Zbl 1034.62022
[67] Silvapulle, M. J. and Sen, P. K. (2004). Constrained Statistical Inference: Inequality, Order, and Shape Restrictions. Hoboken, NJ: John Wiley, second edition. · Zbl 1077.62019
[68] Spiegelhalter, D. J. (2001). “Bayesian methods for cluster randomized trials with continuous responses.” Statistics in Medicine, 20(3): 435–452. URL http://dx.doi.org/10.1002/1097-0258(20010215)20:3<435::AID-SIM804>3.0.CO;2-E
[69] Stein, C. (1985). On the coverage probability of confidence sets based on a prior distribution, 485–514. PWN-Polish Scientific Publishers.
[70] Van Geel, M., Keuning, T., Visscher, A., and Fox, J.-P. (2017). “Changes in educators’ data literacy during a data-based decision making intervention.” Teaching and Teacher Education, 64: 187–198.
[71] Welch, B. and Peers, H. W. (1963). “On formulae for confidence points based on integrals of weighted likelihoods.” Journal of the Royal Statistical Society Series B, 25: 318–29. · Zbl 0117.14205
[72] Westfall, P. and Gönen, M. (1996). “Asymptotic properties of ANOVA Bayes factors.” Communications in Statistics: Theory and Methods, 25: 3101–3123. · Zbl 0887.62032
[73] Ye, K. (1994). “Bayesian reference prior analysis on the ratio of variances for the balanced one-way random effect model.” Journal of Statistical Planning and Inference, 41: 267–280. · Zbl 0816.62029
[74] Zellner, A. (1986). “On assessing prior distributions and Bayesian regression analysis with \(g\)-prior distributions.” In Goel, P. K. and Zellner, A. (eds.), Bayesian inference and decision techniques: Essays in honor of Bruno de Finetti, 233–243. Amsterdam: Elsevier. · Zbl 0655.62071
[75] Zellner, A. and Siow, A. (1980). “Posterior odds ratios for selected regression hypotheses. · Zbl 0457.62004
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.