zbMATH — the first resource for mathematics

Consensus theory for multiple latent traits and consensus groups. (English) Zbl 1448.91243
Summary: We consider a situation in which a group of respondents answers a set of questions and the aim is to identify any consensus among the respondents – that is, shared attitudes, beliefs, or knowledge. Consensus theory postulates that a latent trait determines the respondents’ probability to produce the consensus response. We propose a new version of the variable-response model, which implements consensus theory for numerical continuous responses, ordered categorical responses, unordered categorical responses, or a mixture thereof. The new model also accounts for multiple consensus groups and multiple latent traits underlying the response data. In a series of simulation studies, we identify procedures and conditions that permit an accurate estimation of the number of consensus groups and latent traits. In these simulations, we find that the model recovers the data-generating consensus responses well. We replicate these findings with the empirical data of a memory test.
Reviewer: Reviewer (Berlin)

91E45 Measurement and performance in psychology
Full Text: DOI
[1] Akaike, H., Information theory and an extension of the maximum likelihood principle, (Petrov, B. N.; Csaki, F., Proceeding of the second international symposium on information theory (1973), Akademiai Kiado: Akademiai Kiado Budapest), 267-281 · Zbl 0283.62006
[2] Anders, R.; Batchelder, W. H., Cultural consensus theory for multiple consensus truths, Journal of Mathematical Psychology, 56, 452-469 (2012)
[3] Anders, R.; Batchelder, W. H., Cultural consensus theory for the ordinal data case, Psychometrika, 80, 151-181 (2015) · Zbl 1314.62267
[4] Anders, R.; Oravecz, Z.; Batchelder, W. H., Cultural consensus theory for continuous responses: A latent appraisal model for information pooling, Journal of Mathematical Psychology, 61, 1-13 (2014) · Zbl 1309.91108
[5] Aßfalg, A., Consensus theory for mixed response formats, Journal of Mathematical Psychology, 86, 51-63 (2018) · Zbl 1416.62646
[6] Aßfalg, A.; Erdfelder, E., CAML—Maximum likelihood consensus analysis, Behavior Research Methods, 44, 189-201 (2012)
[7] Birnbaum, A., Some latent train models and their use in inferring an examinee’s ability, (Lord, F. M.; Novick, M. R., Statistical theories of mental test scores (1968), Addison-Wesley: Addison-Wesley Reading, MA), 395-479
[8] Galton, F., Vox populi, Nature, 75, 450-451 (1907) · JFM 38.0288.04
[9] Gelman, A.; Hwang, J.; Vehtari, A., Understanding predictive information criteria for Bayesian models, Statistics and Computing, 24, 997-1016 (2014) · Zbl 1332.62090
[10] Hastings, W. K., Monte Carlo sampling methods using Markov chains and their applications, Biometrika, 57, 97-109 (1970) · Zbl 0219.65008
[11] Karabatsos, G.; Batchelder, W. H., Markov chain estimation for test theory without an answer key, Psychometrika, 68, 373-389 (2003) · Zbl 1306.62447
[12] Metropolis, N.; Rosenbluth, A. W.; Rosenbluth, M. N.; Teller, A. H.; Teller, E., Equation of state calculations by fast computing machines, The Journal of Chemical Physics, 21, 1087-1092 (1953) · Zbl 1431.65006
[13] Mueller, S. T.; Veinott, E. S., Cultural mixture modeling: A method for identifying cultural consensus, ARA Technology Review, 4, 39-45 (2008)
[14] Plummer, M.; Cowles, K.; Vines, K.; Sarkar, D.; Bates, D.; Almond, R., coda: Output analysis and diagnostics for MCMCR package version 0.19-3 (2019)
[15] Waubert de Puiseau, B.; Aßfalg, A.; Erdfelder, E.; Bernstein, D. M., Extracting the truth from conflicting eyewitness reports: A formal modeling approach, Journal of Experimental Psychology: Applied, 18, 390-403 (2012)
[16] Waubert de Puiseau, B.; Greving, S.; Aßfalg, A.; Musch, J., On the importance of considering heterogeneity in witnesses’ competence levels when reconstructing crimes from multiple witness testimonies, Psychological Research, 81, 947-960 (2017)
[17] Rasch, G., On general laws and the meaning of measurement in psychology, (Neyman, J., Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, Vol. 4 (1961), University of California Press: University of California Press Berkeley, CA), 321-333 · Zbl 0107.36805
[18] Reckase, M. D., A linear logistic multidimensional model for dichotomous item response data, (Van Der Linden, W. J.; Hambleton, R. K., Handbook of modern item response theory (1997), Springer: Springer New York), 271-286
[19] Romney, A. K.; Weller, S. C.; Batchelder, W. H., Culture as consensus: A theory of culture and informant accuracy, American Anthropologist, 88, 313-338 (1986)
[20] Schwarz, G., Estimating the dimension of a model, The Annals of Statistics, 6, 461-464 (1978) · Zbl 0379.62005
[21] Spiegelhalter, D. J.; Best, N. G.; Carlin, B. P.; Van Der Linde, A., Bayesian measures of model complexity and fit, Journal of the Royal Statistical Society. Series B. Statistical Methodology, 64, 583-639 (2002) · Zbl 1067.62010
[22] Vehtari, A.; Gelman, A.; Simpson, D.; Carpenter, B.; Bürkner, P. C., Rank-normalization, folding, and localization: An improved \(\hat{R}\) for assessing convergence of mcmc (2019), arXiv preprint: 1903.08008
[23] Watanabe, S., Equations of states in singular statistical estimation, Neural Networks, 23, 20-34 (2010) · Zbl 1396.68106
[24] Weller, S. C., Cultural consensus theory: Applications and frequently asked questions, Field Methods, 19, 339-368 (2007)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.