×
Chen, Yinghan; Culpepper, Steven Andrew; Chen, Yuguo; Douglas, Jeffrey

Bayesian estimation of the DINA \(Q\) matrix. (English) Zbl 1402.62302

Psychometrika 83, No. 1, 89-108 (2018).
Summary: Cognitive diagnosis models are partially ordered latent class models and are used to classify students into skill mastery profiles. The deterministic inputs, noisy “and” gate model (DINA) is a popular psychometric model for cognitive diagnosis. Application of the DINA model requires content expert knowledge of a \(Q\) matrix, which maps the attributes or skills needed to master a collection of items. Misspecification of \(Q\) has been shown to yield biased diagnostic classifications. We propose a Bayesian framework for estimating the DINA \(Q\) matrix. The developed algorithm builds upon prior research [Y. Chen et al., J. Am. Stat. Assoc. 110, No. 510, 850–866 (2015; Zbl 1373.62565)] and ensures the estimated \(Q\) matrix is identified. Monte Carlo evidence is presented to support the accuracy of parameter recovery. The developed methodology is applied to Tatsuoka’s fraction-subtraction dataset.

Cited in 28 Documents

MSC:

62P15 Applications of statistics to psychology
62H30 Classification and discrimination; cluster analysis (statistical aspects)
91E10 Cognitive psychology

Keywords:

cognitive diagnosis models; deterministic inputs; noisy “and” gate (DINA) model; \(Q\) matrix; Bayesian statistics; fraction-subtraction data

Citations:

Zbl 1373.62565
PDF BibTeX XML Cite
\textit{Y. Chen} et al., Psychometrika 83, No. 1, 89--108 (2018; Zbl 1402.62302)
Full Text: DOI

References:

[1] Chen, Y; Liu, J; Xu, G; Ying, Z, Statistical analysis of Q-matrix based diagnostic classification models, Journal of the American Statistical Association, 110, 850-866, (2015) · Zbl 1373.62565
[2] Chiu, C-Y, Statistical refinement of the Q-matrix in cognitive diagnosis, Applied Psychological Measurement, 37, 598-618, (2013)
[3] Chiu, C-Y; Douglas, JA; Li, X, Cluster analysis for cognitive diagnosis: theory and applications, Psychometrika, 74, 633-665, (2009) · Zbl 1179.62087
[4] Chiu, C-Y; Köhn, H-F, The reduced RUM as a logit model: parameterization and constraints, Psychometrika, 81, 350-370, (2016) · Zbl 1345.62142
[5] Chiu, C-Y; Köhn, H-F; Wu, H-M, Fitting the reduced RUM with mplus: A tutorial, International Journal of Testing, 16, 1-21, (2016)
[6] Chung, M. (2014). Estimating the Q-matrix for Cognitive Diagnosis Models in a Bayesian Framework (Unpublished doctoral dissertation). Columbia University. · Zbl 1111.62381
[7] Cowles, M. K., & Carlin, B. P. (1996). Markov chain Monte Carlo convergence diagnostics: A comparative review. Journal of the American Statistical Association, 91, 883-904. · Zbl 0869.62066
[8] Culpepper, SA, Bayesian estimation of the DINA model with Gibbs sampling, Journal of Educational and Behavioral Statistics, 40, 454-476, (2015)
[9] DeCarlo, LT, On the analysis of fraction subtraction data: the DINA model, classification, latent class sizes, and the Q-matrix, Applied Psychological Measurement, 35, 8-26, (2010)
[10] DeCarlo, LT, Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model, Applied Psychological Measurement, 36, 447-468, (2012)
[11] Torre, J, An empirically based method of Q-matrix validation for the DINA model: development and applications, Journal of Educational Measurement, 45, 343-362, (2008)
[12] de la Torre, J. (2009). Estimation code for the G-DINA model. In Presentation at the meeting of the American Educational Research Association. San Diego, CA.
[13] Torre, J; Chiu, C-Y, A general method of empirical Q-matrix validation, Psychometrika, 81, 253-273, (2016) · Zbl 1345.62167
[14] Torre, J; Douglas, JA, Higher-order latent trait models for cognitive diagnosis, Psychometrika, 69, 333-353, (2004) · Zbl 1306.62527
[15] Torre, J; Douglas, JA, Model evaluation and multiple strategies in cognitive diagnosis: an analysis of fraction subtraction data, Psychometrika, 73, 595-624, (2008) · Zbl 1284.62777
[16] Geweke, J. (1992). Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. Bayesian Statistics, 4, 169-188.
[17] Henson, R., & Templin, J. (2007). Importance of Q-matrix construction and its effects cognitive diagnosis model results. In Annual meeting of the national council on measurement in education. Chicago, IL.
[18] Henson, R; Templin, J; Willse, J, Defining a family of cognitive diagnosis models using log-linear models with latent variables, Psychometrika, 74, 191-210, (2009) · Zbl 1243.62140
[19] Huff, K; Goodman, DP; Leighton, JP (ed.); Gierl, MJ (ed.), The demand for cognitive diagnostic assessment, 19-60, (2007), Cambridge
[20] Junker, BW; Sijtsma, K, Cognitive assessment models with few assumptions, and connections with nonparametric item response theory, Applied Psychological Measurement, 25, 258-272, (2001)
[21] Leighton, JP; Gierl, MJ; Leighton, JP (ed.); Gierl, MJ (ed.), Why cognitive diagnostic assessment, 3-18, (2007), Cambridge
[22] Liu, J, On the consistency of Q-matrix estimation: A commentary, Psychometrika, 82, 523-527, (2017) · Zbl 1402.62314
[23] Liu, J; Xu, G; Ying, Z, Data-driven learning of Q-matrix, Applied Psychological Measurement, 36, 548-564, (2012)
[24] Liu, J; Xu, G; Ying, Z, Theory of the self-learning Q-matrix, Bernoulli, 19, 1790-1817, (2013) · Zbl 1294.68118
[25] Mislevy, RJ; Wilson, M, Marginal maximum likelihood estimation for a psychometric model of discontinuous development, Psychometrika, 61, 41-71, (1996) · Zbl 0866.62077
[26] Norris, SP; Macnab, JS; Phillips, LM; Leighton, JP (ed.); Gierl, MJ (ed.), Cognitive modeling of performance on diagnostic achievement tests, 61-84, (2007), Cambridge
[27] Rupp, A. A. (2009). Software for calibrating diagnostic classification models: An overview of the current state-of-the-art. In Symposium conducted at the meeting of the American Educational Research Association, San Diego, CA.
[28] Rupp, AA; Templin, JL, The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model, Educational and Psychological Measurement, 68, 78-96, (2008)
[29] Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: Guilford Press.
[30] Tatsuoka, C, Data analytic methods for latent partially ordered classification models, Journal of the Royal Statistical Society: Series C (Applied Statistics), 51, 337-350, (2002) · Zbl 1111.62381
[31] Tatsuoka, K. K. (1984). Analysis of errors in fraction addition and subtraction problems. Computer-Based Education Research Laboratory, University of Illinois at Urbana-Champaign.
[32] Templin, J. L., & Henson, R. A. (2006). A Bayesian method for incorporating uncertainty into Q-matrix estimation in skills assessment. In Symposium conducted at the meeting of the American Educational Research Association, San Diego, CA.
[33] Templin, JL; Hoffman, L, Obtaining diagnostic classification model estimates using mplus, Educational Measurement: Issues and Practice, 32, 37-50, (2013)
[34] von Davier, M. (2014). The DINA model as a constrained general diagnostic model: Two variants of a model equivalency. British Journal of Mathematical and Statistical Psychology, 67(1), 49-71. · Zbl 1406.91392
[35] Xiang, R. (2013). Nonlinear Penalized Estimation of True Q-matrix in Cognitive Diagnostic Models (Unpublished doctoral dissertation). Columbia University.
[36] Xu, G; Zhang, S, Identifiability of diagnostic classification models, Psychometrika, 81, 625-649, (2016) · Zbl 1345.62166
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.