Regression methods for metacognitive sensitivity. (English) Zbl 1437.91361

Summary: Metacognition is an important component in basic science and clinical psychology, often studied through complex, cognitive experiments. While signal detection theory (SDT) provides a popular and pervasive framework for modelling responses from such experiments, a shortfall remains that it cannot in a straightforward manner account for the often complex designs. Additionally, SDT does not provide direct estimates of metacognitive ability. This latter shortcoming has recently been sought remedied by introduction of a measure for metacognitive sensitivity dubbed meta-\(d^\prime\). The new sensitivity measure, however, further accentuates the need for a flexible modelling framework. In the present paper, we argue that a straightforward extension of SDT is obtained by identifying the model with the proportional odds model, a widely implemented, ordinal regression technique. We go on to develop a formal statistical framework for metacognitive sensitivity by defining a model that combines standard SDT with meta- \(d^\prime\) in a latent variable model. We show how this agrees with the literature on meta-\(d^\prime\) and constitutes a practical framework for extending the model. We supply several theoretical considerations on the model, including closed-form approximate estimates of meta- \(d^\prime\) and optimal weighing of response-specific meta-sensitivities. We discuss regression analysis as an application of the obtained model and illustrate our points through simulations. Lastly, we discuss a software implementation of the model in R. Our methods and their implementation extend the computational possibilities of SDT and meta- \(d^\prime\) and are useful for theoretical and practical researchers of metacognition.


91E10 Cognitive psychology
62P15 Applications of statistics to psychology


R; ordinal; rms; brms
Full Text: DOI


[1] Agresti, A., Foundations of linear and generalized linear models (2015), John Wiley & Sons · Zbl 1309.62001
[2] Ananth, C. V.; Kleinbaum, D. G., Regression models for ordinal responses: a review of methods and applications, International Journal of Epidemiology, 26, 6, 1323-1333 (1997)
[3] Barrett, A. B.; Dienes, Z.; Seth, A. K., Measures of metacognition on signal-detection theoretic models., Psychological Methods, 18, 4, 535-552 (2013)
[4] Breinegaard, N.; Rabe-Hesketh, S.; Skrondal, A., The transition model test for serial dependence in mixed-effects models for binary data, Statistical Methods in Medical Research (2015)
[5] Brockhoff, P. B.; Christensen, R. H.B., Thurstonian models for sensory discrimination tests as generalized linear models, Food Quality and Preference, 21, 3, 330-338 (2010)
[6] Bürkner, P.-C., Brms: An R package for Bayesian multilevel models using Stan, Journal of Statistical Software, 80, 1, 1-28 (2017)
[7] Christensen, R. H.B., Ordinal—regression models for ordinal data (2019), R package version 2019.4-25. http://www.cran.r-project.org/package=ordinal/
[8] Clarke, F. R.; Birdsall, T. G.; Tanner, W. P., Two types of ROC curves and definitions of parameters, The Journal of the Acoustical Society of America, 31, 5, 629-630 (1959)
[9] David, A. S.; Bedford, N.; Wiffen, B.; Gilleen, J., Failures of metacognition and lack of insight in neuropsychiatric disorders, Philosophical Transactions of the Royal Society, Series B (Biological Sciences), 367, 1594, 1379-1390 (2012)
[10] DeCarlo, L. T., Signal detection theory and generalized linear models., Psychological Methods, 3, 2, 186 (1998)
[11] DeCarlo, L. T., Source monitoring and multivariate signal detection theory, with a model for selection, Journal of Mathematical Psychology, 47, 3, 292-303 (2003) · Zbl 1062.91061
[12] DeCarlo, L. T., On the statistical and theoretical basis of signal detection theory and extensions: Unequal variance, random coefficient, and mixture models, Journal of Mathematical Psychology, 54, 3, 304-313 (2010) · Zbl 1190.62005
[13] Demidenko, E., Mixed models: Theory and applications with R (2013), John Wiley & Sons · Zbl 1276.62049
[14] Dienes, Z., Subjective measures of unconscious knowledge, (Progress in brain research, Vol. 168 (2007), Elsevier)
[15] Dienes, Z.; Altmann, G. T.M.; Kwan, L.; Goode, A., Unconscious knowledge of artificial grammars is applied strategically, Journal of Experimental Psychology. Learning, Memory, and Cognition, 21, 5, 1322-1338 (1995)
[16] Fleming, S. M., HMeta-d: hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings, Neuroscience of Consciousness, 3, 1 (2017)
[17] Fleming, S. M.; Daw, N. D., Self-evaluation of decision-making: A general Bayesian framework for metacognitive computation, Psychological Review, 124, 1, 91-114 (2017)
[18] Fleming, S. M.; Lau, H. C., How to measure metacognition, Frontiers in Human Neuroscience, 8 (2014)
[19] Galvin, S. J.; Podd, J. V.; Drga, V.; Whitmore, J., Type 2 tasks in the theory of signal detectability: Discrimination between correct and incorrect decisions, Psychonomic Bulletin & Review, 10, 4, 843-876 (2003)
[20] Green, D. M.; Swets, J. A., (Signal detection theory and psychophysics (1966), John Wiley: John Wiley Oxford, England)
[21] Harrell, F., Regression modeling strategies: With applications to linear models, logistic and ordinal regression, and survival analysis (2015), Springer · Zbl 1330.62001
[22] Hautus, M. J., Corrections for extreme proportions and their biasing effects on estimated values of d’, Behavior Research Methods, Instruments, & Computers, 27, 1, 46-51 (1995)
[23] Jang, Y.; Wallsten, T. S.; Huber, D. E., A stochastic detection and retrieval model for the study of metacognition., Psychological Review, 119, 1, 186-200 (2012)
[24] Johnson, N. L.; Kotz, S.; Balakrishnan, N., Continuous univariate distributions, Vol. 1 (1994), John Wiley & Sons · Zbl 0811.62001
[25] Kiani, R.; Shadlen, M. N., Representation of confidence associated with a decision by neurons in the parietal cortex, Science, 324, 5928, 759-764 (2009)
[26] Maniscalco, B.; Lau, H., A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings, Consciousness and Cognition, 21, 1, 422-430 (2012)
[27] Maniscalco, B.; Lau, H., Signal detection theory analysis of type 1 and type 2 data: Meta-d’, response-specific meta-d’, and the unequal variance SDT model, (Fleming, S. M.; Frith, C. D., The cognitive neuroscience of metacognition (2014), Springer Berlin Heidelberg), 25-66
[28] Maniscalco, B.; Lau, H., The signal processing architecture underlying subjective reports of sensory awareness, Neuroscience of Consciousness, 2016, 1 (2016)
[29] McCullagh, P.; Nelder, J., (Generalized linear models. Generalized linear models, Chapman & Hall/CRC monographs on statistics & applied probability (1989), Taylor & Francis) · Zbl 0744.62098
[30] (Metcalfe, J.; Shimamura, A. P., Metacognition: Knowing about knowing (1996), A Bradford Book: A Bradford Book Cambridge, Mass)
[31] Morey, R. D.; Pratte, M. S.; Rouder, J. N., Problematic effects of aggregation in z ROC analysis and a hierarchical modeling solution, Journal of Mathematical Psychology, 52, 6, 376-388 (2008) · Zbl 1152.91769
[32] Persaud, N.; McLeod, P.; Cowey, A., Post-decision wagering objectively measures awareness, Nature Neuroscience, 10, 2, 257-261 (2007)
[33] R: A language and environment for statistical computing (2016), R Foundation for Statistical Computing: R Foundation for Statistical Computing Vienna, Austria
[34] Ramsøy, T. Z.; Overgaard, M., Introspection and subliminal perception, Phenomenology and the Cognitive Sciences, 3, 1, 1-23 (2004)
[35] Rausch, M.; Zehetleitner, M., Should metacognition be measured by logistic regression?, Consciousness and Cognition, 49, 291-312 (2017)
[36] Rounis, E.; Maniscalco, B.; Rothwell, J. C.; Passingham, R. E.; Lau, H., Theta-burst transcranial magnetic stimulation to the prefrontal cortex impairs metacognitive visual awareness, Cognitive Neuroscience, 1, 3, 165-175 (2010)
[37] Sandberg, K.; Timmermans, B.; Overgaard, M.; Cleeremans, A., Measuring consciousness: Is one measure better than the other?, Consciousness and Cognition, 19, 4, 1069-1078 (2010)
[38] Tutz, G.; Hennevogl, W., Random effects in ordinal regression models, Computational Statistics & Data Analysis, 22, 5, 537-557 (1996) · Zbl 0900.62379
[39] Wright, D. B.; Horry, R.; Skagerberg, E. M., Functions for traditional and multilevel approaches to signal detection theory, Behavior Research Methods, 41, 2, 257-267 (2009)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.