×

Monotonicity of rank order probabilities in signal detection models of simultaneous detection and identification. (English) Zbl 1479.91290

Summary: We examine different models of recognition memory in a simultaneous detection and identification task, which features multiple simultaneously presented test stimuli. A common finding from eyewitness identification research investigating such tasks is that the more confident decision makers are about detecting the presence of a target, the higher the probability that they also correctly identify it. We demonstrate that for members of the signal detection theory (SDT) model framework, predicting such a relationship is – contrary to previous assertions – not entailed by a monotonic diagnosticity ratio. Instead, it can be shown that this prediction follows if latent memory signals’ rank order probabilities exhibit monotonicity under changes in the response criterion. For a selection of common SDT models, we prove that this monotonicity property holds in situations in which two test stimuli are presented simultaneously. Threshold models such as the two-high-threshold model (2HTM), however, do not necessarily possess this feature. Leveraging this fact, we show that in the presence of lures which resemble a target, the 2HTM is unable to make the same predictions as many reasonable SDT models with monotonic rank order probabilities. This enables us to construct a critical, distribution-free test between these models. An empirical investigation implementing this test reveals a clear failure of the 2HTM to account for the qualitative response patterns, which are consistent with the predictions of SDT models with monotonic rank order probabilities.

MSC:

91E10 Cognitive psychology
94A13 Detection theory in information and communication theory

Software:

sdtlu; Stan
PDF BibTeX XML Cite
Full Text: DOI

References:

[1] Akan, M.; Robinson, M. M.; Mickes, L.; Wixted, J. T.; Benjamin, A. S., The effect of lineup size on eyewitness identification, Journal of Experimental Psychology: Applied, 72, 2, 369-392 (2021)
[2] Bapat, R.; Kochar, S. C., On likelihood-ratio ordering of order statistics, Linear Algebra and its Applications, 199, 281-291 (1994) · Zbl 0790.62047
[3] Baricz, A., Mills’ ratio: Monotonicity patterns and functional inequalities, Journal of Mathematical Analysis and Applications, 340, 2, 1362-1370 (2008) · Zbl 1138.60022
[4] Barr, D. J.; Levy, R.; Scheepers, C.; Tily, H. J., Random effects structure for confirmatory hypothesis testing: Keep it maximal, Journal of Memory and Language, 68, 3, 255-278 (2013)
[5] Batchelder, W. H.; Alexander, G. E., Discrete-state models: Comment on Pazzaglia, Dubé, and Rotello (2013), Psychological Bulletin, 139, 6, 1204-1212 (2013)
[6] Benzschawel, T.; Cohn, T. E., Detection and recognition of visual targets, Journal of the Optical Society of America A, 2, 9, 1543-1550 (1985)
[7] Bernbach, H. A., Decision processes in memory, Psychological Review, 74, 6, 462-480 (1967)
[8] Bröder, A.; Kellen, D.; Schütz, J.; Rohrmeier, C., Validating a two-high-threshold measurement model for confidence rating data in recognition, Memory, 21, 8, 916-944 (2013)
[9] Bröder, A.; Schütz, J., Recognition ROCs are curvilinear — or are they? On premature arguments against the two-high-threshold model of recognition, Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 3, 587-606 (2009)
[10] Browne, M. W., Cross-validation methods, Journal of Mathematical Psychology, 44, 1, 108-132 (2000) · Zbl 0946.62045
[11] Carpenter, B.; Gelman, A.; Hoffman, M. D.; Lee, D.; Goodrich, B.; Betancourt, M.; Brubaker, M.; Guo, J.; Li, P.; Riddell, A., Stan: A probabilistic programming language, Journal of Statistical Software, 76, 1 (2017)
[12] Chechile, R. A., Properties of reverse hazard functions, Journal of Mathematical Psychology, 55, 3, 203-222 (2011) · Zbl 1219.62160
[13] Chechile, R. A., A novel method for assessing rival models of recognition memory, Journal of Mathematical Psychology, 57, 5, 196-214 (2013) · Zbl 1292.91153
[14] Cohen, A. L.; Starns, J. J.; Rotello, C. M., sdtlu: An R package for the signal detection analysis of eyewitness lineup data, Behavior Research Methods, 53, 1, 278-300 (2021)
[15] Colloff, M. F.; Wade, K. A.; Wixted, J. T.; Maylor, E. A., A signal-detection analysis of eyewitness identification across the adult lifespan, Psychology and Aging, 32, 3, 243-258 (2017)
[16] Coombs, C. H.; Dawes, R. M.; Tversky, A., Mathematical psychology: An elementary introduction (1970), Prentice Hall · Zbl 0205.23701
[17] Davis, P. J., Gamma function and related functions, (Abramowitz, M.; Stegun, I. A., Handbook of mathematical functions with formulas, graphs, and mathematical tables (1972), U.S. Government Printing Office)
[18] DeCarlo, L. T., Signal detection theory and generalized linear models, Psychological Methods, 3, 2, 186-205 (1998)
[19] DeCarlo, L. T., Signal detection theory with finite mixture distributions: Theoretical developments with applications to recognition memory, Psychological Review, 109, 4, 710-721 (2002)
[20] DeCarlo, L. T., The mirror effect and mixture signal detection theory, Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 1, 18-33 (2007)
[21] Delay, C. G.; Wixted, J. T., Discrete-state versus continuous models of the confidence-accuracy relationship in recognition memory, Psychonomic Bulletin & Review, 28, 2, 556-564 (2021)
[22] Dubé, C.; Rotello, C. M., Binary ROCs in perception and recognition memory are curved, Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 1, 130-151 (2012)
[23] Dubé, C.; Rotello, C. M.; Pazzaglia, A., The statistical accuracy and theoretical status of discrete-state MPT models: Reply to Batchelder and Alexander (2013), Psychological Bulletin, 139, 6, 1213-1220 (2013)
[24] Dubé, C.; Starns, J. J.; Rotello, C. M.; Ratcliff, R., Beyond ROC curvature: Strength effects and response time data support continuous-evidence models of recognition memory, Journal of Memory and Language, 67, 3, 389-406 (2012)
[25] Egan, J. P., Signal detection theory and ROC-analysis (1975), Academic Press
[26] Feller, W., An introduction to probability theory and its applications (vol. 1) (1968), John Wiley & Sons · Zbl 0155.23101
[27] Fisher, R. A.; Tippett, L. H.C., Limiting forms of the frequency distribution of the largest or smallest member of a sample, Proceedings of the Cambridge Philosophical Society, 24, 180-190 (1928) · JFM 54.0560.05
[28] Gelman, A.; Carlin, J. B.; Stern, H. S.; Dunson, D. B.; Vehtari, A.; Rubin, D. B., Bayesian data analysis (2013), Chapman & Hall/CRC Press
[29] Gelman, A.; Hwang, J.; Vehtari, A., Understanding predictive information criteria for Bayesian models, Statistics and Computing, 24, 6, 997-1016 (2014) · Zbl 1332.62090
[30] Glanzer, M.; Hilford, A.; Maloney, L. T., Likelihood ratio decisions in memory: Three implied regularities, Psychonomic Bulletin & Review, 16, 3, 431-455 (2009)
[31] Glanzer, M.; Kim, K.; Hilford, A.; Adams, J. K., Slope of the receiver-operating characteristic in recognition memory, Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 2, 500-513 (1999)
[32] Green, D. M.; Birdsall, T. G., Detection and recognition, Psychological Review, 85, 3, 192-206 (1978)
[33] Green, D. M.; Swets, J. A., Signal detection theory and psychophysics (1966), John Wiley & Sons
[34] Green, D. M.; Weber, D. L.; Duncan, J. E., Detection and recognition of pure tones in noise, The Journal of the Acoustical Society of America, 62, 4, 948-954 (1977)
[35] Gronau, Q. F.; Wagenmakers, E.-J., Limitations of Bayesian leave-one-out cross-validation for model selection, Computational Brain & Behavior, 2, 1, 1-11 (2019)
[36] Gronau, Q. F.; Wagenmakers, E.-J., Rejoinder: More limitations of Bayesian leave-one-out cross-validation, Computational Brain & Behavior, 2, 1, 35-47 (2019)
[37] Gronlund, S. D.; Benjamin, A. S., The new science of eyewitness memory, (Federmeier, K. D., Psychology of learning and motivation (vol. 69) (2018), Academic Press), 241-284
[38] Gupta, B. N., On Mill’s ratio, (Mathematical proceedings of the cambridge philosophical society (vol. 67, no. 2) (1970), Cambridge University Press), 363-364 · Zbl 0206.19602
[39] Haase, S. J.; Theios, J.; Jenison, R., A signal detection theory analysis of an unconscious perception effect, Perception & Psychophysics, 61, 5, 986-992 (1999)
[40] Hintzman, D. L., MINERVA 2: A simulation model of human memory, Behavior Research Methods, Instruments, & Computers, 16, 2, 96-101 (1984)
[41] Iverson, G.; Bamber, D., The generalized area theorem in signal detection theory, (Marley, A. A.J., Choice, decision, and measurement: Essays in honor of R. Duncan Luce (1997), Lawrence Erlbaum Associates), 301-318
[42] Jang, Y.; Mickes, L.; Wixted, J. T., Three tests and three corrections: Comment on Koen and Yonelinas (2010), Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 2, 513-523 (2012)
[43] Jang, Y.; Wixted, J. T.; Huber, D. E., Testing signal-detection models of yes/no and two-alternative forced-choice recognition memory, Journal of Experimental Psychology: General, 138, 2, 291-306 (2009)
[44] Jeffreys, H., The theory of probability (1961), Oxford University Press · Zbl 0116.34904
[45] Johnson, N.; Kotz, S.; Balakrishnan, N., Continuouns univariate distributions (vol. 2) (1995), John Wiley & Sons · Zbl 0821.62001
[46] Judd, C. M.; Westfall, J.; Kenny, D. A., Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem, Journal of Personality and Social Psychology, 103, 1, 54-69 (2012)
[47] Juola, J. F.; Caballero-Sanz, A.; Muñoz-García, A. R.; Botella, J.; Suero, M., Familiarity, recollection, and receiver-operating characteristic (ROC) curves in recognition memory, Memory & Cognition, 47, 4, 855-876 (2019)
[48] Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4401-4410).
[49] Kass, R. E.; Raftery, A. E., Bayes factors, Journal of the American Statistical Association, 90, 430, 773-795 (1995) · Zbl 0846.62028
[50] Kellen, D.; Erdfelder, E.; Malmberg, K. J.; Dubé, C.; Criss, A. H., The ignored alternative: An application of Luce’s low-threshold model to recognition memory, Journal of Mathematical Psychology, 75, 86-95 (2016) · Zbl 1396.91667
[51] Kellen, D.; Klauer, K. C., Discrete-state and continuous models of recognition memory: Testing core properties under minimal assumptions, Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 6, 1795-1804 (2014)
[52] Kellen, D.; Klauer, K. C., Signal detection and threshold modeling of confidence-rating ROCs: A critical test with minimal assumptions, Psychological Review, 122, 3, 542-557 (2015)
[53] Kellen, D.; Klauer, K. C., Elementary signal detection and threshold theory, (Wagenmakers, E. J.; Wixted, J. T., The Stevens’ handbook of experimental psychology and cognitive neuroscience (vol. 5) (2018), John Wiley & Sons), 161-200
[54] Kellen, D.; Klauer, K. C., Selecting amongst multinomial models: An apologia for normalized maximum likelihood, Journal of Mathematical Psychology, 97, Article 102367 pp. (2020) · Zbl 1448.91246
[55] Kellen, D.; Klauer, K. C.; Bröder, A., Recognition memory models and binary-response ROCs: A comparison by minimum description length, Psychonomic Bulletin & Review, 20, 4, 693-719 (2013)
[56] Kellen, D.; Winiger, S.; Dunn, J.; Singmann, H., Testing the foundations of signal detection theory in recognition memory, Psychological Review, 128, 6, 1022-1050 (2021)
[57] Klauer, K. C., Hierarchical multinomial processing tree models: A latent-trait approach, Psychometrika, 75, 1, 70-98 (2010) · Zbl 1272.62126
[58] Klauer, K. C.; Kellen, D., Toward a complete decision model of item and source recognition: A discrete-state approach, Psychonomic Bulletin & Review, 17, 4, 465-478 (2010)
[59] Koen, J. D.; Yonelinas, A. P., Memory variability is due to the contribution of recollection and familiarity, not to encoding variability, Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 6, 1536-1542 (2010)
[60] Koen, J. D.; Yonelinas, A. P., Still no evidence for the encoding variability hypothesis: A reply to Jang, Mickes, and Wixted (2012) and Starns, Rotello, and Ratcliff (2012), Journal of Experimental Psychology: Learning, Memory, and Cognition, 39, 1, 304-312 (2013)
[61] Lee, J.; Penrod, S. D., New signal detection theory-based framework for eyewitness performance in lineups, Law and Human Behavior, 43, 5, 436 (2019)
[62] Luce, R. D., A threshold theory for simple detection experiments, Psychological Review, 70, 1, 61-79 (1963)
[63] Macmillan, N. A.; Creelman, C. D., Detection theory: A user’s guide (2005), Lawrence Erlbaum Associates
[64] Malejka, S.; Bröder, A., Exploring the shape of signal-detection distributions in individual recognition ROC data, Journal of Memory and Language, 104, 83-107 (2019)
[65] Malmberg, K. J., Recognition memory: A review of the critical findings and an integrated theory for relating them, Cognitive Psychology, 57, 4, 335-384 (2008)
[66] Matuschek, H.; Kliegl, R.; Vasishth, S.; Baayen, H.; Bates, D., Balancing Type I error and power in linear mixed models, Journal of Memory and Language, 94, 305-315 (2017)
[67] McAdoo, R. M.; Key, K. N.; Gronlund, S. D., Stimulus effects and the mediation of recognition memory, Journal of Experimental Psychology: Learning, Memory, and Cognition, 44, 11, 1814-1823 (2018)
[68] McAdoo, R. M.; Key, K. N.; Gronlund, S. D., Task effects determine whether recognition memory is mediated discretely or continuously, Memory & Cognition, 47, 4, 683-695 (2019)
[69] Mickes, L.; Gronlund, S. D., Eyewitness identification, (Byrne, J. H., Learning and memory: A comprehensive reference (vol. 2) (2017), Academic Press), 529-552
[70] Mickes, L.; Wixted, J. T.; Wais, P. E., A direct test of the unequal-variance signal detection model of recognition memory, Psychonomic Bulletin & Review, 14, 5, 858-865 (2007)
[71] Mills, J. P., Table of the ratio: Area to bounding ordinate, for any portion of normal curve, Biometrika, 395-400 (1926) · JFM 52.0548.02
[72] Mood, A.; Graybill, F.; Boes, D., Introduction to the theory of statistics (1974), McGraw-Hill · Zbl 0277.62002
[73] Morey, R. D.; Pratte, M. S.; Rouder, J. N., Problematic effects of aggregation in zROC analysis and a hierarchical modeling solution, Journal of Mathematical Psychology, 52, 6, 376-388 (2008) · Zbl 1152.91769
[74] Morrell, H. E.; Gaitan, S.; Wixted, J. T., On the nature of the decision axis in signal-detection-based models of recognition memory, Journal of Experimental Psychology: Learning, Memory, and Cognition, 28, 6, 1095-1110 (2002)
[75] Norman, D. A.; Wickelgren, W. A., Strength theory of decision rules and latency in retrieval from short-term memory, Journal of Mathematical Psychology, 6, 2, 192-208 (1969)
[76] Nyman, T. J.; Lampinen, J. M.; Antfolk, J.; Korkman, J.; Santtila, P., The distance threshold of reliable eyewitness identification, Law and Human Behavior, 43, 6, 527-541 (2019)
[77] Owen, D. B., A table of normal integrals, Communications in Statistics. Simulation and Computation, 9, 4, 389-419 (1980) · Zbl 0462.62089
[78] Parks, C. M.; Yonelinas, A. P., Theories of recognition memory, (Byrne, J. H.; Roediger, H. L., Learning and memory: A comprehensive reference (vol. 2) (2008), Academic Press), 389-416
[79] Pazzaglia, A. M.; Dubé, C.; Rotello, C. M., A critical comparison of discrete-state and continuous models of recognition memory: Implications for recognition and beyond, Psychological Bulletin, 139, 6, 1173-1203 (2013)
[80] Pinelis, I., Exact bounds on the inverse Mills ratio and its derivatives, Complex Analysis and Operator Theory, 13, 4, 1643-1651 (2019) · Zbl 1421.30041
[81] Province, J. M.; Rouder, J. N., Evidence for discrete-state processing in recognition memory, Proceedings of the National Academy of Sciences, 109, 36, 14357-14362 (2012)
[82] Rotello, C. M., Signal detection theories of recognition memory, (Byrne, J. H., Learning and memory: A comprehensive reference (vol. 2) (2017), Academic Press), 529-552
[83] Rouder, J. N.; Lu, J., An introduction to Bayesian hierarchical models with an application in the theory of signal detection, Psychonomic Bulletin & Review, 12, 4, 573-604 (2005)
[84] Rouder, J. N.; Morey, R. D.; Pratte, M. S., Bayesian hierarchical models of cognition, (Batchelder, W. H.; Colonius, H.; Dzhafarov, E. N.; Myung, J., New handbook of mathematical psychology: Foundations and methodology (2017), Cambridge University Press), 504-551
[85] Rouder, J. N.; Pratte, M. S.; Morey, R. D., Latent mnemonic strengths are latent: A comment on Mickes, Wixted, and Wais (2007), Psychonomic Bulletin & Review, 17, 3, 427-435 (2010)
[86] Rouder, J. N.; Province, J. M.; Swagman, A. R.; Thiele, J. E., From ROC curves to psychological theory (2014), Department of Psychological Sciences, University of Missouri
[87] Sampford, M. R., Some inequalities on Mill’s ratio and related functions, The Annals of Mathematical Statistics, 24, 1, 130-132 (1953) · Zbl 0050.13503
[88] Schwarz, W.; Miller, J., GSDT: An integrative model of visual search, Journal of Experimental Psychology: Human Perception and Performance, 42, 10, 1654-1675 (2016)
[89] Singmann, H.; Kellen, D., An introduction to mixed models for experimental psychology, (Spieler, D. H.; Schumacher, E., New methods in cognitive psychology (2019), Routledge), 4-31
[90] Smith, A. M.; Wells, G. L.; Lindsay, R.; Penrod, S. D., Fair lineups are better than biased lineups and showups, but not because they increase underlying discriminability, Law and Human Behavior, 41, 2, 127-145 (2017)
[91] Smith, A. M.; Wilford, M. M.; Quigley-McBride, A.; Wells, G. L., Mistaken eyewitness identification rates increase when either witnessing or testing conditions get worse, Law and Human Behavior, 43, 4, 358-368 (2019)
[92] Snodgrass, J. G.; Corwin, J., Pragmatics of measuring recognition memory: Applications to dementia and amnesia, Journal of Experimental Psychology: General, 117, 1, 34-50 (1988)
[93] Spanton, R. W.; Berry, C. J., The unequal variance signal-detection model of recognition memory: Investigating the encoding variability hypothesis, Quarterly Journal of Experimental Psychology, 73, 8, 1242-1260 (2020)
[94] Spanton, R. W.; Berry, C. J., Variability in recognition memory scales with mean memory strength: Implications for the encoding variability hypothesis (2021), School of Psychology, University of Plymouth, PsyArXiv Preprint. URL: https://psyarxiv.com/3sbnh
[95] Starns, J. J., High-and low-threshold models of the relationship between response time and confidence, Journal of Experimental Psychology: Learning, Memory, and Cognition, 47, 4, 671-684 (2020)
[96] Starns, J. J.; Ma, Q., Guessing versus misremembering in recognition: A comparison of continuous, two-high-threshold, and low-threshold models, Journal of Experimental Psychology: Learning, Memory, and Cognition, 44, 4, 527-539 (2018)
[97] Starns, J. J.; Rotello, C. M.; Ratcliff, R., Mixing strong and weak targets provides no evidence against the unequal-variance explanation of zROC slope: A comment on Koen and Yonelinas (2010), Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 3, 793-801 (2012)
[98] Starr, S. J.; Metz, C. E.; Lusted, L. B.; Goodenough, D. J., Visual detection and localization of radiographic images, Radiology, 116, 3, 533-538 (1975)
[99] Swets, J.; Tanner, W.; Birdsall, T., Decision processes in perception, Psychological Review, 68, 5, 301-340 (1961)
[100] Tekin, E.; Roediger, H. L., The range of confidence scales does not affect the relationship between confidence and accuracy in recognition memory, Cognitive Research: Principles and Implications, 2, 1, 1-13 (2017)
[101] Vehtari, A.; Gelman, A.; Gabry, J., Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC, Statistics and Computing, 27, 5, 1413-1432 (2017) · Zbl 06737719
[102] Voormann, A.; Rothe-Wulf, A.; Starns, J. J.; Klauer, K. C., Does speed of recognition predict two-alternative forced-choice performance? Replicating and extending Starns, Dubé, and Frelinger (2018), Quarterly Journal of Experimental Psychology, 74, 1, 122-134 (2021)
[103] Voormann, A.; Spektor, M. S.; Klauer, K. C., The simultaneous recognition of multiple words: A process analysis, Memory & Cognition, 49, 4, 787-802 (2021)
[104] Wickens, T. D., Elementary signal detection theory (2002), Oxford University Press
[105] Windschitl, P. D.; Chambers, J. R., The dud-alternative effect in likelihood judgment, Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 1, 198-215 (2004)
[106] Wixted, J. T., Dual-process theory and signal-detection theory of recognition memory, Psychological Review, 114, 1, 152-176 (2007)
[107] Wixted, J. T.; Mickes, L., Useful scientific theories are useful: A reply to Rouder, Pratte, and Morey (2010), Psychonomic Bulletin & Review, 17, 3, 436-442 (2010)
[108] Wixted, J. T.; Mickes, L., A signal-detection-based diagnostic-feature-detection model of eyewitness identification, Psychological Review, 121, 2, 262-276 (2014)
[109] Wixted, J. T.; Mickes, L., Evaluating eyewitness identification procedures: ROC analysis and its misconceptions, Journal of Applied Research in Memory and Cognition, 4, 4, 318-323 (2015)
[110] Wixted, J. T.; Vul, E.; Mickes, L.; Wilson, B. M., Models of lineup memory, Cognitive Psychology, 105, 81-114 (2018)
[111] Wixted, J. T.; Wells, G. L., The relationship between eyewitness confidence and identification accuracy: A new synthesis, Psychological Science in the Public Interest, 18, 1, 10-65 (2017)
[112] Yao, Y.; Vehtari, A.; Simpson, D.; Gelman, A., Using stacking to average Bayesian predictive distributions (with discussion), Bayesian Analysis, 13, 3, 917-1007 (2018) · Zbl 1407.62090
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.