×

Fundamental tools for developing likelihood functions within ACT-R. (English) Zbl 1484.91351

Summary: Likelihood functions are an integral component of statistical approaches to parameter estimation and model evaluation. However, likelihood functions are rarely used in cognitive architectures due, in part, to challenges in their derivation, and the lack of accessible tutorials. In this tutorial, we present fundamental concepts and tools for developing analytic likelihood functions for the ACT-R cognitive architecture. These tools are based on statistical concepts such as serial vs. parallel process, convolution, minimum/maximum processing time, and mixtures. Importantly, these statistical concepts are highly composable, allowing them to be combined to form likelihood functions for many models. We demonstrate how to apply these tools within the context of Bayesian parameter estimation using five models taken from the standard ACT-R tutorial. Although the tutorial focuses on ACT-R due to its prevalence, the concepts covered within the tutorial are applicable to other cognitive architectures.

MSC:

91E10 Cognitive psychology
91E40 Memory and learning in psychology
62P15 Applications of statistics to psychology

Software:

JAGS; BayesDA; PyMC; NUTS; Stan
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Akaike, H., A new look at the statistical model identification, IEEE Transactions on Automatic Control, 19, 6, 716-723 (1974) · Zbl 0314.62039
[2] Anderson, J. R., Retrieval of propositional information from long-term memory, Cognitive Psychology, 6, 4, 451-474 (1974)
[3] Anderson, J. R., How can the human mind occur in the physical universe? (2007), Oxford University Press
[4] Anderson, J. R.; Bothell, D.; Byrne, M. D.; Douglass, S.; Lebiere, C.; Qin, Y., An integrated theory of the mind, Psychological Review, 111, 4, 1036-1060 (2004)
[5] Anderson, J. R.; Bothell, D.; Lebiere, C.; Matessa, M., An integrated theory of list memory, Journal of Memory and Language, 38, 4, 341-380 (1998)
[6] Anderson, J. R.; Douglass, S., Tower of hanoi: Evidence for the cost of goal retrieval, Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 6, 1331 (2001)
[7] Berger, J. O.; Wolpert, R. L., The likelihood principle, ((1988), IMS)
[8] Billingsley, P., Probability and Measure (1995), John Wiley & Sons: John Wiley & Sons New York · Zbl 0822.60002
[9] Birnbaum, A., On the foundations of statistical inference, Journal of the American Statistical Association, 57, 298, 269-306 (1962) · Zbl 0107.36505
[10] Brown, S.; Heathcote, A., Averaging learning curves across and within participants, Behavior Research Methods, Instruments, & Computers, 35, 1, 11-21 (2003)
[11] Busemeyer, J. R.; Diederich, A., Cognitive modeling (2010), Sage
[12] Carpenter, B.; Gelman, A.; Hoffman, M. D.; Lee, D.; Goodrich, B.; Betancourt, M., Stan: A probabilistic programming language, Journal of Statistical Software, 76, 1 (2017)
[13] Estes, W. K., The problem of inference from curves based on group data, Psychological Bulletin, 53, 2, 134-140 (1956)
[14] Fisher, C. R.; Houpt, J.; Gunzelmann, G., A comparison of approximations for base-level activation in ACT-R, Computational Brain & Behavior, 1, 3, 228-236 (2018)
[15] Fisher, C. R.; Houpt, J. W.; Gunzelmann, G., Developing memory-based models of ACT-R within a statistical framework, Journal of Mathematical Psychology, 98, Article 102416 pp. (2020) · Zbl 1455.91186
[16] Ge, H.; Xu, K.; Ghahramani, Z., Turing: Composable inference for probabilistic programming, (Storkey, A. J.; Pérez-Cruz, F., International conference on artificial intelligence and statistics. International conference on artificial intelligence and statistics, Proceedings of Machine Learning Research, vol. 84 (2018), PMLR), 1682-1690, URL: http://proceedings.mlr.press/v84/ge18b.html
[17] Gelman, A.; Carlin, J. B.; Stern, H. S.; Dunson, D. B.; Vehtari, A.; Rubin, D. B., Bayesian data analysis (2013), Chapman and Hall/CRC
[18] Grünwald, P. D.; Myung, I. J.; Pitt, M. A., Advances in minimum description length: Theory and applications (2005), MIT Press
[19] Hintzman, D. L., MINERVA 2: A simulation model of human memory, Behavior Research Methods, Instruments, & Computers, 16, 2, 96-101 (1984)
[20] Hoffman, M. D.; Gelman, A., The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo., Journal of Machine Learning Research, 15, 1, 1593-1623 (2014) · Zbl 1319.60150
[21] JASP Team, M. D., JASP (Version 0.11.1)[computer software] (2019), URL: https://jasp-stats.org/
[22] Kangasrääsiö, A.; Jokinen, J. P.; Oulasvirta, A.; Howes, A.; Kaski, S., Parameter inference for computational cognitive models with approximate Bayesian computation, Cognitive Science, 43, 6, Article e12738 pp. (2019)
[23] Kieras, D. E.; Meyer, D. E., An overview of the EPIC architecture for cognition and performance with application to human-computer interaction, Human-Computer Interaction, 12, 4, 391-438 (1997)
[24] Kruschke, J. K., BayesIan assessment of null values via parameter estimation and model comparison, Perspectives on Psychological Science, 6, 3, 299-312 (2011)
[25] Kruschke, J., Doing bayesian data analysis: A Tutorial with R, JAGS, and Stan (2014), Academic Press · Zbl 1300.62001
[26] Lebiere, C., The dynamics of cognition: An ACT-r model of cognitive arithmetic, Kognitionswissenschaft, 8, 1, 5-19 (1999)
[27] Lebière, C.; Anderson, J. R.; Reder, L. M., Error modeling in the ACT-r production system, (Proceedings of the sixteenth annual conference of the cognitive science society (1994), Erlbaum Hillsdale, NJ), 555-559
[28] Lin, Y.-S.; Heathcote, A.; Holmes, W. R., Parallel probability density approximation, Behavior Research Methods, 51, 6, 2777-2799 (2019)
[29] Morey, R. D.; Hoekstra, R.; Rouder, J. N.; Lee, M. D.; Wagenmakers, E.-J., The fallacy of placing confidence in confidence intervals, Psychonomic Bulletin & Review, 23, 1, 103-123 (2016)
[30] Myung, I. J., Tutorial on maximum likelihood estimation, Journal of Mathematical Psychology, 47, 1, 90-100 (2003) · Zbl 1023.62112
[31] Nelder, J. A.; Mead, R., A simplex method for function minimization, The Computer Journal, 7, 4, 308-313 (1965) · Zbl 0229.65053
[32] Newell, A., Unified theories of cognition (1990), Harvard University Press
[33] Nicenboim, B.; Vasishth, S., Models of retrieval in sentence comprehension: A computational evaluation using Bayesian hierarchical modeling, Journal of Memory and Language, 99, 1-34 (2018)
[34] Nosofsky, R. M., Attention, similarity, and the identification-categorization relationship., Journal of Experimental Psychology: General, 115, 1, 39-57 (1986)
[35] Petrov, A. A. (2006). Computationally efficient approximation of the base-level learning equation in ACT-R. In Proceedings of the seventh international conference on cognitive modeling (pp. 391-392).
[36] Plummer, M., JAGS: A program for analysis of Bayesian graphical models using gibbs sampling, (Proceedings of the 3rd international workshop on distributed statistical computing, vol. 124 (2003), Vienna, Austria), 1-10
[37] Rouder, J. N.; Province, J. M.; Morey, R. D.; Gomez, P.; Heathcote, A., The lognormal race: A cognitive-process model of choice and latency with desirable psychometric properties, Psychometrika, 80, 2, 491-513 (2015) · Zbl 1322.62339
[38] Salvatier, J.; Wiecki, T. V.; Fonnesbeck, C., Probabilistic programming in python using PyMC3, PeerJ Computer Science, 2, Article e55 pp. (2016)
[39] Salvucci, D. D.; Taatgen, N. A., Threaded cognition: An integrated theory of concurrent multitasking., Psychological Review, 115, 1, 101 (2008)
[40] Schwarz, G., Estimating the dimension of a model, The Annals of Statistics, 6, 2, 461-464 (1978) · Zbl 0379.62005
[41] Siegler, R. S., The perils of averaging data over strategies: An example from children’s addition., Journal of Experimental Psychology: General, 116, 3, 250-264 (1987)
[42] Siegler, R.; Shrager, J., Strategy choices in addition and subtraction: How do children know what to do?, (Catherine, S., The Origins of Cognitive Skills (1984), Hillsdale: Hillsdale NJ, Erlbaum), 229-293
[43] Smith, A. F.; Roberts, G. O., BayesIan computation via the gibbs sampler and related Markov chain Monte Carlo methods, Journal of the Royal Statistical Society. Series B. Statistical Methodology, 55, 1, 3-23 (1993) · Zbl 0779.62030
[44] Software, A.-R., Simulation-based ACT-R tutorial (2020), URL: http://act-r.psy.cmu.edu/software/
[45] Ter Braak, C. J., A Markov chain Monte Carlo version of the genetic algorithm differential evolution: easy Bayesian computing for real parameter spaces, Statistics and Computing, 16, 3, 239-249 (2006)
[46] Turner, B. M.; Sederberg, P. B., Approximate Bayesian computation with differential evolution, Journal of Mathematical Psychology, 56, 5, 375-385 (2012) · Zbl 1282.62063
[47] Turner, B. M.; Sederberg, P. B., A generalized, likelihood-free method for posterior estimation, Psychonomic Bulletin & Review, 21, 2, 227-250 (2014)
[48] Turner, B. M.; Sederberg, P. B.; Brown, S. D.; Steyvers, M., A method for efficiently sampling from distributions with correlated dimensions, Psychological Methods, 18, 3, 368 (2013)
[49] Van Ravenzwaaij, D.; Cassey, P.; Brown, S. D., A simple introduction to Markov chain Monte-Carlo sampling, Psychonomic Bulletin & Review, 25, 1, 143-154 (2018)
[50] Vehtari, A.; Gelman, A.; Gabry, J., Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC, Statistics and Computing, 27, 5, 1413-1432 (2017) · Zbl 1505.62408
[51] Wagenmakers, E.-J., A practical solution to the pervasive problems of p values, Psychonomic Bulletin & Review, 14, 5, 779-804 (2007)
[52] Weaver, R., Parameters, predictions, and evidence in computational modeling: A statistical view informed by ACT-r, Cognitive Science, 32, 8, 1349-1375 (2008)
[53] Winter, S.; Ryan, O.; Zondervan-Zwijnenburg, M.; Depaoli, S., A systematic review of Bayesian articles in psychology: The last 25 years, Psychological Methods, 22, 2, 217-239 (2017)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.