×

Model selection by minimum description length: lower-bound sample sizes for the Fisher information approximation. (English) Zbl 1304.62053

Summary: The Fisher information approximation (FIA) is an implementation of the minimum description length principle for model selection. Unlike information criteria such as AIC or BIC, it has the advantage of taking the functional form of a model into account. Unfortunately, FIA can be misleading in finite samples, resulting in an inversion of the correct rank order of complexity terms for competing models in the worst case. As a remedy, we propose a lower-bound \(N'\) for the sample size that suffices to preclude such errors. We illustrate the approach using three examples from the family of multinomial processing tree models.

MSC:

62F99 Parametric inference
62B10 Statistical aspects of information-theoretic topics
91E45 Measurement and performance in psychology

Software:

MPTinR; Multitree
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[2] Batchelder, W. H.; Riefer, D. M., Multinomial processing models of source monitoring, Psychological Review, 97, 548-564 (1990)
[3] Batchelder, W. H.; Riefer, D. M., Theoretical and empirical review of multinomial process tree modeling, Psychonomic Bulletin & Review, 6, 57-86 (1999)
[4] Bröder, A.; Schiffer, S., Bayesian strategy assessment in multi-attribute decision making, Journal of Behavioral Decision Making, 193-213 (2003)
[5] Bröder, A.; Schiffer, S., Take the best versus simultaneous feature matching: probabilistic inferences from memory and effects of reprensentation format, Journal of Experimental Psychology: General, 132, 277-293 (2003)
[6] Erdfelder, E.; Auer, T.-S.; Hilbig, B. E.; Assfalg, A.; Moshagen, M.; Nadarevic, L., Multinomial processing tree models: a review of the literature, Zeitschrift fur Psychologie/Journal of Psychology, 217, 108-124 (2009)
[7] Grünwald, P., Model selection based on minimum description length, Journal of Mathematical Psychology, 44, 133-152 (2000) · Zbl 0968.62008
[8] Grünwald, P., The minimum description length principle (2007), MIT Press: MIT Press Cambridge, MA
[9] Grünwald, P.; Myung, J. I.; Pitt, M. A., Advances in minimum description length: theory and applications (2005), MIT Press: MIT Press Cambridge, MA
[10] Harvey, P., Reality monitoring in mania and schizophrenia, Journal of Nervous and Mental Disorders, 173, 67-73 (1985)
[11] Hilbig, B. E.; Moshagen, M., Generalized outcome-based strategy classification: a guide to comparing deterministic and probabilistic choice models, Psychonomic Bulletin & Review (2014), in press
[12] Klauer, K. C.; Wegener, I., Unraveling social categorization in the ‘Who said what?’ paradigm, Journal of Personality and Social Psychology, 75, 1155-1178 (1998)
[13] Moshagen, M., multiTree: a computer program for the analysis of multinomial processing tree models, Behavior Research Methods, 42, 42-54 (2010)
[14] Myung, J. I., The importance of complexity in model selection, Journal of Mathematical Psychology, 44, 190-204 (2000) · Zbl 0946.62094
[15] Myung, J. I.; Navarro, D. J.; Pitt, M. A., Model selection by normalized maximum likelihood, Journal of Mathematical Psychology, 50, 167-179 (2006) · Zbl 1100.94008
[16] Navarro, D. J., A note on the applied use of MDL approximations, Neural Computation, 16, 1763-1768 (2004) · Zbl 1050.62112
[17] Pitt, M. A.; Myung, I. J.; Zhang, S., Toward a method of selecting among computational models of cognition, Psychological Review, 109, 472-491 (2002)
[18] Rissanen, J., Modeling by shortest data description, Automatica, 14, 465-471 (1978) · Zbl 0418.93079
[19] Rissanen, J., Fisher information and stochastic complexity, IEEE Trans. Inform. Theory, 42, 40-47 (1996) · Zbl 0856.94006
[20] Rissanen, J., Strong optimality of the normalized ML models as universal codes and information in data, IEEE Trans. Inform. Theory, 47, 1712-1717 (2001) · Zbl 0999.94016
[21] Schwarz, G., Estimating the dimension of a model, Annals of Statistics, 6, 461-464 (1978) · Zbl 0379.62005
[22] Singmann, H.; Kellen, D., MPTinR: analysis of multinomial processing tree models in R, Behavior Research Methods, 45, 560-575 (2013)
[23] Wu, H.; Myung, J. I.; Batchelder, W. H., On the minimum description length complexity of multinomial processing tree models, Journal of Mathematical Psychology, 54, 291-303 (2010) · Zbl 1190.62192
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.