zbMATH — the first resource for mathematics

Sequential sampling, magnitude estimation, and the wisdom of crowds. (English) Zbl 1396.91647
Summary: Sir Francis Galton [Fr. Galton, Nature, London 75, 450–451, 509–510 (1905; JFM 38.0288.04)] conjectured the psychological process of magnitude estimation caused the curious distribution of judgments he observed at Plymouth in 1906. However, after he published Vox Populi, researchers narrowed their attention to the first moment of judgment distributions and its often remarkable alignment with the truth, while it became customary to explain this wisdom of crowds effect using ideas of statistics more than psychology, and without considering possible interactions with other distribution moments. Recently, however, an exploration of the cognitive foundation of judgment distributions was published [U. W. Nash, “The curious anomaly of skewed judgment distributions and systematic error in the wisdom of crowds”, PLoS One 9, No. 11, e112386, 17 p. (2014; doi:10.1371/journal.pone.0112386)]. The study not only formalized a possible link between signal detection, evidence accumulation, and the shape of judgment distributions, but also in so doing, conjectured that magnitude estimation by independent individuals causes a systematic error in the wisdom of crowds indicated by judgment distribution skewness. The present study reports findings from an experiment on magnitude estimation and supports these predictions. The study moreover demonstrates that systematic errors by groups of people can be corrected using information about the judgment distribution these people together form, before errors might cause damage to decision making. In concluding, we revisit Galton’s data from the West of England Fat Stock and Poultry Exhibition in light of what we have discovered.
91E30 Psychophysics and psychophysiology; perception
62P15 Applications of statistics to psychology
62F07 Statistical ranking and selection procedures
Full Text: DOI
[1] Berniker, M.; Voss, M.; Kording, K., Learning priors for Bayesian computations in the nervous system, PLoS One, 5, 9, 1-9, (2010)
[2] Bitzer, S.; Park, H.; Blankenburg, F.; Kiebel, S. J., Perceptual decision making: drift-diffusion model is equivalent to a Bayesian model, Frontiers in Human Neuroscience, 8, February, 102, (2014)
[3] Britten, K. H.; Shadlen, M. N.; Newsome, W. T.; Movshon, Ja, The analysis of visual motion: a comparison of neuronal and psychophysical performance, The Journal of Neuroscience, 12, 12, 4745-4765, (1992)
[4] Brunswik, E., Organismic achievement and environmental probability, Psychological Review, 50, 3, 255-272, (1943)
[5] Brunswik, E., Perception and the representative design of psychological experiments, (1956), University of California Press Berkeley
[6] Brunton, B. W.; Botvinick, M. M.; Brody, C. D., Rats and humans can optimally accumulate evidence for decision-making, Science, 340, 6128, 95-98, (2013)
[7] Cavonius, C. R.; Hilz, R.; Chapman, R. M., A possible basis for individual differences in magnitude estimation behaviour, British Journal of Psychology, 65, 1, 85-91, (1974)
[8] Cicchini, G. M.; Arrighi, R.; Cecchetti, L.; Giusti, M.; Burr, D. C., Optimal encoding of interval timing in expert percussionists, Journal of Neuroscience, 32, 3, 1056-1060, (2012)
[9] Faisal, aA; Selen, L. P.J.; Wolpert, D. M., Noise in the nervous system, Nature Reviews Neuroscience, 9, 4, 292-303, (2008)
[10] Fechner, G.T. (1897). Kollectivmasslehre, Engleman, Engleman.
[11] Forstmann, B. U.; Ratcliff, R.; Wagenmakers, E. J., Sequential sampling models in cognitive neuroscience: advantages, applications, and extensions, Annual Review of Psychology, 67, 641-666, (2016)
[12] Galton, F., Natural inheritance, (1894), Macmillan
[13] Galton, F. (1907a). The Ballot-Box. Nature Letters to Editor. p. 509.
[14] Galton, F., Vox populi, Nature, 75, 450-451, (1907) · JFM 38.0288.04
[15] Gigerenzer, G.; Goldstein, D. G., Reasoning the fast and frugal way: models of bounded rationality, Psychological Review, 103, 4, 650-669, (1996), oct
[16] Gold, J. I.; Shadlen, M. N., The neural basis of decision making, Annual Review of Neuroscience, 30, 1, 535-574, (2007)
[17] Goldstein, D.G., McAfee, R.P., & Suri, S. (2014). The wisdom of smaller, smarter crowds. In Proceedings of the fifteenth ACM conference on economics and computation - EC’14 (pp. 471-488).
[18] Green, D. M.; Luce, R. D., Detection of auditory signals presented at random times: III, Perception & Psychophysics, 9, 3, 257-268, (1971)
[19] Green, D. M.; Swets, J. A., Signal detection theory and psychophysics, vol. 1, (1966), John Wiley & Sons, Inc. New York
[20] Griffiths, T. L.; Chater, N.; Norris, D.; Pouget, A., How the bayesians got their beliefs (and what those beliefs actually are): comment on bowers and Davis (2012), Psychological Bulletin, 138, 3, 415-422, (2012)
[21] Helson, H., Adaptation-level as frame of reference for prediction of psychophysical data, The American Journal of Psychology, 60, 1, 1-29, (1947)
[22] Hoffman, P. J., The paramorphic representation of clinical judgment, Psychological Bulletin, 57, 116-131, (1960)
[23] Hollingworth, H. L., The central tendency of judgment, The Journal of Philosophy, Psychology and Scientific Methods, 461-469, (1910)
[24] Kayaert, G.; Biederman, I.; Op De Beeck, H. P.; Vogels, R., Tuning for shape dimensions in macaque inferior temporal cortex, European Journal of Neuroscience, 22, 1, 212-224, (2005)
[25] LaBerge, D., A recruitment theory of simple behavior, Psychometrika, 27, 4, 375-396, (1962)
[26] Laplace, P.S. (1812). Theorie analytique des probabilites, Paris, Ve Courcier. · JFM 18.0166.01
[27] Latimer, K. W.; Yates, J. L.; Meister, M. L.; Huk, A. C.; Pillow, J. W., Single-trial spike trains in parietal cortex reveal discrete steps during decision-making, Science, 349, 6244, 184-187, (2015)
[28] Latinus, M.; McAleer, P.; Bestelmeyer, P. E.G.; Belin, P., Norm-based coding of voice identity in human auditory cortex, Current Biology, 23, 12, 1075-1080, (2013)
[29] Leopold, Da; Bondar, I. V.; Giese, Ma, Norm-based face encoding by single neurons in the monkey inferotemporal cortex, Nature, 442, 7102, 572-575, (2006)
[30] Loffler, G.; Yourganov, G.; Wilkinson, F.; Wilson, H. R., FMRI evidence for the neural representation of faces, Nature Neuroscience, 8, 10, 1386-1390, (2005)
[31] Lorenz, J.; Rauhut, H.; Schweitzer, F.; Helbing, D., How social influence can undermine the wisdom of crowd effect, Proceedings of the National Academy of Sciences of the United States of America, 108, 22, 9020-9025, (2011)
[32] Luce, R. D., What sort of measurement is psychophysical measurement?, The American Psychologist, 27, 2, 96-106, (1972)
[33] Luce, R. D.; Mo, S. S., Magnitude estimation of heaviness and loudness by individual subjects: A test of a probabilistic response theory, British Journal of Mathematical and Statistical Psychology, 18, 2, 159-174, (1965)
[34] Marr, D., Vision: A computational investigation into the human representation and processing of visual information, Phenomenology and the Cognitive Sciences, 8, 4, 397, (1982)
[35] Morgan, M. J.; Watamaniuk, S. N.J.; McKee, S. P., The use of an implicit standard for measuring discrimination thresholds, Vision Research, 40, 17, 109-117, (2000)
[36] Muchnik, L.; Aral, S.; Taylor, S. J., Social influence bias: a randomized experiment, Science (New York, NY), 341, 6146, 647-651, (2013)
[37] Muth, J. F., Rational expectations and the theory of price movements, Econometrica, 29, 3, 315-335, (1961)
[38] Nash, U. W., The curious anomaly of skewed judgment distributions and systematic error in the wisdom of crowds, PLoS One, 9, 11, e112386, (2014)
[39] Newsome, William T.; Britten, K. H.; Anthony Movshon, J., Neural correlates of a perceptual decision, Nature, 341, 6237, 52-54, (1989)
[40] Perry-Coste, F.H. (1907). The Ballot-Box. Nature Letters to Editor. (March 28), p. 509. · JFM 38.0288.06
[41] Pike, R., Response latency models for signal detection, Psychological Review, 80, 1, 53-68, (1973)
[42] Ratcliff, R., A theory of memory retrieval, Psychological Review, 85, 2, 59-108, (1978)
[43] Rhodes, G.; Robbins, R.; Jaquet, E.; McKone, E.; Jeffery, L.; Clifford, C. W.G., Adaptation and face perception: how aftereffects implicate norm-based coding of faces, (Fitting the mind to the world: Adaptation and after-effects in high-level vision, (2005), Oxford University Press), 213-235
[44] Roitman, J. D.; Shadlen, M. N., Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task, Journal of Neuroscience, 22, 21, 9475-9489, (2002)
[45] Shadlen, M. N.; Newsome, W. T., Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey, Journal of Neurophysiology, 86, 4, 1916-1936, (2001)
[46] Simons, A. M., Many wrongs: the advantage of group navigation, Trends in Ecology and Evolution, 19, 9, 453-455, (2004)
[47] Stevens, S. S.; Greenbaum, H. B., Regression effect in psychophysical judgment, Perception & Psychophysics, 439-446, (1966)
[48] Stocker, Aa; Simoncelli, E. P., Noise characteristics and prior expectations in human visual speed perception, Nature Neuroscience, 9, 4, 578-585, (2006)
[49] Surowiecki, J., The wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations, (2004), Doubleday
[50] Thurstone, L. L., Psychophysical analysis, American Journal of Psychology, 38, 368, (1927)
[51] Tolman, E. C.; Brunswik, E., The organism and the causal texture of the environment, Psychological Review, 42, 1, 43-77, (1935)
[52] Townsend, J. T.; Ashby, F. G., Stochastic modeling of elementary psychological processes, The American Journal of Psychology, 480, (1983) · Zbl 0595.92015
[53] Vickers, D., Evidence for an accumulator model of psychophysical discrimination, Ergonomics, 13, 1, 37-58, (1970)
[54] Vul, E.; Pashler, H., Measuring the crowd within: probabilistic representations within individuals, Psychological Science, 19, 7, 645-647, (2008)
[55] Wagenmakers, E. J.; Brown, S., On the linear relation between the mean and the standard deviation of a response time distribution, Psychological Review, 114, 3, 830-841, (2007)
[56] Wallis, K. F., Revisiting francis galton’s forecasting competition, Statistical Science, 29, 3, 420-424, (2014) · Zbl 1331.62025
[57] Yang, T.; Shadlen, M. N., Probabilistic reasoning by neurons, Nature, 447, 7148, 1075-1080, (2007)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.