×

Do ‘big losses’ in judgmental adjustments to statistical forecasts affect experts’ behaviour? (English) Zbl 1346.91038

Summary: The behaviour of poker players and sports gamblers has been shown to change after winning or losing a significant amount of money on a single hand. In this paper, we explore whether there are changes in experts’ behaviour when performing judgmental adjustments to statistical forecasts and, in particular, examine the impact of ‘big losses’. We define a big loss as a judgmental adjustment that significantly decreases the forecasting accuracy compared to the baseline statistical forecast. In essence, big losses are directly linked with wrong direction or highly overshooting judgmental overrides. Using relevant behavioural theories, we empirically examine the effect of such big losses on subsequent judgmental adjustments exploiting a large multinational data set containing statistical forecasts of demand for pharmaceutical products, expert adjustments and actual sales. We then discuss the implications of our findings for the effective design of forecasting support systems, focusing on the aspects of guidance and restrictiveness.

MSC:

91A90 Experimental studies
91A80 Applications of game theory
91B06 Decision theory
62C10 Bayesian problems; characterization of Bayes procedures
91A60 Probabilistic games; gambling
91B84 Economic time series analysis
90B50 Management decision making, including multiple objectives
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Armstrong, J. S., Long range forecasting: From crystal ball to computer (1985), Wiley: Wiley New York
[2] Blattberg, R. C.; Hoch, S. J., Database models and managerial intuition: 50
[3] Bonaccio, S.; Dalal, R. S., Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences, Organizational Behavior and Human Decision Processes, 101, 127-151 (2006)
[4] Boulaksil, Y.; Franses, P. H., Experts’ stated behavior, Interfaces, 39, 168-171 (2009)
[5] Brown, L. D., Comparing judgmental to extrapolative forecasts - It’s time to ask why and when, International Journal of Forecasting, 4, 171-173 (1988)
[6] Carbone, R.; Andersen, A.; Corriveau, Y.; Corson, P. P., Comparing for different time series methods the value of technical expertise, individualized analysis and judgmental adjustment, Management Science, 29, 559-566 (1983)
[7] Coval, J. D.; Shumway, T., Do behavioral biases affect prices?, Journal of Finance, 60, 1-34 (2005)
[8] Davydenko, A.; Fildes, R., Measuring forecasting accuracy: The case of judgmental adjustments to SKU-level demand forecasts, International Journal of Forecasting, 29, 510-522 (2013)
[9] de Bruijn, B.; Franses, P. H., Managing sales forecasters, Tinbergen Institute discussion paper, TI 2012-131/III (2012)
[10] Dietvorst, B. J.; Simmons, J. P.; Massey, C., Algorithm aversion: people erroneously avoid algorithms after seeing them err, Journal of Experimental Psychology: General, 144, 114-126 (2015)
[11] Donihue, M. R., Evaluating the role judgement plays in forecast accuracy, Journal of Forecasting, 12, 81-92 (1993)
[12] Eggleton, I. R.C., Intuitive time series extrapolation, Journal of Accounting Research, 20, 68-102 (1982)
[13] Eroglu, C.; Croxton, K. L., Biases in judgmental adjustments of statistical forecasts: The role of individual differences, International Journal of Forecasting, 26, 116-133 (2010)
[14] Fildes, R.; Goodwin, P., Against your better judgment? How organizations can improve their use of management judgment in forecasting, Interfaces, 37, 570-576 (2007)
[15] Fildes, R.; Goodwin, P., Good and bad judgment in forecasting: Lessons from four companies, Foresight: The International Journal of Applied Forecasting, 8, 5-10 (2007)
[16] Fildes, R.; Goodwin, P.; Lawrence, M.; Nikolopoulos, K., Effective forecasting and judgmental adjustments: An empirical evaluation and strategies for improvement in supply-chain planning, International Journal of Forecasting, 25, 3-23 (2009)
[17] Fildes, R.; Hastings, R., The organization and improvement of market forecasting, Journal of the Operational Research Society, 45, 1-16 (1994)
[18] Fildes, R.; Petropoulos, F., Improving forecast quality in practice, Foresight: The International Journal of Applied Forecasting, 36, 5-12 (2015)
[19] Franses, P. H.; Legerstee, R., Properties of expert adjustments on model-based SKU-level forecasts, International Journal of Forecasting, 25, 35-47 (2009)
[20] Franses, P. H.; Legerstee, R., Do experts’ adjustments on model-based SKU-level forecasts improve forecast quality?, Journal of Forecasting, 29, 331-340 (2010) · Zbl 1204.91090
[21] Franses, P. H.; Legerstee, R., Combining SKU-level sales forecasts from models and experts, Expert Systems with Applications, 38, 2365-2370 (2011)
[22] Franses, P. H.; Legerstee, R., Experts’ adjustment to model-based SKU-level forecasts: Does the forecast horizon matter?, Journal of Operational Research Society, 62, 537-543 (2011)
[23] Franses, P. H.; Legerstee, R., Do statistical forecasting models for SKU-level data benefit from including past expert knowledge?, International Journal of Forecasting, 29, 80-87 (2013)
[24] Franses, P. H.; Legerstee, R.; Paap, R., Estimating loss functions of experts, Tinbergen Institute Discussion Paper, TI 2011-177/4 (2011)
[25] Garvey, R.; Murphy, A.; Wu, F., Do losses linger? Evidence from proprietary stock traders, Journal of Portfolio Management, 33, 75-83 (2007)
[26] Goodwin, P., Improving the voluntary integration of statistical forecasts and judgement, International Journal of Forecasting, 16, 85-99 (2000)
[27] Goodwin, P.; Fildes, R., Judgmental forecasts of time series affected by special events: Does providing a statistical forecast improve accuracy?, Journal of Behavioral Decision Making, 12, 37-53 (1999)
[28] Goodwin, P.; Fildes, R.; Lawrence, M.; Nikolopoulos, K., The process of using a forecasting support system, International Journal of Forecasting, 23, 391-404 (2007)
[29] Gorr, W. L.; Ord, J. K., Introduction to time series monitoring, International Journal of Forecasting, 25, 463-466 (2009)
[30] Hämäläinen, R. P.; Luoma, J.; Saarinen, E., On the importance of behavioral operational research: the case of understanding and communicating about dynamic systems, European Journal of Operational Research, 228, 623-634 (2013) · Zbl 1317.90143
[31] Harvey, N., Why are judgments less consistent in less predictable task situations?, Organizational Behavior and Human Decision Processes, 63, 247-263 (1995)
[32] Huang, T.; Fildes, R.; Soopramanien, D., The value of competitive information in forecasting FMCG retail product sales and the variable selection problem, European Journal of Operational Research, 237, 738-748 (2014)
[33] Hyndman, R. J.; Koehler, A. B., Another look at measures of forecast accuracy, International Journal of Forecasting, 22, 679-688 (2006)
[34] Kottemann, J. E.; Davis, F. D.; Remus, W. E., Computer assisted decision making: performance, beliefs, and the illusion of control, Organizational Behavior and Human Decision Processes, 57, 26-37 (1994)
[35] Lawrence, M.; Goodwin, P.; O’Connor, M.; Onkal, D., Judgmental forecasting: A review of progress over the last 25 years, International Journal of Forecasting, 22, 493-518 (2006)
[36] Lawrence, M.; O’Connor, M.; Edmundson, B., A field study of sales forecasting accuracy and processes, European Journal of Operational Research, 122, 151-160 (2000) · Zbl 0962.91503
[37] Legerstee, R.; Franses, P. H., Do experts’ SKU forecasts improve after feedback?, Journal of Forecasting, 33, 69-79 (2014)
[38] Leitner, J.; Leopold-Wildburger, U., Experiments on forecasting behavior with several sources of information a review of the literature, European Journal of Operational Research, 213, 459-469 (2011)
[39] Lim, J. S.; O’Connor, M., Judgmental adjustment of initial forecasts: Its effectiveness and biases, Journal of Behavioral Decision Making, 8, 149-168 (1995)
[40] Mathews, B.; Diamantopoulos, A., Managerial intervention in forecasting. An empirical investigation of forecast manipulation, International Journal of Research in Marketing, 3, 3-10 (1986)
[41] Mathews, B.; Diamantopoulos, A., Judgmental revision of sales forecasts: A longitudinal extension, Journal of Forecasting, 8, 129-140 (1989)
[42] Mathews, B.; Diamantopoulos, A., Judgmental revision of sales forecasts: effectiveness of forecast selection, Journal of Forecasting, 9, 407-415 (1990)
[43] McCarthy, T. M.; Davis, D. F.; Golicic, S. L.; Mentzer, J. T., The evolution of sales forecasting management: A 20-year longitudinal study of forecasting practices, Journal of Forecasting, 25, 303-324 (2006)
[44] McNees, S., The role of judgment in macroeconomic forecasting accuracy, International Journal of Forecasting, 6, 287-299 (1990)
[45] O’Connor, M.; Remus, W.; Griggs, K., Judgmental forecasting in times of change, International Journal of Forecasting, 9, 163-172 (1993)
[46] Önkal, D.; Gönül, M. S., Judgmental adjustment: A challenge for providers and users of forecasts, Foresight: The International Journal of Applied Forecasting, 1, 13-17 (2005)
[47] Petropoulos, F.; Makridakis, S.; Assimakopoulos, V.; Nikolopoulos, K., ‘Horses for Courses’ in demand forecasting, European Journal of Operational Research, 237, 152-163 (2014) · Zbl 1304.62117
[48] Ramnath, S.; Rock, S.; Shane, P., The financial analyst forecasting literature: A taxonomy with suggestions for further research, International Journal of Forecasting, 24, 34-75 (2008)
[49] Remus, W.; O’Connor, M.; Griggs, K., The impact of information of unknown correctness on the judgmental forecasting process, International Journal of Forecasting, 14, 313-322 (1998)
[50] Sauter, V. L., Decision support systems: An applied managerial approach (1997), Wiley: Wiley New York · Zbl 1202.90003
[51] Silver, M. S., Decisional guidance for computer-based decision support, MIS Quarterly, 15, 105-122 (1991)
[52] Smith, G.; Levere, M.; Kurtzman, R., Poker player behavior after big wins and big losses, Management Science, 55, 1547-1555 (2009)
[53] Spranca, M.; Minsk, E.; Baron, J., Omission and commission in judgment and choice, Journal of Experimental Social Psychology, 27, 76-105 (1991)
[54] Staw, B. M., Knee-deep in the big muddy: A study of escalating commitment to a chosen course of action, Organizational Behavior and Human Performance, 16, 27-44 (1976)
[55] Syntetos, A. A.; Nikolopoulos, K.; Boylan, J. E., Judging the judges through accuracy-implication metrics: The case of inventory forecasting, International Journal of Forecasting, 26, 134-143 (2010)
[56] Trapero, J. R.; Pedregal, D. J.; Fildes, R.; Kourentzes, N., Analysis of judgmental adjustments in the presence of promotions, International Journal of Forecasting, 29, 234-243 (2013)
[57] Turner, D. S., The role of judgement in macroeconomic forecasting, Journal of Forecasting, 9, 315-345 (1990)
[58] Tversky, A.; Kahneman, D., Advances in prospect theory: Cumulative representation of uncertainty, Journal of Risk and Uncertainty, 5, 297-323 (1992) · Zbl 0775.90106
[59] Tyebjee, T. T., Behavioral biases in new product forecasting, International Journal of Forecasting, 3, 393-404 (1987)
[60] Vere, D. T.; Griffiths, G. R., Modifying quantitative forecasts of livestock production using expert judgements, an application to the australian lamb industry, Journal of Forecasting, 14, 453-464 (1995)
[61] Wachtel, R. E.; Dexter, F., Review of behavioral operations experimental studies of newsvendor problems for operating room management, Anesthesia and Analgesia, 110, 1698-1710 (2010)
[62] Wolfe, C.; Flores, B., Judgmental adjustment of earnings forecasts, Journal of Forecasting, 9, 389-405 (1990)
[63] Xu, J.; Harvey, N., Carry on winning: the gamblers’ fallacy creates hot hand effects in online gambling, Cognition, 131, 173-180 (2014)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.