×

An experimental investigation into the role of simulation models in generating insights. (English) Zbl 1346.91036

Summary: It is often claimed that discrete-event simulation (DES) models are useful for generating insights. There is, however, almost no empirical evidence to support this claim. To address this issue we perform an experimental study which investigates the role of DES, specifically the simulation animation and statistical results, in generating insight (an ‘Aha!’ moment). Undergraduate students were placed in three separate groups and given a task to solve using a model with only animation, a model with only statistical results, or using no model at all. The task was based around the UK’s NHS111 telephone service for non-emergency health care. Performance was measured based on whether participants solved the task with insight, the time taken to achieve insight and the participants’ problem-solving patterns. The results show that there is some association between insight generation and the use of a simulation model, particularly the use of the statistical results generated from the model. While there is no evidence that insights were generated more frequently from statistical results than the use of animation, the participants using the statistical results generated insights more rapidly.

MSC:

91A90 Experimental studies
90B50 Management decision making, including multiple objectives
91A80 Applications of game theory

Software:

SIMUL8
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[2] Agresti, A., Categorical data analysis (1990), John Wiley & Sons: John Wiley & Sons New York · Zbl 0716.62001
[3] Akinci, C.; Sadler-Smith, E., Intuition in management research: A historical review, International Journal of Management Reviews, 14, 1, 104-122 (2012)
[4] Argyris, C.; Schön, D. A., Organisational learning II: Theory method and practice (1996), Addison Wesley
[5] Bakken, B.; Gould, J.; Kim, D., Experimentation in learning organizations: A management flight simulator approach, European Journal of Operational Research, 59, 1, 167-182 (1992)
[6] Bayer, S.; Bolt, T.; Brailsford, S.; Kapsali, M., Models as Interfaces, (Brailsford, S.; Churilov, L.; Dangerfield, B., Discrete-Event Simulation and System Dynamics for Management Decision Making (2014), John Wiley & Sons: John Wiley & Sons Chichester), 125-139
[7] Bell, P. C.; O’Keefe, R. M., Visual interactive simulation — History, recent developments, and major issues, Simulation, 49, 3, 109-116 (1987)
[8] Bell, P. C.; O’Keefe, R. M., An experimental investigation into the efficacy of visual interactive simulation, Management Science, 41, 1018-1038 (1995)
[9] Belton, V.; Elder, M., Decision support systems: Learning from visual interactive modelling, Decision Support Systems, 12, 355-364 (1994)
[10] Bland, J. M.; Altman, D. G., Multiple significance tests: The Bonferroni method, British Medical Journal, 310, 170 (1995)
[11] Bonner, S. E.; Sprinkle, G. B., The effects of monetary incentives on effort and task performance: Theories, evidence, and a framework for research, Accounting, Organizations and Society, 27, 4, 303-345 (2002)
[12] Campbell, D.; Stanley, J., Experimental and quasi-experimental designs for research (1963), Rand-McNally: Rand-McNally Chicago, IL
[13] Carpenter, M. L.; Bauer, K. W.; Schuppe, T. F.; Vidulich, M. A., Animation: What’s essential for effective communication of military simulation model operation?, (Evans, G. W.; Mollaghasemi, M.; Rusell, E. C.; Biles, W. E., Proceeding of the 1993 winter simulation conference (1993), IEEE: IEEE Los Angeles, California), 1081-1088
[14] Chau, P.; Bell, P. C., Decision support for a new production plant design using visual interactive simulation, Journal of Operational Research Society, 45, 1273-1284 (1994) · Zbl 0812.90057
[15] Chronicle, E. P.; MacGregor, J. N.; Ormerod, T. C., What makes an insight problem? The roles of heuristics, goal conception, and solution recoding in knowledge-lean problems, Journal of Experimental Psychology: Learning, Memory and Cognition, 30, 1, 14-27 (2004)
[16] Cohen, J., Statistical power analysis for the behavioral sciences (1988), L. Erlbaum Associates: L. Erlbaum Associates Hillsdale · Zbl 0747.62110
[17] Cronin, M. A.; Gonzalez, C.; Sterman, J. D., Why don’t well-educated adults understand accumulation? A challenge to researchers, educators, and citizens, Organizational Behavior and Human Decision Processes, 108, 1, 116-130 (2009)
[18] Crookes, J. G., Simulation in 1981, European Journal of Operation Research, 9, 1-7 (1982)
[19] Dane, E.; Pratt, M., Exploring intuition and its role in managerial decision making, Academy of Management Review, 32, 1, 33-54 (2007)
[20] Davidson, J. E., The suddenness of insight, (Sternberg, R. J.; Davidson, J. E., The nature of insight (1995), The MIT Press: The MIT Press Cambridge, MA), 125-155
[21] de Vreede, G.-J.; Verbraeck, A., Animating organizational processes insight eases change, Simulation Practice and Theory, 4, 245-263 (1996)
[22] den Hengst, M.; de Vreede, G.-J.; Maghnouji, R., Using soft OR principles for collaborative simulation: A case study in the Dutch airline industry, Journal of the Operational Research Society, 58, 669-682 (2007) · Zbl 1177.90280
[23] Dickson, G. W., Graphics and managerial decision making: Research based guidelines, Communications of the ACM, 31, 6, 764-774 (1988)
[24] Elder, M. D., Visual interactive modelling: Some guidelines for its implementation and some aspects of its potential impact on operational research. Ph.D. thesis (1992), Department of Management Science, University of Strathclyde: Department of Management Science, University of Strathclyde Edinburgh
[25] Fleiss, J. L.; Levin, B.; Paik, M. C., Statistical methods for rates and proportions (2003), John Wiley and Sons: John Wiley and Sons New Jersey · Zbl 1034.62113
[26] Fletcher, A.; Halsall, D.; Huxham, D.; Worthington, D., The DH accident and emergency department model: A national generic model used locally, Journal of the Operational Research Society, 58, 12, 1554-1562 (2007)
[27] Fone, D.; Hollinghurst, S.; Temple, M.; Round, A.; Lester, N.; Weightman, A., Systematic review of the use and value of computer simulation modelling in population health and health care delivery, Journal of Public Health Medicine, 25, 4, 325-335 (2003)
[28] Forrester, J. W., Industrial dynamics - after the first decade, Management Science, 14, 7, 398-415 (1968)
[29] Gogi, A.; Tako, A. A.; Robinson, S., Generating insights: The effectiveness of simulation models in creative problem solving, (Tjahjono, B.; Heavey, C.; Onggo, S.; van der Zee, D.-J., Proceedings of the operational research society simulation workshop 2014 (2014), The Operational Research Society: The Operational Research Society Birmingham, UK), 133-142
[30] Griffiths, D.; Williams, J. E.; Wood, R. M., Modelling activities at a neurological rehabilitation unit, European Journal of Operational Research, 226, 2, 301-312 (2013) · Zbl 1292.90187
[31] Günal, M. M.; Pidd, M., Discrete event simulation for performance modelling in health care: A review of the literature, Journal of Simulation, 4, 42-51 (2010)
[32] Hämäläinen, R. P.; Luoma, J.; Saarinen, E., On the importance of behavioural operational research: The case of understanding and communicating about dynamic systems, European Journal of Operational Research, 228, 3, 623-634 (2013) · Zbl 1317.90143
[33] Hsiao, N.; Richardson, G. P., In search of theories of dynamic decision making: A literature review, (The 17th international conference of the system dynamics society (1999), Wellington, New Zealand)
[34] Howie, E.; Sy, S.; Ford, L.; Vicente, K. J., Human-computer interface design can reduce misperceptions of feedback, System Dynamics Review, 16, 3, 151-171 (2000)
[35] Hurrion, R. D., Invited review: Visual interactive modelling, European Journal of the Operational Research, 54, 281-287 (1986)
[36] Isaak, M. I.; Just, M. A., Constraints on thinking in insight and invention, (Sternberg, R. J.; Davidson, J. E., The nature of insight (1995), MIT Press: MIT Press Cambridge, MA), 281-325
[37] Jarvenpaa, S., The effect of task and graphical format on information processing strategies, Management Science, 35, 3, 285-303 (1989)
[38] Jun, J. B.; Jacobson, S. H.; Swisher, J. R., Applications of discrete event simulation in health care clinics: A survey, Journal of the Operational Research Society, 50, 109-123 (1999) · Zbl 1054.90597
[39] Kahneman, D., Thinking, fast and slow (2011), Farrar, Straus and Giroux: Farrar, Straus and Giroux New York
[40] Kounios, J.; Beeman, M., The Aha! moment: The cognitive neuroscience of insight, Current Directions in Psychological Science, 18, 4, 210-216 (2009)
[41] Langley, P. A.; Morecroft, J. D.W., Performance and learning in a simulation of oil industry dynamics, European Journal of Operational Research, 155, 3, 715-732 (2004) · Zbl 1043.90529
[42] Lee, C. B., The interactions between problem solving and conceptual change: System dynamic modelling as a platform for learning, Computers & Education, 55, 1145-1158 (2010)
[43] Lehaney, B.; Clarke, S. A.; Paul, R. J., A case of an intervention in an outpatients department, Journal of the Operational Research Society, 50, 877-891 (1999) · Zbl 1054.90599
[44] Luehrmann, M. A.H.; Byrkett, D. L., A pilot study of the impact of animation and decision-making, Simulation, 53, 153-158 (1989)
[45] Macal, C. M., Introduction to special issue on simulation and visualization, Simulation, 77, 90-92 (2001)
[46] MacGregor, J. N.; Ormerod, T. C.; Chronicle, E. P., Information processing and insight: A process model of performance on the 9-dot problem and related problems, Journal of Experimental Psychology: Learning, Memory and Cognition, 27, 1, 176-201 (2001)
[47] Maier, N. R.F., The behavior mechanisms concerned with problem solving, Psychological Review, 47, 1, 43-58 (1940)
[48] Mayer, R. E., Problem solving and reasoning, (Peterson, P.; Baker, E.; McGaw, B., International encyclopedia of education (2010), Elsevier: Elsevier Oxford), 273-278
[49] Metcalfe, J.; Wiebe, D., Intuition in insight and noninsight problem solving, Memory & Cognition, 15, 3, 238-246 (1987)
[50] Monks, T.; Robinson, S.; Kotiadis, K., Learning from discrete-event simulation: Exploring the high involvement hypothesis, European Journal of Operational Research, 235, 1, 195-205 (2014)
[51] Monxes, E., Misperceptions of basic dynamics: The case of renewable resource management, System Dynamics Review, 20, 139-162 (2004)
[52] O’Kane, J. F., Simulating production performance: Cross case analysis and policy implications, Industrial Management & Data Systems, 104, 4, 309-321 (2004)
[53] O’Keefe, R. M.; Pitt, I. L., Interaction with a visual interactive simulation, and the effect of cognitive style, European Journal of Operational Research, 54, 339-348 (1991) · Zbl 06908733
[54] Parker, D. C., Visual interactive financial models: An overview of micro-computer software offerings and discussion of potential decision support, European Journal of Operational Research, 54, 330-338 (1991) · Zbl 06908732
[55] Paul, R. J.; Giaglis, G. M.; Hlupic, V., Simulation of business processes: A review, American Behavioral Scientist, 42, 1551-1576 (1999)
[56] Pidd, M., Why modelling and model use matter, Journal of the Operational Research Society, 61, 14-24 (2010) · Zbl 1193.90156
[57] Polys, N. F.; Bowman, D. A.; North, C., The role of depth and gestalt cues in information-rich virtual environments, International Journal of Human Computer Studies, 69, 30-51 (2011)
[58] Powers, M. J.; Sanchez, S. M.; Lucas, T. W., The exponential expansion of simulation in research, (Laroque, C.; Himmelspach, J.; Pasupathy, R.; Rose, O.; Uhrmacher, A. M., Proceedings of the 2012 winter simulation conference (2012), IEEE: IEEE Berlin, Germany), 1552-1563
[59] Proudlove, N. C.; Black, S.; Fletcher, A., OR and the challenge to improve the NHS: Modelling for insight and improvement in in-patient flows, Journal of the Operational Research Society, 58, 2, 145-158 (2007) · Zbl 1140.90455
[60] Richardson, G. P.; Andersen, D. F.; Maxwell, T. A.; Stewart, T. R., Foundations of mental model research, (Proceedings of the international system dynamics conference 1994. Proceedings of the international system dynamics conference 1994, Stirling, UK (1994))
[61] Richels, R., Building good models is not enough, Interfaces, 11, 4, 48-54 (1981)
[62] Robinson, S., Soft with a hard centre: Discrete-event simulation in facilitation, Journal of the Operational Research Society, 52, 905-915 (2001)
[63] Robinson, S., Discrete-event simulation: From the pioneers to the present, what next?, Journal of the Operational Research Society, 56, 619-629 (2005) · Zbl 1095.68128
[64] Robinson, S., Conceptual modelling for simulation part I: Definition and requirements, Journal of the Operational Research Society, 59, 278-290 (2008) · Zbl 1178.90239
[65] Rouwette, E. A.J. A.; Größler, A.; Vennix, J. A.M., Exploring influencing factors on rationality: A literature review of dynamic decision-making studies in system dynamics, Systems Research and Behavioral Science, 21, 4, 351-370 (2004)
[66] Rouwette, E. A.J. A.; Vennix, J. A.M.; Felling, A. J.A., On evaluating the performance of problem structuring methods: An attempt at formulating a conceptual model, Group Decision and Negotiation, 18, 6, 567-587 (2009)
[67] Rouwette, E. A.J. A.; Korzilius, H.; Vennix, J. A.M.; Jacobs, E., Modeling as persuasion: The impact of group model building on attitudes and behaviour, System Dynamics Review, 27, 1-21 (2011)
[68] Schooler, J. W.; Fallshore, M.; Fiore, S. M., Epilogue: putting insight into perspective, (Sternberg, R. J.; Davidson, J. E., The nature of insight (1995), The MIT Press: The MIT Press Cambridge, MA), 559-587
[70] Smith, S. M., Getting into and out of mental ruts: A theory of fixation, incubation, and insight, (Sternberg, R. J.; Davidson, J. E., The nature of insight (1995), The MIT Press: The MIT Press Cambridge, MA), 229-251
[71] Sterman, J. D., Modeling managerial behavior: Misperceptions of feedback in a dynamic decision making experiment, Management Science, 35, 3, 321-339 (1989)
[72] Sterman, J. D., Learning in and about complex systems, System Dynamics Review, 10, 2-3, 291-330 (1994)
[73] Sterman, J. D.; Booth Sweeney, L., Understanding public complacency about climate change: Adults’ mental models of climate change violate conservation of matter, Climatic Change, 80, 3-4, 213-238 (2007)
[74] Sternberg, R. J., Cognitive psychology (2009), Cengage Learning: Cengage Learning Wadsworth
[75] Swider, C. L.; Bauer, W. B.; Schuppe, T. F., The effective use of animation in simulation model validation, (Tew, J. D.; Manivannah, S.; Sadowski, D. A.; Seila, A. F., Proceedings of the 1994 winter simulation conference (1994), IEEE: IEEE Orlando, FL, USA), 633-640
[76] Taylor, S. J.E.; Eldabi, T.; Riley, G.; Paul, R. J.; Pidd, M., Simulation modelling is 50! Do we need a reality check?, Journal of the Operational Research Society, 60, S69-S82 (2009) · Zbl 1168.90315
[77] Thyer, B. A., Quasi-experimental research design (2012), Oxford University Press, Inc: Oxford University Press, Inc New York
[78] van der Zee, D.&; Slomp, J., Simulation as a tool for gaming and training in operations management - A case study, Journal of Simulation, 3, 1, 17-28 (2009)
[79] Weisberg, R. W., Prolegomena to theories of insight in problem solving: A taxonomy of problems, (Sternberg, R. J.; Davidson, J. E., The nature of insight (1995), The MIT Press: The MIT Press Cambridge, MA), 157-196
[80] Wenzel, S.; Jessen, U., The integration of 3-D visualization into the simulation-based planning process of logistics systems, Simulation, 77, 114-127 (2001)
[81] Wertheimer, M., A gestalt perspective on computer simulations of cognitive processes, Computers in Human Behavior, 1, 19-33 (1985)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.