Validation and verification of social processes within agent-based computational organization models. (English) Zbl 1134.91573

Yilmaz, Levent

Comput. Math. Organ. Theory 12, No. 4, 283-312 (2006).

Summary: The use of simulation modeling in computational analysis of organizations is becoming a prominent approach in social science research. However, relying on simulations to gain intuition about social phenomena has significant implications. While simulations may give rise to interesting macro-level phenomena, and sometimes even mimic empirical data, the underlying micro and macro level processes may be far from realistic. Yet, this realism may be important to infer results that are relevant to existing theories of social systems and to policy making. Therefore, it is important to assess not only predictive capability but also explanation accuracy of formal models in terms of the degree of realism reflected by the embedded processes. This paper presents a process-centric perspective for the validation and verification (V&V) of agent-based computational organization models. Following an overview of the role of V&V within the life cycle of a simulation study, emergent issues in agent-based organization model V&V are outlined. The notion of social contract that facilitates capturing micro level processes among agents is introduced to enable reasoning about the integrity and consistency of agent-based organization designs. Social contracts are shown to enable modular compositional verification of interaction dynamics among peer agents. Two types of consistency are introduced: horizontal and vertical consistency. It is argued that such local consistency analysis is necessary, but insufficient to validate emergent macro processes within multi-agent organizations. As such, new formal validation metrics are introduced to substantiate the operational validity of emergent macro-level behavior.

MSC:

91D10

Models of societies, social and urban evolution

Keywords:

agent-based modeling; computational organization; simulation; social processes; validation; verification

References:

[1]	Aho VA (1990) Algorithms for finding patterns in strings. Handbook of Theoretical Computer Science, Volume A: Algorithms and complexity, North Holland, Amsterdam, Elsevier Science Publishers B. V (North Holland), Amsterdam, pp 255–400
[2]	Axelrod R (1997) Advancing the art of simulation in the social sciences. In: Conte Rosario, Hegselmann Rainer, and Terna Pietro (eds) Simulating Social Phenomena. Springer, Berlin, pp 21–40
[3]	Axtell R (2003) The new coevolution of information science and social science, Working Paper, Brookings Institution 2003
[4]	Balci O, Nance RE (1985) Formulated problem verification as an explicit requirement of model credibility. Simulation 45(2):76–86
[5]	Balci O (1987) Credibility and assessment of simulation results: The State of the Art. In: Balci O (ed) Methodology and Validation proceedings of the conference on methodology and validation, 6–9 April. Orlando Fl. San Diego, CA, Society for Computer Simulation
[6]	Banks J, Gerstein S, Searles SP (1987) Modeling processes, validation, and verification of complex simulations: A survey. In: Balci O (ed) Methodology and Validation Proceedings of the conference on methodology and validation, Orlando Fl, San Diego, CA, Society for Computer Simulation, pp. 13–18
[7]	Barnsley MF et al (1988) The science of fractal images. Springer-Verlag, New York
[8]	Carley MK, Newell A (1994) The nature of the social agent. J Math Sociol 19(4): 221–262
[9]	Carley MK, Butts C (1997) An algorithmic approach to the comparison of partially labeled graphs. In: Proceedings of the 1997 International Symposium on Command and Control Research and Technology, Washington, DC
[10]	Carley MK, Gasser L (1999) Computational organization theory. In: G. Weiss (ed.) Distributed artificial intelligence, MIT Press, Cambridge, MA, (Ch. 7), pp 206–220
[11]	Carley MK (2002) Computational organizational science and organizational engineering. Simulation Practice and Theory. (In press) · Zbl 1011.68839
[12]	Clarke EM, Grumberg O, Long DE (1994) Model checking and abstraction. ACM Transa Programming Lang Sys 16(5):1512–1542
[13]	Dalal PN, Kamath M, Kolarik WJ, Sivaraman E (2004) Toward an integrated framework for modeling enterprise processes. Communications of the ACM, 47(3):83–87 · Zbl 05393476
[14]	Derry R, Landry M, Banville C (1993) Revisiting the issue of model validation in OR: An epistemological view. Eur J Oper Res 66(2):168–184
[15]	Ferber J (1999) Multi agent systems: An introduction to distributed Artificial Intelligence. Addison Wesley
[16]	Epstein J, Axtell R (1996) Growing artificial societies: Social science from the Bottom Up, MIT Press Brookings, MA
[17]	Gasser L, Hulhage I, Leverich B, Lieb J, Majchrzak A (1993) Organization as Complex Dynamic Design Problems, Progress in Artificial Intelligence. Lecture Notes in Artificial Intelligence 727. Springer-Verlag
[18]	Jain R (1991) The art of computer systems performance analysis. Addison-Wesley · Zbl 0824.68013
[19]	Johnson E (1994) Miller and freuds probability and statistics for engineers, 5th Edition, Englewood-Cliffs, N.J, Prentice Hall
[20]	Kaufmann S (1996) At home in the universe: The search for laws of Self-Organization and Complexity, Oxford Univ. Press, Oxford, England
[21]	Kerlinger F (1973) Foundations of behavioral research. Reinhart & Winston, New York
[22]	Landry M, Malouin JL, Oral M (1983) Model validation in operations research. Eur J Oper Res 14(3):207–220
[23]	Law AM, Kelton WD (1991) Simulation modeling and analysis. 2nd edition. McGraw-Hill, New York, NY
[24]	Liskov B, Wing MJ (1997) A behavioral notion of subtyping. ACM TOPLAS 16(6):1811–1841
[25]	MacKenzie G, Schulmeyer G, Yilmaz L (2002) Verification technology potential with different modeling and simulation development and implementation Paradigms, Foundations for V & V in the 21st Century Workshop (Foundations ’02) The Johns Hopkins University Applied Physics Laboratory Laurel, Maryland, USA
[26]	Nance ER (1987) The conical methodology: A framework for simulation model development. Methodology and validation, SCS, San Diego, CA, pp 38–43
[27]	Naylor TH, Finger JM (1963) Verification of computer simulation models. Management Science 14(2)
[28]	Nazareth LD (1989) Issues in the verification of knowledge in rule-based systems. Inter J Man-Machine Studies 30:255–271
[29]	O’Keefe MR, O’Leary DE (1993) Expert system verification and validation. Artif Intell Rev 7(3):3–42
[30]	Oral M, Kettani O (1993) The facets of the modeling and validation process in operations research. Eur J Oper Res 66(2):216–234
[31]	Pace Dale K (2000) Simulation conceptual model issues: Development methods (Part 1), Interaction with Simulation requirements (Part 2), and simulation development costs and V & V Costs (Part 3). In: Proceedings of the 2000 Summer Computer Simulation Conference, July 16–20, 2000, Vancouver, British Columbia
[32]	Pace KD (2003) The roles of metrics in simulation verification and validation. In: Proceedings of the Spring 2003 Simulation Interoperability Workshop, March 30–April 4, 2003, Kissimmee, FL, CD
[33]	Pearl J (1999) Causality: Models, Reasoning, and Inference. Cambridge University Press this month (March 2000) · Zbl 0959.68116
[34]	Sankof D, Kruskal JB (1983) Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison. Addison-Wesley Publishing Company
[35]	Sargent GR (1996) Verifying and validating simulation models. In: Proceedings of the 1996 Winter Simulation Conference, pp 55–64
[36]	Sargent RG (1999) Validation and verification of simulation models. In: Farrington PA, Nembhard HB, Sturrock DT, and Evans GW, (eds), Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp 39–48
[37]	Spriet JA, Vansteenkiste GC (1982) Computer-Aided Modelling and Simulation. Academic Press, London, England)
[38]	Shoham Y (1993) Agent oriented programming. Artif Intell 60:51–92
[39]	Stephen AG (1994). String searching algorithms. World Scientific Publishing, Singapore · Zbl 0831.68028
[40]	Waldrop MM (1993) Complexity: The Emerging science at the edge of order and chaos. Touchstone books
[41]	Wing MJ, Farahani MV (1995) Model checking software systems: A case study. SIGSOFT Software Engineering Notes 20(4):128–39