×

A pragmatic strategy for discovering and testing threats to the validity of sociotechnical experiments. (English) Zbl 1011.68835

Simulation Modelling Practice and Theory 10, No. 3-4, 169-194 (2002).
Summary: This paper presents a new method for structuring decision problems as an essential aspect of solving them. This new method, the method of context validation, is a form of what Campbell (From Evolutionary Epistemology Via Selection Theory to a Sociology of Scientific Validity, 1996) and Dunn (Testing Rival Hypotheses with Pragmatic Eliminative Induction: The Case of National Maximum Speed Limits, unpublished; Am. Behav. Sci. 40 (3), 277 (1997); Knowledge, Power, and Participation in Environmental Policy Analysis, 2001) call pragmatic eliminative induction. By adding the term ”pragmatic” to ”eliminative induction,” the method is distinguished from Mill’s (J. Washington Acad. Sci. 16 (2), 317 (1926)) influential analytical (or logical) variant of eliminative induction, which has been influential in designing social experiments that seek to investigate rival hypotheses, or so-called ”threats to validity,” that must be tested and eliminated to know whether a technological intervention is responsible for changes in a target social system. The paper shows how context validation can be used to estimate an approximately complete set of rival hypotheses, a sine qua non of research on complex sociotechnical systems. The method is exemplified by applying it to a major sociotechnical experiment, the US National Maximum Speed Limit of 1974. The method is shown to mitigate the commission of Type III errors (solving the wrong problem), which in the present case stemmed from the failure to define the relation between speed and traffic safety as a social and political, as well as technical problem. Because of this failure, policy makers concluded that maximum speed limits were effective in saving lives when they were not.

MSC:

68U99 Computing methodologies and applications
68U20 Simulation (MSC2010)
Full Text: DOI