Using Agent-Based Models to Examine Past Scientific Episodes: towards robust findings

AnneMarie Borg, Daniel Frey, Christian Straßer and Dunja Šešelja

Agent-based models (ABMs) have in recent years become an increasingly popular method for the study of social aspects of scientific inquiry. A common feature of ABMs developed in philosophy of science and social epistemology is that they are simple, ‘thin’ representations of scientific inquiry (Pöyhönen and Kuorikoski, 2016; Šešelja, 2018). The primary appeal of such models is that they allow for a sight-forward insight into possible causal mechanisms underlying the phenomenon in question (Reutlinger, Hangleiter, and Hartmann, 2016). The less components a model includes, the easier it gets to study causal dependencies between the given factors. Nevertheless, such simplicity comes at a price: the model will in turn be highly idealized, making it difficult to determine its relation to the real world.

Despite their highly idealized character, many of the ABMs proposed in the literature have been motivated by concrete episodes from the history of science, suggesting potential explanations of the given cases (Holman and Bruner, 2017; Holman and Bruner, 2015; O’Connor and Weatherall, 2017; Weatherall, O’Connor, and Bruner, 2018; Zollman, 2010). This has had two significant consequences for the reception of ABMs of science. On the one hand, these models have been considered to be primarily aiming at explaining real-world phenomena or at least providing ‘how-possibly explanations’ or ‘proofs of principle’ that should be applicable to the given cases. On the other hand, the lack of robustness analyses and empirical validation of the given findings has cast doubt on their link to real-world phenomena, and hence on the relevance of these results for actual scientific inquiry (even in a how-possibly way, see Frey and Šešelja, 2018b). While some have suggested that non-validated ABMs may still be explanatory of abstract theoretical phenomena (Šešelja, 2018), there is also a danger that the current trend of non-validated ABMs leads to the so-called ‘YAAWN’ syndrome: “Yet Another Agent-Based Model . . . Whatever . . . Nevermind . . . ” (O’Sullivan et al., 2016, see also Arnold, 2006, 2013, 2014, 2019), with the upshot that we are missing an opportunity to have more significant epistemic gains by employing simulations. In response to such dangers, two types of validation strategies have been suggested: on the one hand, empirical calibration of ABMs (Harnagel, 2018; Martini and Pinto, 2016), and on the other hand, their robustness analysis, with a special focus on the so-called derivational robustness analysis (Kuorikoski and Ylikoski, 2015; Lehtinen, 2017; Šešelja, 2018; Woodward, 2006). The latter is the examination of the results of a model by means of structurally different models in order to determine to which extent the findings depend on specific idealizations employed in the former and which target phenomena it adequately represents.

The aim of this paper is precisely to undertake the task of derivational robustness analysis and empirical validation with respect to Zollman’s (2010) ABM as a paradigmatic example of highly idealized ABMs of science. In particular, we examine the application of Zollman’s results to the case study from the history of medicine: the research on peptic ulcer disease (PUD). The case study of PUD was previously discussed by Zollman, 2010 who uses his model to explain the development of this historical episode. Frey and Šešelja, 2018a,b have previously examined the robustness of Zollman’s results under changes in idealizing assumptions of his model. Nevertheless, such an enhanced version of Zollman’s ABM is still highly idealized. To determine whether results of those simulations are robust under changes in the idealizing assumptions, it is important to employ a model of scientific inquiry that is structurally different from Zollman-inspired ones. To this end, we employ the argumentation-based ABM of scientific inquiry (ArgABM) (Borg et al., 2017, 2018). What makes ArgABM particularly suitable for the derivational robustness analysis of the previously obtained results is its specific approach to knowledge representation, which is structurally different from Zollman-inspired models. For instance, both defensible and anomalous parts of knowledge can be located as specific parts of the represented scientific theories. Moreover, ArgABM allows for the representation of argumentative dynamics underlying scientific inquiry, which is especially useful for the modeling of rivaling scientific theories. Similarly to Frey and Šešelja, 2018b, we examine which set of factors is more likely to lead to the dynamics of the PUD case by filtering for those results that conform to historical information we have about this episode.

The preliminary analysis of our results supports the findings by Frey and Šešelja that—contra Zollman—the cycle network is more likely to capture the dynamics of the PUD case. Furthermore, our simulations highlight the role of some additional factors that may impact inquiry, such as evaluations underlying theory-choice by scientists. For instance, agents in ArgABM may prefer theories that have a wider scope than their rivals, or they may avoid theories that exhibit more anomalies than their rivals. Our findings indicate that the PUD dynamics seems to fit the scenario in which agents employ the former type of evaluation rather than the latter one.

More generally, our investigation sheds light on the importance of using structurally different ABMs of science when trying to simulate past scientific episodes and to explain which factors are conducive to the dynamics of the given case-study. Due to the highly idealized character of these models it is easy to overlook a potential role of certain factors, which may in turn skew our explanations of concrete historical episodes. Derivational robustness analysis, in terms of structurally different ABMs, together with empirical calibration of the given models can aid in revealing such problems.


  • Arnold, Eckhart (2006). “The Dark Side of the Force: When computer simulations lead us astray and “model think” narrows our imagination—Preconference draft for the Models and Simulation Conference, Paris, June 12-14”. accessed on October 31, 2018. url: https://eckhartarnold.de/papers/2006_simulations/node10.html.
  • —(2013). “Simulation Models of the Evolution of Cooperation as Proofs of Logical Possibilities. How Useful Are They?” In: Etica & Politica/ Ethics & Politics XV (2), pp. 101–138.
  • —(2014). “What’s wrong with social simulations?” In: The Monist 97.3, pp. 359–377.
  • — (2019). “Validation of Computer Simulations from a Kuhnian Perspective”. In: Computer Simulation Validation - Fundamental Concepts, Methodological Frameworks, and Philosophical Perspectives. Cham: Springer.
  • Borg, AnneMarie, Daniel Frey, Dunja Šešelja, and Christian Straßer (2017). “Examining Network Effects in an Argumentative Agent-Based Model of Scientific Inquiry”. In: Logic, Rationality, and Interaction: 6th International Workshop, LORI 2017, Sapporo, Japan, September 11-14, 2017, Proceedings. Ed. by Alexandru Baltag, Jeremy Seligman, and Tomoyuki Yamada. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 391–406.
  • — (2018). “Epistemic effects of scientific interaction: approaching the question with an argumentative agent-based model”. In: Historical Social Research 43.1, pp. 285–309.
  • Frey, Daniel and Dunja Šešelja (2018a). “Robustness and Idealization in Agent-Based Models of Scientific Interaction”. In: British Journal for the Philosophy of Science https://doi.org/10.1093/bjps/axy039.
  • — (2018b). “What is the Epistemic Function of Highly Idealized AgentBased Models of Scientific Inquiry?” In: Philosophy of the Social Sciences https://doi.org/10.1177/0048393118767085.
  • Harnagel, Audrey (2018). “A Mid-Level Approach to Modeling Scientific Communities”. In: Studies in History and Philosophy of Science.
  • Holman, Bennett and Justin Bruner (2017). “Experimentation by industrial selection”. In: Philosophy of Science 84.5, pp. 1008–1019.
  • Holman, Bennett and Justin P Bruner (2015). “The problem of intransigently biased agents”. In: Philosophy of Science 82.5, pp. 956–968.
  • Kuorikoski, Jaakko and Petri Ylikoski (2015). “External representations and scientific understanding”. In: Synthese 192.12, pp. 3817–3837.
  • Lehtinen, Aki (2017). “Derivational robustness and indirect confirmation”. In: Erkenntnis, pp. 1–38.
  • Martini, Carlo and Manuela Fernández Pinto (2016). “Modeling the social organization of science”. In: European Journal for Philosophy of Science, pp. 1–18.
  • O’Connor, Cailin and James Owen Weatherall (2017). “Scientific polarization”. In: arXiv preprint arXiv:1712.04561.
  • O’Sullivan, David, Tom Evans, Steven Manson, Sara Metcalf, Arika Ligmann Zielinska, and Chris Bone (2016). “Strategic directions for agent-based modeling: avoiding the YAAWN syndrome”. In: Journal of land use science 11.2, pp. 177–187.
  • Pöyhönen, Samuli and Jaakko Kuorikoski (2016). “Modeling epistemic communities”. In: The Routledge Handbook of Social Epistemology (forthcoming). Ed. by M. Fricker, P. J. Graham, D. Henderson, N. Pedersen, and J. Wyatt. Routledge.
  • Reutlinger, Alexander, Dominik Hangleiter, and Stephan Hartmann (2016). “Understanding (with) Toy Models”. In: The British Journal for the Philosophy of Science.
  • Šešelja, Dunja (2018). “Exploring Scientific Inquiry via Agent-Based Modeling”. In: Forthcoming.
  • Weatherall, James Owen, Cailin O’Connor, and Justin Bruner (2018). “How to Beat Science and Influence People: Policy Makers and Propaganda in Epistemic Networks”. In: The British Journal for the Philosophy of Science. https://doi.org/10.1093/bjps/axy062.
  • Woodward, Jim (2006). “Some varieties of robustness”. In: Journal of Economic Methodology 13.2, pp. 219–240.
  • Zollman, Kevin J. S. (2010). “The epistemic benefit of transient diversity”. In: Erkenntnis 72.1, pp. 17–35.

Author: Research Group for Non-Monotonic Logics and Formal Argumentation

Created: 2019-03-17 Sun 18:26