Agent-based models in the philosophical literature frequently take scientific communities as their targets. Modelers refer to agents as scientists, epistemic landscapes as research topics; they motivate their models with situations scientists encounter in real communities like reaching consensus on a theory or choosing their next research project (Zollman 2007, Weisberg and Muldoon 2009). While these models and others like them are clearly inspired by real world systems, when it comes to interpreting their results, modelers are justifiably conservative. Agent-based models of scientific communities are often taken to represent possible phenomena, but modelers are careful to avoid claims about how likely their results are to obtain in real world systems. This is largely because such models are highly simplified and abstract away from the complexities of real communities (Rosenstock et al. 2017). One compelling reason (among many) to build agent-based models is to provide insight on optimal ways to organize real scientific communities. In other words, to inform policy and institutional design. To aid in policy decisions we want reasons to believe that model results will obtain in the real world with some non-trivial probability. My project is to describe how to get there from existing ‘how possibly’ models.
To address the problem of loose model-target relation, I propose developing mid-level models. A mid-level model falls between the kinds of highly abstracted models referenced above and a particular real-world context such as a case study or pilot test. This approach uses empirical data to shrink the parameter space of a model to conditions that are in fact encountered in the real world system. The crux of Rosenstock et al.’s concern is that it is often unclear whether a real world community and a model occupy the same parameter space (2017). Calibrating the parameters of a model based on empirical information alleviates this concern.
Building a mid-level model requires making decisions about which features to include, their degree of correspondence to the target system, and how to incorporate empirical data. The agent-based modeling literature is rich in examples of models, and new work is beginning to explore the methodology behind these important modeling decisions. In this paper I adapt ideas from existing modeling theory to guide the construction of my mid-level model.
I first draw on the tripartite account of simulations proposed by Grim et al. (2013). According to this account, simulations are made up of three parts: input conditions, mechanisms and output conditions. Different modeling goals will lend themselves to different objects of inquiry. For example, to build a predictive model, the input conditions and mechanisms of the model are constructed based on what is known about the target system, and new information is gleaned from the output conditions. This tripartite construction points modelers towards which features they must include, and which are to be elucidated through simulation. For mid-level models, the known parts must correspond to the real world system, and therefore are the features that should incorporate empirical data.
The trickiest task of building a mid-level model is determining the degree of correspondence required between the model and the world. According to similarity accounts of model-world relationships, which similarities are relevant and to what degree they are similar depends on the case at hand and the goals of a modeler (Weisberg 2013). This sounds good in theory, but when faced with the task of constructing a model how can one tell which model features require tight correspondence, and for which abstraction or simplification will suffice? One answer is robustness testing. If the output of a model is robust across a number of parameters, this gives us confidence that it is not an artifact of some narrow set of conditions. However, without addressing Rosenstock et al.’s concern, a model may be robust across a wide range of parameter space that has little correspondence with the parameter space of the real system. I propose that key features (particularly the aspects purported to be known according to the tripartite account of model structure) ought to be informed by empirical data as a starting point for reasonable parameter ranges. From there, robustness analysis can advise on which features are particularly sensitive to parameters, and which are robust. For those that are sensitive, it is important that they do in fact correspond to the world.
I use a mid-level model of science funding systems to illustrate how the framework presented here can be used to construct a model. The goal of this model is to predict which funding mechanism will maximize the generation of significant science. Grim et al.’s tripartite structure analysis helps clarify the appropriate model structure. We have information about how science is currently organized (various disciplines, number of scientists, etc.), which constitute the input conditions. We also know how current funding mechanisms work, and we can hypothesize realistic designs for new ones. We seek to determine what output conditions, namely the production of scientific knowledge, will result from the combination of these input conditions and mechanisms. Citation metrics from scientific papers provide a measure of scientific significance, and are used to construct an empirically driven epistemic landscape. The funding mechanisms that select among agents in the model are also informed by empirical studies of science funding.
If philosophers are interested in working towards practical strategies for optimally organizing scientific communities, we need to move from ‘how possibly’ to more predictive models. One way to close the gap between model and target system is to incorporate empirical information. The field is working towards generating best practices for incorporating empirical information into agent-based models and determining what kind of correspondence is required for various model applications. I take a step towards these goals by providing a framework for building mid-level models.
- Grim, P., Rosenberger, R., Rosenfeld, A., Anderson, B., & Eason, R. E. (2013). How simulations fail. Synthese, 190(12), 2367-2390.
- Rosenstock, S., O’Connor C., and Bruner, J.P. (2017). In epistemic networks is less really more? Philosophy of Science, 82(2), 234-252.
- Weisberg, M. and Muldoon, R. (2009). Epistemic landscapes and the division of cognitive labor. Philosophy of Science, 76(2): 225-252.
- Weisberg, M. (2013). Simulation and similarity: Using models to understand the world. Oxford University Press.
- Zollman, K. J. (2007). The communication structure of epistemic communities. Philosophy of Science, 74(5), 547-587.