The USIP Learning Agenda: An Evidence Review

The Application of Systems Science to Peacebuilding

Matt Kasman

Center on Social Dynamics and Policy, Economics Studies Program, The Brookings Institution

Nicole Strombom

Public Health, Brown School, Washington University

Ross A. Hammond

Center on Social Dynamics and Policy, Economics Studies Program, The Brookings Institution Public Health, Brown School, Washington University Santa Fe Institute

MARCH 2023

Introduction

Broadly defined, peacebuilding encompasses a wide variety of activities aimed at halting or preventing violent conflict within or between states. Many people working in peacebuilding and related fields such as international development and international relations recognize the potential value in taking a systems perspective that can help understand and shape how individuals situated in interconnected social systems shift their behaviors from violence to nonviolence.¹ Among others, the United States Institute of Peace (USIP) has taken concrete steps to use this perspective to guide program design, evaluation, and strategy. In conducting this evidence review, we sought to build on these steps by considering specific methodologies within the umbrella of systems science and their potential contributions to the field. We reviewed existing research using such methods to address the following core question: Can systems science methods help peacebuilding practitioners and decision makers address violent conflict more effectively?

To help facilitate the research, we investigated the following three subquestions:

Is there evidence that systems science can help practitioners and decision makers better understand and anticipate the complex dynamics (for example, propagation of trust, violence, or corruption) that peacebuilding interventions seek to address?

Is there evidence that the application of systems science can help practitioners and decision makers identify leverage points for effecting change in peacebuilding interventions?

Is there evidence that the application of systems science can help practitioners and decision makers anticipate and manage unpredictability during the implementation of peacebuilding interventions?

Background

A complex adaptive system has diverse elements that interact with one another and change over time to generate system-level patterns that are often not uniform, linear, or intuitive.² The nature of complex adaptive systems presents challenges for traditional quantitative analytic techniques that (among other things) require strict assumptions about independence. Complex adaptive systems occur across theoretical, natural, and social sciences. They share general properties across these topic spaces, displaying both interdependence and adaptivity at a minimum and potentially heterogeneity. In this paper, for the sake of brevity, we use “system” to refer to a complex adaptive system and focus exclusively on how such systems manifest in social contexts (in other words, those that are concerned with human behavior and its impact).³

Interdependence occurs when phenomena of interest are driven by interpersonal interactions in which people affect others’ behaviors and outcomes. These interactions can be direct, such as coercion, persuasion, cooperation, compromise, harm, aid, or information sharing. They can also be indirect, with behavior affecting environments that then impact others. For example, members of a divided community can participate in rebuilding housing or common infrastructure, and as a result of these efforts, effect sustainable reconciliation between the rest of the community. Interdependence can take place at or between multiple levels of aggregation: individual, organizational, institutional, or state. For example, participation in an electoral system involves interdependence between individuals and institutions.

A social system demonstrates adaptation when people or their environments change over time. People may alter their behavior or decision-making strategies based on additional information, changes in their own circumstances, past experiences, observation of others’ behavior and outcomes, or a shifting landscape. For example, cooperation can be more likely to occur when it has been proven successful, so “getting the ball rolling” is a fundamental challenge in a scenario where productive cooperation has yet to occur. Similarly, the ways in which environments affect, or are affected by, people may change over time. For example, a changing climate might affect agricultural and water resources in ways that make the return of internally displaced people more difficult over time.

Finally, it may be important to consider heterogeneity within systems. Populations and contexts may differ in significant ways that shape the interdependent, adaptive mechanisms driving outcomes of interest. Heterogeneity can include variation in geography, social settings, or relevant properties of people and the environments in which they operate. For example, successfully reintegrating former combatants might be highly dependent on factors related not only to the combatants themselves but also the circumstances in which they are reintroduced.

Using tools from complex systems can allow for a deeper understanding of how social phenomena operate and why interventions may have effects that are unexpected or differ across contexts.⁴ Trying to anticipate patterns in a system with a high degree of complexity solely based on mental models (consisting of little more than intuition) can be limiting or misleading. For example, the design and implementation of “stop and frisk” policing in US cities was based on mental models of law enforcement and the criminal justice system: It was thought that increased detection and enforcement in high-crime areas (even of low-level crimes) would deter criminal activities, including those most dangerous and damaging. However, positive effects on crime rates ended up being more limited, localized, and fleeting than predicted, and there were unanticipated costs.⁵

Available quantitative evidence alone can also prove insufficient to understand how complex systems operate and interventions create effects. For instance, the “gold standard” in causal inference in many fields is employing a randomized controlled trial (RCT). But there are many situations in which doing so is unethical or simply infeasible due to prohibitive cost, risk, or danger, and so RCT data are not available. For example, we cannot and should not use an RCT to estimate the long-term mental health impacts of exposure to ethnic conflict. Even when RCTs are possible, extrapolating from their results to formulate actions can sometimes lead to disappointing outcomes. Interventions that appear promising when measured using a small-scale RCT may turn out to have muted—or even negative—effects when brought to larger scales as interdependent elements of the system respond adaptively over time: An RCT where a treatment group of a few hundred high school students receives large college tuition subsidies may indicate that this has a positive impact on attendance in selective colleges. However, if implemented universally, the effect might be washed out by the collective behavior of college applicants and colleges’ adjustments in admissions practices; if everyone receives this benefit, enrollment patterns do not change much.⁶ Similarly, interventions backed by RCT evidence might have entirely different effects when applied in contexts in which the underlying system is substantially different: An intervention that incentivizes voting might increase participation substantially in a US city but would do little when implemented somewhere where voting is mandatory (such as in Australia).

Using complex systems tools in concert with available data provides additional insight into what works, for whom, and why. This insight can be critical when considering how to best design and implement changes in policy and practice across settings. Effective approaches may be subtle, novel, or unconventional or may leverage hidden synergies; systems science tools can help elucidate these.

Systems Science Tools

We focused our evidence review on four systems science tools that can be used to gain valuable insight into the operation of systems and that are frequently (and successfully) used to provide concrete guidance for policy and practice. This set of tools, described below, is not exhaustive. Rather, it is intended to highlight several different approaches to systems science research.⁷ Our goal was not to provide best practices or “how to” primers for these approaches; there are large bodies of literature that accomplish this.⁸ The aim was to define terms to provide clarity while discussing the case studies we reviewed.

The application of these tools is not mutually exclusive with other research tools within or outside systems science. It is often beneficial to use estimates from traditional statistical analyses (for example, effect estimates from an RCT) to inform systems science models. Similarly, it is possible to combine them into “hybrid” models or to use multiple systems science tools for different functions in the context of a broader research effort.⁹

Following the tools’ general descriptions are case studies that illustrate how each tool, alone or in concert, has been successfully used to answer research questions with important practical implications.

SYSTEM DYNAMICS MODELS

System dynamics is a quantitative approach that involves fully specifying and characterizing the structure and dynamics of systems using mathematics. The approach has long been used to apply concepts familiar to engineers and physical scientists to the study of social behavior and organizations.¹⁰ Although system dynamics models can be presented purely mathematically as sets of variables and dynamic equations, they can also be presented graphically by creating a relatively user-friendly, top-down visualization of a system. This is typically achieved by creating a causal loop diagram (see the Appendix). Elements within system dynamics models include stocks, indicating the specific quantities of a specified resource or construct; flows, which describe how stocks accumulate or deplete over time; and feedback loops, which indicate how relationships within the system might change over time (for example, as more people participate in collaborative behavior, it becomes easier to entice others to join, creating positive or reinforcing feedback).¹¹ System dynamics models can operate at a single level (for example, at the individual level) or a multilevel (for example, characterizing both individuals and intraindividual characteristics such as levels of trust). It is important to note that the formal, quantitative analysis of a system dynamics model requires every element within the system to be fully specified: Stocks must have associated units (for example, the number of people infected with a disease), flows must have rates of change, and all flows and feedback loops must be defined with equations that incorporate all relevant variables.

NETWORK ANALYSIS

Network analysis is fundamentally focused on questions about connections between discrete units.¹² The first step in network analysis involves considering what network or networks are relevant for the questions being addressed. Nodes (alternately referred to in research as vertices or points) in the network can represent individuals, groups, nations, or other units of interest. The edges (alternatively referred to as arcs, lines, or ties) that connect nodes can similarly represent a wide variety of ways that nodes are related to one another (influence, authority, transactions, interactions, or kinship). Both nodes and edges can be assigned properties. For example, if nodes represent individuals, network analysis might incorporate properties such as demographics (gender, ethnicity, or age), attitudes (politically conservative or liberal), or roles (rank in an organization). Edges can take on properties that qualify relationships, such as frequency of interactions, amount of trust, or whether a relationship is between members of the same immediate family. Either network edges or specific edge properties can be characterized as directed, going from one node to the other, vice versa, or both (bidirectional).

Quantitative network analyses involve estimating network properties such as density (the proportion of possible connections that are actually present), average node distance (the mean number of minimum edges that must be traversed to connect each pair of nodes), or clustering (the frequency of transitivity such that if node A is connected to B and A is connected to C, B is also connected to C). Descriptions of network properties might be useful on their own. For example, simply knowing that individuals within one organization are sparsely connected to one another relative to other organizations might be meaningful based on existing research or theories. Longitudinal network data can be analyzed to determine how networks change over time in relevant ways. Analyses might involve estimating whether and in what ways network properties are related to treatment, outcomes of interest, or both. This can be done either correlationally with cross-sectional data or causally with appropriate longitudinal data. Finally, there are types of network analyses that can be employed to gain insight into causal mechanisms that might drive network changes. One of these, stochastic actor-based modeling, is closely related to agent-based modeling, described below.¹³

AGENT-BASED MODELING

Agent-based modeling (ABM) is a bottom-up systems research tool in which entities—as well as their interactions with one another and the environments in which they operate—are all explicitly represented throughout a dynamic computational simulation.¹⁴ Agents in a model can represent a single operational level (for example, individuals) or multiple levels (for example, employees and organizations). Similarly, interactions can occur between agents of the same type, agents of different types, or agents and their environments. ABMs are very flexible with regards to heterogeneity (agent properties, behavioral strategy, and adaptation over time). They are capable of representing a wide variety of environments, including ones that capture geospatial features or social structures. Finally, ABMs can represent different levels of time granularity (for example, simulating activities that occur every hour or aggregating them into days) and time spans (for example, a simulation run might depict one year or a decade).

As opposed to system dynamics models, in which system-level relationships are clear from the design phase onward, ABMs allow for system-level patterns to emerge from individual-level behavior and interaction over time. However, the trade-off is that individual-level attributes and behaviors must all be fully specified in these models, along with the environments in which they take place.

GROUP MODEL BUILDING

Group model building (GMB) is a systems science approach that is inherently and intentionally participatory.¹⁵ As such, it is intended to (1) answer questions about the operation of systems and the potential impact of interventions and (2) develop capacity for systems thinking; build communication, trust, and consensus among diverse stakeholders involved in the process; and create buy-in, enthusiasm, and empowerment for action plans that emerge as a result of GMB.¹⁶

Typically, GMB engages participants in a set of group activities led by one or more facilitators. Facilitators select exercises ahead of time based on the research goals, create scripts to guide implementation of the exercises, and develop clear processes for observing and recording participants’ engagement.¹⁷ A wide variety of activities can be used, but generally researchers will guide participants through articulating a clear problem or problems (a goal that the group believes can be advanced through better understanding a system); formulating hypotheses about the structure and dynamics of the relevant system; collectively producing a plan to address problems, taking into account conjectured (or, potentially, observed or tested) system structure and dynamics; and establishing metrics or assessment tools to determine success.

In addition to fostering action—informed by systems thinking and input from stakeholders with direct knowledge about different areas of the relevant system—GMB can produce data for other systems science tool applications. Most frequently, GMB is paired with system dynamics;¹⁸ as participants engage in exercises to articulate how elements of a system are dynamically linked, they create and iteratively improve upon the causal loop diagram (see the Appendix). Pathways within this diagram can be quantified, potentially through the GMB process, and then used as the basis for the design of a system dynamics model (as described above).¹⁹ Similarly, participants might create a connection circle that describes relationships within networks salient to the system and problem, and this can be used for further network analysis.²⁰ Products created during GMB to describe the structure and operation of a system can also be used in the design and refinement of ABM.²¹

Case Studies Part 1: Systems Science Exemplars in Related Fields

The application of systems science tools to guide peacebuilding is in its infancy. However, the tools have been used extensively for decades to explore other social science and public policy topics. While the below five case studies do not directly pertain to peacebuilding, the examination of systems science models that similarly address dynamic and adaptive social systems can prove useful when considering the development of models that inform peacebuilding efforts: Violent conflict is the result of interactions between organizations and individuals that are affected by resources, social structures, perceptions, incentives, beliefs, and decision-making strategies (among other things). Components of systems science models can be repurposed as needed to avoid redundant efforts. For example, characterizations of social influence can be transposed—albeit not without due consideration of underlying assumptions—from one context to another. Beyond revealing direct model components that can be productively repurposed, the case studies demonstrate how practitioners in related fields have approached the process of modeling social systems. For example, some have convened a collaborative team with appropriate methodological, content, and contextual expertise; designed a model that is “as simple as possible, but not simpler” to address specified research goals;²² utilized appropriate data, potentially from multiple sources; engaged in testing to determine a model’s explanatory power or sensitivity to assumptions; and analyzed model output to provide insight into system etiology or potential intervention effects.

We selected each case study either because we were directly involved with the research or because the research resulted in highly visible, impactful models and applications, or both. This selection approach allowed us to more easily identify the costs and benefits associated with systems science tools across a variety of topics and research goals. For each case study, we summarize the research context, methodology, main findings, practical impact, research costs, and ways in which elements of the case study might be directly applicable to peacebuilding research. We then explore the implications of these case studies, taken together, for each of our three evidence-review subquestions.

SELECTED CASE STUDY 1: AGENT-BASED MODELS OF COLLEGE ENROLLMENT

Overview. ABM was used to explore how changes to policy might affect equitable access to higher education.²³

Context. The US Supreme Court recently considered whether and under what circumstances race-based affirmative action is allowable in college admissions decisions.²⁴ Although the court acknowledged racial diversity as a legitimate goal for public universities, the controlling decision shows clear discomfort with, and uncertainty about, race-based affirmative action as a means to attain such diversity. The court stated that the university has the “ultimate burden of demonstrating, before turning to racial classifications, that available, workable race-neutral alternatives do not suffice” and that “[t]he University … does have a continuing obligation to … [tailor] its approach in light of changing circumstances, ensuring that race plays no greater role than is necessary to meet its compelling interest [in racial diversity].”²⁵ This presented researchers with an opportunity to address a policy-relevant question with wide-ranging societal ramifications: Is it possible to craft race-neutral admissions strategies that generate substantial levels of racial diversity in college enrollment?

Methodology. Because it is infeasible and unethical to approach this question experimentally using real-world colleges and students (for example, by conducting an RCT), the researchers created an ABM for the college enrollment process. This model captured quintessential elements of the system: heterogeneity across colleges and prospective students; bi-directional decision-making, with students deciding where to apply, colleges deciding whom to admit, and admitted students deciding where to enroll; and adaptation over time as colleges and students dynamically adjust to changes in the higher education landscape (for example, changes in the number of students to be admitted in a given year in order to fill available spots). After developing and testing the ABM, it was used experimentally to explore whether race-neutral alternatives that replace race-based affirmative action admissions policies with socioeconomic-based affirmative action and race-based recruitment could deliver levels of diversity observed when selective colleges use race-based affirmative action.

Summary of findings. Combinations of high-magnitude, race-neutral alternatives are required in order to achieve observed levels of racial diversity in college enrollees. However, the presence and strength of such approaches are likely either impractical or prohibitively expensive for most, if not all, colleges to engage in.

Practical impact. The research findings were included as supporting evidence in a brief to the Supreme Court, and it is possible that these findings will play a similar role as the court once again turns its attention to affirmative action in higher education.²⁶ Similarly, an updated version of the enrollment model was used to adjudicate claims made by the 2020 presidential candidates about the impact of vastly expanding college subsidization programs.²⁷

Research costs. The researchers built upon a previously developed ABM of college enrollment.²⁸ Completion of the two ABM phases took over five years. To satisfactorily address research questions about actionable admissions strategies, the ABM drew on multiple sources of high-quality data and theory: nationally representative data on colleges and applicants, theory about and observation of student and college decision-making processes, and available admissions strategies to consider. Thousands of simulation runs were conducted, resulting in large model output datasets (>100GB) to synthesize during analyses.

Potential applications for peacebuilding research. These models tie together issues of equity, racial segregation, access to scarce resources, intergenerational mobility and replication of status, public policy, and individual action.

SELECTED CASE STUDY 2: AGENT-BASED MODELS OF RESIDENTIAL SEGREGATION

Overview. ABM was used to understand potential drivers of persistent patterns in residential segregation.²⁹

Context. As the American Civil Rights movement progressed during the middle of the twentieth century, de jure barriers to racial integration—related to how people lived, attended school, worked, and socialized—steadily eroded. However, activists, policymakers, and social scientists noted that the attendant impact on de facto segregation was noticeably muted. A number of hypotheses were given as to why this was so, ranging from vestiges of formal racial segregation being slow to disappear in practice to the pernicious presence of widespread racial animus preventing true integration. In one of the earliest ABM applications in social science, Thomas Schelling engaged in a computationally aided “thought experiment” to determine whether residential segregation must necessarily stem from strong formal barriers or racial intolerance.³⁰ Since his research on the subject was first published, social scientists have built upon it to answer this thorny etiological question and provide insight into strategies for sustainable racial integration.³¹

Methodology. Schelling’s initial model of residential segregation was elegantly simple in design: equal numbers of two different types of simulated households were placed at random onto grid squares. Each household observed its immediate “neighbors” (in other words, the surrounding eight squares), and if a greater proportion of the neighbors were of a different type than they preferred, then they would move to a randomly selected empty square.³² Subsequent iterations of the model included increasingly sophisticated elements, such as additional household types, heterogenous segregation preferences, realistic geography, and additional housing market factors.³³

Summary of findings. Within the simulation, relatively moderate individual-level preferences for similar neighbors (at least 30 percent) can result in high levels of residential segregation.³⁴ This pattern is less pronounced but still observable when additional complexity is incorporated in the models.³⁵ Taken together, findings from computational models are remarkably consistent with quantitative and qualitative data that have subsequently been collected.³⁶

Practical impact. These models have provided guidance to policymakers weighing alternative approaches to increase integration in their metropolitan areas; there is still much more that ABM can do to inform policies to reduce segregation and its pernicious effects. Similar models have also been used to examine the causes behind, and potential solutions to, racial segregation in public schools, even in settings where school choice policies putatively decouple attendance from residence.³⁷ Specifically, a computational model of student assignment was used to narrowly defeat a proposal under consideration by the school board of the San Francisco Unified School District; the proposal would have increased priority given to residence near schools, which would have allowed residential segregation patterns to be additionally reflected in public school attendance.³⁸

Research costs. Over the course of 50 years, the evolution of ABM from simple “thought experiments” carried out by hand using a game board and pieces to much more sophisticated experiments demonstrates the cost trade-offs in systems science approaches in general and ABM in particular. A very simple model does not require much data, development time, or computational resources. However, it does require making a much larger number of implicit assumptions than a more complex model. For example, the simple model of residential segregation assumes that there are no constraints to making a household move and no purposeful decision-making involved when choosing where to go.³⁹ Such a model can be immensely useful for building and testing theory, as well as for guiding future data collection and modeling efforts.⁴⁰

In contrast, a highly sophisticated model of residential segregation can engage with more empirically grounded assumptions, but it requires more data, some or all of which might need to be collected specifically for the research project. Development time also increases with additional data inputs and specified dynamics. Finally, although computational resources are more readily available today, computing time and storage space can still be prohibitive given that large numbers of simulated entities are making multifactor, strategic decisions.⁴¹ At the same time, these more sophisticated “next generation” models can be useful for retrospectively explaining the underlying causes of observed phenomena or for prospectively selecting approaches with a high probability of success and sustainable favorable change.⁴² In practice, systems science models across the continuum from simple to complex are fruitfully employed.

Potential applications for peacebuilding research. Understanding the causes behind, and possible strategies for, reducing residential segregation across racial or ethnic lines is relevant for the study of conflict, as segregation is associated with animosity or competition.⁴³

SELECTED CASE STUDY 3: AGENT-BASED MODELS OF THE ANCESTRAL PUEBLO

Overview. ABM was used to understand factors leading to historical depopulation in the American Southwest.⁴⁴

Context. The ancestors of the modern Pueblo people lived in the American Southwest for thousands of years. Because of this long duration and favorable environmental conditions, an abundance of archaeological evidence attesting to their presence has been documented. However, archaeological evidence tends to be purely static and descriptive. It can answer the what, where, and when questions quite well, but the how and why questions that researchers often care about most remain tantalizingly out of reach. To address this gap, interdisciplinary teams of researchers have used increasingly sophisticated ABMs to explore the causes of settlement and civilization formation, agricultural behavior, internecine conflict, and migration.

Methodology. From inception, the ABM was highly sophisticated, incorporating dynamic representations of multiple levels within the socioenvironmental system. The models captured salient aspects of human physiology, such as consumption, starvation, and reproduction; social structures and behaviors, such as households and communities; geography, such as hydrology and soil attributes; ecology, such as agriculture and the presence of plant resources and nonhuman animals; and climate, such as rainfall and temperatures. These models were designed to be large in scale, in terms of size of the geographic area, number of individuals, and the timescales used in simulations. The models were capable of generating output with a high degree of spatiotemporal detail, which could help test hypotheses that observable archaeological evidence alone could not address.

Summary of findings. A comparison of the models’ output to archaeological data indicates that the models have a high degree of explanatory power. Collectively, these models provide insight into the formation of settlements, changes and advances in agricultural practices, the emergence of social and economic specialization, the emergence of political hierarchy, conflict between groups, and migration. Taken together, the models suggest that both the violence experience by the Ancestral Pueblo people and their exodus en masse played key roles in the complete depopulation of this region of the American Southwest in the thirteenth century; hypotheses that changes in climate or ecology alone could have generated this outcome are not consistent with the models’ findings.⁴⁵

Practical impacts. Given that this research focused on the distant past, the models’ findings have had few direct, practical impacts on the present. However, field archaeologists and anthropologists have used them to guide their data collection, interpret physical evidence, and increasingly draw lessons from humanity’s past to help address looming challenges, especially climate change.

Research costs. Although the relatively large models did not incorporate highly sophisticated decision-making like the ones used to simulate college application behavior, they nonetheless required fairly high levels of computational resources. A vast amount of data was needed to satisfactorily characterize dynamics, agents, and environments in these models. For example, to incorporate crop yields for the relevant times and places into the models, researchers had to draw from multiple extant data sources.⁴⁶ Determining what data were required, as well as how to best synthesize it, required a highly interdisciplinary team. The researchers involved had expertise in archaeology, cultural anthropology, demography, physiology, geology, ecology, and complex systems.

Potential applications for peacebuilding research. These models were, by design, historical in nature. As such, they might not seem relevant for guiding peacebuilding efforts. However, some of the models’ core elements could be useful, including resource management, climate change response, migration, cooperation, and conflict. Thus, these models could help with forward-looking peacebuilding research that addresses concerns such as violence and displacement.

SELECTED CASE STUDY 4: SYSTEM DYNAMICS RETHINK MODEL

Overview. System dynamics was used to create a tool that can guide positive change in health systems.⁴⁷

Context. The United States has long experienced higher health care costs and underwhelming outcomes relative to other developed countries.⁴⁸ This trend has persisted despite large-scale investment of political capital and resources to transform how health care is delivered, such as through the Affordable Care Act signed in 2010.⁴⁹ Although much is known about linkages among individual-level needs, care, personnel capacity, and costs in public health, limited attention has been paid to how these relationships dynamically interact, especially on a large scale and over a long time frame.⁵⁰ A better understanding of these dynamics could help identify root causes of both the negative health outcomes overall and the persistent disparities across racial and socioeconomic groups, as well as lead to actionable solutions.

Methodology. The ReThink model uses system dynamics to operationalize the factors that interact over time and drive health care outcomes, resource capacities, and costs. The model is designed to take a large-scale, long-term, and highly comprehensive perspective.⁵¹ It is intended to solicit input and participation from stakeholders, whom the modeling team call stewards and who have vested interest, knowledge, and influence within local and regional health care systems.⁵² These stewards identify and prioritize system needs and goals, inform the characterization of a model that represents the contexts in which they operate, suggest feasible and high-potential interventions to explore via computational simulation, and shepherd the real-world implementation of interventions that are selected based on analyses of model findings.⁵³

Summary of findings. In addition to specific local and regional recommendations, the ReThink model provided insight into ways that the overall US health care system might be strategically adjusted to increase efficient and equitable delivery of outcomes over the next two decades (through 2040).⁵⁴

Practical impacts. The ReThink system dynamics model, described by its design team as a “health system in a computer,” has been used to represent a number of specific local and regional contexts and potential strategies.⁵⁵ Findings from these model applications have been used to guide real-world improvements in health policies and practices.

Research costs. There are two major cost categories associated with the ReThink model: data and personnel. Substantial amounts of data are required to characterize relevant model elements on a large scale and over a long time frame.⁵⁶ Not only is the initial outlay of data required by the model large, but this expense is recurring; each application of the model to a new context necessitates the procurement and analysis of appropriate data. One strong element of the ReThink model is its explicit incorporation of individual and organizational stakeholders as stewards. This ensures that the model reflects salient aspects of the specific context, that experimentation reflects meaningful intervention options, and, perhaps most importantly, that recommendations have a high likelihood of being acted upon. However, it takes a significant amount of time and effort to identify, recruit, train, and empower stewards, many of whom might be unfamiliar with the ReThink model or systems science more broadly.⁵⁷

Potential applications for peacebuilding research. This is an example of applying systems science with the express input and engagement of relevant stakeholders. The model is highly extensible—capable of being used across contexts to explore a number of potential intervention options and outputs of interests—and can be used over a long time period to capture delayed impact. A similar approach—which solicits sustained feedback from key individuals or organizations—could be used to develop and apply systems science in peacebuilding research.

SELECTED CASE STUDY 5: WHOLE-OF-COMMUNITY CHILDHOOD OBESITY PREVENTION EFFORTS

Overview. ABM, GMB, and network analysis are being used in combination to help community stakeholders design and implement effective childhood obesity prevention efforts.⁵⁸

Context. Childhood obesity is a persistent problem in many nations, including the United States. It imposes long-term individual health burdens and societal costs. Because of its greater prevalence among low-income and minority populations, it also is a large contributor to health disparities. These problems have become particularly apparent during the COVID-19 pandemic. However, whole-of-community interventions have had some success in addressing childhood obesity at the local level.⁵⁹ These efforts intentionally cultivate the active participation of diverse community stakeholders, who leverage their knowledge of community needs and resources to design and implement changes in policy and practice.⁶⁰ To replicate this success, it is necessary to understand not only what has worked, but also why and how positive impact has been achieved.

Methodology. The Childhood Obesity Modeling for Prevention and Community Transformation (COMPACT) project is an ongoing, iterative collaboration between childhood obesity prevention intervention experts and systems science researchers to (1) evaluate and explain the success of several completed whole-of-community childhood obesity prevention efforts and (2) use the findings to help conceive and implement childhood obesity prevention in new communities. Whole-of-community childhood obesity interventions are inherently predicated on systems thinking and are therefore well suited for the application of systems science tools. COMPACT is designed to deploy and refine an interconnected suite of systems science tools over time. Data are collected using custom-built survey instruments as well as through GMB.⁶¹ Next, a social network analysis of the relevant data is conducted to understand the social structures connecting community stakeholders who directly or indirectly influence children’s health outcomes.⁶² And ABM is used both to help shape intervention design and gauge intervention impact.⁶³

Summary of findings. Data have been used as input into an ABM that captures how an intentionally convened “steering committee” of community stakeholders can build on their knowledge about and engagement with childhood obesity prevention in their community and also create conditions across community sectors (for example, childcare, school nutrition) favorable for positive, sustained change. Finally, lessons learned from these analyses have been used to continually improve intervention design for future use. This includes a suite of GMB activities that steering committee members engage in as they plan and implement changes to policies and practice.⁶⁴

Practical impacts. COMPACT has generated survey tools to measure community contexts that have been tested and iteratively improved, completed social network analyses to quantify relevant social settings, used ABM to understand the mechanisms that drive community readiness to change, and produced GMB scripts to facilitate participant-led positive change. These systems science tools have been and will continue to be used to design and implement whole-of-community interventions with positive and sustainable impacts on children’s health.⁶⁵

Research costs. For five years, this project has brought together intervention and systems science experts from around the world to support concentrated research activity. The work is primarily funded through a multimillion-dollar award from the US National Institutes of Health. At least two dozen individuals across five institutions have participated. Resource-heavy endeavors have included survey design, data gathering, primary data analyses, ABM, and prospective intervention design and implementation.

Potential applications for peacebuilding research. High-quality, intentional, and iterative participatory systems science is the core of this project. Although childhood health might be only tangentially related to peacebuilding, the project’s tools and processes can be productively applied in this field and others. It is not hard to imagine how findings from a preliminary evaluation of social networks, coupled with knowledge related to and engagement with conflict resolution at the community level, might be useful. Similarly, ABM to guide the effective implementation of a whole-of-community conflict prevention initiative—including GMB activities with diverse community stakeholders—could prove fruitful.

CASE STUDIES PART 1: SYNTHESIS OF FINDINGS

Evidence for Understanding Systems

Research subquestion: Is there evidence that systems science can help practitioners and decision makers better understand and anticipate the complex dynamics (for example, propagation of trust, violence, or corruption) that peacebuilding interventions seek to address?

Each of the five case studies provides a concrete example of researchers applying systems science tools to explicate the underlying etiology of systems. In each case, the endeavor was the first of its kind, and practitioners in relevant fields gleaned demonstrable benefits from the insights provided.

Case study 1 highlights the first successful attempt to quantify causal pathways that dynamically operate in concert to generate persistent racial and socioeconomic patterns in higher education. And case study 2 includes one of the first demonstrations of the power of systems science to elucidate how microlevel behavior can have macrolevel effects. Individuals’ actions can affect others, who in turn respond; and the system-level outcomes that eventually emerge seem counterintuitive when only considering individual-level perceptions, motivations, and behaviors.

Case study 3 is an excellent example of how systems science tools can help quantify interactions between multiple levels of a system over a long period of time. Because what causal mechanisms generated observed patterns in the archaeological record was presupposed or much debated, the use of ABM was crucial in providing scientific insight.

Case study 4 shows how system dynamics can meaningfully represent the large array of factors that interact with one another over a long time span and that drive key outcomes of the health care system in a local, regional, or national context.

Finally, case study 5 includes the formulation and subsequent testing of a specific hypothesis for how individual health outcomes result from “upstream” activity in the system. Researchers refer to this hypothesis as “stakeholder-driven community diffusion.”⁶⁶ The hypothesis is that a steering committee of diverse community stakeholders can act to increase knowledge and engagement through their social networks over time. This is a necessary precursor to fostering sustainable and context-appropriate “midstream” changes in childcare, health care, and local government policies and practices that effect the intended, positive health outcomes.

None of these case studies included an application of systems science tools to assess outcomes directly related to current or future peacebuilding efforts. However, they all show how these tools could represent social behavior in ways that provide peacebuilding researchers and practitioners with a greater understanding of systems in conflict settings.

Evidence for Identifying Leverage Points

Research subquestion: Is there evidence that the application of systems science can help practitioners and decision makers identify leverage points for effecting change in peacebuilding interventions?

Each of these selected case studies illustrates how researchers can use systems science tools to identify leverage points through which feasible interventions can have maximal impact on outcomes of interest. These insights can be roughly divided into two types: prospective and retrospective. In the first, salient aspects of a current (or recent) system are represented and then simulations run from that starting point under different counterfactual conditions to reveal which, if any, results in desired impact. In the second, an already completed historical event is simulated and then compared to alternative conditions to reveal what set of actions or circumstances might have yielded a preferable outcome; these findings are then used to inform the formulation of future strategies.⁶⁷

The underlying, iteratively developed model in case study 1 was used to address three broad categories of policy questions; the goal was to identify effective and sustainable interventions to make the higher education landscape more equitable. The first category focused on socioeconomic stratification related to who attends college and where.⁶⁸ A sample finding is that addressing disparities in information quality about the college application process might be a feasible way to reduce inequitable enrollment, but the effort is unlikely to completely level the playing field. The second category focused on different approaches (for example, “race-neutral” alternatives to race-based affirmative action) to addressing inequitable access patterns across students of different races. The third focused on a more nascent application to consider how best to structure college subsidy programs (for example, “free college”) for maximal impact.⁶⁹ Similarly, the set of models described in case study 2 were used to identify promising strategies for counteracting seeming intractable, large-scale negative patterns such as racial segregation.

In case study 3, retrospective models of historical events from the distant past were used to answer interesting—and potentially important—questions about counterfactual scenarios. For instance, these models could provide clues as to whether and in what ways violent interactions between groups might have been avoided had sociocultural or ecological circumstances been different. The clues might then provide lessons that can be applied (with caution, given differences in context) to modern situations.

Conversely, case study 4 highlights a prospective approach to identifying leverage points. That is, given limitations imposed by system structure, the ReThink model was used to reveal feasible ways in which health care systems can operate more equitably and effectively.

Finally, case study 5 shows how the use of multiple, overlapping systems science tools can reveal distinct but related leverage points. First, ABM provided key guidance for how knowledge and engagement can be effectively generated throughout a community (for example, via the optimal composition of a steering committee). This information flow was both more precise and less costly to obtain than it would have been in applying other means, such as RCTs.⁷⁰ Second, ABM was deployed prospectively to guide the formation of the Shape Up Under 5 steering committee.⁷¹ This committee engaged in guided GMB activities designed to facilitate the planning and implementation of effective, systems-based, midstream solutions. This resulted in the Eat Sleep Play campaign in the Somerville, Massachusetts, community, with initial reports indicating cause for optimism.⁷² Based on lessons learned and the substantial investment of time and effort, many other communities are now using the tools and processes that the COMPACT project developed.⁷³

In large part, the ability of systems science tools to identify leverage points through which positive impact can be achieved comes from their ability to consider large numbers of counterfactual scenarios. With guidance from experts and stakeholders, this capability can be directed toward crafting optimal strategies.

Evidence for Handling Uncertainty

Research subquestion: Is there evidence that the application of systems science can help practitioners and decision makers anticipate and manage unpredictability in the implementation of peacebuilding interventions?

Each case study demonstrates how systems science can help manage uncertainty. Broadly speaking, systems science tools can prove useful in two ways. The first is through extrapolation to larger scales of time, geographic, or social space. That is, a pilot intervention might show short- or medium-term positive effects that ultimately fade out or are reversed when the intervention is replicated at scale. Systems science tools can help anticipate long-term outcomes before resources and effort are invested in action. Second, as with any model that abstracts human behavior in a computational framework, assumptions can and should be subjected to sensitivity analyses. By following this best practice, systems science tools can provide guidance on whether and how to engage in additional data collection activities before taking action.

The model used in case study 1 is designed to explore long-term trends in enrollment that might result from changes in both applicant and college behavior. Thus, the model is able to provide insight into delayed effects, countertrends, or cyclical patterns. Repeated simulation runs provide estimates of uncertainty in outcomes of interest such as enrollment of underrepresented minorities in elite schools after a change in policy. Finally, assumptions used to characterize these functions are subjected to extensive sensitivity analyses, thereby providing additional information about the robustness of findings.

The models used in case study 2 can be run on large time scales as well as across fairly large geographic settings (although doing so increases the data requirements and computational costs). Thus, the models can not only capture interventions that might have gradually decreasing impacts or even unintended negative consequences, but also potentially provide insight into the possibility of “spillover” situations (for example, when reducing residential segregation in one community comes at the expense of increasing it elsewhere). As with other ABMs that incorporate stochastic elements, repeated simulation runs provide estimates of uncertainty.

The extremely retrospective historical models included in case study 3 were intentionally designed to observe trends over hundreds of years. As such, they can be used to think about path dependence and unintended consequences. For example, these models could shed light on whether a ninth-century decision to change agricultural practices, which might have made sense in the short term, had negative long-term ramifications. In addition, extensive use of these models has helped determine what additional data are needed and where to look for it.

The ReThink model described in case study 4 was used to capture a health care setting where there is not much current uncertainty about factors such as costs and capacity. However, the model can operate over long time periods, allowing uncertainty to be productively explored (for example, how technology advances might enable more efficient health care delivery).

Finally, the ABM described in case study 5 can provide prospective guidance by analyzing a relatively small sample of initial data. By collecting data on the community context from a small set of individuals—such as steering committee members and those people they frequently interact with—as well as input from community experts, the model can estimate potential long-term changes that might result from the intervention. Then, through repeated model applications, these estimates can reveal the distribution of possible outcomes, which helps quantify uncertainty.⁷⁴

Case Studies Part 2: Direct Applications of Systems Science Research to Peacebuilding

This part of the evidence review aimed to identify any research that was at least somewhat successful in applying systems science methodology in settings directly related to peacebuilding. We explored the very nascent body of literature at the intersection of peacebuilding and systems science in order to gauge the “lay of the land,” address our primary evidence review question and subquestions, and provide a cache of initial literature that may be useful for future systems science attempts. (See the Appendix for details on the literature review methodology.)

Table 1. Summary of Selected Case Studies

Title

Processes or Outcomes of Interest

Context and Data

Systems Science Tools Used

Evidence for Understanding Systems

Evidence for Identifying Leverage Points

Evidence for Handling Uncertainty

Social Network Analysis and Counterinsurgency Operations: The Capture of Saddam Hussein⁷⁵

Describing a covert network.

Individuals and their connections to one another were identified and qualified (for example, kinship or organizational relationships, level of trust) using information gathered during the search for Saddam Hussein by American forces in 2003.

Social Network Analysis

Limited: A detailed social structure can be identified and analyzed through the systems science methods used here, but not an understanding of how this structure forms, changes, and operates.

None

None

Social Network Analysis and Small Group “Dark” Networks: An Analysis of the London Bombers and the Problem of ‘Fuzzy” Boundaries⁷⁶

Describing a covert network.

Publicly available data were used to characterize the social networks of individuals related to two 2005 bombings in London.

Social Network Analysis

Limited: These data are insufficient to satisfactorily describe the relevant social networks, let alone processes that drive events such as terrorist bombings.

None

None

An Agent-based Model of Ethnocentrism and the Unintended Consequences of Violence⁷⁷

The dynamic relationships between ethnocentrism, violence, and cooperative behavior.

This is an extension of an existing ABM of ethnocentrism.⁷⁸ The model is highly stylized—intended to explore broad hypotheses—rather than grounded in any real-world context.

Agent-based Modeling

Limited: Although the processes are fully explicated at the individual level, because the model is not grounded in data, its ability to help understand real-world systems is limited.

Limited: Using a highly stylized model can suggest broad conditions and courses of action that might increase cooperation and reduce ethnocentrism.

None

Social Network Analysis to Characterize Women Victims of Violence⁷⁹

Inferring factors related to patterns of violence against women.

Data from Emergency Department visits in the Lazio region of Italy were used to create a network linking repeated visits by women.

Social Network Analysis

Moderate: Through social network analysis, specific factors related to repeated or escalating violence against women were identified.

Moderate: This case study proposes that the findings be used to create screening tools that identify patients who are experiencing violence and then direct them toward intervention.

Limited: The extent to which patterns of violence are not explainable with available data and models is characterized. In conjunction with the findings presented here, this suggests the need for additional data.

Democracy, War Effort, and the Systemic Democratic Peace⁸⁰

The influence of democratic and autocratic governance patterns on international conflict.

The model is highly stylized—intended to explore broad hypotheses—rather than grounded in any real-world context.

Agent-based Modeling

Moderate: Processes are fully explicated at the national level, and model findings are broadly consistent with empirical data.

None

None

Conquest and Regime Change: An Evolutionary Model of the Spread of Democracy and Peace⁸¹

How interactions between and political dynamics within nations dynamically affect democratic governance and peace.

The model is highly stylized—intended to explore broad hypotheses—rather than grounded in any real-world context.

Agent-based Modeling

Limited: Plausible dynamics through which peaceful transition toward democracy might occur are explicated and explored, but because the model is not grounded in data, its ability to help understand real-world systems is limited.

None

None

Trust and Treason: Social Network Structure as a Source of Flexibility in Peace Negotiations⁸²

The relationship between rebel groups’ social network structures and whether negotiated peace efforts are successful.

Rebels’ social networks were characterized in terms of information sharing, verification, and mutual influence between rebel negotiators and nonnegotiating rebel leaders. The social network structure was compared to the outcome of negotiations between the Liberation Tigers of Tamil Eelam and the Sri Lankan government and between and Gerakan Aceh Merdeka and the Indonesian government.

Social Network Analysis

Moderate: This case study implicitly explores the processes through which individuals, operating within specific social networks, might affect peace efforts.

Limited: The study provides suggestive evidence that specific conditions (in other words, sufficient trust within rebel social networks) are conducive to the success of peace efforts. To the extent that it is possible to affect this change intentionally, this represents a possible leverage point.

None

The Social Life of the Women, Peace and Security Agenda: A Digital Social Network Analysis⁸³

Describing a social network of individuals working on specific peacebuilding topics.

A social network is described using online interactions (on Twitter and Facebook) among individuals working on policy and practice related to the United Nations Security Council Women, Peace and Security (WPS) agenda.

Social Network Analysis

Limited: The analysis indicated that the network included more individuals from the Global North and, based on this, suggested recruitment outside of this population. However, the analysis does little to ensure that the “true” network of WPS work is observed, nor does it capture salient aspects of the network such as trust or influence.

None

None

Spatializing Social Networks: Using Social Network Analysis to Investigate Geographies of Gang Rivalry, Territoriality, and Violence in Los Angeles⁸⁴

Describing a social network of gang rivalries and geography.

Data on gang rivalries and “turf” in a high-violence area of Los Angeles are used to create a combined network of gang rivalry and proximity. This network is then compared to incidences of gang violence.

Social Network Analysis

Limited: The innovative network analysis that combined social and geographic space suggests but does not explicate the behavioral dynamics that generate gang violence.

None

None

Communication Network Analysis to Advance Mapping “Sport for Development and Peace” Complexity: Cohesion and Leadership⁸⁵

Describing a social network of organizations working on a specific peacebuilding topic.

The social network is constructed from data collected through a web-crawler that identifies hyperlinks between the websites of organizations involved in “sport for development and peace.”

Social Network Analysis

Limited: The analysis shows some potentially interesting aspects of the network (for example, low density and overrepresentation of the Global North). However, neither the “true” operational network nor the processes through which it is formed over time are explicated.

None

None

Health Service Resilience in Yobe State, Nigeria in the Context of the Boko Haram Insurgency: A Systems Dynamics Analysis Using Group Model Building⁸⁶

Understanding the causes of and potential solutions to disruption of health services in the Yobe state of Nigeria due to the Boko Haram insurgency.

Data were collected from stakeholder interviews and group model-building activities.

Group Model Building

Strong: A causal loop diagram of the system driving health service disruptions, with “face validity” from a stakeholder perspective, was identified.

Moderate: The causal loop diagram, in conjunction with other group model-building activities, suggests several feasible and high-potential approaches for curbing health system disruptions.

None

Iruba: An Agent-based Model of the Guerrilla War Process⁸⁷

Exploring the context-dependent processes through which insurgencies can be effectively checked by the regime’s response.

A highly stylized model of regional-level guerrilla warfare was created, with dynamics loosely based on data from several real-world insurgencies.

Agent-based Modeling

Limited: Although processes are explicated, the model is not grounded in real-world data and thus has limited ability to explore specific guerrilla warfare settings.

None

None

A Systems Approach to the Sustainability–Peace Nexus⁸⁸

Dynamic processes underlying the sustainability of peace.

A highly stylized, data-free system dynamics model was created to represent a potential system driving the sustainability of peace.

System Dynamics

None

None

None

Understanding Peacebuilding Coordination and Impact Using a Complex Adaptive Systems Method⁸⁹

Dynamics driving the impacts of coordinated peacebuilding efforts.

Data were collected from interviews and focus groups conducted with stakeholders involved with the Kosovo peacebuilding process.

System Dynamics

Strong: Analyses of qualitative data resulted in a complete map of the system that drives the impact of coordinated peacebuilding efforts. Although the model is not fully quantified, links between model elements were assigned relative strengths.

Moderate: Based on the partially quantified system maps, specific leverage points were identified where action could improve the impact of coordinated peacebuilding efforts.

None

The Analysis of Indonesia’s Important Role to Keep Peace in Afghanistan Using a System Dynamic Concept Approach⁹⁰

Understanding the ways in which Indonesia affects peacebuilding in Afghanistan.

Review of relevant literature.

System Dynamics

Limited: A system dynamic model was created based on a review of relevant literature. However, this model has not been tested against real-world data.

None

None

Mapping Networks of Terrorist Cells⁹¹

Description of the social network of a terrorist cell.

A social network is constructed from archival data about the social closeness (for example, living or traveling together) of the 9/11 hijackers.

Social Network Analysis

Limited: The analysis shows some potentially interesting aspects of the network (for example, its overall density made it likely to be highly resilient) as well as identifies central figures in the network (the pilots). However, given that the processes underlying the formation and operation of this network (for example, the coordination activities) are not explicated, the model’s application to further understand the system that resulted in the attack and its application to additional settings ahead of time is limited.

None

None

A Social Network Analysis of Jemaah Islamiyah: The Applications to Counterterrorism and Intelligence⁹²

Description of the social network of a terrorist cell.

A social network was constructed from interaction frequency data gained from observation of the terrorist cell responsible for the 2002 bombing in Bali.

Social Network Analysis

Limited: The analysis shows some potentially interesting aspects of the network (for example, its overall moderate density balanced the competing goals of efficiency and covertness). However, the processes underlying the formation and operation of this network are not explicated.

None

None

Suicide Bomber Mobilization and Kin and Peer Ties⁹³

Whether kin and peer ties are related to willingness to become a suicide bomber (in other words, inferred social influence).

Individuals, network ties, and suicide bomber willingness were identified using documents obtained from an Islamic State informant.

Social Network Analysis

Limited: The social network structure is at least partially explicated along ties of interest and is estimated to be related to the outcome of interest. However, the processes that drive the outcome of interest are not explained.

None

None

Each publication, or case study, that we identified discusses at least one application of systems science tools to a peacebuilding topic. We omitted those that discuss the potential of systems science tools to address a topic or only propose ways in which researchers might use the tools. This is because our goal was to meaningfully assess whether, to what extent, and in which ways systems science research can make a practical contribution to peacebuilding efforts. Table 1 lists the selected case studies; their basic descriptions; the systems science tools used; and the evidence they contribute, divided into three categories (understanding systems, identifying leverage points, and handling uncertainty). This summary allows researchers to easily see which tools were used and how frequently they were employed for different purposes.

Synthesis of Evidence

MAIN FINDINGS

Based on this evidence review, we believe that systems science can help peacebuilding practitioners and decision makers address violent conflict more effectively. Yet it is clear that there has been little direct application of systems science tools to peacebuilding thus far. Relevant efforts have tended to address surface-level research topics, with modest implications for designing and implementing high-impact peacebuilding efforts. The explication of covert networks (or terror cells) was the most common focus among the case studies reviewed. Certainly, identifying and disrupting such networks can alleviate violence and instability, but the existence of these networks is a symptom of conflict rather than a root cause. When we considered the successful application of systems science in related fields, though, we identified a wealth of untapped potential in using systems science tools to provide concrete guidance to peacebuilding efforts. There is reason to be quite optimistic that the application of these methodologies could both reveal why conflict occurs (or might occur) in a given context and also help identify effective strategies to promote sustainable peace.

There is strong evidence that systems science provides a way to better understand the etiologies underlying the structures and processes that help or hinder peacebuilding efforts (for example, intra- and intergroup trust, cycles of violence, or entrenched corruption). The case studies reviewed indicate that not only have systems science approaches been able to accomplish this task in related topic areas, but also that there have been nascent attempts—with varying levels of success—to do so in contexts directly relevant to peacebuilding. Based on this finding, we confidently conclude that future peacebuilding efforts can benefit from the preliminary application of systems science research into how relevant systems are organized and dynamically operate.

The case studies reviewed also indicate that systems science can help practitioners and decision makers identify leverage points for effecting change through peacebuilding interventions. The case studies directly related to peacebuilding topics did not yield any examples of systems science being used to compare potential alternative intervention designs or implementation strategies for maximal positive impact. However, several of these case studies yielded models that could likely be used to do so. Furthermore, a review of in-depth case studies taken from related proxy fields provides strong evidence that it is indeed possible to do so. For example, ABM has revealed effective ways to increase equitable educational access and decrease racial segregation; system dynamics modeling has suggested ways to increase the efficient delivery of high-quality health care; and the synergistic, iterative application of GMB, ABM, and network analysis has guided the deployment of whole-of-community childhood obesity prevention interventions. Just as important, these same research tools have been used to identify intervention approaches that appear intuitively or emotionally enticing (or are otherwise promising on paper), but are nonetheless unlikely to have a positive impact. Therefore, we are convinced that the novel, innovative application of systems science to directly guide the design of peacebuilding initiatives is worthy of significantly more investment in resources and effort.

Finally, the evidence reviewed suggests that the application of systems science can help practitioners and decision makers anticipate and manage unpredictability during the implementation of peacebuilding interventions. Although none of the case studies directly relevant to peacebuilding explicitly addressed unpredictability, the studies that focused on adjacent or overlapping topics suggest two distinct ways to do so: by considering long-term impact patterns and by explicitly incorporating uncertainty about context and dynamics in analyses. The first way makes sustainability a key component in the design of peacebuilding efforts. That is, decision makers can use systems science tools to select an intervention design that has a minimal initial positive impact but grows over time over a design that has a significant initial impact but is subsequently reversed. The second way allows decisions to be made with due consideration of “known unknowns.” Thus, decision makers might select an intervention design that has a high probability of moderate success over a design that has a high probability of considerable success but only under completely favorable conditions.

CLARIFYING EXAMPLE

The following example, informed by the evidence review, shows how different systems science tools could be used to extend research in a specific context in which USIP currently operates, with the goal of enhancing practical, positive impacts.⁹⁴

Project Description

Context. In Burkina Faso, violent extremist groups have been challenging the authority of the Burkinabe state since 2015. As a result, many citizens have steadily lost faith in the state’s capacity to protect their rights, and international resources have been dedicated to supporting Burkina Faso’s justice and security institutions. In 2017, the country’s court and its staff were attacked by a group of citizens dissatisfied with judicial decisions affecting members of their community. About one year later, USIP launched a project to improve the performance of the formal court system in the town of Manga.

Methodology. USIP conducted GMB efforts from 2018 to 2022 in Manga, Burkina Faso. The project brought together approximately twenty-five stakeholders from Manga. The group included members of the formal justice system (judges, prosecutors, bailiffs, judicial staff); members of the informal justice system (religious leaders, traditional chiefs, customary leaders); those with formal authority outside of the justice system (mayor, prefect); and community members with relevant perspectives such as women, youth, and commercial vendors. USIP held preliminary consultations with these community representatives to assess their willingness to work together on the project’s GMB efforts. Once their willingness was confirmed, an 18-month locally led diagnostic research evaluation of the Manga formal court system commenced. During working group sessions, qualitative data on interactions with the formal court system were gathered in the form of more than 250 stories. Each story was then coded and analyzed.

Initial impact. Thirteen recurring problems emerged from the story analyses. They included, among others, social cultural barriers (for example, “Proceeding conducted in French, which I do not speak”); geographic distance (for example, “The court is located 20 km away from my home. I am unable to go” and “I could not stay overnight when my hearing was postponed”); and costs (for example, “I am unable to afford a lawyer or pay fees at the court” and “Many intermediaries promised me to help me with procedural problems on sight in exchange for money. They ran away with the money”). Subsequent working-group sessions focused on supporting the stakeholders in identifying how the 13 common problems interacted with one another and how they interacted with the larger problem of performance, which the USIP project was meant to address. All the information collected was consolidated in an initial common problem tree, which was further consolidated into an initial draft causal loop diagram (CLD). Supported by USIP, the stakeholders then worked collaboratively to validate and adjust the CLD and identify high leverage points. Finally, these points were used to inform the development of a local action plan, as well as a monitoring and evaluation framework to keep track of the plan’s implementation.

Potential Additional Modeling

Group Model Building. Additional GMB efforts could be employed, such as using scripts to collect more information from stakeholders to further refine the action plan. One possibility is a connection circle activity that creates a representation of the convened group’s collective perception of relevant social networks. This product could help determine the extent to which members of the formal and informal judicial systems currently do or are capable of coordinating efforts. Another GMB effort could be recruiting stakeholders who were not included in the original group but who might be able to provide additional insights on the existing CLD and action plan. Given that these endeavors would piggyback on those already underway—building on USIP personnel’s relationships and knowledge, for example—this supplemental GMB work would be similar in nature but easier to initiate and accomplish.

Social Network Analysis. Information about, and trust in, the court system are likely garnered through both formal and informal channels within the community. Learning more about how community members are socially connected—and, more importantly, where there are key gaps in the social networks that can be addressed through explicit action—might suggest ways that USIP can speed up and amplify the impact of its work in Manga. Data required for social network analysis could be collected through interviews or survey instruments, but USIP and local stakeholders would likely need to work together to develop and deploy them.

System Dynamics. The most straightforward application of system dynamics could translate the CLD that was created through stakeholder workshops into a fully quantified form. This modeling could be used to test hypotheses that would have immediate practical implications, such as whether the action plan is focused on the most high-impact leverage points and, if not, which alternatives should be considered. Although the basis for the initial model design is the extant CLD, there are two crucial remaining requirements for this approach. The first is expertise in computational development and use, and the second is data to characterize model elements and inform model calibration and testing. For example, a stock in the model might be community members who would benefit from access to the court system; specific initial values would be needed for the model as well as how these change with flows into and out of this stock.

Agent-Based Modeling. ABM can be used to represent the problems identified by stakeholders at the individual level. A simple ABM might consist of a population of agents representative of Manga community members with respect to key attributes, including geographic location. Over time, some members will need courts; these courts will either perform as intended and expected for the community members (or not) based on their own characteristics as well as those of the courts. Hypotheses could be tested to determine whether specific interventions (for example, providing transportation to courts) are effective and, if so, in what magnitudes or combinations they are most beneficial to community members. As with the system dynamics application, this ABM effort would require a substantial amount of expertise and data.

Additional Takeaways

Based on this evidence review and our experiences as researchers, the following general guidance might be useful in applying systems science to the design and implementation of effective peacebuilding efforts.

Use the right tools for the job. As noted, this evidence review focuses on a small but not exhaustive set of systems science tools, and it is possible to combine systems science approaches within projects (as done in the whole-of-community childhood obesity prevention case study) or even within models.⁹⁵ Successfully applying systems science to achieve positive impacts first and foremost requires selecting the right approach for the task. Considerations should include the perspectives taken by researchers and stakeholders on how the system operates, the nature of active entities within the system, heterogeneity in model elements, and the importance of social or geographic space.⁹⁶ For instance, an examination of how interlinked capacities drive outcomes related to peace might best be done using system dynamics, while an examination of individual motivations and behaviors would use ABM.

Start simple, but you do not need to stop there. One of the biggest strengths of systems science is its applicability to why and how questions. As such, when engaging in systems science research, it is useful to start with a very simple model (for example, Schelling’s initial model of residential segregation) to more readily understand why a model is producing observed behaviors.⁹⁷ Initially, working with systems science models often feels unsatisfying or even alien, especially for people with deep theoretical or practical expertise in a given topic space. When working with stakeholders new to systems science, it can help to call upon George Box’s advice, generally distilled to the maxim “all models are wrong, but some are useful.”⁹⁸ It is generally best to start with a model that leaves much of the real-world complexity out—but whose processes are fully comprehensible—and then add more sophisticated components when it becomes practically useful while remaining easy to understand.

The journey can be as rewarding as the destination. Proceeding through the iterative process of systems science research can yield useful benefits prior to the end of a particular project phase. The design of any model forces implicit assumptions to become explicit (and more easily examined) and helps clarify the locus of uncertainty. Even designing a simple theoretical or prototype model can prove informative. Such models can often cast doubt on previously held assumptions or mental models, inspire new hypotheses, or provide guidance on how to allocate scare resources to data collection activities. The subsequent steps toward a usable model—including model design, data collection, or preliminary analysis—can then have added, often surprising benefits. For example, in several of the case studies summarized in this paper, GMB was part of the overall model design and data collection plan, but it also positively impacted the overall project by increasing stakeholder engagement and systems thinking capacity. Similarly, the collection of data for input into the models can reveal unexpected gaps in knowledge or understanding, thus inspiring further research with practical implications.

You get what you pay for. Systems science models can have a high degree of predictive power, providing timely guidance that is highly context-specific, incorporates precise disaggregation across groups or places of interest, and considers a lengthy time horizon. However, these benefits generally involve substantial costs. As outlined in the case studies earlier, these include data, computational, and personnel costs related to:

developing survey instruments, recruiting participants, deploying surveys, obtaining extant datasets, engaging in literature reviews, and meaningfully and appropriately combining data from multiple sources;

data storage and computational processing; and

convening a team with sufficient expertise, team management, and effort associated with each phase of research.

Ideally, potential costs should be considered alongside project goals, timelines, and available resources prior to and during systems science research. For example, a model with a high degree of explanatory power in a given context can theoretically be used to consider an infinite number of counterfactuals. However, computational and personnel resources are finite and often highly constrained by practical considerations.

Know your audience. The quality of systems science tools and their ultimate impact are improved by early and frequent stakeholder engagement. It is essential that these tools have face validity, given who will eventually use them directly and take actions based on the findings. Facilitating relationships with stakeholders early on in the research and maintaining these relationships ensures that model elements meet this face-validity standard. Fortunately, unlike complicated statistical models, systems science models are relatively easy to communicate to audiences with little or no research training.

Systems science tools are most effective when they are designed with the end users in mind. Research results are only useful if they can be clearly communicated to those who will act on them, and the most salient implications for policy and practice should be readily apparent. Extensive engagement not only accomplishes this, but also confers a sense of trust, buy-in, and ownership that enhances positive outcomes.

Appendix

EXAMPLE OF A CAUSAL LOOP DIAGRAM

This diagram represents the system—as identified by community stakeholders—that generates food and transportation behavior in Latin American cities. Stocks are represented with text; flows are represented with arrows (a plus or minus sign indicates the direction of change, and a double bar indicates a time delay in the relationship); and feedback loops are shown with either an R to denote positive or reinforcing feedback or a B to denote negative or balancing feedback.

Source: Brent A. Langellier, Jill A. Kuhlberg, Ellis A. Ballard, S. Claire Slesinski, Ivana Stankov et al., “Using Community-based System Dynamics Modeling to Understand the Complex Systems that Influence Health in Cities: The SALURBAL Study,” Health & Place 60 (2019): 102215, https://doi.org/10.1016/j.healthplace.2019.102215.

CASE STUDY PART 2, LITERATURE REVIEW METHODOLOGY

We first searched three databases: (1) Google Scholar, (2) Web of Science, and (3) Scopus. We believed that these databases, together, would substantially cover the research most relevant for this evidence review.

Google Scholar Search Strings:

“Group model-building” and “peace”

“Group model-building” and “war”

“Network Analysis” and “peace”

“Network Analysis” and “war”

“Agent-based model” and “peace”

“Agent-based model” and “war”

“System Dynamic” and “peace”

“System dynamic” and “war”

Web of Science Search Strings:

“Peace” or “Conflict” or “War” or “Genocide” or “Violence” or “Secession” or “Extremism” or “Ethnocentrism” or “Development” or “Diplomacy” and “Group model-building” or “Systems Thinking” or “Agent-based model” or “System Dynamics” or “Network Analysis” or “Social Network”

(“Peace” or “Conflict” or “War” or “Genocide” or “Violence” or “Secession” or “Extremism” or “Ethnocentrism” or “Development” or “Diplomacy”) AND (“Group model-building” or “Systems Thinking” or “Agent-based model” or “System Dynamics” or “Network Analysis” or “Social network”)

Scopus Search Strings:

“Peace” or “Conflict” or “War” or “Genocide” or “Violence” or “Secession” or “Extremism” or “Ethnocentrism” or “Development” or “Diplomacy” and “Group model-building” or “Systems Thinking” or “Agent-based model” or “System Dynamics” or “Network Analysis” or “Social Network”

(“Peace” or “Conflict” or “War” or “Genocide” or “Violence” or “Secession” or “Extremism” or “Ethnocentrism” or “Development” or “Diplomacy”) AND (“Group model-building” or “Systems Thinking” or “Agent-based model” or “System Dynamics” or “Network Analysis” or “Social network”)

Following the database searches, we conducted a snowball search phase, in which we collected articles that were cited in publications identified during the first phase. Then, recognizing that this literature review was not exhaustive because of limitations imposed by database inclusion and indexing procedures as well as our selection of search terms, we supplemented our set of case studies with a small set of literature that we had prior knowledge about through external channels (for example, discussions with colleagues).

Next, we examined the literature collected to determine which publications were appropriate for inclusion as case studies in this evidence review. We applied the following inclusion criteria:

Research that was published, in English, in a peer-reviewed journal; this ensured a minimum level of quality and allowed us to judge whether additional criteria were met and to summarize articles as case studies.

Research that was completed and fully described; we specifically excluded proposed research or research protocols. This approach allowed us to fully summarize the case studies, provide a clear description of the methodology (the systems science tool or tools used), make suppositions about the costs of engaging in research (resources, personnel), and indicate findings and implications for the field.

Research that was—or easily could be—applied to provide practical guidance; it is not purely hypothetical or theoretical in nature, but instead can use empirical input data to provide specific, quantifiable output. In general, we erred on the side of inclusion here.

Notes

  1.   Edward N. Luttwak, Strategy: The Logic of War and Peace (Cambridge, MA: Harvard University Press, 2001); Ben Fowler and Elizabeth Dunn, “Evaluating Systems and Systemic Change for Inclusive Market Development” (Washington, DC: United States Agency for International Development, 2014), https://www.marketlinks.org/resources/evaluating-systems-and-systemic-change-inclusive-market-development; Robert Ricigliano and Diana Chigas, “Systems Thinking in Conflict Assessment: Concepts and Application” (Washington, DC: United States Agency for International Development, 2011); Rachel Kleinfeld, “Improving Development Aid Design and Evaluation: Plan for Sailboats, Not Trains” (Washington, DC: Carnegie Endowment for International Peace, 2015), https://carnegieendowment.org/2015/03/02/improving-development-aid-design-and-evaluation-plan-for-sailboats-not-trains-pub-59159.

  2.   John H. Miller and Scott E. Page, Complex Adaptive Systems (Princeton, NJ: Princeton University Press, 2007).

  3.   Social systems can also include important intra-individual elements, but a thorough discussion of this is beyond the scope of this document.

  4.   Matt Kasman, Nancy Breen, and Ross A. Hammond, “Complex Systems Science,” in The Science of Health Disparities Research, ed. Irene Dankwa-Mullan et al. (Hoboken, NJ: Wiley, 2021), 243–56, https://www.researchgate.net/publication/349693618_Complex_Systems_Science; Ross A. Hammond, “Considerations and Best Practices in Agent-based Modeling to Inform Policy,” in Assessing the Use of Agent-based Models for Tobacco Regulation, Robert Wallace, Amy Geller, and V. Ayano Ogawa (Washington, DC: National Academies Press, 2015), https://www.ncbi.nlm.nih.gov/books/NBK305917/; Joshua M. Epstein, “Why Model?,” Journal of Artificial Societies and Social Simulation 11, no. 4 (2008): 12, https://www.jasss.org/11/4/12.html.

  5.   Susan A. Bandes, Marie Pryor, Erin M. Kerrison, and Phillip Atiba, “The Mismeasure of Terry Stops: Assessing the Psychological and Emotional Harms of Stop and Frisk to Individuals and Communities,” Behavioral Sciences & the Law 37, no. 2 (2019): 176–94, https://doi.org/10.1002/bsl.2401; Joseph A. Ferrandino, “The Effectiveness and Equity of NYPD Stop and Frisk Policy, 2003–2014,” Journal of Crime and Justice 41, no. 2 (2018): 119–35, https://doi.org/10.1080/0735648X.2016.1249385.

  6.   Matt Kasman, “Data-Driven Approaches to Subsidizing College Enrollment Costs,” The Brookings Institution, May 26, 2020, https://www.brookings.edu/policy2020/bigideas/data-driven-approaches-to-subsidizing-college-enrollment-costs/.

  7.   For example, the agent-based modeling discussed here is broadly applicable to other microsimulation approaches.

  8.   Hammond, “Considerations and Best Practices in Agent-based Modeling to Inform Policy”; Matt Kasman and L. Kurt Kreuger, “Best Practices for Systems Science Research” (Bethesda, MD: National Institutes of Health, Office of Behavioral and Social Sciences Research, 2022), https://obssr.od.nih.gov/sites/obssr/files/inline-files/best-practices-consensus-statement-FINAL-508.pdf.

  9.   Julia M. Appel, Karen Fullerton, Erin Hennessy, Ariella R. Korn, Alison Tovar et al., “Design and Methods of Shape Up Under 5: Integration of Systems Science and Community-engaged Research to Prevent Early Childhood Obesity,” PLOS ONE 14, no. 8 (2019): e0220169. https://doi.org/10.1371/journal.pone.0220169; Chris Swinerd and Ken R. McNaught, “Design Classes for Hybrid Simulations Involving Agent-based and System Dynamics Models,” Simulation Modelling Practice and Theory 25 (2012): 118–33, https://doi.org/10.1016/j.simpat.2011.09.002.

10.   John Sterman, Rogelio Oliva, Kevin Wayne Linderman, and Elliot Bendoly, “System Dynamics Perspectives and Modeling Opportunities for Research in Operations Management,” Journal of Operations Management 39–40, no. 1 (2015): 1–5, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3118311; John Sterman, “System dynamics at Sixty: The Path Forward,” System Dynamics Review 34, no. 1 (2018): 5–47, https://onlinelibrary.wiley.com/doi/abs/10.1002/sdr.1601; Jay W. Forrester, “System Dynamics—A Personal View of the First Fifty Years,” System Dynamics Review 23, no. 2–3 (2007): 345–58, https://onlinelibrary.wiley.com/doi/10.1002/sdr.382.

11.   John Sterman, Business Dynamics: Systems Thinking and Modeling for a Complex World (New York: McGraw-Hill, Inc., 2000).

12.   Stanley Wasserman and Katherine Faust, Social Network Analysis: Methods and Applications (Cambridge: Cambridge University Press, 1994); John Scott and Peter J. Carrington, The SAGE Handbook of Social Network Analysis (London: SAGE Publishing, 2011); John Scott, “Social Network Analysis: Developments, Advances, and Prospects,” Social Network Analysis and Mining 1, no. 1 (2011): 21–26, https://link.springer.com/article/10.1007/s13278-010-0012-6; John Scott, “Social Network Analysis,” Sociology 22, no. 1 (1988): 109–27, https://journals.sagepub.com/doi/10.1177/0038038588022001007; Peter J. Carrington, John Scott, and Stanley Wasserman, Models and Methods in Social Network Analysis (Cambridge: Cambridge University Press, 2005).

13.   Tom Snijders, Gerhard G. van de Bunt, and Christian Steglich, “Introduction to Stochastic Actor-based Models for Network Dynamics,” Social Networks 32, no. 1 (2010): 44–60, https://www.sciencedirect.com/science/article/abs/pii/S0378873309000069.

14.   Hammond, “Considerations and Best Practices in Agent-based Modeling to Inform Policy”; Steven F. Railsback and Volker Grimm, Agent-based and Individual-based Modeling: A Practical Introduction (Princeton, NJ: Princeton University Press, 2019); Joshua M. Epstein, “Agent-based Computational Models and Generative Social Science,” Complexity 4, no. 5 (1999): 41–60, https://onlinelibrary.wiley.com/doi/10.1002/%28SICI%291099-0526%28199905/06%294%3A5%3C41%3A%3AAID-CPLX9%3E3.0.CO%3B2-F.

15.   Ian M. Thomas and Stacia R. Reilly, “Group Model Building: A Framework for Organizing Healthy Community Program and Policy Initiatives in Columbia, Missouri,” Journal of Public Health Management and Practice 21 (2015): S79–S83, https://doi.org/10.1097/PHH.0000000000000209; Peter S. Hovmand, “Group Model Building and Community-based System Dynamics Process,” in Community Based System Dynamics, ed. Peter S. Hovmand (New York: Springer, 2014), 17–30, https://www.researchgate.net/publication/300892001_Group_Model_Building_and_Community-Based_System_Dynamics_Process; Peter S. Hovmand, David F. Andersen, Etienne Rouwette, George P. Richardson, Krista Rux et al., “Group Model-Building ‘Scripts’ as a Collaborative Planning Tool,” Systems Research and Behavioral Science 29, no. 2 (2012): 179–93, https://onlinelibrary.wiley.com/doi/abs/10.1002/sres.2105; Lloyd S. Peck, “Group Model Building: Facilitating Team Learning Using System Dynamics,” Journal of the Operational Research Society 49, no. 7 (1998): 766–67, https://www.tandfonline.com/doi/abs/10.1057/palgrave.jors.2600567; Céline Bérard, “Group Model Building Using System Dynamics: An Analysis of Methodological Frameworks,” Electronic Journal of Business Research Methods 8, no. 1 (2010): 35–45, https://academic-publishing.org/index.php/ejbrm/article/view/1252.

16.   Thomas and Reilly, “Group Model Building”; Hovmand, “Group Model Building and Community-based System Dynamics Process”; Hovmand et al., “Group Model-Building ‘Scripts’ as a Collaborative Planning Tool”; Steven Allender, Brynle Owen, Jill Kuhlberg, Janette Lowe, Phoebe Nagorcka-Smith et al., “A Community Based Systems Diagram of Obesity Causes,” PLOS ONE 10, no. 7 (2015): e0129683, https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0129683.

17.   Bérard, “Group Model Building Using System Dynamics”; Hovmand et al., “Group Model-Building ‘Scripts’ as a Collaborative Planning Tool.”

18.   Hovmand, “Group Model Building and Community-based System Dynamics Process”; Peck, “Group Model Building: Facilitating Team Learning Using System Dynamics.”

19.   Alastair K. Ager, Martina Lembani, Abdulaziz Mohammed, Garba Mohammed Ashir, Ahmad Abdulwahab et al., “Health Service Resilience in Yobe State, Nigeria in the Context of the Boko Haram Insurgency: A Systems Dynamics Analysis Using Group Model Building,” Conflict and Health 9, no.1 (2015): 1–14, https://pubmed.ncbi.nlm.nih.gov/26442129/.

20.   Matt Kasman, Ross A. Hammond, Benjamin Heuberger, Austen Mack-Crane, Rob Purcell et al., “Activating a Community: An Agent-based Model of Romp & Chomp, a Whole-of-Community Childhood Obesity Intervention,” Obesity 27, no. 9 (2019): 1494–1502, https://doi.org/10.1002/oby.22553.

21.   Appel et al., “Design and Methods of Shape Up Under 5”; Matt Kasman, Brynle Owen, and Joshua Hayward, “A Community-based Complex Systems Approach to High School Completion,” Systems Research and Behavioral Science 34, no. 3 (2017): 267–76, https://doi.org/10.1002/sres.2409.

22.   Andrew Robinson, “Did Einstein Really Say That?,” Nature 557, no. 7703 (2018): 30, https://doi.org/10.1038/d41586-018-05004-4.

23.   Sean F. Reardon, Rachel Baker, Matt Kasman, Daniel Klasik, and Joseph B. Townsend, “What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model,” Journal of Policy Analysis and Management 37, no. 3 (2018): 630–57, https://onlinelibrary.wiley.com/doi/abs/10.1002/pam.22056.

24.   Fisher v. University of Texas at Austin, 133 S. Ct. 2411 (Supreme Court 2013); Fisher v. University of Texas at Austin, 136 S. Ct. 2198 (Supreme Court 2016).

25.   Fisher v. University of Texas at Austin (2013); Fisher v. University of Texas at Austin (2016).

26.   “Brief of 823 Social Scientists as Amici Curiae,” The Civil Rights Project, November 3, 2015, https://www.civilrightsproject.ucla.edu/legal-developments/legal-briefs/amicus-brief-of-social-scientists-in-fisher-case.

27.   Kasman, “Data-Driven Approaches to Subsidizing College Enrollment Costs.”

28.   Sean F. Reardon, Matt Kasman, Daniel Klasik, and Rachel Baker, “Agent-Based Simulation Models of the College Sorting Process,” Journal of Artificial Societies and Social Simulation 19, no. 1 (2016): 8, https://www.jasss.org/19/1/8.html.

29.   Thomas C. Schelling, “Dynamic Models of Segregation,” Journal of Mathematical Sociology 1, no. 2 (1971): 143–86, https://www.tandfonline.com/doi/abs/10.1080/0022250X.1971.9989794; Elizabeth E. Bruch and Robert D. Mare, “Neighborhood Choice and Neighborhood Change,” American Journal of Sociology 112, no. 3 (2006): 667–709, https://www.jstor.org/stable/10.1086/507856; Taylor Anderson, Aaron Leung, Suzana Dragicevic, and Liliana Perez, “Modeling the Geospatial Dynamics of Residential Segregation in Three Canadian Cities: An Agent-based Approach,” Transactions in GIS 25, no. 2 (2021): 948–67, https://onlinelibrary.wiley.com/doi/abs/10.1111/tgis.12712; Babak Mahdavi Ardestani, David O’Sullivan, and Peter Davis, “A Multi-Scaled Agent-based Model of Residential Segregation Applied to a Real Metropolitan Area,” Computers, Environment and Urban Systems 69 (2018): 1–16, https://www.sciencedirect.com/science/article/pii/S0198971517302259; Andrew Crooks, “Constructing and Implementing an Agent-based Model of Residential Segregation through Vector GIS,” International Journal of Geographical Information Science 24, no. 5 (2010): 661–75, https://www.tandfonline.com/doi/abs/10.1080/13658810903569572; Mark Fossett and Warren Waren, “Overlooked Implications of Ethnic Preferences for Residential Segregation in Agent-based Models,” Urban Studies 42, no. 11 (2005): 1893–917, https://journals.sagepub.com/doi/10.1080/00420980500280354; Qingxu Huang, Dawn C. Parker, Tatiana Filatova, and Shipeng Sun, “A Review of Urban Residential Choice Models Using Agent-based Modeling,” Environment and Planning B: Planning and Design 41, no. 4 (2014): 661–89, https://journals.sagepub.com/doi/10.1068/b120043p; Li Yin, “The Dynamics of Residential Segregation in Buffalo: An Agent-based Simulation,” Urban Studies 46, no. 13 (2009): 2749–70, https://www.jstor.org/stable/43197939.

30.   Schelling, “Dynamic Models of Segregation.”

31.   Bruch and Mare, “Neighborhood Choice and Neighborhood Change”; Anderson et al., “Modeling the Geospatial Dynamics of Residential Segregation in Three Canadian Cities”; Ardestani et al., “A Multi-Scaled Agent-based Model of Residential Segregation Applied to a Real Metropolitan Area”; Crooks, “Constructing and Implementing an Agent-based Model of Residential Segregation through Vector GIS”; Fossett and Waren, “Overlooked Implications of Ethnic Preferences for Residential Segregation in Agent-based Models”; Huang et al., “A Review of Urban Residential Choice Models Using Agent-based Modeling”; Yin, “The Dynamics of Residential Segregation in Buffalo.”

32.   Schelling, “Dynamic Models of Segregation.”

33.   Bruch and Mare, “Neighborhood Choice and Neighborhood Change”; Anderson et al., “Modeling the Geospatial Dynamics of Residential Segregation in Three Canadian Cities”; Ardestani et al., “A Multi-Scaled Agent-based Model of Residential Segregation Applied to a Real Metropolitan Area”; Crooks, “Constructing and Implementing an Agent-based Model of Residential Segregation through Vector GIS”; Fossett and Waren, “Overlooked Implications of Ethnic Preferences for Residential Segregation in Agent-based Models”; Huang et al., “A Review of Urban Residential Choice Models Using Agent-based Modeling”; Yin, “The Dynamics of Residential Segregation in Buffalo.”

34.   Schelling, “Dynamic Models of Segregation.”

35.   Bruch and Mare, “Neighborhood Choice and Neighborhood Change”; Anderson et al., “Modeling the Geospatial Dynamics of Residential Segregation in Three Canadian Cities”; Ardestani et al., “A Multi-Scaled Agent-based Model of Residential Segregation Applied to a Real Metropolitan Area”; Crooks, “Constructing and Implementing an Agent-based Model of Residential Segregation through Vector GIS”; Fossett and Waren, “Overlooked Implications of Ethnic Preferences for Residential Segregation in Agent-based Models”; Huang et al., “A Review of Urban Residential Choice Models Using Agent-based Modeling”; Yin, “The Dynamics of Residential Segregation in Buffalo.”

36.   William Julius Wilson and Richard P. Taub, There Goes the Neighborhood: Racial, Ethnic, and Class Tensions in Four Chicago Neighborhoods and Their Meaning for America (New York: Vintage Books, 2007).

37.   Matt Kasman and Jon Valant, “The Opportunities and Risks of K-12 Student Placement Algorithms (Washington, DC: The Brookings Institution, 2019), https://www.brookings.edu/research/the-opportunities-and-risks-of-k-12-student-placement-algorithms/; Matt Kasman, School Selection, Student Assignment, and Enrollment in a School District with Open Enrollment and Mandatory Choice Policies (Stanford, CA: Stanford University, 2014).

38.   Tanya Dzekon, “School Board Stalls on Effort to Address Racial Isolation,” San Francisco Public Press, June 12, 2015, https://www.sfpublicpress.org/school-board-stalls-on-effort-to-address-racial-isolation/.

39.   Kasman and Kreuger, “Best Practices for Systems Science Research.”

40.   Hammond, “Considerations and Best Practices in Agent-based Modeling to Inform Policy”; Kasman and Kreuger, “Best Practices for Systems Science Research.”

41.   Jan van Leeuwen, Handbook of Theoretical Computer Science (Vol. A) Algorithms and Complexity (Cambridge, MA: MIT Press, 1991); Michael Laver, “Policy and the Dynamics of Political Competition,” American Political Science Review 99, no. 2 (2005): 263–81, https://www.researchgate.net/publication/286997821_Policy_and_the_dynamics_of_political_competition.

42.   Kasman et al., “Complex Systems Science”; Hammond, “Considerations and Best Practices in Agent-based Modeling to Inform Policy”; Kasman and Kreuger, “Best Practices for Systems Science Research.”

43.   Marko Klašnja and Natalija Novta, “Segregation, Polarization, and Ethnic Conflict,” Journal of Conflict Resolution 60, no. 5 (2016): 927–55, https://journals.sagepub.com/doi/abs/10.1177/0022002714550084?journalCode=jcrb; Paul Doherty and Michael Poole, “Living Apart in Belfast: Residential Segregation in a Context of Ethnic Conflict,” in Ethnicity and Housing, ed. Frederick W. Boal (London: Routledge, 2020), 179–89, https://www.taylorfrancis.com/chapters/edit/10.4324/9781003073253-23/living-apart-belfast-residential-segregation-context-ethnic-conflict-paul-doherty-michael-poole; Alejandro Corvalan and Miguel Vargas, “Segregation and Conflict: An Empirical Analysis,” Journal of Development Economics 116 (2015): 212–22, https://www.sciencedirect.com/science/article/abs/pii/S0304387815000607.

44.   Stefani A. Crabtree, R. Kyle Bocinsky, Paul L. Hooper, Susan C. Ryan, and Timothy A. Kohler, “How to Make a Polity (in the Central Mesa Verde Region),” American Antiquity 82, no. 1 (2017): 71–95, https://www.jstor.org/stable/26337956; Timothy A. Kohler, R. Kyle Bocinsky, Denton Cockburn, Stefani A. Crabtree, Mark D. Varien et al., “Modelling Prehispanic Pueblo Societies in their Ecosystems,” Ecological Modelling 241 (2012): 30–41, https://www.sciencedirect.com/science/article/abs/pii/S0304380012000038; Jeffrey S. Dean, George J. Gumerman, Joshua M. Epstein, Robert Axtell, Alan Swedlund et al., “Understanding Anasazi Culture Change through Agent-based Modeling,” in Dynamics in Human and Primate Societies: Agent-based Modeling of Social and Spatial Processes, ed. Timothy A. Kohler and George J. Gumerman (Oxford: Oxford University Press, 2000), 179–206, https://academic.oup.com/book/40679/chapter-abstract/348372146?redirectedFrom=fulltext; Robert L. Axtell, Joshua M. Epstein, Jeffrey S. Dean, George J. Gumerman, Alan Swedlund et al., “Population Growth and Collapse in a Multiagent Model of the Kayenta Anasazi in Long House Valley,” Proceedings of the National Academy of Sciences 99, no. 3 (2002): 7275–79, https://www.pnas.org/doi/10.1073/pnas.092080799.

45.   Kohler et al., “Modelling Prehispanic Pueblo Societies in their Ecosystems.”

46.   Dean et al., “Understanding Anasazi Culture Change through Agent-based Modeling.”

47.   “Rethink Health,” Rippel Foundation, accessed June 16, 2022, https://rethinkhealth.org/; David A. Kindig and Bobby Milstein, “A Balanced Investment Portfolio for Equitable Health and Well-being Is an Imperative, and Within Reach,” Health Affairs 37, no. 4 (2018): 579–84, https://www.healthaffairs.org/doi/10.1377/hlthaff.2017.1463; Jack Homer, Gary Hirsch, and Bobby Milstein, “Chronic Illness in a Complex Health Economy: The Perils and Promises of Downstream and Upstream Reforms,” System Dynamics Review 23, no. 2–3 (2007): 313–43, https://doi.org/10.1002/sdr.379; Bobby Milstein, Jack Homer, Peter Briss, Deron Burton, and Terry Pechacek, “Why Behavioral and Environmental Interventions Are Needed to Improve Health at Lower Cost,” Health Affairs 30, no. 5 (2011): 823–32, https://doi.org/10.1377/hlthaff.2010.1116; Jack Homer, Bobby Milstein, Darwin Labarthe, Diane Orenstein, Kristina Wile et al., “Simulating and Evaluating Local Interventions to Improve Cardiovascular Health,” Preventing Chronic Disease 7, no. 1 (2009): A18, https://pubmed.ncbi.nlm.nih.gov/20040233/; Eric C. Schneider, Arnav Shah, Michelle M. Doty, Roosa Tikkanen, Katherine Fields et al., “Reflecting Poorly: Health Care in the US Compared to Other High-Income Countries” (New York: The Commonwealth Fund, 2021), https://www.commonwealthfund.org/publications/fund-reports/2021/aug/mirror-mirror-2021-reflecting-poorly; “The World Health Report 2000—Health Systems: Improving Performance” (Geneva: World Health Organization, 2000), https://www.who.int/publications/i/item/924156198X.

48.   Schneider et al., “Reflecting Poorly: Health Care in the US Compared to Other High-Income Countries”; “The World Health Report 2000—Health Systems: Improving Performance.”

49.   Milstein et al., “Why Behavioral and Environmental Interventions Are Needed to Improve Health at Lower Cost.”

50.   Homer et al., “Chronic Illness in a Complex Health Economy: The Perils and Promises of Downstream and Upstream Reforms.”

51.   Kindig et al., “A Balanced Investment Portfolio for Equitable Health and Well-being Is an Imperative, and Within Reach.”

52.   “Rethink Health.”

53.   “Rethink Health; Jack Homer, Bobby Milstein, Gary B. Hirsch, and Elliot S. Fisher, “Combined Regional Investments Could Substantially Enhance Health System Performance and Be Financially Affordable,” Health Affairs 35, no. 8 (2016): 1435–43, https://doi.org/10.1377/hlthaff.2015.1043.

54.   Homer et al., “Combined Regional Investments Could Substantially Enhance Health System Performance and Be Financially Affordable.”

55.   “ReThink Health.”

56.   Homer et al., “Simulating and Evaluating Local Interventions to Improve Cardiovascular Health.”

57.   “Rethink Health.”

58.   Appel et al., “Design and Methods of Shape Up Under 5”; Kasman et al., “Activating a Community”; “Catalyzing Communities | Helping Children Thrive in Healthy Environments,” Catalyzing Communities, https://catalyzingcommunities.org/; Erin Hennessy, Ariella R. Korn, and Christina D. Economos, “A Community-Level Perspective for Childhood Obesity Prevention,” in Global Perspectives on Childhood Obesity, ed. Debasis Bagchi (Cambridge, MA: Academic Press, 2019), 287–98, https://doi.org/10.1016/B978-0-12-812840-4.00023-2; Ariella R. Korn, Julia Appel, Ross A. Hammond, Erin Hennessy, Louise C. Mâsse et al., “Validation and Refinement of the Stakeholder-Driven Community Diffusion Survey for Childhood Obesity Prevention,” Implementation Science 16, no. 1 (2021): 91, https://doi.org/10.1186/s13012-021-01158-4; Ariella R. Korn, Ross A. Hammond, Erin Hennessy, Aviva Must, Mark C. Pachucki et al., “Evolution of a Coalition Network during a Whole-of-Community Intervention to Prevent Early Childhood Obesity,” Childhood Obesity 17, no. 6 (2021): 379–90, https://doi.org/10.1089/chi.2020.0156; Larissa Calancie, Karen Fullerton, Julia M. Appel, Ariella R. Korn, Erin Hennessy et al., “Implementing Group Model Building with the Shape Up Under 5 Community Committee Working to Prevent Early Childhood Obesity in Somerville, Massachusetts,” Journal of Public Health Management and Practice 28, no. 1 (2022): E43–E55, https://pubmed.ncbi.nlm.nih.gov/32810067/; “Using Systems Applications to Inform Obesity Solutions: Proceedings of a Workshop” (The National Academies Press, 2021), https://doi.org/10.17226/25900; Christina D. Economos and Joseph A. Curtatone, “Shaping Up Somerville: A Community Initiative in Massachusetts,” Preventive Medicine 50, no. 1 (2010): S97–S98, https://pubmed.ncbi.nlm.nih.gov/19897065/; Christina D. Economos, Raymond R. Hyatt, Aviva Must, Jeanne P. Goldberg, Julia Kuder et al., “Shape Up Somerville Two-Year Results: A Community-based Environmental Change Intervention Sustains Weight Reduction in Children,” Preventive Medicine 57, no. 4 (2013): 322–27, https://pubmed.ncbi.nlm.nih.gov/23756187/; de Andrea M. de Silva-Sanigorski, A. Colin Bell, Peter Kremer, Melanie Nichols, Maree Crellin et al., “Reducing Obesity in Early Childhood: Results from Romp & Chomp, an Australian Community-wide Intervention Program,” American Journal of Clinical Nutrition 91, no. 4 (2010): 831–40, https://pubmed.ncbi.nlm.nih.gov/20147472/; Ariella R. Korn, Erin Hennessy, Alison Tovar, Camille Finn, Ross A. Hammond et al., “Engaging Coalitions in Community-based Childhood Obesity Prevention Interventions: A Mixed Methods Assessment,” Childhood Obesity 14, no. 8 (2018): 537–52, https://doi.org/10.1089/chi.2018.0032; Ariella R. Korn, Erin Hennessy, Ross A. Hammond, Steven Allender, Matthew W. Gillman et al., “Development and Testing of a Novel Survey to Assess Stakeholder-Driven Community Diffusion of Childhood Obesity Prevention Efforts,” BMC Public Health 18, no. 681 (2018), https://doi.org/10.1186/s12889-018-5588-1; Christina D. Economos and Ross A. Hammond, “Designing Effective and Sustainable Multifaceted Interventions for Obesity Prevention and Healthy Communities,” Obesity 25, no. 7 (2017): 1155–56, https://doi.org/10.1002/oby.21893; Calancie et al., “Implementing Group Model Building with the Shape Up Under 5 Community Committee Working to Prevent Early Childhood Obesity in Somerville, Massachusetts”; Matt Kasman, Ross A. Hammond, Austen Mack-Crane, Rob Purcell, Ariella R. Korn et al., “Using Agent-based Modeling to Extrapolate Community-Wide Impact from a Stakeholder-Driven Childhood Obesity Prevention Intervention: Shape Up Under 5,” Childhood Obesity (2022), https://doi.org/10.1089/chi.2021.0264.

59.   Economos et al., “Shape Up Somerville Two-Year Results”; de Silva-Sanigorski et al., “Reducing Obesity in Early Childhood.”

60.   Appel et al., “Design and Methods of Shape Up Under 5”; Kasman et al., “Activating a Community”; “Using Systems Applications to Inform Obesity Solutions”; Korn et al., “Engaging Coalitions in Community-based Childhood Obesity Prevention Interventions”; Economos and Hammond, “Designing Effective and Sustainable Multifaceted Interventions for Obesity Prevention and Healthy Communities.”

61.   Appel et al., “Design and Methods of Shape Up Under 5”; Korn et al., “Validation and Refinement of the Stakeholder-Driven Community Diffusion Survey for Childhood Obesity Prevention”; Korn et al., “Development and Testing of a Novel Survey to Assess Stakeholder-Driven Community Diffusion of Childhood Obesity Prevention Efforts.”

62.   Appel et al., “Design and Methods of Shape Up Under 5”; Korn et al., “Evolution of a Coalition Network during a Whole-of-Community Intervention to Prevent Early Childhood Obesity”; “Using Systems Applications to Inform Obesity Solutions.”

63.   Appel et al., “Design and Methods of Shape Up Under 5”; Kasman et al., “Activating a Community”; Kasman et al., “Using Agent-based Modeling to Extrapolate Community-Wide Impact from a Stakeholder-Driven Childhood Obesity Prevention Intervention.”

64.   Appel et al., “Design and Methods of Shape Up Under 5”; Calancie et al., “Implementing Group Model Building with the Shape Up Under 5 Community Committee Working to Prevent Early Childhood Obesity in Somerville, Massachusetts.”

65.   Appel et al., “Design and Methods of Shape Up Under 5”; Kasman et al., “Activating a Community”; Korn et al., “Validation and Refinement of the Stakeholder-Driven Community Diffusion Survey for Childhood Obesity Prevention”; Korn et al., “Evolution of a Coalition Network during a Whole-of-Community Intervention to Prevent Early Childhood Obesity”; Calancie et al., “Implementing Group Model Building with the Shape Up Under 5 Community Committee Working to Prevent Early Childhood Obesity in Somerville, Massachusetts”; Korn et al., “Development and Testing of a Novel Survey to Assess Stakeholder-Driven Community Diffusion of Childhood Obesity Prevention Efforts”; Calancie et al., “Implementing Group Model Building with the Shape Up Under 5 Community Committee Working to Prevent Early Childhood Obesity in Somerville, Massachusetts”; Kasman et al., “Using Agent-based Modeling to Extrapolate Community-Wide Impact from a Stakeholder-Driven Childhood Obesity Prevention Intervention.”

66.   Appel et al., “Design and Methods of Shape Up Under 5”; Kasman et al., “Activating a Community”; Korn et al., “Validation and Refinement of the Stakeholder-Driven Community Diffusion Survey for Childhood Obesity Prevention”; Korn et al., “Engaging Coalitions in Community-based Childhood Obesity Prevention Interventions: A Mixed Methods Assessment”; Korn et al., “Development and Testing of a Novel Survey to Assess Stakeholder-Driven Community Diffusion of Childhood Obesity Prevention Efforts”; Economos and Hammond, “Designing Effective and Sustainable Multifaceted Interventions for Obesity Prevention and Healthy Communities”; Kasman et al., “Using Agent-based Modeling to Extrapolate Community-Wide Impact from a Stakeholder-Driven Childhood Obesity Prevention Intervention.”

67.   Kasman et al., “Complex Systems Science”; Hammond, “Considerations and Best Practices in Agent-based Modeling to Inform Policy”; Kasman and Kreuger, “Best Practices for Systems Science Research.”

68.   Reardon et al., “Agent-based Simulation Models of the College Sorting Process.”

69.   Kasman, “Data-Driven Approaches to Subsidizing College Enrollment Costs.”

70.   Abhijit Banerjee, Arun G. Chandrasekhar, Esther Duflo, and Matthew O. Jackson, “The Diffusion of Microfinance,” Science 341, no. 6144 (2013): 1236498, https://www.science.org/doi/10.1126/science.1236498; Lori Beaman, Ariel BenYishay, Jeremy Magruder, and Ahmed Mushfiq Mobarak, “Can Network Theory-based Targeting Increase Technology Adoption?,” American Economic Review 111, no. 6 (2021): 1918–43, https://www.aeaweb.org/articles?id=10.1257/aer.20200295.

71.   Appel et al., “Design and Methods of Shape Up Under 5.”

72.   Appel et al., “Design and Methods of Shape Up Under 5”; Calancie et al., “Implementing Group Model Building with the Shape Up Under 5 Community Committee Working to Prevent Early Childhood Obesity in Somerville, Massachusetts.”

73.   “Catalyzing Communities | Helping Children Thrive in Healthy Environments.”

74.   Kasman et al., “Activating a Community”; Kasman et al., “Using Agent-based Modeling to Extrapolate Community-Wide Impact from a Stakeholder-Driven Childhood Obesity Prevention Intervention.”

75.   Brian J. Reed and David R. Segal, “Social Network Analysis and Counterinsurgency Operations: The Capture of Saddam Hussein,” Sociological Focus 39, no. 4 (2006): 251–64, https://www.jstor.org/stable/20832304.

76.   Morgan Burcher and Chad Whelan, “Social Network Analysis and Small Group ‘Dark’ Networks: An Analysis of the London Bombers and the Problem of ‘Fuzzy’ Boundaries,” Global Crime 16, no. 2, (2015): 104–22, https://www.tandfonline.com/doi/abs/10.1080/17440572.2015.1005363.

77.   William D. Tilson, Thomas K. Duncan, and Daniel Farhat, “An Agent-based Model of Ethnocentrism and the Unintended Consequences of Violence,” Eastern Economic Journal 46, no. 3, (2020): 483–503, https://link.springer.com/article/10.1057/s41302-019-00151-6.

78.   Ross A. Hammond and Robert Axelrod, “The Evolution of Ethnocentrism,” Journal of Conflict Resolution 50, no. 6 (2006): 926–36, https://www.jstor.org/stable/27638531.

79.   Michela Leone, Enrica Lapucci, Manuela De Sario, Marina Davoli, Sara Farchi et al., “Social Network Analysis to Characterize Women Victims of Violence,” BMC Public Health 19, no. 1 (2019): 1–11, https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-019-6797-y.

80.   Andrew W. Bausch, “Democracy, War Effort, and the Systemic Democratic Peace,” Journal of Peace Research 52, no. 4 (2015): 435–47, https://www.jstor.org/stable/24557430.

81.   Lars-Erik Cederman and Kristian Skrede Gleditsch, “Conquest and Regime Change: An Evolutionary Model of the Spread of Democracy and Peace,” International Studies Quarterly 48, no. 3 (2004): 603–29, https://www.jstor.org/stable/3693525.

82.   Jannie Lilja, “Trust and Treason: Social Network Structure as a Source of Flexibility in Peace Negotiations,” Negotiation and Conflict Management Research 5, no. 1 (2012): 96–125, https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1750-4716.2011.00092.x.

83.   Caitlin Hamilton, Jordan McSwiney, Nyibeny Gum Naam, and Laura J. Shepherd, “The Social Life of the Women, Peace and Security Agenda: A Digital Social Network Analysis,” Global Society 36, no. 1 (2021): 1–24, https://www.tandfonline.com/doi/abs/10.1080/13600826.2021.1875996.

84.   Steven M. Radil, Colin Flint, and George E. Tita, “Spatializing Social Networks: Using Social Network Analysis to Investigate Geographies of Gang Rivalry, Territoriality, and Violence in Los Angeles,” Annals of the Association of American Geographers 100, no. 2 (2010): 307–26, https://www.jstor.org/stable/40645356.

85.   Volha Herasimovich and Aurkene Alzua-Sorzabal, “Communication Network Analysis to Advance Mapping ‘Sport for Development and Peace’ Complexity: Cohesion and Leadership,” International Review for the Sociology of Sport 56, no. 2 (2021): 170–93, https://journals.sagepub.com/doi/10.1177/1012690220909748.

86.   Ager et al., “Health Service Resilience in Yobe State, Nigeria in the Context of the Boko Haram Insurgency.”

87.   James Doran, “Iruba: An Agent-based Model of the Guerrilla War Process,” in Representing Social Reality, Pre-Proceedings of the Third Conference of the European Social Simulation Association, October 2005, 198–205, https://www.researchgate.net/publication/228373416_Iruba_An_Agent-Based_Model_of_the_Guerrilla_War_Process.

88.   Bernard Amadei, “A Systems Approach to the Sustainability–Peace Nexus,” Sustainability Science 16, no. 4 (2021): 1111–24, https://link.springer.com/article/10.1007/s11625-020-00902-x.

89.   Hrach Gregorian, Lara Olson, and Brian Woodward, “Understanding Peacebuilding Coordination and Impact Using a Complex Adaptive Systems Method,” Journal of Peacebuilding & Development 15, no. 1 (2020): 91–110, https://journals.sagepub.com/doi/full/10.1177/1542316619871924.

90.   Arief Rachman, “The Analysis of Indonesia’s Important Role to Keep Peace in Afghanistan Using a System Dynamic Concept Approach,” International Journal of ASRO 11, no. 3, (2020): 125–34, https://www.semanticscholar.org/paper/THE-ANALYSIS-OF-INDONESIA’S-IMPORTANT-ROLE-TO-KEEP-Rachman/c6c835475848c4c3a70c0a23cb2be71f1a605181.

91.   Valdis E. Krebs, “Mapping Networks of Terrorist Cells,” Connections 24, no. 3 (2002): 43–52, https://www.researchgate.net/publication/2490397_Mapping_Networks_of_Terrorist_Cells.

92.   Stuart Koschade, “A Social Network Analysis of Jemaah Islamiyah: The Applications to Counter terrorism and Intelligence,” Studies in Conflict & Terrorism 29, no. 6 (2006): 559–75, https://www.tandfonline.com/doi/abs/10.1080/10576100600798418?journalCode=uter20.

93.   Jared Edgerton, “Suicide Bomber Mobilization and Kin and Peer Ties,” Social Networks 70 (2022): 36–54, https://doi.org/10.1016/j.socnet.2021.10.005.

94.   Personal communication with Philippe Leroux-Martin, June 13, 2022, regarding specifics on USIP’s group modeling project in Manga, Burkina Faso.

95.   Kasman et al., “Complex Systems Science”; Kasman and Kreuger, “Best Practices for Systems Science Research”; Swinerd and McNaught, “Design Classes for Hybrid Simulations Involving Agent-based and System Dynamics Models.”

96.   Kasman et al., “Complex Systems Science”; Kasman and Kreuger, “Best Practices for Systems Science Research.”

97.   Schelling, “Dynamic Models of Segregation.”

98.   George E. Box and Norman R. Draper, Empirical Model-Building and Response Surfaces (New York: John Wiley & Sons, 1987).