The soft underbelly of complexity science adoption in policymaking: towards addressing frequently overlooked non-technical challenges

The literature we surveyed suggests that a range of management and institutional barriers has hindered the integration of complexity science into policymaking. Among the most dominant of these is a concern that there is limited interest in or understanding of complexity science and its related concepts and tools among policymakers and practitioners (El-Jardali et al., 2014; Finegood, 2021; Otto, 2008). Compounding this challenge is the difficulty of embedding complexity science into a field or institution, as this would require a paradigm shift and a drastic departure from the conventional way of dealing with problems (De Greene, 1994b). Without this change in thinking, it becomes increasingly difficult for policymakers to understand, accept and use complexity science within their work (Bale et al., 2015; Collins et al., 2007).

Further exacerbating the above challenges is that complexity science has faced critiques for being too ‘conceptual for policy’ and ill-suited for practical policy applications (Kwamie et al., 2021). Here. Critics suggest that the language and concepts of complexity science are unclear, ambiguous, and ill-suited for decision-makers and practical policy applications (Bale et al., 2015; Loosemore & Cheung, 2015). This sentiment is reflected by Stewart and Ayre (2001: 82), who observed that “the language and perspectives of these [systems thinking] approaches has never been harmonised with that of the policy-maker.” Stewart and Ayre argue that complexity science’s counterintuitive nature can be difficult to grasp and can lead to ‘uncomfortable conclusions’ about the limitations of policymakers to intervene within the complex systems they are tasked to govern. Authors such as Cairney (2012) suggest that translating complexity science and its concepts to the policy domain is incomplete, a sentiment echoed by Morçöl (2014, 2023). Harrison and Geyer (2021a) suggest that among the reasons for this difficulty is that complexity is a relatively young field in policy studies and that it, therefore, still lacks a unifying conceptualisation and language of the approach.

The interplay between the knowledge gaps, limited interest, and misalignment between the language of complexity and the policy domain creates a self-reinforcing cycle, further discouraging the adoption of complexity-informed policies due to the constrained enthusiasm for building capacity, allocating resources, and creating opportunities to apply complexity science (Bale et al., 2015; Haynes et al., 2020; Summers et al., 2015).

Conversely, there are concerns about when complexity-informed approaches are actually used in policymaking. As Levin et al. (2015) highlight, using complexity science for policy development can lead to more holistic proposals but ultimately make them unattainable. Such proposals may exceed or fall outside of the implementing body's institutional, political, or financial purview or capability, resulting in implementation failure. Such issues can be further compounded by bureaucracies' tendency to incentivise efficiency, standardisation, linearity, compliance, and uniformity, further impeding complexity-informed approaches that embrace change and uncertainty (Kwamie et al., 2021; Young, 2017).

While not discussed as frequently as other challenges in the studies we reviewed, some authors have indicated that political considerations can be considerable hindrances to the adoption of complexity science into policymaking (Hartman, 2016). Currie et al. (2018) indicate that one such barrier is the incompatibilities between the time needed to build a model or develop a complexity-informed proposal and the timelines of politicians and public decision-making. Currie et al. suggest that such incompatibilities result in rushed and oversimplified proposals and models, hindering their effectiveness. Additionally, Currie et al. also emphasise that potential issues may also arise due to conflicting goals of the political process, which focuses on short-term outcomes (Barbrook-Johnson et al., 2019), and that of complexity approaches and modelling, which seek long-term and holistic solutions.

This dissonance is exacerbated further by the inherent incompatibility between complexity science and policymakers' traditional, reductionist worldview, who often seek to simplify and control complex systems by eliminating complexity rather than embracing it (Wilkinson et al., 2013). Harrison and Geyer (2021a) echo this sentiment but further highlight that embracing complexity would require governments (and politicians, by extension) to acknowledge their limited control and influence over the policy processes and outcomes within the systems they govern. However, Harrison and Geyer also observe that embracing a complexity approach and acknowledging limited control might reduce elected officials' accountability. Loosemore and Cheung (2015) echoed a similar sentiment by suggesting the potential for complexity-informed approaches to diffuse the allocation of risk and responsibility, thereby reducing accountability when issues arise. They note that this is particularly relevant in high-risk high-responsibility domains, such as construction. To address this concern, Harrison and Geyer (2021a) advocate for a balanced approach, stating that “we need to combine a governmental acknowledgement of the limits to its powers with a societal recognition of the complexity of the governance process and the need to still hold elected policy actors to account in a meaningful way” (Harrison & Geyer, 2021a: 50).

Similarly, other studies have highlighted that most complexity-informed approaches fail to account for power dynamics or the impact of human values in their frameworks (Houchin & MacLean, 2005; Levy et al., 2016). For example, Lane and Oliva (1998) note a lack of social-political theory within systems dynamics, a subbranch of complexity science. Haynes (2018) further suggests that one reason for neglecting values in complexity-informed approaches within social and political science is the historical importation of complexity science from the natural sciences, which typically disregard social norms and values.

The literature also identifies potential barriers to integrating complexity science into policymaking arising from institutional or legal considerations (Currie et al., 2018; Shepherd, 1997). Frohlich et al. (2018), for instance, describe how laws can sometimes impose legal restrictions on adaptive management practices, an approach informed by complexity science. They provide an example of legislation primarily focused on safeguarding an individual species instead of protecting and managing the larger system. They also cite several examples of legislation limiting adaptive management approaches, including regulatory fragmentation, spatial boundaries of jurisdiction, and the mismatch between legal land ownership boundaries and ecosystem boundaries.

Linked to institutional challenges, the interdisciplinary nature of complexity science presents significant challenges to its adoption in the policymaking process. Several authors have reported a lack of collaboration between researchers, institutional departments, and disciplines, hindering the effective implementation of complexity-informed approaches (Schimel et al., 2015; Schlüter et al., 2014). Such challenges highlight the crucial need for fostering interdisciplinary collaboration to bridge the gaps between disciplines and facilitate the effective integration of complexity science into policymaking.

As with many things, certain costs are associated with adopting, using, or implementing a proposal, approach, or tool. Because of its nature, complexity science might be more vulnerable and disproportionately impacted by these costs compared to other established approaches. From the literature studied, we identified a wide range of potential costs associated with, among other things, the development of any tool or model, particularly those using complexity science. The costs identified include general financial costs and funding limitations (Druckenmiller et al., 2007; Lindkvist et al., 2020); financial cost of data collection, creation, or purchasing (Burgess et al., 2020; Summers et al., 2015); time needed to develop a model, collect data, or train users (Terzi et al., 2019); manpower and expertise requirements (Astbury et al., 2023; Balajthy, 1988; Zhuo & Han, 2020); computational costs to develop and run models (Pan et al., 2022; Wen & Li, 2021), which are compounded by computational complexity (Nguyen et al., 2021; Taeihagh et al., 2014).

From a more practical perspective, the literature points to several challenges related to the limited adoption, utilisation, and deployment of complexity science and its related applications within organisations as an issue (Ligmann-Zielinska, 2009; Sharma-Wallace et al., 2018). One reason for the limited uptake of complexity science is that many complexity-informed applications are too domain-specific and cannot be used in broader contexts (Moallemi et al., 2021; Taeihagh et al., 2014). While complexity-informed approaches are not widely used in general policymaking, some limited exceptions exist where complexity-based approaches are the mainstream, such as traffic simulations and epidemiology (Wilkinson et al., 2013). Furthermore, Torrens et al. (2013) note that reductionist tools are easier to work with and understand than complexity-informed tools, even though they do not accurately represent reality. The reluctance to adopt new complexity-informed approaches and tools is likely further hindered because of existing tools, making users and policymakers more reluctant to use different and potentially more complex tools (Cockerill et al., 2007; Rich, 2020). This is especially true as many traditional tools (using statistical analysis) can produce predictions with confidence intervals, encouraging additional trust in the results (Maglio et al., 2014).

We also highlight a few other reasons for the limited uptake of complexity-informed approaches or tools identified from the literature. These challenges include usability issues of applications (Druckenmiller et al., 2007); time, effort, and data required to develop a tool or model (Ecem Yildiz et al., 2020); mismatches between a model's intended purpose (teaching) and the way the model is used (decision support) in practice (Allison et al., 2018); lack of a champion within an institution for complexity science or an application (Shepherd, 1997). We also noted issues of trust in complexity-informed tools and methods, which are linked to the challenges discussed here. However, issues of trust are discussed in more detail in the next section. To summarise the silent challenges identified under the broader thematic group of management, cost, and adoption challenges, a synopsis of the main points has been provided in Table 2.

From the non-technical issues we identified from the literature, challenges related to the trust in complexity science and its applications and methods were among those cited most frequently. Trust in an approach, method, or application is essential, especially for policymakers (Lacey et al., 2018) who must make decisions that are often long-lasting, non-reversible, and can impact many people. Thus, building and maintaining trust in complexity science is essential before those making decisions are willing to embrace it and an alternative approach.

Trust in complexity-informed approaches can be undermined in numerous ways. For example, Ibrahim Shire et al. (2020) highlight, within the context of health care, that the participants not involved in a model building exercise (notably the managers) were less convinced of the model's validity and were unwilling to claim ownership compared to participants involved from the beginning of the process. Vermeulen and Pyka (2016) reflect a similar sentiment by suggesting that the lack of policymakers’ involvement in the modelling process erodes trust and understanding of the process and its outputs. As noted by Levy et al. (2016), the challenge of model validation, particularly in ABM, also contributes to the lack of trust in complexity-informed models. As a distinct issue related to trust, Currie et al. (2018) identify that the problem may not lie with the complexity science application itself but rather how the application is used. Their study implies that working in a multidisciplinary team would likely be most effective in some cases, such as group system dynamics modelling.

Several authors (Kwamie et al., 2021; Wainwright & Millington, 2010) highlight the scarcity of successful deployment of complexity-informed policies as a reason for the hesitancy to adopt complexity into policymaking. Thus, decision-makers are reluctant to put their trust in unproved systems or approaches and seek tangible examples of the efficacy of modelling or an approach (Haynes et al., 2020; Maglio et al., 2014). To gain acceptance, complexity and its modelling applications must showcase their value through successful real-world applications (Haynes, 2018). However, Lorscheid et al. (2019) highlight a compounding issue to this requirement: complexity applications, like Agent-Based Modelling (ABM), need time to mature. Lorscheid et al. cite the time required for calculus to be accepted as a historical parallel. From this point of view, can it be that complexity science is still not yet ready for widespread use despite being around for over 80 years? Lorscheid et al., however, also indicate that approaches such as ABM are becoming more accepted within ecology and that there are indications that the ABM method is starting to mature. Axtell and Shaheen (2021), while optimistic about the future of ABMs, also caution that methods like ABMs are not yet ready to be dependable tools for policymaking due to the technical challenges that still need to be addressed (see Axtell and Shaheen (2021) and Levy et al. (2016) for a discussion of the challenges related to ABM).

Beyond examples of success, Wainwright and Millington (2010) argue that models like ABM need to demonstrate explanatory power before they will be accepted. However, the literature often cites concerns regarding the results' limited predictability and reliability (Allison et al., 2018; Axtell & Shaheen, 2021; Burgess et al., 2020). While a technical issue, the limited predictability of the models primarily stems from the impacts of sensitivity to initial conditions, path dependencies, and stochasticity (Bale et al., 2015; Levy et al., 2016). Additionally, as Whitfield (2013) notes, the increasing complexity and limited reliability of predictions, especially for complex models like those associated with climate modelling, hinder non-experts from understanding the models and results. This erosion of trust and acceptance makes such models more open to politically motivated challenges like those increasingly captured by climate change deniers. Consequently, using the outcomes of such models is more difficult when used to inform climate policy. Astbury et al. (2023) highlight several additional challenges related to modelling: a lack of trust in models as evidence, model complexity that can alienate stakeholders, and stakeholder struggles with interpreting complex results. Notably, they identify the "tension around model complexity," where models must be complex enough to represent the system adequately while remaining interpretable.

Mercure et al. (2016) and Banozic-Tang and Taeihagh (2022) suggest that rather than focusing solely on the complexity or simplicity of models, greater emphasis needs to be placed on the importance of the research-policy interface in relaying the results of models and scientific research findings to policy circles, highlighting the challenges of communication and reporting (discussed in Sect. 5.2.3). These findings underscore the critical need to bridge the gap between the research community and policymakers. Researchers must effectively communicate complexity science's value and limitations, while policymakers must be open to embracing uncertainty and holistic perspectives to address complex societal challenges.

A different aspect of trust discussed across several studies was the lack of transparency in models (Lindkvist et al., 2020; Taeihagh et al., 2014). For example, Torrens et al. (2013) point out that ABMs have been critiqued for being ‘black-box’ models because the simulation does not explicitly show the mechanisms generating emergent behaviours. This lack of transparency can lead policymakers to question the internal validity of the model and doubt its value and related policy recommendations (Vermeulen & Pyka, 2016). Ligmann-Zielinska (2009) underscore the necessity for the transparency and availability of a model’s code to address these concerns. However, Iwanaga (2021) contend that mere access to a model's code doesn’t equate to transparency. Additional contextual information about the model is needed to assess its suitability regarding function and purpose (Burgess et al., 2020; Schlüter et al., 2019). While transparency is a legitimate concern, decision-makers must also consider security issues (Dorri et al., 2018). For example, Luck et al. (2004) describe trust concerns about the safety of multi-agent systems because of agents' self-adaptive nature.

In stark contrast to the challenges discussed above, some studies suggest that policymakers might readily accept the results of models without critical reflection on their purpose, reliability, assumptions, or value (Whitfield, 2013). Allison et al. (2018) and Mercure et al. (2016) note that some models initially designed to enhance social learning, understanding of a system, or built with unreliable data are used as decision-support systems to inform policy. In these cases, the models are used outside of the modeller’s intention, with factors such as the model limitations, reliability or predictability not considered. While no research has explicitly been done on the topic that the authors are aware of, the reliance on models not built for the purpose might also impact the long-term acceptance of complexity science and its applications.

Understanding and accepting complexity science is arguably one of the most critical factors for its wider adoption within policymaking. However, as De Greene (1994b: 445) states,

“the basic challenge facing systems thinkers-and those policymakers, decision-makers, educators, and others who might benefit from systems advice – is not more data, more information systems, more computers, more money, and so on. The challenge is rather for all of us to restructure our very way of thinking.”

Despite its significance, the reviewed literature we studied seldom discusses the challenge of understanding or acceptance of complexity science, particularly within the context of policy and decision support (only 17 instances were identified within the text studied). The limited discussion might be because complexity challenges the ontological and epistemological assumptions of traditional ways of thinking (Morçöl, 2014). The limited discourse on the topic might be because altering beliefs and paradigms towards a more systemic approach is likely one of the most daunting barriers to the widespread adoption of complexity science in policymaking (Mann & Sherren, 2018). Similarly, Cockerill et al., (2007: 39) indicate that if challenges to a person’s beliefs are presented, through a model for example, then the output of such a model is more likely to be ignored than accepted. This suggests, as noted above, that it might be best to include those who are sceptical of complexity-informed approaches into the processes from the beginning to build trust and understanding in the process and outcomes.

Other studies discussing the challenge of improving the understanding and acceptance of complexity science have typically done so in the context of university teaching (York & Orgill, 2020) or training settings (Haynes et al., 2020). These studies highlight the lack of teaching material related to complexity science as a significant issue (Flynn et al., 2019). Additionally, educating students to use complexity science effectively requires support from educators, who themselves require training in complexity thinking before being able to teach students (Schultz et al., 2021). Communicating complexity science concepts to stakeholders and policymakers presents further challenges, similar to those faced in education (Collins et al., 2007; Ibrahim Shire et al., 2020). Stakeholders and policymakers involved in modelling processes can find the concepts or processes difficult and overly complex, leading to information overload (Morais et al., 2021; Weeks et al., 2022). Even well-educated professionals struggle to grasp certain complexity science concepts, such as accumulation (Cronin et al., 2009; Sterman et al., 2015).

A significant challenge to the broader adoption and application of complexity science lies in the inability of policymakers to understand what complexity is and what exactly it offers. While some may point the finger solely at policymakers for this lack of understanding, the blame cannot be laid entirely at their feet. Complexity science has been critiqued for its inherent ambiguity, terminology, and conceptual framework inconsistencies, as noted by several authors. This ambiguity manifests in several ways, hindering both comprehension and collaboration within the field, ultimately limiting its reach and impact. Firstly, the absence of a unifying definition and theoretical framework within complexity science creates a conceptual vacuum, leaving policymakers with little to grasp onto. This issue, highlighted by Kok et al. (2021) and O’Sullivan, (2004), is further compounded by the relative novelty of the field. Scholars are still grappling with its core concepts and methodologies, as Cairney (2012: 352) observes: "the first difficulty with complexity theory is that it is difficult to pin down when we move from conceptual to empirical analysis." Secondly, the terminology employed within complexity science is often plagued by vagueness and inconsistency. As Finegood (2021) and Haynes et al. (2020) emphasise, this lack of standardisation creates significant obstacles for researchers at all levels, hindering their ability to engage effectively with the field. Houchin and MacLean (2005) further echo this concern, critiquing the "variety of definitions, the doubts expressed as to whether it is a theory, theories or a framework, and the different meanings given to the terminology associated with complexity" as detrimental to the field's coherence and credibility. Finally, the terminology's ambiguity extends beyond individual words, impacting the overall conceptual landscape of complexity science. As Teixeira de Melo et al. (2019) point out, different approaches within the field can ascribe different meanings to the same concepts, leading to misinterpretations and hindering effective communication and collaboration between researchers from diverse backgrounds and disciplines. This lack of a shared language, as Harrison and Geyer (2021a: 47) emphasise, further exacerbates the challenges, as "[complexity] is not a unified field with a unifying interdisciplinary language." Given these arguments, addressing the issues of ambiguity and inconsistency is crucial to unlocking the potential of complexity science in policymaking.

The literature has also noted additional barriers to adopting complexity into policymaking. These include policymakers' lack of acceptance and willingness to use complexity-informed models (Levy et al., 2016). Cockerill et al. (2007) suggest that policymakers' limited willingness to use models is based on ‘intentional ignorance’, explaining that in cases where models address controversial topics and provide meaningful insights, decision-makers might ignore the results and fail to address the issue, opting to maintain the status quo, particularly if decision-makers already having existing models or tools (duelling models). Policymakers are, therefore, unwilling to adopt new applications, especially if existing applications suggest some form of certainty or predictability, which is harder to achieve with dynamic and systems models (Cockerill et al., 2007).

Science communication is a significant challenge for many domains of study (Bucchi, 2019), with complexity science being no exception. However, complexity science faces a more significant challenge than many other fields, as it can be challenging to understand and produce unexpected or counter-intuitive results (Stewart & Ayres, 2001). As noted previously, ambiguous language and terminology impact the understanding of complexity science. However, the same linguistic and conceptual inconsistencies, as well as the technical and complex nature of the concepts (Loosemore & Cheung, 2015), also hinder the ability of those using complexity to communicate the concepts and the results effectively (Bale et al., 2015).

Furthermore, Steger et al. (2021) and Taeihagh et al. (2013) highlight that without adequate tools and visualisations, non-technical stakeholders and participants might be overwhelmed by the amount of information required to use complexity-informed approaches. Therefore, as Šucha (2017: 23) states, “there is a job to do in helping policy makers and politicians to develop simple messages to persuade the public of the merits of the solutions arrived at using complex science”, as even the best and most accurate models are of no use if the results cannot be effectively communicated (Lehuta et al., 2016). However, the results of complexity science approaches can be challenging to communicate effectively. For example, describing how initial assumptions in a process can result in specific model outcomes is challenging, particularly as models become more complex (Burgess et al., 2020). Additionally, temporal disturbances, random perturbations, path dependencies, agent learning, model initialisation, and various types of uncertainty add to the burden of communication (Bale et al., 2015; Taeihagh, 2015; Vermeulen & Pyka, 2016).

Linked to the challenge of communication, some authors (Lehuta et al., 2016; Levy et al., 2016) have noted the lack of standards for the evaluation, benchmarking, or implementation of various complexity science models, which hinders trust in the approach and makes communication of a model and its results more challenging. Similarly, the lack of clear guidance on actually applying complexity science-informed approaches in policymaking is also an issue (Currie et al., 2018; Mora et al., 2012) and can produce messy or confusing implementation of an application (Zukowski et al., 2019). For example, many complexity-informed models allow policymakers to test various policy interventions. However, as Amagoh (2016: 3) states, such a “model gives little guidance as to which aspects of the system should be manipulated to achieve policy objectives. In other words, it fails to provide a way forward when constituents of a system are in conflict with each other”. For complexity-informed methodologies to be taken seriously and utilised effectively in policymaking, they must move beyond simply describing a system and its potential future outcomes. There is a critical need to develop methods that provide concrete guidance on how and where to intervene within a system to achieve desired policy goals while minimising unintended consequences and negative externalities.

Table 3 summarises the key sticking points related to the communication, implementation, and acceptance of complexity science in policymaking identified in the literature. Addressing these challenges is crucial to unlocking the transformative potential of this field and fostering its widespread application in policy development and implementation.