Applications of system dynamics modelling to support health policy July 2015, Volume 25, Issue 3

Introduction

Health departments are continually challenged to design and implement policies that will enable health systems to meet the growing demand for high-quality and timely healthcare services. In addition, they are charged with addressing complex public health problems and developing effective systems and processes to support prevention (the prevention system) and deliver population health improvements that will reduce the rate of demand on healthcare services. However, health systems and prevention systems are complex and adaptive. They comprise many interacting components or ‘building blocks’, including governance, workforce, information and communication, service delivery, medicines and technologies, and financing. ¹ Heterogeneous groups of interconnected actors both within and outside these systems intervene at multiple levels through a variety of services and functions.² These components and actors interact in complex, nonlinear and unpredictable ways, creating problems for decision makers attempting to design and administer health policy through standard managerial approaches.² For example, a financial incentive to improve uptake of an initiative by primary care physicians may generate a cascade of effects through interrelated system building blocks that may result in governance challenges relating to accountability and transparency, information system pressures due to tracking and reconciling payments, and service delivery changes that overemphasise incentivised services. ¹

Often, the rules and heuristics that govern routine managerial decision making are inadequate to cope with the design, analysis and implementation of optimal policy actions for health problems embedded in complex systems.³ The use of mental models for long-term target setting are vulnerable to biased intuition which “often fail to account for real-world sources of inertia and delay and may suggest that things can change more rapidly than is actually possible”.⁴ In addition, well-intentioned policy decisions aimed at improving health system or prevention system performance by focusing on one subsystem can lead to unexpected and undesirable side effects, or ‘policy resistance’ in other parts of the system.⁵

Awareness of, and support for, the use of systems science modelling methods (namely, system dynamics, agent-based modelling and discrete event simulation) in the health sector has been gaining momentum as a result of their potential to address the challenges of policy making for complex problems.^{6, 7} However, these modelling methods are not new. System dynamics modelling has been widely and successfully applied to sectors such as engineering, economics, defence, ecology and business since the mid-1950s, and is underpinned by a well-established mathematical theory of nonlinear dynamics.⁸ As the value of systems science is being recognised, its methods are being implemented in the health sector in a number of important ways. It has been used to improve operational aspects of healthcare capacity and delivery, such as patient flows in emergency^{9, 10}, disease screening^{11, 12}, demand for services^{13, 14} and workforce requirements.^{15, 16} It is also being used to help map and understand complex relationships between multilevel risk factors that result in health problems such as childhood obesity¹⁷, substance abuse^{18, 19}, diabetes²⁰ and heart disease.²¹ Of particular promise to health policy makers is the use of systems science modelling methods to map the components of health systems and prevention systems, explore their interaction, and analyse policy options to support the most efficient and effective arrangements of the system.¹ In addition, these modelling methods may be used to develop a quantitative policy analysis tool to guide more efficient and equitable investment, and conscientious disinvestment, to address complex public health problems.²²

For example, in the design of public health policy in Australia and elsewhere, a broader understanding of risk factors for disease and increasing awareness of the social determinants of health have led to the development of more comprehensive, cross-sectoral strategies to tackle complex problems.²² However, comprehensive strategies may not represent the most efficient or effective approach to reducing disease burden at the population level. Rather, they may act to spread finite resources less intensively over a greater number of programs and initiatives, diluting the potential impact of the investment.²² In contrast, a systems approach (while encompassing holistic and cross-sectoral thinking) seeks to identify where best to focus public health action, and with what intensity, and is concerned primarily with what can be reasonably left out of strategies for prevention.²² The potential usefulness of systems science modelling methods lies in their ability to systematically and quantitatively analyse a range of intervention and policy options and identify leverage points in the system (places to intervene) where small inputs might result in large impacts.²³

System dynamics modelling has been a particularly appealing method because it lends itself to a participatory approach to model building. It aims to align stakeholder understanding of the underlying cause of a problem, achieve consensus on a course of action, and facilitate broader policy adoption and successful implementation.^{8, 24, 25} Given the increasing amount of literature promoting the value of system dynamics modelling as a tool to aid policy making, it is important to examine its actual applications in the health sector and explore any evidence of its effectiveness. Literature on the use of simulation modelling within healthcare was systematically reviewed in 2011.²⁶ The review encompassed simulation other than system dynamics modelling, was not limited to its applications to guide policy making, and did not examine evidence relating to the effectiveness of system dynamics modelling for health policy making.²⁶ Therefore, the aim of this systematic review is to identify studies investigating the effectiveness of health policies informed by system dynamics modelling, explore the range and nature of its applications, and identify approaches to potentiate the future use of systems science modelling methods in Australia to inform policy development in the health sector.

Method

Search strategy

A systematic electronic search was made to identify relevant articles published up to April 2015 from the PubMed, Web of Knowledge, Embase, ScienceDirect and Google Scholar databases. The grey literature was searched using the Google search engine. Key terms recognised as descriptors of the systems science discipline were used in the search strategy, which included: ‘system dynamics OR systems approach OR systems model OR systems science OR systems thinking’ AND ‘policy OR decision making’ AND ‘health’. The search was limited to English language publications. Reference lists of selected papers were also reviewed to identify any other eligible studies.

Study selection

Guidelines for study inclusion, quality assessment and data extraction outlined by the Cochrane Effective Practice and Organisation of Care (EPOC) Group were used.²⁷ EPOC guidelines are suitable for design and evaluation of intervention studies aimed at improving the delivery, practice and organisation of effective health services (including organisational, financial and regulatory interventions).²⁷ The scope of this review was the use of system dynamics modelling by any level of government as a tool to inform the design of health sector policy across the areas of prevention and health promotion, healthcare delivery, and financing and regulation. Papers eligible for inclusion were randomised controlled trials, controlled before-and-after studies, and interrupted time series reporting the effectiveness of policies developed using systems dynamics modelling. In addition, the review aimed to compile case studies detailing the use of system dynamics modelling in the design of health policy to explore the range and nature of its applications.

System dynamics modelling refers to the process of transforming a qualitative systems map into a mathematical model for simulating policy or intervention scenarios.²⁸ Given our interest was in policy environments and health systems with similar organisational and technological sophistication as Australia, the review was restricted to papers from the UK, western Europe, Canada, the US and New Zealand. Review papers were excluded, as were papers that described academic research that designed and used system dynamics models for health policy analysis but were not commissioned or applied in the development of ‘real-world’ policy decisions, or if they described models used for decision making at the clinical or organisational level rather than the system or population level. It was intended that data would be extracted on study design, setting, inclusion of a participatory approach, and primary outcomes (such as impact on health outcomes, or impact on safety and quality of healthcare delivery, as defined by each study). It was also anticipated that case studies would be synthesised using a coding frame and thematic analysis.

Results

Figure 1 shows the paper selection process. No analytic studies examining the effectiveness of health policies informed by system dynamic modelling were found. Six papers were identified that reported a total of eight case studies that used system dynamics modelling. Three of these case studies related to policy making in the prevention and health promotion context, and five related to policy development for healthcare delivery. No papers were found that applied system dynamics modelling in the health financing or regulation context (actuarial and financial models appeared to be the primary method used to inform policy making in this area). Three of the eight case studies used system dynamics modelling to map the arrangements of an existing healthcare delivery system and analyse the impact of policy scenarios on the capacity and behaviour of system components (e.g. infrastructure, workforce, financing). The remaining five case studies focused on the use of system dynamics modelling to analyse policy options for effective action to address specific health problems. Table 1 provides a brief description of these case studies.

Figure 1.     Flowchart of paper selection process (click to enlarge)

The few published case studies found by this review yielded insufficient data to justify using formal synthesis methods; however, the features and lessons learnt from these ‘real-world’ applications of system dynamics modelling to support health policy making are described below.

All eight case descriptions detailed the successful development of a system dynamics model used by decision makers for policy analysis. The three cases that included multidisciplinary stakeholders in the process of model design reported that a participatory approach:^{11, 21, 30}

• Facilitated cross-sector communication and synthesis of knowledge
• Developed a common ‘mental map’ for progress and consensus among stakeholders on optimal policy
• Augmented leadership capacity
• Allowed collaborative shaping of policy and legislative changes, leading to broader support for decisions made and catalysing action for implementation.

Two cases^{21, 30} reported the use of evidence from the literature to establish causal relationships between variables in the model. The other six cases relied solely on the synthesis of experiential knowledge of stakeholders. Failing to include evidence from empirical research where it is available may contribute to the perception that systems science modelling methods generally constitute ‘soft’ science and are not sufficiently rigorous to accurately inform policy making.

General benefits of these studies included increased awareness by policy makers of the dynamics of the health problem and how data can be used in a practical way to generate projections to guide policy decisions. The application of system dynamics modelling exposed inadequacies in the selection of indicators for activity reporting and the accuracy of data reported. It provided insight into where data collection systems may need to improve, and highlighted the value of collaboration with nonhealth sectors. The inclusion of financial data allowed decision making on investment and disinvestment, and facilitated exploration of implications of financial instruments such as fees and incentives on the affordability, accessibility and quality of services. Models also reportedly allowed rapid integration and use of new evidence for policy analysis; made trade-offs of policy options explicit; and acted as a vehicle for advancing controversial, contested and value-laden debates. Their use to explore the implications of policy options gave rise to policy scenarios that had not previously been considered.

Discussion

Although the literature promoting the usefulness of system dynamics modelling in public health and redesign of the healthcare system is considerable, the published and grey literature suggests its practical application to inform health policy is not widespread (although there is likely to be a number of unreported applications in the private sector where models are commercial-in-confidence). Rigorous evaluations of system dynamics modelling are essential to assess the effectiveness of policy decisions informed by these methods.

Table 1.     Summary of included papers

Although evaluation of the effectiveness of system dynamics modelling in health policy is lacking, it is important to recognise that system dynamics modelling is not an intervention. Rather, it is an analytical tool that can support policy makers to make more robust decisions in the presence of the significant uncertainty that complex problems create. The modelling is the first step in a broader process that includes policy adoption, alignment of stakeholder actions, implementation and maintenance, all of which are likely to influence whether the policy response has its projected impact. Therefore, evaluation of system dynamics modelling may need to focus on more proximal or intermediate factors such as shifts in (and alignment of) participating stakeholders’ understanding of a problem, changes to their perceptions regarding investment priorities, and commitment to implementing model findings agreed by the stakeholder group to be the optimal course of action.

There is a need to move beyond the rhetoric of the benefits of system dynamics modelling (and other systems science modelling methods) for health policy and explore how best to encourage uptake of their use and their evaluation. Reasons for poor uptake in the health sector may include a lack of locally available expertise in systems science modelling methods, unfamiliarity among policy makers with what these methods can offer, or concerns that these modelling methods present similar challenges to other ‘black box’ modelling approaches within which it can be difficult to establish confidence among stakeholders. Investment in communication to explain and promote systems science modelling methods as robust tools to inform policy in the health sector will need to be made with a range of stakeholders, including policy makers, donors, the media, risk and decision analysts, economists, clinicians, service providers and epidemiologists.^{33, 34} Building a national capacity of people with knowledge of, and expertise in, system science modelling methods will also support uptake of these tools by policy makers. Finally, building rigour into model design while maintaining stakeholder engagement, as well as aligning the modelling with the movement to increase uptake of evidence in policy development, may provide the impetus needed for their use.

The successful use and advantages reported from these applications of system dynamics modelling in the health policy setting appear to be consistent with the advocated benefits of systems science. However, there are several important limitations of these summarised findings. A lack of qualitative or quantitative data presented in these few case descriptions prevented the use of formal synthesis methods. Also absent from these papers were the rigorous capture of the perspectives of policy makers on the usefulness of the model as a tool for policy analysis and stakeholder engagement, an assessment of their learning relating to development and use of the model, and their commitment to model findings. In addition, the case descriptions were all provided by the people responsible for developing the models and who are therefore likely to have a vested interest in promoting their use; unsuccessful cases are likely to be underreported. The potential bias arising from these limitations highlights the need for caution in drawing conclusions from these successful applications of systems methods for health policy making. However, a systematic review and synthesis of 107 applications of system dynamics modelling in nonhealth sectors reported that 42 of 46 cases (where system change was the focus of the project) found that modelling conclusions had led to system changes.²⁴ In addition, participants of collaborative model building endeavours reported increased insights into the problem (in 96 of 107 applications), which led to the development and implementation of new solutions in half of the projects addressing real-world problems.²⁴ These findings provide some support for the benefits described in applications of system dynamics modelling to support policy making in the health sector summarised by the current review.

Conclusion

The potential of systems science modelling methods for health policy development is yet to be realised and there is a lack of evidence to assess their value. Further documentation and robust evaluation is required of their applications to support decision making and consensus on health policy. This will contribute to an evidence base that might lead to their more routine use by policy makers. Advances in software are allowing other systems science modelling methods to be more broadly accessible. This is facilitating the use of a participatory model building approach for sophisticated multimethod modelling (i.e. combining system dynamics modelling, agent-based modelling, and discrete event simulation), which can deliver to policy makers more powerful tools for informing the design of targeted, efficient, effective and equitable policy responses for complex problems in the health sector.

Making policy decisions about complex systemic health issues can be problematic. Available evidence to inform decision making can be vast, noncontextual and insensitive to the social and political milieu in which policy decisions are made. Systems science modelling methods permit the broader involvement of key stakeholders in a more explicit model development process, which may foster trust and transparency in the policy making process and accelerate policy adoption, implementation and health sector change.

Acknowledgements

This research was supported by The Australian Prevention Partnership Centre through the NHMRC partnership centre grant scheme (grant ID: GNT9100001) with the Australian Government Department of Health, NSW Ministry of Health, ACT Health, HCF, and the HCF Research Foundation. The contents of this paper are solely the responsibility of the individual authors and do not reflect the views of the NHMRC or funding partners.

Copyright:

© 2015 Atkinson et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.