Development of effective policy responses to address complex public health problems can be challenged by a lack of clarity about the interaction of risk factors driving the problem, differing views of stakeholders on the most appropriate and effective intervention approaches, a lack of evidence to support commonly implemented and acceptable intervention approaches, and a lack of acceptance of effective interventions. Consequently, political considerations, community advocacy and industry lobbying can contribute to a hotly contested debate about the most appropriate course of action; this can hinder consensus and give rise to policy resistance. The problem of alcohol misuse and its associated harms in New South Wales (NSW), Australia, provides a relevant example of such challenges.
Dynamic simulation modelling is increasingly being valued by the health sector as a robust tool to support decision making to address complex problems. It allows policy makers to ask ‘what-if’ questions and test the potential impacts of different policy scenarios over time, before solutions are implemented in the real world. Participatory approaches to modelling enable researchers, policy makers, program planners, practitioners and consumer representatives to collaborate with expert modellers to ensure that models are transparent, incorporate diverse evidence and perspectives, are better aligned to the decision-support needs of policy makers, and can facilitate consensus building for action.
This paper outlines a procedure for embedding stakeholder engagement and consensus building in the development of dynamic simulation models that can guide the development of effective, coordinated and acceptable policy responses to complex public health problems, such as alcohol-related harms in NSW.
Complex public health problems present challenges to decision makers that can hinder effective action. These challenges include uncertainty relating to what interventions will deliver the greatest impact; competing views and agendas that can undermine effective implementation of interventions; and the need to remain responsive to changing circumstances that can increase service needs, while operating within resource constraints.1-4 As a result of such challenges, the health sector is increasingly recognising the value of dynamic simulation modelling methods. These methods have long been used by other sectors to help address complex problems and optimise the use of limited resources.5 They include system dynamics, which captures population-level influences and whole-system dynamics; agent-based modelling, which captures individual heterogeneity and social network influences; discrete event simulation, which analyses processes or sequences of events; and the integration of these techniques.6 Dynamic simulation modelling provides a platform for systematically integrating diverse evidence sources through a process of mapping, quantifying, testing, validating and refining a model of the underlying causal mechanisms that drive a complex problem in a particular context. These dynamic models provide policy makers with a tailored ‘what-if’ tool to examine the likely impacts of different policy and program options (applied individually or in combination), as well as the comparative cost and system implications of these options over the short and longer term.5
Globally, alcohol consumption results in approximately 3.3 million deaths each year, and the costs of alcohol-related harms amount to more than 1% of gross national product in high-income countries.7,8 In Australia, alcohol is widely used and accounts for an estimated 5.1% of the total burden of disease and injury.9 The harms from alcohol (including ill health, chronic disease, crime, road traffic accidents and lost productivity) have been conservatively estimated to cost Australia $15.3 billion per year.10
Evidence suggests that a range of individual, sociocultural, economic and environmental factors contribute to the problem of alcohol-related harms, resulting in many potential points of intervention. However, despite a range of evidence-based options for intervening11, there is uncertainty about the harm-reduction benefits of such interventions in particular contexts, and the effects of combining them.1 Moreover, there are diverse views about the acceptability of evidence based harm-reduction options, advocacy for options that lack evidence, and industry concerns about interventions that are likely to impose additional costs or limitations on businesses.1-4 Policy development in such a challenging context would benefit from robust and transparent decision-support tools, for which dynamic simulation modelling is well suited.
Successful development of policy responses for reducing alcohol-related harms depends on stakeholder support for the proposed policy actions.3 Transparency and stakeholder engagement in the policy development process, and consensus building are considered to be essential to gaining this support.12 Facilitating stakeholder participation in collaborative simulation model building is one approach that can be used to improve stakeholder communication, advance contentious debates, gain consensus among stakeholders, aid transparency and translation of model outcomes to stakeholders, and garner broader support for collaborative action.13-16 Recent advances in simulation modelling software and more user-friendly interfaces have made participation and the achievement of such outcomes more feasible. A procedure is required for embedding stakeholder engagement and consensus-building processes in the development of dynamic simulation models.
Several guidelines and frameworks exist for best-practice methods of working with small groups to facilitate simulation model conceptualisation, formulation, quantification, calibration and validation, as well as conducting policy analysis and simulation experiments.17-20 The procedure described in this paper draws on such guidelines and frameworks, and on the findings of a systematic review of 107 group model-building projects (almost all of which were in nonhealth sectors).16 The procedure builds on this information to address the need for broader consensus building in the development of public health policy.12,21
The procedure was applied in 2015 to develop a dynamic simulation model of possible policy actions to reduce alcohol-related harms in New South Wales (NSW), Australia. It was developed and applied through collaboration between The Australian Prevention Partnership Centre, NSW Health, local and national alcohol researchers, clinicians, program planners and policy officers.
The procedure consisted of three distinct phases: 1) project planning and engagement; 2) participatory model building and model validation; and 3) consensus building for policy actions (Figure 1.)
Figure 1. Procedure for applying participatory simulation modelling to complex public health policy questions (click to enlarge)
Phase 1 defined the purpose, scope and boundaries of the modelling project to ensure that the simulation modelling tool could be developed in a policy-friendly time frame. Hence, careful planning and early engagement of stakeholders were priorities. The following steps were undertaken in Phase 1.
Step 1.1: Core project staff were identified and assembled, as follows:
Step 1.2: A meeting was held with policy partners (end users of the model) to:
Step 1.3: A project outline was developed and circulated (based on information derived from Step 1.2); experts and key stakeholders were engaged; and modelling workshop materials were developed (i.e. presentations, manuals, and resources to support participatory mapping activities).
Through three participatory workshops, the model-building team collaboratively mapped the key risk factors and likely causal pathways leading to the outcomes of interest. This map was used to develop a computational simulation model considered to be a plausible representation of the problem of alcohol-related harms in NSW. The key products of Phase 2 were a conceptual map and computational model of the complex problem and accompanying documentation to provide transparency in model structure, parameterisation and assumptions.
Step 2.1: The multidisciplinary model-building team was prepared for participation in the workshops by being:
Step 2.2: Two 1-day workshops were conducted (ideally 8–12 weeks apart), proceeding through a process of problem mapping, followed by model conceptualisation, formulation and initial quantification. The workshops included discussion of all potential variables and causal pathways initially considered to be important drivers of the problem. These were later modified as relationships and hypotheses were tested. All assumptions made during the process of building and quantifying the model were documented for transparency and to allow later testing of alternative assumptions. Key assumptions were also discussed and debated during the workshops. Existing data sources (inputs) to inform the structure and parameterisation of the model (Table 1, available from: www.saxinstitute.org.au/table-1-types-of-content-knowledge-contributed-by-participants-and-types-of-literature-and-data-used-2) were identified by the team, and processes required to gain access to these data were discussed. Sessions during the workshops (Appendix A, also available from: www.saxinstitute.org.au/appendix-a-modelling-alcohol-related-harms-agendas-for-workshops-1-3-2) also involved discussion and prioritisation of model outputs, interventions to be integrated (Box 1), and features that might provide appealing visualisation and functionality for the user interface.
Box 1. Interventions prioritised by stakeholders for inclusion in the model of alcohol-related harms in New South Wales
Step 2.3: Between workshops, the core project staff (modeller, JA, EO and GM) built a computational model from the conceptual map. They engaged with key individuals from the model-building team for expert advice, and for direction to relevant literature and data. Further searching of relevant literature was conducted iteratively to supplement information provided by workshop participants. The core project staff led the calibration, testing and validation process (including comparison of model outputs against real-world trends in alcohol-consumption behaviours, harms and health-service impacts), and developed the user interface. The interface allows end users to interact with the model and experiment with policy scenarios.
Step 2.4: In a further half-day workshop, the model was presented to the model-building team for verification, discussion, consensus, feedback of results, and further input on preferred visualisation of model outputs.
Features that were integral to successful consensus building among the multidisciplinary model-building team were the participatory approach to model development; iterative refinement of the model, based on feedback from the model-building team; validation to demonstrate how well the model reproduced a broad range of historical data patterns; and transparency regarding model assumptions. The consensus-building phase is an ongoing process in which the tool will be used to facilitate policy dialogues with a broader range of stakeholders. Stakeholders may interact with the tool to conduct their own policy experiments, discuss the costs and impacts generated by the model, and better understand the trade-offs between different scenarios. It is anticipated that broader engagement of stakeholders with the tool will facilitate wider buy-in and support implementation.
The outputs of the model to date confirm what system scientists have long stated about intervening in complex systems, and the value of dynamic simulation modelling. For example, the model outputs demonstrated that the effects of combining interventions are not necessarily additive. Simulation experiments of state-wide application of the 2014 NSW liquor licence reforms that are currently in place in a number of entertainment precincts in Sydney (comprising 3 am ‘last drinks’ in licensed venues, 1 am ‘lockouts’ and 10 pm closing of bottle shops), combined with a scale-up of treatment services for heavy drinkers, produced forecast reductions in alcohol-related harms, emergency department presentations and hospitalisations that were greater than the sum of impacts seen with each intervention applied individually. Some simulation experiments demonstrate effects of interventions that are counterintuitive and unintended. Others show that the impacts of some interventions would likely occur over longer periods, with effects not evident within the initial 1- to 2-year evaluation period or a 3-year funding cycle. A more detailed description of the model itself and its results for particular policy actions will be given elsewhere.
The model can be updated and maintained as new data or evidence come to hand, through ongoing collaboration between researchers, the model administrators and policy makers.
The design of effective responses to complex problems such as reducing alcohol-related harms raises a range of challenging questions that cannot be adequately answered by the existing systematic reviews of research evidence. Simulation modelling provides policy makers and practitioners with a unique decision-support tool that integrates existing data, research evidence, and expert and local knowledge to examine the likely impacts of different policy and program scenarios simulated over time. Such models can help identify interventions that act synergistically, anticipate unintended consequences, and contribute to business cases for longer-term investment in effective responses to complex problems. Stakeholder participation in collaborative model building and model transparency can provide valuable learning about complex problems, enable constructive dialogues and engagement in the policy development process, and support consensus building for effective action. The model has the capacity to test a large range of scenarios to determine their likely short-term and longer-term impacts on alcohol-related harms in NSW. It can provide decision makers and practitioners with a robust basis for negotiating with stakeholders on effective and appropriate actions to reduce alcohol-related harms, and to inform further research.
We thank Sally Redman, Sonia Wutzke, Alan Shiell and Nate Osgood for their valuable contributions and advice during various stages of the planning and implementation of this work. The work was funded by the National Health and Medical Research Council of Australia (NHMRC) through its Partnership Centre grant scheme (grant GNT9100001). NSW Health, ACT Health, the Australian Government Department of Health, the Hospitals Contribution Fund of Australia and the HCF Research Foundation have contributed funds to support this work as part of the NHMRC Partnership Centre grant scheme. The contents of this paper are solely the responsibility of the individual authors and do not reflect the views of the NHMRC or funding partners.
© 2017 Atkinson et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.