Dynamic simulation modelling of policy responses to reduce alcohol-related harms: rationale and procedure for a participatory approach February 2017, Volume 27, Issue 1

Introduction

Complex public health problems present challenges to decision makers that can hinder effective action. These challenges include uncertainty relating to what interventions will deliver the greatest impact; competing views and agendas that can undermine effective implementation of interventions; and the need to remain responsive to changing circumstances that can increase service needs, while operating within resource constraints.^1-4 As a result of such challenges, the health sector is increasingly recognising the value of dynamic simulation modelling methods. These methods have long been used by other sectors to help address complex problems and optimise the use of limited resources.⁵ They include system dynamics, which captures population-level influences and whole-system dynamics; agent-based modelling, which captures individual heterogeneity and social network influences; discrete event simulation, which analyses processes or sequences of events; and the integration of these techniques.⁶ Dynamic simulation modelling provides a platform for systematically integrating diverse evidence sources through a process of mapping, quantifying, testing, validating and refining a model of the underlying causal mechanisms that drive a complex problem in a particular context. These dynamic models provide policy makers with a tailored ‘what-if’ tool to examine the likely impacts of different policy and program options (applied individually or in combination), as well as the comparative cost and system implications of these options over the short and longer term.⁵

The problem of alcohol misuse

Globally, alcohol consumption results in approximately 3.3 million deaths each year, and the costs of alcohol-related harms amount to more than 1% of gross national product in high-income countries.^7,8 In Australia, alcohol is widely used and accounts for an estimated 5.1% of the total burden of disease and injury.⁹ The harms from alcohol (including ill health, chronic disease, crime, road traffic accidents and lost productivity) have been conservatively estimated to cost Australia $15.3 billion per year.¹⁰

Challenges for reducing alcohol-related harms

Evidence suggests that a range of individual, sociocultural, economic and environmental factors contribute to the problem of alcohol-related harms, resulting in many potential points of intervention. However, despite a range of evidence-based options for intervening¹¹, there is uncertainty about the harm-reduction benefits of such interventions in particular contexts, and the effects of combining them.¹ Moreover, there are diverse views about the acceptability of evidence based harm-reduction options, advocacy for options that lack evidence, and industry concerns about interventions that are likely to impose additional costs or limitations on businesses.^1-4 Policy development in such a challenging context would benefit from robust and transparent decision-support tools, for which dynamic simulation modelling is well suited.

The importance of a participatory approach

Successful development of policy responses for reducing alcohol-related harms depends on stakeholder support for the proposed policy actions.³ Transparency and stakeholder engagement in the policy development process, and consensus building are considered to be essential to gaining this support.¹² Facilitating stakeholder participation in collaborative simulation model building is one approach that can be used to improve stakeholder communication, advance contentious debates, gain consensus among stakeholders, aid transparency and translation of model outcomes to stakeholders, and garner broader support for collaborative action.^13-16 Recent advances in simulation modelling software and more user-friendly interfaces have made participation and the achievement of such outcomes more feasible. A procedure is required for embedding stakeholder engagement and consensus-building processes in the development of dynamic simulation models.

Methods

Development of the procedure

Several guidelines and frameworks exist for best-practice methods of working with small groups to facilitate simulation model conceptualisation, formulation, quantification, calibration and validation, as well as conducting policy analysis and simulation experiments.^17-20 The procedure described in this paper draws on such guidelines and frameworks, and on the findings of a systematic review of 107 group model-building projects (almost all of which were in nonhealth sectors).¹⁶ The procedure builds on this information to address the need for broader consensus building in the development of public health policy.^12,21

Application of the procedure

The procedure was applied in 2015 to develop a dynamic simulation model of possible policy actions to reduce alcohol-related harms in New South Wales (NSW), Australia. It was developed and applied through collaboration between The Australian Prevention Partnership Centre, NSW Health, local and national alcohol researchers, clinicians, program planners and policy officers.

The procedure

The procedure consisted of three distinct phases: 1) project planning and engagement; 2) participatory model building and model validation; and 3) consensus building for policy actions (Figure 1.)

Figure 1.     Procedure for applying participatory simulation modelling to complex public health policy questions (click to enlarge)

Phase 1: project planning and engagement (time frame – 1 month)

Phase 1 defined the purpose, scope and boundaries of the modelling project to ensure that the simulation modelling tool could be developed in a policy-friendly time frame. Hence, careful planning and early engagement of stakeholders were priorities. The following steps were undertaken in Phase 1.

Step 1.1: Core project staff were identified and assembled, as follows:

• Project lead (JA). The project lead facilitated the brokering and management of the project, and was primarily responsible for engaging and maintaining relationships with stakeholders and policy partners (end users of the model) throughout the process. The project lead shared the duties of facilitating modelling workshops with the lead domain expert, and oversaw model development, the production of supporting documentation and external communications
• Lead domain expert (JW). A well-respected alcohol expert was identified and engaged as a project sponsor and lead domain expert. In collaboration with the policy partners, the lead domain expert played an integral role in identifying and recruiting key stakeholders to participate in model development, and jointly facilitated the modelling workshops
• Dynamic simulation modeller (computer programmer) and data analyst. This person was contracted to build the model according to agreed specifications
• Research officer (EO). The research officer was responsible for logistical arrangements, and liaising directly with the modeller and modelling workshop participants to source and manage evidence and data requirements of the model-building process
• Expert technical adviser (GM). The technical adviser provided high-level oversight of model conceptualisation and development to ensure that the model was computationally efficient, interactive, and deliverable in a policy-friendly time frame.

Step 1.2: A meeting was held with policy partners (end users of the model) to:

• Clearly define the aspects of the problem to be modelled, including its scope and boundaries, and identify key outputs of interest (e.g. alcohol-related mortality, health service use, prevalence of alcohol-related chronic disease, costs and cost savings)
• Identify and select possible intervention options to be included in, and tested by, the model. The inventory of possible interventions included existing programs and interventions, those currently under policy consideration, those supported by evidence but not implemented, and those recommended by stakeholder groups but not supported by evidence
• Identify a small group of experts and key stakeholders (10–15 people) who would comprise the collaborative model-building team. Members of this multidisciplinary team were purposefully selected on the basis of their diverse perspectives and recognition as research, clinical, policy and community leaders in the following fields: drug and alcohol clinical service delivery; population health policy making and service delivery; epidemiology; social, behavioural and implementation science; and health economics. In addition, representation from law enforcement was sought
• Establish a timeline and deliverables.

Step 1.3: A project outline was developed and circulated (based on information derived from Step 1.2); experts and key stakeholders were engaged; and modelling workshop materials were developed (i.e. presentations, manuals, and resources to support participatory mapping activities).

Phase 2: participatory model building and model validation (time frame – 6 to 9 months)

Through three participatory workshops, the model-building team collaboratively mapped the key risk factors and likely causal pathways leading to the outcomes of interest. This map was used to develop a computational simulation model considered to be a plausible representation of the problem of alcohol-related harms in NSW. The key products of Phase 2 were a conceptual map and computational model of the complex problem and accompanying documentation to provide transparency in model structure, parameterisation and assumptions.

Step 2.1: The multidisciplinary model-building team was prepared for participation in the workshops by being:

• Provided with information on the modelling process and its purpose
• Invited to contribute documents and literature they considered important to inform model design and parameterisation
• Provided with documentation detailing the draft model.

Step 2.2: Two 1-day workshops were conducted (ideally 8–12 weeks apart), proceeding through a process of problem mapping, followed by model conceptualisation, formulation and initial quantification. The workshops included discussion of all potential variables and causal pathways initially considered to be important drivers of the problem. These were later modified as relationships and hypotheses were tested. All assumptions made during the process of building and quantifying the model were documented for transparency and to allow later testing of alternative assumptions. Key assumptions were also discussed and debated during the workshops. Existing data sources (inputs) to inform the structure and parameterisation of the model (Table 1, available from: www.saxinstitute.org.au/table-1-types-of-content-knowledge-contributed-by-participants-and-types-of-literature-and-data-used-2) were identified by the team, and processes required to gain access to these data were discussed. Sessions during the workshops (Appendix A, also available from: www.saxinstitute.org.au/appendix-a-modelling-alcohol-related-harms-agendas-for-workshops-1-3-2) also involved discussion and prioritisation of model outputs, interventions to be integrated (Box 1), and features that might provide appealing visualisation and functionality for the user interface.

Box 1.     Interventions prioritised by stakeholders for inclusion in the model of alcohol-related harms in New South Wales

• Brief interventions (delivered by health professionals)
• Improved access to alcohol treatment services
• Restriction of hours of sale of alcohol
• ‘Lockouts’
• Limits on the density of licensed venues
• Advertising bans
• Minimum pricing
• Improved enforcement of responsible service of alcohol

Step 2.3: Between workshops, the core project staff (modeller, JA, EO and GM) built a computational model from the conceptual map. They engaged with key individuals from the model-building team for expert advice, and for direction to relevant literature and data. Further searching of relevant literature was conducted iteratively to supplement information provided by workshop participants. The core project staff led the calibration, testing and validation process (including comparison of model outputs against real-world trends in alcohol-consumption behaviours, harms and health-service impacts), and developed the user interface. The interface allows end users to interact with the model and experiment with policy scenarios.

Step 2.4: In a further half-day workshop, the model was presented to the model-building team for verification, discussion, consensus, feedback of results, and further input on preferred visualisation of model outputs.

Phase 3: consensus building for policy actions (time frame – ongoing)

Features that were integral to successful consensus building among the multidisciplinary model-building team were the participatory approach to model development; iterative refinement of the model, based on feedback from the model-building team; validation to demonstrate how well the model reproduced a broad range of historical data patterns; and transparency regarding model assumptions. The consensus-building phase is an ongoing process in which the tool will be used to facilitate policy dialogues with a broader range of stakeholders. Stakeholders may interact with the tool to conduct their own policy experiments, discuss the costs and impacts generated by the model, and better understand the trade-offs between different scenarios. It is anticipated that broader engagement of stakeholders with the tool will facilitate wider buy-in and support implementation.

Outputs of the alcohol model

The outputs of the model to date confirm what system scientists have long stated about intervening in complex systems, and the value of dynamic simulation modelling. For example, the model outputs demonstrated that the effects of combining interventions are not necessarily additive. Simulation experiments of state-wide application of the 2014 NSW liquor licence reforms that are currently in place in a number of entertainment precincts in Sydney (comprising 3 am ‘last drinks’ in licensed venues, 1 am ‘lockouts’ and 10 pm closing of bottle shops), combined with a scale-up of treatment services for heavy drinkers, produced forecast reductions in alcohol-related harms, emergency department presentations and hospitalisations that were greater than the sum of impacts seen with each intervention applied individually. Some simulation experiments demonstrate effects of interventions that are counterintuitive and unintended. Others show that the impacts of some interventions would likely occur over longer periods, with effects not evident within the initial 1- to 2-year evaluation period or a 3-year funding cycle. A more detailed description of the model itself and its results for particular policy actions will be given elsewhere.

The model can be updated and maintained as new data or evidence come to hand, through ongoing collaboration between researchers, the model administrators and policy makers.

Conclusion

The design of effective responses to complex problems such as reducing alcohol-related harms raises a range of challenging questions that cannot be adequately answered by the existing systematic reviews of research evidence. Simulation modelling provides policy makers and practitioners with a unique decision-support tool that integrates existing data, research evidence, and expert and local knowledge to examine the likely impacts of different policy and program scenarios simulated over time. Such models can help identify interventions that act synergistically, anticipate unintended consequences, and contribute to business cases for longer-term investment in effective responses to complex problems. Stakeholder participation in collaborative model building and model transparency can provide valuable learning about complex problems, enable constructive dialogues and engagement in the policy development process, and support consensus building for effective action. The model has the capacity to test a large range of scenarios to determine their likely short-term and longer-term impacts on alcohol-related harms in NSW. It can provide decision makers and practitioners with a robust basis for negotiating with stakeholders on effective and appropriate actions to reduce alcohol-related harms, and to inform further research.

Acknowledgements

We thank Sally Redman, Sonia Wutzke, Alan Shiell and Nate Osgood for their valuable contributions and advice during various stages of the planning and implementation of this work. The work was funded by the National Health and Medical Research Council of Australia (NHMRC) through its Partnership Centre grant scheme (grant GNT9100001). NSW Health, ACT Health, the Australian Government Department of Health, the Hospitals Contribution Fund of Australia and the HCF Research Foundation have contributed funds to support this work as part of the NHMRC Partnership Centre grant scheme. The contents of this paper are solely the responsibility of the individual authors and do not reflect the views of the NHMRC or funding partners.

Copyright:

© 2017 Atkinson et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.