In Practice

Dynamic simulation modelling of policy responses to reduce alcohol-related harms: rationale and procedure for a participatory approach

Jo-An Atkinson, Eloise O'Donnell, John Wiggers, Geoff McDonnell, Jo Mitchell, Louise Freebairn, Devon Indig, Lucie Rychetnik

Published 15 February 2017. doi: http://dx.doi.org/10.17061/phrp2711707
Citation: Atkinson J, O’Donnell E, Wiggers J, McDonnell G, Mitchell J, Freebairn L, Indig D, Rychetnik L. Dynamic simulation modelling of policy responses to reduce alcohol-related harms: rationale and procedure for a participatory approach. Public Health Res Pract. 2017;27(1):e2711707.

  • Citation

  • PDF

About the author/s

Jo-An Atkinson | The Australian Prevention Partnership Centre, Sax Institute, Sydney, NSW; School of Medicine, University of Sydney, NSW, Australia

Eloise O'Donnell | The Australian Prevention Partnership Centre, Sax Institute, Sydney, NSW

John Wiggers | The Australian Prevention Partnership Centre, Sax Institute, Sydney, NSW; Hunter New England Population Health, Newcastle, NSW, Australia; School of Medicine and Public Health, University of Newcastle, NSW, Australia

Geoff McDonnell | The Australian Prevention Partnership Centre, Sax Institute, Sydney, NSW

Jo Mitchell | Centre for Population Health, NSW Ministry of Health, Sydney, Australia

Louise Freebairn | The Australian Prevention Partnership Centre, Sax Institute, Sydney, NSW; Knowledge Translation and Health Outcomes, Epidemiology Section, ACT Health, Canberra, Australia

Devon Indig | The Australian Prevention Partnership Centre, Sax Institute, Sydney, NSW; School of Public Health, University of Sydney, NSW, Australia

Lucie Rychetnik | The Australian Prevention Partnership Centre, Sax Institute, Sydney, NSW; School of Medicine, University of Notre Dame, Sydney, NSW, Australia

Corresponding author

Jo-An Atkinson | Jo-An.Atkinson@saxinstitute.org.au

Competing interests

None declared

Author contributions

All authors contributed to the conception of the paper. JA drafted the manuscript. All authors made important intellectual contributions to multiple draft revisions. All authors read and approved the final manuscript.

Abstract

Development of effective policy responses to address complex public health problems can be challenged by a lack of clarity about the interaction of risk factors driving the problem, differing views of stakeholders on the most appropriate and effective intervention approaches, a lack of evidence to support commonly implemented and acceptable intervention approaches, and a lack of acceptance of effective interventions. Consequently, political considerations, community advocacy and industry lobbying can contribute to a hotly contested debate about the most appropriate course of action; this can hinder consensus and give rise to policy resistance. The problem of alcohol misuse and its associated harms in New South Wales (NSW), Australia, provides a relevant example of such challenges.

Dynamic simulation modelling is increasingly being valued by the health sector as a robust tool to support decision making to address complex problems. It allows policy makers to ask ‘what-if’ questions and test the potential impacts of different policy scenarios over time, before solutions are implemented in the real world. Participatory approaches to modelling enable researchers, policy makers, program planners, practitioners and consumer representatives to collaborate with expert modellers to ensure that models are transparent, incorporate diverse evidence and perspectives, are better aligned to the decision-support needs of policy makers, and can facilitate consensus building for action.

This paper outlines a procedure for embedding stakeholder engagement and consensus building in the development of dynamic simulation models that can guide the development of effective, coordinated and acceptable policy responses to complex public health problems, such as alcohol-related harms in NSW.

Full text

Key points

  • Effective policy responses to complex public health problems are challenged by uncertainty around the most effective intervention combinations, political considerations, community advocacy, and lack of consensus on a course of action
  • Simulation models are ‘what-if’ tools for testing the impacts of alternative policy scenarios before implementing solutions in the real world
  • This paper outlines a procedure for embedding stakeholder engagement and consensus building in the development of simulation models

Introduction

Complex public health problems present challenges to decision makers that can hinder effective action. These challenges include uncertainty relating to what interventions will deliver the greatest impact; competing views and agendas that can undermine effective implementation of interventions; and the need to remain responsive to changing circumstances that can increase service needs, while operating within resource constraints.1-4 As a result of such challenges, the health sector is increasingly recognising the value of dynamic simulation modelling methods. These methods have long been used by other sectors to help address complex problems and optimise the use of limited resources.5 They include system dynamics, which captures population-level influences and whole-system dynamics; agent-based modelling, which captures individual heterogeneity and social network influences; discrete event simulation, which analyses processes or sequences of events; and the integration of these techniques.6 Dynamic simulation modelling provides a platform for systematically integrating diverse evidence sources through a process of mapping, quantifying, testing, validating and refining a model of the underlying causal mechanisms that drive a complex problem in a particular context. These dynamic models provide policy makers with a tailored ‘what-if’ tool to examine the likely impacts of different policy and program options (applied individually or in combination), as well as the comparative cost and system implications of these options over the short and longer term.5

The problem of alcohol misuse

Globally, alcohol consumption results in approximately 3.3 million deaths each year, and the costs of alcohol-related harms amount to more than 1% of gross national product in high-income countries.7,8 In Australia, alcohol is widely used and accounts for an estimated 5.1% of the total burden of disease and injury.9 The harms from alcohol (including ill health, chronic disease, crime, road traffic accidents and lost productivity) have been conservatively estimated to cost Australia $15.3 billion per year.10

Challenges for reducing alcohol-related harms

Evidence suggests that a range of individual, sociocultural, economic and environmental factors contribute to the problem of alcohol-related harms, resulting in many potential points of intervention. However, despite a range of evidence-based options for intervening11, there is uncertainty about the harm-reduction benefits of such interventions in particular contexts, and the effects of combining them.1 Moreover, there are diverse views about the acceptability of evidence based harm-reduction options, advocacy for options that lack evidence, and industry concerns about interventions that are likely to impose additional costs or limitations on businesses.1-4 Policy development in such a challenging context would benefit from robust and transparent decision-support tools, for which dynamic simulation modelling is well suited.

The importance of a participatory approach

Successful development of policy responses for reducing alcohol-related harms depends on stakeholder support for the proposed policy actions.3 Transparency and stakeholder engagement in the policy development process, and consensus building are considered to be essential to gaining this support.12 Facilitating stakeholder participation in collaborative simulation model building is one approach that can be used to improve stakeholder communication, advance contentious debates, gain consensus among stakeholders, aid transparency and translation of model outcomes to stakeholders, and garner broader support for collaborative action.13-16 Recent advances in simulation modelling software and more user-friendly interfaces have made participation and the achievement of such outcomes more feasible. A procedure is required for embedding stakeholder engagement and consensus-building processes in the development of dynamic simulation models.

Methods

Development of the procedure

Several guidelines and frameworks exist for best-practice methods of working with small groups to facilitate simulation model conceptualisation, formulation, quantification, calibration and validation, as well as conducting policy analysis and simulation experiments.17-20 The procedure described in this paper draws on such guidelines and frameworks, and on the findings of a systematic review of 107 group model-building projects (almost all of which were in nonhealth sectors).16 The procedure builds on this information to address the need for broader consensus building in the development of public health policy.12,21

Application of the procedure

The procedure was applied in 2015 to develop a dynamic simulation model of possible policy actions to reduce alcohol-related harms in New South Wales (NSW), Australia. It was developed and applied through collaboration between The Australian Prevention Partnership Centre, NSW Health, local and national alcohol researchers, clinicians, program planners and policy officers.

The procedure

The procedure consisted of three distinct phases: 1) project planning and engagement; 2) participatory model building and model validation; and 3) consensus building for policy actions (Figure 1.)

Figure 1.     Procedure for applying participatory simulation modelling to complex public health policy questions (click to enlarge)

PHRPFEB17-Modelling-Figure-01

Phase 1: project planning and engagement (time frame – 1 month)

Phase 1 defined the purpose, scope and boundaries of the modelling project to ensure that the simulation modelling tool could be developed in a policy-friendly time frame. Hence, careful planning and early engagement of stakeholders were priorities. The following steps were undertaken in Phase 1.

Step 1.1: Core project staff were identified and assembled, as follows:

  • Project lead (JA). The project lead facilitated the brokering and management of the project, and was primarily responsible for engaging and maintaining relationships with stakeholders and policy partners (end users of the model) throughout the process. The project lead shared the duties of facilitating modelling workshops with the lead domain expert, and oversaw model development, the production of supporting documentation and external communications
  • Lead domain expert (JW). A well-respected alcohol expert was identified and engaged as a project sponsor and lead domain expert. In collaboration with the policy partners, the lead domain expert played an integral role in identifying and recruiting key stakeholders to participate in model development, and jointly facilitated the modelling workshops
  • Dynamic simulation modeller (computer programmer) and data analyst. This person was contracted to build the model according to agreed specifications
  • Research officer (EO). The research officer was responsible for logistical arrangements, and liaising directly with the modeller and modelling workshop participants to source and manage evidence and data requirements of the model-building process
  • Expert technical adviser (GM). The technical adviser provided high-level oversight of model conceptualisation and development to ensure that the model was computationally efficient, interactive, and deliverable in a policy-friendly time frame.

Step 1.2: A meeting was held with policy partners (end users of the model) to:

  • Clearly define the aspects of the problem to be modelled, including its scope and boundaries, and identify key outputs of interest (e.g. alcohol-related mortality, health service use, prevalence of alcohol-related chronic disease, costs and cost savings)
  • Identify and select possible intervention options to be included in, and tested by, the model. The inventory of possible interventions included existing programs and interventions, those currently under policy consideration, those supported by evidence but not implemented, and those recommended by stakeholder groups but not supported by evidence
  • Identify a small group of experts and key stakeholders (10–15 people) who would comprise the collaborative model-building team. Members of this multidisciplinary team were purposefully selected on the basis of their diverse perspectives and recognition as research, clinical, policy and community leaders in the following fields: drug and alcohol clinical service delivery; population health policy making and service delivery; epidemiology; social, behavioural and implementation science; and health economics. In addition, representation from law enforcement was sought
  • Establish a timeline and deliverables.

Step 1.3: A project outline was developed and circulated (based on information derived from Step 1.2); experts and key stakeholders were engaged; and modelling workshop materials were developed (i.e. presentations, manuals, and resources to support participatory mapping activities).

Phase 2: participatory model building and model validation (time frame – 6 to 9 months)

Through three participatory workshops, the model-building team collaboratively mapped the key risk factors and likely causal pathways leading to the outcomes of interest. This map was used to develop a computational simulation model considered to be a plausible representation of the problem of alcohol-related harms in NSW. The key products of Phase 2 were a conceptual map and computational model of the complex problem and accompanying documentation to provide transparency in model structure, parameterisation and assumptions.

Step 2.1: The multidisciplinary model-building team was prepared for participation in the workshops by being:

  • Provided with information on the modelling process and its purpose
  • Invited to contribute documents and literature they considered important to inform model design and parameterisation
  • Provided with documentation detailing the draft model.

Step 2.2: Two 1-day workshops were conducted (ideally 8–12 weeks apart), proceeding through a process of problem mapping, followed by model conceptualisation, formulation and initial quantification. The workshops included discussion of all potential variables and causal pathways initially considered to be important drivers of the problem. These were later modified as relationships and hypotheses were tested. All assumptions made during the process of building and quantifying the model were documented for transparency and to allow later testing of alternative assumptions. Key assumptions were also discussed and debated during the workshops. Existing data sources (inputs) to inform the structure and parameterisation of the model (Table 1, available from: www.saxinstitute.org.au/table-1-types-of-content-knowledge-contributed-by-participants-and-types-of-literature-and-data-used-2) were identified by the team, and processes required to gain access to these data were discussed. Sessions during the workshops (Appendix A, also available from: www.saxinstitute.org.au/appendix-a-modelling-alcohol-related-harms-agendas-for-workshops-1-3-2) also involved discussion and prioritisation of model outputs, interventions to be integrated (Box 1), and features that might provide appealing visualisation and functionality for the user interface.

 


Box 1.     Interventions prioritised by stakeholders for inclusion in the model of alcohol-related harms in New South Wales

  • Brief interventions (delivered by health professionals)
  • Improved access to alcohol treatment services
  • Restriction of hours of sale of alcohol
  • ‘Lockouts’
  • Limits on the density of licensed venues
  • Advertising bans
  • Minimum pricing
  • Improved enforcement of responsible service of alcohol

 

Step 2.3: Between workshops, the core project staff (modeller, JA, EO and GM) built a computational model from the conceptual map. They engaged with key individuals from the model-building team for expert advice, and for direction to relevant literature and data. Further searching of relevant literature was conducted iteratively to supplement information provided by workshop participants. The core project staff led the calibration, testing and validation process (including comparison of model outputs against real-world trends in alcohol-consumption behaviours, harms and health-service impacts), and developed the user interface. The interface allows end users to interact with the model and experiment with policy scenarios.

Step 2.4: In a further half-day workshop, the model was presented to the model-building team for verification, discussion, consensus, feedback of results, and further input on preferred visualisation of model outputs.

Phase 3: consensus building for policy actions (time frame – ongoing)

Features that were integral to successful consensus building among the multidisciplinary model-building team were the participatory approach to model development; iterative refinement of the model, based on feedback from the model-building team; validation to demonstrate how well the model reproduced a broad range of historical data patterns; and transparency regarding model assumptions. The consensus-building phase is an ongoing process in which the tool will be used to facilitate policy dialogues with a broader range of stakeholders. Stakeholders may interact with the tool to conduct their own policy experiments, discuss the costs and impacts generated by the model, and better understand the trade-offs between different scenarios. It is anticipated that broader engagement of stakeholders with the tool will facilitate wider buy-in and support implementation.

Outputs of the alcohol model

The outputs of the model to date confirm what system scientists have long stated about intervening in complex systems, and the value of dynamic simulation modelling. For example, the model outputs demonstrated that the effects of combining interventions are not necessarily additive. Simulation experiments of state-wide application of the 2014 NSW liquor licence reforms that are currently in place in a number of entertainment precincts in Sydney (comprising 3 am ‘last drinks’ in licensed venues, 1 am ‘lockouts’ and 10 pm closing of bottle shops), combined with a scale-up of treatment services for heavy drinkers, produced forecast reductions in alcohol-related harms, emergency department presentations and hospitalisations that were greater than the sum of impacts seen with each intervention applied individually. Some simulation experiments demonstrate effects of interventions that are counterintuitive and unintended. Others show that the impacts of some interventions would likely occur over longer periods, with effects not evident within the initial 1- to 2-year evaluation period or a 3-year funding cycle. A more detailed description of the model itself and its results for particular policy actions will be given elsewhere.

The model can be updated and maintained as new data or evidence come to hand, through ongoing collaboration between researchers, the model administrators and policy makers.

Conclusion

The design of effective responses to complex problems such as reducing alcohol-related harms raises a range of challenging questions that cannot be adequately answered by the existing systematic reviews of research evidence. Simulation modelling provides policy makers and practitioners with a unique decision-support tool that integrates existing data, research evidence, and expert and local knowledge to examine the likely impacts of different policy and program scenarios simulated over time. Such models can help identify interventions that act synergistically, anticipate unintended consequences, and contribute to business cases for longer-term investment in effective responses to complex problems. Stakeholder participation in collaborative model building and model transparency can provide valuable learning about complex problems, enable constructive dialogues and engagement in the policy development process, and support consensus building for effective action. The model has the capacity to test a large range of scenarios to determine their likely short-term and longer-term impacts on alcohol-related harms in NSW. It can provide decision makers and practitioners with a robust basis for negotiating with stakeholders on effective and appropriate actions to reduce alcohol-related harms, and to inform further research.

Acknowledgements

We thank Sally Redman, Sonia Wutzke, Alan Shiell and Nate Osgood for their valuable contributions and advice during various stages of the planning and implementation of this work. The work was funded by the National Health and Medical Research Council of Australia (NHMRC) through its Partnership Centre grant scheme (grant GNT9100001). NSW Health, ACT Health, the Australian Government Department of Health, the Hospitals Contribution Fund of Australia and the HCF Research Foundation have contributed funds to support this work as part of the NHMRC Partnership Centre grant scheme. The contents of this paper are solely the responsibility of the individual authors and do not reflect the views of the NHMRC or funding partners.

Copyright:

Creative Commons License

© 2017 Atkinson et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.

References

  • 1. Loxley W, Gray D, Wilkinson C, Chikritzhs T, Midford R, Moore D. Alcohol policy and harm reduction in Australia. Drug Alcohol Rev. 2005;24(6):559–68. CrossRef | PubMed
  • 2. McKee PA, Nelson TF, Toomey TL, Shimotsu ST, Hannan PJ, Jones-Webb RJ. Adopting local alcohol policies: a case study of community efforts to regulate malt liquor sales. American J Health Promot. 2012;26(3):e86–94. CrossRef | PubMed
  • 3. Tindall J, Groombridge D, Wiggers J, Gillham K, Palmer D, Clinton-McHarg T, et al. Alcohol-related crime in city entertainment precincts: public perception and experience of alcohol-related crime and support for strategies to reduce such crime. Drug Alcohol Rev. 2016;35(3):263–72. CrossRef | PubMed
  • 4. Wilkinson C, Room R, Livingston M. Mapping Australian public opinion on alcohol policies in the new millennium. Drug Alcohol Rev. 2009;28(3):263–74. CrossRef | PubMed
  • 5. Homer JB, Hirsch GB. System dynamics modeling for public health: background and opportunities. Am J Public Health. 2006;96(3):452–8. CrossRef | PubMed
  • 6. Marshall DA, Burgos-Liz L, IJzerman MJ, Crown W, Padula WV, Wong PK, et al. Selecting a dynamic simulation modeling method for health care delivery research–part 2: report of the ISPOR dynamic simulation modeling emerging good practices task force. Value Health. 2015;18(2):147–60. CrossRef | PubMed
  • 7. World Health Organization. Global status report on alcohol and health 2014. Luxembourg: WHO; 2014 [cited 2016 Nov 4]. Available from: apps.who.int/iris/bitstream/10665/112736/1/9789240692763_eng.pdf
  • 8. Rehm J, Mathers C, Popova S, Thavorncharoensap M, Teerawattananon Y, Patra J. Global burden of disease and injury and economic cost attributable to alcohol use and alcohol-use disorders. Lancet. 2009;373(9682):2223–33. CrossRef | PubMed
  • 9. Australian Institute of Health and Welfare. Australian burden of disease study: impact and causes of illness and death in Australia 2011. Canberra: AIHW; 2016 [cited 2016 Nov 11]. Available from: www.aihw.gov.au/WorkArea/DownloadAsset.aspx?id=60129555476
  • 10. Collins DJ, Lapsley HM. The costs of tobacco, alcohol and illicit drug abuse to Australian society in 2004/05. Canberra: Commonwealth of Australia; 2008 [cited 2016 Nov 11]. Available from: www.health.gov.au/internet/drugstrategy/publishing.nsf/Content/34F55AF632F67B70CA2573F60005D42B/$File/mono64.pdf
  • 11. Martineau F, Tyner E, Lorenc T, Petticrew M, Lock K. Population-level interventions to reduce alcohol-related harm: an overview of systematic reviews. Prev Med. 2013;57(4):278–96. CrossRef | PubMed
  • 12. Casswell S, Thamarangsi T. Reducing harm from alcohol: call to action. Lancet. 2009;373(9682):2247–57. CrossRef
  • 13. Loyo HK, Batcher C, Wile K, Huang P, Orenstein D, Milstein B. From model to action: using a system dynamics model of chronic disease risks to align community action. Health Promot Pract. 2013;14(1):53–61. CrossRef | PubMed
  • 14. Royston G, Ayesha D, Townshend J, Turner H. Using system dynamics to help develop and implement policies and programmes in health care in England. Syst Dyn Rev. 1999;15(3):293–313. CrossRef
  • 15. Atkinson J, Wells R, Page A, Dominello A, Haines M, Wilson A. Applications of system dynamics modelling to support health policy. Public Health Res Pract. 2015;25(3):e2531531. CrossRef | PubMed
  • 16. Rouwette EAJA, Vennix JAM, van Mullekom T. Group model building effectiveness: a review of assessment studies. Syst Dyn Rev. 2002;18(1):5–45. CrossRef
  • 17. Andersen DF, Richardson GP. Scripts for group model building. Syst Dyn Rev. 1997;13(2):107–29. CrossRef
  • 18. Bernard C. Group model building using system dynamics: an analysis of methodological frameworks. Electronic J Business Res Methods. 2010;8(1):35–45. Article
  • 19. Hammond R. Considerations and best practices in agent-based modeling to inform policy. In: Institute of Medicine, editor. Assessing the use of agent-based models for tobacco regulation. Washington, DC: The National Academies Press; 2015.
  • 20. Vennix JAM, Andersen DF, Richardson GP, Rohrbaugh J. Model-building for group decision support: issues and alternatives in knowledge elicitation. European J Operational Res. 1992;59(1):28–41. CrossRef
  • 21. Ansell C, Gash A. Collaborative governance in theory and practice. J Public Adm Res Theory. 2008;18(4):543–71. CrossRef