Abstract
Background: Evidence synthesis is an important tool to inform decision-making in public health policy and practice. Collaborative approaches to evidence synthesis involving researchers and the end-users of their research can enhance the relevance of the evidence for policy and practice and overcome the limitations of traditional evidence synthesis methods. Despite its benefits, collaboration is not consistently integrated into evidence-synthesis methods.
Type of program or service: Collaborative evidence synthesis for public health policy and practice.
Methods: Reflecting on our experiences of undertaking collaborative evidence syntheses with end-users to inform policy and practice around preventive health in the first 2000 days of life, we have collated our key learnings to inform future collaborations in public health research.
Results: Key themes generated from our reflections were: 1) establish genuine partnerships early on with stakeholders, leveraging existing trusted relationships; 2) identify common goals; 3) prioritise evidence synthesis aims and objectives to ensure they are policy and practice relevant; and 4) maintain transparent, two-way communication.
Lessons learnt: Collaboration involving researchers and end-users enhances knowledge synthesis methodologies, increases relevance and accessibility of the evidence for end-users, and strengthens research-policy relationships.
Full text
Introduction
Evidence synthesis is the process of examining and summarising research evidence on a specific topic to answer research, clinical or policy questions.1 There are numerous approaches to evidence synthesis, including scoping reviews, systematic reviews and meta-analyses, and reviewers should select the approach that best suits their question and purpose.2 Evidence syntheses are essential to informing evidence-based policy and practice, as single studies are unlikely to provide sufficient evidence when considered alone.3 Evidence syntheses provide an overall understanding of the available evidence, highlight commonalities in the evidence and improve confidence in findings.
To support the integration of research into policy and practice, evidence syntheses need to be relevant and accessible to stakeholders beyond the research community. This includes end-users such as public servants, policymakers, and practitioners, who use research to make evidence-informed decisions. There are several identified barriers to consider in ensuring evidence syntheses are relevant to end-users.4,5 These include a narrow research focus, lack of applicability to policy and practice contexts, lack of access to evidence, variable evidence quality, and a slow production time that can limit the inclusion of up-to-date evidence.4,5 Furthermore, researchers undertaking evidence syntheses may not be fully cognisant of the policy and practice perspectives that are important to end-users.
Collaboration between researchers and end-users can help overcome well-documented barriers to using research evidence in policy and practice.6 Collaborative approaches can inform the design and conduct of reviews, ensuring research questions are relevant to end-users, and facilitate the use of findings in policy and practice.4,5 While it is important to acknowledge that collaboration is not without challenges, the benefits of collaborative approaches to evidence syntheses can outweigh the additional time and resources that collaboration requires.7 Collaborations with other stakeholders such as consumers (for example, parents and caregivers who benefit from evidence-informed policy and practice) are also important to advance evidence synthesis and have been outlined elsewhere.8 In this perspective we focus on collaborations with policy and practice end-users.
This article aims to share learnings from our experiences undertaking collaborative evidence syntheses to inform policy and practice decisions around preventive health in the first 2000 days based on two case studies (See Box 1). The first 2000 days is the period of early life from conception to age 5 years and a critical time for prevention to support lifelong trajectories of health and wellbeing. We present this perspective as a multidisciplinary team of early- to mid-career researchers with experience in public health research and practice. Our roles in the projects described include applying for and managing grant funding, project conceptualisation and management, stakeholder engagement, evidence synthesis, and developing research communication products. The insights that we describe reflect our own experiences and viewpoints.
Box 1. Case studies
Case study 1: Prevention in the First 2000 Days knowledge synthesis
In 2020, the Collaboration for Enhanced Research Impact (CERI) – a joint initiative between The Australian Prevention Partnership Centre (Prevention Centre) and National Health and Medical Research Centre (NHMRC) Centres of Research Excellence (CREs) – established a collaboration comprising researchers from three Australian CREs, state and federal health policymakers, and science communication experts to undertake a knowledge synthesis on prevention in the first 2000 days. The overarching aim was to collaboratively produce an evidence synthesis that met policymaker needs. A knowledge synthesis process9, already established by The Prevention Centre, formed the framework for the project. The Prevention Centre initiated and managed the engagement process, which included two policy roundtable meetings, leveraging its established relationships with policymakers and researchers. During the first meeting, policymakers were explicitly asked what their evidence needs were, and research questions to guide the evidence synthesis were agreed upon. At the second roundtable, researchers presented synthesised evidence10 to policymakers, the group discussed implications for policy and practice, and guidance on research translation and communication to a policy audience was sought. Engaging with end-users resulted in a targeted evidence synthesis co-produced with researchers and policymakers, that was reported by policymakers to be relevant and accessible.10
Case study 2: The TOPCHILD Collaboration
The TOPCHILD (Transforming Obesity Prevention for CHILDren) Collaboration, established in 2020, includes more than 100 researchers internationally and aims to transform the thinking and practices around obesity prevention in early childhood. It engages primary researchers, policymakers, practitioners and caregivers to tailor evidence syntheses and communication products to ensure findings are relevant, and solutions are feasible, acceptable, and scalable. TOPCHILD is led by researchers at the University of Sydney in New South Wales and Flinders University in South Australia, funded through project grants and fellowships. The Collaboration applies innovative evidence synthesis approaches to answer questions that are impossible to resolve by traditional evidence synthesis approaches alone (e.g., What components of parent-focused behavioural interventions are effective in preventing early childhood obesity for different population groups?). Annual systematic searches of databases and clinical trial registries identify planned, ongoing, and completed trials. Investigators of identified trials are invited to join the Collaboration. Intervention materials are coded to understand the components of different interventions11, and individual participant data meta-analysis brings together raw de-identified data from more than 50 trials to understand which intervention works best and for whom.12 A nested prospective meta-analysis approach coordinates ongoing research efforts, to answer additional questions and capture evidence as it is generated. TOPCHILD maintains engagement with stakeholders through annual collaborator meetings held online and duplicated for different time zones, and newsletters provide project updates and success stories. Targeted meetings with end-users (e.g., parents, practitioners) are held to align key research questions, support data analysis, and interpret results ahead of developing dissemination products. The day-to-day activities are managed by a steering group that meets regularly; an advisory group (including senior researchers, end-users and consumers) meets quarterly to address big-picture issues; and purpose-driven working groups meet as needed. Clear governance structures were established and communicated early on, including disclosure and management of conflicts of interest.
Methods
Approaches to collaborative evidence synthesis
Various approaches to evidence synthesis have different benefits and, while the findings can be used to inform policy and practice, many approaches rely on researchers to articulate implications with little or no input from policymakers or practitioners.13 We propose that any of these approaches could be used in a collaborative context and present an overview of methods that, in our experience, are amenable to collaboration with public health policy and practice stakeholders (Table 1).
Table 1. Examples of evidence synthesis approaches suitable for collaboration to inform public health policy and practice
| Type of evidence synthesis | When to use in collaboration with end-users in public health |
| Scoping review14 | For exploratory research questions
To identify available evidence on a given topic |
| Rapid review15 | When evidence synthesis is required promptly and/or limited resources are available |
| Systematic review16 | For well-defined, focused research questions, including critical appraisal of the identified evidence
To inform or confirm current policy or practice To guide decision-making around evidence-based policy or practice |
| Realist review17 | To evaluate complex interventions
To guide policy and practice decisions about what interventions work for whom |
| Evidence mapping18 | To map evidence characteristics across a broad topic area |
| Individual participant data meta-analysis19 | To perform more complex analyses that are not possible with aggregate data (e.g., subgroup analyses, harmonised primary outcome, controlling for important confounders) and for which high underlying data quality is required
To determine differential intervention effects for different populations |
| Prospective meta-analysis20 | To answer research questions with limited existing evidence, but for which new studies are expected/ongoing
To enable ongoing studies to coordinate efforts in a way similar to multicentre trials, but with higher generalisability |
Results
Involving stakeholders in evidence synthesis
Addressing complex issues such as childhood obesity prevention requires extensive collaboration and coordination across sectors and disciplines.8 For example, policymakers offer experiential knowledge and contextual insights to which researchers do not have access.13 Collaborating with policymakers can ensure the research is relevant to the policy agenda and increase the likelihood that findings will be used to guide policy and program development.21 Robust partnerships between decision-makers and research teams, and the consideration of context and end-user needs in developing research questions and reporting findings have been identified as success factors in collaborative research projects.22 This was demonstrated in our projects leveraging existing relationships between researchers and policymakers. For example, in the First 2000 Days case study, alignment between policymakers’ evidence needs, and researchers’ objectives was likely the result of longstanding relationships between policymakers and the chief investigators of participating CREs.
Opportunities to engage end-users can be created at any point during the knowledge synthesis process. However, the greatest value is achieved when stakeholders are involved throughout the whole process – from the early stages of generating research questions 23 to interpreting findings of the evidence synthesis to guiding the development of the subsequent communication products. The TOPCHILD Collaboration evolved from an international collaboration of primarily researchers and consumers (reimbursed for their time using grant funds) to partner with end-users to understand their information needs, refine the evidence synthesis database and develop tailored communication products. The First 2000 Days knowledge synthesis engaged policymakers from the outset, led by The Prevention Centre, which provided a central point of contact, facilitated communication between stakeholders, and maintained project momentum.
Strong leadership is critical to the success of any collaborative evidence synthesis24, and our projects benefited from clear leadership structures. The TOPCHILD Collaboration has a chair and deputy chairs whose roles include clear communication with all involved stakeholders, ensuring the project stays within scope and timelines, and resolving and making final decisions on any disputes. The First 2000 Days knowledge synthesis was led by The Prevention Centre, which had roles spanning stakeholder engagement, coordinating and facilitating meetings, and developing and designing research communication products.
Evidence synthesis collaborations can be initiated by researchers or policymakers and likely stem from different needs. Policymakers may seek evidence to inform policy decisions, while researchers may initiate evidence syntheses that advance the science and achieve academic metrics.7 Communication and consideration of each other’s needs and goals are key to ensuring successful collaborations. The First 2000 Days knowledge synthesis used iterative roundtables to establish shared goals, prioritise aims, and identify suitable communication products. In the TOPCHILD Collaboration, one-on-one meetings with end-users helped to develop a shared understanding of goals for the evidence synthesis and its alignment with organisational priorities.
The case studies demonstrated that while collaborators can have divergent opinions at times, resulting in robust discussions, if managed correctly, these are desirable since they can lead to broad consensus and a multidisciplinary perspective. Planning and allowing time for such discussions within project timelines enables stakeholders to voice their opinions or concerns in a safe and respectful environment. A clear governance structure, responsive leadership team, open communication, and a regular meeting schedule, with the flexibility to allow for additional meetings, can help resolve issues that may arise.
The complexity and novelty of the advanced evidence synthesis methods in the TOPCHILD case study posed additional challenges. For example, the timeframe required for an individual participant data meta-analysis is lengthy compared to a meta-analysis using published aggregate data. Extra time is necessary to accommodate data sharing, outcome harmonisation and data processing, as well as to ensure all data contributors agree with the analysis plans. Managing expectations around project timelines is an important part of the stakeholder relationship that requires clear and regular communication.
Drawing on our experiences, we have identified key principles for successful collaborations for evidence synthesis in public health policy and practice (Figure 1).
Figure 1. Key principles for collaborative evidence synthesis in public health policy and practice (click figure to enlarge)
Translating synthesised evidence into policy and practice
Research translation involves the dissemination and application of evidence for policy and practice. Research dissemination can take many forms including, but not limited to, research reports, policy briefs, presentations, and submissions to government consultations. In each case, key messages must be tailored to the end-user and their needs.25 In the First 2000 Days case study, research translation was led by The Prevention Centre’s science communications experts, who participated in the roundtable meetings and used their knowledge and skills to develop key messages and create tailored products that communicated evidence to policy and practice audiences.
Collaboration takes the guesswork out of research translation by ensuring end-user needs are clear from the outset. This is important to provide end-users with relevant research evidence, but on its own, does not ensure that the research will be applied.23 Policymakers’ use of research evidence also depends on factors including alignment with political goals and interests, and interactions between researchers and policymakers via policy networks, such as formal advisory committees and informal relationships.24
Collaborating to undertake evidence syntheses can create the opportunity to form strategic alliances between research teams and policy and practice end-users with the potential for ongoing exchanges beyond the evidence synthesis process.24 In our experiences, this also creates opportunities for early career researchers to begin forming relationships with policymakers and practitioners. In the longer term, these collaborations can help to guide further research and increase the likelihood that researchers are sought by policymakers in the future.7
Lessons learnt
Collaborative approaches to evidence syntheses offer benefits to both researchers and the end-users of their findings in public health policy and practice. In our experience – as demonstrated by the case studies – collaborations for evidence syntheses have brought stakeholders together, strengthened relationships between researchers and end-users, given researchers greater insights into policymaker and practitioner needs, and led to the development of tailored evidence that is relevant for policy and practice.
Acknowledgements
The work of authors VB, BJ, KK and AS is supported by the National Health and Medical Research Council (NHMRC) Centre of Research Excellence (CRE) in Translating the Early Prevention of Obesity in Childhood (GNT2006999).
AC is supported by a Medical Research Future Fund (MRFF) Preventive Health Research Initiative (GNT1199826) and a VicHealth Postdoctoral Research Fellowship.
BJ is supported by an Early-Mid Career Research Fellowship from The Hospital Research Foundation Group and an NHMRC Ideas Grant (GNT1186363).
AS is supported by an NHMRC Investigator Grant (GNT2009432) and an NHMRC Ideas Grant (GNT1186363) which funds the TOPCHILD project referred to in the manuscript.
AH is supported by the National Centre Of Implementation Science, an NHMRC Centre for Research Excellence (APP1153479)
This paper is part of a special issue of the journal focusing on: ‘Collaborative partnerships for prevention: health determinants, systems and impact’. The issue has been produced in partnership with the CERI, a joint initiative between The Prevention Centre and NHMRC CREs. The Prevention Centre is managed by the Sax Institute in collaboration with its funding partners.
Peer review and provenance
Externally peer reviewed, invited.
Copyright:
© 2024 Chung et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.
References
- 1. Gough D, Davies P, Jamtvedt G, Langlois E, Littell J, Lotfi T, et al. Evidence synthesis international (ESI): position statement. Syst Rev. 2020;9(1):155. CrossRef | PubMed
- 2. Munn Z, Pollock D, Barker TH, Stone J, Stern C, Aromataris E, et al. The Pandora's box of evidence synthesis and the case for a living evidence synthesis taxonomy. BMJ Evid Based Med. 2023;28(3):148–50. CrossRef | PubMed
- 3. Littell JH, White H. The Campbell Collaboration: providing better evidence for a better world. Research on Social Work Practice. 2018;28(1):6–12. CrossRef
- 4. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Ser Res. 2014;14(1):2. CrossRef | PubMed
- 5. Tricco AC, Cardoso R, Thomas SM, Motiwala S, Sullivan S, Kealey MR, et al. Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review. Implementation Sc. 2016;11(1):4. CrossRef | PubMed
- 6. Oliver K, Cairney P. The dos and don’ts of influencing policy: a systematic review of advice to academics. Palgrave Communications. 2019;5(1):21. CrossRef
- 7. Williamson A, Tait H, El Jardali F, Wolfenden L, Thackway S, Stewart J, et al. How are evidence generation partnerships between researchers and policy-makers enacted in practice? A qualitative interview study. Health Research Policy and Systems. 2019;17(1):41. CrossRef | PubMed
- 8. Seidler AL, Johnson BJ, Golley RK, Hunter KE. The complex quest of preventing obesity in early childhood: describing challenges and solutions through collaboration and innovation. Front Endocrinol (Lausanne). 2022;12:803545. CrossRef | PubMed
- 9. Heenan M, Chung A, Howse E, Signy H, Rychetni L. Combining public health evidence, policy experience and communications expertise to inform preventive health: reflections on a novel method of knowledge synthesis. Health Res Policy Syst. 2023;21(1):112. CrossRef | PubMed
- 10. The Australian Prevention Partnership Centre and Collaboration for Enhanced Research Impact. Prevention in the first 2000 days. Sydney, NSW: Prevention Centre & CERI; 2022 [cited 2024 Feb 08]. Available from: preventioncentre.org.au/wp-content/uploads/2022/08/First-2000-days-full-report-FINAL-1.pdf
- 11. Johnson BJ, Hunter KE, Golley RK, Chadwick P, Barba A, Aberoumand M, et al. Unpacking the behavioural components and delivery features of early childhood obesity prevention interventions in the TOPCHILD Collaboration: a systematic review and intervention coding protocol. BMJ Open. 2022;12(1):e048165. CrossRef | PubMed
- 12. Hunter KE, Johnson BJ, Askie L, Golley RK, Baur LA, Marschner IC, et al. Transforming Obesity Prevention for CHILDren (TOPCHILD) Collaboration: protocol for a systematic review with individual participant data meta-analysis of behavioural interventions for the prevention of early childhood obesity. BMJ Open. 2022;12(1):e048166. CrossRef | PubMed
- 13. Cairney P, Oliver K. Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy? Health Res Policy Syst. 2017;15(1):35. CrossRef | PubMed
- 14. Colquhoun HL, Levac D, O'Brien KK, Straus S, Tricco AC, Perrier L, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014;67(12):1291–4. CrossRef | PubMed
- 15. Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med 2015;13(1):224. CrossRef | PubMed
- 16. Munn Z, Stern C, Aromataris E, Lockwood C, Jordan Z. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol. 2018;18(1):5. CrossRef | PubMed
- 17. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review – a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10 Suppl 1:21–34. CrossRef | PubMed
- 18. Bragge P, Clavisi O, Turner T, Tavender E, Collie A, Gruen RL. The Global Evidence Mapping Initiative: scoping research in broad topic areas. BMC Med Res Methodol. 2011;11:92. CrossRef | PubMed
- 19. Riley RD, Tierney JF, Stewart LA, editors. Individual Participant Data Meta-Analysis: A handbook for healthcare research: John Wiley & Sons Ltd; US; 2021. CrossRef
- 20. Seidler AL, Hunter KE, Cheyne S, Ghersi D, Berlin JA, Askie L. A guide to prospective meta-analysis. BMJ. 2019;367:l5342. CrossRef | PubMed
- 21. Moore G, Redman S, Rudge S, Haynes A. Do policy-makers find commissioned rapid reviews useful? Health Res Policy Syst. 2018;16(1):17. CrossRef | PubMed
- 22. Bornstein S, Baker R, Navarro P, Mackey S, Speed D, Sullivan M. Putting research in place: an innovative approach to providing contextualized evidence synthesis for decision makers. Syst Rev. 2017;6(1):218. CrossRef | PubMed
- 23. Langlois EV, Daniels K, Akl EA, editors. Evidence synthesis for health policy and systems: a methods guide. Geneva: World Health Organization, Alliance for Health Policy and Systems Research; 2018 [cited 2024 Feb 12]. Available from: iris.who.int/bitstream/handle/10665/275367/9789241514552-eng.pdf?sequence=1
- 24. Lawrence LM, Bishop A, Curran J. Integrated knowledge translation with public health policy makers: a scoping review. Healthc Policy. 2019;14(3):55–77. CrossRef | PubMed
- 25. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Imp Science. 2012;7(1):50. CrossRef | PubMed
