Experiences of research-policy engagement in policy-making processes April 2024, Volume 34, Issue 1

Introduction

The coronavirus disease 2019 (COVID-19) crisis has highlighted that policy action and collaborative decision making can occur rapidly across sectoral boundaries¹ while reinforcing the need for evidence-informed policy. The policy process is non-linear and ‘messy’, involving a set of interdependent activities², and informing policy and program decision making with research evidence is a similarly complex process.³ Both processes require a mix of political ‘know-how’, scientific and technical analysis, and practice/professional experience⁴, and evidence-informed decision-making involving various stages of adoption and rejection of evidence in policy processes.⁵

A long history of learned experience on research use in policy⁶ confirms the range of persistent challenges to evidence-informed decision making at the individual, organisational, and structural level. These include limited time and skills, unsupportive organisational culture for evidence-informed decision making, and competing priorities and crises.^7,8 A logical and increasingly popular solution proposed to overcome these challenges is collaboration between research producers and policy actors (public sector, non-government organisations, industry) at all stages of the research process.⁹ Known also as co-production¹⁰, this type of collaboration represents an interactive method to increase engagement between research and policy stakeholders, theoretically increasing the likelihood that research is co-produced or addresses needs of decision makers and stakeholders.^11,12 Many examples of collaboration, co-production and integrated knowledge translation exist^10,12,13, which generally all seek to promote research-policy engagement and impact.

Despite the extensive body of literature emphasising the importance of collaborations in evidence-informed policy making^14-16, most research has centred on interdisciplinary collaborations among researchers or research teams and with community groups, practitioners and industry bodies, with limited attention given to partnerships involving policy makers and researchers.¹⁵ Consequently, there is a lack of understanding about the experiences of both researchers and policy actors in these interactions.^15,17 Previous studies examining research-policy partnership experiences have frequently focused on specific project cases, and only a few have explored the perspectives and experiences of those involved more broadly.¹⁵ This study aims to provide quantitative data on the experiences, barriers, and enablers encountered by researchers and policy actors in research-policy partnerships. It seeks to further enhance our understanding of the complexities of these partnerships.^15,17

Understanding attitudes, experiences, and cultures of collaboration across research and policy settings could help scaffold collaboration-focused interventions and ultimately build capacity and advance system reform. This study used a national survey in Australia to gain insights on how both policy actors and academics/researchers experience collaboration and aims to contribute to the literature on research-policy collaboration, which is an important area of study for public health researchers, policy makers and practitioners.

Methods

Building upon the experience from the South Australian Health in All Policies Initiative that operated from 2007 to 2021¹⁸, a survey was co-designed by a cross-disciplinary working group with research and policy experience. The survey drew upon items from previous surveys about collaboration across sectors for policy decision making.¹⁹ The working group, including academics/researchers and policy actors from across multiple Australian jurisdictions and disciplines, guided survey development, implementation, analysis and interpretation. The survey instrument was pilot tested with both the working group members and the expert panel, a small group of international experts who advised and supported the project.

Convenience sampling was used initially to identify and recruit eligible individuals by sending the survey invitation to the working group’s research and policy networks. Individuals were invited to participate if they self-identified as either: 1) a researcher interested in research being used to inform public policy and practice or 2) a policy actor interested in using research as part of policy development, implementation, and evaluation. Survey recruitment was supplemented by snowball sampling. It sought contacts from identified key stakeholders in each Australian state and territory to whom the survey could be sent and invited initial recruits to forward the invitation to their networks. Contact nodes were identified in each state and territory, drawing on project team connections – this included one policy lead and one research lead who acted as distribution nodes for the survey. The survey was also promoted through newsletters of key professional groups in South Australia (South Australia only; Institute of Public Administrative Australia SA and the SA Policy Officers Network).

The survey was entered into Research Electronic Data Capture (REDCap) for online completion by participants between 14 April 2021 and 30 June 2021.

All survey data were collected anonymously, and all individual responses were aggregated for analysis. Quantitative data were analysed descriptively. Content analysis was used to identify themes in open-text responses within survey questions related to what strategies/solutions were needed to facilitate collaboration between research and policy. Comments were first organised as associated with either a policy actor or academic/researcher, then were coded deductively according to the knowledge translation/engagement strategy focus of ‘push’, ‘pull’ or ‘exchange’. Comments describing production and dissemination of research/tools were coded ‘push’; comments describing capacity building (of either policy actors or academics) or funding/incentive processes enabling collaboration were coded ‘pull’; and comments describing reciprocal relationship-building and co-production activities were coded ‘exchange’²⁰

Ethics

Ethical approval was granted by UniSA Business Human Research Ethics Committee (Application: 020-2021020-2021).

Results

Table 1 displays the sample characteristics. In total, 172 respondents completed the survey. Just over half identified as policy actors (57.6%), and the remainder identified as researchers or academics (41.3%). Only two did not identify as either (1.2% unknown), and as a result, they were excluded from the analysis. The majority of included respondents resided in South Australia (38%), followed by Victoria (13%) and Tasmania (12%). Academics/researchers were mostly working within universities or academic institutes (62%) in the health sciences field (52%), while policy actors were predominantly employed by state governments (63%) or in the healthcare and social assistance sector (37%).

Table 1.     Sample characteristics (N = 170)

ACT = Australian Capital Territory; NSW = New South Wales; NT = Northern Territory; Qld = Queensland; SA = South Australia; Tas = Tasmania; WA = Western Australia; Vic = Victoria

Experiences in engagement and collaboration

The majority of policy actors (90%) and academics/researchers (94%) reported ever having worked with the other to inform a policy-making process.

Half of the policy actors, compared to a smaller proportion (43%) of academics/researchers, reported they were ‘often’ or ‘very often’ the ones to initiate engagement. Both policy actors and academics/researchers-initiated contact mostly by ‘utilising existing networks or relationships’ with the other party (Table 2).

Table 2.     How have you initiated working together to inform the policymaking process?

^a Respondents could nominate multiple methods of working together

Purpose of collaboration

The primary purpose for collaboration reported by both policy actors (61%) and academics/researchers (50%) was to promote ‘evidence-informed policy making’ (Table 3).

Table 3.     What has been the primary purpose for working together?

ARC = Australian Research Council

Barriers and enablers

Reported (‘actual’) barriers

Among policy actors, ‘budget constraints’ was reported most commonly (43.4%) as a barrier faced when working with academics/researchers in the policy-making process.

Among academics/researchers, ‘budget’ was among the most substantial barriers reported, along with political risk (of research findings being politically unpalatable) and ‘structural’ (government/non-government organisations and university/academic institutes work in silos) (35.2% for all).

Perceptions of other barriers

Figure 1 and Figure 2 show a mix of agreement and discordance between actual and perceived barriers to collaboration. Some barriers commonly perceived by researchers, such as ‘politics’, negative past ‘experiences’ (of collaboration with academics/researchers), ‘lack of trust’ and ‘differing priorities’ were not explicitly reported as such by policy actors. There was greater agreement between researchers’ perceptions and actual barriers reported by policy makers regarding ‘budget’ and ‘capacity’ (insufficient or lack of resources/time to support collaboration). While policy actors perceived that ‘contacts’ (not knowing who to approach/collaborate with) and ‘rigour’ (compromising research rigour to meet policy making time constraints) would be seen as barriers by researchers, most academics and researchers did not consider them as actual barriers. There was a greater alignment between policy makers’ perceptions and the actual barriers reported by researchers for ‘expectations’ (research output unlikely to resolve the policy issues) and ‘understanding’ (not knowing each other’s processes).

Figure 1.     Actual barriers reported by policy actors vs barriers as perceived by academics/researchers (click figure to enlarge)

Figure 2.     Actual barriers reported by academics/researchers vs barriers as perceived by policy actors 

Reported enablers

More than half the policy actors (57%) and most academics/researchers (71%) indicated that their organisation was very/extremely supportive of facilitating engagement with the other.

Among policy actors, common enablers of collaboration with academics/researchers were reported to be ‘leadership’ (29%) (their government agency/NGO encourages collaboration with academics/researchers and promotes evidence-informed policy); ‘connections’ (22%) (knowledge of who to engage within the university/academic institutes), and ‘priorities’ (23%) (a ‘common purpose’ or shared agenda).

Among academics/researchers, important enablers reported were also ‘connections’ (26%) (knowledge of who to engage within the government/non-government organisation) and ‘leadership’ (23%) (their university/institute encourages collaboration with policy actors).

The themes from an open-text field suggested numerous common enablers or solutions to overcome barriers provided by policy actors and academics/researchers. Examples of enablers and suggested ways to support working together were categorised by type of knowledge translation strategy (push, pull or exchange).²⁰ See Supplementary Table S1 for further details (available from figshare.com/s/cdf7bf55dac775cabf73).

Difficult and successful experiences

Many academics/researchers (65%) and policy actors (49%) reported having had a poor experience working with the other. At the same time, the vast majority (80%) of both academics/researchers and policy actors also reported having had a successful collaboration with the other. Few respondents provided further information, but some open-text comments revealed that misalignment between the expectations of policy actors and academics/researchers created additional barriers. For example, a researcher commented that: “the policy actor had demonstrated a lack of genuine and shared agenda” while a policy actor commented: “the researcher was just looking for money to fund their project. Project did not [meet] our organisational goals”. Further comments reinforced the challenge of time and money for policy actors – “non-alignment of timing, budget, priorities etc.” as well as academics/researchers – “time, capacity and funding issues”.

More than one-third of policy actors (34%) and academics/researchers (40%) reported having had an opportunity to collaborate but choosing not to do so, with reasons provided including a lack of real-world insight from the academics/researchers. For example, policy actors commented they were: “put off by previous experience of, lack of real-world insight and leadership from the academic lead researcher”. Researchers commented on a lack of commitment or tokenistic approach to the use of evidence by the policy actors; for instance, “as the project progressed, it was clear that the appetite to implement the findings was lacking. It seemed pointless to keep swimming against the stream”; and “could see it was a tokenistic approach”.

Skills for collaboration

Policy actors and academics/researchers were asked to rank the top three behaviours/skills for collaboration, among which substantial commonalities were identified, as shown in Table 4.

Table 4.     The top three behaviours/skills for collaboration as ranked by participants

Discussion

This study presents experiences and insights from both policy actors and academics/researchers regarding research-policy collaborations. It also identifies key enablers and barriers to research-policy collaborations, highlighting the discordance between perceived and actual barriers. With participation from all Australian states and territories, the survey includes a balanced mix of senior officers (executive, senior, or middle management) and project officers, making the findings applicable to various decision-making contexts. This study adds to the expanding body of literature on research-policy collaborations by offering quantitative data that enhances the understanding of collaboration experiences for both policy actors and academics/researchers. It complements recent qualitative studies, thereby contributing to a more comprehensive understanding of the topic.^15,17,21

The majority of policy actors and academics/researchers reported having worked collaboratively with the other party, which is likely a reflection of the study recruitment process purposively targeting those with experience of engagement. This study suggests that policy actors are more likely to initiate engagement than academics/researchers. Previous studies present mixed results on who initiates the collaborations, with some indicating that such partnerships are mostly initiated by policy makers²², others suggesting researchers take the initiative²³, and some reporting joint or equivalent initiatives by both parties.^9,24 Our finding is unsurprising given researchers have fewer drivers for engagement compared to policy makers.²³ The finding can also be partly attributed to Australian research funding structures, which primarily incentivise the end of the grant stage while overlooking the initial stages, such as synthesis and initiation.²⁵ This is critical, given time and funding constraints are crucial determinants of whether or not researchers can initiate contact for engagement.²³ While policy-initiated collaborations are thought by both researchers and policy makers to be the most likely to result in impact¹⁵, other authors have found that outcomes did not appear to be associated with the initiator of the partnership.²² Overall, it is widely recognised that work co-initiated and co-designed by researchers and policy makers is more likely to have a significant impact in the real world^15,22,23, although it is practically challenging to conduct.¹⁵

The purpose of engagement is closely related to the initiation of engagement, which for the majority of both policy actors and academics/researchers in this survey, was evidence-informed policy making, or improving public policy outcomes (rather than, for example, needs identification, funding decisions or advocacy). While this is in line with previous studies that show the reason most cited by researchers for wanting to partner with policy makers is to increase the likelihood of research impact^15,21, the survey findings showed that policy makers initiate engagement with researchers for the same reason. A previous study found that policy makers primarily initiate partnerships to access additional skills and capacity not available within their agency.¹⁵ This inconsistency may be attributed to differences in research methodology. Our study used a survey that provided limited response options, whereas qualitative interviews allow for unexpected topics to emerge.²⁶ Consistent with existing literature in the public health policy field, this study highlighted the crucial role of existing networks in initiating and sustaining research-policy partnerships.^9,16

Despite Australian academic institutions encouraging academics and researchers to establish connections with industry and business for enterprise and commercialisation purposes, there are few internal incentives for them to engage with policy actors. This is particularly true in fields such as health sciences and humanities, where the knowledge produced is intended for public policy but where engagement brokering is limited Furthermore, the policy-making process itself is embedded in the dynamics of political discourse and debate, which can create uncertainty, instability and, importantly, opportunity. Our findings indicate that neither the academics/researchers nor the policy actors took full advantage of the potential partnership opportunities presented in the policy-making process. The reasons for not engaging align with previous reports that highlight a lack of engagement from researchers²⁷, while policy makers tended to disregard external academic research²⁸ due to concerns about timeliness, relevance, inadequate communication, and controversial nature.²⁹ Additionally, policy makers generally face constraints in terms of time and resources, making it challenging for them to establish connections with researchers or effectively use the available evidence²⁷, even when it is pertinent.³⁰

The key barriers cited were structural, namely, that there are limited budgets, structures and systems that encourage collaboration between policy actors and academics/researchers and that government agencies operate in silos. Collectively, this creates challenges when trying to initiate relationships and form collaborations across government sectors and departments.³¹ The findings align with a Canadian study that revealed resource limitations (financial/time) and differences in participation and contribution among team members hindered successful collaborations between researchers and policy makers.³² The same issue applies to academic institutions, adding a major bi-directional structural impediment to the forming of research-policy partnerships. These findings are unsurprising considering that government agencies and non-government organisations are traditionally arranged in siloed portfolios, and as noted earlier, there are few incentives for academics to reach out to policy actors in Australia.^25,32 However, we believe there is great promise in the emerging ‘horizontal’ partnership arrangements that provide an authorising environment for different sectors to work together within governments. Some examples include South Australia’s Public Health Partner Authority agreements and international joined-up policy efforts such as Health in All Policies.³³ Further, while the policy impact of funded research is often evaluated in Australia, the effectiveness of the actual partnering mechanisms/arrangements and the conditions that lead to a successful collaboration between policy actors and academics/researchers have not yet been objectively studied.

Emerging literature on the role of trust in the formation of research-policy translation partnerships offers promise and points to the need for government and universities to invest in structures and systems that link academics/researchers and policy actors together, enabling relationships and trust to develop.³⁴ This is consistent with a recent scoping review that found interpersonal links are indeed important in the production and use of relevant evidence but need to be underpinned by long-term strategic and institutional support.¹¹ The majority of policy actors in our study agreed that it is important for the policy-making process to be linked to research, and many experienced supportive organisational cultures for policy-research engagement. An important enabler contributing to this was perceived to be leadership in government agencies, as well as in universities/academic institutes. There is a need to evaluate policy-research engagement initiatives more rigorously, their preceding conditions, processes, and impacts/outcomes.

Based on our findings, suggestions for improving engagement between academics/researchers and policy actors in the policy-making process include developing strategies that create space for academics/researchers and policy actors to meet, connect and establish trusting relationships. Such interaction and exchange has been identified as a key to promoting the use of research evidence in policy.⁹ Interactions can include professional development and training courses, joint networks, and other engagement platforms. As well as the need to define motivations for engagement and co-production³⁵, the survey results point to the importance of skills that underpin collaboration, co-design and co-production – key strategies to support evidence-informed policy making. Policy actors and academics/researchers need to invest in building their collaboration and relationship skills, as these are primary requirements for successful research-policy partnerships.²¹ Both groups can play an important role in challenging the structural issues that inhibit research-policy translation.

Why this matters in public health

Collaboration with researchers is a key facilitator to evidence uptake by policy makers¹⁴, which is critical for improving public health. The study has particular relevance to public health as participants were predominantly health science researchers and policy makers in the healthcare and social assistance sector. The survey responses and comments shed light on the experiences and perceptions of those involved in collaborations in public health. The findings contribute to the growing body of literature on research-policy collaboration – an important area of study for public health researchers – and can inform future research and practice in this area, both in Australia and globally.

The strengths and limitations of the research

The survey sought to understand the research-policy experiences of both academics/researchers and policy actors using the same instrument, enabling the comparison of both groups within a single sample. The survey instrument was adapted from previous work led by the South Australian Health in All Policies initiative to assess collaboration experiences across the public sector and between state and local government. The adapted survey instrument was piloted with policy actors and academics/researchers from across jurisdictions and disciplines.

The research used a convenience sample and snowballing technique, relying on the networks of the working group members. This resulted in oversampling from South Australia and from the field of public health. The sample favoured people who had some collaboration experiences, so findings about why people do not engage in collaborative processes may not be generalisable to all policy actors or academics.

Conclusion

This study provides a deeper understanding of how policy actors and academics/researchers collaborate for the purpose of advancing evidence-informed decisions within the policy-making process. It also highlights the importance of existing networks and demonstrates challenges faced by both the research and policy communities in knowing who to collaborate with in the research-policy translation process.

The discordance in perceived versus actual barriers to collaboration between both academics/ researchers and policy actors illustrates the level of disconnection and disengagement between the research and policy worlds. It demonstrates the need for capacity-building initiatives to improve the alignment between policy and research.

Potential impacts of these collaboration experiences on research and decision making include the need for academic institutions and the public sector to be deliberate in their efforts to deliver research-informed policy. Future initiatives could use the study findings to implement preferred methods of collaboration, together with rigorous evaluation, to understand ‘what works’ in promoting engagement for evidence-informed policy.

Peer review and provenance

Externally peer reviewed, not commissioned.

Copyright:

© 2023 Williams et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.