Objectives: We examine how health policy and systems research (HPSR) is produced and funded in 14 post-Soviet states to inform possible strategies to improve the supply and quality of research and advance evidence-based health policy making in these states.
Study type: Mixed methods.
Methods: Using mixed methods – secondary data analysis, desk review and in-depth interviews – this qualitative study is exploratory and explanatory.The secondary data analysis involved a comparative analysis of available data from: a) the ‘fiscal space’ (national economic resources) – using gross domestic expenditure on research and development for the years 2013–2018; and b) capacity for HPSR – using the number of published papers and average citation per paper (as a quality proxy) in the years 2015–2019. To explain the secondary data analysis findings, we used the approach proposed by Hallerberg et al., highlighting the importance of institutional context, actors, and their incentives and influence in budget allocation decisions. The desk review of available documents and 32 in-depth interviews were conducted remotely to obtain insights on the context and actors. The interview transcripts were analysed using Nvivo 12 software with an inductive approach.
Results: In all studied countries, except the Baltic states, funding levels for HPSR remain inadequate. Most research and development funding is allocated to fundamental sciences and biomedical research – fields with more influential long-standing institutional legacies. The low volume and poor quality of published HPSR research appears to be adversely affecting the credibility of researchers in this field in the eyes of critical beneficiaries – policy makers, who do not prioritise and advocate for funding of HPSR.
Conclusions: HPSR funding in most post-Soviet countries is caught in a vicious cycle of inadequate funding and poor quality. International collaborative projects focused on post-Soviet states and involving science funders, academic institutions and researchers from those countries may help strengthen HPSR capacity, improve research quality and help boost priority funding and the credibility of researchers in this field.
Health policy and systems research (HPSR) is a relatively new field of scientific inquiry that explores how societies achieve collective health goals and how different policy actors interact in policy formulation and implementation processes to produce health policy outcomes. HPSR is interdisciplinary and uses economics, sociology, anthropology, political science, public health and epidemiology to enrich knowledge about the interconnection of health systems and health policies, and how they are affected by broader determinants of health.1 The purpose of this study is to gain a broader understanding of HPSR funding in post-Soviet states to inform possible strategies, policy dialogues and advocacy work with state funders, and global health organisations, to help increase overall resources for HPSR. Geographic scope is limited to former Soviet states, including diverse subgroupings such as the Baltic states, which became part of the European Union and therefore are contrasted with other subregions. The other commonly accepted subgroupings are the European subregion: Russia, Belarus, Ukraine and Moldova, and the subregions of Central Asia and South Caucasus.
To gain insights into the level of HPSR development and funding, we based our work on the understanding that public budget decisions result from a collective process involving various political actors (institutions and individuals), each with their motivations. Because of the preferences of ‘participants’, budget processes may lead to several different results/outcomes under various institutional arrangements. Thus, we used the approach proposed by Hallerberg et al.2, where it is essential to know the institutional context within which the actors interact, who the actors are, and what their power and incentives are when making budget allocation decisions.
We first interrogated gross domestic expenditure on research and development (GERD) using UNESCO data3 for 2013–2018 and drilled down to research fields, where HPSR logically belongs, to reveal potential ‘fiscal space’ for HPSR. Next, we explored research ‘capacity’ using SCImago Journal & Country Rank (the largest abstract and citation database of peer-reviewed literature)4 for the period 2015–2019 (lagging science outputs by a couple of years) and evaluated the volume of published peer-reviewed papers categorised as ‘health policy’, ‘health informatics’, ‘health information management’, ‘health social science’, and ‘public health’ in the database. We used these categories as a proxy for estimating ‘HPSR production capacity’ and an average citation per paper as a proxy for the ‘quality’ of the scientific outputs.
After that, to analyse the institutional context within which the actors interact and explore their incentives and influence in the budget process, we used qualitative information from a desk review and 32 in-depth interviews. For the institutional context analysis, we covered 14 countries of the former Soviet Union. To understand actor incentives, we used convenience sampling of respondents representing policy makers in science funding institutions, decision makers from ministries of health, and researchers working and publishing in the HPSR field. This list was compiled from agency-specific websites, and researchers were identified using published papers in English or Russian. The respondents were selected from six countries (Belarus, Russia, Kazakhstan, Uzbekistan, Kyrgyzstan and Georgia), representing all subregions except the Baltic states. Interviews were conducted remotely using the Microsoft Teams platform and transcripts were analysed using Nvivo software (Melbourne, Australia: QSR International; NVivo 12) with an inductive approach.
Ethics approvals were not sought as most of the data for the analysis were from the public domain and informed consent for the use of the anonymised data was obtained from each individual interviewed, as per local policy. Each interviewee was sent an invitation letter from the WHO Alliance for Health Policy and Systems Research (AHPSR) with information about the study and an informed consent form. The AHPSR letter (in Russian) is available here: figshare.com/articles/media/Letter_of_Support_Russian_and_Consent_Form/16533183/1
Using data on GERD spending for the countries in our scope helped us evaluate overall funding for science, and specifically for the medical, health and social science fields. Table 1 presents the overall research and development funding by country and spending levels on medical, health and social science research fields, through which HPSR is likely to be funded. Although there is no internationally recommended optimal level of funding for HPSR, for example for overall health research at 2% of national health expenditure5, the comparative analysis of funding per capita still sheds light on the level of research prioritisation across countries.
As shown in Table 1, poorer countries devote 0.1–0.3% of GDP to GERD, with the lowest levels in Central Asia. The wealthier Baltic states and countries in the European subregion (except Moldova) spend about two to four times more (0.6–1.4%).
Of these GERD resources, the share allocated to medical, health and social science research fields ranges from 5.9% in the Russian Federation (2013) to 30.3% in Georgia (2013), pointing to the varying importance of these fields in science funding priorities. However, when funding for medical, health and social science is measured as a percentage of GDP, countries in the South Caucasus, Central Asia and the European subregion devote 5 to 10 times less than the Baltic states. These levels translate into a minimum of $0.4 per year per capita for Kyrgyzstan to $77 per capita for Estonia in purchasing power parity dollars ($PPP). Finally, in 2018 Central Asia, South Caucasus and the European subregion on average devoted about 12% of their GERD to the medical, health and social science fields while the Baltic states gave greater importance and, on average, allocated 16.4% of their GERD to this field.
When looking at sources for GERD, governments’ critical role in overall research funding is evident. The business sector also features prominently across all countries except for Georgia, where higher education institutions and the government provide almost 85% of GERD. Extraterritorial funders (e.g., the European Community or non-government organisations) contribute about 10% (depending on the country), except in Ukraine, where their share is the largest – 22% of GERD.
Table 1. Spending on overall research and development, and on medical, health and social science research3 in post-Soviet countries
Country | GERD as a percentage of GDP | Medical, health and social science research as a percentage of GDPa | Medical, health and social science spend per capita (in current $PPP) | |||
2013 | 2018 | 2013 | 2018 | 2013 | 2018 | |
South Caucasus | ||||||
Armenia | 0.22 | 0.19 | 0.02 | 0.03 | 1.98 | 3.12 |
Azerbaijan | 0.21 | 0.18 | 0.03 | 0.03 | 4.75 | 4.74 |
Georgia | 0.08 | 0.30 | 0.02 | na | 2.02 | na |
European subregion | ||||||
Belarus | 0.65 | 0.61 | 0.05 | 0.04 | 9.65 | 8.04 |
Republic of Moldova | 0.30 | 0.25 | 0.05 | 0.05 | 2.86 | 3.23 |
Russian Federation | 1.03 | 0.99 | 0.06 | 0.06 | 15.80 | 18.09 |
Ukraine | 0.76 | 0.47 | 0.07 | 0.03 | 6.02 | 2.89 |
Central Asia | ||||||
Kazakhstan | 0.17 | 0.12 | 0.02 | 0.01 | 4.18 | 2.45 |
Kyrgyzstan | 0.15 | 0.11 | na | 0.01 | na | 0.40 |
Tajikistan | 0.12 | 0.10 | 0.02 | 0.02 | 0.52 | 0.73 |
Uzbekistan | 0.20 | 0.13 | na | 0.02 | na | 1.79 |
Baltic states | ||||||
Estonia | 1.72 | 1.43 | 0.25 | 0.26 | 69.69 | 76.93 |
Latvia | 0.61 | 0.63 | na | 0.10 | na | 23.45 |
Lithuania | 0.95 | 0.94 | 0.21 | 0.14 | 53.86 | 45.03 |
Figures 1a and 1b reveal increased production of published peer-reviewed papers, especially during the past 2 years in the selected science fields, primarily from the European subregion that produced almost 80% of all papers (although Russia mainly drove the growth). In other subregions, the volumes are low but growing except in the Baltic states. When published papers are adjusted to population, Central Asia produced about eight papers per million inhabitants during 2015–2018 and the Baltic states produced 178, followed by South Caucasus (32.3) and the European subregion (30.5), indicating subregional differences in production capacity. Although countries in the European subregion produce more in absolute terms, the average citation per paper is the lowest at 3.8. In contrast, the absolute number of papers published in the Baltic states is six times less, but with the highest average citation – 13.6, followed by South Caucasus – 9.3. Notably, subregional averages hide significant country differences, such as papers from Georgia having on average 10 citations and from Kyrgyzstan 17.2, while countries in Baltic states do not have substantial variability.
The low level of publications per capita and low citations (except the Baltic states and Kyrgyzstan) may indicate a weak HPSR production capacity generating lower impact papers.
Figure 1a. Citable documents in selected fieldsa, 2015–2019 (click to enlarge)
Figure 1b. Citable documents and average citation by subregion in selected fieldsa, 2015–2019 (click to enlarge)
After the dissolution of the Soviet Union, newly independent states inherited the Soviet legacy system of science organisation and financing, primarily focused on funding institutions and not researchers (or research ideas/projects). Two funding schemes for science featured prominently in the reviewed countries for some time: a) funding research institutions from a science budget flowing through academies of science (which organised, managed and funded the research production) and b) funding institutions organised under ministry lines (in the health sector, ministries of health) from sectoral budget allocations. In both instances, funding was focused on state-defined priorities and institutions. With some rare exceptions, universities were focused on education without clear linkages to science production and/or science funding.
The fall of the Soviet Union, accompanied by significant economic decline in all republics6, led to a reduction in science funding in all countries, although to varying degrees. The limited science budgets were spent on ‘sustaining’ the extensive research infrastructure institutions, covering subsistence-level salaries for scientists and utility expenses of institutions with no linkages to scientific outputs. The focus on sustaining institutions, and underpaid jobs, made science unappealing for younger generations.7-11 Consequently, underfunded and institutionally obsolete science fields existed for almost two decades until the reforms described in Table 2 began. To restructure the science sector and reduce inherited inefficiencies, and the institutional influence of the academies of science or research institutions under ministries, most countries established ‘science funds’ and switched to competitive grant–based financing, with variable results.
Table 2. Institutional arrangements for science funding for medical, health and social fields
Region/country | Science funding organisations and year of their establishment |
South Caucasus | |
Armeniaa | Ministry of Education and Science through State Committee of Science, 2007 National Academy of Sciences of Armenia |
Azerbaijan | Azerbaijan National Academy of Sciences, 2003 |
Georgiaa | Shota Rustaveli National Science Foundation of Georgia, 2010 The Georgian Innovation and Technology Agency, 2014 |
European Region | |
Belarus | The Belarusian Republican Foundation for Fundamental Research, 1991 National Academy of Sciences of Belarus The Ministry of Health and subordinated research institutions |
Republic of Moldova | Academy of Sciences of Moldova, 2004 The Ministry of Health and subordinated research institutions |
Russian Federationa | Russian Science Foundation, 2014 Ministry of Health and subordinated research institutions |
Ukrainea | National Research Foundation of Ukraine, 2018 National Academy of Sciences of Ukraine The Ministry of Health and subordinated research institutions |
Central Asia | |
Kazakhstana | The Science Fund, 2006 The Ministry of Health and subordinated health research institutions |
Kyrgyzstana | Ministry of Education and Science – Managing Science Fund with the help of the Academy of Science, 2015 The Ministry of Health and subordinated research institutions |
Tajikistan | Academy of Sciences of the Republic of Tajikistan, 2011 |
Uzbekistana | Science and Technology Agency, 2017 |
Baltic states | |
Estoniaa | Estonian Research Council, 2012 |
Latviaa | Latvian Council of Science, 2009 |
Lithuaniaa | The State Studies Foundation, 2008 |
Although competitive funding schemes were introduced, most reports and respondents confirm that old institutions and ‘influential researchers’ still dominate funding priorities. According to in-depth interview respondents, institutional arrangements for priority setting in science (and in the medical and HPSR fields) are blurred and patchy at best or nonexistent in most instances, except in Kazakhstan. In most studied countries, 50–80% of the science budget is allocated for fundamental research, compared with the developed world, where about 20% is spent on fundamental/basic science. The rest is allocated to applied science fields, to which HPSR belongs. According to the respondents, and corroborated with the desk review results, the low level of financing for applied fields indicates low demand for evidence from policy/decision makers.8-12 The influence of the ‘old’ institutions is further confirmed by national budget allocations for science where ‘politically important’ institutions receive a higher share of budget funds; e.g, in Russia the Academy of Sciences absorbs almost 40% of the capital budget for science. In contrast, universities receive only 5.2%.7 Similar trends were noted elsewhere by interview respondents.
A lack of transparency in priority setting is compounded by a lack of trust in the newly established grant-making systems among scientists in the region7,9,11,13 because most funding goes to those involved in reviewing proposals and/or to people closely affiliated with the grant decision-making process. These individuals and/or institutions determine ‘priorities’ for research and funds flow to ‘well-connected’ institutions and their research priorities, often lacking relevance to national and societal needs. Competitive funding mechanisms face additional challenges, except in Baltic states, related to application/proposal review processes. Establishing independent and objective review panels, free from conflicts of interest (institutional or individual), proved challenging across the post-Soviet region. Only the Russian Federation and Georgia are showing initial progress in increasing the objectivity of review processes. With its sizable pool of reviewers, Russia introduced blinded peer review, and Georgia, due to its small size, outsourced the process to an extraterritorial entity.
The pattern described above for general science repeats itself in the health research area, where most domestic resources are diverted to biomedical research and almost none to HPSR. The exception is funding flowing from ministries of health to subordinated institutions, where funds are used for HPSR and the ministry officials define priorities. However, these funds are not accessible to scientists based in universities or other research institutions. According to respondents, only in countries where bi- or multilateral donor assistance exists (e.g., Georgia, Kyrgyzstan, Uzbekistan) are HPSR priorities defined and funded through discussions and negotiations with funders. Although the process is not inclusive and/or participatory, it forces governments to think through and produce HPSR priority lists even if only for 2 years at most.
Quantitative findings about HPSR production capacity were also reconfirmed during the in-depth interviews. The respondents named several explanatory factors:
Based on the results of this study, it seems that the HPSR field in post-Soviet states, except the Baltic states, is caught in a vicious developmental cycle rooted in the Soviet legacy, as schematically depicted in Figure 2. In the studied countries, funding levels for GERD, while slowly growing, remain low compared with levels seen in the Baltic states. The low dedicated GDP shares also translate into meagre absolute dollar amounts for science, especially in poorer countries. Furthermore, those involved in fundamental research, and having legacy institutional standing, are still influencing budget priorities, leaving little to the medical, health and social science fields, and marginalising applied science fields, to which HPSR belongs. HPSR is also losing within health research allocations to dominant players representing biomedical research.
Figure 2. Vicious cycle of HPSR funding in post-Soviet states (click to enlarge)
These ‘lost battles’ for budget dollars have their explanations. Namely, those expected to champion the importance of HPSR seem to be weak compared with long-standing and influential research institutions inherited from Soviet times under academies of science or line ministries. The Soviet legacy of science funding, although evolving, is still ‘forcing’ public health schools to focus more on education than on research, thus limiting any positive spill-over effects for HPSR from the internationalisation of higher education (except schools in the Baltic states). Although the situation is slowly tilting towards conducting more research, progress seems extremely slow and not sufficient to empower participants in HPSR to fight for budget funds. Furthermore, the quality and volume of HPSR research produced is low, further inhibiting the empowerment of HPSR researchers and possibly challenging their credibility in the eyes of policy makers. Consequently, interactions between policy makers and the HPSR community are weak at best or non-existent in most instances, causing negative effects on research priority setting and funding allocations for HPSR within national science budgets.
Finally, as described in the in-depth interviews, the complexities are compounded by institutionally weak science funders, who make non-transparent funding decisions and remain under the influence of dominant establishments like the academies of science. In an environment where transparent processes for national priority setting are weak or absent; where proposal review processes are non-transparent and are not free from institutional or personal conflicts of interest; and where legacies of ‘renowned’ individuals or institutions have a significant bearing on funding decisions, it seems HPSR will lose the battle for domestic dollars for some years to come unless timely solutions are found.
This study has a number of limitations. Firstly, the international classification of research spending14 includes six broad categories, including ‘medical and health sciences’ and ’social sciences’, which include the HPSR field. The lack of a statistical code for HPSR makes it impossible to separate HPSR funding within these groups. Also, some applied HPSR is embedded in sectoral programs and may not be classified as research spending under national accounting frameworks, making HPSR expenditure invisible. Therefore, we have used proxy measures to reveal ‘potential’ fiscal space for HPSR funding.
Secondly, the use of the volume and citation index for measuring the capacity and quality of HPSR, while accepted for comparative studies of research capacity and quality7,15,16, has significant limitations. For example, the ‘Matthew effect’, under which more prominent researchers may have their work cited more, simply due to name recognition rather than the quality of the publication17 and low-quality papers may have a high citation index because many authors spent efforts on their critiques.17 In our studied region, language bias may feature more prominently among these limitations because countries publish mostly in native or Russian languages and in national journals not linked to international citation databases. According to the 2019 annual report of the Russian Science Foundation18, only half of the scientific publications produced by Russian scholars were listed on the Web of Science (WoS) databases. According to the Master Journal List, Russia has 161 national journals indexed in the WoS, Ukraine and Estonia have 11 each, Azerbaijan two and Armenia, Belarus, Georgia and Uzbekistan one each. Kazakhstan, Kyrgyzstan, Moldova, Tajikistan, and Turkmenistan have no national journals indexed.19 Thus, important research papers that may have influenced health policies in these countries may have been ‘invisible’ due to language bias. However, the low share of papers listed in the WoS could also be seen as a significant weakness of the science sector in these countries.19
Thirdly, due to time and resource constraints we were unable to interview key informants from the Baltic countries to fully explore the institutional context and factors that may have determined the observed major difference in HPSR funding and capacity compared with other former Soviet countries.
HPSR funding and development challenges in most post-Soviet countries are multifaceted, mostly inherited from Soviet times and persisting due to a vicious cycle. Breaking the cycle would require external assistance, as seen for the Baltic states after joining the European Union. Extensive international collaboration in HPSR, involving primarily academic institutions through twinning and collaborative projects through European Commission science funding schemes20 such as Horizon Europe and Erasmus+, with existing capacity from European countries, may help to foster production capacity and improve quality. World Health Organization networks and partnerships, such as the Alliance for Health Policy and Systems Research, and the European Observatory on Health Systems and Policies, could intensify their HPSR capacity building efforts directed at former Soviet countries. They could also mobilise more support from bilateral donors and the European Commission by advocating for HPSR as a funding priority. Involving national science funders through collaborative projects may also be beneficial for increasing awareness of, and long-term involvement in, HPSR. Finally, external collaborators could offer greater legitimacy to national actors and increase their credibility in the eyes of national policy makers to positively influence science funding.
This work was supported by the Alliance for Health Policy and Systems Research, World Health Organization. The authors are themselves alone responsible for the views expressed in the article. This article does not represent the views, decisions, or policies of the Alliance for Health Policy and Systems Research.
Externally peer reviewed, invited.
Copyright:
© 2021 Gotsadze and Zoidze. This article is licensed under a Creative Commons Attribution 3.0 IGO Licence, which allows others to redistribute, adapt and share this work for any purpose provided they attribute the work and indicate if changes were made.