The ‘how to’ of scaling up public health interventions for maximum reach and outcomes is receiving greater attention; however, there remains a paucity of practical tools to guide those actively involved in scaling up processes in high-income countries. To fill this gap, the New South Wales Ministry of Health developed Increasing the scale of population health interventions: a guide (2014). The guide was informed by a systematic review of scaling up models and methods, and a two-round Delphi process with a sample of senior policy makers, practitioners and researchers actively involved in scaling up processes. Although it is a practical guide to assist health policy makers, health practitioners and others responsible for scaling up effective population health interventions, it can also be used by researchers in the design of research studies that are potentially suitable for scaling up, particularly where research–practice collaborations are involved. The guide is divided into four steps: step 1, ‘scalability assessment’, aims to determine if an intervention is scalable; step 2, ‘developing a scale up plan’, aims to develop a practical and workable scaling up plan that can be used to convince stakeholders there is a compelling case for action. Step 3, ‘preparing for scale up’, aims to identify ways of securing resources needed for going to scale, operating at scale, and building a foundation of legitimacy and support to sustain the scaling up effort through the implementation stage; and step 4, ‘scaling up the intervention’, involves putting the plan developed in step 2 into place. Although the guide is written as though the user is starting from the point of assessing the scalability of an intervention, later steps can be used by those already involved in scaling up to review their implementation processes. The guide is not intended to be prescriptive. Its purpose is to help policy makers, practitioners, researchers and other decision makers decide on appropriate methodological and practical choices, and balance what is desirable with what is feasible.
Population-wide health improvements depend on wide-scale implementation of health interventions that have been found to be effective in a controlled research setting. This usually requires scaling up interventions to reach broader populations or multiple settings. Although the scale of an intervention may seem an obvious concept, the terms ‘scalability’ and ‘scaling up’ have been applied in different ways and contexts.1 They have been variously applied in the literature to describe the dissemination of a new technique, prototype product or process innovation; epidemiological and economic forecasting; ‘growing’ an organisational or system capacity to implement to a new level; and translating a small-scale initiative into a government policy.2,3
For clarity, specific definitions for ‘scalability’ and ‘scaling up’ are needed. In the health sector, scalability is “… the ability of a health intervention shown to be efficacious on a small scale and or under controlled conditions to be expanded under real-world conditions to reach a greater proportion of the eligible population, while retaining effectiveness”.4 Scaling up is “… deliberate efforts to increase the impact of successfully tested health interventions so as to benefit more people and to foster policy and program development on a lasting basis”.5 Scalability is the capacity of an individual intervention to be scaled up, while scaling up is a process of increasing the scale of a successfully tested health intervention. Of course, some health interventions that have been shown to be effective in research contexts will never be scalable because of a lack of strategic alignment, cost or incompatibility with existing infrastructure.
The issue of how to scale up health interventions is receiving greater attention in the health literature.1,4-7 Although a number of frameworks and guides exist4,5,8 they have not been informed by empirical examination of how scaling up decision making and processes occur in high-income countries. To fill this gap, the New South Wales (NSW) Ministry of Health developed Increasing the scale of population health interventions: a guide.9 The guide was developed using a systematic review of scaling up models and methods3, and a two-round Delphi process with a sample of senior policy makers, practitioners and researchers actively involved in scaling up processes.2,10 It was developed as a practical guide to assist health policy makers, health practitioners, and others with responsibility for scaling up effective population health interventions.9 The guide is divided into four steps: scalability assessment, developing a scaling up plan, preparing for scaling up, and scaling up the intervention. This article provides a concise synthesis of the guide’s key scale-up steps (Table 1).
Table 1. Steps in the scaling up process
|Step 1. Scalability assessment: assess the suitability of the intervention for scaling up||1.1 Assess effectiveness||Determine effectiveness, intervention effect size, unintended consequences and differential effects|
|1.2 Assess potential reach and adoption||Determine if the likely reach and adoption of the intervention is extensive enough to have a population impact|
|1.3 Assess alignment with the strategic context||Determine whether the intervention is consistent with national, state or regional policy directions. Even highly effective interventions may struggle to obtain funding if they are not aligned with the priorities of funding agencies|
|1.4 Assess acceptability and feasibility||Judge whether the intervention could realistically be scaled up, given what is known about its costs, workforce requirements, time required, infrastructure requirements and acceptability to stakeholders|
|Step 2. Develop a scaling up plan: outline a vision of scale-up and a compelling case for action||2.1 Document a rationale for scale-up||Draw up a rationale for scaling up from the information in Step 1, noting that further investigation and analysis may be necessary to provide a compelling case for action|
|2.2 Describe the intervention||Describe ‘what’ will be scaled up and where possible the original intervention should be simplified and streamlined|
|2.3 Complete a situational and stakeholder analysis||Map the social, political and organisational environment(s) in which the intervention will be scaled up and identify potential barriers and enablers to scale-up|
|2.4 Determine who could be involved in scale-up and what their role will be||Consider who might perform key functions when the intervention is scaled up by mapping key functions and matching them to those who could potentially be involved|
|2.5 Select an approach to scaling up||There are two main approaches to scaling up. A vertical approach involves the introduction of an intervention simultaneously across a whole system and results in institutional change through policy, regulation, financing or health systems change. A horizontal approach involves the introduction of an intervention across different sites or groups in a phased manner. These approaches are not mutually exclusive, and a combination of approaches can be used|
|2.6 Consider options for evaluation and monitoring||Determine what variables are important to measure over time and determine feasibility and associated cost of these systems|
|2.7 Estimate resources required for scale-up||Estimate the human, technical and financial resources needed to scale up the intervention|
|2.8 Write up the scaling up plan||The plan should present a clear and concise case for scaling up the intervention, as well as an overview of how this will be brought about, including a vision of what scaling up will look like if successfully completed|
|Step 3. Prepare for scale-up: secure resources and build a foundation of legitimacy for the scaling up plan||3.1 Consult with stakeholders||Assess the appropriateness and acceptability of the intervention and the scaling up plan and use this information to design advocacy and communication strategies|
|3.2 Legitimise change||Gain the support of decision makers who must be convinced that scaling up the intervention is a credible and superior solution to a pressing problem, is for a priority population and that it is affordable|
|3.3 Build a constituency||Mobilise the broader ‘community of practice’ required to successfully scale up an intervention|
|3.4 Realign and mobilise resources||Mobilise financial resources through existing channels or through new funding streams. Ensure that resources are directed to address skill and other capacity deficits in delivery organisations|
|Step 4. Scale up the intervention: implement the scale-up plan, making necessary adjustments based on performance data||4.1 Modify and strengthen organisations||When scaling up interventions, most organisations need to adapt. Manage organisational change through processes such as staff retraining, mentoring, leadership development and coaching|
|4.2 Coordinate action and governance||Develop and implement concrete and detailed agreements about how, when, where and by whom resources are to be used, and the governance structures that will be used to identify issues and resolve any disputes that may arise|
|4.3 Monitor performance and efficiency||Develop systems that have an ongoing focus on measuring effectiveness, reach, fidelity, acceptability and costs, with a particular focus on the efficiency of intervention delivery|
|4.4 Ensure sustainability||Implement organisational and cultural changes to institutionalise an intervention so that it becomes part of routine practice|
Although the guide is written as if the user is starting from the point of assessing the scalability of an intervention, later steps can be used by those already involved in scaling up to review their implementation processes. The guide is not intended to be prescriptive. Its purpose is to help policy makers, practitioners, researchers and other decision makers make appropriate methodological and practical choices. It has grown out of population health experience and is therefore written from a population health perspective; however, its core concepts could be applied to other human service endeavours, particularly those that involve programmatic interventions.
The guide can also be used by researchers in the design of research studies that may be suitable for scaling up. For example, step 1 could be used to identify research gaps and to guide researchers towards seeking funding to address such gaps. Similarly, it may be used to assist researchers to present intervention research findings, thus providing the information necessary for health practitioners and policy makers to assess the scalability of an intervention. In addition, the later stages of the guide can be used by researchers to identify opportunities for partnering with evaluation and monitoring efforts when interventions are scaled up.
A number of key concepts in the guide merit closer examination. The key prerequisite for scaling up a population health intervention is that it is effective.4-6 For the purpose of this guide, an effective intervention is one that has achieved significant intervention effects either through a highly controlled efficacy study or an effectiveness study implemented in more diverse real-world contexts.
Ideally, evidence of effectiveness should be provided from randomised controlled research trials; however, it is increasingly acknowledged that evidence of effectiveness, especially for public health interventions, can be derived from a broader range of research designs, including pragmatic trials, quasi-experimental studies, cluster randomised trials, the relatively new ‘stepped wedge design’, and multiple baseline designs.11,12 In addition, natural experimental studies may be the only option when it is impossible to manipulate exposure to the intervention.13 As with other quasi‐experimental designs, natural experiments cannot definitively determine causation and are subject to bias; however, they can provide a useful inferential tool that may not be obtainable in any other way.
It is also important to consider whether the size of the effect of the intervention is known and whether this may be of policy significance. Effects of interventions are likely to be smaller as they are scaled up; therefore, relatively large effect sizes should be demonstrated in the efficacy stage if an acceptable level of effect is to be maintained when interventions are scaled up.4 This reduction in effect is in part because of difficulties maintaining the dose and fidelity of the original intervention in real-world settings. It is rare for interventions to remain unchanged as they are scaled up, because of the need to adapt them to suit the local context and the organisational, financial and human resources available for scaling up.4,6,10 These adaptations may reduce effectiveness, but they can improve acceptability and efficiency, highlighting the importance of measuring intervention effectiveness throughout the scaling up process.
Reach and adoption are at the heart of scalability.4 Reach refers to the level of individual participation of an intended target population in an intervention.14 Adoption is the proportion of settings, practices or organisations that adopt an intervention.14 In either case, it is important that interventions reach as large a proportion as possible of those eligible to receive them once they are scaled up, and that interventions are adopted by as large a proportion of eligible settings as possible. Whether an intervention has differential rates of reach and adoption should also be determined.
It is important to acknowledge that all the information that would ideally be available to assess the scalability of an intervention may not be available at the time the assessment is completed. For example, it might not be possible to accurately determine how large the effect size at a population level needs to be to achieve a population health gain, or how much the program can be changed (to reduce cost or suit different contexts) while still retaining fidelity and outcomes. Where there are gaps in the available research evidence, decision makers may need to consider information from other sources such as expert advice, practice-based knowledge or parallel evidence from similar programmatic interventions. Where no information is available, a judgement is required about how important the missing information is, whether any gaps can be addressed during implementation, or whether further research is required before scaling up can be recommended.
A central part of any scaling up process is selecting the approach. Step 2 outlines the two main approaches: vertical and horizontal.5 Scaling up using a vertical approach involves the introduction of an intervention simultaneously across a whole system and results in institutionalisation of a change through policy, regulation, financing or health systems. Examples of successful vertically scaled up interventions include the introduction of mandatory seatbelt legislation15, smoking bans in outdoor eating areas16 and the introduction of new health system financing models.17 Scaling up using a horizontal approach involves the introduction of an intervention across different sites or groups in a phased manner, often beginning with a pilot program, followed by stepwise expansion, learning lessons along the way to help refine further expansion. Examples of successful horizontally scaled up interventions include the stepwise expansion of the effective falls prevention in older people intervention, Stepping On18, in Local Health Districts across NSW, and the expansion of lifestyle-based diabetes prevention programs in community settings in the US using a YMCA implementation model.19
It is also important that appropriate evaluation frameworks are built into intervention delivery from the outset4, particularly for steps 3 and 4. Formative evaluation before scale-up will be required to test the appropriateness and acceptability of the scaled up intervention with the target audience and other stakeholders. Subsequent evaluation and monitoring efforts during scale-up should focus on measuring effectiveness over time, rates of reach and adoption, acceptability, compatibility with existing interventions and costs. The emphasis placed on measuring each of these aspects during scale-up will depend on what is already known about the intervention (gaps in evidence identified in step 1) and the approach to scaling up the intervention. A scarcity of resources often means that key program outcomes, which require specific data collection efforts, are either not measured or are measured with data that may or may not be intended for that purpose.2 In light of this, questions of the validity of performance measures are important, as is an understanding of the limitations of using performance data to inform decision making.
Although the issues outlined in the guide are important, many issues surrounding the scale-up of interventions are still being debated, and methods will continue to develop. However, it is clear from the literature and our research that, despite the unpredictable outcomes of scaling up attempts, a good plan can guide the scaling up process in the right direction and make success more likely.2,3 Moreover, successful scale-up requires systematic use of evidence2, and it is particularly important that data from monitoring is linked to decision making throughout the scaling up process. This guide argues that plans for scaling up need to consider a broad range of factors and balance what is desirable with what is feasible.
The publication, Increasing the scale of population health interventions: a guide, can be downloaded from the NSW Ministry of Health’s website
© 2016 Milat et al. This article is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.