A call to action for undertaking and sharing formative evaluations of public health campaigns June 2021, Volume 31, Issue 2

Background

In early 2019, then Australian Federal Health Minister the Hon Greg Hunt MP announced the ‘Heart Health Check’: two new Medicare-funded items for use by general practitioners (GPs) to assess and manage cardiovascular disease risk in their patients.¹ This was a major policy win for Australia’s peak heart disease advocacy body, the Heart Foundation, and GPs, as they had advocated for better investment in early detection and prevention of cardiovascular diseases within primary care settings. The Heart Foundation should be commended for this successful policy change which further embeds preventive medicine in primary care settings and acknowledges the time-intensive nature of chronic disease prevention and management.

In May 2019, following the policy announcement, the Heart Foundation released a new campaign entitled ‘Heartless Words’ which aimed to: 1) raise awareness about the impact of heart disease for Australians; and 2) promote the Heart Health Check. The main campaign advertisement featured a series of people with heart disease telling their family and friends that they did not love or care for them, with the intention of highlighting that heart disease affects not just the person who is ill, but those around them.²

The campaign advertisement provoked instant controversy, with comments predominantly aired on social media by advertising experts, public health professionals and commentators, who criticised the advertisement for implying that individuals who had heart disease did not care about their families.³ Other criticisms of the campaign advertisement included that it was stigmatising individuals with heart disease, and that it failed to take into account broader social determinants and inequities in healthcare and prevention in Australia, in particular those faced by specific groups such as Aboriginal and Torres Strait Islander peoples.⁴ After initially defending and amending the advertisement, the Heart Foundation pulled it from air after 5 days, and published advertisements in print newspapers and online websites to apologise for it.⁵

While some commentators argued the advertisement’s appeal to negative emotions such as guilt and shame stigmatised individuals with heart disease^3,4, we note that public health researchers have questioned the ethics of negative emotional appeals used in other public health campaigns, including in tobacco control and HIV/AIDS.^6,7 This is not a new debate, nor is it limited to Australian public health campaigns – Cancer UK was also called on to defend its 2019 campaign highlighting the link between obesity and cancer, amid concerns it could inadvertently contribute to weight stigma and discrimination.⁸ Without rehashing this debate, we note the available evidence indicates that negative, emotive appeals can have a place in public health campaigns.^9,10 However, only focusing on and critiquing the messaging of Heartless Words and other campaigns runs the risk of ignoring a broader and more fundamental issue for public health campaigns. Namely, the way we develop and test our campaigns – what is termed a ‘formative evaluation’ – and whether we are sharing the outcomes of such evaluations as part of good public health practice.

The role of formative evaluations

Formative evaluations are designed to ensure optimal development and testing of intervention components pre-implementation.¹¹ For interventions such as mass media campaigns, formative evaluations may use both qualitative and quantitative methods to review the existing literature, develop messages, pre-test materials with the target population, and ensure communication channels are appropriate. Evidence-informed guidelines for best practice in mass media campaigns in public health include formative research as a critical step to ensure the relevance of the public health message for the target population and to ensure it will reach and likely influence the intended audience.^12,13 Formative evaluations should allow for the forensic examination of message development and testing, such that any problems with messaging can be addressed prior to campaign launch. This is particularly important for campaigns that are using potentially controversial or negative appeals as a specific communications technique or strategy, given that such approaches could have unintended consequences, including stigmatisation and victim blaming (or perceptions thereof). Formative evaluations also contribute to our understanding of why campaigns might be successful (or not) and, crucially, whether our pre-campaign exploratory research or problem analysis is sufficient. It helps us to answer: Is our intervention appropriate, and for whom? Will our intervention help us to address the problem we have identified – and how?

Despite the importance of this stage of evaluation, we note that formative evaluations of mass media campaigns outside of tobacco control are rarely included in impact evaluations, or published in the peer-reviewed literature.^14,15 We are unaware of any publicly available protocol or data from the Heart Foundation that could yield insights into the formative evaluation that informed the development of the Heartless Words campaign, despite a thorough and systematic search. This includes any exploratory research, pre-testing of materials, or appropriate problem analysis, and how these informed the final material development and release. Sharing any such information on the formative evaluation of the campaign would help in understanding whether the issues associated with the advertisement were related to its design or execution (or both). Can messages like those in the Heartless Words advertisement ever be effective at increasing knowledge and changing the identified behaviour? We cannot know without undertaking and sharing rigorous evaluations, including formative, process and outcome (impact) evaluations.

We believe undertaking comprehensive formative evaluation is critical for good public health campaigns. This view is shared by many working in public health and health promotion^15-17 and by those more broadly in social marketing and mass media communications.¹⁸ Equally, we believe that sharing this information publicly is valuable for researchers, practitioners and policy makers, to help us better understand the process of successful campaign development prior to implementation. We acknowledge this may be difficult given many public health agencies now commission external advertising and creative agencies to develop campaigns.¹⁹ Agencies may be wary of sharing their work and formative research for reasons relating to the protection of intellectual property and commercial interests. Yet the opportunity to learn the best ways to position important public health messages is sacrificed when the processes that give rise to a campaign are not publicly available for research purposes.

Call to action: informing future public health campaigns

The key lesson to take from the Heartless Words campaign is not to dismiss negative or emotively framed public health campaigns that provoke discussion. Rather, we should ensure any new mass media campaign is supported by a robust formative evaluation framework that includes exploratory research and pre-testing, with the results shared among fellow public health researchers, policy makers, practitioners and students. Sharing the results would mean that future campaigns are informed by the lessons (good and bad) from past campaigns, including those like Heartless Words. It should not be up to academics or individuals to privately request this information from the campaign designers and implementers; rather, we need to develop a culture where such knowledge is already in the public domain.

This is a call to action to undertake formative evaluation activities for mass media campaigns. These activities should go beyond message testing and encompass a thorough problem analysis as well as providing the foundation for other stages of evaluation (process, outcome and impact).

We also call for better sharing of findings from formative evaluations in both peer-reviewed literature and public reports to improve the way in which public health campaigns are developed and received, noting that many in the health workforce and broader community may not have access to academic literature and reports, outside of open-access publications. We urge all agencies developing public health campaigns to share the results from any formative evaluation for the purpose of improving empirical knowledge and research methods in the area of public health and mass media campaigns.

Peer review and provenance

Externally peer reviewed, not commissioned. BF is an Associate Editor with Public Health Research & Practice. She had no part in the peer-review process for this paper.

Copyright:

© 2021 Howse et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.