Using Facebook to recruit for a public health campaign evaluation July 2019, Volume 29, Issue 2

Introduction

Facebook is the largest social media platform, so it is unsurprising that public health groups are using Facebook to deliver campaigns.¹ However, assessing the contribution of Facebook to campaign impact requires different approaches to traditional campaign evaluations.² This includes trialling novel approaches to participant recruitment.

Campaign evaluations typically involve recruitment via telephone or research panels, methods that require significant investment. Both methods require a large sample to capture enough people that use Facebook and have been exposed to the campaign on Facebook, which increases recruitment costs. Recruiting participants through Facebook, on the other hand, presents a potentially effective way to obtain reasonable sample sizes on a modest budget.³

In this paper, we report on our use of Facebook as a recruitment tool for the evaluation of a public health campaign by the New South Wales (NSW) Government in Australia, the Make healthy normal (MHN) Facebook page.

Method

In this study, approved by the University of Sydney’s Human Research Ethics Committee (protocol number: 2017/145), we recruited NSW adult Facebook users to complete an online survey about why users have ‘liked’ the MHN Facebook page, how often they engage with MHN and other similar Facebook pages, and their opinions about the page. Participants could be fans of the MHN Facebook page (i.e. had ‘liked’ the page) or non-fans.

Potential participants were invited via two Facebook advertisements (Figure 1). Advertising started on 29 May 2017. Participants were offered a chance to win a gift voucher as compensation for the time taken to complete the survey. We applied Facebook’s advertising filters to target MHN fans, as well as non-fans who had an interest in health (e.g. had ‘liked’ I quit sugar, LiveLighter, etc.). Filters were also applied for age (more than 18 years old), location (NSW) and household income (A$40 000–$60 000), reflecting the campaign’s target audience. These filters provided a total audience of approximately 65 000 people. The total budget was A$6000, including advertising spend, marketing fees and prizes.

Figure 1.     Facebook advertisements for recruiting survey participants (click to enlarge)

Results

The initial response to the survey was slow, with only 159 surveys completed after 2 weeks of advertising. We then expanded the audience size by removing targets based on user interests and relaxing the filter for household income (making it $40 000–$80 000) and then adding a ‘lookalike audience’ (users similar in profile to fans of the MHN page). This increased the total audience size to 480 000. Finally, two additional lookalike audiences were created to increase the total potential audience to approximately 2 million people.

The advertising was ‘in field’ (i.e. the period in which the advertising was displayed to Facebook audiences) for 8 weeks, reaching 88 030 unique Facebook users in NSW. We achieved 591 completed surveys from a spend of $5060, giving a cost per completed survey (CPS) of $8.56 (Table 1). Women were more likely than men to click on the advertising. Of the two advertisements, Figure 1b achieved a slightly higher click-through rate (i.e. the ratio of users who click on the advertisement to the total number of users who view it).

Table 1.     Advertising metrics and survey results

^a The number of times the advertising was displayed to a Facebook user. This figure does not represent unique users e.g. if one user was served the advertisement five times, this would constitute one unique user reached and five impressions.

Detailed characteristics of the participants are available elsewhere.⁴ In brief, nearly 90% of the sample were women and almost two-thirds (63%) were aged 18 to 34 years, partly reflecting the fanbase of MHN (83% women and 52% aged 18 to 34 years). Additionally, more participants from high socio-economic areas completed the survey than would be expected in the general population: 75% were from the least disadvantaged areas (Socio-Economic Index for Areas quintiles 1 to 3). However, we do not know whether this reflects the characteristics of the MHN fanbase or Facebook users more generally because this information is not available.

Discussion

Our experience shows that Facebook may have potential as a recruitment method for campaign evaluations, particularly as the CPS was low. However, significant concerns remain about the representativeness of the sample.

Comparing survey costs is difficult because there are many variables involved.⁵ Nonetheless, at $8.56, our CPS compares favourably with random digit dialling, which can cost more than $100 per completed survey. In addition, the cost is similar to an online panel but has the advantage of not requiring involvement of a third party in the form of a panel provider. Further, improving the appeal of the advertisements we used (e.g. experimenting with images and text) would be expected to attract a higher click-through rate and a lower CPS.

Limitations are that the characteristics of the participants were heavily skewed towards women and young people, even more so than MHN’s fanbase, and there was an overrepresentation of participants from high socio-economic areas. Although we cannot know what the sample would have looked like had we used other recruitment methods, our results do little to remove concerns about whether social media can recruit a comparable sample to that of other methods.³ Improving the representativeness of participants could be achieved by, for example, running gender-specific messages to attract more men.

Within the context of the evaluation of a campaign like MHN, Facebook advertising has potential as a relatively inexpensive option for recruitment. However, further research is required to determine whether a comparable sample can be obtained. Experimenting with advertising design and implementation may help future campaigns to reduce the CPS and time in field, and improve representativeness.

Acknowledgements

We would like to thank the MHN campaign team at the NSW Ministry of Health and Office of Preventive Health for supporting this work. The NSW Ministry of Health provided the funds to run the advertising and for the prizes.

Peer review and provenance

Externally peer reviewed, not commissioned.

Copyright:

© 2019 Kite et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.