The quality of Australian Indigenous primary health care research focusing on social and emotional wellbeing: a systematic review October 2017, Volume 27, Issue 4

Introduction

Health research that focuses on Aboriginal and Torres Strait Islander (Indigenous) people is needed to ensure that the healthcare provided is evidence based and culturally appropriate. Concerns about this type of research arise from the perception that, despite the large amount of research completed, community-level benefit¹ and improvements in health outcomes are limited.² To guide Indigenous-focused research, a set of guidelines for ethical conduct in Indigenous health research (Values and ethics) was developed.³ This document guides researchers and ethics committees on the conduct of culturally appropriate, community-acceptable research.

Conducting research that is culturally appropriate and acceptable to the Indigenous community where it is being completed may require modification of traditional research designs and processes. For example, use of participatory action research designs⁴ or research processes that involve extensive community consultation⁵ may mean that traditional approaches need to be adapted. Increasingly, Indigenous research methods and designs are being used.⁶ When conducting research that is culturally appropriate, researchers must balance using culturally appropriate methods with the need for research that is of high scientific quality.

The term ‘social and emotional wellbeing’ (SEWB) is preferred by many Indigenous people to ‘mental health’, as it implies a holistic, strengths-based perspective of mental health.⁷ SEWB is an important aspect of health, and the SEWB of Australia’s Indigenous people is reported to be poor compared with the non-Indigenous population.⁸ Culturally appropriate, evidence based research strategies are needed to effectively improve the SEWB of Indigenous people.

As the health system’s ‘front line’, primary health care services often provide SEWB-related care, including screening, early intervention and management. The stigma associated with seeking help for SEWB-related issues⁹ and the perception by some people that hospitals are unwelcoming¹⁰ may hinder access to mainstream mental health and state-run services. Primary health care services offer a discreet and independent alternative, and these services are often where Indigenous-focused SEWB research is conducted. This research is commonly conducted by teams that include primary health care staff, community members and researchers external to the community. Primary health care research is a challenging and resource-intensive process¹¹, and Indigenous-focused primary health care research must also comply with Values and ethics³ and be acceptable to the community.

In this review, we aim to identify the study designs, processes, outcomes and quality indicators of Indigenous-focused SEWB primary health care research conducted by teams that include researchers who are located outside the community. A subsequent review will describe actions relating to Values and ethics³ and local protocols.

Methods

A protocol for this review has been published previously¹², and is in accordance with PRISMA and MOOSE guidelines. This study is registered with PROSPERO (CRD42015024994). Database searches were conducted in Medline, Embase, CINAHL, Informit databases and HealthInfoNet, an Indigenous-specific research and project website, using the following terms: ‘Indigenous’, ‘social and emotional wellbeing’, ‘mental health’ and ‘primary health care’. To capture studies conducted since the development of Values and ethics³, a date limit was applied from January 2003 to February 2015.

Published studies were included if they used qualitative, quantitative or mixed methods, focused on Indigenous SEWB, and were conducted in one or more primary health care services. Journal articles, reports and evaluations were included. We included studies involving research teams, including primary health care staff, community members and researchers located outside the community. SEWB describes a strengths-based holistic perspective of mental health that acknowledges the sociohistorical and personal influences on mental health.¹³ We included SEWB/mental health, depression disorders, anxiety disorders, dual diagnosis (SEWB and drug or alcohol use), and smoking and alcohol use.

We excluded studies that did not generate original data or where at least half the research occurred outside a primary health care service. Primary health care services included Aboriginal Medical Services (AMSs)¹³, Aboriginal community controlled health services, and health services that provide primary health care or had general practitioners as staff members. In this review, the term AMS includes Aboriginal community controlled health services and Indigenous health services. ‘Community’ refers to primary health care or AMS staff, patients, families or community members. For the purpose of this review, Indigenous refers to Australian Aboriginal and/or Torres Strait Islander people.

One reviewer screened titles and abstracts, and excluded obviously irrelevant studies and duplicates. Two reviewers examined full-text versions of the articles remaining after screening. Data were extracted using data extraction forms developed for this review. In accordance with our protocol¹², we assessed risk of bias using adapted versions of existing risk-of-bias assessment tools. Studies using mixed methods were assessed according to the dominant method used. To assess community acceptance, we considered common aspects described in key Indigenous research documents^3,14, and identified if the following criteria were reported: 1) community governance; 2) community representation in study development; 3) community representation in study conduct (data collection, data analysis); and 4) community representation in reporting. Refer to the supplementary tables (available from: www.researchgate.net/publication/317099307_FINAL_2017_05_25_Farnbach_Systematic_Review_Supp_Tables) for details of the community acceptance assessment. We considered studies meeting three or four of the criteria as acceptable.

Results

A total of 2288 articles and program reports were identified (Figure 1). We removed 402 duplicates and excluded 1491 studies based on their title or abstract. There were 395 studies that required full-text assessment. To ensure all relevant articles and program reports were identified, we attempted to contact 50 authors to request additional data. Of these, 36 replied and 24 provided new data. A total of 37 articles relating to 25 studies were included in the review.

Figure 1.     PRISMA diagram of the systematic review process (click to enlarge)

Multiple articles that reported findings relating to the same study (such as one evaluation^15,16, one project^17-20, one survey^21,22, one interview/focus group session^23-26, or one questionnaire^27,28) were considered as a single study, and all references were included. The included studies focused on:

• SEWB (nine studies)^19,20,25-34
• Alcohol misuse (five studies)^21,22,35-38
• Smoking cessation (four studies)^23,24,39-41
• Dual diagnosis – SEWB and drug or alcohol misuse (three studies)^17,18,42,43
• Depression^44,45, depression or anxiety⁴⁶, or a mental health worker program (four studies).^15,16

Nineteen studies were conducted in AMSs^{17,18,21,22,25-29,32-45}, four in services aimed at providing primary health care^{19,20,23,24,30,31} and one in a service where general practitioners were staff members.¹⁵ One study involved community organisations, but most of the research appeared to take place in primary health care services or AMSs.⁴⁶ See Supplementary Table 1 (available from: www.researchgate.net/publication/317099307_FINAL_2017_05_25_Farnbach_Systematic_Review_Supp_Tables) for a reference list and descriptions.

Study design

Qualitative methods were used in 18 studies^{15-20,23-26,29-32,34,35,37,39,41-44,46}, six of which used mixed methods^{17-20,30,31,37} and one of which was a quasi-experimental design.³⁴ Quantitative methods on their own were used in five studies.^{21,22,27,28,33,36,45} One case study was included.⁴⁰ Participatory action research principles were used in combination with yarning techniques⁴¹, a social–ecological perspective^23,24 or as part of a mixed-methods study.^19,20 Two-way learning approaches were described by three studies.^{15,16,19,20,23,24} Sociological action research principles were used once.⁴² One study involved a review of existing case management models, followed by a staff survey and training.³⁸

Three studies were part of the Australian Integrated Mental Health Initiative (AIMhi).^19,20,3032 AIMhi aimed to improve outcomes for Indigenous clients of remote mental health services. AIMhi 1 developed a mental health ‘brief intervention’ and conducted a randomised controlled trial to evaluate the intervention.^19,20 AIMhi 2 used mixed methods to examine service-level challenges³⁰, and AIMhi 3 developed and assessed an electronic mental health resource.³¹ AIMhi was followed by the AIMhi Priority Driven Research Partnership, which involved the community, AMS and external researchers.³²

Voices United for Harmony constituted three substudies^27,28,33,34 to develop and assess the effectiveness of a participatory singing program to improve SEWB and physical health. AMS staff coordinated the studies’ activities and participants were AMS patients. Three related studies used qualitative, mixed-methods and quantitative designs to examine staff practices³⁵ and experiences³⁷, and quantify the effect of staff training on alcohol screening and brief interventions in AMSs.³⁶

Primary health care staff and patients were the most common participants. In four studies, families^29,43 and carers^19,20 of primary health care patients were participants. Community elders, families and residents were involved in the establishment of the AIMhi Priority Driven Research Partnership.³² Voices United for Harmony involved community leaders during study design and implementation.^27,28,33,34

Study initiation process

Seven studies appeared to be initiated by researchers external to the primary health care service^{23,24,27,28,33-37}, seven arose from research partnerships^{15,16,19,20,30-32,38,46} and three appeared to be jointly initiated.^25,26,40,42 A community also invited a researcher from outside the community to evaluate a SEWB service.²⁹ It was unclear how the remaining seven studies were initiated.^{17,18,21,22,39,41,43-45}

Study outcomes

Primary outcomes were identified and met in 16 studies.^{19-29,31,33-37,39,42,43,45} Outcomes related to identifying participant perceptions and experiences^{23,24,26,34,35,39,42,43}; evaluating an intervention^{19,20,27,28,33,34,36}, a service²⁹ or training³⁷; and developing and assessing the acceptability of a resource.^25,31

Two primary outcomes were identified and met in three studies.^21-26 For example, one study assessed the acceptability of an alcohol-related intervention²¹ and identified cut-off scores of an alcohol dependence screening tool for Indigenous clients.²² Two of these articles reported on data that appear to have been collected at one time point.^25,26

We were unable to identify primary outcomes in eight studies. These included a case study⁴⁰; AIMhi 2³⁰; the AIMhi Priority Driven Research Partnership³²; a study to develop and assess a psychological assessment tool⁴⁴; and projects focused on depression⁴⁶, a case management model³⁸ and capacity development relating to dual diagnosis.^17,18 The primary outcome was somewhat met in one study, where a workplace policy was developed as planned, but acceptability testing was pending.⁴¹ The AIMhi 1 and Voices United for Harmony evaluations demonstrated improved outcomes for participants who received the intervention.^{19,20,27,28,33,34}

Risk of bias and community acceptance

We included peer-reviewed journal articles and articles from other publications describing processes, including an evaluation, description of a partnership and development of a model. Consultation, training and project reports were also included. Four studies did not align with the standard risk-of-bias assessment tools and therefore could not be assessed.^15-18,32,38 In 12 studies, the risk of bias was judged to be high^{19,20,27,28,30,33-35,37,46} or unclear^{21,22,36,40,45}, four were at moderate risk of bias^{19,20,25,26,41,42,44} and five were at a low risk of bias.^{23,24,29,31,39,43} AIMhi 1 was assessed using the qualitative¹⁹ and randomised controlled trial²⁰ risk-of-bias tools, because these methods were reported separately.

Using the qualitative risk-of-bias tool, most studies were deemed to have a moderate^{25,26,41,42,44} or high^{19,20,30,35,37,46} risk of bias. These ratings were because of missing or unclear reporting of many of the criteria used to assess bias. For example, actions related to ethical issues were not reported in more than half the studies. Processes related to informed consent, confidentiality or consideration of the impact of the authors’ relationship with the participants were described by authors of six studies.^{23,24,29,31,39,42,43}

Assessment of quantitative studies presented specific challenges. Quantitative studies included a survey^21,22, quality improvement cohort study³⁶, case study⁴⁰, validation study⁴⁵ and cohort analytic quasi-experimental design.^27,28,33,34 Many assessment criteria were not applicable for these designs. For example, assessing intervention integrity was not applicable when assessing the validation study⁴⁵ or the quality improvement cohort study.³⁶

The Voices United for Harmony quasi-experimental designs^27,28,33,34 were found to have high risk of bias because of participant self-selection (selection bias), confounding, lack of blinding and the high number of dropouts in two of the studies.^33,34 In these studies and the case study, participant selection methods were developed in response to community feedback and were based on participants’ ability or willingness to take part in the program or intervention, rather than using sampling, randomised or consecutive methods.^{27,28,33,34,40} Two of the Voices United for Harmony initiatives used valid and reliable data collection tools^27,28,33, leading to a low risk of bias for this criteria.

The AIMhi 1 randomised controlled trial was judged to have a moderate risk of bias.²⁰ Documentation of allocation concealment and blinding of participants and personnel during outcome assessment were unclear.

Communities’ perspectives were rarely reported. However, nine studies were assessed as having high community acceptance (three to four criteria met)^{21-24,27,28,32,39,41,44-46}, and 12 as low acceptance (two or fewer criteria met)^{15-20,25,26,29,33-37,42,43} according to our four criteria. Four studies were part of large, ongoing research partnerships, which may have involved extensive community engagement, but this was not described.^{19,20,25,26,30,31} We were unable to determine community acceptance in four studies.^30,31,38,40 This was because of the lack of reporting between linked articles^30,31 or because primary health care service staff were co-authors and the extent of their involvement during research development, conduct and reporting was not described.^38,40 Two studies were judged as having high community acceptance and low risk of bias.^23,24,39

Discussion

We identified only two studies that were judged to be scientifically robust and acceptable to the community.^23,24,39 Other studies with high community acceptance were deemed to have a moderate^41,44, unclear or high risk of bias^{21,22,27,28,45,46}, or were unable to be assessed³² using standard assessment tools. This results in uncertainty about the strength and generalisability of their findings. Where community perspectives were unclear, it was difficult to determine if this was because of underreporting (possibly related to publication word limits) or if it reflected community dissatisfaction. Although not explicitly reported, involving community members in key positions or extensive community consultation may suggest acceptance and have led to culturally appropriate designs.

A variety of designs and processes were used in the included studies. These depended on the study aim, the collaboration and the community involved. This variation, together with the diversity among Indigenous communities, makes drawing general conclusions about designs challenging.

Qualitative studies appeared to have greater community acceptance and lower risk of bias than quantitative and mixed-methods studies. However, qualitative research is considered Level IV evidence in the scientific community, meaning there is a lack of certainty when drawing conclusions from its findings. In addition, the primary outcomes identified in most of the qualitative studies involved identifying perceptions or experiences, suggesting limited impact on primary health care delivery.

Concerns about randomised controlled trials that involve Indigenous communities include the perception that randomisation is unethical.⁴⁷ However, randomised controlled trials are considered Level I evidence in the scientific community. Two studies in this review involved randomised controlled trials, and both used flexible randomisation processes. In one^19,20, participants were randomised into ‘early’ and ‘late’ intervention groups, meaning that all participants received the intervention at different time points. Although the authors did not provide justification for this approach, the study was part of a large, ongoing initiative, suggesting collaboration with the community. In the other study, the design was modified to a nonrandomised, quasi-experimental design in response to community feedback.^27,28 Both studies were assessed as having a high risk of bias, demonstrating the challenges of implementing study designs in Indigenous communities that are considered high quality in the academic community.

These challenges surrounding randomised controlled trials have been reported previously, including by the authors of one study⁴⁸, who described modifying the design to address challenges and encourage recruitment. The researchers ceased this study, citing clinic, patient, staff and study design–related factors that made the project untenable. Evidence based research methods have developed within a Western cultural perspective, which does not incorporate Indigenous social, cultural or historical perspectives. These examples demonstrate how evidence based research methods may not be appropriate for Indigenous communities, because of these differing perspectives.

Culturally sensitive approaches, including two-way learning^{15,16,19,20,23,24}, participatory^{19,20,23,24,41}, social–ecological^23,24 and phenomenological approaches³⁹ were used in five studies. In one, participatory action research was used to localise an intervention and study design.^19,20 These approaches appeared to improve community acceptance by incorporating local perspectives. We propose that research projects incorporate these culturally sensitive approaches, as identified in this review.

There is increasing focus on methods that incorporate Indigenous perspectives and Indigenous ways of knowing, being and doing.⁴⁹ Regardless of the topic under investigation, research incorporating Indigenous perspectives will lead to primary health care that is better aligned with the needs of Indigenous people. However, there appear to be few examples of their implementation in practice.⁵⁰

In this review, we identified outcomes related to evaluating interventions, services or training in seven studies^{19,20,27-29,33,34,36,37}or assessing resources in two studies.^25,31 These outcomes indicate potential impact at the community level. Research should improve health and result in community-level benefit.² We propose that research is reported according to the following outcomes:

1. Process outcomes that describe steps taken during planning and implementation

2. Academic outcomes that describe dissemination and academic achievements

3. Clinical outcomes that describe efficacy, impact, cost-effectiveness and research translation

4. Community outcomes that describe ongoing implementation and efficacy, cost savings, access changes, community engagement and other outcomes that the community determines to be relevant.

Reporting these outcomes will provide a balanced description of how to achieve high-quality, community-endorsed research that is likely to affect clinical practice and health outcomes. We suggest considering these outcomes, together with the community-acceptance principles highlighted in this review, when assessing the quality of Indigenous-focused research.

There are several limitations to this review. The breadth of formats included (evaluations, reports and journal articles) did not fit easily with standard risk-of-bias assessment tools, and we modified these tools to make assessment feasible. We were restricted to the information reported in articles, which may exclude some information. Although we have identified criteria to indicate community involvement and acceptance, we recognise this may not comprehensively capture all aspects of culturally appropriate research. In addition, we recognise that the diversity of Indigenous communities throughout Australia means that a process that is suitable in one community may not be suitable in another.

Conclusion

There are few examples of Indigenous-focused SEWB primary health care research that are of high scientific quality and acceptable to the community. This provides many opportunities for improvements for research in all domains. Use of participatory action research, social–ecological approaches and incorporation of two-way learning principles appears to facilitate research that incorporates Indigenous perspectives. We recommend that consideration of community-level outcomes and the community-acceptance principles highlighted in this review are kept at the forefront throughout research. This will improve culturally appropriate research that positively impacts the SEWB of Indigenous people.

Acknowledgements

We acknowledge staff at the University of Sydney Medical Library for their help developing the search strategy. AE was in receipt of a National Health and Medical Research Scholarship (APP1056434). JF was in receipt of the Rowan Nicks Russell Drysdale Fellowship and MH was in receipt of a National Heart Foundation Future Leader Fellowship (100034). These funding bodies had no role in the conduct or reporting of this review.

Copyright:

© 2017 Farnbach et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.