Objective: Multilingual patient education materials (PEMs) in Australia are normally prepared initially in English (source text) and then translated into other languages. The aim of this study was to evaluate whether the source texts for publicly available multilingual diabetes PEMs in Australia were written at the reading level recommended by health literacy guidelines (eighth-grade reading level).
Study type: Nonexperimental descriptive study.
Methods: All publicly accessible multilingual fact sheets on diabetes self-management from the Diabetes Australia and National Diabetes Services Scheme websites were collected. Readability was analysed using five different readability indices: Flesch Kincaid Grade Level (FKGL), Gunning Fog Score (GFS), Coleman Liau Index (CLI), Simplified Measure of Gobbledygook Index (SMOG) and Automated Readability Index (ARI). The average number of syllables per word and the average number of words per sentence were also calculated.
Results: The average reading grade level of included PEMs was above Grade 10 (mean 10.4; standard deviation [SD] 0.9). The average number of syllables per word was 1.5 (SD 0.1), and the average number of words per sentence was 17 (SD 0.9).
Conclusions: English-language source texts for national multilingual diabetes PEMs examined in this study were written at a readability level significantly higher than that recommended in health literacy guidelines. This was likely due to the use of polysyllabic words and complex medical terms, which are especially problematic when they are not defined. Improving readability of English-language source texts may help to ensure that the translated PEMs are more readable and accessible to their target readers. In conjunction with addressing other features that can make written materials easier to understand, this may help to better support diabetes self-management.
In Australia, an estimated 1.2 million people had diabetes in 2014–151, and 2 million more people were at high risk of developing diabetes.2 Diabetes is particularly prevalent in culturally and linguistically diverse (CALD) communities in Australia; the prevalence of diabetes in people born in eastern and southern Europe (14.5%), and North Africa and the Middle East (22.4%) is more than triple the national average.3 As a result, the Australian National Diabetes Strategy 2016–2020 recognises CALD groups as a priority population in strategic planning and implementation, and recommends tailored actions to meet the specific needs of CALD communities.4
The complex nature of diabetes makes self-management challenging. Its success is influenced by a number of factors, including health literacy. Education is an integral component of diabetes self-management.5 Given that education is often delivered using a written format, the reader’s health literacy skills (i.e. their ability to understand and act on the information) will be crucial to their self-management efforts. The only nationally representative survey to directly measure health literacy in Australia to date suggests that 60% of Australians have low health literacy, a disproportionate number of whom are from CALD backgrounds.6 Addressing health literacy has been identified as a strategic priority both nationally and internationally.7 In Australia, all diabetes health professionals are encouraged to address health literacy according to the joint position statement from the Australian Diabetes Educators Association and the Australian Government (via the National Diabetes Services Scheme [NDSS]).8
Health literacy can be addressed in several ways. A risk approach to health literacy prioritises removing literacy-related barriers in the environment and making health information easier to understand.9 Written patient education materials (PEMs) are a common means of communicating diabetes self-management information. From a risk perspective, ideal diabetes PEMs would employ health literacy principles and appropriately meet the needs of the target audience to support them to make more effective decisions and take appropriate health actions. One common approach is to write PEMs at an eighth-grade (or below) reading level.10 In addition, it has been recommended that sentences should be no more than 15 words in length.11
Use of multilingual PEMs can improve the understanding of diabetes self-management practices among CALD groups. However, the utility of multilingual PEMs may be limited if the readability level is too high, because individuals may have low levels of literacy in their first language. Multilingual PEMs in Australia are normally prepared initially in English (‘source text’) and then translated into other languages “without distortion or omission and preserving the content and intent of the source message or text”.12 Assessing the readability of source texts can provide a preliminary indication of the readability of translated language versions. To the authors’ knowledge, no research has looked into the readability of source texts that have been directly translated into other languages.
The purpose of this study was to explore the readability of the source texts for publicly available multilingual diabetes PEMs in Australia. Our primary aim was to evaluate whether these publicly available diabetes PEM source texts were written at the reading level recommended by health literacy guidelines.10,11
This study used a non-experimental and descriptive design that involved calculating computerised readability scores of the source texts of selected PEMs. No ethical approval was required.
Diabetes Australia is the key national diabetes authority in Australia. The NDSS is an Australian Government initiative that is administered with the assistance of Diabetes Australia to enhance the capacity of people with diabetes to understand and self-manage their life with diabetes. Diabetes Australia and the NDSS have jointly published PEMs on different topics. Both organisations are perceived as sources of authoritative information in this field, which is widely used in diabetes education.13
PEMs were included only if they were published by the NDSS and Diabetes Australia as a printable fact sheet (in PDF), publicly accessible from their official websites at no cost, and available in a language other than English. Information was excluded if it was only available in English; only available in video, pictorial or graphic form; or not directly related to diabetes self-management. The search was conducted in February–March 2017. The PEMs evaluated in this study were the versions published in 2016. Diabetes Australia and the NDSS updated the PEMs in October 2018, but the new materials were not included in this study.
PEMs were categorised into four main content areas: general, nutritional, physical activity and medical information. The classification was completed independently by four senior bilingual diabetes educators. Disagreements were resolved through discussion.
All documents were then prepared in Microsoft Word by removing any text that was not in full sentences (including titles, headings, subheadings, short captions and bullet lists), embedded punctuation and document design elements (such as gaps, white spaces, pictures and images, and text boxes), as per published protocols for assessment by readability tools.14
The prepared text was assessed for readability using an automated online readability calculator. This is a cost-effective, time-effective and objective method to evaluate the literacy demand of written information materials.15 The calculator includes five commonly used readability indices: Flesch Kincaid Grade Level (FKGL), Gunning Fog Score (GFS), Coleman Liau Index (CLI), Simplified Measure of Gobbledygook Index (SMOG) and Automated Readability Index (ARI). These readability indices are used to assess health information, and have been recommended by health authorities and organisations.16 The readability calculator reports US grade reading levels, which can be used to estimate equivalent reading levels within the Australian educational system; according to the International Standard Classification of Education and the Australian Standard Classification of Education, US grade levels are estimated to be equivalent to Australian years of education in most states.17 The average score was calculated to provide a fairer prediction of the overall readability, because each measure predicts reading difficulty according to different linguistic variables; for example, the FKGL accounts for sentence length (number of words) and word length (number of syllables), whereas the SMOG calculates readability based on the number of polysyllabic words.18 See Table 1.
We also calculated the average number of words per sentence and the average number of syllables per word for each included PEM, as well as the proportion of complex terms (more than two syllables)19 and complex medical terms (medical terms with more than two syllables). A medical term was defined as a healthcare word or phrase that describes anatomy, diseases, procedures, medicines and experts in the field of health.20 Final terms were checked against the medical terminology criteria in Taber’s Medical Dictionary Online.21 The terms matched in the search were considered medical terms. Prefaced medical expert terms (such as the provider registration status) were counted once, based only on the medical expert term (e.g. ‘registered podiatrist’ was counted as ‘podiatrist’). Medical terms that would be familiar to the target audience (e.g. Medicare, diabetes, obesity) and their medical abbreviations (including NDSS) were not counted. Complex medical terms were identified independently by three authors. Disagreements were resolved through discussion.
Data were entered and analysed using SPSS for Windows (Armonk, NY: IBM Corp, version 24.0).
Table 1. Descriptions of readability measures
|Flesch Kincaid Grade Level||Word length (syllables)
|0.39 × (words/sentences) + 11.8 × (syllables/words) – 15.59|
|Gunning Fog Score||Sentence length
Complex words (3 syllables or more)
|0.4 × ((words/sentences) + 100 × (complex words/words))|
|Simplified Measure of Gobbledygook Indexb||Sentence length
|1.0430 ×√(30 x complex words/sentences) + 3.1291|
|Coleman Liau Index||Word length (characters)
|5.89 × (characters/words) – 0.3 × (sentences/words) – 15.8|
|Automated Readability Index||Word length (characters)
|4.71 × (characters/words) + 0.5 × (words/sentences) – 21.43|
Eight PEMs that met the inclusion criteria for this study were identified on the NDSS and Diabetes Australia websites. They covered a broad range of content relating to diabetes self-management (Table 2). All PEMs were available in 10 languages or more.
The average reading grade level of included PEMs was above Grade 10 (mean 10.4; standard deviation [SD] 0.9) (Table 2). On average, there were 1.5 syllables per word (mean 1.5; SD 0.1) and 17 words per sentence (mean 17.1; SD 0.9). The percentage of complex words ranged from 10.8% to 17.8% of the total number of words. The GFS and the CLI yielded the highest readability scores of all readability formulas, with readability scores above Grade 10 for each PEM. The average readability score was 10 for two-page PEMs, 11 for three-page PEMs, 10 for four-page PEMs and 11 for six-page PEMs (Table 3).
Table 2. Description of patient education materials and readability statistics
|Title||Content category||Availability in other languages||Pagesa||FKGLb||GFSb||SMOGb||CLIb||ARIb||Average readability scoreb||% of words that are complex||Average words per sentence||Average syllables per word|
|Understanding type 1 diabetes||General information||Arabic, simplified Chinese, traditional Chinese, Greek, Italian, Korean, Spanish, Turkish, Urdu, Vietnamese||2||8.7||12||8.8||11.5||8.4||10||14.2||16.0||1.5|
|Understanding type 2 diabetes||General information||Arabic, simplified Chinese, traditional Chinese, Greek, Italian, Korean, Spanish, Turkish, Urdu, Vietnamese||2||8.9||12||8.8||11.6||8.9||10||13.4||16.8||1.5|
|Understanding gestational diabetes||General information||Arabic, Bengali, simplified Chinese, traditional Chinese, Greek, Hindi, Italian, Korean, Samoan, Spanish, Turkish, Urdu, Vietnamese||3||10||13.2||9.7||13.3||10.5||11||16.1||17.2||1.6|
|Food choices for people with diabetes||Nutrition information||Arabic, simplified Chinese, traditional Chinese, Greek, Italian, Korean, Spanish, Turkish, Urdu, Vietnamese,||4||9.4||12.4||9.1||11.9||9.8||11||13.4||18.1||1.5|
|Hypoglycaemia||Medical information or instruction||Arabic, simplified Chinese, traditional Chinese, Greek, Italian, Korean, Spanish, Turkish, Urdu, Vietnamese||4||9.1||11.2||8.1||11.1||9||10||10.8||17.9||1.5|
|Diabetes complications||Medical information or instruction||4||7.7||10.4||7.5||12||8.6||9||10.6||15.4||1.5|
|Pregnancy and diabetes||Medical information or instruction||6||9.6||12.8||9.3||13.3||10.6||11||14.6||17.5||1.6|
|Physical activity||Physical activity information||6||10.4||14||10.2||12.7||10.1||11||17.8||17.5||1.6|
Table 3. Average readability across patient education materials based on number of pages
|Number of pages||Number of PEMS||FKGL||GFS||SMOG||CLI||ARI||Average|
In total, there were 95 unique complex medical terms across the eight PEMs, most commonly used for diseases, experts in the health field and medicines. Among those, there were 62 (65.3%) instances in which the term was not explained or defined after its first appearance (Table 4). The most frequently undefined medical terms were the medical expert terms (3–5 syllables), which occurred in all PEMs. The most frequently explained terms were ‘endocrinologist’ and ‘hypoglycaemia’, appearing three times across eight PEMs.
Table 4. Number of complex medical terms in patient education materials and average readability
|PEM title||Terms with definition||Terms without definition||Average readability||Number of unique complex medical terms||Proportion of complex medical terms without definition (%)|
|Total||With definition||Without definition|
|Understanding type 1 diabetes||endocrinologist, insulin, ketoacidosis||antibodies, diabetes educator, dietitian, exercise physiologist, nurse practitioner, pancreas, podiatrist||10||10||3||7||70.0|
|Understanding type 2 diabetes||endocrinologist, haemoglobinA1c, insulin||antipsychotic, diabetes educator, dietitian, exercise physiologist, injectable medication, nurse practitioner, pancreas, pathology test, podiatrist, polycystic ovary syndrome, psychologist||10||14||3||11||78.6|
|Hypoglycaemia||Acarbose (Glucobay®) , glucagon, hypoglycaemia||diabetes educator, glucose jellybean, insulin, Lucozade||10||7||3||4||57.1|
|Diabetes complications||atherosclerosis, autonomic, cardiovascular, cataract, gastroparesis, genital, gingivitis, glaucoma, macrovascular, microvascular, neuropathy, ophthalmologist, peripheral arterial disease, peripheral neuropathy, retinopathy||amputation, cholesterol, diabetes educator, dietitian, optometrist, triglycerides||9||21||15||6||28.6|
|Pregnancy and diabetes||complication screening||contraception, dietitian, haemoglobin A1c, hypoglycaemia, insulin, intrauterine devices, miscarriage, nutrient supplement, pre-eclampsia, reversible methods||11||11||1||10||90.9|
|Physical activity||hypoglycaemia, peripheral vascular disease||cholesterol, diabetes educator, exercise physiologist, Medicare, muscular pain, osteoporosis, podiatrist||11||9||2||7||77.8|
|Understanding gestational diabetes||endocrinologist, gestational diabetes, hypoglycaemia||antipsychotic, Caesarean, diabetes educator, dietitian, exercise physiologist, insulin, metformin, nurse practitioner, obesity, pancreas, physiotherapist, placenta, polycystic ovary syndrome||11||16||4||12||75.0|
|Food choices for people with diabetes||HDL-cholesterol, LDL-cholesterol||cholesterol, dietitian, glycaemic index, insulin, triglycerides||115||7||2||5||71.4|
There is an ethical imperative to consider the readability of national multilingual diabetes PEMs, given that CALD groups have a higher prevalence of diabetes and poorer clinical outcomes than their Caucasian counterparts.22 The current study highlights that the national diabetes PEMs that serve as source texts for translation into multiple languages were written at a readability level higher than recommended, requiring above Year 10 Australian education to read. PEMs had an average sentence length greater than recommendations for readers with lower literacy.11 They also contained a large number of complex medical terms, a high proportion (65.3%) of which were without in-text explanation or definition. Our findings suggest that the more pages a PEM contained, the higher its readability score (i.e. the lower its readability), although this was not tested for statistical significance.
Although this was the first study to assess the readability of source texts (in English) for national multilingual diabetes PEMs in Australia, our findings are consistent with a large and growing body of literature that suggests that written information materials are written at a readability level that is too high for most patients.20 An examination of culturally adapted English materials for African Americans, for example, showed that the mean readability of anticoagulation PEMs was Grade 11, when the reading skills of the target audience were between Grade 7 and Grade 8.23 Together, these findings highlight the discrepancy between the skills and abilities of patients and the level of materials that are designed to support their care.
Poor readability of English-language source texts may also have implications for translated materials. Australian translators are guided by a code of ethics and conduct that emphasises preservation of the content and intent of a text.12 Australian translators do not require accreditation to specialise in medical translation, and this may potentially lead to poor or inconsistent use of a medical corpus. Consequently, a good source text for any health education material lays the foundation for a good translated PEM, including from a health literacy perspective. Using a direct translation technique, it is possible that the translated PEMs (especially for the languages that have a similar lexical–semantic system to English) might have similarly poor readability.
This may be further exacerbated by additional translation issues. For example, other research examining the simplified Chinese version of the Diabetes Australia ‘Physical activity’ PEM found a number of inappropriate medical translation errors (e.g. ‘glucose gel’ mistranslated as ‘glucose jelly’).24 Given that there is currently no universal medical corpus for translation, specific professional guidance for translating complex medical terms using health literacy principles is an important step for future research and practice.
To date, our study is the first to evaluate the readability of national diabetes PEMs that have been translated for CALD populations. We employed five readability measures, and averaged scores across all measures as per recommended guidelines. We also examined factors (e.g. average number of words per sentence, average number of syllables per word, proportion of complex terms) that most strongly contribute to poor readability scores. Although only eight PEMs were analysed, the included PEMs were produced by the most reputable diabetes professional organisation in Australia, in conjunction with the Australian Government. As a result, they can be considered more influential than any other diabetes PEMs in this context.
This study only considered readability. A broad array of features beyond readability can make materials easier to understand, such as pictures, headers and visual cues. Although our findings may provide a proxy readability level for national multilingual diabetes PEMs in Australia, we did not assess the linguistic fluency (e.g. syntactic, phonological and lexical fluency) in the languages other than English and did not compare different language versions to examine differences in readability.
Additional work is needed to examine visual design, linguistic and structural features that promote or inhibit understanding in the source texts included in this study. Use of validated readability formulas for different languages would be useful to provide a deeper and richer view of the accessibility of multilingual PEMs. In addition, linguistic analyses would be useful to offer more sophisticated insights beyond the sentence level, and to provide more social–cultural understandings. It would also be valuable to evaluate the PEMS using cognitive interviews with consumers, to provider a deeper understanding of how meaning is extracted from the text.
In practice, producers of health materials must begin to consider the number of polysyllabic words, the use of complex medical terms and sentence length when developing PEMs, to reduce readability demands. Where complex medical terms are essential, they should be clearly defined (such as in a glossary of terms), or given simple explanations or descriptions when first introduced. It could also be worthwhile to systematically develop an Australian guide for medical translation (including a corpus) to assist decision making in translation of health materials. Lastly, value could be added by involving CALD consumers in the creation of content.
The English-language source texts for national multilingual diabetes PEMs examined in this study were of lower readability than recommended in health literacy guidelines. This was likely to be the result of use of polysyllabic words and complex medical terms, which are especially problematic when undefined. Improving readability of English-language source texts may help to ensure that the translated PEMs are more readable and accessible to their target readers. In conjunction with addressing other features that can make written materials easier to understand, this could better support diabetes self-management.
SL received a Research Training Program Stipend Scholarship from the Australian Government.The authors acknowledge the valuable contributions of Vania Khoury, Jennifer Zhen and Wenhui Zhang in categorising the diabetes PEMs, and the guidance Meng Ji has provided SL throughout the PhD research.
Externally peer reviewed, not commissioned.
© 2020 Lin et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.