Ascertainment of self-reported prescription medication use compared with pharmaceutical claims data October 2017, Volume 27, Issue 4

Introduction

High-quality pharmacoepidemiological research relies on accurate ascertainment of medication exposure. The primary ways of ascertaining medication exposure in pharmacoepidemiological studies are using pharmaceutical claims or prescription data, self-reporting, or individual medication inventory at interview. Each has advantages and disadvantages. Claims data are relatively cheap to use, document use over time and are available on a large scale; however, they only document prescribing or dispensing of medications, rather than their actual use, and generally do not include medications that are not subsidised.¹ Self-report methods can capture information on a wide range of prescription, nonprescription, complementary and alternative medicines reported as being used by the patient. However, there may be issues with accuracy of self-report and recall, and these methods are potentially costly to apply on a large scale. A formal medication review is often seen as the gold standard of medication ascertainment², but is particularly resource intensive.

A number of studies have assessed the validity and accuracy of self-reported medication data compared with pharmaceutical claims data. Although they have demonstrated that self-reporting is generally appropriate for measuring prescription medication exposure^3-6, most have evaluated specific medications, or chronically used medications such as cardiovascular medications.⁴ It is possible that the validity of self-reported data varies according to the medication type in terms of clinical indication. For example, there may be issues with capturing intermittent medication use or medications associated with stigma.

Thus, the aim of this study was to compare the accuracy of self-reported prescription-only medication use across a very wide range of medication types with the gold standard method of linking routine data on claims for dispensed prescription-only medications. We used data from a sample of participants from the 45 and Up Study, a large-scale Australian cohort study.

Methods

Study population

This study used baseline questionnaire data from the Sax Institute’s 45 and Up Study, which includes 267 153 men and women aged ≥45 years from New South Wales, Australia, randomly sampled from the Medicare Australia database.⁷ Participants completed a self-administered postal questionnaire. Recruitment was conducted from February 2006 to April 2009, with an 18% response rate. For this study, we selected a random sample of 500 individuals who were healthcare concession cardholders, and hence had complete and consistent capture of their dispensed prescriptions through the Pharmaceutical Benefits Scheme (PBS). The sample was enriched for medication use, with 95% of participants having any self-reported medication recorded as free text, in addition to medications listed in check boxes, described in detail elsewhere.⁸ This study was approved by the UNSW Human Research Ethics Committee, Sydney, Australia (05035).

Self-reported medication data

In the 45 and Up Study, participants are asked to provide information on medication use in response to the question, “Have you taken any medications, vitamins or supplements for most of the last 4 weeks?” There is a checklist of commonly used medications and a free-text response box for medications not included as check-box items. The self-reported medicines included both those listed in the check box and those listed in the free-text box. The check-box option included the 32 most common medications, vitamins and supplements used in the Australian population at the time the baseline survey was conducted. The free-text fields were reviewed and coded by medication experts.⁸ If a check-box medication was both ticked and listed in the free-text box, we only counted it once. We defined medication exposure as use of any self-reported prescription medication as listed in the Australian registered product information⁹ in the 4 weeks before the survey (coded as 28 days for the purposes of our analysis). We chose this approach because prescription-only medications are more likely to appear consistently in PBS records. We classified participants as ‘unexposed’ if there was no record of prescription-only medications in the check box or free-text box.

Medication names were converted to active ingredient names using the Australian registered product information.¹⁰ Active ingredient names were coded automatically to Anatomical Therapeutic Chemical (ATC) classification codes to chemical substance code level (7 digits), chemical subgroup code level (5 digits), pharmacological subgroup code level (4 digits) or therapeutic subgroup code level (3 digits). The results for the 7- and 3-digit codes are presented in this paper. We did not include vaccines (ATC code J07), because these agents are not typically considered to be medications and they are not used regularly.

Pharmaceutical claims data

Questionnaire data from participants were linked to PBS data from 2004 to 2011, supplied by the Australian Government Department of Human Services and linked by the Sax Institute. The PBS dataset contains information on subsidised claims for dispensing of prescribed medicines listed on the Schedule of Pharmaceutical Benefits in Australia. During the study period, complete dispensing histories were generally recorded only for healthcare concession cardholders. Hence, the study sample was restricted to these individuals.

We defined distinct medications at the 7-digit ATC level, and therapeutic subgroup using 3-digit codes. We classified ‘gold standard’ exposure as an individual being dispensed a PBS medicine up to 90 days before the 4-week (28-day) window before the survey, as this dispensing is likely to include the 4-week period relating to the survey response. Hence, an individual was considered ‘exposed’ to a specific medication if they were dispensed a PBS medicine up to 90 days before the 4-week period before the survey. The 90-day period is well established in the literature and is used for the primary analysis presented in this paper, with 60- and 30-day estimates used for sensitivity analysis.^11,12 The three time periods account for different lengths of time that a dispensed prescription might last in the community.

Data analysis

The PBS data were considered the gold standard. The main comparisons were performed in relation to the therapeutic subgroup (3-digit ATC) and individual medications (7-digit ATC). In the questionnaire, two check-box medications, namely ‘Coversyl (C09AA04), Coversyl plus (C09BA04)’ and ‘Avapro (C09DA04), Karvea (C09CA04)’ could be assigned two ATC codes because these refer to two different generic names. To account for this, these medications were included for comparison at the 3-digit level only. We calculated sensitivity (i.e. the proportion of PBS medication use that is identified correctly by self-reporting), positive predictive values (PPVs; proportion of self-reported use that is confirmed by the PBS data) and specificity (i.e. proportion of non-use of a PBS medication that is correctly identified by self-reporting) at the 3- and 7-digit ATC levels, and used the ‘exact’ binomial method to calculate the corresponding 95% confidence interval (CI). Data analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC).

Results

The mean age of participants was 70.1 years (SD 10.3), 55.6% were female and 71.0% were born in Australia (Table 1). Compared with the 45 and Up Study cohort members, similar to other concession cardholders, our participants were generally older, were more likely to be born overseas, not be married and have fewer educational qualifications.⁸ Overall, 92.8% (464) of participants self-reported use of at least one prescription-only medication, 6.6% (n = 33) self-reported use of nonprescription medications, and 0.6% (n = 3) did not self-report use of any medication in the previous 4 weeks.

Table 1.      Baseline characteristics of the study population (n = 500)

SD = standard deviation
^a Not married included divorced, separated, single or widowed
^b Diabetes, stroke, blood clots or heart disease
Note: Some data are missing for the following characteristics: country of birth (n = 3), marital status (n = 3), household income (n = 34), education (n = 17), alcohol use (n = 16) and self-rated health (n = 18).

The proportion of participants with medications recorded and the sensitivity and PPV of self-reported versus PBS data for prescription-only medications for therapeutic medication classes using the 90-day window period are shown in Table 2, and for individual medications in Supplementary Table 1 (available from: hdl.handle.net/1885/119323).

Table 2.      Frequency, sensitivity and positive predictive value of 45 and Up Study self-reported data for prescription-only medications to 3-digit level ATC classification compared with PBS pharmaceutical claims data (gold standard) considering 90-day exposure definition

ATC = Anatomical Therapeutic Chemical; na = not applicable; PBS = Pharmaceutical Benefits Scheme; PPV = positive predictive value
^a Table sorted by order of pharmaceutical claims data frequency
^b Correct survey entry is where a pharmaceutical claims record exists for the same medication
Note: Only medication classes with 10 or more entries identified using pharmaceutical claims data are presented.

The sensitivity/PPV of self-reported use versus PBS data at the 3-digit ATC level (Table 2) were highest for chronic cardiovascular medications, being 94.4% (95% CI 90.6, 97.0)/96.9% (95% CI 93.7, 98.7), respectively, for lipid-lowering agents; 92.5% (88.5, 95.4)/97.5% (94.6, 99.1) for angiotensin agents; 88.8% (81.2, 94.1)/93.1% (86.4, 97.2) for beta-blockers; and 88.0% (80.3, 93.4)/96.9% (91.3, 99.4) for calcium-channel blockers. The sensitivity of self-reported centrally acting medications was 78.9% (95% CI 69.0, 86.8) for psychoanaleptics; 42.1% (26.3, 59.2) for analgesics; 26.0% (16.5, 37.6) for psycholeptics; and 4.8% (2.0, 9.7) for antibacterial agents. Overall, the PPVs for these medication types were ≥65%, with PPVs of ≥75% for 79% of the medications examined.

The sensitivity/PPV of self-reported use versus PBS data at the 7-digit ATC level (see Supplementary Table 1, available from: hdl.handle.net/1885/119323) varied across individual medications. The highest sensitivity/PPV for self-reported versus claims data at the 7-digit ATC level were 96.9% (95% CI 83.8, 99.9)/96.9% (95% CI 83.8, 99.9) for warfarin; 94.5% (88.4, 98.5)/92.0% (85.3, 96.3) for atorvastatin; 94.3% (80.8, 99.3)/84.6% (69.5, 94.1) for pantoprazole, and 93.3% (81.7,98.6)/95.5% (84.5, 99.4) for atenolol. The lowest sensitivities were observed for a range of individual medications, being 16.7% (95% CI 5.6, 34.7) for temazepam; 15.2% (5.1, 31.9) for perindopril; 11.5% (4.4, 23.4) for irbesartan; 11.1% (1.4, 34.7) for oxazepam, and 3.3% (0.1, 17.2) for amoxicillin.

Overall, the PPVs for individual medications were ≥75% for 81% of the medications examined. Findings from analyses using 30- and 60-day exposure periods (see Supplementary Table 2, available from: hdl.handle.net/1885/119323) did not differ from those in the main analyses.

Data on specificity using the 90-day period are provided in Supplementary Table 3 (available from: hdl.handle.net/1885/119323). Specificities of self-reported use versus PBS data at the 3-digit ATC level ranged from 96.5% (95% CI 94.4, 97.9) for cardiac therapy agents to 100% for ophthalmologic agents.

Discussion

The findings of this study indicate that sensitivity and PPVs for self-reported versus PBS claims data for prescription-only medications are good overall, and vary by medication type. The sensitivity and PPVs for self-reported cardiovascular medications were good to excellent compared with the claims data. This is consistent with previous studies, which have shown that self-reported data can be used to accurately identify the use of statins and diuretics.^3,13

Our results are similar to a recently published paper that compared a self-administered mailed form for specific medications, and found near-perfect agreement between self-reported data and pharmacy records for recent use of long-term medicines such as statins, beta-blockers and calcium-channel blockers.¹⁴ In this study, participants were asked to refer to their prescription labels while completing the form; therefore, this approach may be useful to collect self-reported data on recent medication use. The accuracy of self-reporting exposure to cardiovascular medications is consistently good across studies, regardless of study population, study designs and methods used to capture medication use.^13-15

Our study found that self-reported data of the type used here may be less useful for identifying exposure to medications such as anxiolytics, sedatives, analgesics and antibacterial agents. These medications are typically taken for short periods or intermittently, so may be less likely to be identified as having been used for “most of the last 4 weeks” by study participants. In general, the findings for sensitivities and PPVs for medication classes (3-digit level) reflected consistent findings for individual medications (7-digit level), including for lipid-modifying agents, medications for acid-related disorders, beta-blockers, antithrombotic agents, diuretics and calcium-channel blockers. However, findings could vary within medication classes, as demonstrated by high sensitivities for most angiotensin II–receptor antagonists and angiotensin-converting enzyme inhibitors, apart from irbesartan and perindopril. The reasons for this variation are unclear, but should be borne in mind in future studies.

Importantly, PPVs remained relatively high for most of the medication types, including those with low prevalence. This means that, for exposure classification, most of those identified as exposed to the medicine by self-report will have been exposed. Even if sensitivity is not particularly high, sound comparisons can be made between exposed and unexposed when medication exposure is relatively uncommon. Contamination of the unexposed group (i.e. low sensitivity/inclusion of people exposed to a medication in the unexposed or control group) may occur where medication exposure is common, and would generally mean that estimates of relative risk would be biased towards the null (i.e. towards a relative risk of 1), leading to more conservative findings. However, low sensitivity and specificity may compromise study power, and complementary information on sensitivity and specificity, clinical context, research design, and research questions should be considered. Hence, self-reported data are likely to be useful to identify and compare exposed individuals for many medications.

To our knowledge, our study is the first to compare the accuracy of self-reported medication data with pharmaceutical claims data for a range of medications. This is essential to better understand the merits of self-reported data across the full spectrum of prescribed medications. Additional analyses were conducted to account for different exposure periods, because the degree of misclassification of medication exposure may vary across different fixed-time periods and may partly explain differences observed across studies. The PBS dataset was our gold standard. However, this is imperfect because participants may be dispensed medications that they do not use, and pharmaceutical claims datasets in Australia do not capture all prescription medications, including medications supplied by hospitals or available as private prescriptions. These two factors may oppose each other, meaning that PBS dispensings may overestimate exposure to medications that were not then used (this might apply more to short-term or one-off dispensings) and may underestimate exposure to certain medicines. Hence, some of the PBS dispensings that were not accompanied by self-reporting may be medications that were not taken, consistent with the finding of low validity for use of certain medications that are generally taken short term or as a one-off, and some of the ‘false positives’ observed in our study may be medications that participants were receiving from other sources. These factors suggest that our findings regarding validity may be underestimated.

To allow reliable comparisons between self-reporting and dispensing data, our study was restricted to concession cardholders. This group is generally older and more likely to have chronic disease and be prescribed multiple medications than the general population. These factors may decrease the accuracy of self-reporting, indicating the potential for the validity of self-reporting to be somewhat better in the general population. We enriched the sample for medication users – about 99% of our sample reported use of any medication during the past 4 weeks, compared with 91% of all healthcare concession cardholders in the broader 45 and Up Study. This is unlikely to have had a large effect on validity of reporting for medication classes and individual medications, but a slight elevation in the tendency to self-report cannot be excluded. The survey structure may have also affected the accuracy of self-reported medications.¹⁶ One study found that some medications, such as antibacterial agents, are more likely to be reported if participants are prompted (73% sensitivity) rather than asked to list medications in open-ended question (50% sensitivity).¹⁷ Interestingly, although studies have noted that participants may be less inclined to report the use of psychoanaleptics such as antidepressants due to self-stigmatisation¹⁸, this was not observed in our study.

Conclusion

In summary, self-reported medication data of the type examined here are likely to be useful for identifying exposure to a range of prescription medications, particularly those used long term, in future pharmacoepidemiological studies. However, caution is warranted when using such data to identify prescription-only medications used in the short term or intermittently. These findings have important implications for design and interpretation of pharmacoepidemiological studies using self-reported data to examine use and safety of prescription medications in real-world settings.

Acknowledgements

This research was completed using data collected through the 45 and Up Study (www.saxinstitute.org.au/our-work/45-up-study). The 45 and Up Study is managed by the Sax Institute in collaboration with major partner Cancer Council NSW, and other partners: the National Heart Foundation of Australia (NSW Division), the NSW Ministry of Health, beyondblue, the NSW Government Family and Community Services – Carers, Ageing and Disability Inclusion, and the Australian Red Cross Blood Service. We thank the many thousands of people participating in the 45 and Up Study. This study was funded by the National Health and Medical Research Council of Australia (NHMRC project grant number 1024450 and NHMRC Centre of Research Excellence in Medicines and Ageing, 1060407). DG is supported by an NHMRC Early Career Fellowship and EB is supported by an NHMRC Senior Research Fellowship.

Copyright:

© 2017 Gnjidic et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.