Diagnosis-based and external cause-based criteria to identify adverse drug reactions in hospital ICD-coded data: application to an Australia population-based study

Wei Du, Sallie-Anne Pearson, Nicholas A Buckley, Cathy Day, Emily Banks

Published 27 April. doi:
Citation: Du W, Pearson S, Buckley NA, Day C, Banks E. Diagnosis-based and external cause–based criteria to identify adverse drug reactions in hospital ICD-coded data: application to an Australian populationbased study. Public Health Res Pract. 2017;27(2):e2721716.

  • Citation

  • PDF

About the author/s

Wei Du | National Centre for Epidemiology and Population Health, Australian National University, Canberra, ACT

Sallie-Anne Pearson | Faculty of Pharmacy, University of Sydney, NSW, Australia

Nicholas A Buckley | Discipline of Pharmacology, University of Sydney, NSW, Australia

Cathy Day | National Centre for Epidemiology and Population Health, Australian National University, Canberra, ACT

Emily Banks | National Centre for Epidemiology and Population Health, Australian National University, Canberra, ACT; Sax Institute, Sydney, NSW, Australia

Corresponding author

Wei Du |

Competing interests

None declared

Author contributions

WD and EB conceived the original idea for the study, conducted the analysis and drafted the first manuscript; all authors contributed to conceptual development, the literature review, data analysis, critical revision of the first manuscript and subsequent drafts.


Objectives: External cause International Classification of Diseases (ICD) codes are commonly used to ascertain adverse drug reactions (ADRs) related to hospitalisation. We quantified ascertainment of ADR-related hospitalisation using external cause codes and additional ICD-based hospital diagnosis codes.

Methods: We reviewed the scientific literature to identify different ICD-based criteria for ADR-related hospitalisations, developed algorithms to capture ADRs based on candidate hospital ICD-10 diagnoses and external cause codes (Y40–Y59), and incorporated previously published causality ratings estimating the probability that a specific diagnosis was ADR related. We applied the algorithms to the NSW Admitted Patient Data Collection records of 45 and Up Study participants (2011–2013).

Results: Of 493 442 hospitalisations among 267 153 study participants during 2011–2013, 18.8% (n = 92 953) had hospital diagnosis codes that were potentially ADR related; 1.1%    (n = 5305) had high/very high–probability ADR-related diagnosis codes (causality ratings: A1 and A2); and 2.0% (n = 10 039) had ADR-related external cause codes. Overall, 2.2%       (n = 11 082) of cases were classified as including an ADR-based hospitalisation on either external cause codes or high/very high–probability ADR-related diagnosis codes. Hence, adding high/very high–probability ADR-related hospitalisation codes to standard external cause codes alone (Y40–Y59) increased the number of hospitalisations classified as having an ADR-related diagnosis by 10.4%. Only 6.7% of cases with high-probability ADR-related mental symptoms were captured by external cause codes.

Conclusion: Selective use of high-probability ADR-related hospital diagnosis codes in addition to external cause codes yielded a modest increase in hospitalised ADR incidence, which is of potential clinical significance. Clinically validated combinations of diagnosis codes could potentially further enhance capture.

Full text

Key points

  • Use of external cause–based codes alone gives slightly more conservative estimates of the incidence of adverse drug reactions than use of combined diagnosis and external cause codes
  • Selectively adding specific drug-induced diagnosis codes to external cause codes increases hospitalisations attributed to adverse drug reactions by about 10%


Drug-related adverse medical events are a major health problem.1-4 Surveillance is a key strategy to quantify the magnitude of the adverse drug reaction (ADR) problem, set priorities, identify countermeasures and evaluate interventions.5

ADR surveillance may use International Classification of Diseases (ICD) codes (9th or 10th revisions) to ascertain ADRs from routinely collected data on deaths, hospitalisations and emergency department visits.6-13 These ICD codes comprise diagnoses describing specific clinical symptoms and signs indicating any ADRs (e.g. D61.1 – drug-induced aplastic anaemia), and supplementary classifications documenting external causes to ADR indications (e.g. Y40.0 – penicillin causing adverse effects in therapeutic use).14

Using different ICD code–based ADR selection criteria is likely to result in different incidence estimates. Hohl et al. reported varying sensitivity of 6.8–28.1% and specificity of 87.7–98.7% for different ICD code sets to capture ADR-related emergency department presentations among 1574 patients.12 Wu reported consistent sensitivities of 56% and specificities of 99% for different code sets to capture ADR-related hospitalisations in Canada.13 These findings of relatively low sensitivity and high specificity indicate incomplete capture of ADR-related events using routinely collected ICD-coded health administrative data. Most commonly, ADR ascertainment uses external cause–based criteria alone9,10, but external cause codes and/or diagnosis codes are also used.7,8,12,13 The implications of using different approaches are unclear, and may differ by location and over time.7,9,11,13

This study aims to quantify the impact of using different external cause code combinations and hospital diagnosis–based criteria to ascertain ADR-related hospitalisations.


Data sources

The Sax Institute’s 45 and Up Study is an ongoing large-scale Australian cohort study of 267 153 participants from New South Wales (NSW), Australia’s most populous state.15 During 2006–2009, participants aged 45 and over were randomly selected from the Medicare Australia enrolment database, which covers virtually the whole population. Participants provided signed consent for linkage to health data, including routinely collected hospitalisation records linked through the Centre for Health Record Linkage (CHeReL).16

We extracted linked hospitalisation data for participants with a minimum age of 47 years during 2011–2013 from the NSW Admitted Patient Data Collection, a complete census of inpatient separations from NSW public, private and repatriation hospitals, and private day procedure centres.16 Separation records were coded using the Australian modification of ICD-10 (ICD-10-AM, 7th version).14 Like other ICD versions, this has external cause codes indicating ADR occurrence and clinical diagnosis codes indicating ADR symptoms (i.e. unintended consequences during normal pharmaceutical therapies).17 We considered consecutive episodes of care (i.e. a patient was transferred rather than discharged) as a nested incident case to avoid multiple counting of episodes (including same-day admissions). We used the ADR-related external cause codes and diagnosis codes to derive different case sets for comparison.

ADR-related external cause codes

We selected ICD-10 Chapter XX external cause codes Y40–Y59 to ascertain ADR events caused by a medicine properly administered in therapeutic or prophylactic dosage. This selection is reasonably well correlated with the widely accepted ADR definition, developed by the World Health Organization, as “noxious and unintended response to a drug that occurs at doses normally used in humans”.17

ADR-related diagnosis codes

We summarised 279 ADR-related ICD-10 diagnosis code entries for analysis (see Supplementary Table 1, available from:, based on 1) a recent systematic review18; 2) additional codes with supplementary specification of “Use additional external cause code (Chapter XX), if desired, to identify drug, if drug-induced” (e.g. G43 – migraine) by a manual search14; and 3) additional codes listed in the adverse drug events chapter of the Classification of Hospital Acquired Diagnoses (CHADx, version 5).19

We adopted causality rating categories A1, A2, B, C and D, based on the systematic review.18 We created an additional ‘other’ category for ICD codes with causality ratings other than A–D18 and the newly added codes that did not have validated causality ratings (Supplementary Table 1). We considered diagnosis codes of A1 or A2 causality ratings as having a very high or high probability of being ADR related. A1 includes a description of ADR causality (e.g. D52.1 – drug-induced folate deficiency anaemia), and A2 has “due to drug” causality but suspecting other causal substances (e.g. E03.2 – hypothyroidism due to medicaments and other exogenous substances). Categories B–D and ‘other’ have a lower probability of being drug related, but contain medication safety signals for potential in-depth review and evaluation of cases. We further categorised candidate hospital diagnosis codes to system–organ groups based on ICD-10 anatomical chapters, considering different capture probabilities across diagnostic groups. Episodes related to intentional self-poisoning (external cause codes X60–X69) were excluded.

Statistical analysis

We derived multiple ADR-related case sets based on different combinations of external cause codes and diagnosis codes to identify ADR cases (Table 1), including different levels of causality for the diagnosis codes. We calculated numbers and proportions of hospitalised ADR cases for each case set, and the concordance between diagnosis and external cause codes. Individual code counts were based on the total number of cases having the relevant code. Multiple ICD-10 diagnosis codes within one case were counted separately. We used all hospitalisations as the basis for calculating proportions.

We carried out all analyses using SAS version 9.3 (SAS Institute, 2008) or Microsoft Excel version 2010 (Microsoft Corporation, USA). The NSW Population and Health Services Human Research Ethics Committee and the Australian National University Human Research Ethics Committee approved the study (HREC/10/CIPHS/31).


Table 1.      Number and percentage of adverse drug reaction (ADR) incident cases with additional Y40–Y59 external cause codes, by diagnostic groups with different causality ratings

Diagnostic group causality rating18 % with Y40–Y59 codes (n) Total % of all hospitalisations (n = 493 422) Change in % with ADR by adding the diagnostic group to standard Y40–Y59 codesa
A1: induced by drug/medication included in the code description 92.5 (3413) 3 690 0.7 0.1
A2: induced by drug/medication included in the code description, but suspecting other causal substances 54.6 (932) 1 707 0.3 0.2
A1 or A2 80.3 (4262) 5 305 1.1 0.2
B: poisoning by medication included in the T-code description, but suspecting other causal substances 16.9 (68) 403 0.1 0.1
C: induced by drug/medication not included in the code description, but deemed to be very likely 38.7 (420) 1 085 0.2 0.1
D: induced by drug/medication not included in the code description, but deemed to be likely 14.4 (2955) 20 575 4.2 3.6
Other: induced by drug/medication not included in the code description, but deemed to be possible, unlikely or unknown 8.9 (7322) 82 415 16.7 15.2
Any causality rating 9.7 (9059) 92 953 18.8 17.0
a We used the Y40–Y59 external cause code–based ADR cases (n = 10 039) as the numerator and all hospitalisations (n = 493 442) as the denominator for calculating proportion and proportional change when using either diagnosis or external cause codes.



During 2011–2013, 267 153 participants in the 45 and Up Study had a total of 493 442 hospitalisations. Of these, 1.1% (n = 5305) had high/very high (or both)–probability ADR-related diagnosis codes (causality ratings A1 and A2), made up of 0.7% (n = 3690) very high–probability and 0.3% (n = 1707) high-probability codes. A total of 18.8% (n = 92 953) had any potentially ADR-related diagnosis codes (A1 to D and other; see Table 1), and 2.0% (n = 10 039) had Y40–Y59 external cause codes, which are specified as ADR related. Overall, 2.2% (n = 11 082) of hospitalisations were identified as including an ADR, based on either external cause codes or high/very high–probability ADR-related diagnosis codes. Using either external cause or high/very high–probability ADR-related diagnosis codes increased the number of identified ADR cases by 10.4% (n = 1043) relative to using external cause codes alone (11 082 and 10 039, respectively).

Concordance between diagnosis codes and external cause codes

Most cases with A1 diagnosis codes (92.5%, n = 3413) also had ADR-related external cause codes, but only 54.6% (n = 932) of A2 diagnosis codes also had ADR-related external cause codes (see Table 1). The Y40–Y59 external cause–based selection demonstrated different capture probabilities across diagnostic groups (e.g. higher capture of cases with respiratory, digestive, endocrine or dermatological signs, and lower capture in cases with mental symptoms; Table 2).


Table 2.      Number of high-probability ADR-related cases by diagnostic group and percentage with external cause codes

Diagnostic group Subgroup Number withY40–Y59 codes/totala % with Y40–Y59 codesa
A1 causality rating Anaemias 1814/2013 90.1
Circulatory 1013/1062 95.4
Fever 278/284 97.9
Nervous 198/218 90.8
Respiratory or digestive 66/66 100.0
Musculoskeletal 48/55 87.3
Endocrine or skin 45/48 93.8
Total cases with ≥1 A1 causality code 3413/3690 92.5
A2 causality rating Endocrine or skin 492/506 97.2
Mental 24/358 6.7
Genitourinary 97/124 78.2
Circulatory 9/10 90.0
Unclassifiable 324/733 44.2
Total cases with ≥1 A2 causality code 932/1707 54.6
a We excluded intentional self-poisoning cases and used the total numbers as the basis for calculating proportion with external cause codes.


In NSW, external cause–based selection alone ascertained a larger number of ADR cases than high-probability ADR diagnosis-based selection alone, consistent with previous findings.7,8,11 The Y40–Y59 external cause codes captured about 80% of hospitalisations with “drug-induced” or “due to drug” diagnosis codes, a much higher concordance than the 12–15% reported using English hospital statistics more than a decade ago.11 Adding high-probability ADR-related diagnosis codes to external cause–based selection yielded a 10.4% increase in ADR numbers on a frequency basis, such that 2.2% of hospitalisations, in this Australian context, were classified as an ADR-related event; this is consistent with a 2–3% national figure based on a recent review.20 Therefore, external cause–based codes alone give slightly more conservative estimates of ADR incidence than those based on combined diagnosis–external cause methods.

Of note is that 4.2% of hospitalisations included possible ADRs of causality ratings A–D, based on diagnosis codes. Many cases diagnosed as having a lower probability of being ADR related were not captured by the external cause codes. Should ADR causality have been confirmed for all these cases, the burden of ADR-related hospitalisations would have increased approximately twofold. However we were unable to validate these cases using another data source. Although the exact study results may not be generalisable, the finding of underascertainment by use of external cause codes alone is likely to hold.

Routinely collected hospitalisation data do not necessarily include medication use, especially when use occurs outside hospital. Without this information, it is legitimate to use diagnosis code–based selection to supplement external cause–based case selection. However, ADR-related diagnosis codes should be carefully scrutinised for ADR causality where possible by means of additional chart reviews and/or record linkage to medication use. Alternatively, refinement of ICD-10 codes by emphasising the use of additional codes to identify medicines14 may facilitate coders assigning external causes when reviewing medical records.

Diagnosis codes include clinical information that provides potential medication safety signals, which could be useful for ADR prevention efforts. This study demonstrates that the capture of high/very high–probability diagnosis codes by external cause–based selection varies across diagnostic groups, with the highest for respiratory or digestive symptoms and signs (100%), followed by fever (98%), and the lowest for mental symptoms (7%). Price et al. also reported that external cause–based selection was associated with case underascertainment and variation across different symptoms.21 This finding is relevant to future studies considering specific ADR signals, suggesting that diagnosis codes may be particularly useful for ascertaining certain ADRs, particularly acute-onset symptoms and signs that might be more distinctively ADR related. It is concerning that the capture of mental symptoms by external cause–based selection remains low, consistent with the previously reported 0–1% capture rate using 1996–2000 hospitalisation data in England.11 Although external cause–based selection might be less useful for capturing chronic versus acute ADRs22, possible underlying reasons for the low capture of mental health symptoms might include the lack of formal medication reviews when using less specific external codes14, which may be necessary for identifying ADR psychiatric symptoms. Further research is needed to explore reasons for discrepancies between external cause and diagnosis codes to substantially improve ADR identification.

The identification of ADR-related hospitalisations relies on accurate and specific coding of diagnoses and external causes in routinely collected hospital records. We were unable to verify whether a diagnosis-based or external cause–based ADR-related hospitalisation was due to medicine use because relevant medical records were not accessible. Nor were we able to verify causality ratings and administered dose information for selected diagnosis codes, particularly given the large number of potential ADR cases based on diagnosis codes alone (n = 92 953). Such verification could be completed through independent chart review or medical record audit. In their absence, we did not estimate corresponding sensitivity and specificity for selected ICD code sets to ascertain ADR-related hospitalisations. Previous findings of low sensitivity indicate that many ADR-related events are not captured by application of ICD code–based selection criteria12,13, which would result in an underestimation of ADR-related incidence, particularly when using relatively narrow ICD code–based definitions. Different coding practice between jurisdictions and hospitals may explain a part of the observed variation of using different ICD codes to detect ADRs. Hence, any estimated ADR incidence rates based on administrative data alone should be interpreted with caution. Nevertheless, we applied the most recent systematic review of ADR-related ICD-10 codes18, and limited our analysis to high-probability or very high–probability ADR diagnosis only. The study findings demonstrate a modest increase in hospitalised ADR incidence using relevant diagnosis codes in addition to external cause codes for identifying ADR-related hospitalisations.

ADR-related hospitalisations represent a significant and potentially avoidable burden for healthcare systems.1 Increased identification of ADRs contributes to more accurate quantification of the problem, and hence more appropriate prioritisation and targeting of interventions to prevent them.

In conclusion, the addition of high-probability ADR-related diagnosis codes to the common practice of using external cause codes to identify ADR-related hospitalisations yields a modest, but potentially clinically meaningful, increase in estimates of incidence. Clinically validated selection criteria based on routinely collected ICD-coded databases should contribute to more accurate and complete quantification of the ADR burden and inform the development of preventive strategies.


This research was completed using data collected through the 45 and Up Study ( The 45 and Up Study is managed by the Sax Institute in collaboration with major partner Cancer Council NSW; and partners the National Heart Foundation of Australia (NSW Division); the NSW Ministry of Health; the NSW Department of Family and Community Services – Carers, Ageing and Disability Inclusion; and the Australian Red Cross Blood Service. We thank the many thousands of people participating in the 45 and Up Study. This study is funded by the National Health and Medical Research Council of Australia (NHMRC; Project Grant 1024450, and Centre of Research Excellence in Medicines and Ageing; 1060407). EB is supported by the NHMRC, and SAP is supported by the Cancer Institute NSW.


Creative Commons License

© 2017 Du et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.


  • 1. Runciman WB, Roughead EE, Semple SJ, Adams RJ. Adverse drug events and medication errors in Australia. Int J Qual Health Care. 2003;15 Suppl 1:i49–59. CrossRef | PubMed
  • 2. Pirmohamed M, James S, Meakin S, Green C, Scott AK, Walley TJ, et al. Adverse drug reactions as cause of admission to hospital: prospective analysis of 18 820 patients. BMJ. 2004;329:15–9. CrossRef | PubMed
  • 3. Kongkaew C, Noyce PR, Ashcroft DM. Hospital admissions associated with adverse drug reactions: a systematic review of prospective observational studies. Ann Pharmacother. 2008;42(7):1017–25. CrossRef | PubMed
  • 4. Leendertse AJ, Visser D, Egberts ACG, van den Bemt PM. The relationship between study characteristics and the prevalence of medication-related hospitalizations: a literature review and novel analysis. Drug Saf. 2010;33(3):233–44. CrossRef | PubMed
  • 5. Edwards IR, Aronson JK. Adverse drug reactions: definitions, diagnosis, and management. Lancet. 2000;356(9237):1255–9. CrossRef | PubMed
  • 6. Chyka P. How many deaths occur annually from adverse drug reactions in the United States? Am J Med. 2000;109(2):122–30. CrossRef | PubMed
  • 7. Patel H, Bell D, Molokhia M, et al. Trends in hospital admissions for adverse drug reactions in England: analysis of national hospital episode statistics 1998–2005. BMC Clin Pharmacol. 2007;7:9. CrossRef | PubMed
  • 8. Wu C, Bell CM, Wodchis WP. Incidence and economic burden of adverse drug reactions among elderly patients in Ontario emergency departments: a retrospective study. Drug Saf. 2012;35(9):769–81. CrossRef | PubMed
  • 9. Burgess CL, Holman CD, Satti AG. Adverse drug reactions in older Australians, 1981–2002. Med J Aust. 2005;182(6):267–70. PubMed
  • 10. Zhang M, Holman CD, Preen DB, Brameld K. Repeat adverse drug reactions causing hospitalization in older Australians: a population-based longitudinal study 1980–2003. Br J Clin Pharmacol. 2007;63(2):163–70. CrossRef | PubMed
  • 11. Waller P, Shaw M, Ho D, Shakir S, Ebrahim S. Hospital admissions for ‘drug-induced’ disorders in England: a study using the Hospital Episodes Statistics (HES) database. Br J Clin Pharmacol. 2005;59(2):213–19. CrossRef | PubMed
  • 12. Hohl CM, Kuramoto L, Yu E, Rogula B, Stausberg J, Sobolev B. Evaluating adverse drug event reporting in administrative data from emergency departments: a validation study. BMC Health Serv Res. 2013;13:473–84. CrossRef | PubMed
  • 13. Wu C. Adverse drug reactions in the emergency department population in Ontario: analysis of national ambulatory care reporting system and discharge abstract database 2003–2007. Toronto: Wu C; 2009 [cited 2017 Jan 13]. Available from:
  • 14. National Centre for Classification in Health. International statistical classification of diseases and related health problems, 10th revision, Australian modification (ICD-10-AM) 7th edition. Sydney: National Centre for Classification in Health; 2010.
  • 15. 45 and Up Study Collaborators, Banks E, Redman S, Jorm L, Armstrong B, Bauman A, et al. Cohort profile: the 45 and Up Study. Int J Epidemiol. 2008;37(5):941–7. CrossRef | PubMed
  • 16. CHeReL: Centre for Health Record Linkage. Sydney: CHeReL; 2017 [cited 2017 Jan 13]. Available from:
  • 17. World Health Organization. International drug monitoring: the role of national centres. WHO technical report series 498. Geneva: WHO; 1972 [cited 2017 Jan 13]. Available from:
  • 18. Hohl CM, Karpov A, Reddekopp L, Doyle-Waters M, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc. 2014;21(3):547–57. CrossRef | PubMed
  • 19. Jackson TJ, Michel JL, Roberts RF, Jorm CM, Wakefield JG. A classification of hospital-acquired diagnoses for use with routine hospital data. Med J Aust. 2009;191(10):544–8. PubMed
  • 20. Roughead L, Semple S, Rosenfeld E. Literature review: medication safety in Australia. Sydney: Australian Commission on Safety and Quality in Health Care; 2013. Available from:
  • 21. Price SD, Holman CD, Sanfilippo FM, Emery JD. Use of case-time-control design in pharmacovigilance applications: exploration with high-risk medications and unplanned hospital admissions in the Western Australian elderly. Pharmacoepidemiol Drug Saf. 2013;22(11):1159–70. CrossRef | PubMed
  • 22. Hodgkinson MR, Dirnbauer NJ, Larmour I. Identification of adverse drug reactions using the ICD-10 Australian modification clinical coding surveillance. J Pharm Pract Res. 2009;39(1):19–23. CrossRef