Enhanced self-harm presentation reporting using additional ICD-10 codes and free text in NSW emergency departments September 2023, Volume 33, Issue 3

Introduction

Emergency departments (EDs) are often the first point of contact for Australians with self-harm or suicidal thoughts.¹ Self-harm presentation rates have grown steadily over the last decade^1,2, which may have accelerated since the coronavirus disease 2019 (COVID-19) pandemic, particularly in adolescents.³ Australian health systems need accurate data to understand and respond to these increases.

Diagnosis or presenting problem codes in ED datasets usually form the basis for analysing and reporting ED system activity. Reporting is often based on the International Classification of Diseases (ICD) diagnosis codes. In ICD-10, codes X60-84 and Y87 indicate deliberate self-harm and code R45.81 indicates suicidal ideation. Previously, ICD-9 identified self-harm by combining codes for poisoning or injury with external cause codes identifying self-inflicted events (E950-E959). However, file audit studies find that routinely recorded ED diagnoses or presenting problem codes identify fewer than half of clinically documented self-harm attempts.^4,5 This reflects the inherent challenges of collecting and coding ED data. In Australian EDs data are typically entered by clinicians rather than being coded by trained Health Information Managers. ED information systems often provide clinicians with a limited choice of codes in settings where workloads are high, assessments may be incomplete at the time of coding, and presentations may involve a mix of self-injury, mental health and/or substance use conditions.⁶

Therefore, up until now, Australian health reporting has relied on other measures of self-harm trends. Hospital admissions rates for self-harm are based on data extracted by manual coding of diagnostic and clinical information. Ambulance callout rates for self-harm or suicidal ideation are currently calculated using clinical coding of ambulance data, with monthly snapshots from participating states and territories.⁷ These measures provide important information about trends and risk groups. However, they are incomplete and may be biased towards some demographic groups or types of presentation, leading to calls for renewed efforts to improve reporting of ED data.^6,8

Three main strategies have been used to improve the identification of suicide and self-harm-related presentations in large ED datasets. First, sensitivity may be increased by expanding target ICD-10 codes to include codes for poisoning with psychotropics, analgesics, anti-inflammatories or other agents where a substantial proportion of poisonings are likely to be non-accidental.⁴ Second, as free text data becomes more available in health datasets, diagnosis codes may be supplemented by text-based methods such as rule-based keyword detection⁹ or machine learning.^10,11 Third, ED information systems in some Australian states and territories employ a “human intent” flag to identify self-inflicted injuries.¹² Each health system may need to use a different combination of these approaches to best suit their local information systems, data availability and reporting purpose.

In this study, we describe the development, testing and implementation of an enhanced detection method for reporting suicide and self-harm-related presentations to New South Wales (NSW) EDs. The method combines two of the strategies above: i) an expanded set of ICD-10 codes; and ii) rule-based processing of a presenting problem free-text field. The method has recently been implemented in routine statewide reporting.

Methods

Setting and context

This work was undertaken by the NSW Ministry of Health, the NSW government agency responsible for planning, operation and monitoring of NSW hospital and community health services. The state of NSW had a population of 8.2 million people in 2020.¹³ The NSW Emergency Department data collection currently includes more than 2.9 million ED presentations per year from EDs in 174 NSW government (“public”) hospitals.¹⁴

Ethics approval

Method development was conducted as part of the NSW Mental Health Living Longer project, an ongoing data linkage examining causes of premature mortality in people using NSW Mental Health services.¹⁵ The project has been approved by the NSW Population and Health Service Research Ethics Committee (AU RED Reference number: HREC/17/CIPHS/48, CINSW Reference numbers: 2017/HRE1105, 2018/UMB0302).

Data sources and linkage

Data were obtained from the NSW Emergency Department Data Collection (EDDC) and Admitted Patient Data Collection (APDC). Data were linked by the NSW Centre for Health Record Linkage (CHeReL) using probabilistic linkage procedures.¹⁶

Stage 1: Method development and piloting

Method development and validation included all NSW emergency department presentations from 1 January 2005 to 31 December 2018. We examined two available fields in NSW ED data: i) presenting problem text, a 50-character text string typically entered by nursing triage staff as free text or from locally customised drop-down lists; and ii) a problem code (the “Principal ED Diagnosis”) recorded by attending ED clinicians after an episode of ED care. In most NSW EDs, clinicians select a diagnosis description from a drop-down list, and the corresponding identifier is recorded using the Standardised Nomenclature for Medical Terminology (SNOMED-CT).¹⁷ A small number of NSW EDs also record diagnosis codes in ICD-10 or ICD-9 formats. We first converted all diagnosis and problem codes to ICD-10-AM (Australian Modification). SNOMED-CT codes were converted using Australian Digital Health Agency mapping tables augmented with additional codes identifying self-harm. Presenting problem text fields were standardised by conversion to lowercase, replacement of non-alphanumeric, non-word and multiple space characters, stop word removal and grouping of related forms of common words (lemmatisation).

To supplement standard ICD-10 self-harm codes (X60–X84, Y87.0), we identified additional ICD-10 codes likely to include a substantial proportion of deliberate self-harm based on clinical consultation and literature reviews. These included ICD-10 codes: T39 (poisoning by nonopioid analgesics, antipyretics and antirheumatics); T42 (poisoning by antiepileptic, sedative-hypnotic and anti-parkinsonian drugs); and T43 (poisoning by psychotropic drugs, not elsewhere classified). We initially screened presenting problem text by i) searching for terms likely to indicate suicide or self-harm; and ii) analysing terms most frequently associated with clinical problem codes for deliberate self-harm or suicidal ideation. A preliminary method was defined using a combination of these approaches. Presentations by people aged under 10 were excluded after pilot testing, because of low rates of specific self-harm coding but high rates of accidental poisoning or injury.

We distinguished two types of suicide and self-harm-related presentations, using terminology proposed by Ceniti et. al.¹⁸ These were i) suicide-related behaviour, where self-harm had occurred; and ii) suicide-related ideation, where harm had not occurred but the person expressed thoughts or plans of self-harm or suicide. In this terminology, suicide-related behaviour includes suicide attempts and self-harm without an intent to die.¹⁸ We did not attempt to capture the dimension of intent in our definition, because we did not believe this important issue could be reliably distinguished using the limited data sources available.

Stage 2: Method validation

In pilot analyses, approximately 45% of suicide and self-harm-related presentations of suicide-related behaviour resulted in hospital admission. For that subgroup, we joined ED records to the subsequent hospital admission using unique person identifiers provided by the CHeReL’s data linkage process, along with the date, time and facility details. We used diagnoses from the corresponding inpatient admission as reference diagnoses to validate the ED diagnoses. In NSW, diagnoses for admitted hospital episodes are clinically coded by Health Information Managers based on discharge summaries and clinician records. A principal diagnosis and up to 50 additional diagnoses or external cause ICD-10 codes are recorded. Initial examination of inpatient ICD-10 codes suggested that suicide-related ideation (R45.81) was not consistently coded in inpatient records, and therefore validation focused on suicide-related behaviour (X60–X84, Y87.0). Our focus was to increase sensitivity of detection by reducing ‘false negative’ coding: we assumed that where an ED clinician deliberately applied a code for suicide-related behaviour that should be treated as a ‘true positive’ diagnosis. Therefore, our reference diagnoses for validation comprised records with either i) a clinician-allocated ED diagnosis code; or ii) an inpatient code indicating suicide-related behaviour (X60–X84, Y87.0).

After excluding planned or non-emergency ED visit types and presentations by people aged under 10 years, we identified approximately 8 million ED records with a linked inpatient hospital record. These records were split into 70% training and 30% testing datasets stratified by hospital, year of presentation and self-harm status. This stratification aimed to preserve the distribution of self-harm reference diagnoses in both datasets and to balance any systematic differences in coding practices across different hospitals or over time.

Stage 3: Method revision

The performance of the method was evaluated by calculating sensitivity, specificity and positive predictive value (PPV). We calculated term frequencies associated with the self-harm class in the training data for single words (unigram), two-word phrases (bigram) and ICD-10 codes. The top 25 keywords from each calculation were saved for further screening. The most plausible words and phrases were consolidated into a single list, along with the initial selection of ICD-10 codes above. We monitored changes in the performance metrics for each inclusion and exclusion of the keywords and ICD-10 codes. For keywords like “self-harm” or “suicide”, regular expressions were used to identify instances combined with terms (“thoughts”, “feelings”, “risks”, etc.) that indicate suicide-related ideation. The algorithm, as currently implemented in NSW Health, is shown in Figure 1.

Figure 1.    Algorithm for defining suicide-related behaviour or ideation presentations in New South Wales Emergency Department (ED) data using presenting problem codes and text. (click figure to enlarge)

Note: Clinician codes entered using Standardised Nomenclature for Medical Terminology (SNOMED-CT) are first mapped to the International Classification of Diseases, version 10 (ICD).

 

To examine the face validity of estimates, we examined summary statistics for NSW ED data from January 2016 to December 2020, to focus on recent data that were more easily interpreted by services. Findings at facility level were shared with NSW local health districts and compared to broader NSW Health reporting of ED activity and mental health presentations. Age and sex distributions of ED self-harm presentations were examined for alignment with known age and sex distributions. Incidence estimates were compared with projected NSW incidence estimates calculated from comparable population studies.

Results

1. Changes in incidence estimates

From 2016 to 2020, core ICD-10 self-harm codes identified an average of 21 797 suicide and self-harm-related presentations per year to NSW Emergency Departments, of which 79% were for suicide-related ideation (R45.81). For the subgroup with suicide-related behaviour, the inclusion of additional ICD-10 codes for overdose or poisoning more than doubled the number of suicide attempts or self-harm presentations identified, from 4519 to 10 087 (Table 1). The further inclusion of keyword searching of presenting problem text added 8 898 suicide-related behaviour presentations and 11 039 suicide-related ideation presentations, which were not identified using ICD-10 codes alone. The enhanced method identified an average of 51 822 episodes per year of suicide-related behaviour or ideation in NSW EDs, of which only 42% were identified using core ICD-10 suicide and self-harm codes alone.

Table 1.     Annual presentations to NSW Emergency Departments with suicide-related behaviour or ideation.

	Suicide-related behaviour	Suicide-related ideation	Total	% of total ED presentations1	Rate per 100 0002
	N (%)	N (%)	N (%)
Core ICD self-harm codes	4 519 (19%)	17 278 (61%)	21 797 (42%)	0.7%	313
+ additional ICD codes	10 087 (43%)	-(-)	10 087 (19%)	0.3%	145
+ presenting problem triage text	8 898 (38%)	11 039 (39%)	19 937 (38%)	0.7%	286
Total	23 504 (100%)	28 317 (100%)	51 822 (100%)	1.8%	745

Note: Defined using core International Classification of Diseases version 10 (ICD-10) codes, supplemented with additional ICD codes and keyword searching of presenting problem text (average presentations per year 2016–20. People aged 10 and over).
¹ Denominator = 2 926 025 average annual ED presentations from 2016–17 to 2020–21.^14
2 Denominator = 6 960 150 average NSW population aged 10 and over 2016–2020.¹⁹

2. Validation against inpatient diagnoses

Classification performance for the test data set is shown in Table 2. Including additional ICD-10 codes and problem text substantially increased detection sensitivity with little impact on specificity. Compared to a reference diagnosis of standard ICD-10 suicide-related behaviour codes (X60–84, Y87.0) in either ED or inpatient data, the sensitivity of those codes in ED data alone was 12.2%. This increased to 73.9% with the addition of extra ICD-10 codes and problem text. However, positive predictive value (PPV) was reduced with the addition of extra ICD-10 codes (to 75.4%) and problem text (to 63.6%).

Table 2.     Comparison of emergency department (ED) diagnosis methods with inpatient diagnoses of self-harm for episodes where ED presentation ended in hospital admission.

	Sensitivity	Specificity	PPV
Standard ICD-10 codes	12.2%	100%	100%
+ Extra ICD-10 codes	55.3%	99.7%	75.4%
+ Problem text	73.9%	99.4%	63.6%

Note: Reference diagnosis is any hospital or ED diagnosis code for self-harm [(International Classification of Diseases version 10 Australian modification(ICD-10-AM)) codes X60–X84] in the same episode of care.
PPV = positive predictive value

3. Characteristics of presentations

Age- and sex-specific per capita rates showed that suicide-related behaviour and ideation presentations were more common in females than males, with the highest rates in females aged 15–24 years (Figure 2). Approximately 57% of presentations arrived in the ED by ambulance. Fifty-four percent were triaged as urgent (Category 3), and 30% were triaged as Emergency or Resuscitation (Categories 1 and 2). Presentations classified as suicide-related behaviour were more likely to receive triage codes in categories 1 and 2 than presentations classified as suicidal-related ideation (37% compared to 24%). Forty percent of both groups were admitted to hospital following ED assessment.

Figure 2.    Suicide-related presentations to New South Wales Emergency Departments, as defined using the enhanced algorithm, by age group and sex (click figure to enlarge)

Discussion

In NSW Health ED data, standard ICD-10 codes for deliberate self-harm or suicidal ideation (X60–X84, Y87.0, R45.81) identified only 19% of suicide-related behaviour presentations and 42% of total suicide-related behaviour or ideation presentations, compared to an enhanced method which combined automated keyword searching of presenting problem text with additional ICD-10 codes for medications commonly used in overdose. The enhanced method increased the estimated annual number of ED presentations for suicide-related behaviour or ideation in NSW from 21 797 to 51 822. Validation against hospital admission diagnoses for a subset of presentations showed that the enhanced method had substantially increased sensitivity and high specificity. However, the inclusion of additional ICD-10 codes and text also reduced positive predictive value (PPV). This may reflect an increase in false positives due to the inclusion of the overdose codes of unknown intent and the general search terms such as “overdose” or “poisoning”. It may also be due to false negatives in inpatient reference diagnoses, where suicidal behaviour or ideas may also be under-recorded.²⁰

Is this increased estimate plausible? Standard ED data approaches substantially underestimate self-harm presentation rates.^4,5 From recent US²¹, Northern Ireland²² and Victorian²³ data, we derived ED self-harm presentation rates of 380 to 486 per 100 000 population, or 1.1% to 1.5% of total ED presentations. Our estimates using core ICD-10 codes alone are lower than this range (313 presentations per 100,000 population aged 10 and over, 0.7% of total ED presentations), while our enhanced estimates (745 per 100 000 population aged 10 and over, 1.8% of total ED presentations) are higher. This may reflect greater sensitivity due to the inclusion of keyword text searching in the enhanced method. A recent study in the state of Queensland⁸ used detailed file audit to assess text analysis via machine learning, reporting a higher estimate of self-harm presentations of 887 per 100 000 population, or 3.1% of ED presentations.

Ambulance data provide another possible source of external validation. In NSW in 2020–21, there were around 43 000 ambulance attendances per year for self-injury, suicidal ideation or suicide attempts.⁷ Of these, more than 90% (up to 38 000 per year) were transported to ED.²⁴ We found that core ICD-10 self-harm codes identified only around 10 000 self-harm presentations per year who arrived by ambulance. Estimates using the enhanced method are more consistent with NSW ambulance data.

The enhanced method described here has now been implemented in routine NSW Health reporting. Reporting of weekly data to NSW local health districts commenced in May 2020. Data have been shared regularly with other Australian governments as part of combined monitoring efforts during the COVID-19 pandemic. Feedback from health services has been that the enhanced method produces plausible rates for local facilities. Estimates derived using the enhanced method have appeared sensitive to changes in other measures of community distress or service activity during the pandemic. They have been consistent with rates and patterns from other Australian states which have used different methods. The enhanced method provides estimates for subgroups of age and sex, which are consistent with other sources²², and appear to have been sensitive to long-term trends and recent changes in self-harm in young people.³ Data using the method are now included in routine NSW Health public reporting.²⁵

Limitations and future development

The enhanced method requires ongoing development and refinement. It is limited by available data sources, relying on a single problem/diagnosis field and a brief (50-character) presenting issue text field. Compared to inpatient diagnoses, the estimated sensitivity of 73.9% suggests that the method continues to underestimate the true rate of self-harm-related presentations to NSW EDs.

Routine ED data and reporting struggle to capture the complex spectrum of ED self-harm-related presentations. We have focused on ED presentations where self-injury has occurred, or the person has thoughts or fears of self-harm. We combine these in routine reporting because suicidal behaviour and thoughts are predictors of future suicide risk²⁶ and require assessment and access to suitable follow-up care. Our method does not address the dimension of intent: it cannot distinguish whether self-injury occurred with suicidal intent, deliberately but without suicidal intent, or accidentally. In clinical assessment, a person’s intent is an important issue but is often mixed or unclear. We do not believe that a person’s intent can be reliably distinguished from the routine administrative data currently available.

Our method may include sources of bias. Information entered by ED clinicians, either as text or problem codes, may differ systematically for different groups based on age, sex, cultural background or self-harm method.⁵ Changes to coding standards, local training efforts or local differences in ED information systems may lead to apparent differences in rate between sites or over time.⁸ Our inclusion of additional ICD-10 codes for overdoses of unknown intent may cause selective over-identification of suicide-related presentations in groups such as older people, where accidental overdose may be more common due to cognitive or social factors.

We have used available data in the NSW Emergency Department Data Collection (EDDC). The number of hospitals participating in the EDDC has increased over time from approximately 52 in 1996–97 to 174 in 2020–21. The collection currently includes more than 98% of all NSW ED presentations, omitting only some very small rural sites. We have used earlier years (before 2016) for method development, but because of changing coverage over time, we have not commented on trends and only reported summary statistics on more recent (2016–2020) data where coverage is close to complete.

The current NSW method has been developed to reflect the available data and limitations of NSW Health’s ED systems and collections, and it may not be applicable for other health systems. NSW Health is currently implementing a new enterprise data warehouse (EDW) which will make additional diagnosis codes and more detailed triage assessment text available for analysis of ED presentations. We will revise and revalidate the current method as new data items become available. Approaches to natural language processing of ED free-text data are developing rapidly^10,11, and additional data will allow testing of some of these more complex approaches. However, we found that around half of the increased sensitivity for suicide-related behaviours was due to the simple strategy of including codes for poisoning by psychotropics, analgesics or other medications where accidental overdose is uncommon in adults: including these in self-harm detection algorithms may increase sensitivity without significant false positives.

Conclusion

Understanding and responding to trends in emergency department self-harm presentations is a critical priority. However, ED information systems consistently underestimate these presentations. Until recently, this prevented routine reporting on this issue in NSW, requiring reliance on important but incomplete measures such as hospital admissions. An enhanced method incorporating additional ICD-10 codes and simple rules-based processing of a presenting problem text field produces plausible estimates which are consistent with local service data and the known demographics of self-harm. Our goal has been to implement a ‘good enough’ measure to support policy, planning and health system management. Data sources and tools to support better ED reporting continue to evolve, and health systems should continue to refine and share their analysis and reporting methods.

Acknowledgements

GS is a member of National Suicide Monitoring System Expert Advisory Group, a member of the Beyond Blue Way Back Support National Advisory Committee, and a member of the Black Dog Institute Lifespan Research and Advisory Committee.

Peer review and provenance

Externally peer reviewed, not commissioned.

Copyright:

© 2023 Sara and Wu. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.