Abstract
Objective: Results of routine maternal antenatal hepatitis B (HBV) screening have been recorded in the New South Wales (NSW) Perinatal Data Collection (PDC) since January 2011. We evaluated the accuracy of this reporting in 2012, the first year that comprehensive data were available, by linking the PDC to HBV notifications.
Methods: PDC records of mothers giving birth in 2012 were probabilistically linked to HBV notifications recorded in the NSW Notifiable Conditions Information Management System (NCIMS). Sensitivity and specificity of the PDC record of HBV status were determined using a linked HBV notification from the NCIMS database as the gold standard. Results were also examined according to health service (area health service, hospital level, public or private) and individual factors (maternal age, country of birth, Aboriginality, parity, timing of first antenatal visit).
Results: Among 99 510 records of women giving birth in NSW in 2012, positive HBV status was recorded for 0.69% of the women according to the PDC record and 0.90% from linked NCIMS records. The overall sensitivity of the HBV status variable in the PDC data was 65.5% (95% confidence interval [CI] 62.4, 68.7) and positive predictive value was 85.3% (95% CI 82.6, 87.9). In general, the low prevalence of HBV meant we had limited statistical power to assess differences between health service factors and maternal factors; however, sensitivity was significantly lower in PDC data for HBV in Australian-born non-Aboriginal women (37.0%; 95% CI 27.5, 46.7) than in overseas-born women (69.9%; 95% CI 66.6, 73.1; p < 0.001).
Conclusions: PDC records of HBV status for women giving birth in 2012 had high specificity but poor sensitivity. Sensitivity varied across area health services and levels of maternal services, and by various maternal factors. Because the results of maternal HBV screening can be used to monitor HBV prevalence in adults, analysis of the PDC records in subsequent years is necessary to track whether sensitivity improves over time.
Full text
Introduction
Chronic hepatitis B (HBV) infection contributes to a significant burden of disease worldwide through its causal role in liver cirrhosis, liver failure and cancer. The majority of chronic infections are acquired through maternal transmission1, but this risk can be virtually eliminated with administration of HBV immunoglobulin and HBV vaccine within 48 hours of birth.2 Therefore, antenatal screening for chronic HBV is an important public health measure. Routine antenatal screening for HBV in pregnant women using HBV surface antigen (HBsAg) has been in place since 19873 in New South Wales (NSW), Australia. The NSW Health policy directive, effective from 1 January 2011, on Perinatal Data Collection (PDC) reporting requires a birthing mother’s HBsAg status to be submitted as part of the pregnancy details.4 According to the NSW HBV Strategy, 99% of pregnant women were screened for HBV as part of routine antenatal care.5 As previously shown, information on HBsAg collected as part of antenatal screening can be used to assess and monitor HBV prevalence in the population and the impact of vaccination programs6,7, and therefore it is important to assess how accurate these data are in the PDC records.
By linking maternal records from the PDC to HBV notifications from the NSW Notifiable Conditions Information Management System (NCIMS), we can compare the accuracy of HBV status reporting in the PDC.
Methods
Data sources
The NSW PDC is a statutory record of all births in NSW of at least 400 grams birthweight or at least 20 weeks gestation, along with information on maternal health, including HBsAg status, demographics, pregnancy details and outcomes. Data are entered by the attending midwife or doctor either at the time of booking for delivery or around the time of birth for unbooked deliveries.
The NSW NCIMS is a statutory population-based surveillance system with records of all notifiable conditions under the Public Health Act 2010.8 All laboratory-detected HBV infections are notifiable to the NCIMS. The records include personal identifying details, the classification of the report into either newly acquired HBV or HBV of unspecified duration based on standard definitions9, date of notification and estimated onset date. The NCIMS database available for this study included all notifications from 1 January 1994.
Linkage
We used probabilistic matching of personal identifying details to link records from the PDC and the NCIMS. The linkage was conducted by the NSW Centre for Health Record Linkage (CHeReL), independent of the study investigators, and linked, de-identified data were provided to the investigators for analysis. Full details of the linkage process can be found at www.cherel.org.au/how-record-linkage-works. CHeReL’s reported linkage false positive and false negative rate is <0.5%.
Analysis
Although collection of the HBsAg status of birthing women started in the PDC from 1 January 2011, it was known to be incomplete for that year. Thus, this study was limited to PDC records between 1 January 2012 and 31 December 2012, the second year of the collection. A woman was defined as having an HBV infection based on NCIMS records if she had at least one linked record of an HBV notification from the NCIMS database with a notification date before the birthing date on the PDC record. Women with no linked HBV notifications were assumed not to have HBV infection.
We compared the proportion of women with a positive HBV record from each data source using a chi-square test. The NCIMS notifications linked to the PDC were taken as the gold standard for HBV infection, and the accuracy of the PDC data entry for HBsAg status was compared to the gold standard. We calculated sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). The sensitivity and specificity for the PDC records were also estimated according to area health service (AHS), the hospital maternity service level (levels 1–4: local, district, metropolitan and regional referral metropolitan services for moderate-risk pregnancies; level 5: regional referral metropolitan services for high-risk pregnancies; level 6: tertiary specialist obstetric services with neonatal intensive care and private hospitals), public or private hospital, maternal age, maternal country of birth, maternal Aboriginal and Torres Strait Islander status (hereafter referred to as Aboriginal), maternal parity (0, 1, ≥2), and timing of first antenatal assessment (before or after 20 weeks). For Aboriginal status, we improved the reporting in the PDC using linkage, as done previously.10 Records with no country of birth recorded were included in the non-Australian-born category.
The study was approved by the NSW Population and Health Services Research Ethics Committee (reference number 2009/11/193) and the Aboriginal Health & Medical Research Council Human Research Ethics Committee (r841/12).
Results
Between January and December 2012, there were 99 510 records of mothers giving birth in the PDC. Table 1 shows that the proportion of women with a record of a positive HBsAg test according to their PDC record (685/99 510, or 0.69%) was significantly lower than the proportion of women linking to a HBV notification in the NCIMS (891/99 510, or 0.90%; p < 0.001). Using the linked NCIMS notifications as the gold standard for HBV infections, the overall sensitivity of the PDC data was 65.5% (95% confidence interval [CI] 62.4, 68.7) and specificity was 99.9% (95% CI 99.9, 99.9). The PPV of the PDC data was 85.3% (95% CI 82.6, 87.9) and NPV was 99.7% (95% CI 99.7, 99.7) (Table 2).
Table 1. Comparison of positive HBsAg status recorded in the NSW Perinatal Data Collection in 2012 with linked hepatitis B notifications from the NCIMS
| NSW Perinatal Data Collection record | Linked notification (NCIMS) | ||
| HBV positive | HBV negative | Total | |
| HBV positive | 584 | 101 | 685 |
| HBV negative | 307 | 98 518 | 98 825 |
| Total | 891a | 98 619 | 99 510 |
HBsAg = hepatitis B surface antigen; HBV = hepatitis B; NCIMS = Notifiable Conditions Information Management System; NSW = New South Wales
a 17 of the 891 linked NCIMS hepatitis B notifications were for newly acquired hepatitis B
Comparisons between the PDC data and linked NCIMS notifications by AHS and by other hospital, sociodemographic and birth characteristics are shown in Tables 2 and 3. There were variations between AHSs in both the estimated HBV prevalence and the measures of agreement between records (see Table 2). For example, Northern Sydney and Central Coast AHS had the highest sensitivity of 81.0% (95% CI 73.4, 88.5) with moderate HBV prevalence (0.66%; 95% CI 0.53, 0.78), while Greater Western AHS had both the lowest sensitivity of 33.3% (95% CI 2.5, 64.1) and the lowest HBV prevalence (0.23%; 95% CI 0.08, 0.38). The estimate of HBV prevalence using linked NCIMS records was consistently higher than that from the PDC in all AHSs. The biggest absolute difference in prevalence was in the Sydney West AHS (0.42%). Specificity and NPV were high in all AHSs.
Of the other characteristics examined (see Table 3), there was no significant difference in estimated sensitivity between hospital maternity service level (levels 1–4 versus levels 5–6, p = 0.3), private and public hospitals (p = 0.4), maternal age (<29 versus ≥29 years, p = 0.1) and parity (0 versus ≥1, p = 0.05). However, there were differences in sensitivity when comparing records of women who presented on time for antenatal care (at or before 20 weeks; sensitivity 64.4%; 95% CI 60.9, 67.9) with those who presented late for antenatal care (sensitivity 74.0%; 95% CI 67.1, 81.0; p = 0.03). Sensitivity also differed by country of birth and was highest in overseas-born women (69.9%; 95% CI 66.6, 73.1), followed by Aboriginal women (48.4%; 95% CI 30.8, 66.0), then Australian-born non-Aboriginal women (37.0%; 95% CI 27.5, 46.7), with a highly significant difference in reporting between overseas-born women and Australian-born non-Aboriginal women (p < 0.0001).
Table 2. Sensitivity, specificity, PPV and NPV of hepatitis B recorded in the PDC, using linked hepatitis B notifications as the gold standard, by area health service, 2012
| Area health service | No. birthing records | HBV prevalence using NCIMS, % (95% CI) | HBV prevalence using PDC alone, % (95% CI) | Sensitivity, % (95% CI) | Specificity, % (95% CI) | PPV, % (95% CI) | NPV, % (95% CI) |
| Sydney South West | 19 147 | 1.55 (1.38, 1.73) | 1.26 (1.10, 1.42) | 71.7 (66.6, 76.8) | 99.9 (99.8, 99.9) | 88.4 (84.3, 92.4) | 99.6 (99.5, 99.7) |
| Sydney West | 20 073 | 1.33 (1.17, 1.49) | 0.91 (0.78, 1.04) | 60.7 (54.8, 66.5) | 99.9 (99.8, 99.9) | 88.5 (83.9, 93.1) | 99.5 (99.4, 99.6) |
| South Eastern Sydney and Illawarra | 18 736 | 0.78 (0.66, 0.91) | 0.61 (0.50, 0.73) | 59.2 (51.2, 67.1) | 99.8 (99.8, 99.9) | 75.7 (67.8, 83.5) | 99.7 (99.6, 99.8) |
| Northern Sydney and Central Coast | 15 972 | 0.66 (0.53, 0.78) | 0.61 (0.49, 0.73) | 81.0 (73.4, 88.5) | 99.9 (99.9, 100.0) | 87.6 (81.1, 94.2) | 99.9 (99.8, 99.9) |
| Greater Southern | 3 927 | 0.41 (0.21, 0.61) | 0.25 (0.10, 0.41) | 50.0 (25.5, 74.5) | 99.9 (99.9, 100.0) | 80.0 (55.2, 100.0) | 99.8 (99.7, 99.9) |
| Hunter New England | 11 714 | 0.27 (0.18, 0.37) | 0.20 (0.12, 0.29) | 59.4 (42.4, 76.4) | 100.0 (99.9, 100.0) | 79.2 (62.9, 95.4) | 99.9 (99.8, 99.9) |
| North Coast | 5 908 | 0.30 (0.16, 0.45) | 0.19 (0.08, 0.30) | 38.9 (16.4, 61.4) | 99.9 (99.9, 100.0) | 63.6 (35.2, 92.1) | 99.8 (99.7, 99.9) |
| Greater Western | 3 910 | 0.23 (0.08, 0.38) | 0.10 (0.00, 0.20) | 33.3 (2.5, 64.1) | 100.0 (99.9, 100.0) | 75.0 (32.6, 100.0) | 99.8 (99.7, 100.0) |
| Totala | 99 510 | 0.90 (0.84, 0.95) | 0.69 (0.64, 0.74) | 65.5 (62.4, 68.7) | 99.9 (99.9, 99.9) | 85.3 (82.6, 87.9) | 99.7 (99.7, 99.7) |
CI = confidence interval; HBV = hepatitis B; NCIMS = Notifiable Conditions Information Management System; NPV = negative predictive value; PDC = NSW Perinatal Data Collection; PPV = positive predictive value
a The total includes an additional 123 records that did not have valid area health service information
Table 3. Sensitivity, specificity, PPV and NPV of hepatitis B recorded in the PDC, using linked hepatitis B notifications as the gold standard, by hospital maternity service level and maternal factors
| Factor | Category | No. maternal birth records | NCIMS prevalence, % (95% CI) | Sensitivity, % (95% CI) | Specificity, % (95% CI) | PPV, % (95% CI) | NPV, % (95% CI) |
| Hospital maternity service level in public hospitalsa | 1–4 | 29 766 | 1.03 (0.92, 1.15) | 68.7 (63.5, 73.9) | 99.9 (99.9, 100.0) | 92.1 (88.7, 95.6) | 99.7 (99.6, 99.7) |
| 5 | 16 747 | 0.66 (0.53, 0.78) | 61.8 (52.7, 70.9) | 99.9 (99.8, 99.9) | 75.6 (66.7, 84.4) | 99.7 (99.7, 99.8) | |
| 6 | 28 801 | 1.09 (0.97, 1.21) | 65.0 (59.7, 70.2) | 99.9 (99.8, 99.9) | 86.1 (81.7, 90.5) | 99.6 (99.5, 99.7) | |
| Public vs private | Public | 75 336 | 0.97 (0.90, 1.04) | 66.1 (62.6, 69.5) | 99.9 (99.9, 99.9) | 86.9 (84.1, 89.7) | 99.7 (99.6, 99.7) |
| Private | 24 174 | 0.66 (0.56, 0.76) | 63.1 (55.6, 70.6) | 99.9 (99.8, 99.9) | 78.3 (71.2, 85.4) | 99.8 (99.7, 99.8) | |
| Parityb | 0 | 43 741 | 0.84 (0.76, 0.93) | 69.4 (64.7, 74.1) | 99.9 (99.8, 99.9) | 81.0 (76.7, 85.3) | 99.7 (99.7, 99.8) |
| 1 | 32 963 | 0.83 (0.73, 0.92) | 68.0 (62.5, 73.6) | 99.9 (99.9, 99.9) | 87.3 (82.8, 91.8) | 99.7 (99.7, 99.8) | |
| 2+ | 22 770 | 1.10 (0.96, 1.23) | 57.2 (51.1, 63.3) | 99.9 (99.9, 100.0) | 91.1 (86.6, 95.5) | 99.5 (99.4, 99.6) | |
| Gestation at first antenatal assessment | ≤20 weeks | 84 939 | 0.85 (0.79, 0.92) | 64.4 (60.9, 67.9) | 99.9 (99.9, 99.9) | 85.5 (82.6, 88.5) | 99.7 (99.7, 99.7) |
| >20 weeks | 11 992 | 1.28 (1.08, 1.49) | 74.0 (67.1, 81.0) | 99.8 (99.8, 99.9) | 85.1 (79.0, 91.1) | 99.7 (99.6, 99.8) | |
| Unknown | 2 579 | 0.47 (0.20, 0.73) | 25.0 (0.5, 49.5) | 99.9 (99.8, 100.0) | 60.0 (17.1, 100.0) | 99.7 (99.4, 99.9) | |
| Maternal agec | <20 | 3 185 | 0.19 (0.04, 0.34) | 66.7 (28.9, 100.0) | 100.0 (100.0, 100.0) | 100.0 (100.0, 100.0) | 99.9 (99.9, 100.0) |
| 20–29 | 39 933 | 0.84 (0.75, 0.93) | 68.0 (63.0, 72.9) | 99.9 (99.9, 99.9) | 84.1 (80.2, 88.8) | 99.7 (99.7, 99.8) | |
| 30–39 | 51 693 | 0.97 (0.88, 1.05) | 64.3 (60.1, 68.5) | 99.9 (99.9, 99.9) | 85.6 (82.1, 89.2) | 99.7 (99.6, 99.7) | |
| ≥40 | 4 684 | 1.00 (0.72, 1.29) | 61.7 (47.8, 75.6) | 99.9 (99.8, 100.0) | 85.3 (73.4, 97.2) | 99.6 (99.4, 99.8) | |
| Country of birth and Aboriginality | Australian-born, non-Aboriginal | 61 264 | 0.16 (0.13, 0.19) | 37.0 (27.5, 46.7) | 100.0 (100.0, 100.0) | 64.3 (51.7, 76.8) | 99.9 (99.9, 99.9) |
| Australian-born, Aboriginal | 3 474 | 0.89 (0.58, 1.21) | 48.4 (30.8, 66.0) | 99.9 (99.8, 100.0) | 83.3 (66.1, 100.0) | 99.5 (99.3, 99.8) | |
| Overseas-born | 34 772 | 2.19 (2.04, 2.35) | 69.9 (66.6, 73.1) | 99.8 (99.7, 99.8) | 87.2 (84.6, 89.9) | 99.3 (99.2, 99.4) |
CI = confidence interval; NCIMS = Notifiable Conditions Information Management System; NPV = negative predictive value; PDC = New South Wales Perinatal Data Collection; PPV = positive predictive value
a Excludes 22 records with no information on hospital maternity service level
b Excludes 36 records with no information on parity
c Excludes 15 records with no information on maternal date of birth/maternal age
Discussion
This study is the first analysis of the accuracy of HBsAg status reporting from the NSW PDC since the start of routine reporting in January 2011. The level of agreement of HBsAg status that we found appeared to be lower than that from previous validation studies on other variables included in PDC records. A 1998 validation study showed high levels of accuracy and reliability compared with medical records for dichotomised perinatal variables such as preterm birth, low and high birthweight, Apgar scores, perineal trauma, regional analgesia and stillbirth (Kappa 0.95–1.00, sensitivities 94.7–100.0%).11 The lower estimate of HBV prevalence from the PDC compared with prevalence using linked NCIMS records is consistent with previous analyses of PDC data for rarer events. For example, a study of births from 2001 to 2011 in NSW showed that PDC records reported lower rates of severe perineal injuries than linked records from the NSW Admitted Patient Data Collection. However, the discrepancy improved with time, with lower agreement before 2006 (Kappa 0.78; 95% CI 0.78, 0.79) than after 2006 (Kappa 0.89; 95% CI 0.89, 0.89).12 The level of agreement was similar for private and public hospitals, in contrast to a study of PDC versus hospital records of postpartum haemorrhage requiring transfusion, which found poorer agreement for private hospitals.13
Because the collection of HBsAg status in the PDC was only in its second year, and 2012 was the first year we were able to conduct such analyses, we were unable to assess the trends in accuracy over time. However, we hypothesise that, as the data collection practice becomes more ingrained in the health system, sensitivity will improve, as it did for the perineal trauma12 and postpartum haemorrhage data13, and to the level of accuracy and reliability of other dichotomised variables suggested by previous validation studies.11 If reporting of maternal HBV status in the PDC data becomes more reliable, it could be used to determine HBV prevalence across different AHSs and population groups, as well as to monitor prevalence changes over time, as has been done in previous studies using linked data.6,7
Variations in accuracy across AHSs, and important sociodemographic factors such as country of birth also require further analysis over time to determine if there are consistent factors that affect the accuracy of reporting. The better sensitivity in women presenting late to antenatal clinics and those born overseas suggest greater vigilance in HBV-status recording in populations considered to be at higher risk of HBV.
In this study, we were limited to using linked NCIMS records as the gold standard because we did not have access to medical records and patient pathology records, which would have provided us with the most accurate information on the patient’s HBsAg status and thus be the ultimate gold standard. It is possible that some linked NCIMS notifications of HBV of unspecified duration could have been acute HBV infections that cleared by the time the woman became pregnant. Thus, the woman may have tested negative during her pregnancy and this would be recorded as such on the PDC. It is also possible that the woman tested positive after her characteristics were entered into the PDC and that her record was not updated. The study is also limited by false positive and false negative linkages that may have affected the accuracy of the linked NCIMS records.
Conclusion
Accurate perinatal recording of HBV infection status based on routine screening provides a simple and cost-effective means of monitoring population patterns and trends in HBV prevalence in birthing women, and could be used in Australia6,7 and internationally.14 This study showed that HBV prevalence as reported on NSW PDC records for birthing women in 2012, its second year of reporting, was lower than the estimate obtained by linking PDC records to NCIMS HBV notifications. There was also some variability in measures of agreement according to AHS, timing of first antenatal assessment, country of birth and Aboriginal status. Further assessment of the accuracy of PDC reporting over time will determine whether it can be used for monitoring prevalence in NSW, and whether similar reporting systems should be implemented elsewhere in Australia.
Acknowledgements
We thank the NSW Ministry of Health for providing the data for this study, and CHeReL for conducting the record linkage. We thank Lee Taylor and Tim Harrold from the NSW Ministry of Health for their advice on this manuscript. BL and JK are supported by National Health and Medical Research Council Fellowships.
Copyright:
© 2017 Deng et al. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.
References
- 1. World Health Organisation. Geneva: WHO; 2017. Hepatitis; 2002 [cited 2015 Nov 25]; [about 3 screens]. Available from: www.who.int/hepatitis/en/
- 2. Australian Technical Advisory Group on Immunisation. The Australian immunisation handbook 10th edition: Chapter 4.5 hepatitis B. Canberra: Australian Government Department of Health; 2016 [cited 2016 Nov 1]. Available from: www.immunise.health.gov.au/internet/immunise/publishing.nsf/Content/Handbook10-home~handbook10part4~handbook10-4-5#4-5-6
- 3. Jayaram S. Neonatal Hepatitis B vaccination program. N S W Public Health Bull. 1993;3(30):27–8. Article
- 4. NSW Ministry of Health. Perinatal data collection (PDC) reporting and submission requirements. Sydney: NSW Health; 2010 [cited 2015 Nov 10]. Available from: www0.health.nsw.gov.au/archive/policies/pd/2010/pdf/PD2010_072.pdf
- 5. NSW Health. NSW hepatitis B strategy 2014–2020. Sydney: NSW Ministry of Health; 2014 [cited 2015 Sep 10]. Available from: www.health.nsw.gov.au/hepatitis/Publications/HepatitisBStrategy.pdf
- 6. Liu B, Guthridge S, Li SQ, Markey P, Krause V, McIntyre P, et al. The end of the Australia antigen? An ecological study of the impact of universal newborn hepatitis B vaccination two decades on. Vaccine. 2012;30(50):7309–14. CrossRef | PubMed
- 7. Reekie J, Gidding HF, Kaldor JM, Liu B. Country of birth and other factors associated with hepatitis B prevalence in a population with high levels of immigration. J Gastroenterol Hepatol. 2013;28(9):1539–44. CrossRef | PubMed
- 8. NSW Government. NSW Public Health Act 2010 No 127. Sydney: New South Wales Government; 2017 [cited 2015 Nov 25]. Available from: www.legislation.nsw.gov.au/#/view/act/2010/127/full
- 9. Australian Government Department of Health. Surveillance case definitions for the Australian National Notifiable Diseases Surveillance System. Canberra: Australian Government Department of Health; 2016 [cited 2016 Feb 20]. Available from: www.health.gov.au/internet/main/publishing.nsf/Content/cdna-casedefinitions.htm/$File/consolidated-case-definitions.pdf
- 10. Taylor LK, Bentley J, Hunt J, Madden R, McKeown S, Brandt P, Baker D. Enhanced reporting of deaths among Aboriginal and Torres Strait Islander peoples using linked administrative health datasets. BMC Med Res Methodol. 2012;12:91. CrossRef | PubMed
- 11. Ampt AJ, Ford JB, Taylor LK, Roberts CL. Are pregnancy outcomes associated with risk factor reporting in routinely collected perinatal data? N S W Public Health Bull. 2013;24(2):65–9. CrossRef | PubMed
- 12. Ampt AJ, Ford JB. Ascertaining severe perineal trauma and associated risk factors by comparing birth data with multiple sources. Public Health Res Pract. 2015;25(4):e2541544. CrossRef | PubMed
- 13. Patterson JA, Roberts CL, Taylor LK, Ford JB. Reporting postpartum haemorrhage with transfusion: a comparison of NSW birth and hospital data. N S W Public Health Bull. 2014;24:153–8. CrossRef | PubMed
- 14. Lin CC, Hsieh HS, Huang YJ, Huang YL, Ku MK, Hung HC. Hepatitis B virus infection among pregnant women in Taiwan: comparison between women born in Taiwan and other southeast countries. BMC Public Health. 2008;8:49. CrossRef | PubMed
