Building a best-in-class automated de-identification tool for electronic health records through ensemble learning

Karthik Murugadoss, Ajit Rajasekharan, Bradley Malin, Vineet Agarwal, Sairam Bade, Jeff R. Anderson, Jason L. Ross, William A. Faubion, John D. Halamka, Venky Soundararajan, Sankar Ardhanari

Research output: Contribution to journalArticlepeer-review


The presence of personally identifiable information (PII) in natural language portions of electronic health records (EHRs) constrains their broad reuse. Despite continuous improvements in automated detection of PII, residual identifiers require manual validation and correction. Here, we describe an automated de-identification system that employs an ensemble architecture, incorporating attention-based deep-learning models and rule-based methods, supported by heuristics for detecting PII in EHR data. Detected identifiers are then transformed into plausible, though fictional, surrogates to further obfuscate any leaked identifier. Our approach outperforms existing tools, with a recall of 0.992 and precision of 0.979 on the i2b2 2014 dataset and a recall of 0.994 and precision of 0.967 on a dataset of 10,000 notes from the Mayo Clinic. The de-identification system presented here enables the generation of de-identified patient data at the scale required for modern machine-learning applications to help accelerate medical discoveries.

Original languageEnglish (US)
Article number100255
Issue number6
StatePublished - Jun 11 2021


  • DSML 4: Production: Data science output is validated, understood, and regularly used for multiple domains/platforms
  • anonymization
  • de-identification
  • ensemble
  • mayo
  • nference
  • obfuscation

ASJC Scopus subject areas

  • Decision Sciences(all)


Dive into the research topics of 'Building a best-in-class automated de-identification tool for electronic health records through ensemble learning'. Together they form a unique fingerprint.

Cite this