Going Beyond Administrative Data: Retrospective Evaluation of an Algorithm Using the Electronic Health Record to Help Identify Bleeding Events Among Hospitalized Medical Patients on Warfarin

To reliably assess quality, a standardized electronic approach is needed to identify bleeding events. The study aims were the following: (1) clinically validate an electronic health record–based algorithm for bleeding and (2) assess interrater results to determine validity and reliability. Data were analyzed before and after implementation of a pharmacist-managed warfarin protocol. Bleeding was based on ≥2 of 3 criteria: (1) diagnosis indicating bleeding, (2) lab value decrease suggesting bleeding, and (3) blood product use. All suspected bleeds (234) and a sample (58) not meeting criteria were compared with clinical review. There were 234 bleeding cases identified electronically. Reviewer agreement was 78.2% (κ = 0.565). Algorithm sensitivity was 93.9% and positive predictive value 46.2%. Algorithm identification was least accurate for those with only 2 criteria but good for those with all criteria. This study supports using multiple electronic criteria to identify bleeding events. However, cases having exactly 2 criteria may require manual review for validation.