Reliability of causality assessment for drug, herbal and dietary supplement hepatotoxicity in the Drug-Induced Liver Injury Network (DILIN)

Background & Aims: Because of the lack of objective tests to diagnose drug-induced liver injury (DILI), causality assessment is a matter of debate. Expert opinion is often used in research and industry, but its test-retest reliability is unknown. To determine the test-retest reliability of the expert opinion process used by the Drug-Induced Liver Injury Network (DILIN). Methods: Three DILIN hepatologists adjudicate suspected hepatotoxicity cases to one of five categories representing levels of likelihood of DILI. Adjudication is based on retrospective assessment of gathered case data that include prospective follow-up information. One hundred randomly selected DILIN cases were re-assessed using the same processes for initial assessment but by three different reviewers in 92% of cases. Results: The median time between assessments was 938 days (range 140-2352). Thirty-one cases involved >1 agent. Weighted kappa statistics for overall case and individual agent category agreement were 0.60 (95% CI: 0.50-0.71) and 0.60 (0.52-0.68) respectively. Overall case adjudications were within one category of each other 93% of the time, while 5% differed by two categories and 2% differed by three categories. Fourteen per cent crossed the 50% threshold of likelihood owing to competing diagnoses or atypical timing between drug exposure and injury. Conclusions: The DILIN expert opinion causality assessment method has moderate interobserver reliability but very good agreement within one category. A small but important proportion of cases could not be reliably diagnosed as ≥50% likely to be DILI.