Reliability among central readers in the evaluation of endoscopic disease activity in pouchitis

Mark A. Samaan, Bo Shen, Mahmoud H. Mosli, Guangyong Zou, William J. Sandborn, Lisa M. Shackelton, Sigrid Nelson, Larry Stitt, Stuart Bloom, Darrell Spencer Pardi, Paolo Gionchetti, James Lindsay, Simon Travis, Ailsa Hart, Mark S. Silverberg, Brian G. Feagan, Geert R. D'Haens, Vipul Jairath

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Background and Aims: Pouchitis is a common adverse event after proctocolectomy with ileal pouch anal anastomosis for ulcerative colitis. Evaluation of pouchitis disease activity and response to treatment requires use of validated indices. We assessed the reliability of items evaluating endoscopic pouchitis disease activity. Methods: Twelve panelists used a modified RAND appropriateness methodology to rate the appropriateness of items evaluating endoscopic pouchitis disease activity derived from a systematic review and also identified additional potential endoscopic items based on expert opinion. Four central readers then evaluated 50 pouchoscopy videos in triplicate, in random order. Intra- and inter-rater reliability for each item was assessed by calculating and comparing intraclass correlation coefficients (ICCs). A Delphi process identified common sources of disagreement among the readers. Results: Ten existing endoscopic items were identified from the systematic review and an additional 7 exploratory items from the panelists. ICCs for inter-rater reliability were highest for the existing item of pouch ulceration (.72; 95% confidence interval [CI],.60-.82) and for the exploratory item of ulcerated surface in the pouch body (.67; 95% CI,.53-.75). Inter-rater reliability for all other existing and exploratory items was “moderate” (ICC <.60). The item “ulcerated surface in the pouch body” demonstrated the best correlation with a global evaluation of lesion severity (r =.80; 95% CI,.73-.85). Conclusion: Substantial reliability was observed only for the endoscopic items of ulceration and ulcerated surface in the pouch body. Future studies should assess responsiveness to treatment in the next stage toward development of an endoscopic pouchitis disease activity index.

Original languageEnglish (US)
Pages (from-to)360-369.e2
JournalGastrointestinal endoscopy
Volume88
Issue number2
DOIs
StatePublished - Aug 1 2018

Fingerprint

Pouchitis
Confidence Intervals
Colonic Pouches
Expert Testimony
Ulcerative Colitis

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Gastroenterology

Cite this

Samaan, M. A., Shen, B., Mosli, M. H., Zou, G., Sandborn, W. J., Shackelton, L. M., ... Jairath, V. (2018). Reliability among central readers in the evaluation of endoscopic disease activity in pouchitis. Gastrointestinal endoscopy, 88(2), 360-369.e2. https://doi.org/10.1016/j.gie.2018.04.2330

Reliability among central readers in the evaluation of endoscopic disease activity in pouchitis. / Samaan, Mark A.; Shen, Bo; Mosli, Mahmoud H.; Zou, Guangyong; Sandborn, William J.; Shackelton, Lisa M.; Nelson, Sigrid; Stitt, Larry; Bloom, Stuart; Pardi, Darrell Spencer; Gionchetti, Paolo; Lindsay, James; Travis, Simon; Hart, Ailsa; Silverberg, Mark S.; Feagan, Brian G.; D'Haens, Geert R.; Jairath, Vipul.

In: Gastrointestinal endoscopy, Vol. 88, No. 2, 01.08.2018, p. 360-369.e2.

Research output: Contribution to journalArticle

Samaan, MA, Shen, B, Mosli, MH, Zou, G, Sandborn, WJ, Shackelton, LM, Nelson, S, Stitt, L, Bloom, S, Pardi, DS, Gionchetti, P, Lindsay, J, Travis, S, Hart, A, Silverberg, MS, Feagan, BG, D'Haens, GR & Jairath, V 2018, 'Reliability among central readers in the evaluation of endoscopic disease activity in pouchitis', Gastrointestinal endoscopy, vol. 88, no. 2, pp. 360-369.e2. https://doi.org/10.1016/j.gie.2018.04.2330
Samaan, Mark A. ; Shen, Bo ; Mosli, Mahmoud H. ; Zou, Guangyong ; Sandborn, William J. ; Shackelton, Lisa M. ; Nelson, Sigrid ; Stitt, Larry ; Bloom, Stuart ; Pardi, Darrell Spencer ; Gionchetti, Paolo ; Lindsay, James ; Travis, Simon ; Hart, Ailsa ; Silverberg, Mark S. ; Feagan, Brian G. ; D'Haens, Geert R. ; Jairath, Vipul. / Reliability among central readers in the evaluation of endoscopic disease activity in pouchitis. In: Gastrointestinal endoscopy. 2018 ; Vol. 88, No. 2. pp. 360-369.e2.
@article{7f1fe8f25fd448c19fa0d8d999877277,
title = "Reliability among central readers in the evaluation of endoscopic disease activity in pouchitis",
abstract = "Background and Aims: Pouchitis is a common adverse event after proctocolectomy with ileal pouch anal anastomosis for ulcerative colitis. Evaluation of pouchitis disease activity and response to treatment requires use of validated indices. We assessed the reliability of items evaluating endoscopic pouchitis disease activity. Methods: Twelve panelists used a modified RAND appropriateness methodology to rate the appropriateness of items evaluating endoscopic pouchitis disease activity derived from a systematic review and also identified additional potential endoscopic items based on expert opinion. Four central readers then evaluated 50 pouchoscopy videos in triplicate, in random order. Intra- and inter-rater reliability for each item was assessed by calculating and comparing intraclass correlation coefficients (ICCs). A Delphi process identified common sources of disagreement among the readers. Results: Ten existing endoscopic items were identified from the systematic review and an additional 7 exploratory items from the panelists. ICCs for inter-rater reliability were highest for the existing item of pouch ulceration (.72; 95{\%} confidence interval [CI],.60-.82) and for the exploratory item of ulcerated surface in the pouch body (.67; 95{\%} CI,.53-.75). Inter-rater reliability for all other existing and exploratory items was “moderate” (ICC <.60). The item “ulcerated surface in the pouch body” demonstrated the best correlation with a global evaluation of lesion severity (r =.80; 95{\%} CI,.73-.85). Conclusion: Substantial reliability was observed only for the endoscopic items of ulceration and ulcerated surface in the pouch body. Future studies should assess responsiveness to treatment in the next stage toward development of an endoscopic pouchitis disease activity index.",
author = "Samaan, {Mark A.} and Bo Shen and Mosli, {Mahmoud H.} and Guangyong Zou and Sandborn, {William J.} and Shackelton, {Lisa M.} and Sigrid Nelson and Larry Stitt and Stuart Bloom and Pardi, {Darrell Spencer} and Paolo Gionchetti and James Lindsay and Simon Travis and Ailsa Hart and Silverberg, {Mark S.} and Feagan, {Brian G.} and D'Haens, {Geert R.} and Vipul Jairath",
year = "2018",
month = "8",
day = "1",
doi = "10.1016/j.gie.2018.04.2330",
language = "English (US)",
volume = "88",
pages = "360--369.e2",
journal = "Gastrointestinal Endoscopy",
issn = "0016-5107",
publisher = "Mosby Inc.",
number = "2",

}

TY - JOUR

T1 - Reliability among central readers in the evaluation of endoscopic disease activity in pouchitis

AU - Samaan, Mark A.

AU - Shen, Bo

AU - Mosli, Mahmoud H.

AU - Zou, Guangyong

AU - Sandborn, William J.

AU - Shackelton, Lisa M.

AU - Nelson, Sigrid

AU - Stitt, Larry

AU - Bloom, Stuart

AU - Pardi, Darrell Spencer

AU - Gionchetti, Paolo

AU - Lindsay, James

AU - Travis, Simon

AU - Hart, Ailsa

AU - Silverberg, Mark S.

AU - Feagan, Brian G.

AU - D'Haens, Geert R.

AU - Jairath, Vipul

PY - 2018/8/1

Y1 - 2018/8/1

N2 - Background and Aims: Pouchitis is a common adverse event after proctocolectomy with ileal pouch anal anastomosis for ulcerative colitis. Evaluation of pouchitis disease activity and response to treatment requires use of validated indices. We assessed the reliability of items evaluating endoscopic pouchitis disease activity. Methods: Twelve panelists used a modified RAND appropriateness methodology to rate the appropriateness of items evaluating endoscopic pouchitis disease activity derived from a systematic review and also identified additional potential endoscopic items based on expert opinion. Four central readers then evaluated 50 pouchoscopy videos in triplicate, in random order. Intra- and inter-rater reliability for each item was assessed by calculating and comparing intraclass correlation coefficients (ICCs). A Delphi process identified common sources of disagreement among the readers. Results: Ten existing endoscopic items were identified from the systematic review and an additional 7 exploratory items from the panelists. ICCs for inter-rater reliability were highest for the existing item of pouch ulceration (.72; 95% confidence interval [CI],.60-.82) and for the exploratory item of ulcerated surface in the pouch body (.67; 95% CI,.53-.75). Inter-rater reliability for all other existing and exploratory items was “moderate” (ICC <.60). The item “ulcerated surface in the pouch body” demonstrated the best correlation with a global evaluation of lesion severity (r =.80; 95% CI,.73-.85). Conclusion: Substantial reliability was observed only for the endoscopic items of ulceration and ulcerated surface in the pouch body. Future studies should assess responsiveness to treatment in the next stage toward development of an endoscopic pouchitis disease activity index.

AB - Background and Aims: Pouchitis is a common adverse event after proctocolectomy with ileal pouch anal anastomosis for ulcerative colitis. Evaluation of pouchitis disease activity and response to treatment requires use of validated indices. We assessed the reliability of items evaluating endoscopic pouchitis disease activity. Methods: Twelve panelists used a modified RAND appropriateness methodology to rate the appropriateness of items evaluating endoscopic pouchitis disease activity derived from a systematic review and also identified additional potential endoscopic items based on expert opinion. Four central readers then evaluated 50 pouchoscopy videos in triplicate, in random order. Intra- and inter-rater reliability for each item was assessed by calculating and comparing intraclass correlation coefficients (ICCs). A Delphi process identified common sources of disagreement among the readers. Results: Ten existing endoscopic items were identified from the systematic review and an additional 7 exploratory items from the panelists. ICCs for inter-rater reliability were highest for the existing item of pouch ulceration (.72; 95% confidence interval [CI],.60-.82) and for the exploratory item of ulcerated surface in the pouch body (.67; 95% CI,.53-.75). Inter-rater reliability for all other existing and exploratory items was “moderate” (ICC <.60). The item “ulcerated surface in the pouch body” demonstrated the best correlation with a global evaluation of lesion severity (r =.80; 95% CI,.73-.85). Conclusion: Substantial reliability was observed only for the endoscopic items of ulceration and ulcerated surface in the pouch body. Future studies should assess responsiveness to treatment in the next stage toward development of an endoscopic pouchitis disease activity index.

UR - http://www.scopus.com/inward/record.url?scp=85048489478&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048489478&partnerID=8YFLogxK

U2 - 10.1016/j.gie.2018.04.2330

DO - 10.1016/j.gie.2018.04.2330

M3 - Article

C2 - 29660321

AN - SCOPUS:85048489478

VL - 88

SP - 360-369.e2

JO - Gastrointestinal Endoscopy

JF - Gastrointestinal Endoscopy

SN - 0016-5107

IS - 2

ER -