Observer agreement in the assessment of endovascular aneurysm therapy and aneurysm recurrence

Research output: Contribution to journalArticle

51 Citations (Scopus)

Abstract

BACKGROUND AND PURPOSE: Assessments of completeness of endovascular cerebral aneurysm therapy are commonly reported in the literature. We studied several aneurysm assessment scales with regard to observer variability, which directly affects validity of these scales. MATERIALS AND METHODS: Initial aneurysm occlusion and occlusion at a follow-up angiogram at 3-6 months were assessed independently by 2 experienced observers. Assessments of each aneurysm were made using 3 different scales: 4-response (complete, dog ear, neck remnant, incomplete), 3-response (complete, near-complete, incomplete), and 2-response (complete or near-complete, incomplete). Assessments were also made of comparisons of initial treatment angiogram with follow-up angiogram using 2 different scales: 3-response (better, same, worse) and 2-point response (not worse, worse). RESULTS: With assessments of both initial and follow-up angiograms, interobserver and intraobserver agreement was progressively worse with increasing response choices in the scales. Observer agreement on assessments of initial angiograms (κ values 0.48-0.67) was worse than that for follow-up angiograms (κ values 0.66-0.97). For the comparisons of the initial angiogram with the follow-up angiogram, there was worse observer agreement with the 3-response scale (κ values 0.64-0.71) than with the 2-response scale (κ values 0.78-0.89). CONCLUSION: Interobserver and intraobserver variability are inherent to assessment scales of completeness of cerebral aneurysm therapy. Observer variability is substantially better in scales that offer fewer observer responses. However, scales with fewer observer responses may not identify aneurysm subgroups that have differing risks of recurrence and/or rehemorrhage.

Original languageEnglish (US)
Pages (from-to)497-500
Number of pages4
JournalAmerican Journal of Neuroradiology
Volume28
Issue number3
StatePublished - Mar 2007

Fingerprint

Aneurysm
Angiography
Recurrence
Intracranial Aneurysm
Therapeutics
Observer Variation
Ear
Neck
Dogs

ASJC Scopus subject areas

  • Clinical Neurology
  • Radiology Nuclear Medicine and imaging
  • Radiological and Ultrasound Technology

Cite this

Observer agreement in the assessment of endovascular aneurysm therapy and aneurysm recurrence. / Cloft, Harry J.; Kaufmann, Timothy J; Kallmes, David F.

In: American Journal of Neuroradiology, Vol. 28, No. 3, 03.2007, p. 497-500.

Research output: Contribution to journalArticle

@article{d22fd8c60ee94311b58ae82c69ee9819,
title = "Observer agreement in the assessment of endovascular aneurysm therapy and aneurysm recurrence",
abstract = "BACKGROUND AND PURPOSE: Assessments of completeness of endovascular cerebral aneurysm therapy are commonly reported in the literature. We studied several aneurysm assessment scales with regard to observer variability, which directly affects validity of these scales. MATERIALS AND METHODS: Initial aneurysm occlusion and occlusion at a follow-up angiogram at 3-6 months were assessed independently by 2 experienced observers. Assessments of each aneurysm were made using 3 different scales: 4-response (complete, dog ear, neck remnant, incomplete), 3-response (complete, near-complete, incomplete), and 2-response (complete or near-complete, incomplete). Assessments were also made of comparisons of initial treatment angiogram with follow-up angiogram using 2 different scales: 3-response (better, same, worse) and 2-point response (not worse, worse). RESULTS: With assessments of both initial and follow-up angiograms, interobserver and intraobserver agreement was progressively worse with increasing response choices in the scales. Observer agreement on assessments of initial angiograms (κ values 0.48-0.67) was worse than that for follow-up angiograms (κ values 0.66-0.97). For the comparisons of the initial angiogram with the follow-up angiogram, there was worse observer agreement with the 3-response scale (κ values 0.64-0.71) than with the 2-response scale (κ values 0.78-0.89). CONCLUSION: Interobserver and intraobserver variability are inherent to assessment scales of completeness of cerebral aneurysm therapy. Observer variability is substantially better in scales that offer fewer observer responses. However, scales with fewer observer responses may not identify aneurysm subgroups that have differing risks of recurrence and/or rehemorrhage.",
author = "Cloft, {Harry J.} and Kaufmann, {Timothy J} and Kallmes, {David F}",
year = "2007",
month = "3",
language = "English (US)",
volume = "28",
pages = "497--500",
journal = "American Journal of Neuroradiology",
issn = "0195-6108",
publisher = "American Society of Neuroradiology",
number = "3",

}

TY - JOUR

T1 - Observer agreement in the assessment of endovascular aneurysm therapy and aneurysm recurrence

AU - Cloft, Harry J.

AU - Kaufmann, Timothy J

AU - Kallmes, David F

PY - 2007/3

Y1 - 2007/3

N2 - BACKGROUND AND PURPOSE: Assessments of completeness of endovascular cerebral aneurysm therapy are commonly reported in the literature. We studied several aneurysm assessment scales with regard to observer variability, which directly affects validity of these scales. MATERIALS AND METHODS: Initial aneurysm occlusion and occlusion at a follow-up angiogram at 3-6 months were assessed independently by 2 experienced observers. Assessments of each aneurysm were made using 3 different scales: 4-response (complete, dog ear, neck remnant, incomplete), 3-response (complete, near-complete, incomplete), and 2-response (complete or near-complete, incomplete). Assessments were also made of comparisons of initial treatment angiogram with follow-up angiogram using 2 different scales: 3-response (better, same, worse) and 2-point response (not worse, worse). RESULTS: With assessments of both initial and follow-up angiograms, interobserver and intraobserver agreement was progressively worse with increasing response choices in the scales. Observer agreement on assessments of initial angiograms (κ values 0.48-0.67) was worse than that for follow-up angiograms (κ values 0.66-0.97). For the comparisons of the initial angiogram with the follow-up angiogram, there was worse observer agreement with the 3-response scale (κ values 0.64-0.71) than with the 2-response scale (κ values 0.78-0.89). CONCLUSION: Interobserver and intraobserver variability are inherent to assessment scales of completeness of cerebral aneurysm therapy. Observer variability is substantially better in scales that offer fewer observer responses. However, scales with fewer observer responses may not identify aneurysm subgroups that have differing risks of recurrence and/or rehemorrhage.

AB - BACKGROUND AND PURPOSE: Assessments of completeness of endovascular cerebral aneurysm therapy are commonly reported in the literature. We studied several aneurysm assessment scales with regard to observer variability, which directly affects validity of these scales. MATERIALS AND METHODS: Initial aneurysm occlusion and occlusion at a follow-up angiogram at 3-6 months were assessed independently by 2 experienced observers. Assessments of each aneurysm were made using 3 different scales: 4-response (complete, dog ear, neck remnant, incomplete), 3-response (complete, near-complete, incomplete), and 2-response (complete or near-complete, incomplete). Assessments were also made of comparisons of initial treatment angiogram with follow-up angiogram using 2 different scales: 3-response (better, same, worse) and 2-point response (not worse, worse). RESULTS: With assessments of both initial and follow-up angiograms, interobserver and intraobserver agreement was progressively worse with increasing response choices in the scales. Observer agreement on assessments of initial angiograms (κ values 0.48-0.67) was worse than that for follow-up angiograms (κ values 0.66-0.97). For the comparisons of the initial angiogram with the follow-up angiogram, there was worse observer agreement with the 3-response scale (κ values 0.64-0.71) than with the 2-response scale (κ values 0.78-0.89). CONCLUSION: Interobserver and intraobserver variability are inherent to assessment scales of completeness of cerebral aneurysm therapy. Observer variability is substantially better in scales that offer fewer observer responses. However, scales with fewer observer responses may not identify aneurysm subgroups that have differing risks of recurrence and/or rehemorrhage.

UR - http://www.scopus.com/inward/record.url?scp=33947598336&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33947598336&partnerID=8YFLogxK

M3 - Article

VL - 28

SP - 497

EP - 500

JO - American Journal of Neuroradiology

JF - American Journal of Neuroradiology

SN - 0195-6108

IS - 3

ER -