Inter-rater reliability of the PIPES tool: Validation of a surgical capacity index for use in resource-limited settings

Abraham Markin, Roxana Barbero, Jeffrey J. Leow, Reinou S. Groen, Greg Perlman, Elizabeth B Habermann, Keith N. Apelgren, Adam L. Kushner, Benedict C. Nwomeh

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Introduction: In response to the need for simple, rapid means of quantifying surgical capacity in low resource settings, Surgeons OverSeas (SOS) developed the personnel, infrastructure, procedures, equipment and supplies (PIPES) tool. The present investigation assessed the inter-rater reliability of the PIPES tool. Methods: As part of a government assessment of surgical services in Santa Cruz, Bolivia, the PIPES tool was translated into Spanish and applied in interviews with physicians at 31 public hospitals. An additional interview was conducted with nurses at a convenience sample of 25 of these hospitals. Physician and nurse responses were then compared to generate an estimate of reliability. For dichotomous survey items, inter-rater reliability between physicians and nurses was assessed using the Cohen's kappa statistic and percent agreement. The Pearson correlation coefficient was used to assess agreement for continuous items. Results: Cohen's kappa was 0.46 for infrastructure, 0.43 for procedures, 0.26 for equipment, and 0 for supplies sections. The median correlation coefficient was 0.91 for continuous items. Correlation was 0.79 for the PIPES index, and ranged from 0.32 to 0.98 for continuous response items. Conclusions: Reliability of the PIPES tool was moderate for the infrastructure and procedures sections, fair for the equipment section, and poor for supplies section when comparing surgeons' responses to nurses' responses - an extremely rigorous test of reliability. These results indicate that the PIPES tool is an effective measure of surgical capacity but that the equipment and supplies sections may need to be revised.

Original languageEnglish (US)
Pages (from-to)2195-2199
Number of pages5
JournalWorld Journal of Surgery
Volume38
Issue number9
DOIs
StatePublished - 2014

Fingerprint

Equipment and Supplies
Nurses
Physicians
Surgical Equipment
Interviews
Bolivia
Public Hospitals
Surgeons

ASJC Scopus subject areas

  • Surgery

Cite this

Inter-rater reliability of the PIPES tool : Validation of a surgical capacity index for use in resource-limited settings. / Markin, Abraham; Barbero, Roxana; Leow, Jeffrey J.; Groen, Reinou S.; Perlman, Greg; Habermann, Elizabeth B; Apelgren, Keith N.; Kushner, Adam L.; Nwomeh, Benedict C.

In: World Journal of Surgery, Vol. 38, No. 9, 2014, p. 2195-2199.

Research output: Contribution to journalArticle

Markin, A, Barbero, R, Leow, JJ, Groen, RS, Perlman, G, Habermann, EB, Apelgren, KN, Kushner, AL & Nwomeh, BC 2014, 'Inter-rater reliability of the PIPES tool: Validation of a surgical capacity index for use in resource-limited settings', World Journal of Surgery, vol. 38, no. 9, pp. 2195-2199. https://doi.org/10.1007/s00268-014-2522-2
Markin, Abraham ; Barbero, Roxana ; Leow, Jeffrey J. ; Groen, Reinou S. ; Perlman, Greg ; Habermann, Elizabeth B ; Apelgren, Keith N. ; Kushner, Adam L. ; Nwomeh, Benedict C. / Inter-rater reliability of the PIPES tool : Validation of a surgical capacity index for use in resource-limited settings. In: World Journal of Surgery. 2014 ; Vol. 38, No. 9. pp. 2195-2199.
@article{3ffc181782214b679271c0762d72f564,
title = "Inter-rater reliability of the PIPES tool: Validation of a surgical capacity index for use in resource-limited settings",
abstract = "Introduction: In response to the need for simple, rapid means of quantifying surgical capacity in low resource settings, Surgeons OverSeas (SOS) developed the personnel, infrastructure, procedures, equipment and supplies (PIPES) tool. The present investigation assessed the inter-rater reliability of the PIPES tool. Methods: As part of a government assessment of surgical services in Santa Cruz, Bolivia, the PIPES tool was translated into Spanish and applied in interviews with physicians at 31 public hospitals. An additional interview was conducted with nurses at a convenience sample of 25 of these hospitals. Physician and nurse responses were then compared to generate an estimate of reliability. For dichotomous survey items, inter-rater reliability between physicians and nurses was assessed using the Cohen's kappa statistic and percent agreement. The Pearson correlation coefficient was used to assess agreement for continuous items. Results: Cohen's kappa was 0.46 for infrastructure, 0.43 for procedures, 0.26 for equipment, and 0 for supplies sections. The median correlation coefficient was 0.91 for continuous items. Correlation was 0.79 for the PIPES index, and ranged from 0.32 to 0.98 for continuous response items. Conclusions: Reliability of the PIPES tool was moderate for the infrastructure and procedures sections, fair for the equipment section, and poor for supplies section when comparing surgeons' responses to nurses' responses - an extremely rigorous test of reliability. These results indicate that the PIPES tool is an effective measure of surgical capacity but that the equipment and supplies sections may need to be revised.",
author = "Abraham Markin and Roxana Barbero and Leow, {Jeffrey J.} and Groen, {Reinou S.} and Greg Perlman and Habermann, {Elizabeth B} and Apelgren, {Keith N.} and Kushner, {Adam L.} and Nwomeh, {Benedict C.}",
year = "2014",
doi = "10.1007/s00268-014-2522-2",
language = "English (US)",
volume = "38",
pages = "2195--2199",
journal = "Presentations from the 9th Annual Electric Utilities Environmental Conference",
issn = "0364-2313",
publisher = "Springer New York",
number = "9",

}

TY - JOUR

T1 - Inter-rater reliability of the PIPES tool

T2 - Validation of a surgical capacity index for use in resource-limited settings

AU - Markin, Abraham

AU - Barbero, Roxana

AU - Leow, Jeffrey J.

AU - Groen, Reinou S.

AU - Perlman, Greg

AU - Habermann, Elizabeth B

AU - Apelgren, Keith N.

AU - Kushner, Adam L.

AU - Nwomeh, Benedict C.

PY - 2014

Y1 - 2014

N2 - Introduction: In response to the need for simple, rapid means of quantifying surgical capacity in low resource settings, Surgeons OverSeas (SOS) developed the personnel, infrastructure, procedures, equipment and supplies (PIPES) tool. The present investigation assessed the inter-rater reliability of the PIPES tool. Methods: As part of a government assessment of surgical services in Santa Cruz, Bolivia, the PIPES tool was translated into Spanish and applied in interviews with physicians at 31 public hospitals. An additional interview was conducted with nurses at a convenience sample of 25 of these hospitals. Physician and nurse responses were then compared to generate an estimate of reliability. For dichotomous survey items, inter-rater reliability between physicians and nurses was assessed using the Cohen's kappa statistic and percent agreement. The Pearson correlation coefficient was used to assess agreement for continuous items. Results: Cohen's kappa was 0.46 for infrastructure, 0.43 for procedures, 0.26 for equipment, and 0 for supplies sections. The median correlation coefficient was 0.91 for continuous items. Correlation was 0.79 for the PIPES index, and ranged from 0.32 to 0.98 for continuous response items. Conclusions: Reliability of the PIPES tool was moderate for the infrastructure and procedures sections, fair for the equipment section, and poor for supplies section when comparing surgeons' responses to nurses' responses - an extremely rigorous test of reliability. These results indicate that the PIPES tool is an effective measure of surgical capacity but that the equipment and supplies sections may need to be revised.

AB - Introduction: In response to the need for simple, rapid means of quantifying surgical capacity in low resource settings, Surgeons OverSeas (SOS) developed the personnel, infrastructure, procedures, equipment and supplies (PIPES) tool. The present investigation assessed the inter-rater reliability of the PIPES tool. Methods: As part of a government assessment of surgical services in Santa Cruz, Bolivia, the PIPES tool was translated into Spanish and applied in interviews with physicians at 31 public hospitals. An additional interview was conducted with nurses at a convenience sample of 25 of these hospitals. Physician and nurse responses were then compared to generate an estimate of reliability. For dichotomous survey items, inter-rater reliability between physicians and nurses was assessed using the Cohen's kappa statistic and percent agreement. The Pearson correlation coefficient was used to assess agreement for continuous items. Results: Cohen's kappa was 0.46 for infrastructure, 0.43 for procedures, 0.26 for equipment, and 0 for supplies sections. The median correlation coefficient was 0.91 for continuous items. Correlation was 0.79 for the PIPES index, and ranged from 0.32 to 0.98 for continuous response items. Conclusions: Reliability of the PIPES tool was moderate for the infrastructure and procedures sections, fair for the equipment section, and poor for supplies section when comparing surgeons' responses to nurses' responses - an extremely rigorous test of reliability. These results indicate that the PIPES tool is an effective measure of surgical capacity but that the equipment and supplies sections may need to be revised.

UR - http://www.scopus.com/inward/record.url?scp=84905708445&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84905708445&partnerID=8YFLogxK

U2 - 10.1007/s00268-014-2522-2

DO - 10.1007/s00268-014-2522-2

M3 - Article

C2 - 24696058

AN - SCOPUS:84905708445

VL - 38

SP - 2195

EP - 2199

JO - Presentations from the 9th Annual Electric Utilities Environmental Conference

JF - Presentations from the 9th Annual Electric Utilities Environmental Conference

SN - 0364-2313

IS - 9

ER -