Validation of a Teaching Effectiveness Assessment in Psychiatry Continuing Medical Education

Brian A. Palmer, Mark A Frye, Kristin S. Vickers Douglas, Jeffrey P Staab, Robert P. Bright, Cathy D. Schleck, Jayawant Mandrekar, Saswati Mahapatra, Thomas J. Beckman, Christopher M. Wittich

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Objective: Little is known about factors associated with effective continuing medical education (CME) in psychiatry. The authors aimed to validate a method to assess psychiatry CME teaching effectiveness and to determine associations between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Methods: This cross-sectional study was conducted at the Mayo Clinic Psychiatry Clinical Reviews and Psychiatry in Medical Settings. Presentations were evaluated using an eight-item CME teaching effectiveness instrument, its content based on previously published instruments. Factor analysis, internal consistency and interrater reliabilities, and temporal stability reliability were calculated. Associations were determined between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Results: In total, 364 participants returned 246 completed surveys (response rate, 67.6%). Factor analysis revealed a unidimensional model of psychiatry CME teaching effectiveness. Cronbach α for the instrument was excellent at 0.94. Item mean score (SD) ranged from 4.33 (0.92) to 4.71 (0.59) on a 5-point scale. Overall interrater reliability was 0.84 (95% CI, 0.75–0.91), and temporal stability was 0.89 (95% CI, 0.77–0.97). No associations were found between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Conclusions: This study provides a new, validated measure of CME teaching effectiveness that could be used to improve psychiatry CME. In contrast to prior research in other medical specialties, CME teaching effectiveness scores were not associated with use of case-based or interactive presentations. This outcome suggests the need for distinctive considerations regarding psychiatry CME; a singular approach to CME teaching may not apply to all medical specialties.

Original languageEnglish (US)
Pages (from-to)458-463
Number of pages6
JournalAcademic Psychiatry
Volume42
Issue number4
DOIs
StatePublished - Aug 1 2018

Fingerprint

Continuing Medical Education
psychiatry
Psychiatry
Teaching
education
Statistical Factor Analysis
factor analysis
Medicine
cross-sectional study
Cross-Sectional Studies

Keywords

  • Continuing medical education
  • Continuous professional development
  • Medical education
  • Validation study

ASJC Scopus subject areas

  • Education
  • Psychiatry and Mental health

Cite this

Validation of a Teaching Effectiveness Assessment in Psychiatry Continuing Medical Education. / Palmer, Brian A.; Frye, Mark A; Vickers Douglas, Kristin S.; Staab, Jeffrey P; Bright, Robert P.; Schleck, Cathy D.; Mandrekar, Jayawant; Mahapatra, Saswati; Beckman, Thomas J.; Wittich, Christopher M.

In: Academic Psychiatry, Vol. 42, No. 4, 01.08.2018, p. 458-463.

Research output: Contribution to journalArticle

Palmer, BA, Frye, MA, Vickers Douglas, KS, Staab, JP, Bright, RP, Schleck, CD, Mandrekar, J, Mahapatra, S, Beckman, TJ & Wittich, CM 2018, 'Validation of a Teaching Effectiveness Assessment in Psychiatry Continuing Medical Education', Academic Psychiatry, vol. 42, no. 4, pp. 458-463. https://doi.org/10.1007/s40596-017-0763-8
Palmer, Brian A. ; Frye, Mark A ; Vickers Douglas, Kristin S. ; Staab, Jeffrey P ; Bright, Robert P. ; Schleck, Cathy D. ; Mandrekar, Jayawant ; Mahapatra, Saswati ; Beckman, Thomas J. ; Wittich, Christopher M. / Validation of a Teaching Effectiveness Assessment in Psychiatry Continuing Medical Education. In: Academic Psychiatry. 2018 ; Vol. 42, No. 4. pp. 458-463.
@article{d8e2326baa944cc2a4c2ad4e6dd2f817,
title = "Validation of a Teaching Effectiveness Assessment in Psychiatry Continuing Medical Education",
abstract = "Objective: Little is known about factors associated with effective continuing medical education (CME) in psychiatry. The authors aimed to validate a method to assess psychiatry CME teaching effectiveness and to determine associations between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Methods: This cross-sectional study was conducted at the Mayo Clinic Psychiatry Clinical Reviews and Psychiatry in Medical Settings. Presentations were evaluated using an eight-item CME teaching effectiveness instrument, its content based on previously published instruments. Factor analysis, internal consistency and interrater reliabilities, and temporal stability reliability were calculated. Associations were determined between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Results: In total, 364 participants returned 246 completed surveys (response rate, 67.6{\%}). Factor analysis revealed a unidimensional model of psychiatry CME teaching effectiveness. Cronbach α for the instrument was excellent at 0.94. Item mean score (SD) ranged from 4.33 (0.92) to 4.71 (0.59) on a 5-point scale. Overall interrater reliability was 0.84 (95{\%} CI, 0.75–0.91), and temporal stability was 0.89 (95{\%} CI, 0.77–0.97). No associations were found between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Conclusions: This study provides a new, validated measure of CME teaching effectiveness that could be used to improve psychiatry CME. In contrast to prior research in other medical specialties, CME teaching effectiveness scores were not associated with use of case-based or interactive presentations. This outcome suggests the need for distinctive considerations regarding psychiatry CME; a singular approach to CME teaching may not apply to all medical specialties.",
keywords = "Continuing medical education, Continuous professional development, Medical education, Validation study",
author = "Palmer, {Brian A.} and Frye, {Mark A} and {Vickers Douglas}, {Kristin S.} and Staab, {Jeffrey P} and Bright, {Robert P.} and Schleck, {Cathy D.} and Jayawant Mandrekar and Saswati Mahapatra and Beckman, {Thomas J.} and Wittich, {Christopher M.}",
year = "2018",
month = "8",
day = "1",
doi = "10.1007/s40596-017-0763-8",
language = "English (US)",
volume = "42",
pages = "458--463",
journal = "Academic Psychiatry",
issn = "1042-9670",
publisher = "American Psychiatric Publishing Inc.",
number = "4",

}

TY - JOUR

T1 - Validation of a Teaching Effectiveness Assessment in Psychiatry Continuing Medical Education

AU - Palmer, Brian A.

AU - Frye, Mark A

AU - Vickers Douglas, Kristin S.

AU - Staab, Jeffrey P

AU - Bright, Robert P.

AU - Schleck, Cathy D.

AU - Mandrekar, Jayawant

AU - Mahapatra, Saswati

AU - Beckman, Thomas J.

AU - Wittich, Christopher M.

PY - 2018/8/1

Y1 - 2018/8/1

N2 - Objective: Little is known about factors associated with effective continuing medical education (CME) in psychiatry. The authors aimed to validate a method to assess psychiatry CME teaching effectiveness and to determine associations between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Methods: This cross-sectional study was conducted at the Mayo Clinic Psychiatry Clinical Reviews and Psychiatry in Medical Settings. Presentations were evaluated using an eight-item CME teaching effectiveness instrument, its content based on previously published instruments. Factor analysis, internal consistency and interrater reliabilities, and temporal stability reliability were calculated. Associations were determined between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Results: In total, 364 participants returned 246 completed surveys (response rate, 67.6%). Factor analysis revealed a unidimensional model of psychiatry CME teaching effectiveness. Cronbach α for the instrument was excellent at 0.94. Item mean score (SD) ranged from 4.33 (0.92) to 4.71 (0.59) on a 5-point scale. Overall interrater reliability was 0.84 (95% CI, 0.75–0.91), and temporal stability was 0.89 (95% CI, 0.77–0.97). No associations were found between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Conclusions: This study provides a new, validated measure of CME teaching effectiveness that could be used to improve psychiatry CME. In contrast to prior research in other medical specialties, CME teaching effectiveness scores were not associated with use of case-based or interactive presentations. This outcome suggests the need for distinctive considerations regarding psychiatry CME; a singular approach to CME teaching may not apply to all medical specialties.

AB - Objective: Little is known about factors associated with effective continuing medical education (CME) in psychiatry. The authors aimed to validate a method to assess psychiatry CME teaching effectiveness and to determine associations between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Methods: This cross-sectional study was conducted at the Mayo Clinic Psychiatry Clinical Reviews and Psychiatry in Medical Settings. Presentations were evaluated using an eight-item CME teaching effectiveness instrument, its content based on previously published instruments. Factor analysis, internal consistency and interrater reliabilities, and temporal stability reliability were calculated. Associations were determined between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Results: In total, 364 participants returned 246 completed surveys (response rate, 67.6%). Factor analysis revealed a unidimensional model of psychiatry CME teaching effectiveness. Cronbach α for the instrument was excellent at 0.94. Item mean score (SD) ranged from 4.33 (0.92) to 4.71 (0.59) on a 5-point scale. Overall interrater reliability was 0.84 (95% CI, 0.75–0.91), and temporal stability was 0.89 (95% CI, 0.77–0.97). No associations were found between teaching effectiveness scores and characteristics of presentations, presenters, and participants. Conclusions: This study provides a new, validated measure of CME teaching effectiveness that could be used to improve psychiatry CME. In contrast to prior research in other medical specialties, CME teaching effectiveness scores were not associated with use of case-based or interactive presentations. This outcome suggests the need for distinctive considerations regarding psychiatry CME; a singular approach to CME teaching may not apply to all medical specialties.

KW - Continuing medical education

KW - Continuous professional development

KW - Medical education

KW - Validation study

UR - http://www.scopus.com/inward/record.url?scp=85051114216&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85051114216&partnerID=8YFLogxK

U2 - 10.1007/s40596-017-0763-8

DO - 10.1007/s40596-017-0763-8

M3 - Article

C2 - 28685348

AN - SCOPUS:85051114216

VL - 42

SP - 458

EP - 463

JO - Academic Psychiatry

JF - Academic Psychiatry

SN - 1042-9670

IS - 4

ER -