Evaluating technology-enhanced learning: A comprehensive framework

David Allan Cook, Rachel H. Ellaway

Research output: Contribution to journalArticle

33 Citations (Scopus)

Abstract

Background: The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies. Purpose: To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors. Methods and results: Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL. Conclusions: No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.

Original languageEnglish (US)
JournalMedical Teacher
DOIs
StateAccepted/In press - Mar 18 2015

Fingerprint

Learning
Technology
evaluation
learning
instructor
Medical Education
Documentation
documentation
education
experience
sustainability
Observation
Outcome Assessment (Health Care)
Education
Costs and Cost Analysis
costs

ASJC Scopus subject areas

  • Medicine(all)
  • Education

Cite this

Evaluating technology-enhanced learning : A comprehensive framework. / Cook, David Allan; Ellaway, Rachel H.

In: Medical Teacher, 18.03.2015.

Research output: Contribution to journalArticle

@article{c537a6320c1345049d50042f180b8136,
title = "Evaluating technology-enhanced learning: A comprehensive framework",
abstract = "Background: The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies. Purpose: To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors. Methods and results: Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL. Conclusions: No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.",
author = "Cook, {David Allan} and Ellaway, {Rachel H.}",
year = "2015",
month = "3",
day = "18",
doi = "10.3109/0142159X.2015.1009024",
language = "English (US)",
journal = "Medical Teacher",
issn = "0142-159X",
publisher = "Informa Healthcare",

}

TY - JOUR

T1 - Evaluating technology-enhanced learning

T2 - A comprehensive framework

AU - Cook, David Allan

AU - Ellaway, Rachel H.

PY - 2015/3/18

Y1 - 2015/3/18

N2 - Background: The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies. Purpose: To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors. Methods and results: Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL. Conclusions: No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.

AB - Background: The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies. Purpose: To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors. Methods and results: Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL. Conclusions: No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.

UR - http://www.scopus.com/inward/record.url?scp=84940783714&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940783714&partnerID=8YFLogxK

U2 - 10.3109/0142159X.2015.1009024

DO - 10.3109/0142159X.2015.1009024

M3 - Article

C2 - 25782599

AN - SCOPUS:84940783714

JO - Medical Teacher

JF - Medical Teacher

SN - 0142-159X

ER -