TY - JOUR
T1 - Evaluating technology-enhanced learning
T2 - A comprehensive framework
AU - Cook, David A.
AU - Ellaway, Rachel H.
PY - 2015/1/1
Y1 - 2015/1/1
N2 - Background: The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies. Purpose: To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors. Methods and results: Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL. Conclusions: No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.
AB - Background: The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies. Purpose: To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors. Methods and results: Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL. Conclusions: No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.
UR - http://www.scopus.com/inward/record.url?scp=84940783714&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84940783714&partnerID=8YFLogxK
U2 - 10.3109/0142159X.2015.1009024
DO - 10.3109/0142159X.2015.1009024
M3 - Article
C2 - 25782599
AN - SCOPUS:84940783714
SN - 0142-159X
VL - 37
SP - 961
EP - 970
JO - Medical teacher
JF - Medical teacher
IS - 10
ER -