Facilitating Student Success in Introductory Chemistry with Feedback in an Online Platform

Sam Van Horne, Maura Curran, Anna Smith, John VanBuren, David Zahrieh, Russell Larsen, Ross Miller

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Instructional technologists and faculty in post-secondary institutions have increasingly adopted learning analytics interventions such as dashboards that provide real-time feedback to students to support student’ ability to regulate their learning. But analyses of the effectiveness of such interventions can be confounded by measures of students’ prior learning as well as their baseline level of self-regulated learning. For this research study, we sought to examine whether the frequency of accessing a dashboard was associated with learning outcomes after matching subjects on confounding variables. And because prior research has suggested that measures of prior learning are associated with students’ likelihood to use learning analytics interventions, we sought to adequately control for learners’ likelihood to access the feedback by using a propensity score matching with a non-binary treatment variable. We administered the Motivated Strategies for Learning Questionnaire and also collected demographic information for a propensity score matching process. Users’ frequency of accessing the intervention was categorized as High, Moderate, or Low/No usage. After matching users on characteristics associated with dashboard usage (gender, high school GPA, and the “Test Anxiety” and “Self Efficacy” factors) we found that both the “High” and “Moderate” users achieved significantly higher course grades than the “Low/No” users. The results suggest learners benefited from regularly accessing the feedback, but extreme amounts of usage were not necessary to achieve a positive effect. We discuss the implications for recommending how students use learning analytics interventions without excessively accessing feedback.

Original languageEnglish (US)
Pages (from-to)21-40
Number of pages20
JournalTechnology, Knowledge and Learning
Volume23
Issue number1
DOIs
StatePublished - Apr 1 2018

Fingerprint

Chemistry
chemistry
Students
Feedback
learning
student
Propensity Score
Likelihood
Self-efficacy
Learning
Anxiety
Confounding
Questionnaire
self-efficacy
Baseline
Extremes
anxiety
Real-time
Necessary
questionnaire

Keywords

  • Feedback
  • Learning analytics
  • Motivation
  • Propensity score matching
  • Self-regulated learners

ASJC Scopus subject areas

  • Mathematics (miscellaneous)
  • Education
  • Human-Computer Interaction
  • Computer Science Applications

Cite this

Facilitating Student Success in Introductory Chemistry with Feedback in an Online Platform. / Van Horne, Sam; Curran, Maura; Smith, Anna; VanBuren, John; Zahrieh, David; Larsen, Russell; Miller, Ross.

In: Technology, Knowledge and Learning, Vol. 23, No. 1, 01.04.2018, p. 21-40.

Research output: Contribution to journalArticle

Van Horne, Sam ; Curran, Maura ; Smith, Anna ; VanBuren, John ; Zahrieh, David ; Larsen, Russell ; Miller, Ross. / Facilitating Student Success in Introductory Chemistry with Feedback in an Online Platform. In: Technology, Knowledge and Learning. 2018 ; Vol. 23, No. 1. pp. 21-40.
@article{c8864da2538c44b0987ce952d09b46a2,
title = "Facilitating Student Success in Introductory Chemistry with Feedback in an Online Platform",
abstract = "Instructional technologists and faculty in post-secondary institutions have increasingly adopted learning analytics interventions such as dashboards that provide real-time feedback to students to support student’ ability to regulate their learning. But analyses of the effectiveness of such interventions can be confounded by measures of students’ prior learning as well as their baseline level of self-regulated learning. For this research study, we sought to examine whether the frequency of accessing a dashboard was associated with learning outcomes after matching subjects on confounding variables. And because prior research has suggested that measures of prior learning are associated with students’ likelihood to use learning analytics interventions, we sought to adequately control for learners’ likelihood to access the feedback by using a propensity score matching with a non-binary treatment variable. We administered the Motivated Strategies for Learning Questionnaire and also collected demographic information for a propensity score matching process. Users’ frequency of accessing the intervention was categorized as High, Moderate, or Low/No usage. After matching users on characteristics associated with dashboard usage (gender, high school GPA, and the “Test Anxiety” and “Self Efficacy” factors) we found that both the “High” and “Moderate” users achieved significantly higher course grades than the “Low/No” users. The results suggest learners benefited from regularly accessing the feedback, but extreme amounts of usage were not necessary to achieve a positive effect. We discuss the implications for recommending how students use learning analytics interventions without excessively accessing feedback.",
keywords = "Feedback, Learning analytics, Motivation, Propensity score matching, Self-regulated learners",
author = "{Van Horne}, Sam and Maura Curran and Anna Smith and John VanBuren and David Zahrieh and Russell Larsen and Ross Miller",
year = "2018",
month = "4",
day = "1",
doi = "10.1007/s10758-017-9341-0",
language = "English (US)",
volume = "23",
pages = "21--40",
journal = "Technology, Knowledge and Learning",
issn = "2211-1662",
publisher = "Springer Science + Business Media",
number = "1",

}

TY - JOUR

T1 - Facilitating Student Success in Introductory Chemistry with Feedback in an Online Platform

AU - Van Horne, Sam

AU - Curran, Maura

AU - Smith, Anna

AU - VanBuren, John

AU - Zahrieh, David

AU - Larsen, Russell

AU - Miller, Ross

PY - 2018/4/1

Y1 - 2018/4/1

N2 - Instructional technologists and faculty in post-secondary institutions have increasingly adopted learning analytics interventions such as dashboards that provide real-time feedback to students to support student’ ability to regulate their learning. But analyses of the effectiveness of such interventions can be confounded by measures of students’ prior learning as well as their baseline level of self-regulated learning. For this research study, we sought to examine whether the frequency of accessing a dashboard was associated with learning outcomes after matching subjects on confounding variables. And because prior research has suggested that measures of prior learning are associated with students’ likelihood to use learning analytics interventions, we sought to adequately control for learners’ likelihood to access the feedback by using a propensity score matching with a non-binary treatment variable. We administered the Motivated Strategies for Learning Questionnaire and also collected demographic information for a propensity score matching process. Users’ frequency of accessing the intervention was categorized as High, Moderate, or Low/No usage. After matching users on characteristics associated with dashboard usage (gender, high school GPA, and the “Test Anxiety” and “Self Efficacy” factors) we found that both the “High” and “Moderate” users achieved significantly higher course grades than the “Low/No” users. The results suggest learners benefited from regularly accessing the feedback, but extreme amounts of usage were not necessary to achieve a positive effect. We discuss the implications for recommending how students use learning analytics interventions without excessively accessing feedback.

AB - Instructional technologists and faculty in post-secondary institutions have increasingly adopted learning analytics interventions such as dashboards that provide real-time feedback to students to support student’ ability to regulate their learning. But analyses of the effectiveness of such interventions can be confounded by measures of students’ prior learning as well as their baseline level of self-regulated learning. For this research study, we sought to examine whether the frequency of accessing a dashboard was associated with learning outcomes after matching subjects on confounding variables. And because prior research has suggested that measures of prior learning are associated with students’ likelihood to use learning analytics interventions, we sought to adequately control for learners’ likelihood to access the feedback by using a propensity score matching with a non-binary treatment variable. We administered the Motivated Strategies for Learning Questionnaire and also collected demographic information for a propensity score matching process. Users’ frequency of accessing the intervention was categorized as High, Moderate, or Low/No usage. After matching users on characteristics associated with dashboard usage (gender, high school GPA, and the “Test Anxiety” and “Self Efficacy” factors) we found that both the “High” and “Moderate” users achieved significantly higher course grades than the “Low/No” users. The results suggest learners benefited from regularly accessing the feedback, but extreme amounts of usage were not necessary to achieve a positive effect. We discuss the implications for recommending how students use learning analytics interventions without excessively accessing feedback.

KW - Feedback

KW - Learning analytics

KW - Motivation

KW - Propensity score matching

KW - Self-regulated learners

UR - http://www.scopus.com/inward/record.url?scp=85035084698&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85035084698&partnerID=8YFLogxK

U2 - 10.1007/s10758-017-9341-0

DO - 10.1007/s10758-017-9341-0

M3 - Article

AN - SCOPUS:85035084698

VL - 23

SP - 21

EP - 40

JO - Technology, Knowledge and Learning

JF - Technology, Knowledge and Learning

SN - 2211-1662

IS - 1

ER -