Mixing web and mail methods in a survey of physicians

Timothy J. Beebe, G. Richard Locke, Sunni A. Barnes, Michael E. Davern, Kari J. Anderson

Research output: Contribution to journalArticle

82 Citations (Scopus)

Abstract

Objective. To assess the effects of two different mixed-mode (mail and web survey) combinations on response rates, response times, and nonresponse bias in a sample of primary care and specialty internal medicine physicians. Data Sources/Study Setting. Primary data were collected from 500 physicians with an appointment in the Mayo Clinic Department of Medicine (DOM) between February and March 2005. Study Design. Physicians were randomly assigned to receive either an initial mailed survey evaluating the Electronic Medical Record (EMR) with a web survey follow-up to nonrespondents or its converse - an initial web survey followed by a mailed survey to nonrespondents. Response rates for each condition were calculated using standard formula. Response times were determined as well. Nonresponse bias was measured by comparing selected characteristics of survey respondents to similar characteristics in the full sample frame. In addition, the distributions of results on key outcome variables were compared overall and by data collection condition and phase. Principal Findings. Overall response rates were somewhat higher in the mail/web condition (70.5 percent) than in the web/mail condition (62.9 percent); differences were more pronounced before the mode switch prior to the mailing to nonrespondents. Median response time was 2 days faster in the web/mail condition than in the mail/web (median=5 and 7 days, respectively) but there was evidence of under-representation of specialist physicians and those who used the EMR a half a day or less each day in the web/mail condition before introduction of the mailed component. This did not translate into significant inconsistencies or differences in the distributions of key outcome variables, however. Conclusions. A methodology that uses an initial mailing of a self-administered form followed by a web survey to nonrespondents provides slightly higher response rates and a more representative sample than one that starts with web and ends with a mailed survey. However, if the length of the data collection period is limited and rapid response is important, perhaps the web survey followed by a mailed questionnaire is to be preferred. Key outcome variables appear to be unaffected by the data collection method.

Original languageEnglish (US)
Pages (from-to)1219-1234
Number of pages16
JournalHealth Services Research
Volume42
Issue number3 I
DOIs
StatePublished - Jun 2007

Fingerprint

Postal Service
physician
Physicians
response behavior
Reaction Time
medicine
electronics
Electronic Health Records
Surveys and Questionnaires
data collection method
trend
Information Storage and Retrieval
Internal Medicine
questionnaire
methodology
Primary Health Care
Appointments and Schedules

Keywords

  • Methods
  • Mixed mode
  • Physician survey
  • Response bias
  • Response rate

ASJC Scopus subject areas

  • Health Policy
  • Nursing(all)
  • Health(social science)
  • Health Professions(all)

Cite this

Beebe, T. J., Locke, G. R., Barnes, S. A., Davern, M. E., & Anderson, K. J. (2007). Mixing web and mail methods in a survey of physicians. Health Services Research, 42(3 I), 1219-1234. https://doi.org/10.1111/j.1475-6773.2006.00652.x

Mixing web and mail methods in a survey of physicians. / Beebe, Timothy J.; Locke, G. Richard; Barnes, Sunni A.; Davern, Michael E.; Anderson, Kari J.

In: Health Services Research, Vol. 42, No. 3 I, 06.2007, p. 1219-1234.

Research output: Contribution to journalArticle

Beebe, TJ, Locke, GR, Barnes, SA, Davern, ME & Anderson, KJ 2007, 'Mixing web and mail methods in a survey of physicians', Health Services Research, vol. 42, no. 3 I, pp. 1219-1234. https://doi.org/10.1111/j.1475-6773.2006.00652.x
Beebe TJ, Locke GR, Barnes SA, Davern ME, Anderson KJ. Mixing web and mail methods in a survey of physicians. Health Services Research. 2007 Jun;42(3 I):1219-1234. https://doi.org/10.1111/j.1475-6773.2006.00652.x
Beebe, Timothy J. ; Locke, G. Richard ; Barnes, Sunni A. ; Davern, Michael E. ; Anderson, Kari J. / Mixing web and mail methods in a survey of physicians. In: Health Services Research. 2007 ; Vol. 42, No. 3 I. pp. 1219-1234.
@article{54076036020b439692a0198e6b67f901,
title = "Mixing web and mail methods in a survey of physicians",
abstract = "Objective. To assess the effects of two different mixed-mode (mail and web survey) combinations on response rates, response times, and nonresponse bias in a sample of primary care and specialty internal medicine physicians. Data Sources/Study Setting. Primary data were collected from 500 physicians with an appointment in the Mayo Clinic Department of Medicine (DOM) between February and March 2005. Study Design. Physicians were randomly assigned to receive either an initial mailed survey evaluating the Electronic Medical Record (EMR) with a web survey follow-up to nonrespondents or its converse - an initial web survey followed by a mailed survey to nonrespondents. Response rates for each condition were calculated using standard formula. Response times were determined as well. Nonresponse bias was measured by comparing selected characteristics of survey respondents to similar characteristics in the full sample frame. In addition, the distributions of results on key outcome variables were compared overall and by data collection condition and phase. Principal Findings. Overall response rates were somewhat higher in the mail/web condition (70.5 percent) than in the web/mail condition (62.9 percent); differences were more pronounced before the mode switch prior to the mailing to nonrespondents. Median response time was 2 days faster in the web/mail condition than in the mail/web (median=5 and 7 days, respectively) but there was evidence of under-representation of specialist physicians and those who used the EMR a half a day or less each day in the web/mail condition before introduction of the mailed component. This did not translate into significant inconsistencies or differences in the distributions of key outcome variables, however. Conclusions. A methodology that uses an initial mailing of a self-administered form followed by a web survey to nonrespondents provides slightly higher response rates and a more representative sample than one that starts with web and ends with a mailed survey. However, if the length of the data collection period is limited and rapid response is important, perhaps the web survey followed by a mailed questionnaire is to be preferred. Key outcome variables appear to be unaffected by the data collection method.",
keywords = "Methods, Mixed mode, Physician survey, Response bias, Response rate",
author = "Beebe, {Timothy J.} and Locke, {G. Richard} and Barnes, {Sunni A.} and Davern, {Michael E.} and Anderson, {Kari J.}",
year = "2007",
month = "6",
doi = "10.1111/j.1475-6773.2006.00652.x",
language = "English (US)",
volume = "42",
pages = "1219--1234",
journal = "Health Services Research",
issn = "0017-9124",
publisher = "Wiley-Blackwell",
number = "3 I",

}

TY - JOUR

T1 - Mixing web and mail methods in a survey of physicians

AU - Beebe, Timothy J.

AU - Locke, G. Richard

AU - Barnes, Sunni A.

AU - Davern, Michael E.

AU - Anderson, Kari J.

PY - 2007/6

Y1 - 2007/6

N2 - Objective. To assess the effects of two different mixed-mode (mail and web survey) combinations on response rates, response times, and nonresponse bias in a sample of primary care and specialty internal medicine physicians. Data Sources/Study Setting. Primary data were collected from 500 physicians with an appointment in the Mayo Clinic Department of Medicine (DOM) between February and March 2005. Study Design. Physicians were randomly assigned to receive either an initial mailed survey evaluating the Electronic Medical Record (EMR) with a web survey follow-up to nonrespondents or its converse - an initial web survey followed by a mailed survey to nonrespondents. Response rates for each condition were calculated using standard formula. Response times were determined as well. Nonresponse bias was measured by comparing selected characteristics of survey respondents to similar characteristics in the full sample frame. In addition, the distributions of results on key outcome variables were compared overall and by data collection condition and phase. Principal Findings. Overall response rates were somewhat higher in the mail/web condition (70.5 percent) than in the web/mail condition (62.9 percent); differences were more pronounced before the mode switch prior to the mailing to nonrespondents. Median response time was 2 days faster in the web/mail condition than in the mail/web (median=5 and 7 days, respectively) but there was evidence of under-representation of specialist physicians and those who used the EMR a half a day or less each day in the web/mail condition before introduction of the mailed component. This did not translate into significant inconsistencies or differences in the distributions of key outcome variables, however. Conclusions. A methodology that uses an initial mailing of a self-administered form followed by a web survey to nonrespondents provides slightly higher response rates and a more representative sample than one that starts with web and ends with a mailed survey. However, if the length of the data collection period is limited and rapid response is important, perhaps the web survey followed by a mailed questionnaire is to be preferred. Key outcome variables appear to be unaffected by the data collection method.

AB - Objective. To assess the effects of two different mixed-mode (mail and web survey) combinations on response rates, response times, and nonresponse bias in a sample of primary care and specialty internal medicine physicians. Data Sources/Study Setting. Primary data were collected from 500 physicians with an appointment in the Mayo Clinic Department of Medicine (DOM) between February and March 2005. Study Design. Physicians were randomly assigned to receive either an initial mailed survey evaluating the Electronic Medical Record (EMR) with a web survey follow-up to nonrespondents or its converse - an initial web survey followed by a mailed survey to nonrespondents. Response rates for each condition were calculated using standard formula. Response times were determined as well. Nonresponse bias was measured by comparing selected characteristics of survey respondents to similar characteristics in the full sample frame. In addition, the distributions of results on key outcome variables were compared overall and by data collection condition and phase. Principal Findings. Overall response rates were somewhat higher in the mail/web condition (70.5 percent) than in the web/mail condition (62.9 percent); differences were more pronounced before the mode switch prior to the mailing to nonrespondents. Median response time was 2 days faster in the web/mail condition than in the mail/web (median=5 and 7 days, respectively) but there was evidence of under-representation of specialist physicians and those who used the EMR a half a day or less each day in the web/mail condition before introduction of the mailed component. This did not translate into significant inconsistencies or differences in the distributions of key outcome variables, however. Conclusions. A methodology that uses an initial mailing of a self-administered form followed by a web survey to nonrespondents provides slightly higher response rates and a more representative sample than one that starts with web and ends with a mailed survey. However, if the length of the data collection period is limited and rapid response is important, perhaps the web survey followed by a mailed questionnaire is to be preferred. Key outcome variables appear to be unaffected by the data collection method.

KW - Methods

KW - Mixed mode

KW - Physician survey

KW - Response bias

KW - Response rate

UR - http://www.scopus.com/inward/record.url?scp=34247859109&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34247859109&partnerID=8YFLogxK

U2 - 10.1111/j.1475-6773.2006.00652.x

DO - 10.1111/j.1475-6773.2006.00652.x

M3 - Article

VL - 42

SP - 1219

EP - 1234

JO - Health Services Research

JF - Health Services Research

SN - 0017-9124

IS - 3 I

ER -