Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys

Michael Davern, Donna McAlpine, Timothy J. Beebe, Jeanette Ziegenfuss, Todd Rockwood, Kathleen Thiede Call

Research output: Contribution to journalArticle

61 Citations (Scopus)

Abstract

Objective. To examine the impact of response rate variation on survey estimates and costs in three health telephone surveys. Data Source. Three telephone surveys of noninstitutionalized adults in Minnesota and Oklahoma conducted from 2003 to 2005. Study Design. We examine differences in demographics and health measures by number of call attempts made before completion of the survey or whether the household initially refused to participate. We compare the point estimates we actually obtained with those we would have obtained with a less aggressive protocol and subsequent lower response rate. We also simulate what the effective sample sizes would have been if less aggressive protocols were followed. Principal Findings. Unweighted bivariate analyses reveal many differences between early completers and those requiring more contacts and between those who initially refused to participate and those who did not. However, after making standard poststratification adjustments, no statistically significant differences were observed in the key health variables we examined between the early responders and the estimates derived from the full reporting sample. Conclusions. Our findings demonstrate that for the surveys we examined, larger effective sample sizes (i.e., more statistical power) could have been achieved with the same amount of funding using less aggressive calling protocols. For some studies, money spent on aggressively pursuing high response rates could be better used to increase statistical power and/or to directly examine nonresponse bias.

Original languageEnglish (US)
Pages (from-to)1324-1344
Number of pages21
JournalHealth Services Research
Volume45
Issue number5 PART 1
DOIs
StatePublished - Oct 2010

Fingerprint

Health Surveys
Telephone
Sample Size
Social Adjustment
Information Storage and Retrieval
Health
Demography
Costs and Cost Analysis
Surveys and Questionnaires

Keywords

  • drug use
  • health care access
  • health insurance
  • Health survey
  • response rates
  • survey methods

ASJC Scopus subject areas

  • Health Policy

Cite this

Davern, M., McAlpine, D., Beebe, T. J., Ziegenfuss, J., Rockwood, T., & Call, K. T. (2010). Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys. Health Services Research, 45(5 PART 1), 1324-1344. https://doi.org/10.1111/j.1475-6773.2010.01128.x

Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys. / Davern, Michael; McAlpine, Donna; Beebe, Timothy J.; Ziegenfuss, Jeanette; Rockwood, Todd; Call, Kathleen Thiede.

In: Health Services Research, Vol. 45, No. 5 PART 1, 10.2010, p. 1324-1344.

Research output: Contribution to journalArticle

Davern, M, McAlpine, D, Beebe, TJ, Ziegenfuss, J, Rockwood, T & Call, KT 2010, 'Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys', Health Services Research, vol. 45, no. 5 PART 1, pp. 1324-1344. https://doi.org/10.1111/j.1475-6773.2010.01128.x
Davern, Michael ; McAlpine, Donna ; Beebe, Timothy J. ; Ziegenfuss, Jeanette ; Rockwood, Todd ; Call, Kathleen Thiede. / Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys. In: Health Services Research. 2010 ; Vol. 45, No. 5 PART 1. pp. 1324-1344.
@article{8ee2dc24189c4fbbaadbfe95a505bb1e,
title = "Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys",
abstract = "Objective. To examine the impact of response rate variation on survey estimates and costs in three health telephone surveys. Data Source. Three telephone surveys of noninstitutionalized adults in Minnesota and Oklahoma conducted from 2003 to 2005. Study Design. We examine differences in demographics and health measures by number of call attempts made before completion of the survey or whether the household initially refused to participate. We compare the point estimates we actually obtained with those we would have obtained with a less aggressive protocol and subsequent lower response rate. We also simulate what the effective sample sizes would have been if less aggressive protocols were followed. Principal Findings. Unweighted bivariate analyses reveal many differences between early completers and those requiring more contacts and between those who initially refused to participate and those who did not. However, after making standard poststratification adjustments, no statistically significant differences were observed in the key health variables we examined between the early responders and the estimates derived from the full reporting sample. Conclusions. Our findings demonstrate that for the surveys we examined, larger effective sample sizes (i.e., more statistical power) could have been achieved with the same amount of funding using less aggressive calling protocols. For some studies, money spent on aggressively pursuing high response rates could be better used to increase statistical power and/or to directly examine nonresponse bias.",
keywords = "drug use, health care access, health insurance, Health survey, response rates, survey methods",
author = "Michael Davern and Donna McAlpine and Beebe, {Timothy J.} and Jeanette Ziegenfuss and Todd Rockwood and Call, {Kathleen Thiede}",
year = "2010",
month = "10",
doi = "10.1111/j.1475-6773.2010.01128.x",
language = "English (US)",
volume = "45",
pages = "1324--1344",
journal = "Health Services Research",
issn = "0017-9124",
publisher = "Wiley-Blackwell",
number = "5 PART 1",

}

TY - JOUR

T1 - Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys

AU - Davern, Michael

AU - McAlpine, Donna

AU - Beebe, Timothy J.

AU - Ziegenfuss, Jeanette

AU - Rockwood, Todd

AU - Call, Kathleen Thiede

PY - 2010/10

Y1 - 2010/10

N2 - Objective. To examine the impact of response rate variation on survey estimates and costs in three health telephone surveys. Data Source. Three telephone surveys of noninstitutionalized adults in Minnesota and Oklahoma conducted from 2003 to 2005. Study Design. We examine differences in demographics and health measures by number of call attempts made before completion of the survey or whether the household initially refused to participate. We compare the point estimates we actually obtained with those we would have obtained with a less aggressive protocol and subsequent lower response rate. We also simulate what the effective sample sizes would have been if less aggressive protocols were followed. Principal Findings. Unweighted bivariate analyses reveal many differences between early completers and those requiring more contacts and between those who initially refused to participate and those who did not. However, after making standard poststratification adjustments, no statistically significant differences were observed in the key health variables we examined between the early responders and the estimates derived from the full reporting sample. Conclusions. Our findings demonstrate that for the surveys we examined, larger effective sample sizes (i.e., more statistical power) could have been achieved with the same amount of funding using less aggressive calling protocols. For some studies, money spent on aggressively pursuing high response rates could be better used to increase statistical power and/or to directly examine nonresponse bias.

AB - Objective. To examine the impact of response rate variation on survey estimates and costs in three health telephone surveys. Data Source. Three telephone surveys of noninstitutionalized adults in Minnesota and Oklahoma conducted from 2003 to 2005. Study Design. We examine differences in demographics and health measures by number of call attempts made before completion of the survey or whether the household initially refused to participate. We compare the point estimates we actually obtained with those we would have obtained with a less aggressive protocol and subsequent lower response rate. We also simulate what the effective sample sizes would have been if less aggressive protocols were followed. Principal Findings. Unweighted bivariate analyses reveal many differences between early completers and those requiring more contacts and between those who initially refused to participate and those who did not. However, after making standard poststratification adjustments, no statistically significant differences were observed in the key health variables we examined between the early responders and the estimates derived from the full reporting sample. Conclusions. Our findings demonstrate that for the surveys we examined, larger effective sample sizes (i.e., more statistical power) could have been achieved with the same amount of funding using less aggressive calling protocols. For some studies, money spent on aggressively pursuing high response rates could be better used to increase statistical power and/or to directly examine nonresponse bias.

KW - drug use

KW - health care access

KW - health insurance

KW - Health survey

KW - response rates

KW - survey methods

UR - http://www.scopus.com/inward/record.url?scp=77956562015&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77956562015&partnerID=8YFLogxK

U2 - 10.1111/j.1475-6773.2010.01128.x

DO - 10.1111/j.1475-6773.2010.01128.x

M3 - Article

C2 - 20579127

AN - SCOPUS:77956562015

VL - 45

SP - 1324

EP - 1344

JO - Health Services Research

JF - Health Services Research

SN - 0017-9124

IS - 5 PART 1

ER -