Pick Me! Incentivizing Web Response in a Concurrent Mixed-Mode Design
Submission ID: 5586

Date: Friday, 1:15 PM to 2:45 PM 
Session: Session I: F1:15 - 2:45 PM

Primary Presenter

Rebecca Medway, American Institutes for Research

Additional Authors or Round Table Presenters

Mahi Megra, American Institutes for Research ,

, ,

, ,

, ,

, ,

, ,

, ,

, ,

Abstract

Offering both web and paper modes of response is an attractive – and increasingly employed – design feature for maximizing response rates, minimizing bias, and controlling costs. However, there are challenges to employing such designs. While researchers often prefer web response, sample members may prefer paper (Shih & Fan 2008). Offering both modes concurrently may reduce the response rate (Medway & Fulton 2012) but offering them sequentially may prematurely dissuade sample members from responding who would have done so if they knew the second mode would be offered. Biemer et al. (2018) reported promising results of a “choice plus” protocol that addresses some of these challenges; it offers both response modes concurrently but incentivizes web response by offering a larger contingent incentive for web response than for paper response.

This presentation reports the results of an experiment that builds on these findings. Households sampled for the 2019 National Household Education Survey (NHES:2019) were randomly assigned to: (1) choice plus, (2) web-push (sequential web-then-paper), or (3) paper-only. Within the choice plus condition, households were randomly assigned to receive a $10 or $20 contingent incentive for web response. Compared to web-push and paper-only, choice plus resulted in a higher response rate, both overall and among subgroups that typically have lower-than-average NHES response rates. Compared to web-push, choice plus also increased response to early mailings but did not increase the percentage of responses by web – the increase in response was almost entirely by paper. Among those in the choice plus condition, $20 led to more response to early mailings and more response by web than did $10. However, choice plus also was more expensive than the other conditions, particularly when the $20 incentive was used. This presentation will be of interest to practitioners interested in maximizing response rates in mixed-mode surveys.