Evaluating the Effectiveness of Pressure-Sealed Mailing Materials in a Web-Push Design
Submission ID: 6078

Date: Thursday, 10:15 AM to 11:45 AM 
Session: Session D: T10:15 - 11:45 AM

Primary Presenter

Jiashan Cui, American Institutes for Research

Additional Authors or Round Table Presenters

Mahi Megra, American Institutes for Research ,

Tzu-Jou Wan, American Institutes for Research ,

Rebecca Medway, American Institutes for Research ,

, ,

, ,

, ,

, ,

, ,

Abstract

The results of the 2017 National Household Education Survey web test (NHES:2017) suggested a pressure-sealed envelope could be a promising option as a survey reminder (Medway et al. 2018). However, the NHES:2017 administration was web-only and thus not entirely comparable to a mixed mode web-push design. In addition, it was not a randomized experiment, in which a reminder postcard and pressure-sealed envelope are tested simultaneously within the same data collection. The NHES:2019 administration predominately used a web-push design and experimented with the use of a pressure-sealed envelope as a survey reminder. For this experiment, sample members in the baseline group received a regular reminder postcard, while the sample members in the treatment group received a pressure-sealed envelope. Both types of survey reminders were sent a week after the initial screener package. The NHES:2019 pressure-sealed envelope included the web survey URL and the household’s unique web login credentials, but the regular reminder postcard did not (because the postcard format did not allow for sufficient protection of this information). It was hypothesized that the pressure-sealed envelope’s ability to include the survey URL and the household’s web login credentials would increase internet response (Census, 2018).

We will present, within the context of a web-push design, whether a pressure-sealed envelope reminder motivated survey participation. The results show that it is more effective to send a pressure-sealed envelope (a 4-percentage-point difference). We also evaluated the effect on response rates by selected household characteristics. There were a few subgroups for which sending a pressure-sealed envelope led to a somewhat larger-than-average increase in response relative to the reminder postcard. Our presentation will provide insights into the effectiveness of pressure-sealed mailing materials in a web-push design.