Applying Quasi-Experimental Methods to Measure the Impact of Unanticipated Mid-Field Events on Political Attitudes
Submission ID: 5465

Date: Thursday, 3:45 PM to 5:15 PM 
Session: Session F: T3:45 - 5:15 PM

Primary Presenter

Michael Jackson, SSRS

Additional Authors or Round Table Presenters

Jennifer Agiesta, CNN ,

, ,

, ,

, ,

, ,

, ,

, ,

, ,

Abstract

Recent polling errors have prompted exploration of alternatives to random digit dialing for political polling, with the goal of mitigating differential nonresponse. One alternative is address-based sampling (ABS). Advantages of ABS include the ability to offer multiple modes, provide pre- and post-incentives, and use linked auxiliary data to direct follow-up efforts to underrepresented subgroups.

A tradeoff is that a mixed-mode ABS with mailing components requires a relatively long field period. This may limit the suitability of ABS for measuring political outcomes such as presidential approval, which are sensitive to fast-breaking external news events. Such events are often unanticipated and therefore difficult to design for in advance. Often, when a high-profile news event occurs during an ABS field period, the only option for assessing its impact is an “after-the-fact” comparison between pre- and post-event respondents.

Therefore, in assessing the utility of ABS for political polling, an important question is whether such “pre-post” comparisons can validly estimate the impact of mid-field events. The inferential challenge is that later respondents may differ from earlier respondents on characteristics that are unrelated to the event but could confound observed changes in political attitudes.

Using an ABS poll whose field period overlapped with the August 2021 fall of Kabul, Afghanistan, this presentation will assess methods of estimating the impact of high-profile, mid-field events on presidential approval and related outcomes.

Simple demographic reweighting will be compared to quasi-experimental methods drawn from the causal inference literature, including propensity score and regression discontinuity approaches. We will evaluate the extent to which these methods can control for underlying differences between early and late respondents and obtain plausibly unconfounded estimates of event-driven changes in attitudes.

Results will provide further insight into the tradeoffs of using ABS for political polling and the types of questions that can be answered with such samples.