Evaluating An Alternative Raking Strategy for A National Household Survey Program
Submission ID: 5716

Date: Thursday, 1:15 PM to 2:45 PM 
Session: Session E: T1:15 - 2:45 PM

Primary Presenter

Tzu-Jou (Carol) Wan, American Institutes for Research

Additional Authors or Round Table Presenters

Jiashan Cui, American Institutes for Research ,

Rachel Hanson, American Institutes for Research ,

Danielle Battle, American Institutes for Research ,

Michael Jackson, SSRS ,

, ,

, ,

, ,

, ,

Abstract

Public-use datasets such as the American Community Survey (ACS) are often used to create population benchmarks for raking. However, researchers often do not consider how differences, even seemingly subtle, in the measurement of weighting variables can affect the accuracy of the resulting weights. For example, the National Household Education Surveys Program of 2019 (NHES:2019) overestimated the number of kindergarteners and underestimated the number of 4-year-olds in the child population due to the misalignment in reference periods between the survey being raked (i.e. NHES:2019) and the benchmark survey (i.e. ACS).  An alternative raking strategy was developed to improve the precision of population estimates of children in the NHES:2019. The alternative raking strategy stacks the two child surveys in NHES:2019 together for the raking process when developing the weights. This strategy was expected to address the estimation issue by avoiding reliance on variables that are sensitive to the differing reference periods.

This study compared the population estimates of children in the NHES between the current and alternative raking strategies for the overall child population and by select demographic characteristics. Additionally, key survey estimates and associated standard errors in the surveys were evaluated between the two raking schemes for statistically significant differences. The results suggest that the alternative raking strategy mitigated the estimation issues in both surveys, with minimal disruptions to the cross-year trends of survey estimates. The study demonstrates the importance of consistently defined populations and weighting variables between the surveys being raked and the benchmark survey, and cautions on the use of demographic characteristics that could lead to extreme weighting adjustments for raking.