OMB Information Collection Request
Supporting Statement B
U.S. Department of Commerce
U.S. Census Bureau
Household Pulse Survey
During the Coronavirus Pandemic
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
As noted in Part A, the Household Pulse Survey was initially fielded as a proof of concept to test the Federal statistical system’s ability to produce data in near real-time in the face of a national emergency. The survey is conducted under the auspices of the U.S. Census Bureau’s Experimental Data Products Series (see https://www.census.gov/data/experimental-data-products.html).
Planning for the survey began March 23, 2020, and with OMB emergency clearance granted on April 19, 2020 for 3 months of data collection (OMB No. 0607-1013, Exp. 7/31/2020), the Census Bureau launched the Household Pulse Survey on April 23, 2020. Data were first released on May 20, 2020, with updated releases made subsequently on a weekly basis. Since then, the data have been used widely to guide response and recovery efforts on the part of Federal agencies and state governments. Given the continuing nature of the pandemic, the Census Bureau sought, and OMB approved, an extension to data collection for the balance of the 180 days allowable for emergency clearance under 5 CFR Part 1320, Section 1320.13, Emergency Processing. The Census Bureau then put forward a regular (non-emergency) Information Collection Request (ICR) for OMB review on September 17, 2020. OMB approved this ICR on October 30, 2020 for three years (OMB No. 0607-1013; expiration October 31, 2023). Given information about the trajectory of the pandemic in early December, the Census Bureau submitted a non-substantive request to OMB on December 22, 2020 to request approval for continuing collection of the current Phase 3 instrument through January and February, 2021. This Information Collection Request (ICR) is submitted for the purposes of requesting a revision to an Existing Collection for a revised, Phase 3.1 questionnaire.
For ease of reference, we refer to the initial approval by OMB to conduct the Household Pulse Survey as “Phase 1” (April – July, 2020), and the period approved for the balance of the emergency clearance as “Phase 2” (August – October, 2020). “Phase 3” is in reference to normal clearance granted starting October 30, 2020 through October 30, 2023. This ICR requests approval to launch “Phase 3.1” with a revised questionnaire beginning in March, 2021.
Given the rapid response nature of this survey, the Census Bureau continued to make adjustments in content and methods throughout Phase 1, documenting those through the submission of non-substantive change requests to OMB. As data collection progressed, there was increasing demand from other Federal agencies to revise or add content to the Household Pulse questionnaire to produce data in support of their own missions and response to the pandemic. Rather than change the survey on a constant basis, the Census Bureau consolidated those requests into one comprehensively revised questionnaire, which was launched as Phase 2.
The Census Bureau’s approach to Phase 3 was similar to Phase 2, including the content, contact strategies, mode, data collection cycle and sampling approach. However, we continued to monitor data needs over the course of the pandemic and its recovery to determine if revisions (including additions and deletions of items) were needed. For Phase 3.1, upon review of Household Pulse Survey instrument, the Bureau identified questions for which relevancy or utility is declining, and to address requests for new information emanating from previous public comment and consultation with other Federal agencies. Additional questions address the following new topics: disability, child health access, telehealth and childcare.
As in Phases 1, 2 and 3, we will provide all information about sampling, weighting, post-survey processing, and cognitive testing results on our website to maximize transparency for the public. This transparency is especially important given the value to decision makers of having rapid bi-weekly state-by-state estimates of the health, social, and economic characteristics of their package.
Universe and Respondent Selection
The universe for the Household Pulse Survey is the approximately 140,000,000 housing units in the United States. For operational efficiency, only housing units linked to one or more email addresses or cell phone numbers will be eligible for the sample. One respondent from each housing unit is selected to respond for him or herself. The sample for each data collection cycle is approximately 1,165,000 housing units.
In Phase 1, the Census Bureau observed response rates in the range 6-7 percent for households who were invited to participate for the first time and asked to respond in a one-week period. In Phase 2, we extended the response period to 13 days and sent additional reminders to respondents. These efforts had the effect of increasing response rates to a range of 8.1 to 10.3 percent. In Phase 3, we continued with the 13-day response period and observed a response rate of 9%. Based on this experience, we are assuming a 9 percent response rate for Phase 3.1 and expect to receive 105,000 responses each data collection cycle. Sample from each data collection cycle will be independent from the prior cycles.
Procedures for Collecting Information
Housing units linked to one or more email addresses or cell phone numbers are eligible for the sample. These housing units are stratified by state and the top 15 metropolitan areas. For an estimate of 40 percent, the sample size was determined to produce a margin of error of 2.8 percent for most states and 3.3 percent for the 11 smallest states with eligible sample units (Alaska, Delaware, District of Columbia, Hawaii, Maine, Montana, North Dakota, Rhode Island, South Dakota, Vermont, and Wyoming) with 90 percent confidence. The sample was then allocated to states and metropolitan areas proportionally, using the number of housing units as the measure of size. Samples cases will be matched to the Census Bureau’s Contact Frame to assign email addresses and cell phone numbers. See Attachment A for information on the Contact Frame.
A sampled address will have up to five associated email addresses and up to five cell phone numbers. Starting on the first Wednesday of the data collection period, an email invitation will be sent in the morning and an SMS invitation in the afternoon to the first set of contact information for the unit. If a response is not received by Wednesday evening, on Thursday an email and SMS will be sent to the second set of contact information for the unit (if it exists). This process will be repeated on weekdays through the fifth contact. If there is still no response received, the process will be repeated one time, cycling through the contact information again. Responses are due by the second Monday in the data collection period (12 days after the initial invitation).
The Census Bureau will conduct this information collection online using Qualtrics as the data collection platform. Qualtrics is currently used at the Census Bureau for research and development surveys and provides the necessary agility to deploy the Household Pulse Survey quickly and securely. It operates in the Gov Cloud, is FedRAMP authorized at the moderate level, and has an Authority to Operate from the Census Bureau to collect personally identifiable and Title 13-protected data.
Attachment B sets forth details on the Census Bureau’s weighting plan, which will use a multi-step process to help reduce nonresponse bias in the estimates.
Methods to Maximize Response
The survey is designed to meet the goal of accurate and timely bi-weekly estimates. It will be conducted by an internet questionnaire, with links sent by email and SMS text message. Up to five email addresses and cell phone numbers will be used for each unit to increase response for nonrespondents. One reminder will be sent to each piece of contact information if a response is not received. These modes are expected to yield response rates much lower than traditional surveys. The benefits are implementation efficiency, cost, and timeliness of responses.
Standard errors will be large but will be available for data users to understand the quality of the estimates. Nonresponse bias is likely to be an issue, but measures such as the demographic distribution of the survey respondents compared to benchmarks will be produced for data users to consider in their analyses. Weighted distributions of key demographic covariates are included in the detailed tables, and are accompanied by companion standard error tables. These distributions of these characteristics can be compared with other federal and non-federal data sources. The microdata allows for the comparison of weighted and unweighted estimates and evaluations of gaps in response. Source and Accuracy documents include coverage rates both before and after weighting raking procedures, and are released with each data cycle release. There are no other known data sources that can provide comprehensive, relevant, and timely information. At the very least, this survey will provide non-anecdotal evidence about the effects of the pandemic on the population even if the quality is lower than would be traditionally acceptable.
Source and Accuracy statements and other technical documentation is available at https://www.census.gov/programs-surveys/household-pulse-survey/technical-documentation.html.
Testing of Procedures
Prior to Phase 1, the Center for Behavioral Science Methods completed a thorough expert review that consisted of nine independent experts reviewing the questionnaire independently, followed by a consensus meeting to discuss comments and come up with recommendations. The recommendations were then passed to the Demographic Directorate, who accepted most of the expert recommendations and responded with subject-matter justifications for those recommendations that were not accepted. In the end, the questionnaire comprised a consensus of the expert survey methodologist recommendations and the subject matter needs. A similar process was followed prior to Phase 2 and Phase 3 as the Census Bureau considered questionnaire revisions to keep up with evolving data needs.
We will take a similar course for Phase 3.1: feedback from Federal agency partners, data users and other stakeholders on diminishing, changing or emerging data needs will inform changes to the questionnaire.
Given the rapid response nature of this effort and in keeping with the agile approach in which the Household Pulse Survey is being deployed, the Census Bureau, in partnership with BLS and NCHS, concurrently launched a web probing study with participants who had opted in to participate in a research study with the Census Bureau via email. These sampled participants were sent email invitations to the survey with embedded probes. Two rounds of web probing were conducted in Phase 1, and an additional round in Phase 2 and Phase 3. The Household Pulse Survey Team used results from these studies to refine content and cut questions to stay within a 20-minute limit for the survey.
Questions identified as priorities for testing in all rounds include those that were developed specifically to address the pandemic; items developed for potential future deployment to understand households’ decision-making in light of receiving economic stimulus payments; and questions deemed to be overly complex from a cognitive standpoint, likely as a consequence of asking agencies contributing content to keep the number of items to a minimum to reduce burden (e.g., questions on food security, questions on educational disruption).
Cognitive testing will be conducted in preparation for Phase 3.1. This will be submitted separately under the Census Bureau’s Generic Clearance for Pretesting (0607-0725).
The Census Bureau will release these data under the auspices of its Experimental Statistical Product Series. Information on the Series is available at https://www.census.gov/data/experimental-data-products.html#.
Contacts for Statistical Aspects and Data Collection
Statistical Design:
Anthony Tersine
Demographic Statistical Methods Division
Demographic Programs Directorate
Data Collection/Survey Design:
Jason Fields
Social Economic and Housing Statistics Division
Demographic Programs Directorate
Jennifer Hunter Childs
Center for Behavioral Science Methods
Associate Director Research and Methodology
Statistical Analysis:
David Waddington
Social Economic and Housing Statistics Division
Demographic Programs Directorate
Attachments
Census Bureau Contact Frame
Household Pulse Survey: Weighting Approach
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Dumas, Sheleen (Federal) |
| File Modified | 0000-00-00 |
| File Created | 2021-02-04 |