SUPPORTING STATEMENT A 0971_ renewal_review_v12

Department of Commerce

United States Census Bureau

OMB Information Collection Request

Generic Clearance for Census Bureau Field Tests and Evaluations

OMB Control Number 0607-0971

Part A – Justification

Question 1. Necessity of the Information Collection

The U.S. Census Bureau is committed to conducting research towards census and survey operations that costs less while maintaining high quality results. The Census Bureau requests a reinstatement, with change, of our previous OMB approval to conduct a series of studies to research and evaluate how to improve data collection activities for data collection programs at the Census Bureau. These studies will explore how the Census Bureau can improve efficiency, data quality, and response rates and reduce respondent burden in future census and survey operations, evaluations and experiments.

This information collection will operate as a generic clearance. The estimated number of respondents and annual reporting hours requested cover both the known and yet to be determined tests. A generic clearance is needed for these tests because though each share similar methodology, the explicit details of each test to be performed has yet to be determined. Once information collection plans are defined, they will be submitted on an individual basis in order to keep OMB informed as these tests progress.

The Census Bureau plans to test the use of new and improved data collection techniques for self-enumeration and interviewer data-collection tasks surrounding and following the ongoing census and survey operations. The research and evaluation may include: research on public outreach, messaging, and similar invitations to participate in surveys; developing alternative enumeration or follow-up questionnaires; usability issues; conducting interviews or debriefings; and non-English language training and interviews. The questions asked in these studies will be typical census or survey questions and questions related to that content or attitudinal, opinion and satisfaction questions related to responding to Census Bureau surveys.

The Census Bureau plans to conduct these tests in small geographic areas or national samples over the next three years. We will follow the protocol of past generic clearances: at least 30 days before the scheduled start date of each field test, we will provide OMB with a detailed background on the activity, estimates of respondent burden, and samples of pertinent forms and/or questions.

In addition to the Field Tests listed above, economic programs are developing research activities, including field tests, to support survey program re-engineering, including, but not limited to, recommendations published by the National Academy of Sciences consensus study report on “Re-engineering the Census Bureau’s Annual Economic Surveys.¹ ” In addition, economic programs, together with their reimbursable partners, co-sponsor a number of survey collections using Title 13 authority, and plan to conduct field tests prior to production, to ensure effective data collection instruments and procedures.

Question 2. Needs and Uses

This research program is for respondent communication, questionnaire and procedure development and evaluation purposes. We will use data tabulations to evaluate the results of questionnaire testing. The information collected in this effort will not be released as official statistics. However, summarized results might be released in methodological reports, or as a methodological appendix or footnote in a report containing data from a larger data collection effort. The results of this research may also be prepared for presentation at professional meetings or publication in professional journals.

Information quality is an integral part of the pre-dissemination review of the information disseminated by the Census Bureau (fully described in the Census Bureau's Information Quality Guidelines). Information quality is also integral to the information collections conducted by the Census Bureau and is incorporated into the clearance process required by the Paperwork Reduction Act.

Question 3. Use of Information Technology

The information will be collected through observations, self-response, face-face interviews, and/or telephone interviews. Approximation 75 percent of the data covered by this generic clearance will be collected through electronic instruments, with the remainder on paper questionnaires.

Question 4. Efforts to Identify Duplication

This research does not duplicate any other questionnaire design work being done by the Census Bureau or other Federal agencies. The purpose of this clearance is to stimulate additional research, which would not be done under other circumstances due to time constraints. This research may involve collaboration with staff from other agencies that are sponsoring the surveys conducted by the Census Bureau. The research may also involve joint efforts with staff from other Federal Statistical Agencies. All efforts would be collaborative in nature, and no duplication in this area is anticipated.

To the maximum extent possible, we will make use of previous information, reviewing results of previous evaluations of survey data before we attempt to revise survey communication materials, invitations, or questionnaires. However, this information is not sufficient to refine our census and survey invitations or questionnaires without conducting additional research which can be conducted concurrently with census and survey production work.

This generic clearance request is an explicit addition to the current generic clearance for pretesting (Number 0607-0725) that allows small-scale cognitive and usability testing research and the generic clearance for internet pretesting (Number 0607-0978) that allows medium-scale online pretesting. This generic clearance is different from the previous clearances in two ways. First, this clearance alone can collect data using the mandatory authorities provided for Decennial Census operations and testing, as well as for the Economic Census, the Census of Governments, annual economic survey programs, selected sub-annual economic programs, and/or reimbursable surveys co-sponsored with the Census Bureau or otherwise conducted with mandatory authority. These mandatory authorities have been repeatedly shown to increase response rates. Although we do not expect to invoke the mandatory authority on most studies, in some cases, where we need to study effects on response rate or to ensure adequate response to assess data quality reflecting the effectiveness of survey questions and collection instruments, it may be necessary to use the mandatory authority. This will be secondary to using the voluntary authority as the primary collection authority. Second, this clearance focuses on field tests and evaluations, which typically occur concurrently with production census and survey work, in order to efficiently use existing resources and processes, rather than preceding census and survey work as is the case with the previously mentioned clearances. However, where no concurrent collections exist, such as for new initiatives or re-engineered surveys expected to carry mandatory authority, separate field tests may be needed to ensure adequate data quality and appropriate collection procedures prior to undertaking a full-scale survey.

Question 5. Minimizing Burden

To the extent possible, and as appropriate to address research questions, sample sizes for this research will be based on power analyses that identify the minimum number of cases needed to be able to detect substantive and statistically significant differences in treatment. This will minimize the number of respondents needed to test improvements to questionnaire design, test field data collection procedures and new technologies. For field tests involving households and individuals, we will attempt to exclude housing units that have been in sample for ACS or the 2019 Census Test for any experiment that is a follow-up to the 2020 Census. For economic census/survey program field tests, sample sizes and selection may be targeted based on pre-determined criteria to support the goals of the field test; these will be detailed in each individual clearance request.

Question 6. Consequences of Less Frequent Collection

Many responses to these activities are on a one-time basis. Responses may be used in future research studies that build upon the results of the current production census and survey work. Failure to research and test new methods could result in increased costs and less accuracy of future census and survey operations.

Question 7. Special Circumstances

There are no special circumstances preventing the meeting of all OMB Guidelines.

Question 8. Consultations Outside the Agency

The Census Bureau regularly consults with outside parties in its census and survey planning. For example, consultation with staff from other Federal agencies that sponsor surveys conducted by the Census Bureau will occur in conjunction with the testing program for the individual survey. Consultation with staff from other Federal laboratory facilities may also occur as part of joint research efforts. These consultations will include discussions concerning potential response problems, clarity of questions and instructions, and other aspects of respondent burden. Additional efforts to consult with potential respondents to obtain their views on the availability of data, clarity of instructions, etc., may be undertaken as part of the testing that is conducted under this clearance.

The Census Bureau published a notice in the Federal Register on October 16, 2018 soliciting public comments on our plans to submit this request (83 FR 52190). We received three comments. One, dated October 18, 2018, expressed comments from the National Federation of Independent Business (NFIB). These comments suggested that the Census Bureau was not adhering to the requirements of the Paperwork Reduction Act. In response, the Census Bureau clarified the intent of this Generic Clearance package and the associated supporting statements. Another, dated December 17, 2018, from the Arab American Institute urged the Census Bureau to research the impacts of moving to an internet-first response on Hard-to-Count populations, like Arab Americans. The Census Bureau notes that this is in scope for the 2020 Census Privacy and Confidentiality Evaluation. This letter also mentions incorporating MENA as a response category in the census and hiring Arabic-speaking interviewers. The latter two comments were passed along to appropriate areas within the Census Bureau. The third comment, dated August 26, 2019, was critical of the data collection efforts.

Question 9. Paying Respondents

In general, respondents will not be paid or provided any cash incentive for their participation. In certain situations, a token incentive may be proposed. Details of this incentive will be outlined with each individual request submitted by the Census Bureau to OMB.

Question 10. Assurance of Confidentiality

All respondents who participate in research under this clearance will be informed that the information they provide will not be made available to anyone who is not a Census employee or does not have Special Sworn Status in any way that would personally identify an individual, household, business or other organization. They will also be informed that their participation is either mandatory or voluntary, depending on the collection authority. This disclosure will be made prior to any data collection.

Data collection for this project is authorized under the authorizing legislation for the questionnaire being tested. This may be Title 13, Sections 91, 131, 141, 161, 181, 182, 193, and 301 for Census Bureau sponsored surveys, and Title 13 and 15 for surveys sponsored by other Federal agencies. Other titles may be referenced depending on the nature and source of questionnaires being pretested during the course of the clearance. Appropriate legal authority for pretesting other survey questionnaires, yet to-be-determined, will be referenced in each individual request for a field test under this generic clearance. Per the Federal Cybersecurity Enhancement Act of 2015, data are protected from cybersecurity risks through screening of the systems that transmit data.

Question 11. Justification for Sensitive Questions

For most demographic surveys conducted by the Census Bureau, most of the questions included on the questionnaires are not of a sensitive nature and are not anticipated to pose a problem to respondents.  For many establishment surveys conducted by the Census Bureau, most questions may be answered based on business/organizational records; this may impose minimal reporting burden in excess of that experienced under current economic data collections. However, it is possible that some questions could be perceived as sensitive, or, in establishment surveys, requested information may be considered by the business to be strategic or proprietary.  One of the purposes of testing included here is to identify such questions, determine sources of sensitivity and/or reporting burden, and alleviate them insofar as possible in future production surveys. Additionally, confidentiality pledges will be emphasized to relieve respondent concerns.

Question 12. Estimate of Hour Burden

While this clearance includes prospective field tests yet to be determined, this chart shows the calculation of the estimated burden for each known demographic survey project.

Title	Description	Burden Hours	N	Total burden hours
Privacy and Confidentiality CATI/CAPI Study	Data will be collected in two modes: telephone and in-person. Self-respondents to the decennial census will be interviewed by telephone and NRFU census respondents will be interviewed in-person and may be offered a small incentive. The survey will be conducted in both English and Spanish.	15 minutes	105,000	26,250
Privacy and Confidentiality Focus Group Study	The study will include four components: 1) Qualitative interviews with cultural experts recommended by the partnership program, 2) focus groups with trusted messengers, 3) observations of partnership events, and 4) focus groups with respondents. We will leverage the partnership program for this study since people who might have confidentiality concerns might not speak directly with government employees. The partnership program seeks to partner with people in hard-to-count communities who are already trusted in those communities. Researchers will conduct focus groups with members of hard-to-count populations who might have privacy or confidentiality concerns. Focus groups will be conducted in English and other languages. This set of focus groups will allow us to assess the impact of privacy and confidentiality concerns for respondents who might not be captured in the larger quantitative survey due to the sampling strategy and sample size constraints. 18 groups (max 12 per group) 15 interviews	90 minutes	231 participants	347
Hard to Count: Language	For the 2020 NRFU observation study, initially a potential 15-20 sites will be chosen based on meeting criteria of demographic characteristics associated with under-coverage and non-English speaking households. The criteria for potential sites will include percentage of households speaking each of the seven target languages, percentage of population that identifies as non-White, percentage of households in poverty, percentage of complex households, percentage of households with a child under 5 present, as well as other characteristics that previous research has shown are correlated with under-coverage. Although we cannot guarantee how many interviews will actually occur and how many of those interviews will be in the target language, about 25 observations for each of the 6 languages (150 interviews) and at least 150 English interviews will be the goal. During the 2020 Census, Census Bureau researchers and contractors will follow bilingual NRFU interviewers in these sites to observe interview interactions and conduct 5-minute follow-up interviews with the target populations. Ideally, HQ staff and contractors will be able to audio record the entire NRFU interview, but at the very least, they can record the follow-up portion of the interview in addition to taking notes.	5 minutes	300 households	25
Undercount of Young Children	Conduct focus groups and cognitive interviews with individuals in households at-risk of erroneously omitting young children (under age 5) on the census form. Testing to be conducted in three modes (paper, internet self-response and interviewer-administered non-response follow-up). A total of 18 focus groups and 90 interviews is planned, Sample from post-enumeration survey administrative record non-matches to the 2020 census	Focus groups (at 90 minutes each) Cognitive interviews (at 60 minutes each)	360 focus group participants 180 cognitive interview participants	540 hours 180 hours
National Survey of College Graduates	Revising and testing for potential mode differences between web and CATI/CAPI data Collection	30 minutes	120,000	60,000 hours
Re-engineering the Census Bureau’s Annual Economic Surveys	Census Bureau economic programs are currently considering conducting up to six field tests, averaging 500 cases per test, with an expected average reporting burden of 2-3 hours per case. In addition, each field test would be followed up with debriefing interviews with an average of 60 responding businesses, averaging one hour each.	2.5 hours per case 1 hour/ debriefing	6 field tests X 500 cases = 3000 cases 6 field tests X 60 debriefing interviews = 360 debriefing interviews Plus an additional 100 hours recruiting for debriefing	7,960 hours
Subtotal		25	229,431	95,302
	Yet Unknown	30	111,942	55,971
		27	341,373	151,273
Total Burden		minutes/ participant	Participants	Hours

Specific plans for each economic survey program’s field test, including pre-specified selection criteria, such as business size and industry, will be provided in Individual Request for Clearance documents submitted to OMB prior to conducting the pretest.

The total estimated respondent burden is 151,273 hours in total for the period from April 2019 to March 2022. A variety of activities will be conducted under this clearance and the exact dates and number of households, businesses/organizations, and respondents for all activities are estimated at this time. The estimate of hour burden is based on our current expectations and planned work.

Question 13. Estimate of Cost Burden

There are no costs to demographic survey respondents other than that of their personal time to respond. For field tests of establishment surveys, we do not expect respondents to incur any costs other than that of their time to respond. The information requested is of the type and scope normally carried in company records and no special hardware or accounting software or system is necessary to provide answers to prospective information collections.

Question 14. Cost to Federal Government

The annual costs to the Federal Government associated with each specific test will be provided in the Individual Request for Clearance document submitted to the OMB that will precede the test.

Question 15. Reason for Change in Burden

We are requesting an increase in hours from 16,667 to 47,771 annually because we incorporated the evaluation needs for the 2020 Census into this request. Due to the timeline of this Generic Clearance and one needed for 2020 Census evaluations, it was most efficient to combine what otherwise could have been two separate requests.

We have also added 7,960 burden hours to be available for field tests of economic surveys, as needed, to meet the scope and intent of re-engineering economic annual programs per the recommendations of the National Academy of Sciences panel as well as any potential associated impact on collection of the economic census, as well as other field test collections for reimbursable surveys.

Question 16. Project Schedule

An overall schedule is above in the section on respondent burden. A schedule for completing each activity will be provided in the Individual Request for Clearance document submitted to the OMB that will precede the activity.

Question 17. Request to Not Display Expiration Date

No exemption is requested.

Question 18. Exceptions to the Certification

There are no exceptions to the certification.

1 URL link: https://www.nap.edu/catalog/25098/reengineering-the-census-bureaus-annual-economic-surveys

6