2018EndtoEndCensusTest SupportingStatementPart A_v3.1

Department of Commerce

United States Census Bureau

OMB Information Collection Request

2018 End-to-End Census Test – Peak Operations

OMB Control Number XXXX-XXXX

Part A. Justification

1. Necessity of the Information Collection

During the years preceding the 2020 Census, the U.S. Census Bureau will pursue its commitment to conducting a quality decennial census through rigorous research and testing of operations and systems. In 2018, the Census Bureau will be performing the 2018 End-to-End Census Test. This last major test before the 2020 Census is designed to (1) test and validate 2020 Census operations, procedures, systems, and field infrastructure to ensure proper integration and conformance with requirements, and (2) produce prototypes of geographic and data products.

The 2018 End-to-End Census Test will encompass operations and systems related to (1) Address Canvassing; (2) Optimizing Self-Response, including contact strategies, questionnaire content, and language support; (3) Update Leave, including technological and operational testing; (4) Nonresponse Followup, including technological and operational improvements; and (5) Group Quarters, including technological and operational testing. The operations other than Address Canvassing are referred to collectively as Peak Operations, because they are the enumeration data collection operations of the census. These operations are the focus of this package.

The Address Canvassing operation ((1) from the above), beginning in the summer of 2017, is the first operation in the 2018 End-to-End Census Test and was included in a separate Address Canvassing Operation package because of timing considerations.

New approaches to the design of the 2020 Census are classified into four key innovation areas. These areas have been the subject of Census Bureau testing this decade to identify methodological improvements, technological advances, and possibilities for new operational efficiencies over repeating the design of the 2010 Census. One of these innovation areas is Optimizing Self-Response, which is focused on improving methods for increasing the number of people who take advantage of self-response options.

There is a change in test site since the Presubmission Federal Register Notice was published. This test will occur within Providence County, Rhode Island. 

Optimizing Self-Response:

The 2018 End-to-End Census Test will include two different mailing strategies to optimize the rate at which the public self-responds to the decennial census, thereby reducing costs of the 2020 Census by decreasing the workload for following up at nonresponding units. Previous Census Bureau tests have evaluated a number of different contact strategies, including variations on the format, order, and timing of the mailings. This test will use both the Internet First and Internet Choice strategies to optimize self-response, and this approach is also currently planned for the 2020 Census.

Internet First is the primary mail contact strategy proposed for the 2020 Census and has been used in Census Bureau research and testing efforts since 2012. (In previous tests, this strategy was called Internet Push.) This strategy includes the mailing of a letter encouraging respondents to complete the questionnaire online and two follow-up reminders, after which a paper questionnaire is mailed to nonresponding housing units. A final reminder postcard is the last mailing.

Internet Choice includes a paper questionnaire in the first mailing, along with an invitation to complete the questionnaire online, providing a choice of internet or paper response from the beginning of the contact strategy. (Subsequent mailings are of the same number and type as the Internet First strategy.) This strategy is targeted to households in areas least likely to respond online, as indicated by a number of factors, including internet availability and historical census response rates.

In addition, the 2018 End-to-End Census Test provides the Census Bureau with an opportunity to enhance the user experience, performance, and functionality of the internet self-response instrument compared to previous census tests this decade.

The Census Bureau plans to produce a variety of metrics about self-response in the 2018 End-to-End Census Test including: overall self-response rates in each panel (Internet First and Internet Choice), response rates by mode of enumeration, device usage for online respondents, and demographics for each response mode. Note that the criteria for determining whether households are in the Internet First or Internet Choice strategy has not changed since the 2017 Census Test. The proportion of households in each strategy is different for this test as compared to the 2017 Census Test due to differences in the selection of cases for inclusion in the tests.

Nonresponse Followup:

The 2018 End-to-End Census Test will allow the Census Bureau to continue to refine, optimize, and assess the operational procedures and technical design of the Nonresponse Followup (NRFU) operation. NRFU is a field operation for determining housing unit status (occupied, vacant, or delete) and for gathering the enumeration data at addresses for which no self-response was received. This test will build on the results of previous field tests this decade where the NRFU operation has been conducted. In particular, NRFU is now a fully automated operation, whereas it was performed using paper materials in the 2010 Census.

For this test, the Census Bureau will examine:

• Operational procedures

  • Testing continued refinements to the field data collection instrument for enumeration, particularly where previous testing has shown potential problems in our question branching or issues with the interview software user interface. The Census Bureau will critically assess navigation within the instrument through debriefing sessions for field enumerators after operations complete.

  • Continuing refinement of administrative record modeling to determine the NRFU universe and reduce the NRFU workload overall, along with an assessment of the accuracy of administrative record modeling.

  • Continuing refinement of methods for enumerating multiunit structures, particularly identifying vacant households in multiunits with a minimal number of contact attempts, thereby minimizing respondent burden.

  • Continuing refinement of procedures for interviewing proxy respondents to gather information from hard-to-enumerate households.

  • Continuing refinement of processes used to detect and deter falsification by field enumerators.

  • Continuing evaluation of enumerator training procedures and materials, including both online training modules and classroom training.

  • Integrating a Field Verification assignment into the NRFU workload. The Field Verification cases are intended to verify the existence and location of certain types of self-responses that were received without a preassigned census identification code (called a User ID) and which are not currently in the Census Bureau’s address frame (known as the Master Address File).

  • Expanding the NRFU universe to include geographic areas focusing on Update Leave.

  • Integrating into NRFU the ability to designate an area for an earlier NRFU operation where needed to enumerate college and university students living in off-campus housing before the end of the spring semester.

• Technical Design

  • Continuing refinement of the management alerts to identify potentially problematic field behavior in real time.

  • Continuing refinement of the optimization and routing algorithms used to make daily NRFU field assignments.

  • Continuing work to integrate into the Census Bureau’s enterprise data collection systems.

Update Leave (UL):

The Update Leave (UL) operation is designed for areas where the Census Bureau has concerns about accurate mail delivery and needs to determine the Census block location of each housing unit. The current design capitalizes on 2020 Census methodological improvements such as internet self-response and automated field operations. UL is conducted mostly in geographic areas that have one or more of the following characteristics:

• Do not have city-style addresses like 123 Main Street.

• Do not receive mail through city-style addresses.

• Receive mail at post office boxes rather than at physical addresses.

• Have unique challenges associated with accessibility, such as dirt roads or seasonal access.

• Have recently been affected by natural disasters.

• Have high concentrations of seasonally vacant housing.

The following objectives are being tested for Update Leave:

• Integrating listing operation and systems.

• Testing the ability to link a questionnaire ID to an address.

• Testing field supervisor to enumerator ratios.

Group Quarters (GQ):

The 2018 End-to-End Census Test will inform Census Bureau technological and operational planning and design for the enumeration of the population residing in group quarters (GQs). GQs are living quarters where people who are typically unrelated have group living arrangements and frequently are receiving some type of service. College/University student housing and nursing/skilled-nursing facilities are examples of GQs. To date, some small-scale testing has been done to test electronic transmission of GQ’s enumeration responses. The 2018 End-to-End Census Test expands on these results to allow the opportunity to evaluate procedures and technologies for conducting GQ enumeration operations. The set of operations planned for GQ enumeration is GQ Advance Contact, Service-Based Enumeration, and, finally, GQ Enumeration. These operations have been used in previous censuses. The GQ Advance Contact is an operation where facility contact and planning data are collected, including the ability of the GQ facility to provide electronic records for the enumeration. Service-Based Enumeration has the objective of counting individuals who will not be enumerated at a living quarter but are receiving some type of service. The GQ Enumeration is the final stage of enumerating individuals residing at the GQ.

• Operational Procedures

  • Testing updated procedures for handling newly discovered GQs during field operations.

  • Continuing testing of the various GQ operations, process flows, estimated staffing levels, supporting processes, and workload estimates.

  • Continuing refinement of procedures for linking paper questionnaire response records collected by multiple enumerators during enumeration at a single GQ.

  • Continuing evaluation and refinement of the optimal enumerator to GQ ratios for multiple GQ types.

  • Testing multiple modes of enumeration.

• Technical Design

  • Testing the use of electronic methodologies to:

    • Create the initial universe for the GQ Advance Contact.

    • Conduct In-Office GQ Advance Contact.

    • Update the GQ frame before GQ enumeration.

    • Accept electronically transmitted response data in a Census Bureau provided template.

  • Integrating GQ operations with listing and enumeration operations and systems.

Content:

The Census Bureau recognizes that the Office Of Management and Budget (OMB) is continuing to lead the discussion among federal agencies and other stakeholders on race/ethnicity from the perspective of data collection and dissemination guidance and standards, and that the final determination has not been made on the format of the race/ethnicity question for the 2020 Census. If it is determined that the combined race/ethnicity question format may be used for the 2020 Census (versus the separate race and Hispanic Origin questions used for the 2010 Census), it will be crucial for the Census Bureau to ensure that critical operations are fully prepared to go into production for the 2020 Census using the combined question. Therefore, the 2018 End-to-End Census Test data collection operations will use the combined race/ethnicity question version (which also includes a Middle Eastern or North African category) to further its analysis and understanding of mode differences for the race/ethnicity responses before deploying the 2020 Census questionnaire.

• Internet Self-Response: Continue testing the combined race/ethnicity question under the further enhancements of the internet self-response instrument for the 2018 End-to-End Census Test in regards to user experience, performance, and functionality; and ensure that the resulting response data and paradata meet the requirements of follow-up and data processing operations.

• Nonresponse Followup: Continue testing the combined race/ethnicity question under the further enhancements of the field enumeration instrument; assess enumerators’ experience with the field enumeration instrument and their navigation of the race/ethnicity question within the instrument. Input will be gathered during the post-operation field enumerator debriefing sessions.

• Update Leave and Group Quarters: Examine the 2018 End-to-End Census Test results by mode, including Update Leave and Group Quarters operations, which will be fielded for the first time this decade.

Supporting Documents about the 2020 Census Design and the 2018 End-to-End Census Test Objectives

The Census Bureau is submitting with this package links to a number of official Census Bureau documents that provide background on the plans for the 2020 Census. This list includes in particular the 2020 Census Operational Plan v2.0, which documents at a high level the objectives for the census tests both already completed and planned for the future. This document shows the planned design of the 2020 Census as of September 2016 and identifies design decisions made, as well as remaining decisions to be made using census test results. Key design components related to Peak Operations for the 2018 End-to-End Census Test are discussed in Chapter 4 (specifically 4.1.5) and in Sections 5.5.4, 5.5.5, 5.5.6, 5.5.7, 5.5.9, 5.5.10, 5.6.1, and 5.6.2. See Appendix C.

In addition, for most of the 2020 Census operations, the Census Bureau is developing a Detailed Operational Plan (DOP) to document objectives and procedures of the operation, major tasks involved in implementation, the overall workflow, and the overall resources required. See Appendix C for a list of the Detailed Operational Plans completed to date or soon to be completed and links to these plans within the 2020 Census Memorandum Series.

In addition, we include in Appendix B a graphic showing the census operations that are included in this test.

Site

The 2018 End-to-End Census Test for Peak Operations will take place in Providence County, Rhode Island. This location allows for testing systems and operations integration with a number of different operations, some of which have not been included in integrated tests this decade. The 2018 End-to-End Census Test for Address Canvassing took place in three sites: Bluefield-Beckley-Oak Hill, West Virginia; Providence County, Rhode Island; and Pierce County, Washington. Collectively these three sites allowed the Census Bureau to gain valuable experience in conducting the challenging process of building the address list in sites with a diversity of physical geography, housing structures, and residence types. For this Peak Operations part of this test, the focus is on systems and operations integration for more operations than have been tested in prior census tests this decade. Providence County, RI is an ideal community to simulate the 2020 Census experience and provide sufficient systems and operations testing.

Self-Response

The housing units in the areas selected for inclusion in the 2018 End-to-End Census Test will be contacted by mail and invited to complete their questionnaire using the internet. Optimizing Self-Response contact methods include follow-on letter and postcard reminders. The Census Bureau will also deliver paper questionnaires to households that do not or cannot respond online, as measured by low internet connectivity or low internet usage rates.

The Census Bureau will continue to test Non-ID Processing methodology as another strategy for Optimizing Self-Response. A User ID that links to a unique housing unit is on many of the mailed materials, but respondents can also submit a response without using the ID, particularly on the internet or telephone. Non-ID Processing refers to address matching and geocoding for census responses that lack this preassigned User ID. This processing allows such responses to be linked up with the associated census enumeration data and can occur through automated or clerical procedures. Additionally, the 2018 End-to-End Census Test will allow the Census Bureau to continue to develop the capability to conduct real-time Non-ID Processing, where a respondent is prompted interactively (while they are still online filling out the questionnaire) for additional address and location information if the provided address cannot be matched through automation to an address with a User ID.

This test will allow the Census Bureau to better understand the requirements related to scalability of planned systems and to determine metrics for each of the Non-ID Processing steps. If the address match is not resolved during the initial automated or real-time processing, Census Bureau staff will attempt manually to match or geocode the address. It is estimated that 2 percent of the overall non-ID respondents will be contacted by telephone as part of the manual matching process. Non-ID Processing also includes an office-based address verification (OBAV) component. OBAV uses available geographic reference materials to verify the existence and location of an address. Responses processed during the manual matching process may become part of the OBAV workload, depending on the results of the matching and geocoding. OBAV is performed in an effort to avoid more costly fieldwork. However, any address that is worked in OBAV but cannot be verified in OBAV will be sent to Field Verification (discussed in more detail below as a suboperation of NRFU).

Additionally, with the Re-Collect component, a sample of self-response cases are selected for re-contact, which may occur through centralized phone contact or in-field enumeration. Re-Collect is intended to validate the information from a respondent, confirming the existence of the address and the people enumerated at that address. Re-Collect is also one aspect of fraud detection.

Content

Decades of research on different race/ethnicity question designs have shown that individual identities can be impacted by societal changes, attitudes, and perceptions. The 2018 End-to-End Census Test design can help us understand whether respondent reporting of racial/ethnic identities is impacted by the types of data that the Census Bureau is collecting (e.g., detailed race/ethnic responses and new categories), as well as whether or not respondent privacy concerns and expectations for data protection are addressed and the process is trusted by the general public.

It will be crucial for the Census Bureau to ensure that critical operations are fully prepared to go into production for the 2020 Census using the combined race/ethnicity question, if it is determined that the combined race/ethnicity question format may be used for the 2020 Census. The Census Bureau plans to deploy the combined race/ethnicity question along with a Middle Eastern or North African category during the 2018 End-to-End Census Test to further examine item nonresponse to the combined race/ethnicity question and levels of overall race/ethnicity group reporting (e.g., White, Hispanic, Black, etc.), as well as detailed reporting levels for these groups (e.g., German, Mexican, Jamaican, etc.).

Research has found that, over time, there have been a growing number of people who do not identify with any of the race categories, and this means that an increasing number of respondents have been classified as “Some Other Race” (SOR). The combined question format with detailed checkboxes has greatly helped to improve the accuracy of self-identified race/ethnicity data, as demonstrated by testing during the 2015 National Content Test (NCT). In addition, the Census Bureau conducted an additional matching study to explore how respondents self-identify their race and/or Hispanic origin in a combined question (with an SOR category) versus when they are presented with two separate questions (without an SOR category). In this study, respondents’ Hispanic origin and race responses from the NCT were linked with the Hispanic origin and race responses provided for the same respondents in the 2011-2016 Current Population Survey Annual Social and Economic Supplement (CPS ASEC). The analyses of these matched data showed that among those who responded Hispanic White alone in the CPS ASEC and link to the NCT Separate Format Panel, the majority selected Hispanic SOR alone or in combination.

The Census Bureau expects the results of the 2018 End-to-End Census Test will help inform our growing body of knowledge regarding the combined race/ethnicity question and the collection of major group responses and detailed race/ethnicity responses. This test also presents an important opportunity to test imputation procedures for missing data with a combined race/ethnicity question. The 2018 End-to-End Census Test will enable the Census Bureau to determine which records to use (e.g., ancestry, Hispanic origin, race), and in what hierarchy they should be used for imputation. Additionally, the development of data products and testing their dissemination is a critical objective of the 2018 End-to-End Census Test. The question format used in data collection and processing is also the source of the redistricting tabulation, and the Census Bureau must be prepared to meet the needs of the states as well as 2020 Census data users if it is determined that the combined race/ethnicity question format will be used for the 2020 Census.

As previously stated, the Census Bureau recognizes that OMB is continuing to lead the discussion among federal agencies and other stakeholders on race/ethnicity from the perspective of data collection and dissemination guidance and standards, and that the final determination has not been made on the format of the race/ethnicity question for the 2020 Census. In the event that the 2020 Census does not proceed with the combined race/ethnicity question, the Census Bureau is prepared to make necessary adjustments to deploy the separate race and Hispanic origin questions by consulting the various versions of the separate race and Hispanic origin questions that were tested during the 2015 National Content Test. The Census Bureau will properly configure all downstream operationssuch as response processing and data tabulation, imputation, analysis, and data disseminationto ensure a successful deployment of the race/ethnicity question(s) regardless of the question format.

Language Services

Individuals of Limited English Proficiency (LEP) require language assistance in order to complete their census questionnaires. The Census Bureau has identified the largest LEP populations in the United States using American Community Survey data and has established a program for providing non-English materials for LEP respondents. For the 2018 End-to-End Census Test, respondents will be able to provide their information in English, Spanish, Chinese, Vietnamese, Korean, Russian, Arabic, and Tagalog.

Nonresponse Followup (NRFU)

For all housing unit addresses included in the test universe, if no response is received by a specified date, the address will be included in the universe for the NRFU portion of the test. In NRFU, enumerators will attempt to follow up at addresses for which no self-response was received to determine their status and to collect their data for addresses determined to be occupied.

At certain points during the NRFU operation, administrative records modeling will be performed on all NRFU cases, resulting in one of four modeled status outcomes: 1) occupied, 2) vacant, 3) delete, or 4) no determination. The modeled status will guide the contact strategy used for followup, and the conditions for removing the case from the NRFU workload.

To allow sufficient time for self-response, the NRFU operation will begin in mid-May. However, some students who reside in off-campus housing units will have left the campus area by the time NRFU is scheduled to begin. Early NRFU will be conducted starting in April in blocks near colleges and universities with a high percentage of off-campus housing to enumerate at these units while students are still in town.

The Census Bureau will conduct NRFU with mobile devices. The devices will use a secure Census Bureau-provided enumeration application solution for conducting the NRFU field data collection.

Nonresponse Followup Reinterview (NRFU-RI)

A sample of the cases enumerated by NRFU will be selected for reinterview (RI). This NRFU-RI operation is intended to help pinpoint possible cases of enumerator falsification. The Census Bureau will use centralized phone contacts of the NRFU-RI cases before sending them to an enumerator in the field, providing opportunities for additional efficiency. Enumerators working NRFU-RI cases will use the same mobile devices and software as for the NRFU cases.

Field Verification (FV)

Households that self-respond to the census test without a User ID with addresses that cannot be either matched to our address frame or verified in Non-ID Processing may be sent to a Field Verification operation, performed by NRFU enumerators. This suboperation is designed for verification that the housing unit exists, confirmation of the census block location for the address, and if possible, collection of Global Positioning System coordinates to facilitate accurate determination of the census block.

Update Leave (UL)

Update Leave for the 2018 End-to-End Census Test will test the two planned components of the operation: Update Leave Production and Update Leave Quality Control (QC). During the field operation, the Census Bureau will test linking a questionnaire ID to each housing unit and, using the Internet Choice methodology, leaving a choice package with a paper questionnaire for households to self-respond. Any responses not received will be sent to NRFU for additional attempts. If a household self-responds, this is a cost savings for the test since another enumerator will not have to return and spend time collecting these data.

Update Leave (UL) Production

Enumerators will visit specific geographic areas to identify every place where people could live or stay, comparing what they see on the ground to the existing census address list. The enumerator will update the address list, either verify or correct the address and location information, and classify each living quarter (LQ) as a housing unit (HU), group quarter (GQ) or transitory location (TL). If the LQ is classified as a GQ, it will be designated for enumeration within the GQ operations.

Enumerators will attempt to contact and leave a questionnaire package for each housing unit found. If someone answers the door, the enumerator will provide a Confidentiality Notice and ask questions to verify or update the address. The enumerator will then ask if there are any additional LQs in the structure or on the property. If there are additional LQs, the enumerators will collect/update that information. The enumerator will then link a paper questionnaire to the updated address, place it in the package, and leave it for the respondent. The choice package invites a respondent to go online with a User ID or to fill out a paper questionnaire to complete the 2018 End-to-End Census Test. The package will include a letter with the phone number for Census Questionnaire Assistance (CQA) if the respondent has any questions or would prefer to respond to the test on the telephone.

Update Leave QC

A sample of addresses listed by UL production will be selected for QC. The intention of this operation is to verify enumerators understand correct listing procedures and to pinpoint possible address errors. UL QC will use the Census Bureau’s listing software on laptops to re-collect listing data to be used for a comparison.

Group Quarters Advance Contact

The GQ Advance Contact operation will contact group quarters before enumeration. In an in-office Advance Contact, GQs will be contacted to verify information such as: preferred modes of enumeration, expected population on Census Day, and whether there are available electronic response data records the Census Bureau could use for the enumeration. Census Bureau staff at area census offices will follow up with GQs by phone to obtain the necessary pre-enumeration information.

Group Quarters Service-Based Enumeration (SBE)

Enumerators will conduct SBE at selected shelters, soup kitchens, regularly scheduled mobile food vans, and nonsheltered outdoor locations, providing an opportunity to test new response collection procedures on a larger scale than has been tested so far this decade.

Group Quarters Enumeration (GQE)

GQE will involve multiple modes of enumeration. During the 2018 End-to-End Census Test, electronic response for GQs will be tested on a broad scale to determine if there are gains in efficiency through self-response. For GQs that do not request electronic response, the modes of enumeration include in-person interview, drop off and pickup of paper questionnaires, which are completed by self-enumeration or facility self-enumeration, and paper response data listing, also known as paper administrative records.

Group Quarters QC

A sample of cases that have been enumerated by GQE will be selected for reinterview. This operation is intended to help us pinpoint possible cases of enumerator falsification.

Coverage Improvement

Coverage Improvement is conducted to resolve potential erroneous enumerations and omissions from the initial self-response data collection and from all field enumeration data collections. Coverage questions are included in both the self-response and NRFU instruments to aid in the identification of overcount and/or undercount cases. Coverage edits are also used to identify cases based on large households (because of limitations on the paper form) and count discrepancies (where the sum from the roster does not equal the provided population count). After the Coverage Improvement workload is selected, telephone follow-up with the respondent will occur. For cases left unresolved after the phone attempts, limited in-office follow-up will include evaluating usual-home-elsewhere address data from GQ enumerations and assessing the potential person duplication, as identified through person matching on collected data.

Response Processing, Redistricting Data, Data Products and Dissemination 

Any non-English response data from detailed write-in fields, such as those on the combined race/ethnicity question, will need to be translated, output, processed, coded, edited, and tabulated. In addition, prototype P.L. 94-171 Redistricting Data tabulations and geographic support products will be delivered. Ensuring these interfaces meet the requirements for data tabulation will be a crucial step in preparing to tabulate the test data.

The design of this data product and its dissemination is a critical final objective of the 2018 End-to-End Census Test, as the Census Bureau must be prepared to meet the needs of various stakeholders for 2020 Census data.

2. Needs and Uses

The Census Bureau needs to validate that the operational design of the Peak Operations is ready for 2020 Census production, from a systems, operational, and architectural perspective. The results of this test will inform the Census Bureau’s final preparations in advance of the 2020 Census. In particular, conducting a live operation will ensure all the systems, instruments, and processes are functioning correctly or will provide indicators of what needs to be fixed. In addition, metrics collected during the operation will provide additional data to be used for budget and operational planning purposes. The operational assessment questions that the Census Bureau plans to answer using data collected from this test are documented in the following series of Assessment Study Plans, which are also being submitted with this package:

• 2018 End-to-End Census Test Group Quarters Enumeration Operational Assessment

• 2018 End-to-End Census Test Group Quarters Advance Contact Operational Assessment

• 2018 End-to-End Census Test Internet Self-Response Operational Assessment

• 2018 End-to-End Census Test Paper Data Capture Operational Assessment

• 2018 End-to-End Census Test Decennial Logistics Management Support Operational Assessment

• 2018 End-to-End Census Test Decennial Logistics Management: Space Acquisition and Lease Management Operational Assessment

• 2018 End-to-End Census Test Non-ID Processing Operational Assessment

• 2018 End-to-End Census Test Census Questionnaire Assistance Operational Assessment

• 2018 End-to-End Census Test Geographic Programs Operational Assessment

• 2018 End-to-End Census Test Forms Printing and Distribution Operational Assessment

• 2018 End-to-End Census Test Recruiting, Onboarding and Training (Field Infrastructure) Operational Assessment

• 2018 End-to-End Census Test: Field Office Administration and Payroll (Field Infrastructure) Operational Assessment

• 2018 End-to-End Census Test Service-Based Enumeration (Group Quarters) Operational Assessment

• 2018 End-to-End Census Test Update Leave Operational Assessment

• 2018 End-to-End Census Test Response Processing Operational Assessment

• 2018 End-to-End Census Test Nonresponse Followup Operational Assessment

• 2018 End-to-End Census Test Redistricting Data Program Operational Assessment

While the Census Bureau will endeavor to answer the documented operational assessment questions, there are some known analysis limitations. The 2018 End-To-End Census Test is a site test with its own special characteristics, so test results cannot be generalized to the entire nation and do not necessarily predict trends or rates/estimates expected in the 2020 Census.

There are two topics for which there will be research questions to be answered during this test. The Census Bureau is submitting with this package the following study plans:

• 2018 End-to-End Census Test Response Rates Study

• 2018 End-to-End Census Test: Administrative Records Usage for Nonresponse Followup

Information Quality

Information quality is an integral part of the predissemination review of the data disseminated by the Census Bureau (fully described in the Census Bureau’s Information Quality Guidelines at https://www.census.gov/quality/guidelines/). Information quality is also integral to the data collections conducted by the Census Bureau and is incorporated into the clearance process required by the Paperwork Reduction Act.

The data collected from households and individuals during the 2018 End-to-End Census Test will be used to evaluate the readiness of operations and systems for the 2020 Census. Additionally, the Census Bureau will produce a prototype of the geographic and data products that will be used for the 2020 Census and make the prototypes available to the general public.

3. Use of Information Technology

The 2018 End-to-End Census Test will use the internet, telephone centers, and data collection software residing on electronic devices to communicate with respondents. Respondents will initially have the option to respond to this Test using the internet on various devices (e.g. computers, tablets, smartphones), by mailed paper questionnaires, or on the telephone through CQA.

The 2018 End-to-End Census Test is heavily dependent on specific information technology systems, and a key objective of the test is devoted to validating the integration of systems with each other and with the operations they support. The test will employ automated systems to administer and manage training, manage workloads, route field workers, alert supervisors of potential problems, create management reports, and process responses.

The IT systems to be used in this test and in the 2020 Census undergo a number of stages of testing and validation. These fall into the major categories of Systems Readiness and Operational Readiness.

Systems Readiness

Systems Readiness reviews are a series of milestone reviews completed at the end of each Software Development Life Cycle (SDLC) phase. The purpose of the reviews is to: (1) Give an indication of system readiness at the end of each SDLC phase. (2) Highlight completed activities, identify potential issues, and enable managers to decide if system development is ready to begin the next SDLC phase. (3) Provide managers and other stakeholders with transparency into the system development processes used by cross-directorate System Development Teams (SDT). Decennial Systems Engineering & Integration (SE&I) will monitor project-level readiness status throughout the life cycle. Each system will report weekly on status, issues, and risks. In addition to this regular monitoring, SE&I will also assess systems readiness to support an operation through Production Readiness Reviews (PRRs). Each PRR will serve as a gateway to assess systems readiness to support a particular operation.

Operational Readiness

Operational readiness reviews will be conducted to assess and ensure the readiness throughout the program life cycle. Once the systems are thoroughly tested during Systems Readiness, an assessment of systems will be made during the PRR, and the systems will be handed over to operations. The operations team will perform multiple Operation Readiness Testing (ORT) activities to ensure all other components in addition to the systems are tested thoroughly and ready to support the operation.

4. Efforts to Identify Duplication

As part of its efforts to conduct a complete and accurate decennial census, the Census Bureau identified new methods for the public to respond to the census and new ways to automate and more efficiently manage field data collection that have not previously been examined or used in a decennial census.

We are using research results when possible from other Census Bureau surveys. Research on American Community Survey paradata from their internet data collection operations was informative on the usefulness of continuing to contact respondents after they initially fail to complete the survey. We also worked with other countries with similar issues and goals (for example, Australia, Canada, and England) to share information on these matters. The 2020 Census Research and Testing Program also conducted additional literature reviews on results seen in other surveys about paradata, NRFU procedures, fieldwork efficiencies, telephone contacts, internet response, and geographic differences. However, most survey results cannot be directly applied to a decennial census environment. The size, scope, mandatory nature, importance of results (for such things as congressional apportionment, state redistricting efforts, and the allocation of billions of dollars in federal funds each year), and timing constraints (legal deadlines for producing apportionment and redistricting data) of the decennial census are unique. Thus, thorough and separate research and integrated testing must be conducted to ensure that new methods and operations will work in a decennial census environment.

5. Minimizing Burden

The collection of data is only for households and individuals and should have no effect on small businesses.

6. Consequences of Less Frequent Collection

If this collection of information does not occur, it would significantly impact the Census Bureau’s ability to conduct integrated, end-to-end testing of 2020 Census systems, procedures, and field infrastructure, and thus the ability to refine operations before the 2020 Census. Frequency cannot be decreased, as this is a one-time data collection activity and the last major test prior to the 2020 Census.

7. Special Circumstances

No special circumstances exist.

8. Consultations outside the Agency

In developing this test, the Census Bureau consulted with a variety of stakeholders, including, but not limited to, academics, national researchers, community and organizational leaders, and the Census Bureau’s Advisory Committees. In addition, external consultants from the National Academy of Sciences provided feedback about the Census Bureau’s objectives for the 2018 End-to-End Census Test. Recommendations from audits by the Government Accountability Office and the Department of Commerce’s Office of Inspector General were also factored in.

The results from this test will be shared widely with decennial census stakeholders and released to the public. To help evaluate and assess the results of the 2018 End-to-End Census Test, the Census Bureau will conduct debriefings with field staff who work on the test.

The notice for public comment, titled, “2018 End-to-End Census Test,” was published in the Federal Register March 29, 2017 (Vol. 82 No. 59, pp.15486-15492). The Census Bureau received five messages or documents with comments during the 60-day period. One comment reflected general discontent with the government and said that the only questions that the Census Bureau should ask are about illegal immigrants.

Comments from the Center for American Progress (CAP) indicated strong support of the Census Bureau’s proposed collection of information, and CAP commended the Census Bureau for utilizing the combined race/ethnicity question in the 2018 End-to-End Census Test to perform further analysis using the results.

For the remaining comments, the Census Bureau has prepared a separate document to provide complete responses to all the comments. A summary of the comments is provided here for reference.

One comment was received regarding a request for the lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ) community to be fully acknowledged positively as members of the American family in the 2020 Census.

Comments from the Nevada state demographer indicated a variety of concerns about the impacts of Census Bureau’s plans to reduce in-field Address Canvassing and other time spent on the ground in field operations. Specifically there were concerns about identification of Group Quarters and identification of units where people live for short terms compounded by potential undercoverage of these same populations within administrative records.

Comments from the National Associated of Latino Elected and Appointed Officials (NALEO) Educational Fund included recommendations on a variety of topics and situations. The categories for the recommendations specific to this test were internet response and other technological advances, evaluation of the combined question on Hispanic origin and race, the use of Administrative Records within the NRFU operation, and enumeration of the populations within Group Quarters.

The notice for public comment, titled, “2018 End-to-End Census Test,” was published in the Federal Register July 13, 2017 (Vol. 82 No. 133, pp. 32322-32324). The Census Bureau received one document with comments during the 30-day period, from the NALEO Educational Fund. These comments mostly fell into the category of concerns about the reduction to one test site rather than the original planned three. NALEO also expressed a particular concern about not having sufficient diversity to test the combined race and ethnicity question for full utilization in a census and comparability of datasets between decades.

Census Bureau Response

The Administration’s FY 2018 budget request funds the 2020 Census operations and systems development activities that are required to ensure a fair, accurate, and complete 2020 Census. For the 2018 End-to-End Census Test, the final major field test before the 2020 Census, the Census Bureau has made decisions that will prioritize the readiness and testing of the integrated system-of-systems in the field in a Census-like environment. The Census Bureau must commit the bulk of its resources in FY 2018 to ensuring that systems are ready and secured and that the infrastructure is built out well in advance of the 2020 Census.

The lessons learned from how these systems interact with each other, with the operations being tested, and, where relevant, with the field staff and residents in the test sites, will be invaluable to finalizing the operational plan and putting the finishing refinements on the systems and operations in advance of the 2020 Census. We are confident in the ability of the Providence County test site to allow us to test nearly everything that we would have been able to test under three sites. Providence County is an ideal community to simulate a microcosm of the 2020 Census experience, as its demographics mirror those of the nation. The county includes historically hard-to-count populations and has a large Latino presence.

Additionally, the replanned 2017 Census Test successfully conducted this spring was a self-response test of 80,000 housing units that allowed testing of all our public-facing systems for self-response together in the field for the first time, and in a cloud environment for the first time. This was a major step in our readiness for the 2018 End-to-End Census Test and the 2020 Census itself. This test included the Spanish language version of these systems.

The Census Bureau agrees that the 2020 Census nationwide partnership program is vital to counting everyone. With this program, the Census Bureau will have trusted community partners encouraging community members to respond to the 2020 Census. The Census Bureau also recognizes that not everyone has the same level of internet connectivity and access to technology and has designed alternatives that will provide ample response options for anyone. One of these options is sending a paper form to about 20 percent of housing units with the first mailing in areas where American Community Survey data show there is less connectivity. In the 2020 Census all households that do not self-respond initially will receive a paper form.

In addition, by minimizing the Nonresponse Followup operation to collect responses from nonrespondents, the Census Bureau has the potential to save $1.4 billion (based on the life cycle cost estimate released in 2015) in the 2020 Census by using information individuals have already provided to the government, called administrative records. These cost savings primarily come in two ways:

1) After giving the population an opportunity to self-respond to the 2020 Census and we know the addresses that did not respond, we will use multiple sources of high quality administrative records to identify vacant units so enumerators do not have to visit these addresses.

2) For the remaining cases, enumerators will make one visit to collect the information in person. If the visit to collect the information in person is not successful and only where we have high quality administrative records from trusted sources, we will use the administrative records as the response data for the household.

To be clear, where high-quality administrative records are not available from trusted sources, we will continue in-person visits to reach nonresponding housing units until the case is resolved.

The Census Bureau has acknowledged the undercount of young children, those ages 0 to 4, in the decennial census, and we have been actively researching this coverage problem. This is a difficult issue that goes back to at least the 1970 census. Many countries around the world share this issue of undercounting young children.

A research team has been conducting several projects to answer key questions about why the census disproportionately misses young children. One common finding has been that children that are not the biological or adopted child of the householder are more likely to be missed. These research results are being published in the 2020 Census Memorandum Series. This research effort will continue. We are also researching working with different advocacy groups who want to help us with this issue.

Regarding the comment about using a single combined race and ethnicity question, we note that the Census Bureau comprehensively tested different question designs in the 2015 National Content Test (NCT). The 2015 NCT was a large nationally representative test of 1.2 million addresses across a broad spectrum of different demographic and socio-economic groups. Earlier this year, the Census Bureau released the 2015 NCT Race and Ethnicity Analysis Report for the 2020 Census, which explored several promising ways to improve the census’ race and ethnicity questions so that they better measure our nation. These findings build upon our previous research, and are instrumental to improving respondents’ understanding of their options to report multiple race and ethnic groups, and see themselves as identified in the census. Research findings from the 2015 National Content Test indicate that an optimal race and ethnicity question design would include elements, such as: 

• A combined race and ethnicity question, with detailed checkboxes for response.

• Using "Race/Ethnicity" terminology.

• A separate "Middle Eastern or North African" response category can produce high quality data.

• New instructions to "Mark all that apply" (for paper data collections) or "Select all that apply" (for internet data collections). 

In addition to the 2015 NCT research, the combined race and ethnicity question was also evaluated in the 2016 American Community Survey Content Test, with testing in all response modes including self-response and in-person followup.

The Census Bureau and other federal statistical agencies are awaiting guidance from the U.S. Office of Management and Budget (OMB) on their review of the current federal standards for race/ethnicity data. These standards will inform how the Census Bureau collects race/ethnicity data for the 2020 Census. An OMB decision is expected later this year.

If the Census Bureau were to move to a combined question format, we would also expect that conversion techniques (e.g., bridging/imputation) would be developed to allow for comparison between different data collection designs. Ultimately, the U.S. Census Bureau will incorporate the guidance from OMB’s decisions on race/ethnicity standards into the planning and design for race/ethnicity on the 2020 Census. Final question wording for the 2020 Census must be delivered to the U.S. Congress before April 2018.

9. Paying Respondents

Respondents participating in the 2018 End-to-End Census Test will not receive any form of compensation for their participation.

10. Assurances of Confidentiality

The Census Bureau will conduct the 2018 End-to-End Census Test under the authority of Title 13 United States Code Sections 141, 191, and 193. All respondents who participate in the 2018 End-to-End Census Test will be informed that the information they provide is confidential under that law but that the same law makes participation mandatory. All collected information that identifies individuals will be held in strict confidence according to the provisions of Title 13 United States Code, Section 9 and 214.

11. Justification for Sensitive Questions

The Census Bureau collects racial and ethnic data in accordance with the 1997 OMB standards on race and ethnicity. Information on race and Hispanic origin is required for many federal programs and is critical in making policy decisions, particularly for civil rights. States use these data to meet legislative redistricting principles. Racial and ethnic statistics are used in planning and evaluating government programs and policies to ensure they fairly serve the needs of each community and to monitor against discrimination in these programs and in society. Race and Hispanic origin data are also used to promote equal employment opportunities and to assess racial and ethnic disparities in health and environmental risks.

12. Estimate of Burden Hours

Test Site: Providence County, Rhode Island.
Operation or Category	Estimated Number of Respondents	Estimated Time per Response	Total Burden Hours
Self-Response - Internet/Telephone/Paper	114,000	10 minutes	19,000 hours
Nonresponse Followup	163,000	10 minutes	27,167 hours
Nonresponse Followup Reinterview	16,300	10 minutes	2,717 hours
Update Leave Production	2,000	5 minutes	167 hours
Update Leave QC	300	5 minutes	25 hours
GQ Advance Contact (facility)	500	10 minutes	83 hours
GQ SBE – facility contact	40	10 minutes	7 hours
GQ SBE – person contact	1,600	10 minutes	267 hours
GQ Enumeration – facility contact	500	10 minutes	83 hours
GQ Enumeration – person contact	28,000	10 minutes	4,667 hours
Group Quarters QC	50	5 minutes	4 hours
Non-ID Processing Phone Followup	115	5 minutes	10 hours
Re-collect	4,600	10 minutes	767 hours
Field Verification	140	2 minutes	5 hours
Coverage Improvement	5,500	10 minutes	917 hours
Totals	336,645		55,886 hours

13. Estimate of Cost Burden to Respondents

There are no costs to respondents other than their time to participate in this data collection.

14. Cost to Federal Government

The cost of this collection is covered under the requested budget for the 2018 End-to-End Census Test, Research and Testing Program, and is estimated to be $45,970,000. This amount includes conducting, planning, managing, and supporting this test, which includes funding for space, field infrastructure and equipment, recruiting and hiring processes, postage, and print contracts (“other objects”). This estimate also includes salaries for field workers, data capture processing staff, and the staff in headquarters providing program management and/or support for the test. This estimate does not include data product creation or CQA agents.

15. Reason for Change Burden

The increase in burden is attributable to the information collection being submitted as a new collection.

16. Project Schedule

Activity/Milestone	Date/Range
Conduct GQ Advance Contact	February 5, 2018 - March 9, 2018
Conduct Self-Response, including Non-ID Processing	March 16, 2018 - August 7, 2018
Conduct UL	March 19, 2018 - April 13, 2018
Conduct UL QC	March 26, 2018 - April 20, 2018
Conduct GQ Enumeration (including SBE)	March 28, 2018 - June 8, 2018
Conduct Coverage Improvement	March 30, 2018 - August 7, 2018
Census Day	April 1, 2018
Conduct Early NRFU	April 5 2018 - May 9, 2018
Conduct NRFU Reinterview	April 4, 2018 - July 31, 2018
Conduct Re-collect	April 4, 2018 - July 31, 2018
Conduct Group Quarters QC	April 4, 2018 - July 17, 2018
Conduct NRFU (including Field Verification)	May 10, 2018 - July 24, 2018
Conduct Late GQ Enumeration	June 11, 2018 - July 24, 2018
Publish Prototype P.L.94-171 Data and Support Products	By April 1, 2019

17. Request to Not Display Expiration Date

No exemption is requested.

18. Exceptions to the Certification

There are no exceptions to the certification.

Appendix A – Documents Included in the 2018 End-to-End Census Test Package

1. Supporting Statement A (this document)

2. Supporting Statement B

3. 2018 End-to-End Census Test – Index of Forms

4. 2018 End-to-End Census Test mailing materials (including questionnaires)

5. 2018 End-to-End Census Test mail materials and panel design matrix with form assignment

6. 2018 End-to-End Census Test Self Response Instrument Screenshots

7. 2018 End-to-End Census Test Group Quarters Advance Contact script

8. 2018 End-to-End Census Test Group Quarters Facility Managers Letters

9. 2018 End-to-End Census Test Group Quarters Field Materials

10. 2018 End-to-End Census Test Field Enumeration Instrument Specifications

11. 2018 End-to-End Census Test Coverage Improvement Instrument Screenshots

12. 2018 End-to-End Census Test Census Questionnaire Assistance Specifications

13. 2018 End-to-End Census Test 60-Day Federal Register Notice

14. Comments received on 60-Day Federal Register Notice

15. 2018 End-to-End Census Test 30-Day Federal Register Notice

16. Comments received on 30-Day Federal Register Notice

17. 2018 End-to-End Census Test 83-I

18. 2018 End-to-End Census Test Paperwork Reduction Act Executive Summary Form

19. 2018 End-to-End Census Test Nonresponse Followup Operational Assessment Study Plan

20. 2018 End-to-End Census Test Redistricting Data Program Operational Assessment Study Plan

21. 2018 End-to-End Census Test Update Leave Operational Assessment Study Plan

22. 2018 End-to-End Census Test Field Infrastructure Recruiting, Onboarding, and Training Operational Assessment Study Plan

23. 2018 End-to-End Census Test Field Infrastructure Field Office Administration Payroll Operational Assessment Study Plan

24. 2018 End-to-End Census Test Response Rates Study Plan

25. 2018 End-to-End Census Test: Administrative Records Usage for Nonresponse Followup Study Plan

26. Template for 2018 End-to-End Census Test and 2020 Census Operational Assessment Study Plans

27. Operational Assessment Content Guidelines for the 2018 End-to-End Census Test and the 2020 Census

28. List of Detailed Operational Plans with links to these and 2020 Census Operational Plan – Supporting Statement A Appendix C

29. Additional responses to comments received on the 60-day Federal Register Notice

Note that the additional study plans listed within the document above as to be included within this package will be made available through Nonsubstantive Changes as soon as they are fully approved through the Census Bureau. The schedule is given in Appendix D.

Appendix B

Appendix C

The Operational Plan and all Detailed Operational Plans are available at the following website link:

<https://www.census.gov/programs-surveys/decennial-census/2020-census/planning-management/memo-series.html>

Specific links:

The 2020 Census Operational Plan, v2.0

Detailed Operational Plans related to Peak Operations for the 2018 End-to-End Census Test that have been or will soon be published under the 2020 Census Memorandum Series are:

1. Census Questionnaire Assistance (Memo 2016.15) <CQA DOP v1.0, September 15, 2016>

2. Content and Forms Design (Memo 2016.12) <CFD DOP v1.0, September 7, 2016>

3. Forms Printing and Distribution (TBD)

4. Geographic Programs, Geographic Data Processing Component (Memo 2016.18) <GP GDP DOP, v1.0, September 13, 2016>

5. Geographic Programs, Geographic Delineations Component (Memo 2016.19) <GP GD DOP, v1.0, September 28, 2016>

6. Geographic Programs, Geographic Partnership Programs Component (Memo 2016.20) <GP GP DOP, v1.0, September 9, 2016>

7. Internet Self-Response (TBD)

8. Language Services (Memo 2016.11) <LNG DOP, v1.0, September 1, 2016>

9. Non-ID Processing (Memo 2016.13) <NID DOP, v1.0, August 24, 2016>

10. Nonresponse Followup (TBD)

11. Paper Data Capture (TBD)

12. Security, Privacy, and Confidentiality (TBD)

13. Redistricting Data Program (Memo 2016.14) <RDP DOP, v1.0, September 8, 2016>

14. Response Processing (TBD)

Appendix D

Schedule for submissions of 2018 End-to-End Census Test Operational Assessments and Studies

1. Group Quarters Enumeration – November 30, 2017

2. Group Quarters Advance Contact – November 30, 2017

3. Internet Self-Response – December 29, 2017

4. Paper Data Capture – November 30, 2017

5. Decennial Logistics Management Support – November 10, 2017

6. Decennial Logistics Management: Space Acquisition and Lease Management – November 10, 2017

7. Non-ID Processing – September 29, 2017

8. Census Questionnaire Assistance – October 31, 2017

9. Geographic Programs – October 31, 2017

10. Forms Printing and Distribution – October 31, 2017

11. Recruiting, Onboarding and Training (Field Infrastructure) – in initial package

12. Field Office Administration and Payroll (Field Infrastructure) – in initial package

13. Service-Based Enumeration (Group Quarters) – October 31, 2017

14. Update Leave – in initial package

15. Response Processing – October 31, 2017

16. Nonresponse Followup – in initial package

17. Redistricting Data Program – in initial package

In addition, there will be the following research result reports:

1. Response Rates Study – in initial package

2. Administrative Records Usage for Nonresponse Followup – in initial package

18