2018EndtoEndCensusTest_RDP_Study Plan

Version 0.7

Study Plan for the 2018 End-to-End Census Test Operational Assessment Redistricting Data Program Census Redistricting & Voting Rights Data Office Draft Pending Final Census Bureau Executive Review and Clearance.
	June 6, 2017

1. Introduction

The 2018 End-to-End Census Test is an important opportunity for the U.S. Census Bureau to ensure an accurate count in 2020 of the nation’s increasingly diverse and rapidly growing population. It is the first opportunity to apply much of what has been learned from census tests conducted throughout the decade in preparation for the nation’s next once-a-decade population census in 2020. The 2018 End-to-End Census Test will be held in three locations, covering more than 700,000 housing units: Pierce County, Washington; Providence, Rhode Island; and the Bluefield-Beckley-Oak Hill, West Virginia, area.

The 2018 End-to-End Census Test will test and validate 2020 Census operations, procedures, systems, and field infrastructure to ensure proper integration and conformance with functional and nonfunctional requirements. The test will also produce a prototype of geographic and data products and will validate the 2020 Census design and cost estimate. Note that the 2018 End-to-End Census Test results are based on three sites that were purposely selected and cannot be generalized to the entire United States. Additionally, since it is not conducted in a “full decennial census environment,” the results may not replicate the results to be obtained in the 2020 Census.

The purpose of the Redistricting Data Program (RDP) operation is to provide each state the opportunity to identify the small area geographies needed for legislative redistricting and to deliver the legally required Public Law (P.L.) 94-171 redistricting data tabulations by the mandated deadline of one year from Census Day: April 1, 2021. The RDP will test delivery of the data tabulations and geographic support products in the 2018 End-to-End Census Test. This includes testing that the Data Products and Dissemination (DPD) team can tabulate the data with the format and content specified by the RDP and deliver the data on a flow by state. The data delivery will initially be “embargoed,” or accessible only to the governor, state legislative leadership, and members of the public bodies responsible for redistricting in the state being released. After those people have had access to the data for 24 hours, the data will be released to the public.

The RDP will also test that the Geographic Operations (GEOP) team can create the geographic support products as specified by the RDP and release them to the public on the Census Bureau’s website and to state stakeholders on removable media. The RDP will test that all products can be generated and delivered to stakeholders in the time frame required by law (no more than one year from Census Day).

This study plan documents how the Redistricting Data Program’s piece of the 2018 End-to-End Census Test will be assessed, as guided by questions to be answered.

2. Background

The 2020 RDP addresses the Census Bureau’s responsibilities under P.L. 94-171. The U.S. Congress passed P.L. 94-171 in 1975 to address the states’ need for data tabulated for specific geographies to meet the judicially established requirement of “one person, one vote” and other requirements under the Voting Rights Act of 1965. P.L. 94-171 requires the Census Bureau, through the Secretary of Commerce, to establish a program for states to identify the geographic areas for which data tabulations are needed to conduct their legislative redistricting. Historically, states have identified census tabulation blocks (blocks) and voting districts (VTDs) as those geographies. The law also requires that the program be conducted in a nonpartisan manner and that delivery of tabulations to the governors, legislative leadership, and the officers or public bodies with responsibility for legislative redistricting (hereafter referred to as the state stakeholders) take place no later than one year from Census Day. Since the program was first established for the 1980 Census, the specifics of the program have adapted from census to census to meet changing circumstances and needs of the states.

Changing circumstances and needs of the states were assessed after the 2010 Census as part of Phase 5 of the 2010 RDP: Evaluation of the 2010 Census Redistricting Data Program and Recommendations for the 2020 Census Redistricting Data Program. This phase included discussions by group emails where states provided the Census Bureau with feedback on how things went in 2010 and their vision or wish list for the 2020 Census. Based on this feedback and the requirements of P.L. 94-171, the Census Bureau plans to continue to deliver to stakeholders state data aggregated to 2020 Census tabulation blocks and other census geographies on a flow by state, delivering the last state no later than April 1, 2021. The Census Bureau will also deliver the data to the stakeholders in advance of the public and media (embargoed release). Data will not be released to the media and public until the Census Bureau has verified receipt by the state stakeholders. (In 2010, the Census Bureau released the data to the public immediately after releasing to the state stakeholders, which was a departure from previous censuses. Stakeholders overwhelmingly requested that they have time to access the 2020 data before the press and the public.) The data will include tables with race, ethnicity, and age characteristics; housing characteristics; and group quarters counts down to the block level.

The RDP Integrated Project Team (IPT) is developing specifications for the tabulations, but in 2010 they included five tables at many levels of geography down to the block level:

• A count of everyone by race.

• A count of the population age 18 and over by race.

• A count of Hispanic or Latino and a count of not Hispanic or Latino by race.

• A count of Hispanic or Latino and a count of not Hispanic or Latino by race for population 18 and over.

• A count of vacant and occupied housing units.

For 2020, the RDP IPT is also considering a group quarters table with the same race, ethnicity, and age breakdowns as above for adult corrections facilities, military barracks, college dormitories, and all other group quarters. Additionally, any changes the Office of Management and Budget makes to its Standards for Federal Data on Race and Ethnicity between now and 2020 may require the Census Bureau to alter the format and content of our tabulations accordingly.

The Census Bureau also plans to continue to provide geographic support products in December 2020/January 2021, in advance of the P.L. 94-171 redistricting data tabulations release for the 2020 Census.

The RDP has not yet been a direct part of any post-2010 testing. The 2018 End-to-End Census Test will be the first test of the data tabulation and creation systems and the delivery methods. A prototype P.L. 94-171 redistricting data tabulation product and geographic support products will be generated reflecting the data collected from the test sites. After these products are released to the states, the RDP will also solicit feedback from the states pertaining to the content, format, and delivery of these products.

3. Assumptions

• The RDP will work internally with the DPD and the GEOP team on the specifications for the prototype P.L. 94-171 redistricting data tabulations and the geographic support products. The content and format of the products will mirror as closely as possible our plans for the products in the 2020 Census.

• The RDP IPT will develop specifications for the prototype P.L. 94-171 redistricting data tabulation and deliver them to the DPD no later than January 2018. Specifications for the geographic support products were delivered to GEOP in November 2016.

• While the RDP will strive to make the file schema, format, and content for the prototype P.L. 94-171 redistricting data tabulations match what is planned for the 2020 P.L. 94-171 tabulations, final decisions from the Office of Management and Budget on the combined race/ethnicity question and inclusion of a Middle Eastern/North African category, the residence rule criteria decision, and state stakeholder feedback on the prototype products, may necessitate changes to the products for the 2020 Census.

• The Center for Enterprise Dissemination Services and Consumer Innovation (CEDSCI) will be used as the vehicle for the embargoed and public release of the prototype P.L. 94-171 redistricting data tabulations on the website. The data tabulations will also be available on removable media and mailed as a backup/contingency if necessary.

• Geographic support products will be released by the GEOP on the public website and (for state stakeholders only) on removable media, using standard procedures for geographic product release. States have requested these data products on removable media as well as on the web, because the size of map PDF files and shapefiles often makes it difficult and time-intensive to download from the website.

• The prototype product release schedule will mirror the release deadlines required under P.L. 94-171 for the 2020 Census data. However, because the geographic support products and prototype data product will only be produced for the test sites in the 2018 End-to-End Census Test (as opposed to all states in the 2020 Census), the data will not be released in a flow from December through March as is planned for 2020. Geographic support products for the test sites will be released in January 2019, and the prototype P.L. 94-171 redistricting data tabulations will be released no later than April 1, 2019.

4. Scope of Assessment Content and Questions-To-Be-Answered

The RDP provided geographic product requirements to the GEOP in November 2016. The GEOP will deliver the geographic support products in January 2019. The RDP will provide tabulation data product requirements to the DPD no later than January 2018. The DPD will notify the RDP when prototype P.L. 94-171 redistricting data products are tabulated and ready for release, and the RDP will authorize their embargoed release to the state stakeholders and the RDP’s official state liaison no later than April 1, 2019. The RDP will then confirm that the state stakeholders have accessed the data and provide authorization to DPD to release the products to the media and the public. The RDP requirements to DPD will ask CEDSCI to track metrics on the number of people who accessed and downloaded the prototype P.L. 94-171 redistricting data tabulations, who from the supplied embargoed delivery list has done so, and when they accessed them.

To address the risk that CEDSCI may fail to deliver the data in 2020, the RDP has developed a contingency plan where the prototype P.L. 94-171 redistricting data products will be provided to states on removable media. The RDP will ask DPD in conjunction with Application Development and Services Division (ADSD) to create data for the test sites on removable media to ensure the systems required to implement the contingency plan are operational. This will be included in the requirements to DPD.

The RDP assessment will include variance analysis on schedule and budget, a measure on how closely the final deliveries matched the requirements, and metrics on the number of data users who download the embargoed data and when. The assessment will also include debriefings with our state liaisons on how the prototype P.L. 94-171 redistricting data tabulations and geographic support products met their needs in terms of content, format, and method of delivery.

The operational assessment will answer the following questions:

• Did the prototype P.L. 94-171 redistricting data tabulations product and the geographic support products produced by the DPD and GEOP, respectively, meet the requirements submitted by the RDP?

• Were the products released to the states no later than April 1, 2019?

• How much variance was there between planned and actual spending?

• Was the prototype P.L. 94-171 redistricting data tabulations product successfully embargoed and released initially only to the state stakeholders and the RDP’s official state liaison?

• Was the prototype P.L. 94-171 redistricting data tabulations product successfully released publically on the Census Bureau’s webpage only after the RDP authorized it (after confirming state stakeholders had received the data)?

• How many state-level stakeholders downloaded the prototype P.L. 94-171 redistricting data tabulations product via CEDSCI and how quickly after the data were embargoed?

• Did DPD successfully create a prototype P.L. 94-171 redistricting data tabulation data on removable media for the test sites as described in the requirements?

• Did the structure and content of the prototype P.L. 94-171 redistricting data tabulations product and the geographic support products meet our state stakeholders’ redistricting needs? If not, what changes would they like to see?

• Did the delivery method (via the Census Bureau website) of the prototype P.L. 94-171 redistricting data tabulations product and the geographic support products meet our state stakeholders’ redistricting needs? If not, what changes would they like to see?

5. Methodology

The methodology for answering the research questions is as follows:

Did the prototype P.L. 94-171 redistricting data tabulations product and the geographic support products produced by the DPD and GEOP respectively meet the requirements submitted by the RDP?

To ensure all requirements were met, the RDP will compare the final products to the requirements document.

Were the products released to the states no later than April 1, 2019?

A schedule variance report will compare the release dates with the planned dates.

How much variance was there between planned and actual spending?

Financial Management Reports (FMRs) generated by the Census Bureau’s Decennial Budget Office will compare planned and actual spending and calculate any variances.

Was the prototype P.L. 94-171 redistricting data tabulation product successfully embargoed and released initially only to state stakeholders and the RDP’s official state liaison?

The RDP will analyze data from CEDSCI to check that only those stakeholders on the embargo list accessed the data during the embargo period. RDP staff not on the embargo list will make attempts to access the data to ensure they cannot.

Was the prototype P.L. 94-171 redistricting data tabulations product successfully released publically on the Census Bureau’s webpage only after the RDP authorized it (after confirming state stakeholders had received the data)?

RDP staff not on the embargo list will attempt to access the data tabulations while the data are embargoed for state stakeholders only, and after the RDP authorizes it for release to the public from the embargo environment. Any instance where someone not on the embargo list is able to access the data tabulations before public release (during the embargo period) would indicate a failure of the system. Any instance where someone is unable to access the data tabulations after public release would indicate a failure of the system.

How many of our state-level stakeholders downloaded the prototype P.L. 94-171 redistricting data tabulations product via CEDSCI and how quickly after the data were embargoed?

The method for answering this research question will require data from the CEDSCI system. CEDSCI should be able to provide to the RDP a list of data users who logged into the embargoed environment to access the data and the date they did so. CEDSCI will also provide to the RDP the date the data first went into the embargoed environment.

Was DPD able to successfully create a prototype P.L. 94-171 redistricting data tabulation data on removable media for the test sites as described in the requirements?

The RDP will review data on removable media to ensure that all the files are present for the appropriate test site and can be opened. The product will be compared against the requirements provided to DPD to ensure all requirements were met.

Did the structure and content of the prototype P.L. 94-171 redistricting data tabulations product and the geographic support products meet our state stakeholders’ redistricting needs? If not, what changes would they like to see? Did the delivery method (via the Census Bureau website) of the prototype P.L. 94-171 redistricting data tabulations product and the geographic support products meet our state stakeholders’ redistricting needs? If not, what changes would they like to see?

The method for answering these research questions will come from a debriefing conducted by the RDP with the state stakeholders. The RDP will conduct qualitative interviews, where the RDP staff will ask the states if the content and data structure of the products was satisfactory to them and sufficient for them to conduct their redistricting. RDP staff will also ask open-ended questions to see if the state stakeholders found the delivery method met their needs or if they would prefer some other way of receiving the data. RDP staff will give states an opportunity to suggest alternate delivery methods or suggest changes to the products.

6. Risks/Limitations

If the CEDSCI delivery method cannot support an embargoed data release, then the RDP will not be able to assess the planned delivery method for the 2020 Census.

If GEOP and DPD cannot generate the prototype data and geographic support products, then the RDP will not be able to assess the delivery method, or assess if the data structure and content of the products meet the states’ redistricting needs.

7. Measures of Success

1. Prototype P.L. 94-171 redistricting data tabulation products and geographic support products are created on schedule and within budget.

2. Prototype P.L. 94-171 redistricting data tabulation products and geographic support products are released to state stakeholders on schedule.

3. Prototype P.L. 94-171 redistricting data tabulation products are released initially in an embargoed environment accessible only by state stakeholders.

4. Prototype P.L. 94-171 redistricting data tabulation products are not released publicly until RDP confirms state stakeholders have accessed the data and approves public release (24 hours after state stakeholders have accessed the data).

5. Prototype P.L. 94-171 redistricting data tabulation products and geographic support products meet specified requirements in their content and data structure.

8. Data Requirements

To perform an assessment of our research questions, the RDP will need the following data:

• The prototype P.L. 94-171 redistricting data tabulation products and the geographic support products, including any data on removable media, from DPD and GEOP, respectively. These products should be available no later than April 1, 2019.

• The dates the P.L. 94-171 prototype data product and the geographic support products were released to the embargoed website (where applicable) and the public. These dates will be provided by DPD and GEOP no later than April 1, 2019.

• Financial Management Reports generated by the Census Bureau’s Decennial Budget Office.

• A report from CEDSCI showing the login information (i.e., names, emails or login credentials yet to be determined) and time of login for the embargoed release website. This report should also include any information on what files, if any, were downloaded while the user was logged in.

• Reports generated from RDP staff indicating attempts made to access the geographic support products in the public release environment. These reports will include the date the RDP authorized the release, the date the attempt was made to access the data, and if the staff person was able to successfully access the data. These data will come from the RDP staff and be available no later than April 15, 2019.

• Reports generated from RDP staff indicating attempts made to access the Prototype P.L. 94-171 redistricting data tabulation products in the embargoed and public release environments, the date the attempts were made and if the staff person was able to successfully access the data. Also required will be the date the RDP authorized the Prototype P.L. 94-171 redistricting data tabulation products for public release. These data will all come from the RDP staff and be available no later than April 15, 2019.

• Feedback from the RDP state stakeholders obtained by RDP staff though open-ended qualitative interviews and debriefings. These data will be available no later than May 1, 2019.

9. Division and Office Responsibilities

• The Census Redistricting and Voting Rights Data Office (CRVRDO) will be heavily involved with the RDP team in developing requirements for the GEOP and DPD teams and in conducting feedback interviews with the state stakeholders.

• The Geography Division (GEO) will be responsible for creating the geographic support products based on requirements provided by the RDP, delivering the products to the RDP, and releasing the products on the Census Bureau’s website.

• The Decennial Census Management Division (DCMD) will be involved in the P.L. 94-171 redistricting data product requirements development.

• The American Community Survey Office (ACSO) is responsible for tabulating data from the test sites to create the P.L. 94-171 redistricting data tabulations file product.

• The Decennial Information Technology Division (DITD) is responsible for the CEDSCI system, the system that will control the release of the P.L. 94-171 redistricting data tabulations file, including managing the embargoed release.

• The Administrative and Customer Services Division (ACSD) is responsible for creating data on backup removable media.

Milestone Schedule

Below are the standard schedule activities for the development of the study plan and operational assessment report. Regard those highlighted in bold block-face as the key milestone activities. Definitions of acronyms are noted in the glossary section.

Activity ID	Activity Name	Orig Duration	Start	Finish
Redistricting Data Program (RDP) Assessment Study Plan
First Draft
	Prepare First Draft of RDP Assessment Study Plan	15	12/17/2016	12/27/2016
	Distribute First Draft of RDP Assessment Study Plan to the Assessment Sponsoring DCMD ADC and Other Reviewers	1	12/27/2016	12/27/2016
	Incorporate CRVRDO Chief and Other Comments to RDP Assessment Study Plan	5	12/28/2016	01/04/2017
Initial Draft
	Prepare Initial Draft RDP Assessment Study Plan	5	01/05/2017	01/11/2017
	Distribute Initial Draft RDP Assessment Study Plan to Evaluations & Experiments Coordination Brach (EXC)	1	01/12/2017	01/12/2017
	EXC Distributes Initial Draft RDP Assessment Study Plan to the DROM Working Group for Electronic Review	1	01/12/2017	01/12/2017
	Receive Comments from the DROM Working Group on the Initial Draft RDP Assessment Study Plan	5	01/13/2017	01/20/2017
	Discuss DROM Comments on Initial Draft RDP Assessment Study Plan	1	01/23/2017	02/06/2017
Final Draft
	Prepare Final Draft of RDP Assessment Study Plan	15	02/07/2017	02/28/2017
	Distribute Final Draft of RDP Assessment Study Plan to the DPMO and the EXC	1	03/01/2017	03/01/2017
	Schedule and Discuss Final Draft RDP Assessment Study Plan with the 2020 PMGB	14	03/02/2017	03/15/2017
	Incorporate 2020 PMGB Comments for RDP Assessment Study Plan	5	03/16/2017	03/29/2017
	Prepare FINAL Programs RDP Assessment Study Plan	5	03/16/2017	03/29/2017
	Distribute FINAL Geographic Programs RDP Assessment Study Plan to the EXC	1	03/29/2017	03/29/2017
	EXC Staff Distributes the RDP Assessment Study Plan and 2020 Memorandum to the DCCO	3	3/30/2017	4/3/2017
	DCCO Staff Process the Draft 2020 Memorandum and the RDP Assessment Study Plan to Obtain Clearances (DCMD Chief, Assistant Director, and Associate Director)	30	4/4/2017	5/16/2017
	DCCO Staff Formally Release the RDP Assessment Study Plan in the 2020 Memorandum Series	1	5/17/2017	5/17/2017
RDP Assessment Report
First Draft of Assessment Report
	Receive, Verify, and Validate RDP Assessment Data	20	4/1/2019	5/3/2019
	Examine Results and Conduct Analysis	20	5/6/2019	5/31/2019
	Prepare First Draft of RDP Assessment Report	15	06/03/2019	6/21/2019
	Distribute First Draft of RDP Assessment Report to the Assessment Sponsoring DCMD ADC and Other Reviewers	1	06/24/2019	06/24/2019
	Incorporate DCMD ADC and Other Comments RDP Assessment Report	7	06/25/2019	07/08/2019
Initial Draft of Assessment Report
	Prepare Initial Draft RDP Assessment Report	8	07/09/2019	07/19/2019
	Distribute Initial Draft RDP Assessment Report to Evaluations & Experiments Coordination Br. (EXC)	1	07/22/2019	07/22/2019
	EXC Distributes Initial Draft RDP Assessment Report to the DROM Working Group for Electronic Review	1	07/23/2019	07/23/2019
	Receive Comments from the DROM Working Group on the Initial Draft RDP Assessment Report	10	07/24/2019	08/07/2019
	Schedule the RDP Report for the IPT Lead to Meeting with the DROM Working Group	10	08/08/2019	08/22/2019
	Discuss DROM Comments on Initial Draft RDP Assessment Report	1	08/23/2019	08/23/2019
Final Draft of Assessment Report
	Prepare Final Draft of RDP Assessment Report	25	08/26/2019	09/27/2019
	Distribute Final Draft of RDP Assessment Report to the DPMO and the EXC	1	09/30/2019	09/30/2019
	Schedule and Discuss Final Draft RDP Assessment Report with the 2020 PMGB	14	10/01/2019	10/21/2019
	Incorporate 2020 PMGB Comments RDP Assessment Report	10	10/22/2019	11/05/2019
Final Assessment Report
	Prepare FINAL RDP Assessment Report	10	11/06/2019	11/20/2019
	Deliver FINAL RDP Assessment Report to the EXC	1	11/21/2019	11/21/2019
	EXC Staff Distribute the FINAL RDP Report and 2020 Memorandum to the DCCO	3	11/22/2019	11/26/2019
	DCCO Staff Process the Draft 2020 Memorandum and the FINAL RDP Report to Obtain Clearances (DCMD Chief, Assistant Director, and Associate Director)	30	11/27/2019	01/10/2020
	DCCO Staff Formally Release the FINAL RDP Report in the 2020 Memorandum Series	1	01/13/2020	01/13/2020
	EXC Staff Capture Recommendations of the FINAL RDP Report in the Census Knowledge Management SharePoint Application	1	01/14/2020	01/14/2020

10. Review/Approval Table

Role	Electronic Signature	Date
Author’s Division Chief (or designee)	<James Whitehorne>	01/11/2017
DCMD ADC	N/A
DROM DCMD co-executive sponsor (or designee)	<Deborah Stempowski>
DROM DSSD co-executive sponsor (or designee)	<Patrick Cantwell>
Associate Director for R&M (or designee)	<John Abowd>
Associate Director for Decennial Census Programs (or designee), 2020 PMGB	<Lisa Blumerman>

11. Document Revision and Version Control History

VERSION/EDITOR	DATE	REVISION DESCRIPTION	EAE IPT CHAIR APPROVAL
v. 0.1	12/23/2016	First Draft to CRVRDO
v. 0.2	1/12/2017	Initial Draft for EXC
v. 0.3	2/28/2017	Final Draft to PMGB
v. 0.4	3/29/2017	Final Plan to EXC
v. 0.5	4/4/2017	Incorporates editorial comments from DCMD
v. 0.6	5/2/2017	Incorporates additional editorial comments from DCMD
v. 0.7	6/5/2017	Incorporates additional editorial comments from DCMD

12. Glossary of Acronyms

Acronym	Definition
ADC	Assistant Division Chief
CEDSCI	Center for Enterprise Dissemination Services and Consumer Innovation
CRVRDO	Census Redistricting and Voting Rights Data Office
DCCO	Decennial Communications Coordination Office
DPD	Data Products and Dissemination
DPMO	Decennial Program Management Office
DROM	Decennial Research Objectives and Methods Working Group
DSSD	Decennial Statistical Studies Division
EXC	Evaluations & Experiments Coordination Branch
GEOP	Geographic Operations
IPT	Integrated Project Team
PMGB	Portfolio Management Governance Board
R&M	Research & Methodology Directorate
RDP	Redistricting Data Program
VTD	Voting District

13. References

U.S. Census Bureau (2015), “2020 Census Detailed Operational Plan for: 22. Redistricting Data Program Operation (RDP),” Version 1.0 Final, September 8, 2016.

U.S. Census Bureau (2014), “Designing P.L. 94-171 Redistricting Data for the Year 2020 Census: The View from the States,” December 2014.

U.S. Census Bureau (2016), “Customer Requirements Document for Phase 3 of the Census Redistricting Data Program – Delivery of the 2018 End-To-End Test Redistricting Data (P.L 94-171) Geographic Products,” Version 1.0, November 22, 2016.

14