2018EndtoEndCensusTest_Operational Assessment Content Guidelines

Version 1.2

Operational Assessment Content Guidelines for 2018 End-to-End Census Test and the 2020 Census

May 19, 2017

Program Scope

Decennial operations are required to produce operational assessments for the 2018 End-to-End Census Test and the 2020 Census. Since the Decennial Census Management Division (DCMD) is accountable and responsible for decennial operations, the Program Managers ultimately determine whether any operation is exempt from this requirement for the 2020 Census. They also ultimately advise the Integrated Project Teams (IPT) that they sponsor on the scope of assessment content for each operation. Since the nature of operations varies, the content of what’s to be reported for each operation will also vary. For example, where the assessment for the Forms Printing and Distribution operation includes lessons learned and planned versus actual variance analyses on budget, workload, and schedule, other operations will have additional content. The Nonresponse Followup assessment would also include, in the least, staffing ratios, training, enumerator productivity rates, debriefings, etcetera.

For the 2020 Census all 35 operations, below, are required to produce study plans and reports. Note that some operations may produce more than one assessment. As examples, Address Canvassing may separately assess In-Office and In-Field operations; Non-ID Processing may separately assess fraud detection; and Field Infrastructure may separately assess recruiting/hiring, payroll, training, and office administration. For the 2018 End-to-End Census Test, the 18 operations highlighted in bold block-face are required to produce study plans and reports.

Program Management	Systems Engineering & Integration	Security, Privacy, & Confidentiality
Content and Forms Design	Language Services	Geographic Programs
Local Update of Census Addresses	Address Canvassing	Forms Printing and Distribution
Paper Data Capture	Integrated Partnership & Communication	Internet Self-Response
Non-ID Processing	Update Enumerate	Group Quarters
Enumeration of Transitory Locations	Census Questionnaire Assistance (CQA)	Nonresponse Followup
Response Processing	Federally Affiliated Americans Count Overseas	Data Products and Dissemination
Redistricting Data Program	Count Review	Count Question Resolution
Archiving	Island Areas Censuses	Post-Enumeration Survey Design & Estimation
Post-Enumeration Survey Matching	Post-Enumeration Survey Field Operations	Evaluations and Experiments
Decennial Service Center	Field Infrastructure	Decennial Logistics Management
IT Infrastructure	Update Leave

In addition to operational assessments, other non-evaluative studies will be produced. The Decennial Statistical Studies Division (DSSD) will prepare cross-operation studies for the 2018 End-to-End Census Test and the 2020 Census. One would be specific to self­-response, including the Census Questionnaire Assistance, Internet Self-response, and Paper Data Capture operations and that would include self-response rates overall and by mode and geographic area. Other examples would include a report on item nonresponse and imputation rates, mail response/return rates, administrative records usage, and the Master Address File (MAF) Coverage Study.

Assessment Responsibilities

Integrated Project Teams (IPTs) are responsible for specifying general scope (including assessment topics or questions), critically reviewing study plans and reports, and specifying when the final results are needed. IPT representatives from the DCMD and the DSSD will work as co-authors of assessment study plans and reports. The DCMD team representative will take the lead in preparing the assessment study plans and reports. The DSSD team representative will be responsible for specifying operational data requirements, deriving appropriate methods, and producing frequency distributions and standard demographic and address frame tables, where appropriate. Other divisions may be called on to help develop assessment study plans and reports.

The Evaluations and Experiments (EAE) IPT supports implementation of operational assessments by providing content guidelines, workflows, study plan and report templates, checklists, email transmittal templates, standard schedule activities, and standard demographic and address table shells. The EAE IPT is chaired by the Branch Chief for Evaluations and Experiments in the DCMD. The chair will manage the IPT in close collaboration with the Branch Chiefs for Census Experiments and Evaluations in the DSSD. The DSSD Branch Chiefs will take the lead on technical issues related to program design, implementation, evaluation/experiment operations, assessment data requirements, and analysis. DCMD will take the lead on the coordination, communication, documentation, management issues, and the supporting research processes.

2020 Census Research Governance Framework

Study plans and reports will undergo gate reviews under the 2020 Census Research Governance Framework. The four stages of study plan and report development are First Draft, Initial Draft, Final Draft, and Final. The First Draft is for review by the sponsoring DCMD ADC of the IPT, the home division/office of the author, and/or subject matter experts. This is an “internal” clearance step before the report is distributed to the research governing bodies. The Initial Draft is the first formal review that involves a research governing body. Initial Draft gate reviews are conducted by the Decennial Research Objectives and Methods (DROM) working group. Once Initial Draft Reports are vetted through the DROM, the results can be regarded as preliminary and be shared publicly on request. Recommendations also are to be captured in the Census Knowledge Management application.

Final Draft gate reviews are conducted by the 2020 Census Portfolio Management Governing Board (2020 PMGB). Briefings for the 2020 Census Executive Steering Committee are conducted on an as requested basis, regardless of the development stage.

Guidelines for Operational Assessment Content

The content guidelines apply to the 2018 End-to-End Census Test and the 2020 Census, but for instances where guidelines vary between them, an explanation will be provided. Operational assessments include some discussion of data, but do not include analysis and explanation of error. In general, operational assessments report out on: planned to actual variances as it relates to budget, schedules, and production and training workloads; on outcomes of meeting the success criteria--as documented in the study plan; frequency distributions that are used to document volumes and rates; results from field debriefings and focus groups; lessons learned derived by the IPT; on knowledge management action plans and recommendations; for enumeration operations, on standard demographic tables; for frame operations, on standard address tables. In the least, all operations should have lessons learned.

The full range of content that could be in an operational assessment is listed below. For all content items that will be included in the operational assessment, the study plan should define questions that will be addressed in the report. IPT Leads should work with their sponsoring DCMD Program Manager to obtain their approval of the proposed content; this can be accomplished early on during the First Draft gate review.

Operational Assessment Content

1. Executive Summary. Summarize key findings based on questions to be answered, and recommendations. Add summary tables or figures if useful to convey key findings of operational assessment. When incorporating conclusions of the operational assessment, paraphrase the conclusions in the Executive Summary. Include limited introduction, background, and methodology, as necessary, but focus on the main results and recommendations. Include high-level research questions and the answers to the questions. Include a table or chart, if needed, to present main findings. Use the full citation if a reference is cited. Do not use acronyms in the Executive Summary. Do not include any information in the Executive Summary that isn’t also in the body of the report.

2. Introduction and Background. In consideration of four innovation areas¹ for 2020, describe the background of whether and how this operational assessment is tied and useful to innovations under implementation, if applicable. Not all operations will have a direct alignment here. Using In-Office Address Canvassing as an example, there would be narrative on how the reduction in the in-field address canvassing workload would reduce overall cost. In addition and if applicable, tie the study background to major external studies that are relevant to the operational assessment. As an example for Non-ID Processing, there’s the JASONs group report, Respondent Validation for Non-ID Processing in the 2020 Decennial Census.

3. State relevant assumptions, if decisions have yet to be made about some operational component.

4. Limitations. As a reminder for eventual external audiences, include the following for 2018 End-to-End Census Test operational assessments.

“The 2018 End-to-End Census Test is an important opportunity for the Census Bureau to ensure an accurate count of the nation’s increasingly diverse and rapidly growing population. It is the first opportunity to apply much of what has been learned from census tests conducted throughout the decade in preparation for the nation’s once-a-decade population census. The 2018 End-to-End Census Test will be held in three locations, covering more than 700,000 housing units: Pierce County, Washington; Providence, Rhode Island; and the Bluefield-Beckley-Oak Hill, West Virginia area.”

“The 2018 End-to-End Census Test will be a dress rehearsal for most of the 2020 Census operations, procedures, systems, and field infrastructure to ensure there is proper integration and conformance with functional and non-functional requirements. The test also will produce a prototypes of geographic and data products. Note that the 2018 End-to-End Census Test results are based on three sites that were purposely selected and cannot be generalized to the entire United States.”

5. Workloads and workflow. Study plan to address by posing questions-to-be-answered. Document production and where appropriate, quality control, case workloads and final dispositions.

0. Workload examples include: caseloads, incoming counts; outgoing counts; the actions or final dispositions; mode of case completion or resolution [e.g., self-response by Internet survey, CQA in-bound telephone, Computer-Assisted Personal Interview (CAPI), administrative records (AR)]; use of language options; contact and attempt history when available by case; reassigned work number and reasons; and any other operational tallies, as required.

0. Workflow examples include incoming work by day², workflow of resolved or completed cases to processing over time; workflow of in-office and in-field address canvassing; and workflow of reinterview cases from field through Sampling, Matching, Review, and Coding System (SMARCs), back to field by resolution and timeliness.

This includes interface functionality/efficiencies on data receipts/deliveries with Headquarters processing/servicing areas, such as with Census Enterprise Data Collection and Processing (CEDCaP).

6. Schedule – How did actual start and completion dates compare to planned start and completion dates? In other words, document the original schedule and any departures from it. Include impact assessments for any schedule variance greater than 7 days. Study plan to address by posing questions-to-be-answered. So in this case, the question could be, What’s the explanation for variances of more the seven days between actual milestone completions versus what was planned?

7. Budget – How does the planned budget compare with the actual costs? In other words, document the budget for the operation and the actual costs. Provide explanations for variances greater than $100,000. <<As more guidance becomes available from the Decennial Census Management Division's budget offices, more detailed requirements will be provided.>>

8. Staffing – Field or National Processing Center (NPC) staffing by position and over time (authorized, invited to training, trained, worked, replacements) – both field and office. If new staffing ratios were operationalized, provide an assessment on efficiencies. Study plan to address by posing questions-to-be-answered-- Were there challenges posed by the new staffing ratios and was there an impact on the throughput of workload? Were there problems or issues for the field supervisor to handle the number of enumerators? Could the field supervisor have handled more enumerators?

9. Training – For example, number and duration of training sessions and number of replacement training sessions. Study plan to address by posing questions-to-be-answered—Did any staggered training schema work? What was the effectiveness of the on-line and classroom training modules? Were there significant gaps in enumerators’ knowledge that needs to be addressed?

10. Automation implementation, as appropriate – For example, equipment performance/breakage/downtime/repair/loss/maintenance records and summary of automation problems documented by transmission logs, help tickets, etc. (need to try to quantify impact). Study plan to address by posing questions-to-be-answered.

11. Major findings from structured observation reports.

12. Results from focus groups with respondents.

13. Debriefings – Information derived from debriefings with field and/or office staff.

14. Lessons Learned captured by the IPT – what worked well and what needs improvements. Emphasize lessons learned that are informative on evolving the operation for 2030. In other words, what is critical for capturing in Knowledge Management so that it’s acted upon in early 2030 research and testing efforts?

15. Results from answering specific assessment questions, which have not been included in other sections.
    
    

16. Cost-data quality trade-off analysis where applicable. Even though trade-off analysis may not be appropriate for some operations, most operations should do what’s possible. Since trade-off analysis is not possible in the context of an end-to-end test operations, IPTs should focus on the derivation of cost and quality measures.

17. Standard Demographic Tables are proposed for the operations listed in Appendix A. IPTs for the following operations will not need to provide standard demographic tables: Census Questionnaire Assistance, Internet Self-Response, and Paper Data Capture. The Decennial Statistical Studies Division (DSSD) will prepare one cross-operation assessment specific to self-response for the 2018 End-to-End Census Test and the 2020 Census, which may include the demographic tables.

If demographic tables are to be used, the standard demographic tables are to be used at a minimum. Other customized tables can be derived at the discretion of the IPT.

18. Standard Address Tables are proposed for the operations listed in Appendix B. If standard address tables are to be used for the following operations, the standard demographic tables are to be used at a minimum. Other customized tables can be derived at the discretion of the IPT.

19. Census Knowledge Management. For the study plan, document whether existing action plans of past recommendations assigned to the IPT will be addressed. For the report, document how past recommendations were addressed and identify new recommendations that need to be addressed during the 2020 Census and/or the 2022-2025 Research and Testing Phase.

20. Performance measurement assessment, derived by addressing the success criteria that were established in the study plan.
    
    

For 2020 Census operations, the corresponding Measures of Success will be documented in the operational assessment study plans and final reports. The operational assessment study plan documents the criteria that will be used to define successful completion of the operation. The operational assessment report will provide results on whether the criteria were met.

Types of success measures include:

• Process Measures – that indicate how well the process works, typically including metrics related to completion dates, rates, and productivity rates.

• Cost Measures – that drive the cost of the operation and comparisons of actual costs to planned budgets. Costs can include workload as well as different types of resource costs.

• Quality Measures – of the quality of the results of the operation, typically including metrics such as rework rates, error rates, and coverage rate

• Field Productivity Measures – that indicate degree of lister or enumerator efficiency as measured by productivity rates against expectations.

• Other measures that are unique to the operation.

21. Risks
    
    Project Management Body of Knowledge (PMBOK) 2013 defines risk as an uncertain event or condition that, if it occurs, has a positive or negative effect on a project objective. Summarize if there are any risks associated with the successful completion of the operational assessment.
    
    For example, when administrative records are used, the results leveraged by the linked data may be subject to various errors of administrative records.
    
    

22. Conclusions and/or recommendations for the 2020 Census and/or the 2022-2025 Research and Testing Phase.

Indexing and Fact-Checking Requirements

During report development, operational assessments are required to be indexed and independently fact-checked. Indexing goes beyond the creation of references, citations, and a bibliography and serves a very different function. References are used to cite the source of information in a very general manner. Indexing is used to ensure that specific data items are reflected correctly from source documents to the report.

Starting with the Initial Draft development, the author begins compiling all sources needed to prepare results and to index. The process of indexing provides specific references for each fact, number, percentage, or other information in the report. By identifying the exact source of information throughout the report, authors will be providing a link between results and recommendations to the substantiating source.

Some form of validation and verification are required to ensure that all numbers and statements have been correctly included in the report. If double programming and/or code review are formally implemented, full formal fact-checking is not required. However, efforts must take place to ensure data items in the report are without error, such as transposition errors. Absent of these validation efforts, full fact-checking of the report is required. Examples include: ensuring that numbers in tables match the corresponding numbers in the text; that numbers haven’t been transposed; and that statements taken from another document are correct and did not change the meaning of the original document.

Study Plan and Report Review Protocol

Protocols for phase gate sign-off for study plans and reports are as follows:

First Draft

• Fact Checker or independent validation

• IPT Lead

• Author’s Division Chief, or designee

Initial Draft for the DROM research governing body

• DCMD Program Manager

• DROM co-executive sponsor – DCMD, or designee

• DROM co-executive sponsor – DSSD, or designee

• Associate Director for Research and Methodology (R&M), or designee

Final Draft for the 2020 PMGB research governing body

• 2020 PMGB – Associate Director for Decennial Census Programs, or designee

Final study plans and reports will undergo 2020 Census memorandum clearance for internal release. Given the visibility of the topic, some study plans and reports will be released in the external 2020 Census Memorandum series.

Ultimately for the 2020 Census program, indexing and fact-checking helps ensure the release of accurate, validated, and sound reports. See Guidelines for Indexing and Fact-Checking 2018 End-to-End Census Test and 2020 Census Reports [forthcoming], for detailed steps on the process.

Document Revision and Version Control History

VERSION/EDITOR	DATE	REVISION DESCRIPTION	EAE IPT CHAIR APPROVAL
v. 1.0 / Erin Love	12/08/2016	Final	Randall Neugebauer
v 1.1/Randall Neugebauer	04/27/2017	Revisions from the DROM working group
v 1.2/Randall Neugebauer/Miranda Chung	0519/2017	Revisions from Maryann Chapin and the added Update Leave operation

STANDARD DEMOGRAPHIC TABLES FOR THE FOLLOWING OPERATIONS:

IPTs for the following operations will not need to provide standard demographic tables: Census Questionnaire Assistance, Internet Self-Response, and Paper Data Capture. The Decennial Statistical Studies Division (DSSD) will prepare one cross-operation assessment specific to self-response for the 2018 End-to-End Census Test and the 2020 Census, which may include the demographic tables.

Standard Demographic Tables are proposed for the following operations. If demographic tables are to be used, the standard demographic tables are to be used at a minimum. Other customized tables can be derived at the discretion of the IPT.

Nonresponse Followup

Update Enumerate

Group Quarters Enumeration

Non-ID Processing

Coverage Measurement

Response Processing

Enumeration at Transitory Locations

Federally Affiliated Americans Count Overseas

Island Area Censuses

Administrative Records comprehensive

Coverage Followup

TABLE SHELLS FOR ASSESSMENT STANDARD DEMOGRAPHIC TABLES

Table XX1. Standard Assessment Demographic Table for Sex

Sex	Number	Percent
Total Population		100.0
Male
Female
Both*
Missing

*This category is only valid in paper data capture

Source: DRF

Table XX2. Standard Assessment Demographic Table for Age and Sex

Age and Sex	Number	Percent
Total Population		100.0
Under 5 years
5 to 9 years
10 to 14 years
15 to 19 years
20 to 24 years
25 to 29 years
30 to 34 years
35 to 39 years
40 to 44 years
45 to 49 years
50 to 54 years
55 to 59 years
60 to 64 years
65+ years
Missing
Male
Under 5 years
5 to 9 years
10 to 14 years
15 to 19 years
20 to 24 years
25 to 29 years
30 to 34 years
35 to 39 years
40 to 44 years
45 to 49 years
50 to 54 years
55 to 59 years
60 to 64 years
65+ years
Missing
Female
Under 5 years
5 to 9 years
10 to 14 years
15 to 19 years
20 to 24 years
25 to 29 years
30 to 34 years
35 to 39 years
40 to 44 years
45 to 49 years
50 to 54 years
55 to 59 years
60 to 64 years
65+ years
Missing

Source: DRF

Note: (add notes to explain table-specific hints helpful to the reader (i.e. rounding, cautions, surrounding interpretations, etc.)

Table XX3. Standard Assessment Demographic Table for Race and Ethnicity

Race and Ethnicity	Number	Percent
Total Population		100.0
White, Alone
Hispanic, Latino, or Spanish, Alone**
Black or African American, Alone
Asian, Alone
American Indian or Alaska Native, Alone
Middle Eastern or North African, Alone***
Native Hawaiian or Other Pacific Islander, Alone
Some Other Race, Ethnicity, or Origin, Alone
Two or More
Write-In Only*
Missing

*Write-in accepted if write-in box was filled and no other race categories were selected

** The wording of this category could change for 2018 and 2020, please make sure correct category wording is used

***Pending OMB final definitions for 2020

Source: DRF

Table XX4. Standard Assessment Demographic Table for Relationship

Relationship	Number	Percent
Total Population		100.0
Householder
Opposite-sex Husband/Wife/Spouse
Opposite-sex Unmarried Partner
Same-sex Husband/Wife/Spouse
Same-sex Unmarried Partner
Biological Son or Daughter
Adopted Son or Daughter
Stepson or Stepdaughter
Brother or Sister
Father or Mother
Grandchild
Parent-in-law
Son-in-law or Daughter-in-law
Other Relative
Roommate or Housemate
Foster Child
Other Nonrelative
On Extended Roster*
Related to Householder
Not Related to Householder
Both
None
Two or More Relationships*	(N/A)	(N/A)
Missing

*This category is only valid in paper data capture Source: DRF

Table XX5. Standard Assessment Demographic Table for Tenure

Tenure	Number	Percent
Total Housing Units		100.0
Owned with a mortgage or a loan
Owned without a mortgage or a loan
Rented
Occupied without payment of rent
Multiple*
Missing

*This category is only valid in paper data capture Source: DRF

Notes about the inclusion of the Standard Demographic Tables:

Not every table will be included in every assessment. If an assessment calls for a table within this document please use the table shell provided. For example, if the assessment includes a table for sex counts, use Table XX1 in the assessment.

In addition, these tables contain the minimum information to be reported for each demographic category, if reported. An operation may add more detail, if warranted. For example, if the author wanted to include more details on the older population then adding more age categories (65-69, 70-74, 75-79, 80-84, 85-89, 90+) is welcome as long as the original categories in the age table shell are present.

• One main purpose of the standard demographic tables is so assessments have table shells to use and, if used, the tables will be consistent across assessments.

• Authors will need to adjust the table so that a demographic item does not split across pages.

• Include Data Defined Persons (DDP) or residents in calculating the tables, as appropriate to the assessment.

• Only the boilerplate language should be included with the tables. Comment on each table to describe what the table shows. Basic conclusions will be based off demographic results.

• The tables may be placed in an Appendix or appear as part of the report. If one of the research questions pertains to demographics then the table(s) should appear in the body of the report.

• When coding the demographic tables you should not use the STD_* variables but use the individual checkboxes instead.

• Use the variable GROUPTAGID or P_ROSTER_INDEX_INT³ (person number) to identify extended roster persons.

• Race and Ethnicity Extended Roster: if you are working with forms that contain an extended roster you should address the issue that some persons were not asked the race and ethnicity question. You can include these people in the missing total and then explain it. No standard wording is provided for this since it will vary from report to report, but a suggestion that you can modify is as follows:

  • Persons 7 through 10 on English-only mailback forms and persons 7 through 10 on Bilingual mailback forms, called extended roster persons, were not asked all demographic information. A shortened relationship question was asked for them. No information on race and ethnicity was collected. Missing rates for race and ethnicity will be inflated because of this.

  • You could possibly footnote the missing rates for race and ethnicity and include counts of how many of these people are extended roster persons.

• Race and Ethnicity Coding:

  • The ‘Write-in Only’ category should include cases where none of the checkboxes are marked and any one or more of the write-in fields are filled.

  • (Explain the Hispanic Ethnicity placement in the race question if that is the final decision).

• Relationship: If the household returns an electronic form and does not answer relationship, the reference person is considered the householder. If the household returns a paper form and does not answer relationship, person 1 should be reported as the householder. If you are working with forms that contain an extended roster you should address the issue that some persons were not asked detailed Relationship.

• Age: Clarifications on the original algorithm provided with the agenda and minutes were received from POP. Below is an updated algorithm. You can also obtain SAS code from Julia Combs or Elizabeth Poehler which has been independently double programmed and validated.

Variables Used: P_AGE_INT, P_BIRTH_MONTH_INT, P_BIRTH_DAY_INT, P_BIRTH_YEAR_INT.

Create calculated age as follows:

1. Range check:

a. If P_BIRTH_MONTH_INT not in 01-12, then set to blank===>INTERMEDIATE_MOB

b. If P_BIRTH_YEAR_INT not in [(censusyr-116)-(censusyr)] then set to blank====>INTERMEDIATE_YOB

(do not convert 2-digit years to 4-digit years, use whatever is in the field as is)

2. Calculating an age (CALCULATED_AGE):

a. Calculate an age where P_BIRTH_MONTH_INT is not blank nor 4:

If INTERMEDIATE_YEAR and INTERMEDIATE_MONTH are not blank and P_BIRTH_MONTH_INT is not 4, then:

Create a string variable (used in formula for calculated age) = STRING_DATE1

STRING_DATE1=INTERMEDIATE_YOB*100+INTERMEDIATE_MOB

CALCULATED_AGE=floor(((censusyr)04-STRING_DATE1)/100)

Ignore decimal, do not round.

Now you have a calculated age (CALCULATED_AGE) and a reported age (P_AGE_INT).

If CALCULATED_AGE exists and is in range 0-115, then FINAL_AGE=CALCULATED_AGE

Else if CALCULATED_AGE does not exist and P_AGE_INT does exist and is in range 0-115, then FINAL_AGE=P_AGE_INT

Else if calculated_age is in range 116-125 or P_AGE_INT is in range 116-125 then final_age=BLANK

Else, FINAL_AGE=blank.

If the date of birth is reported after census day but P_AGE_INT is 0 then final age should be reported as missing

b. Calculate an age where P_BIRTH_MONTH_INT=4

If P_BIRTH_DAY_INT not in (01-30 for months=04) then set to blank====> INTERMEDIATE_DOB

If INTERMEDIATE_YEAR and INTERMEDIATE_MONTH and INTERMEDIATE_DAY exist, then:

Create a string variable (used in formula for calculated age) = STRING_DATE2

STRING_DATE2=INTERMEDIATE_YOB*10000+INTERMEDIATE_MOB*100+INTERMEDIATE_DOB

CALCULATED_AGE=floor(((censusdate in YYYYMMDD format)-STRING_DATE2)/10000)

Ignore decimal, do not round.

Now you have a calculated age (CALCULATED_AGE) and a reported age (P_AGE_INT).

If CALCULATED_AGE exists and is in range 0-115, then FINAL_AGE=CALCULATED_AGE

Else if CALCULATED_AGE does not exist and P_AGE_INT does exist and is in range 0-115, then FINAL_AGE=P_AGE_INT

Else if calculated_age is in range 116-125 or P_AGE_INT is in range 116-125 then final_age= BLANK

Else, FINAL_AGE=blank.

If the date of birth is reported after census day but P_AGE_INT is 0 then final age should be reported as missing

c. If CALCULATED_AGE is not able to be calculated then:

if P_AGE_INT exists and is in the range of 0-115 then FINAL_AGE=P_AGE_INT

else if P_AGE_INT exists and is in the range of 116-125 then FINAL_AGE=BLANK

else FINAL_AGE=blank

Boiler Plate Language

Before the tables, include the following:

There were XXX,XXX data defined persons included on XXX,XXX (operation name) forms in the 2018 Census End-to-End Test [or 2020 Census]. This section will present the demographic characteristics for these persons on the (operation name) form. Tables XX1-XX5 gives (operation name) person demographic characteristics: sex, age, race and ethnicity, and relationship to person 1. Age was calculated based on the date of birth provided; if no date of birth was provided then the reported age was used. Age was calculated only if the date of birth fell within valid date ranges. Similarly, the calculated age or reported age was used only if it fell within valid age ranges; otherwise, it was considered missing. Table XX5 gives the distribution of tenure responses for housing units included in the (operation name) operation.

Because the demographic data used in this (evaluation/assessment/experiment) are unedited, direct comparisons with published 2018 Census End-to-End Test [or 2020 Census] results are not possible. These tables include a row for people with missing values for the specific characteristic. The data in published Census reports have undergone editing and imputation, and therefore will have no missing values.

After the (last) table, include the following statement:

These distributions may vary across different census operations due to differences in corresponding populations and census procedures.

STANDARD ADDRESS TABLES FOR THE FOLLOWING OPERATIONS:

Standard Address Tables are proposed for the following operations. If standard address tables are to be used for the following operations, the standard demographic tables are to be used at a minimum. Other customized tables can be derived at the discretion of the IPT.

Address Canvassing – In-Office and In-Field

Geographic Programs

2020 Local Update of Census Addresses

Non-ID Processing

SAMPLE TABLE SHELLS FOR ASSESSMENT STANDARD

ADDRESS TABLES

DRAFT 2018 (TEST SITES) TABLE SHELLS: Housing Unit Tables

DSSD / Program Evaluation

Version X.X, released (Month Date, Year)

	Table N1.
	2018 (TEST SITES) <<Operation>>
	Addresses	Count*	Percent of total+
	Total ......................................................................................
	United States .........................................................................
	Puerto Rico ............................................................................
	*Counts and percentages are unweighted.
	*Counts and percentages are weighted; standard errors in parentheses.
	+Percentages may not sum to 100 due to rounding.
	1Footnote.
	Source: X.

Table N2.
2018 (TEST SITES) <<Operation>>: <<Universe>> by Occupancy Status
Housing Unit Type	Count*	Percent of total+
Total......................................................................................		100.00
Occupied Unit.......................................................................
Vacant Unit.........................................................................
For Sale/Sold Not Occupied…...........................................
For Rent/Rented Not Occupied.......................................... Seasonal Unit………………………………………………….
Other.....................................................................
Missing………………………………………………………….
*Counts and percentages are unweighted.
*Counts and percentages are weighted; standard errors in parentheses.
+Percentages may not sum to 100 due to rounding.
1Footnote.
Source: X.

Table N3.
2018 (TEST SITES) <<Operation>>: <<Universe>> by Address Type
Type of Address Information	Count*	Percent of total+
Total ......................................................................................		100.00
Complete City-Style ................................................................
With complete Rural Route
and/or complete P.O. Box and/or location description .........
Without complete Rural Route
or complete P.O. Box or location description ......................
Complete Rural Route .............................................................
With location description .....................................................
Without location description .................................................
Complete P.O. Box .................................................................
With location description .....................................................
Without location description .................................................
Incomplete address information ................................................
With location description .....................................................
Without location description .................................................
No address information ............................................................
With location description .....................................................
Without location description .................................................
*Counts and percentages are unweighted.
*Counts and percentages are weighted; standard errors in parentheses.
+Percentages may not sum to 100 due to rounding.
1Footnote.
Source: X.

Table N4.
2018 (TEST SITES) <<Operation>>: <<Universe>> by Type of Enumeration Area
Type of Enumeration Area (TEA)	Count*	Percent of total+
Total ......................................................................................		100.00
TEA 1: Self Response ..........................................
TEA 2: Update Enumerate.....................................................
TEA 4: Remote Update Enumerate.................................................
TEA 5: Military.............................................
TEA 6: Island Areas......................................................................
*Counts and percentages are unweighted.
*Counts and percentages are weighted; standard errors in parentheses.
+Percentages may not sum to 100 due to rounding.
1Footnote.
Source: X.

Table N5.
2018 (TEST SITES) <<Operation>>: <<Universe>> by <<Collection or Tabulation>> Block Size
Block Size Based on Number of Addresses	Count*	Percent of total+
Total ......................................................................................		100.00
0 ...........................................................................................
1 ...........................................................................................
2-9 ........................................................................................
10-19 .....................................................................................
20-49 .....................................................................................
50-99 .....................................................................................
100-499 .................................................................................
500-999 .................................................................................
1000+ ....................................................................................
*Counts and percentages are unweighted.
*Counts and percentages are weighted; standard errors in parentheses.
+Percentages may not sum to 100 due to rounding.
1Footnote.
Source: X.

Table N6.
2018 (TEST SITES) <<Operation>>: <<Universe>> by Census Region and Division
Census Region and Division	Count*		Percent of total+
Total ......................................................................................			100.00
Region 1: Northeast ................................................................
Division 1: New England (CT,ME,MA,NH,RI,VT) ...................
Division 2: Middle Atlantic (NJ,NY,PA) ................................
Region 2: Midwest ................................................................
Division 3: East North Central (IL,IN,MI,OH,WI) ....................
Division 4: West North Central (IA,KS,MN,MO,NE,ND,SD) ...
Region 3: South ......................................................................
Division 5: South Atlantic (DC,DE,FL,GA,MD,NC,SC,VA,WV)
Division 6: East South Central (AL,KY,MS,TN) ....................
Division 7: West South Central (AR,LA,OK,TX) ....................
Region 4: West ......................................................................
Division 8: Mountain (AZ,CO,ID,MT,NM,NV,UT,WY) .............
Division 9: Pacific (AK,CA,HI,OR,WA) ................................
Puerto Rico ............................................................................
*Counts and percentages are unweighted.
*Counts and percentages are weighted; standard errors in parentheses.
+Percentages may not sum to 100 due to rounding.
1Prior to June 1984, the Midwest Region was designated as the North Central Region.
Source: X.

Table N7.
2018 (TEST SITES) <<Operation>>: <<Universe>> by State
State	Count*	Percent of total+
Total ......................................................................................		100.00
Alabama ................................................................................
Alaska ...................................................................................
Arizona ..................................................................................
Arkansas ...............................................................................
California ................................................................................
Colorado ................................................................................
Connecticut ............................................................................
Delaware ................................................................................
District of Columbia .................................................................
Florida ...................................................................................
Georgia ..................................................................................
Hawaii ...................................................................................
Idaho .....................................................................................
Illinois.....................................................................................
Indiana ...................................................................................
Iowa ......................................................................................
Kansas ..................................................................................
Kentucky ...............................................................................
Louisiana ...............................................................................
Maine ....................................................................................
Maryland ................................................................................
Massachusetts .......................................................................
Michigan ................................................................................
Minnesota ..............................................................................
Mississippi .............................................................................
Missouri .................................................................................
Montana ................................................................................
Nebraska ...............................................................................
Nevada ..................................................................................
New Hampshire ......................................................................
New Jersey ............................................................................
New Mexico............................................................................
New York ...............................................................................
North Carolina ........................................................................
North Dakota ..........................................................................
Ohio ......................................................................................
Oklahoma ..............................................................................
Oregon ..................................................................................
Pennsylvania ..........................................................................
Rhode Island ..........................................................................
South Carolina ........................................................................
South Dakota .........................................................................
Tennessee .............................................................................
Texas ....................................................................................
Utah ......................................................................................
Vermont .................................................................................
Virginia ..................................................................................
Washington ............................................................................
West Virginia .........................................................................
Wisconsin ..............................................................................
Wyoming ...............................................................................
Puerto Rico ............................................................................
*Counts and percentages are unweighted.
*Counts and percentages are weighted; standard errors in parentheses.
+Percentages may not sum to 100 due to rounding.
1Footnote.
Source X

1 Reengineering Address Canvassing, Optimizing Self-Response, Utilizing Administrative Records and Third-Party Data, and Reengineering Field Operations

2 Important for QC workloads which derive from completed cases, operations that may flow work out in waves to field, recycled cases such as deletes in Address Canvassing that need to be rechecked, and cases recycled through supervisory review such as non-interviews and Pop 99s (questionnaires with no person data for one person).

3 The person number variable will either be GROUPTAGID or P_ROSTER_INDEX_INT, we will chose the correct choice when decided.

7