Download:
pdf |
pdf2017 Census Test
Goals, Objectives, Success Criteria (GOSC) and Research Questions
November 21, 2016
Version 3.1
Scope Refinement
Pursuant to Decision Memo 2016.21
Contents
List of the Operations Participating in the Test and Program Managers ........................................................................................................................................................................................................................ 2
2017 Census Test Timeline................................................................................................................................................................................................................................................................................................. 4
Key Information ................................................................................................................................................................................................................................................................................................................. 5
Site Selection Criteria .................................................................................................................................................................................................................................................................... 5
Test Focus ..................................................................................................................................................................................................................................................................................... 5
Scope and Limitations ................................................................................................................................................................................................................................................................... 5
Operations.................................................................................................................................................................................................................................................................................... 5
Overall Test Assumptions .............................................................................................................................................................................................................................................................. 5
Test Focus Operations with Test Objectives and Research Questions .............................................................................................................................................................................................................................. 6
Support Operations with Goals, Objectives, Success Criteria and Research Questions .................................................................................................................................................................................................... 7
Support Operations Only – Summary of How the Operation will Support the Test ......................................................................................................................................................................................................... 9
Page 1 of 17
List of the Operations Participating in the Test and Program Managers
Operation/Support
Program Manager
Geographic Programs
Non-ID Processing
Evan Moffett
Program Management
Deidre Hicks
Census Questionnaire Assistance
Paper Data Capture
Forms Printing and Distribution
Sheila Proudfoot
Alexa Jones-Puthoff
Alexa Jones-Puthoff
IT Infrastructure
Systems Engineering and Integration
Security, Privacy and Confidentiality
Pete Boudriault
Raphael Corrado
Language Services
Internet Self-Response
Content and Forms Design
Jennifer Kim
Alexa Jones-Puthoff
Jennifer Kim
Response Processing
Raphael Corrado
2017 Census Test
Deborah Stempowski
Page 2 of 17
2017 Census Test Timeline – (10/25/2016)
10/28/2016
Full Package
to OMB
7/26/2016
Pre-Submission
to OMB
4/29/2016
Finalize GOSC
4/12/2016
Site Selection
announcement
& Deploy website
Sample Size
National Sample – 80,000
12/30/2016
OMB Approval
Content Reinterview Sample – 15,000
3/20/2017
Begin CQA
5/6/2016
SRR
11/14/2016
Printing and Mailing
TRR1
6/3/2016
CDR
5/16
4/1/2017
Census Day
6/16
7/16
8/16
9/16
10/16
11/16
12/16
4/10/2017 - 7/10/2017
Content Reinterview
1/17
4/1/2016
2/17
3/17
4/17
5/17
3/20/2017 - 5/12/2017
Self Response
5/5/2016
PBR
8/1/2016
ORR Kick-Off
1/3/2017
Self Response
TRR2
4/3/2017
Mailing 3
3/23/2017
Mailing 2
4/20/2017
3/20/2017
Mailing 1
Mailing 5
4/10/2017
Mailing 4
Page 3 of 17
6/17
7/17
7/31/2017
2017 Census Test
Focus of the
Test
Required to
Support the Test
NOT Included in
the Test
SUPPORT
Program Management
Program
Management
Systems
Engineering
and
Integration
Security,
Privacy, and
Confidentiality
Content and
Forms Design
FRAME
Geographic Programs
Infrastructure
Census/Survey Engineering
Language
Services
Decennial
Service
Center
Field
Infrastructure
RESPONSE DATA
Forms Printing and Distribution
Decennial
Logistics
Management
IT Infrastructure
PUBLISH DATA
Update
Enumerate
Nonresponse Followup
Data Products and
Dissemination
Local Update of
Census Addresses
Paper Data Capture
Group Quarters
Response Processing
Address Canvassing
(in-office only)
Integrated Partnership &
Communications
Enumeration at Transitory
Locations
Federally Affiliated Americans
Count Overseas
Internet Self-Response
Census Questionnaire
Assistance
Redistricting Data
Program
Count Review
Count Question
Resolution
Archiving
Non-ID Processing
TEST AND EVALUATION
OTHER CENSUSES
Island Areas Censuses
Coverage Measurement
Design and Estimation
Coverage
Measurement Matching
s Test Timeline20me
Page 4 of 17
Coverage Measurement
Field Operations
Evaluations and
Experiments
Key Information
Sample Selection
Description
Sites
Test Focus
Scope and Limitations
Operations
Focus of the test is:
Test the feasibility of collecting tribal enrollment information
The 2017 Census Test cannot be generalized to the entire nation. The results do not predict national trends or rate estimates expected in the 2020 Census.
Test Focus Operations - Goals, Objectives, Success
Criteria and Research Questions:
Overall Test
Assumptions
A nationwide sample of 80,000 households for a self-response only operation, oversampled for areas with relatively higher concentrations of people estimated to identify as
AIAN (Source: Mike Bentley)
A subsample of 15,000 households for a reinterview operation.
Content and Forms Design
Support Operations - with Goals, Objectives, Success Criteria
and Research Questions:
Non-ID Processing
Support Operations Only – Summary on how the operation
will support the test:
Program Management
Systems Engineering and Integration
Security, Privacy and Confidentiality
Language Services
IT Infrastructure
Geographic Programs
Forms Printing & Distribution
Paper Data Capture
Internet Self-response
Response Processing
Census Questionnaire Assistance
1. Mailing package will include a paper questionnaire
Internet Choice Panel receives the paper questionnaire in Contact 1 (and again in Contact 4 if they have not yet responded)
Internet First Panel receives the paper questionnaire in Contact 4 if they have not yet responded
2. The National sample test is conducted through the Self Response contact strategy only [no nonresponse followup]
Page 5 of 17
Test Focus Operations with Test Objectives and Research Questions
Goals
Success Criteria
(should be quantifiable)
Objectives
Research Questions
Identify which are implementation vs research
questions?
Content and Forms Design (Jenny Kim)
Test feasibility of collecting Tribal Enrollment
information (offer option of collecting tribal
enrollment via Internet Self Response (SR),
Telephone SR (CQA), and Paper SR
Test Tribal Enrollment question.
Have a reinterview component to
assess the validity of the tribal
enrollment question fielded
(conducted by the NPC Call
Center).
The tribal enrollment reinterview
shows that the tribal enrollment
question captures tribal enrollment
in at least one federally- or staterecognized tribe(s).
Page 6 of 17
What are the item and unit nonresponse
rates from the initial survey?
What is the consistency rate for tribal
enrollment status when comparing the
initial survey response to the
reinterview?
Support Operations with Goals, Objectives, Success Criteria and Research Questions
Goals
Objectives
Success Criteria (should be
quantifiable)
Research Questions
Maximum number of Non-ID
responses match to a valid address
record in the Census universe during
real time processing.
What were the results from real-time
matching and geocoding Stateside
addresses during self-response?
What were the results from Asynchronous
Non-ID Processing (e.g., how many
additional matches and geocodes were
derived)?
Non-ID Processing (Evan Moffett and Frank McPhillips)
Conduct Real Time Non-ID Processing
(RTNP)
Implement real-time address
processing (standardization, MAF
matching, geocoding) for Stateside
addresses.
Conduct post-RTNP automated
matching and geocoding (also known as
Asynchronous Non-ID Processing)
Utilize administrative records data
to enhance respondent-provided
address data, and then make a
further attempt to match to an
MTdb record and/or derive a
census block geocode. This will
occur on a transactional basis for
each case not matched during RTNP
(i.e., individually, not in batches).
Additional Non-ID responses are
matched to MTdb records and/or
assigned to census blocks
Conduct manual Non-ID processing
concurrent with self-response
processing
Complete all outstanding manual
processing before the first cut for
UE workload, and keep up with
daily internet and CQA response
turnaround by the nominal date
when UE or NRFU operations would
have started (this will simulate a
2020 environment where Non-ID is
Manual processing catches up with
the backlog of cases not resolved
during automated processing by the
time the initial UE or NRFU cut
would have been taken. This will
give us a measure of how much Non-
ID can reduce the UE or NRFU
workload before the operations
Page 7 of 17
What additional matches were derived
during manual Non-ID processing that
could have reduced the UE and NRFU
workload (e.g., how many, geographic
distribution, address characteristics, etc.)
What additional/updated geocodes were
derived during manual processing?
Support Operations with Goals, Objectives, Success Criteria and Research Questions
Goals
Objectives
Success Criteria (should be
quantifiable)
reducing UE and NRFU workload as
quickly as possible, given that selfresponse will remain open during
UE and NRFU).
Conduct office-based address
verification (OBAV) for eligible Non-ID
cases
Attempt to verify the existence and
census block location of all eligible
addresses from Non-ID processing
using geographic reference sources
in an office-base environment in
order to reduce field address
verification workload. Reference
sources may include available
online mapping and imagery
services.
Research Questions
even start.
The number of addresses verified for
eligible Non-ID cases in the officebased operation is maximized.
Page 8 of 17
How many of the Non-ID cases eligible for
address verification could be verified in an
office-based operation?
Support Operations Only – Summary of How the Operation will Support the Test
Goals
Objectives
Issues/Constraints
Program Management (Deidre Hicks)
Define and implement program
Implement:
management policies, processes, and Governance
the control functions for planning and Strategic Communication
implementing the 2017 Census Test.
(internal/external)
Change Management
Knowledge Management
Acquisition and Sourcing
Management
Budget Management
Schedule Management
Human Capital Management Risk
and Issues Management
Performance Measurement
Systems Engineering and Integration (Pete Boudriault and Daniel Lewis)
Gain experience moving to new IT
infrastructure – cloud computing
The Census Enterprise Data
Collection and Processing (CEDCaP)
Commercial Off-the-Shelf (COTS)
Platform and the real-time non-ID
processing systems will be hosted in
a commercial cloud.
Census will obtain metrics regarding
Page 9 of 17
Support Operations Only – Summary of How the Operation will Support the Test
Goals
Objectives
Issues/Constraints
cost versus performance as a result
of the testing to inform decisions
regarding future cloud
implementation, our ability to scale
nationally for 2020, etc.
Gain experience moving to new IT
infrastructure – services
Ensure the integration of new
systems into the Field Test that will
support the 2020 Census.
Utilize an enterprise data model.
Final list of systems supporting the field test will be determined once capability
requirements have been finalized and allocated to systems.
Integrate the following new systems
into the Test:
•
•
•
CEDCaP COTS Platform Enterprise
Census And Survey Enabling
Platform (ECaSE)
CQA
PEARSIS (Production Environment
for Administrative Records
Staging, Integration and Storage)
Each new system meets all functional
and non-functional requirements in
support of the Test.
Page 10 of 17
Support Operations Only – Summary of How the Operation will Support the Test
Goals
Objectives
Issues/Constraints
Security, Privacy, and Confidentiality (Pam Mosley, Pete Boudriault)
All planned security systems listed in
the scope for Security, Privacy and
Confidentiality are in place and
operating.
The following systems are in place
and operating:
•
•
•
•
•
•
•
Enterprise IT infrastructure and
Telecommunications Systems
Enterprise Security Operations
Center (SCC)
Enterprise Intrusion Detection
Systems (IDS)
Enterprise Security Vulnerability
management systems
Enterprise Audit Log Repository
Enterprise Operating System (OS)
Security Baseline Management
Application Security Baseline
Management System.
Language Services (Jenny Kim)
Support language needs of nonEnglish respondents.
Provide functionality enabling nonEnglish speakers to respond
Internet self-response
Page 11 of 17
Support Operations Only – Summary of How the Operation will Support the Test
Goals
Objectives
Issues/Constraints
instrument (Spanish)
Paper (Spanish)
CQA aids for the CQA agents
in Chinese, Vietnamese,
Korean, Russian, Arabic;
Spanish instrument
Provide mailing materials in Spanish
that include:
Letters, postcards
IT Infrastructure (Raphael Corrado and Pete Boudriault)
Provide the information technologyrelated infrastructure support to the
2017 Census Test, including:
•
Enterprise systems and
applications.
•
Decennial specific systems,
applications, and interfaces.
Geographic Programs (Evan Moffett and Ross Davis)
MAF/TIGER will provide MAF Extracts
and Geographic Reference Files to
support the operation.
Page 12 of 17
Support Operations Only – Summary of How the Operation will Support the Test
Goals
Objectives
Issues/Constraints
Forms Printing and Distribution (Alexa Jones-Puthoff and Mark Wolfram)
Print and distribute Internet
invitation letters, reminder postcards,
and questionnaire mailing packages
to support the 2017 Census Test
mailing strategy and enumeration of
the population.
Paper Data Capture (Alexa Jones-Puthoff and Heidi Brady)
Capture and convert data from the
2017 Census Test paper
questionnaires, including document
preparation, scanning, Optical
Character Recognition, Optical Mark
Recognition, Key from Image, and
checkout.
Internet Self-Response
(Alexa Jones-Puthoff and Jason Reese)
Test full mail contact strategies (5
mailings) in the mailout areas of the
national sample.
Refine final mail contact strategy
prior to the 2018 End to End Test.
Page 13 of 17
Support Operations Only – Summary of How the Operation will Support the Test
Goals
Response Processing
Objectives
Issues/Constraints
(Raphael Corrado and Acting Branch Chief for Response Processing)
Establish the testing enumeration
universe
Manage the self-response
enumeration strategy
Distribute workload files required
for self-response enumeration
operation
Track self-response enumeration
status by case and support
determining the course of selfresponse enumeration based on
established business rules.
Perform required response data
collection process editing and
race/Hispanic origin coding
Perform required post-data
collection processing actions in
order to prepare the data for final
decennial response file
Evaluate the ability to support
the Data Collection Phase from a
response processing perspective
including Internet, Paper, and
CQA modes.
Evaluate the ability to check for
sufficient data and identify datadefined persons.
Evaluate support for the Non-ID
processing operation to
successfully transmit and receive
geocoding data from GEO.
Evaluate the ability to conduct
required response data
processing steps, such as
standardizing demographic data
responses, race coding,
residence coding, determining
population counts, etc. in
preparation for future
unduplication work.
Evaluate the ability to produce
Page 14 of 17
Support Operations Only – Summary of How the Operation will Support the Test
Goals
Perform required steps to create
a Census Unedited File for data
analysis purposes, including
unduplicating data via the
primary selection algorithm and
performing count imputation.
Serve as the final test data
repository for input to required
analysis
Test methods for detecting
fraudulent responses
Objectives
Issues/Constraints
final data for analysis that has
completed an unduplication
effort using the primary selection
algorithm methodology
Evaluating the performance of
count imputation and creating a
Census Unedited File to be used
for test analysis and results.
Examine responses from
internet, CQA, and paper in an
attempt to identify potential
fraud, using multiple methods.
There is likely low motivation for submitting fraudulent responses during a
Census Test (as opposed to the 2020 Census), so we will likely have to rely on
simulated fraudulent responses to help test and refine our processes.
Page 15 of 17
Support Operations Only – Summary of How the Operation will Support the Test
Goals
Objectives
Issues/Constraints
Census Questionnaire Assistance (Sheila Proudfootand Kevin Zajac)
Provide contact center
infrastructure and staff to handle
inbound assistance calls from
respondents (no web chat or
email capabilities in 2017)
Assist respondents in completing
the Census questionnaire,
including capturing responses
Answer questions about Census
processes and operations
Automate certain tasks using
Interactive Voice Response (IVR)
self service solutions
Assess the interface(s) between
the CQA contractor and Census
Bureau’s systems
Page 16 of 17
Document History
Version
Date
Changes
1.0
May 4, 2016
1.1
September 21, 2016
Draft for review incorporating updates since May.
2.0
September 30, 2016
Baseline 2 incorporating updates since May.
3.0
October 19, 2016
3.1
November 21, 2016
Baseline prior to Project Baseline Review (PBR)
Refinement of scope per 2020 Census Decision Memo 2016.21
Added detail for response processing.
Page 17 of 17
File Type | application/pdf |
File Title | November 21 |
Author | Darlene L Monaco (CENSUS/DMD FED) |
File Modified | 2016-12-01 |
File Created | 2016-11-22 |