2015_Census_Test_OMB_Part_B Final

SUPPORTING STATEMENT

U.S. DEPARTMENT OF COMMERCE

U.S. CENSUS BUREAU

2015 Census Test

OMB CONTROL NUMBER 0607-XXXX

Part B – Collection of Information Employing Statistical Methods

1. Universe and Respondent Selection

The Census Bureau will conduct two mandatory surveys in 2015 to test data collection methods for the 2020 Census. To conduct the 2015 Census Tests, the Census Bureau will select a sample of up to 90,000 housing addresses from selected counties in and surrounding Savannah, GA, and adjacent counties in South Carolina, and 165,000 housing addresses in portions of Maricopa County, Arizona. Also in the Maricopa site, there will be a Nonresponse Follow-up (NRFU) portion of the 2015 Census Test that will include no more than 70,000 housing units (5,000 of which will be re-interviewed in a follow up operation to evaluate administrative records information).

The areas selected in the Savannah site will contain three sampling strata defined by high, medium, and low Internet response propensities (i.e., the expected response rates in the specific geographic area based on historical information) for households. The strata will be delineated at the census tract level, with each tract in the Savannah media market being assigned to one of the three strata. There will be no operations for or sampling of group quarters. Address sampling will be a two-part design: select the areas containing the two strata and then sample the addresses within those areas.

In the Maricopa site, the Census Bureau identified different areas of block groups based on a diversity of socio-economic population characteristics. The selection of block groups was a balance between identifying diverse areas to include in the site to test administrative record and field reenginnering strategies with the need to have clustered workloads to estimate cost differentials. The first set of block groups were located in Central Maricopa county. These block groups had high concentrations of both vacant and Hispanic populations. These were included to test the administrative record objectives related to assess the implementation in areas with high concentrations of both vacants and Hispanic populations. The second set of block groups were located in the cities of Chandler and Mesa. For both of these areas, the Census Bureau identified areas with higher 2010 return rates and lower mobility. These areas included allow the testing of the performance of administrative records in areas with higher return rates and less mobility. The third area is the outer ring of block groups on the northern border of Maricopa county. These block groups were included to allow the inclusion of areas that are located in a more remote location.

The Maricopa sample design is a two-stage sample design. The block groups were initially allocated to the control panel and two experimental panels . The sample design allocated to the block groups was determined to detect a significant difference for the productivity rate between each of the two experimental panels and the control panel of 0.12. The productivity rate is the number of fieldwork cases worked per hour. One of the objectives of this test is to measure how cost parameter estimates compare from the two experimental panels as compared to the control panel. Assumptions were made for mail return rate and the administrative record removal for the two experimental panels. These assumptions led to the identification of 250 block groups. The goal was to have 20,000 NRFU units in each panel. These block groups were randomly allocated to the 3 panels. Since the experimental panels have more cases removed due to administrative records, they had additional block groups allocated. The sample design projects to have about 22,000 NRFU cases in each panel. An additional subsampling operation of blockgroups will happen after the initial mailout to reduce the sample to about 20,000 NRFU cases per panel. Since we are measureing cost implications, we are subsampling out blockgroups so we can measure cost implications of attempting to enumerate entire block groups. The significant difference testing described above took into account this additional subsampling.

Further, to reduce burden on respondents, any housing units selected for the 2015 American Community Survey (ACS) will be excluded from sample selection. No quality control operations will be tested during this census test. However, to minimize any risk of data falsification, field representatives will be carefully supervised and data will be retained for possible future quality analysis.

The Census Bureau estimates an overall self-response rate of 55 percent in the Maricopa site, and self- response rates of 45 percent across 3 panels designed to compare different self- response strategies in the Savannah site. These estimates are based on analyses from prior testing, including the 2012 National Census Test, the 2014 Census Test, the 2010 Census with adjustments based on other recent tests, and literature reviews from other studies of self -response methods.

The sample size in the Savannah site for the panels (30,000 per panel) is designed to compare different self- response strategies in the 2015 Census Test based on a power analysis of expected measurable differences on key measures for response rates and analysis of responses by address type (city-style and non-city-style). With an alpha level of 0.10 and power of 0.80, we expect to be able to detect statistically significant differences between two contact strategy panels of 1.4 percentage points.

The sample size for the Maricopa site is driven by the need to ensure a sufficient workload for NRFU for comparing key metrics of efficiency and quality. For the nonresponse follow-up strategies, power analysis was conducted to detect a difference for the productivity rate. The productivity rate is the estimate of cases completed per hour. We expect to be able to detect statistically significant differences between the panels of 0.1 to 0.12 cases per hour.

2. Procedures for Collecting Information in 2015 Census Test (Self-Response and NonResponse Follow-Up Operations)

Census Day for the 2015 Census Test will be April 1, 2015. The test will employ four response modes: Internet self -response on a secure web site, self -response on a paper questionnaire, Computer Assisted Telephone Interviewing (Web TQA), and Computer Assisted Personal Interviewing (CAPI) in Nonresponse Follow-up (NRFU). Self-response operations apply to both 2015 Census Test sites - Savannah and Maricopa, while NRFU operations will be conducted only in the Maricopa site.

The Census Bureau will accept responses from respondents who do not have a unique Census ID at the time of the response. By accepting responses without a unique ID, the Census Bureau is testing its ability to increase self -response rates and reduce respondent burden. Non-ID cases need to be validated against the Census Bureau’s address frame. The key survey measure for this data collection is the number of Non-ID cases successfully resolved (i.e., matched to an address on the frame or assigned to a census block) through automated matching and geocoding both during self -response (i.e., in real time) and in subsequent batch processing, which will occur for those Non-ID respondent addresses not matched to a valid record in the Master Address File. Additionally, the test provides an additional opportunity to test improvements to the automated Non-ID processing utilized during previous 2020 Non-ID research and testing in 2013 and 2014.

Self-Response Operation for 2015 Census Test Savannah Site

All addresses in the Savannah site for the 2015 Census Test (Savannah, GA and adjacent counties in South Carolina) will be eligible to participate in the test. The respondents in the sample/panel for “Notify Me” are invited to provide their preferred contact information (email or cell phone/text) beginning on February 23, 2015, with the Census Bureau then sending the initial contact (and follow-up reminders) later according to their preferred contact method.

Internet Push Mail Contact Strategy

On or about March 23, 2015, two samples/panels of addresses from the Savannah site will receive direct invitations to respond to the Census test, either via the Internet or by calling the Telephone Questionnaire Assistance number to provide their responses via a telephone interview. The notification in one panel will contain a Census ID that is unique to the address in sample. The initial notification in the second panel, and for any households that pre-registered with an email address or cell phone number, will include the link for Internet data collection, but these respondents will not be provided with an ID and are considered part of the Non-ID workload. One week after the initial contact notification, the Census Bureau will send a reminder message (by mail, postcard, email, or text message). A second reminder will be sent to non-respondents (by mail postcard, email message, or text message) the following week. Approximately three weeks after the initial notification, the Census Bureau will send a final reminder along with a paper questionnaire (either bilingual or English) to non-respondents. Respondents will be instructed to complete the census questionnaire and return it in a business reply envelope provided, or they can still choose to respond online or by telephone. There is no field Nonresponse Follow-up operation as part of the 2015 Census Test Savannah site.

Self -Response: Internet Push Mail Contact Strategy for 2015 Census Test in Maricopa County Site

All addresses in the Maricopa site sample for the 2015 Census Test (parts of Maricopa County) will be sent an Internet Push mail contact (defined below) asking them to respond to the census questionnaire on a secure Internet site or by calling the Telephone Questionnaire Assistance number to provide their responses via a telephone interview. Two reminders will be sent (by mail postcard) in each of the following two weeks (the second reminder will only be sent to non-respondents). Approximately three weeks after the initial notification, the Census Bureau will send a final reminder along with a paper questionnaire (either bilingual or English) to non-respondents. Respondents will be instructed to complete the census questionnaire and return it in a business reply envelope provided, or they can still choose to respond online or by telephone.

Nonresponse Follow-up in the Maricopa County Site

Between May 14 and June 26, 2015, Census enumerators will contact non-responding housing units to conduct the NRFU operation in portions of the Maricopa site. The enumerators will determine the Census Day status of the housing units and collect the test census data for occupied housing units. A difference from procedures in the 2010 Census is that the enumerators will use an automated handheld device for data collection on a smart phone device.

During this NRFU field operation, enumerators will contact non-responding households in person. Enumerators will be trained on how to convince reluctant respondents to participate, such as reassuring them of their data privacy and explaining the benefits of this work. The automated instrument will contain translations into Spanish. Cases with language barriers can be reassigned to be completed by bilingual enumerators with those language skills. Efforts will be made to recruit and assign bilingual field staff for areas known to be bilingual. Enumerators will be provided with “Notice of Visit” forms that will be translated into four languages: Spanish, Vietnamese, Chinese, and Korean. The “Notice of Visit” forms contain the Telephone Questionnaire Assistance number and a link to the internet collection site. If a household member for a sample address is still not willing to provide information, enumerators can utilize proxy respondents to gather the necessary information. Sample units in the control panel believed to be occupied become proxy eligible after the third unsuccessful contact attempt (or sixth when a telephone number is present). Sample units in the two experimental panels believed to be vacant or non-housing units become proxy eligible for 90 minutes after each unsuccessful contact attempt. Enumerators will also be trained on how to verify whether an address is vacant, determine if an address has changed, or whether the address or housing unit no longer exists.

Evaluation Follow-up (EFU) in Maricopa Site

Between July 6 and July 25, Census enumerators specially trained in re-interviewing techniques will conduct an Evaluation Follow-up (EFU) operation on a subsample of NRFU cases (no more than 5,000) in the Maricopa site using a handheld device. NRFU cases from the control panel will be compared to administrative record processing results in order to identify households with housing status and population differences to be included in the EFU operation. While the control panel does not remove administrative record cases from the workload, the processing is still done on the cases in the panel so comparisons can be made.

The following addresses will be eligible for the evaluation follow-up.

• The NRFU interview was conducted with a household member and the population count is different than the administrative record determination.

• The NRFU interview was conducted by a proxy interview and the population count is different than the administrative record determination.

• The NRFU interview determine the unit to be occupied but the administrative record determination was vacant.

• The NRFU interview determined the unit to be vacant but the administrative record determination was occupied.

• Proxy vs “Close” administrative records: The administrative records processing will meet the criteria to be identified as being administrative record occupied. There will be other cases that have administrative records but don’t meet the threshold. Examples of cases that may not be used include having too large of a population count or being certain household compositions. We will identify housing units where we have these “close” administrative records but had a proxy househonld enumeration in NRFU Including these will allow us to be able to examine when proxy versus administrative records should be utilized.

• The last group are households that have had changes in their administrative record composition from the previous year. This group does not make a comparison to a NRFU enumeration status. For these cases, we will do the evaluation followup interview to learn more information about the change from the previous year.

The enumerators will determine the Census Day status of the housing units and collect the evaluation questionnaire data for selected housing units, with a special focus on where household members lived during the year to understand the timing of changes to housing unit status and to the people listed as residents of occupied households during the calendar year. This information will be collected to help us understand information in the administrative records used during the 2015 Census Test in the Maricopa site.

3. Methods to Maximize Response

Self -Response

One of the primary objectives during the 2015 Census Test is to experiment with different methods of maximizing self -response. The unifying theme underlying each of the strategies is the goal of increasing the number of households that respond online. The Maricopa site will have a self- response component but it will not have separate panels, and will not be included in the “Notify Me” section of the test. In the Savannah site, three contact strategy panels will be implemented to encourage response. In addition, there will be an integrated communications campaign to promote the test and encourage all households in the site to participate in the “Notify Me” opportunity and to participate and respond to the Census Test. The communications campaign will include traditional paid media (e.g. television, newspaper, etc.) as well as targeted digital advertising and partnership efforts with local communities and stakeholder groups.

All panels incorporate the “Internet Push” approach, in which sample housing units will not initially receive a paper questionnaire. Sample households either receive (a) a letter that describes the survey and includes instructions about how to respond to the survey online; or (b) an email/text with a description of the survey and a prompt to respond to the survey online with the URL for Internet collection. The households who receive an email or text will be those who provided their contact information as their contact preference via the “Notify Me” panel (see below).

Traditional paper questionnaires will be sent to non-respondents in the three panels about three weeks after the initial contact notification. The paper questionnaire is an opportunity for those without Internet access or a telephone, or with reservations about providing their data online, to respond to the data collection. Households that receive the paper questionnaire will still have the option of responding to the survey online or through the telephone questionnaire assistance line. The three self- response contact strategy panels are:

• “Notify Me” (Savannah site only): Sample households in this panel in the Savannah site will receive a pre-notice postcard inviting them to visit the 2015 Census Test “Notify Me” website to sign up to receive a notification when it is time to respond to the 2015 Census Test. Pre-registrants can opt to be notified by text or email. Notifications will be sent up to three times in the chosen format. If there is no response after the third notification/reminder (from the households in the test panel) a paper questionnaire will be sent. Sample households in the “Notify Me” panel who do not pre-register will be sent the standard Internet Push mail materials (described below). Note that as a result of advertising, promotion, or word of mouth, households in one of the other two panels (or outside of the sample) may pre-register. Any households that pre-register will receive a text or email in lieu of their assigned mail materials.

• Non-ID Panel: This panel in the Savannah site is designed to test the Census Bureau’s automated matching and geo-coding processes for cases without a pre-assigned, unique Census ID. Sample households in this panel will receive standard Internet Push mail materials, however they will not be provided with a Census ID. Instead, respondents provide their address as part of the response process. After the initial contact letter, sample households receive up to two additional reminder postcards by mail. If there is no response, a paper questionnaire will be sent. The magnitude of respondent engagement in Non-ID response will also be studied. All mail materials used in the Maricopa site will contain Census IDs, but respondents will still have the option of responding without that pre-assigned Census ID.

• ID Panel: For this panel, the first contact is a letter along with an instruction card on how to respond online using the household’s Census ID. After the initial contact letter, sample households receive up to two additional reminder postcards by mail. If there is no response a paper questionnaire will be sent. This strategy will be used for self response in the Maricopa site.

The full contact strategy panel design is summarized in the following table:

Table 1. 2015 Census Test – Self-Response Contact Strategy Panels (Savannah Site Only)

Panel/ Site	Sample Size	Pre-Notice	Contact 1 (March 23)	Contact 2 (April 1)	Contact 3 (April 8)	Contact 4 (April 15 or 16)
1. “Notify Me” (Pre-registration) / Savannah	30,000	Postcard invite to register Feb 23	Choice of email or text	Choice of email or text	Choice of email or text	Mail questionnaire
2. Non ID / Savannah	30,000	N/A	Letter (no ID)	Postcard (no ID)	Postcard (no ID)	Mail questionnaire
3. ID- Both Sites	30,000 in Savannah site & 160,000 in Maricopa site to support NRFU test objectives	N/A	Letter	Postcard	Postcard	Mail questionnaire

*Targeted only to non-respondents

We also anticipate receiving Non-ID responses from non-sample units in both sites due to promotional efforts and word-of-mouth. If the respondent-provided address includes an eligible ZIP code from the Savannah site, it will be subject to real-time Non-ID processing. In all other cases, however, it will be matched and geocoding via batch processing.

Communications and Advertising (Savannah Site Only)

Some communication elements will happen across the entire test site including television, radio, and print advertising. There will also be an earned media and partnership campaign. Finally, the Census Bureau will randomly select some households to receive reminder telephone recorded calls, which will be distributed across the panels (site design in Table 2.). We will test whether prerecorded local influencer reminder phone calls are a cost-effective tool to increase response. Calls with local influencers will be compared to calls from a more national figure to test the benefit of local community leaders providing tailored messages to communities. These calls will be made to a random sample of households not receiving self-response treatments, and will be evenly distributed across the five digital treatments. We have cell and/or landline phone numbers available for about 76% of households.

Voice	Community Leader “A”: Potential elected official	Community Leader “B”: Potential civic leader	“National” Voice Professional voice actor	Control No call
Recordings	Two recordings: Message if voice connection Message if voicemail	Two recordings: Message if voice connection Message if voicemail	Two recordings: Message if voice connection Message if voicemail	No calls
Households	20,000	20,000	20,000	~ 145,000
	60,000 total

We will have a five panel design. The first aspect will be spending level (per household). This will help identify the return on investment for various levels of online advertising. The second aspect will be whether the advertising is a general or targeted messaging campaign. Targeted refers to advertising creative that is designed and served to a specific audience. The fifth panel serves as a control that does not receive any digital advertising. This will allow us to distinguish the effects of the other communications channels from the low-spend levels (e.g. partnership activities, reminder calls, television/radio/print advertising, and non-promoted social media).

Table 2. 2015 Census Test – Advertising Panels (Savannah Site Only)

Elements spread evenly across entire site: Broadcast and cable/satellite TV, radio ads, print ads, earned media, reminder calls, social media, and partnerships.
	Targeted	Non-targeted
High spend level (per household)	Panel A General digital advertising + Target-specific advertising	Panel B General digital advertising
Low spend level (per household)	Panel C General digital advertising + Target-specific advertising	Panel D General digital advertising
No spend	Panel E No digital (control condition for the other communication elements)

The five panels will be developed at the ZIP code-level because of the ability to target digital advertising. Each of the 106 non-P.O. box ZIP codes in the media market will be assigned to one of the panels.

When assigning the ZIP codes to specific panels, there are three objectives that we are trying to achieve: 1.A balanced number of households in each panel, 2. Maximize demographic parity, and 3. Maximize geographic cohesion of each panel to minimize the effects of advertising bleed between ZIP codes.

Nonresponse Follow-up (NRFU)

A major objective of the 2015 Census Test in the Maricopa site is to evaluate the difference in productivity rates between the control panel and alternative experimental panel treatments. The 2015 Census Test alternative treatments varies the approach by removing cases from the NRFU workload prior to fieldwork and changes contact strategies for cases that are visited in the field. It also varies the way cases are assigned and managed in the background. We will study the efficiencies and drawbacks associated with these two NRFU approaches in 2015. The NRFU panels and associated mail out sample size, as well as the NRFU field workload, are shown in Table 2 below.

Panel 1 is the control panel. It will employ similar NRFU procedures to those used in the 2010 Census. Enumerators will be instructed to make no more than six contact attempts. The first contact attempt must be made in person with no more than two additional in-person attempts. The main difference from the 2010 Census is that enumerators will be using an automated instrument instead of paper for data collection in the field.

Panel 2 will reduce the initial workload for NRFU to exclude any housing units identified as vacant or occupied based on administrative records. Vacant housing units will be identified by utilizing “Undeliverable-As-Addressed” information from the United States Postal Service combined with unit information from the Master Address File and block group information from American Community Survey estimates. Occupied units will be identified based on administrative records combined with the unit and block group area information just described. For remaining NRFU cases, all units will be visited at least once. The reenginnered field work and adaptive design procedures will be utilized to determine the number and timing of visits when to contact units.

Panel 3 will reduce the initial workload for NRFU to exclude any housing units that were identified as vacant using administrative records. In addition, for all housing units that have not been removed due to vacancy enumerators will make one personal visit. After that initial personal visit, the remaining NRFU workload will be further reduced to remove any housing units that could be enumerated using administrative records. The methods to reduce the NRFU workloads using administrative records are the same as those employed in Panel 2. For remaining NRFU cases at that point, all units will be visited at least once. The reengineered field work and adaptive design procedures will be used to determine the number and timing of visits when contacting units.

For NRFU cases in Panels 2 and 3, if the enumerator cannot reach someone during an in-person contact attempt the enumerator will leave a “Notice of Visit” with information for the respondent to either respond via Internet or Telephone Questionnaire Assistance (TQA) for each unresolved case. For NRFU cases in Panel 1, to simulate the 2010 methods the enumerator will leave a Notice of Visit requesting that the respondent call the enumerator to schedule an appointment for the enumerator to return to collect information from the respondent. “Notice of Visit” forms will be translated into four languages: Spanish, Vietnamese, Chinese, and Korean.

Table 3. 2015 Census Test - Nonresponse Follow-up Contact Strategy Panels (Maricopa Site only)

Panel	Treatment	Operation	Housing Units
1	Control – similar to 2010 Census	Mailout Size NRFU Fieldwork	46,000 23,000
2	Full Removal – Remove both cases identified as vacant and those identified as occupied with administrative records prior to conducting NRFU fieldwork, optimize assignments given to enumerators, apply adaptive stopping rules for fieldwork.	Mailout Size NRFU Fieldwork	66,000 23,500
3	Hybrid Removal – Remove cases identified as vacant with administrative records prior to conducting fieldwork, optimize assignments given to enumerators, remove cases identified as occupied with administrative records after one contact attempt, apply adaptive stopping rules for fieldwork	Mailout Size NRFU Fieldwork	48,00 23,500
Total		NRFU Mailout Size NRFU Fieldwork	160,000 70,000

Evaluation Follow-up (EFU) Operation

The two major objectives of the EFU test in Maricopa County, AZ are to obtain the most accurate status of the housing unit on Census Day and to identify people associated with an occupied housing unit during the calendar year, as well as their timing of association with that housing unit. This will help determine the most accurate household status and roster for Census Day. Cases, numbering no more than 5,000 in this portion of the test mostly include those housing units where information collected in the NRFU portion of the test conflicts with information we might have from administrative records for that housing unit. Other cases being followed up include housing units where information from administrative record sources has had changes in household composition over a two-year period. We are following up with these cases to better understand how changes in persons associated with administrative records in those units compare to the household there on Census Day.

4. Tests of Procedures or Methods

In developing these tests, the Census Bureau consulted with a variety of stakeholders, including, but not limited to, academics, national researchers, community and organizational leaders, and the Census Bureau’s Advisory Committees. In addition, external consultants from the National Academy of Sciences shared information about other relevant studies and provided quarterly feedback about the Census Bureau’s research plans and objectives for the 2015 Census Test. The objectives for the 2015 Census Test are largely building on findings and lessons learned from the 2014 Census Test. The results from these tests also will be shared widely with decennial census stakeholders.

To help evaluate and assess the results of these tests, the Census Bureau will conduct debriefings with enumerators and others who work on the tests, and will conduct focus groups, as described earlier in this supporting statement. Also, cognitive testing participants will be recruited from outside the Census Bureau to provide their views on the wording of questionnaire items, the use of emails and text messages, and the navigation of the Internet application, including the new screens for non-ID processing. The respondents will be asked to provide their reactions to the content of the messages and the mode of delivery, in order to test for comprehension, saliency, and sensitivity of the messages.

The paradata collection screens used in this NRFU operation are not public-facing. They are adapted from the Contact History Instrument employed by current Census Bureau Field Representatives for the American Community Survey and other current surveys conducted by Census. Additional questions are adapted from the CATI instrument used by Census interviewers for telephone contacts from call centers. This adaptation of questions will undergo usability testing before being released for the operation.

5. Contacts for Statistical Aspects and Data Collection

For more information, contact Erin Love (301) 763-2034, [email protected].

6