2018EndtoEndCensusTest_Response to Public Comments from 60day FRN

Responses to comments received on the Presubmission Notice for the 2018 End-to-End Census Test Peak Operations

(Federal Register, Volume 82, Number 59, March 29, 2017, pp 15486 - 15492)

Comment received from Bahati Louis:

I would urge the Census Bureau to add an LGBTQ option to the National Census so our government can make better efforts to protect another marginalized group of people.

Census Bureau response:

On March 28, 2017, the Census Bureau delivered the planned subjects for the 2020 Census and the American Community Survey (ACS) to Congress. The proposal to Congress was that the planned subjects remain unchanged from the 2010 Census and will cover age, gender, race/ethnicity, relationship, and homeownership status. It did not include sexual orientation or gender identity.

Determining these subjects is a rigorous, iterative process completed in conjunction with the Office of Management and Budget (OMB) and coordinated across many federal agencies. In order for a subject to be included, there must be a clear statutory or regulatory need for data collection. The Census Bureau did receive several requests from federal agencies in 2016 to add questions about sexual orientation and gender identity to the ACS. However, at the end of the process, there was no demonstrated federal data need for the addition of this content, and subsequently no changes to the planned census and ACS subjects.

The Census Bureau is committed to reflecting the information needs of our changing society and is constantly examining the effectiveness of census and survey questions to collect accurate data on families and people. The Census Bureau does have experience collecting information on sexual orientation and gender identity in demographic surveys. The Census Bureau collaborated with the Bureau of Justice Statistics (BJS) to cognitively test and implement sexual orientation and gender identity questions on the National Crime Victimization Survey in July 2016, which the Census Bureau administers for BJS. The Census Bureau also administers the National Health Interview Survey on behalf of the National Center for Health Statistics, which introduced questions on sexual orientation in 2013. The new questions improved data on the general health behaviors, health status and healthcare utilization of LGBT individuals. The Census Bureau also participates on the OMB Interagency Working Group on Measuring Sexual Orientation and Gender Identity and remains actively engaged in the discussion of how to develop and implement quality measures of both sexual orientation and gender identity.

Comments received from Jeff Hardcastle, Nevada state demographer:

Comments were primarily on the topic of ensuring a complete count, such as through the inclusion of nontraditional types of living quarters, particularly those that have historically been identified through extensive field operations. The full document received is included with this package.

One example of these living quarters is units that are rented on a daily, weekly, or monthly basis. Mr. Hardcastle is concerned that these might not be picked up in the United States Postal Service’s Delivery Sequence File of mail delivery points or in the Local Update of Census Addresses file updates received from participating governmental units. With reductions in on-the-ground field operations, the concern is that these also might not be picked up by Address Canvassing or Nonresponse Followup as they likely would have been in prior censuses.

A second example was the identification of Group Quarters (GQs) and Transitory Locations (TLs) for inclusion on the address list. The concern is that with less fieldwork some of these units would be more likely to be missed.

The third comment was about potential differential coverage within Administrative Records and the impacts this could have on census results if they are used in conjunction with Nonresponse Followup.

Census Bureau response:

The Census Bureau is actively updating the address lists for 100 percent of the blocks in the nation. This decade much of this work will occur in the office using partner data and imagery to make updates to the frame, which has been proven in testing to provide strong address coverage. There is significant effort to partner with parties that can inform us of special situations on the ground, including short-term rentals, GQs, and TLs. In 2019 the Census Bureau will conduct in-field address canvassing in blocks that cannot be resolved using in-office address canvassing processes, further identifying less-typical living quarters. In addition, new GQs can be discovered and included in the census during both GQ advance contact and GQ enumeration. As for the development of the address frame of TLs, the Census Bureau plans to conduct research to identify hotels/motels that have long-term stay units and possibly to add them to this frame. The Federal-State Cooperative for Population Estimates will also help with building this frame and making sure that the Census Bureau has collected data at all possible GQs and TLs during their respective data collection phases.

The Census Bureau’s use of administrative records focuses on usage where and when we have high confidence in the reliability of the administrative records data available, placing great emphasis on trusted federal sources and validating the information on households by assessing the consistency of that information among several different data sources. Only after providing opportunities for households to respond via Internet, telephone, or mail will we use administrative records to identify vacant units so enumerators do not have to visit these addresses. For the remaining cases that did not self-respond through any of the available modes, enumerators will make one visit to collect the information in person. If the visit to collect the information in person is not successful, only where we have high quality administrative records from trusted source will we use the administrative records as the response data for the household. Where high-quality administrative records are not available from trusted sources, we will continue in-person visits to reach nonresponding housing units until the case is resolved, which could include a successful enumeration, determination that the address is vacant, or we have exhausted a maximum number of attempts to reach someone at the address. In addition, for cases where we use administrative records to identify vacants or as the response data for occupied households, we will mail an additional piece of correspondence to encourage self-response.

Comments received from the National Association of Latino Elected and Appointed Officials (NALEO) Education Fund

These comments were classified by NALEO Education Fund into seven categories. The Census Bureau provides responses to the five of these that are on the topic of research or methods. The five included statements will be quoted or summarized below. The full document received is included with this package.

I. Internet/Technology Response Option

NALEO comment summary:

The concern noted by NALEO was about internet or mobile phone use and the differential impact that may occur due to a lower rate of usage among Latinos, as well as challenges in some areas with having access to the internet.

Census Bureau response:

The Census Bureau is committed to optimizing self-response across all demographic groups, particularly for traditionally hard-to-count populations. While our goal is to maximize response rates and reduce the reliance on paper, the Census Bureau is aware that there are disparities in internet-usage and connectivity across certain populations and is taking steps to ensure that all households have the opportunity to respond to the 2020 Census so that everyone is counted. One strategy is to include the “internet choice” mail design, in which a paper questionnaire will be provided in the first mailing, using our research to optimally identify the areas that are estimated to have lower online response propensity. Other efforts in the 2018 End-to-End Census Test that make it easier for all households to self-respond include:

• An internet instrument optimized for mobile devices such as smartphones.

• Internet response available with and without a pre-assigned Census ID number.

• Census Questionnaire Assistance, which allows respondents to call over the phone to ask questions or to provide their census information by phone.

• Support in both English and Spanish.

• Delivery of a paper questionnaire – in the initial mailing to all households in areas designated for Internet Choice and in later mailings to nonrespondents in areas designated for Internet First.

Note that in-person Nonresponse Followup (NRFU) is designed to collect census data from any households that do not self-respond. 

Census Bureau assessment reports for the 2018 End-to-End Census Test will include a detailed analysis of self-response rates overall and by response mode, for both the “internet choice” and “internet first” strategies. This will include a study of the demographic characteristics of the respondents in each mode (internet, telephone, paper, and also NRFU), including site, age, sex, race and ethnicity, and household size. The assessment will also report the language used to access the online instrument, and device type such as desktop or laptop computer, smartphone, or a tablet, for these demographics. The study plans are included as part of this clearance package. 

NALEO comment summary:

One particular comment made by the NALEO Educational Fund was one this organization has made about previous tests, as well. This is about the extent to which other household members help with completing the questionnaires on the internet in Latino households.

Census Bureau response:

The only way the Census Bureau is able to study this particular question is through focus groups. In the Census Bureau’s OMB materials for the 2017 Census Test we provided extensive results on our tests on this question, which we provide again here for the sake of complete documentation.

From the 2016 Census Test focus groups the Census Bureau conducted in LA, California and Harris County, Texas, the Census Bureau found that most participants were comfortable in filling out the survey online. Several respondents in the focus groups mentioned completing the Census test by phone (internet). A few Latino responders preferred completing the Census Test by phone; that way they could ask questions throughout the phone call. Nonresponse Followup responders did not respond because they forgot to complete the Census or did not see the correspondence. However, in all focus groups each head of the family mentioned completing the 2016 Census Test on their own (or with their spouse but it was not because they needed help). It is important to note that for the Latino focus groups, the groups were not divided by age (Morales).

For the 2015 Census Test focus groups, which were divided by age and ethnicity, the older Latino focus group mentioned not having access to emails or were not well versed in the new technology such as mobile devices. Many mentioned having an old phone that could not access the internet well, whereas others did not understand how to use a phone well or did not want to try to use new technology. Having said all of this, this group were all Nonresponse Followup respondents, therefore no one in their families helped them, but they were helped by a Census enumerator. (Morales, et al)

Separately, the 2011 Census Barriers, Attitudes, and Motivators Survey II provided some results into how the Latino population perceives the decennial census. < "Census Knowledge and Census Participation Among Hispanics" Blog post> (Conrey, Frederica R., Randal ZuWallack and Robynne Locke.)

In particular, this study found the following results about who completes the census form.

ce3. Did you personally fill out the form or did someone else in your household fill out the form?
% who answered…	General Population	Hispanics
	%	%
Filled out the form	73	62
Someone else filled out the form	22	31
We worked on it together	4	6
Don't know	1	1
Refused/No answer	0	N/A
	100	100
N	2974	444
Source: Census Barriers and Motivators Survey II (2011)

II. Evaluation of Combined Question on Hispanic origin and Race

NALEO comment summary:

NALEO urged the Census Bureau to continue to assess: (1) the extent to which the combined question format improves the level of overall race/ethnicity reporting, as well as detailed reporting for specific Latino national origin and subgroups; (2) the impact of the combined question format on the reporting of Latinos who identify with multiple major race/ethnic categories or detailed categories. NALEO expressed support for the Census Bureau’s plans to continue evaluating comparability between Census Bureau test data and OMB’s federal data standards on this question and for plans to produce prototype data products from this test. NALEO specifically recommended: (1) that the Census Bureau determine the level of detail about race and ethnicity that will be provided in products and to produce as much detail as possible; (2) that the Census Bureau develop plans to ensure comparability to historical statistics, in particular in response to those who may self-identify only as Latino, without identifying a race.

Census Bureau response:

Census Bureau researchers plan to examine the results of the 2018 End-to-End Census Test to ensure that combined race/ethnicity question format continues to perform well. The Census Bureau is considering explorations like those suggested by NALEO as we continue our work and preparation for the 2020 Census. This includes examining major race/ethnicity group responses, and the reporting of multiple responses across major categories. Additionally, as NALEO knows, enabling the reporting of detailed identities for all race/ethnicity groups has been a major objective of the Census Bureau’s research for improving data on race/ethnicity over the past decade. We expect this will continue to be true with the 2018 End-to-End Census Test as well, as respondents recognize their opportunity to report multiple detailed groups within and across categories. The results from the 2015 National Content Test and the 2010 Census Alternative Questionnaire Experiment research demonstrated that the combined question approach for race/ethnicity effectively enables respondents to more accurately and more easily self-identify their racial/ethnic identities, and we expect the 2018 End-to-End Test will continue to show the effectiveness of this conceptual framework.

As we at the Census Bureau continue our preparations for the 2020 Census, we are exploring different tabulation ideas to both enable comparisons to previous data but also to provide data that meet current needs. The Census Bureau currently bridges race data based on 1997 OMB Standards with Some Other Race and Vital Statistics data without Some Other Race. As part of our ongoing work with the OMB and the Federal Interagency Working Group, the Census Bureau and other federal agencies are in dialogues about the NCT results, other data inputs, and feedback from the public through the Federal Register Notice process to discuss and develop solutions to recommend to OMB for improving race/ethnicity data. If OMB decides to allow the use of the combined question, the Census Bureau will perform the research, evaluation, and processing in order to provide a high quality conversion system between the two classification schemes. Initial plans include developing bridging factors from Census 2020 data, linking records with missing race detail to previous Census or ACS data, developing socioeconomic modeling techniques from ACS data, and using the Demographic Characteristics File method currently used to assign race and Hispanic origin to all new entrants into the domestic migration universe who could not be found in previous data (e.g., children or new immigrants since 2010). Combinations of these techniques would not only allow conversion (and therefore compatibility amongst agency reporting, historical series, and research data), but it would also potentially allow us to update the conversion factors over time as race reporting continues to evolve.

All of this work is crucial for the Census Bureau to ensure that critical operations are fully prepared to go into production for the 2020 Census using the combined race/ethnicity question, if it is determined that the combined question format may be used for the 2020 Census. The Census Bureau will continue to engage with NALEO and other organizations as these discussions continue, and we acutely recognize and value the importance of these dialogues. The Census Bureau appreciates NALEO’s encouragement for this research as it supports our ongoing analyses and goals to understand and describe the increasingly complex racial/ethnic identification and growing diversity of our nation’s population.

III. NRFU and Use of Administrative Records

NALEO comment:

The 2018 Test will examine several technological improvements and operational procedures for conducting NRFU. In light of the 2018 Test’s emphasis on the Internet response option, traditionally harder-to-count populations may be even more likely to require NRFU than they would in a paper-based enumeration. In some cases, these residents live in areas that do not have Internet access, such as rural areas or those with non-traditional housing. Some also cannot afford Internet access or lack the skills or information needed to navigate the online questionnaire. Thus, in general, we urge the Bureau to examine the overall effectiveness of the 2018 Test’s NRFU technological improvements and operational approaches in reaching and improving the quality of data collected on harder-to-count populations.

Census Bureau Response:

In development of the technological improvements that will support data collection during the Nonresponse Followup, the Census Bureau is mindful of the need to collect and safeguard data in situations where there is no or limited availability of cellular coverage. To that end, our approach and solution accounts for the need to collect data in a disconnected state, meaning, in an area with no or limited cellular coverage, an enumerator can collect response data in the same way as when in an area with cellular coverage. This enables consistent collection and treatment of all data collected. Data collected when in a disconnected state are stored securely on the data collection device until the enumerator has cellular coverage, at which point the data will be securely transmitted to the Census Bureau.

NALEO comment:

In addition, we note that the Bureau will continue to evaluate the efficacy of automated data collection during NRFU, and make field data collection instruments and materials available in both English and Spanish. The Bureau should work with enumerators and other members of its workforce to evaluate the usability of automated NRFU devices and the related software applications by diverse employees. The Bureau should also examine the effectiveness of Spanish-language instruments and materials for enhancing NRFU within Latino communities.

Census Bureau Response:

The Census Bureau with each of its field tests solicits input and feedback from its enumerators on their experiences with the data collection device, software, training, and procedures. We will collect and capture their experiences and use the information to inform enhancements as we finalize our approaches for the 2020 Census. We will continue this practice with the 2018 End-to-End Census Test.

NALEO comment summary:

The Census Bureau should use the opportunity of the 2018 End-to-End Census Test NRFU operation to examine whether Latino respondents understand the purpose of the combined Hispanic origin and race question, the question’s instructions, and whether respondents understand that they can choose multiple major and detailed race/ethnicity categories. Similarly, the Census Bureau should examine the extent to which young Latinos provide assistance to or serve as proxies for the heads of households during NRFU.

In light of the fact that many Latinos are extremely mobile or live in non-traditional housing, the Census Bureau should examine the impact of NRFU methods for enumerating at multiunit structures and identifying vacant units with minimal contact attempts on reaching Latino households and obtaining an accurate count of their members.

As the Census Bureau evaluates its refinements to its administrative records strategy, the agency should be mindful of any use of the records that would create disparate results for traditionally hard-to-count communities or diminish the quality or accuracy of data on the Latino community. In addition, the Census Bureau should continue to communicate with Latino stakeholders about the proposed use of administrative records and obtain their feedback as the agency finalizes its plans. As the Bureau trains its field enumerators, it should educate them about the fact that part of their role is to ameliorate some of the potential limitations in the use of administrative data. The Bureau should empower enumerators to alert the Census Bureau of instances where information based on administrative records does not match their observations, which could help alert the Bureau to potential systemic issues with the generation or use of such information.

Census Bureau Response:

The Census Bureau’s use of administrative records focuses on usage where and when we have high confidence in the reliability of the administrative records data available, placing great emphasis on trusted federal sources and validating the information on households by assessing the consistency of that information among several different data sources. Only after providing opportunities for households to respond via internet, telephone, or mail, will we use administrative records to identify vacant units so enumerators do not have to visit these addresses. For the remaining cases, enumerators will make one visit to collect the information in person. If the visit to collect the information in person is not successful, only where we have high quality administrative records from trusted source will we use the administrative records as the response data for the household. Where high-quality administrative records are not available from trusted sources, we will continue in-person visits to reach nonresponding housing units until the case is resolved which could include a successful enumeration, determination that the address is vacant, or we have exhausted a maximum number of attempts to reach someone at the address. In addition, for cases where we use administrative records to identify vacants or as the response data for occupied households, we will mail an additional piece of correspondence to encourage self-response.

At no time will the field enumerators have access to or knowledge regarding administrative records usage for a nonresponding case.

IV. Group Quarters

NALEO comment summary:

The Census Bureau should employ a diverse workforce for field operations, especially for Group Quarters operations, which may require special linguistic skills and cultural competencies. In addition, response mode can affect of the accuracy of data in a differential fashion.

Census Bureau response:

The Census Bureau has set recruiting goals at the census tract level to help ensure that employees are hired at the local level to work in the neighborhoods in which they live. In addition, the Census Bureau’s Planning Database is used to determine where significant non-English speaking populations exist, and goals for recruiting applicants with non-English language skills are set. Local recruiting staff will contact organizations in these census tracts and ask them to assist the Census Bureau in getting the word out about jobs and the need for applicants who are bilingual.

The Census Bureau will also have a strong partnership program for the 2020 Census called the Community Partnership and Engagement Program (CPEP), which will execute a robust local partnership outreach campaign. This program will focus on grassroots organizations and ensure our messages reach all segments of the population for the 2020 Census. Similar to previous censuses, CPEP will work very closely with the Latino communities.

VI. The Need for Adequate Resources for the 2018 Test and Census 2020 Preparations

NALEO Comment:

In light of the technological and operational innovations the Bureau intends to implement in Census 2020, robust and effective communications and partnership efforts are crucial to educate Latinos about how to participate in Census 2020 and ensure that the Latino community understands the different options available for participation. NALEO Educational Fund has consistently urged the Census Bureau to maintain and strengthen the Partnership Program to achieve this important goal. The Bureau’s Partnership Program was an integral component of 2010 Census outreach efforts, and helped engage harder-to-count populations in the enumeration. The program helps keep national, state, and local stakeholders fully informed about prospective design changes for the 2020 Census, as well as ongoing efforts to preserve a robust American Community Survey (ACS) and other important demographic and socio-economic surveys. In addition, maintaining and strengthening the network of stakeholders in the program will help ensure their full engagement in the outreach efforts for the 2020 enumeration and other surveys.

Effective Integrated Communications and Partnerships efforts are also essential to ameliorating the undercount of very young Latino children. The Bureau must incorporate messaging in its communications campaign to ensure that individuals completing the census forms include all household members, including very young children, in their responses. It must also must engage partners who work closely with Latino families, such as institutions which reach new or expectant Latino mothers, community organizations, and public assistance programs.

Moreover, in the Federal Register notice, the Bureau acknowledges that with respect to its assessment of the combined question, it needs to understand “whether or not respondent privacy concerns and expectations for data protection are addressed and the process is trusted by the general public.” The absence of a communications effort for the 2018 Test represents a lost opportunity for the Bureau to evaluate the role of such an effort in building public trust in the privacy and confidentiality of information provided to the Census Bureau. This issue is particularly salient in light of the current technology environment in the nation. Cybersecurity risks and breaches have generated significant public attention, and many residents are likely to have serious concerns about the confidentiality of information submitted to the Bureau through the Internet.

Census Bureau Response:

The 2020 Census Integrated Partnership and Communications Program must reach every household in the nation, delivering the right messages to the right audiences at the right time. The Census Bureau appreciates the concerns NALEO Educational Fund expresses about ensuring we engage the Latino community in ways that help them understand the different options available for participating in the 2020 Census and making sure individuals completing the census forms include all household members. Comments like yours help us to better understand our target audiences, as well as issues and concerns we should keep in mind as we develop our program.

While it is true the absence of a communications effort in the 2018 End-to-End Census Test does remove the opportunity for the Census Bureau to evaluate the role of such an effort in building public trust in our privacy and confidentiality, we believe our testing to date has provided valuable insights. In particular, the 2015 Census Test looked at messaging regarding new response capabilities, digital advertising, and other communications techniques. The communications effort during this test was supported by partnership activities to provide for an integrated approach to our testing. Partnership efforts were also included during the 2016 Census Test. As we get closer to the 2020 Census, the Census Bureau will keep building on the lessons learned from the 2010 Census, the 2015 Census Test, and the 2016 Census Test.

More importantly, the Census Bureau is working with the 2020 Integrated Communications Campaign contractor, Team Y&R, to develop and implement a communications research plan that will be a strong foundation for executing partnership and communication activities. The initial effort of this research, the Census Barriers, Attitudes, and Motivators Survey (CBAMS), is a quantitative research project that is focused on providing insights into what messages will resonate with respondents, particularly hard-to-count audiences, such as Latinos. Privacy and confidentiality are crucial topics that will be explored. The Census Bureau will be focusing efforts in late 2017 and 2018 on developing the best possible research, including creating segmentation models and building rapid response systems, to start our outreach efforts to hard-to-count audiences in 2019.

In deciding how best to reach the Latino community, this research will look at characteristics such as general makeup, language, attitudes/beliefs, income/education, and media habits of the Latino community. Evaluation factors include the likelihood of responding and what should be the paid media approach to this group. These insights and the results of research projects like CBAMS will play a role in determining how best to engage the Latino community.

The Census Bureau will also have the Community Partnership and Engagement Program (CPEP) that will execute a robust local partnership outreach campaign. This program will focus on grassroots organizations and ensure 2020 Census messaging reaches all segments of the population. Similar to previous censuses, CPEP will work closely with the Latino community, including trusted voices such as the Latin American Chamber of Commerce, the Latino Coalition, and the League of United Latin American Citizens.

The Census Bureau and Team Y&R will continue to build and expand partnerships with national, local, and multicultural leaders who have strong ties to traditionally underrepresented groups and hard-to-count audiences. The Census Bureau recognizes the need for outreach with various affinity groups and service providers that are trusted voices and influencers within these communities to help educate individuals on the importance of the census, in addition to communicating and sharing tailored messages to this audience. We also recognize the need for customizable messaging materials that speak to various communities about the undercount of young children and the risks and ramifications of not being counted. Both the Census Bureau and Team Y&R have started working with several organizations to better understand this issue and develop targeted messages on the matter. Through these messages, the Census Bureau will ensure the privacy and confidentiality of all respondents.

Appendix

Conrey, Frederica R., Randal ZuWallack and Robynne Locke. 2011. “Census Barriers, Attitudes, and Motivators Survey II. Final Report.” Statistical Research Division. U.S. Census Bureau. Washington, D.C. 20233. Available at: CBAMS_II_Final_Report

Morales, Holzberg and Eggleston (2017). 2015 Census Test Focus Groups

Morales (Forthcoming). 2016 Census Test Focus Groups