OMB Memo

Generic Information Collection Request Addendum

Request: The purpose of this letter is to request additional burden hours and note minor changes to the testing protocol for joint cognitive and usability testing of the 2020 decennial census online questionnaire. The original request, “2017 Census Test Usability Testing” was approved 07/20/2016 with Elizabeth Nichols as the primary contact (see Enclosure 10 for approved memo from 2017). This new testing will occur in 11 additional non-English languages, which requires us to work with RTI International and Research Support Services contractor staff. The Census Bureau plans to conduct this additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). Testing will be a continuation of the 2017 Census Test in the 2018 End-to-End Census Test and 2020 Census instrument builds (see Purpose section below for more information on the individual builds).

Purpose: Additional languages will now be available for pretesting and minor lay-out changes have been made to the decennial census online questionnaire that would benefit from usability and cognitive testing. Census Bureau (the Center for Survey Measurement) will test the 2018 build of the instrument in July and August of 2018 in English and Spanish. This testing is being conducted because the 2018 build of the instrument is available in an environment that respondents can access using mobile devices. In October, November, and December of 2018, Census Bureau (the Center for Survey Measurement) as well as contractors (RTI International and Research Support Services) will jointly test the 2020 build of the instrument. This testing is being conducted because the 2020 build includes more languages (Chinese, Vietnamese, Korean, Arabic, Tagalog, Russian, Haitian Creole, Portuguese, French, Polish, and Japanese) than just English and Spanish. Additionally, the 2020 build of the instrument also includes the addition of a question about citizenship. We do not anticipate that the 2020 build of the instrument will be available in an environment in which respondents can access the instrument using mobile devices.

As with the previously approved request, the goal of this research is to improve the design of the census online instrument. Updates to the protocol include:

• Additional probes on redesigned landing page and log-in page

• Additional language-specific probes on understanding of survey terms in target languages

• Additional probes on redesigned mailing materials

• Additional probes on users’ expectations for survey functionality

The same protocol and materials will be used in testing of both the 2018 and 2020 build of the instrument. When testing is conducted on the 2018 build, no probing will be done on the citizenship question as there is no citizenship question in that build. The protocols and supplementary materials will be translated into all languages relevant to the testing (i.e., the 2018 build testing will have materials available in English and Spanish, and the 2020 build will have materials in English/Spanish/ Chinese/Vietnamese/Korean/Arabic/Tagalog/Russian/Haitian/ Creole/Portuguese/French/Polish/Japanese).

Population of Interest: The usability and cognitive evaluation will focus on assessing and improving the user experience for the general population as well as for speakers of Spanish, Chinese, Vietnamese, Korean, Russian, Arabic, Tagalog, Haitian Creole, Portuguese, French, Polish, and Japanese.

Timeline: Testing will be from July 2018 through December 2018. Recruiting will start July 2^nd 2018.

Language: Testing will be conducted in English, Spanish, Chinese, Vietnamese, Korean, Arabic, Tagalog, Russian, Haitian Creole, Portuguese, French, Polish, and Japanese.

Method: Respondents will participate using a variety of devices, include smartphones, tablets, and laptops. Devices may be provided by Census Bureau or the contractor or may be respondents’ own devices.

Sample: We originally requested and received clearance to conduct testing with 54 participants. To date, we have conducted 15 interviews because the development cycle did not allow us to conduct multiple rounds of testing, as we had initially planned. As a result, we have 39 interviews remaining for which we requested burden hours and received cash incentives, but have not used the burden hours or incentives. We would like to modify the scope for those 39 hours to include the new tasks described above in the Purpose section of this memo. Additionally, Census Bureau would like to conduct 16 more interviews, in addition to those 39 remaining from our clearance last year. In total, Census Bureau would conduct 55 interviews split between English and Spanish on the 2018 and 2020 builds of the instrument.

We are also requesting clearance for contractors (RTI International and Research Support Services) to conduct 200 additional interviews in Chinese, Vietnamese, Korean, Arabic, Tagalog, Russian, Haitian Creole, Portuguese, French, Polish, and Japanese in the 2020 build of the instrument. This will allow the contractor to test the instrument with a minimum of 10 participants per language, and more as needed.

In sum, we are requesting to modify the scope of the clearance for 39 interviews from last year, and for clearance to conduct 216 new interviews (16 for Census Bureau and 200 for contractors). Overall, 255 interviews will be conducted.

Different household sizes will be recruited – from single-person households to larger households with six or more people. Some participants will live with people who are not related to them. Participants will not have participated in previous testing of a census online instrument. Participants will have at least one year’s experience with using the Internet for things other than email. For those participants using their own smartphone or tablet, they will know how to use that device to access the Internet. Participants in languages other than English will be native speakers of their non-English language.

Recruitment: Participants will be recruited using a combination of word-of-mouth, flyers, online and paper advertisements. Census Bureau participants will be recruited in the DC metro area. Contractor participants will be located in Pennsylvania, California, Illinois and Michigan. Interviews will take place in the Census Bureau usability laboratory or in private locations that are convenient for respondents such as local libraries and community centers.

Protocol: Participants will be asked to complete a demographics questionnaire and mobile experience questionnaire when applicable. Then participants will complete the questions within the Internet Self-Response instrument using a think-aloud protocol. Participants will either be given mailing materials with their login information containing a Census ID or they will be told to complete the census without a Census ID. After completing the online form, each participant will be asked to complete a satisfaction questionnaire (which was also done in 2017), asked debriefing questions about their expectations for the survey functionality (this is new since 2017), asked to complete up to two vignettes (this is similar to 2017), and asked debriefing probes about the design and content of survey screens and mailing materials (this is similar to 2017).

Use of Incentive: This pretesting project was approved with an incentive of $60, and our addendum seeks to conduct additional cases at this incentive level. This incentive level was requested in 2017 to account for respondents using their own data plans, the need for many of our respondents to make childcare arrangements covering a 90-minute interview plus travel time to and from interview locations, and the need to recruit respondents with minimal English-speaking ability and adequate internet proficiency for many of our interviews during tight testing windows.

Below is a list of materials to be used in the current study. Included is a note on whether each attachment is new or has already been approved by OMB under the original submission.

1. Protocol used for the study (Enclosure 1) including possible debriefing probes and vignettes [This is a revised version of a document already approved by OMB]

2. Mailing materials (Enclosure 2) (New)

3. PowerPoint of the Census Test screens as a placeholder (this includes all the questions asked, like the citizenship question) (Enclosure 3) (Revised version of a document already approved by OMB.)

4. Demographic questionnaire (Enclosure 4) (Previously approved by OMB in original clearance submission)

5. Mobile experience questionnaire (Enclosure 5) (Previously approved by OMB in original clearance submission)

6. Satisfaction questionnaire (Enclosure 6) (Previously approved by OMB in original clearance submission)

7. RTI International and RSS Screener (Enclosure 7) (New)

8. RTI Consent form (Enclosure 8) (New)

9. Login Card (Enclosure 9) (New)

10. Approved memo from 2017 testing (Enclosure 10)

Length of interview: We estimate 90 minutes per respondent for each interview X 216 respondents, which is 324 additional interviewing hours. (This does not include the burden hours for the 39 respondents Census Bureau did not interview last year that we plan to interview this year as a continuation of our OMB approval. Those hours were already requested and approved in 2017.) Census Respondents will be screened using the Census Bureau’s generic screener; thus the burden hours for screenings are covered under a separate request and have not been included in this request. RTI International and RSS expect to screen up to 600 respondents in order to complete 200 interviews. Their screening take approximately 6 minutes per respondent, and thus the screening burden for the contractors’ 200 interviews would be 60 hours.

Overall we are requesting 384 additional burden hours in this addendum (324 additional interviewing hours + 60 additional screening hours).

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Mikelyn Meyers

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-9008

[email protected]

Enclosures

cc:
Elizabeth Nichols (ADRM)” with Enclosures

Erica Olmsted-Hawala (ADRM)” “

Lin Wang (ADRM)” “

Mikelyn Meyers (ADRM)” “

Patricia Goerman (ADRM)” “

Marisa Hotchkiss (DCMD)” “

Gregory White (DCMD)” “

Jennifer Hunter Childs (ADRM)” “

Jasmine Luck (ADRM)” “

3