Memo

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We will be conducting a cognitive/usability evaluation of the 2016 Census Test online questionnaire. As a part of this current submission we are seeking approval for usability/cognitive interviews conducted by internal Census Bureau staff and RTI, International contractor staff.

Offering the census questionnaire and mailing materials in different languages is the primary goal of the 2016 Census Test. During the usability evaluation, we will test the English text, the Spanish translation, the Chinese translation and the Korean translation of the online instrument. Other modifications to the online instrument to be tested include different coverage questions and different relationship response categories.

Usability testing will identify issues that are problematic and frustrating to the user; testing in non-English languages will also include cognitive testing of the translation. Testing will evaluate the usability of the screen design for different types of devices including desktop/laptop, tablets and smartphones.

Testing will be conducted between February 1 and February 12, 2016. Recruiting will start January 11, 2016. Testing will be conducted for multiple languages in collaboration with RTI International. Languages will include English, Spanish, Chinese and Korean.

• Twelve English, three Chinese, and (5-10) bilingual Spanish sessions will be conducted by U.S. Census Bureau staff. These will take place either at the U.S. Census Bureau headquarters, or at locations more convenient for participants (what we refer to as off-site testing). If off-site testing occurs, they will likely be conducted at local community centers or local libraries.

• The contractor will conduct additional testing in Chinese (10-15 cases) and Korean (10-15 cases). They will also conduct a maximum of 20 Spanish cases. Sessions will take place at off-site locations.

If participants are testing on mobile devices, they will use their own smartphones or tablets to complete the survey. If participants are testing on desktop or laptop devices, those will be provided by Census Bureau or the contractor.

Participants will be from the general public. Different household sizes will be recruited – from single-person households to larger households with six or more people. Some participants will live with people who are not related to them. Participants will not have participated in previous testing of a census online instrument. Participants will have at least one year’s experience with using the Internet for things other than email. For those participants using their own smartphone or tablet, they will know how to use that device to access the Internet. Participants will be recruited using a combination of word-of-mouth, flyers, online and paper advertisements.

Participants will first be asked to complete a demographics questionnaire and mobile experience questionnaire. Then participants will complete the 2016 Census Test online survey using a think-aloud protocol. To complete the 2015 Census Test, most participants will be given a language-appropriate mailing materials with their login information containing a Census ID. A few participants in each language will be given language appropriate mailing materials without an ID. After completing the online form, each participant will be asked to complete a satisfaction questionnaire, and asked debriefing questions about different screens.

Due to the length and complexity of the task as well as the necessity to travel to test locations, and use of their data plan for mobile devices, we plan to compensate respondents $60 for 90 minutes of their time.

Below is a list of materials to be used in the current study. Included is a note on whether each attachment is new or has already been approved by OMB.

1. Protocol used for the study (Enclosure 1)

2. Mailing materials with and without their ID (Enclosure 2) (These are concurrently being approved by OMB for the 2016 Census Test.)

3. PowerPoint slide shows with debriefing probes (Enclosure 3) (Debriefing questions are new. Placeholder screen shots are concurrently being approved by OMB for the 2016 Census Test.)

4. Demographic questionnaire (Enclosure 4) (Previously approved by OMB for usability testing in the spring of 2015 for the 2015 Census Test.)

5. Mobile experience questionnaire (Enclosure 5) (Previously approved by OMB for usability testing with the November 2015 Compass-NRFU submission.)

6. Satisfaction questionnaire (Enclosure 6) (Previously approved by OMB for usability testing in the spring of 2015 for the 2015 Census Test.)

7. RTI screener (Enclosure 7) (Previously approved by OMB for usability testing with the November 2015 Compass-NRFU submission.)

For the 12 English cases, respondents will be screened using the Census Bureau’s generic screener, thus the burden hours for screenings are covered under a separate request. The total estimated respondent burden for Census cases conducted in English for this study is 18 hours.

The Census Bureau expects to screen 9 respondents to arrive at 3 completed cases in Chinese, 30 respondents to arrive at 10 completed cases in Spanish, while RTI expects to screen up to 162 respondents to arrive at 50 completed cases for Chinese, Korean, and Spanish testing. Screening takes approximately 6 minutes per case. Thus the maximum burden for the non-English language cases, 50 RTI cases and the 13 Census cases, is 115 hours total, 95 hours for interviewing and 20 hours for recruiting. Including the 18 hours of burden for English cases, the grand total is 133 hours.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Elizabeth Nichols

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-1724

[email protected]