Memo

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We will be conducting a cognitive/usability evaluation of the 2017 Census Test online questionnaire. As a part of this current submission we are seeking approval for usability/cognitive interviews conducted by Census Bureau staff.

Two of the 2017 Census Test objectives are testing in American Indian reservations and in Puerto Rico. However, the planned usability evaluation will focus on assessing and improving the user experience for the general population. The 2017 instrument uses the 2016 Census Test instrument as a baseline. The usability goal for 2017 is to improve upon the 2016 Census Test instrument. There was not enough time before the 2016 Census Test production to program all the changes recommended based on the usability testing of last round. This is especially true for the mobile-optimized version.

Testing will be conducted iteratively from June 2016 through February 2017. Recruiting will start early June 2016. We plan on testing approximately six people each month for a total of 54 participants. Testing will be conducted primarily in English, but we leave open the opportunity to also test in Spanish during those months. The 2017 instrument will not offer languages other than English and Spanish. Testing will evaluate the usability of the screen design for different types of devices including desktop/laptop, tablets and smartphones.

If participants are testing on mobile devices, they will use their own smartphones or tablets to complete the survey. If participants are testing on desktop or laptop devices, those will be provided by Census Bureau or the contractor.

Participants will be from the general public. Different household sizes will be recruited – from single-person households to larger households with six or more people. Some participants will live with people who are not related to them. Participants will not have participated in previous testing of a census online instrument. Participants will have at least one year’s experience with using the Internet for things other than email. For those participants using their own smartphone or tablet, they will know how to use that device to access the Internet. Participants will be recruited using a combination of word-of-mouth, flyers, online and paper advertisements.

Participants will first be asked to complete a demographics questionnaire and mobile experience questionnaire. Then participants will complete the most up-to-date 2017 Census Test online survey using a think-aloud protocol. Participants will either be given mailing materials with their login information containing a Census ID or they will be told to complete the census without a Census ID. After completing the online form, each participant will be asked to complete a satisfaction questionnaire, and asked debriefing questions about different screens.

Due to the length and complexity of the task as well as the necessity to travel to test locations, and use of their data plan for mobile devices, we plan to compensate respondents $60 for 90 minutes of their time.

Below is a list of materials to be used in the current study. Included is a note on whether each attachment is new or has already been approved by OMB.

1. Protocol used for the study (Enclosure 1) including possible debriefing probes and vignettes

2. Placeholder mailing materials (Enclosure 2) (This is an example from the 2016 Census Test, mocked up for the usability tests.)

3. PowerPoint of the current 2016 Census Test screens as a placeholder (Enclosure 3) (Approved by OMB for the 2016 Census Test.)

4. Demographic questionnaire (Enclosure 4) (Previously approved by OMB for usability testing in the spring of 2015 for the 2015 Census Test.)

5. Mobile experience questionnaire (Enclosure 5) (Previously approved by OMB for usability testing with the November 2015 Compass-NRFU submission.)

6. Satisfaction questionnaire (Enclosure 6) (Previously approved by OMB for usability testing in the spring of 2015 for the 2015 Census Test.)

Respondents will be screened using the Census Bureau’s generic screener, thus the burden hours for screenings are covered under a separate request. The total estimated respondent burden for this study is 81 hours.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Elizabeth Nichols

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-1724

[email protected]