Memo

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We will be conducting a usability test on the non-response follow-up (NRFU) components of the COMPASS (Census Operations Mobile Platform for Adaptive Services and Solutions) mobile application. The COMPASS serves as an enumeration platform including such activities as the questionnaire instrument, case management, location and security services. The portions that will be tested in this usability study include the interview screens administered during non-response follow up procedures (see Attachment F).

The purpose of the test is to identify how potential enumerators and respondents will interact with the design features and content of the COMPASS application. Usability testing will assess users’ ability to perform tasks using the application accurately and in an efficient and satisfying manner. Testing will identify issues that are problematic and frustrating to the user and provide recommendations for improvement. Testing will also identify content that is difficult for users to understand and assess whether someone who is monolingual will be able to respond to the survey items.

The testing will be conducted in December 2015 through January of 2016. Testing will be conducted for multiple languages in collaboration with RTI International. Languages will include English, Spanish, Chinese and Korean. Fourteen (14) English and 6 Spanish sessions will be conducted by the U.S. Census Bureau and take place either at the U.S. Census Bureau headquarters, or at locations more convenient for participants (what we refer to as off-site testing). If off-site testing occurs, they will likely be conducted at local community centers or local libraries. All testing in Chinese (30 cases) and Korean (30 cases) and 20 cases in Spanish will be conducted by RTI International and will take place at off-site locations as well. Table 1 provides a summary of the number of participants taking part in each evaluation including the research agency conducting the test.

Table 1. Summary of Usability tests being conducted by U,S, Census Bureau and RTI International

Research organization	Number of participants	Language being tested
U.S. Census Bureau Human Factors and Usability Research group	14	English
U.S. Census Bureau Language and Cross-Cultural Research Group	6	Spanish
RTI International	30	Chinese
	30	Korean
	20	Spanish

All participants will have at least one year of experience using the Internet on a mobile device (such as with checking e-mail, getting mapping directions, reading the news, or shopping online, etc.) Participants will either act as interviewers/enumerators or survey respondents in the evaluations. We anticipate that the sessions will take 90 minutes for both types of participants. When participants act as enumerators, they will have to complete a demographic background questionnaire, a mobile background questionnaire, a training exercise, 2-4 interview scenarios, a satisfaction questionnaire, and a battery of debriefing questions with targeted probes about the screens they encountered. When participants act as survey respondents, they will have to complete a demographic background questionnaire, a mobile background questionnaire, a survey interview, and a battery of debriefing questions with targeted probes about the questions they were asked. Due to the length and complexity of the task as well as the necessity to travel to test locations, we plan to compensate respondents $60 for 90 minutes of their time.

The following materials will be used in the study and are attached with this letter:

• Attachment A_1 (Census Bureau) & A_2 (RTI International):Consent form (To obtain participant consent for participation and recording of the session)

• Attachment B: Mobile experience questionnaire (To get an understanding of participants’ experience with a mobile device)

• Attachment C: Demographic questionnaire (To obtain participant characteristics):

• Attachment D: Protocol used for the study (To outline how the research study will be conducted)

• Attachment E: Interview scenarios (To assess how the application works for participants when reporting for various household scenarios)

• Attachment F: Screenshots of the COMPASS NRFU screens that will be tested

• Attachment G: Satisfaction questionnaire (Allows for a subjective satisfaction measure to be gathered based on the participants’’ experience with the application).

• Attachment H: Debriefing questions with targeted probes (Allows for a conversational exchange between test administrator and the participant about topics related to the application that was not yet covered or need further clarification)

• Attachment I: Example of a local advertisement that will be used to recruit participants

For the 14 Census cases conducted in English respondents will be screened using the Census Bureau’s generic screener, thus the burden hours for screening are covered under a separate request. The total estimated respondent burden for Census cases conducted in English for this study is 21 hours.

The Census Bureau expects to screen 18 respondents to arrive at 6 completed cases in Spanish, while RTI expects to screen up to 260 respondents to arrive at 80 completed cases for Chinese, Korean, and Spanish testing. Screening takes approximately 6 minutes per case. Thus the maximum burden for the 80 RTI cases and the 6 Census cases is 156 hours total, 129 hours for interviewing and 27 hours for recruiting .

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Contact:

Temika Holland

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-5241

[email protected]