OMB Memo Revised

Generic Information Collection Request

Cognitive Testing of National Crime Victimization Survey Mail Materials

Request: The U.S. Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct cognitive interviews to pretest revised mail materials for the National Crime Victimization Survey (NCVS). We are seeking approval for this project.

Background: Beginning in 2018, the NCVS will utilize updated mail materials for outreach with sampled addresses (advance letters, envelopes and two versions of a fact sheet) and respondents (thank-you letters). The Census Bureau’s Demographic Statistical Methods Division (DSMD) Survey Methodology (SM) team was tasked with enhancing the mail materials in preparation for the 2018 implementation. The SM team updated the letters and fact sheet based on an expert review of the materials that are currently used. The next step is to conduct cognitive interviews to evaluate how well the revised materials work and to identify any ways they may be improved.

Purpose: The purpose of this research is to interview adults from the general U.S. population to assess their understanding of the messages and information presented in the proposed NCVS mail materials. In the cognitive interviews, we will assess the clarity and potential effectiveness of the revised materials and identify respondent issues with comprehension or interpretation, with the goal of producing clear, succinct and motivating messages. The results of the cognitive interviews will influence decisions about final content and design of the mail materials for the 2018 NCVS.

Population of Interest: The planned research will focus on assessing and improving the NCVS mail materials for the general population.

Language: The interviews will be conducted in English only.

Timeline: Cognitive interviews will be conducted from June 1, 2017 to July 7, 2017.

Project scope: The cognitive testing of the proposed NCVS mail will be carried out according to the schedule described in Table 1.

Table 1. Estimated Project Schedule

Recruiting	Monday 5/29 - Monday 6/19
Conduct Cognitive Interviews	Thursday, 6/1 - Friday 7/7
Data Analysis – Preliminary Results	Monday 7/10 - Monday 7/17
Final Recommended Materials	Monday 7/17 - Tuesday 8/1
Report	Tuesday 8/1 - Wednesday 9/6

General Protocol: Participants will be asked to review the proposed materials for each mailing (advance letters, thank-you letters, envelopes and fact sheets). Participants will be asked to think aloud. We will use retrospective probing to get a full understanding of the participant’s thought processes and interpretations of specific terminology and overall messaging of the materials as well as preferences for how information is presented. After completing the task, each participant will be asked debriefing questions. All interviews will be audio-recorded to facilitate a summary of the results. Participants will be asked to sign consent forms and for permission to be recorded

Sample: A total of 15 participants will be interviewed in the Washington, DC and Raleigh-Durham, NC metropolitan areas. Participants will be screened during the study recruitment to ensure they are likely to receive/open the mail for their household so they are representative of the NCVS target population; those who would receive mail materials that may be sent to their address.

Recruitment: We will use a multi-pronged strategy to recruit participants, such as posting ads on Craigslist, distributing flyers, and using personal networks. While not nationally representative, an attempt will be made to recruit participants with varying demographic characteristics (e.g., sex, education, age, race and ethnicity).

Use of Incentive: Due to the length and complexity of the task as well as the necessity to travel to test locations, we plan to offer an incentive of $40 to offset the costs of participation, such as travel and parking.

Below is a list of materials to be used in this research.

1. NCVS 593_Cognitive Interviewing Version (Thank-you Letter)

2. NCVS 573_Cognitive Interviewing Version (Advance Letter)

3. NCVS 572_Cognitive Interviewing Version (Advance Letter)

4. NCVS-110_2014 Factsheet_English_12_22_15 (Fact Sheet)

5. NCVS_Infographic_v5_green (Fact Sheet Alternative)

6. NCVS BC335A envelope (Mailing Envelope)

Length of interview: Based on past experience with similar pretesting efforts it requires four screener conversations to recruit one participant. Each screener conversation lasts approximately three minutes. We estimate it will take 3 hours to screen and recruit 15 participants. We estimate it will take 60 minutes per participant to complete the cognitive interview. For 15 participants, the estimated burden for the interviews is therefore 15 hours, bringing the total burden to no more than 18 hours.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

M. Christopher Stringer

Demographic Statistical Methods Division

U.S. Census Bureau

Washington, D.C. 20233

[email protected]

2