Cognitive Interviewing Plan -- ICP Aided Awareness Questions

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB Number 0607-0725). We will be conducting cognitive interviews to test the aided awareness questions developed for the Census Integrated Communications Program (ICP) Evaluation (CICPE). These items assist in determining whether or not survey respondents have viewed specific ads within the 2010 ICP. Accurate measurement of exposure to paid media is essential to the success of the ICP evaluation.

We will be testing two types of items related to paid media. The first one cues respondents with specific details of an advertisement, then asks for additional details to confirm that respondents indeed saw the referenced advertisement. These questions have been previously tested without the context of the specific advertisements being planned for the ICP, and this testing will include those advertisements. The second type of item presents respondents with a set of statements about the messages embedded in the referenced ads and asks about their level of agreement with the statements. These two types of items are directly rooted in marketing evaluation and evaluation of public service campaigns.

To set the stage for conducting these cognitive interviews, respondents will be asked to view four to five advertisements on-line. One week later they will participate in an in-person cognitive interview.

In September, approximately fifteen cognitive interviews will be conducted under contract by NORC at the University of Chicago. Participants will be recruited primarily using the website CraigsList.com, as internet access and facility is essential for participation in the testing. We will strive to interview participants from a variety of racial/ethnic groups and with a range of media use. Special efforts will be made to select at least one individual from each of the 6 race/ethnic groups comprising sub-samples in the CICPE survey design: Non-Hispanic Blacks, Hispanics, Asians, American Indian and Alaska Natives, Native Hawaiians and other Pacific Islanders, and all others (including non-Hispanic Whites). In addition, the screener questionnaire will collect respondents’ age and gender, which will be used to help select participants within racial/ethnic groups when necessary. Our experience using CraigsList.com to recruit respondents has yielded a natural distribution of male and female respondents from a variety of age groups and educational backgrounds. Respondents will be from the Chicago metropolitan area and the interviews will be conducted at NORC offices. All participants will be paid $40 for their assistance with the project.

Once an individual has been selected to participate, they will be mailed a letter with their login information and the URL for the website where they can access materials they are required to watch before the interview. A copy of the letter is attached. The materials that participants will view include five advertisements (storyboards) under development by DraftFCB. Participants will be allowed to view each advertisement only once. If a participant breaks off in the middle of viewing the ads, s/he will be allowed to re-enter the website, but will be able to access only those ads that they have not yet viewed.

Once a participant views the selected ads, s/he will be asked to schedule an appointment through the website with a NORC interviewer to answer some follow-up questions approximately a week later. On the last screen of the website participants will be presented with a variety of timeslots when they can come to the NORC office to participate in the interview and asked to select the most convenient time for them. NORC staff will call the day before their scheduled interview to remind them of their appointment. During the cognitive interview, participants will be given a statement about informed consent which states that their participation is voluntary. A copy of which is attached.

Participants will be asked roughly seven questions from the paid media messaging section of the CICPE questionnaire about each of the five ads that they viewed and one ad that they didn’t view. These questions are a subset of the questions developed during prior rounds of cognitive testing for this project. In addition to the survey questions, cognitive probes will be used to gain greater insight into how respondents interpret the questions and formulate answers. A copy of the survey questions and cognitive interview protocol is attached.

Interviews will be taped with participants’ permission, and the tapes will be destroyed within 60 days of completion of cognitive interviewing.

We estimate that the time required to complete this information will average 40 minutes. Thus, the total estimated burden for this research is 10 hours.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Donna Souders

Decennial Management Division

4600 Silver Hill Road

Rm 3H470A

Washington, DC 20233

301-763-1810

[email protected]