Various Demographic Pretesting Activities

PROTOCOL GUIDE

Cognitive Testing for the Survey of Income and Program Participation

Event History Calendar Questionnaire

1. INTERVIEW PROCEDURES

Gather all necessary materials for the cognitive interview:

• Protocol Guide

• Paper Instrument

• CONTROL CARD table

• Flashcards

• Envelope containing letter and address verification and update from

• Consent form

• Payment voucher form

• Payment ($40 cash)

• Tape recorder, batteries, and one cassette tape (this will be set for you if the interview is going to be conducted in the laboratory)

2. INTRODUCTION

Greeting: “Hello. My name is __________. I work for the Census Bureau. Thanks for agreeing to participate in our study.”

3. INFORMED CONSENT

PLACE THE CONSENT FORM IN FRONT OF PARTICIPANT: “Before we start, I would like you to read over the document in front of you. This document explains a little bit about this interview and provides information about your rights as a participant. In addition, the back page asks for your permission to have this session audio recorded. Please ask me any questions you have about this document. Once you have finished reading the document, please sign it.”

4. BRIEFLY EXPLAIN THE SURVEY AND PROCEDURES

“Let me begin by telling you a little more about what we’ll be doing today. The United States Census Bureau counts population in the U.S and also conducts various kinds of surveys.

Today, with your help, we will be testing some new questions and evaluating a new letter and form that were developed for a national survey called the Survey of Income and Program Participation. I will first ask you some survey questions and you’ll answer the questions just like you would if you were doing it with an interviewer in a regular survey. We are interested in how you understand these questions and how these questions work for you. I am interested in your answers, but I am also interested in the process you go through in your mind when you answer the questions. So I’d like you to think aloud as you answer the questions, just tell me everything you are thinking about as you go about answering each question. From time to time, I’ll ask you some questions about your answers, or about the questions themselves.

I will also ask you to review a letter and to fill out an address verification and update form as if you have received it in the mail after you have completed a survey. I will ask you to read certain parts out loud to me and will ask you some further questions. For example, we might ask you if there were any words that seemed unclear or confusing or if you found something positive or negative about the letter or the address update form.

Our goal here is to get a better idea how well the new questions and materials work with people such as yourself before we actually use them in the survey. So the purpose of our session today is to collect your thoughts and opinions on these new questions and materials. I just want to remind you that I'm only asking for your thoughts and opinions on these materials. There is no right or wrong answer. We are simply trying to make sure the question wording and the content of these materials are clear and easy for most people to understand. Your feedback will be very useful for helping make sure these questions, letters and forms will make sense to other people.

Your participation in this interview and the review of the materials is very important because it will help the Census Bureau in revising these questions and materials.

Do you have any questions before we begin?”

5. THINK-ALOUD PRACTICE

Let’s begin with a practice question. Remember to try to think aloud as you answer.

Practice Question 1. How many windows are there in the house or apartment where you live? WINDOWS _____________

[IF NEEDED:] Try to visualize the place where you live, and think about how many windows there are in that place. As you count up the windows, tell me what you are seeing and thinking about.

PROBES:

• How did you come up with that answer?

• Tell me more about that. Why did you say [ANSWER]?

• I noticed that you hesitated. Tell me what you were thinking.

6. PERMISSION TO TAPE RECORD

“I will be tape-recording the interview so that I can go back and listen to it later if it is necessary when I need to summarize the interview results. And this way I can focus today on what you’re saying rather than having to concentrate on taking notes. As I mentioned earlier, please be assured that all your responses will be kept confidential and your name, and any personal information you provide will never be used in any of our reports. I will now turn on the tape recorder. START RECORDING. May I have your permission to tape record this session?”

7. COGNITIVE TEST – See instrument

SECTION ONE: Demographics Section

• Use Control Card to collect household and demographic information from the respondent

SECTION TWO: Screener Questions

SECTION THREE: Multiple Partner Fertility Questions

SECTION FOUR: Letter and Address Verification & Update Form

METHOD

Paraphrase and think aloud

Term/concept that are difficult to understand

Retrospective Probes

Debriefing

GUIDE TO OBSERVATION, PROBING AND DEBRIEFING

1. Are the screener questions doing their job?

2. Did respondents have any difficulty answering or were uncomfortable with the multiple partner fertility questions?

3. Any issue with the letter and the form? Is that the most effective way for us to update their information?

Assessing Comprehension:

• Observe/document instances of interpretation/understanding difficulty

• Observe/document how questions and messages in the letter were interpreted--were they correctly and consistently interpreted?

Assessing Respondents' Reactions to the Questions and Materials and Incentives:

• Were any questions sensitive, alarming or unsettling to respondents?

• Would the letter and form encourage or discourage future survey participation?

• What did they think about the debit card?

8. FINAL COMMENTS

• Do you have any final comments or any questions?

CLOSING: “I want to thank you very much for participating today. I will now give you $40 as a token of our appreciation. Please sign this receipt form verifying that you received the money.”