Cognitive Interviews for the 2010 National Household Education Survey (NHES) Study Draft Questionnaires

PFI Protocol

General Introduction

I would like to begin by thanking you for coming here today to help us out. We’re scheduled to be here for 90 minutes, but I’ll let you go early if we finish sooner. The reason we’ve asked you to come is that the US Department of Education conducts a survey roughly every two years to describe children’s educational experiences. Today I’m going to ask you to fill out a questionnaire that is being developed for 2011 and to provide feedback so that we can help the Department of Education make sure the questionnaire will be understood by all different types of parents.

Because it would be hard to keep track of everything you say today, we're going to tape-record this session. And I’ll also be taking notes, so you might notice me writing as we are talking. There’s a one-way mirror behind me, and there are a few of my coworkers behind it watching. I want you to know that your responses are voluntary and confidential.

In just a minute I'm going to hand you the questionnaire and I'd like you to fill it out the same way you would if it came to you at home in the mail. Unlike being at home, though, I’ll ask you to stop periodically so that we can talk about what you were thinking as you answered the questions.

There will also be a few questions where I will ask you to think “out loud” as you answer them. For these few questions, it is important that you tell me what you are thinking and feeling while you are actually in the process of coming up with your answers.

Now because some people aren't used to reading, thinking and expressing their feelings aloud, I'd like to begin today with a short practice question. I’d like you to practice with this by reading the question I’m about to give you out loud and telling me your thoughts and feelings from the moment I hand you the question until you are finished explaining your answer.

QUESTION: How many times did you eat a meal out at a restaurant in the past week?

Thank you. I’m going to hand you the questionnaire now. Please read it and fill it out as if you were actually responding to the survey. As you go through the survey, if any question sticks out to you in any way (maybe you think a question is hard to answer, hard to understand, in a strange place, or you just don’t like it)—anything at all that you would like to say about an item, please tell me so that we can discuss it.

For the purposes of this survey, the sampled child is ____________________________

(GIVE QUESTIONNAIRE TO RESPONDENT)

Question-by-Question Follow-Ups

PAGES 1-3	Did respondent carefully read instructions?
	Did respondent have any concerns or questions about the instructions?
SECTION 1
Q1-6	Stop respondent after completing the first page to gather feedback.
	Which child are you answering these questions about? How easy or difficult is it to keep your focus on the “sampled child” as you are answering these questions?
Q4	What is this question asking, in your own words?
	What were you thinking about when you answered this question?
Q5	What do you think of when you hear the words “charter school?” What does it mean to you? Do you know any examples of charter schools? How confident are you that CHILD’s school is/is not a charter school?
Q1-6	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS PAGE)
	Are there any other items on this page you would like to comment on?
Q7-19	As respondent fills out this section, please stop them at Q14 to do a “concurrent” think aloud.
Q7	Under what circumstances would someone answer yes to this question?
	How easy or difficult is it for you to recall whether you considered other schools?
Q9	What was this question asking, in your own words?
	What were you thinking about as you answered this question?
Q11	(Monitor to make sure respondent enters numbers instead of characters)
Q12	Were you satisfied with these response options?
Q14	STOP: Please think out loud as you are coming up with your answer to this question. Tell me what is going through your mind as you determine how to answer it.
	How could this question be improved?
Q14E	How did you come up with your answer for this one?
Q7-19	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS SECTION)
	Did you find the arrows and the instructions easy or difficult to follow?
	Are there any other items in this section you would like to comment on?
SECTION 2
Q21	STOP: Please think out loud as you are coming up with your answer to this question. Tell me what is going through your mind as you determine how to answer it.
Q20-25	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS SECTION)
	Are there any other items in this section you would like to comment on?
SECTION 3
Q26-31	For this whole section, please think out loud as you are coming up with your answers to these questions. Tell me what is going through your mind as you determine how to answer them.
	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS SECTION)
	Are there any other items in this section you would like to comment on?
SECTION 4
Q32-34	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS SECTION)
	Are there any other items in this section you would like to comment on?
SECTION 5
Q35-43	Did you find any questions in this section to be difficult to answer?
Q36	Were there any items in this question that you did not understand or did not know how to answer?
	What did the term “condition” mean to you and how did you determine whether or not to classify something as a condition?
Q37	Please tell me how you interpreted the skip pattern before Q37. Is there anything that could be changed to make this clearer?
	What was Q37 asking, in your own words? What does the term “services” mean to you in this question?
Q35-43	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS SECTION)
	Are there any other items in this section you would like to comment on?
SECTION 6
Q44-51	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS SECTION)
	Did you find any questions in this section to be difficult to answer?
SECTION 7
Q87	Please think out loud as you are coming up with your answer to this question. Tell me what is going through your mind as you determine how to answer it.
	Did you feel any confusion about who to think about when responding?
Q88	Who are you answering this question about? How did you determine who the “first” parent or guardian is?
	Did you notice the instruction at the bottom of the 1st column? How could this instruction be more clear?
Q103	Who were you thinking about when you answered Q103?
Q87-117	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS SECTION)
	Are there any other items in this section you would like to comment on?
SECTION 8
Q121	Please tell me what this question is asking, in your own words.
Q126	What were you thinking as you answered this question? How easy or difficult was it for you to determine how to answer these questions?
Q121-127	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS SECTION)
	Are there any other items in this section you would like to comment on?
SECTION 9
Q128-138	(GO OVER ANY OBSERVATIONS YOU HAD ON ANY OTHER ITEMS IN THIS SECTION)
	Are there any other items in this section you would like to comment on?
FINAL OBSERVATIONS
	Do you have anything else you would like to tell me about this questionnaire that you haven’t had a chance to mention?
	Okay, that’s all the questions. I want to thank you very much for all of the help you have given us. We appreciate it very much.

5