National Household Education Surveys Program (NHES) 2019 Types of Schooling Cognitive Interviews – Revision 2

National Household Education Surveys Program (NHES)

2019 Types of Schooling Cognitive Interviews

Attachment 2 – Interview Protocol

OMB# 1850-0803 v.216

(revised v.212)

June 2017

revised November 2017

Table of Contents

Navigation Protocol – English 2

Navigation Protocol – Spanish 7

New Content Protocol – English 12

New Content Protocol – Spanish 21

NHES PFI-C Cognitive Interview Protocol (Navigation)

RECORD DATE: _________________

RECORD START TIME: _________________

INTERVIEWER’S INITIALS: _________________

NOTETAKER’S INITIALS: _________________

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Hello, my name is <interviewer name> and I work for <Activate Research/Child Trends/ Sanametrix>. It’s nice to meet you. Thanks for speaking with me today.

We are having a conversation today to help with a survey that we are working on with the U.S. Department of Education. This survey asks about your family’s experiences with your child’s schooling, family activities, and your child’s health and background. A survey is a set of questions about your experiences. I will ask you to take the survey as if you were taking a real survey. There are no right or wrong answers, and it’s ok if you do not know some of the answers. After you get a chance to go through the survey, I will ask you questions. This will help us make the survey better and clearer for other parents just like you.

Because the information you provide is so important, I am going to be taking notes while you are working and while we are talking [IF THERE IS AN OBSERVER: and my colleague will also be observing our session today].

Also, this interview will be recorded so researchers can review the recordings later. Any information that refers to you or your family will not be included in our report. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

As a thank you for your time and effort, you will receive $75.

Remember, you are not being tested; there are no wrong answers and we want you to tell us what you think about the survey. I did not write these survey questions, so I won’t be upset by anything you tell me about them. Would you still like to be a part of this study? [If respondent agrees, continue. If not, end the interview.]

If at any time you want to stop, just let me know. [If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

Before we continue, do you have any questions about what I just said? [Answer any questions the participant may ask.]

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form (or read it to him/her, depending on respondent preference), answer any questions, and have the respondent sign the form. Leave a separate copy of the form with the respondent.

Fill out receipt of incentive form, have respondent sign and date.

Give the respondent the incentive.

The following is a pre-interview checklist. Please ensure the following are complete before continuing.]

• COLLECT SIGNED CONSENT FORM

• PROVIDE COPY OF CONSENT FORM TO RESPONDENT

• COLLECT SIGNED RECEIPT OF INCENTIVE FORM

• PROVIDE INCENTIVE TO RESPONDENT

• START THE RECORDER (IF THE RESPONDENT HAS CONSENTED)

STEP 2. DISCUSSION OF FAMILY’S EDUCATION EXPERIENCES

Before you start the survey, I would like to ask you to help me understand a typical week of school for <this child>. Can you please walk me through where <this child> is for school on each weekday (Monday through Friday)?

STEP 3: COMPLETION OF THE QUESTIONNAIRE

[IF IN-PERSON] Now we’re ready to begin. [Hand respondent a copy of the questionnaire and a pen.] Complete the questionnaire as you would do if you were home and had received this survey in the mail. When you are finished, I will be asking you some questions. You may see me take some notes.

[IF REMOTE] Now we’re ready to begin. Please open the email attachment named questionnaire.pdf. If you have printed the questionnaire out, please place it in front of you. Complete the questionnaire as you would do if you were home and had received this survey in the mail. When you are finished, I will be asking you some questions.

[NOTE ANY OBSERVATIONS OF RESPONDENT CONFUSION, ETC.

STOP RESPONDENT ANY TIME AFTER ITEM 91 IS COMPLETED.]

Question-by-Question Observation Guides

Sections: CHILD’S SCHOOLING through CHILD’S BACKGROUND
General	I’m going to stop you here and ask you some questions now about the survey. Do you have any initial reactions to the survey that you want to tell me about first? I noticed a few things while you were completing the survey. I’m going to ask you about those things first, and then we’ll go through the survey in order. [For each of the above areas where you noticed hesitance, confusion, or apparently incorrect skip behaviors] Let’s look back at question <number>. [Wait while respondent turns back to that page of the questionnaire.] Can you walk me through how you figured out the answer to this question? [If misunderstanding occurred based on instructions] Tell me what this introduction/instruction is telling you. [If misunderstanding occurred based on apparently incorrect answer choice or skip pattern] I noticed you skipped this/checked this. How did you choose this answer and not this other option?
Q2	[Explore how respondent answers each subpart and note how the answers compare to the description of child’s schooling that the respondent gave at the beginning of the interview.] Please tell me about your answer for Question number 2. What question did you answer after you completed Question number 2?
Q3, Q4	[Explore how respondent answers each of these questions with respect to their description of child’s schooling. Observe whether they understand/do not understand the skip patterns.] Please walk me through your answers to the next few questions.
Q12	[If applicable] What school were you thinking about when you answered the items in Question 12?
Q17	[If applicable] Walk me through your answer to Question 17.
After Q37	[Explore respondent’s process after the completion of this item (i.e., does s/he go straight to the next item, as intended? Is s/he confused about where to go next?)] Next, we are going to turn to item 38. Before we do that, are there any items before 38 that you want to tell me something about?
Q38	[Explore how respondent answers this question with respect to the description of child’s schooling that the respondent gave at the beginning of the interview.] [Explore whether they understand/do not understand the skip patterns.] Tell me about how you answered Q38.
Q39	[Explore how respondent answers this question with respect to the description of child’s schooling that the respondent gave at the beginning of the interview.] [Explore whether they understand/do not understand the skip patterns.] [If applicable] Walk me through your answers to the next few items after 38. What were your answers? Where did you go in the survey next?
Instructions before Q60	[Explore whether respondent understands the intended skip patterns here and whether s/he goes to the item we would have guessed based on the conversation at the beginning of the interview.] Next, we are going to turn to item 60. Before we do that, are there any items before 60 that you want to tell me about? When you got to that Families and School section, what did you read first? Where did you go next?

GENERAL FEEDBACK

Do you have anything else you would like to tell me about this questionnaire that you haven’t had a chance to mention?

[If this is a remote interview] May I please have your mailing address so that we may send you the $75 incentive for your participation today?

Name ____________________________________

Street ____________________________________

City ____________________________________

State ___________ Zip ________________

[At the end of the interview] Thank you so much for taking the time to work through this questionnaire. We appreciate your time and your insight.

NHES PFI-C Cognitive Interview Protocol (Navigation)

RECORD DATE: _________________

RECORD START TIME: _________________

INTERVIEWER’S INITIALS: _________________

NOTETAKER’S INITIALS: _________________

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Hola, mi nombre es <interviewer name> y trabajo para <Activate Research/Child Trends/ Sanametrix>. Es un gusto conocerle. Gracias por tomar el tiempo para hablar conmigo hoy.

Nuestra conversación hoy ayudará con una encuesta que estamos realizando junto con el Departamento de Educación de los Estados Unidos. Esta encuesta busca información sobre las experiencias de su familia con la educación de su hijo/a, actividades de la familia, y el salúd y historia personal del niño/a. Una encuesta es una serie de preguntas sobre sus experiencias. No hay respuestas incorrectas, y no hay problema si no sabe algunas respuestas. Una vez que haya tenido la oportunidad de revisar la encuesta, le voy a hacer una serie de preguntas. Eso nos ayudará a mejorar y hacer más clara la encuesta para otros padres como usted.

Porque esta información es tan importa, voy a tomar notas mientras usted trabaja y mientras nosotros hablamos [IF THERE IS AN OBSERVER: y mi colega también va a observar nuestra entrevista hoy].

Además, esta entrevista se grabará para que los investigadores puedan revisar las grabaciones más tarde. Cualquier información que se refiere a usted o a su familia no será incluida en nuestro reporte. Toda la información que usted proporcione puede ser usada solamente para propósitos estadísticas y no puede ser divulgada, o usada, de manera identificable para cualquier otro propósito excepto como requerido por la ley (20 U.S.C. §9573 y 6 U.S.C. §151).

En agradecimiento por su tiempo y esfuerzo, recibirá $75.

Recuerde, no está siendo probado; no hay respuesta incorrecta y queremos que usted nos diga que piensa acerca de la encuesta. Yo no escribí las preguntas de esta encuesta, así que no me va a molestar por cualquier cosa que me diga acerca de la encuesta. ¿Todavía le interesa participar en este estudio? [If respondent agrees, continue. If not, end the interview.]

Me avisa si es que quiere parar en cualquier momento. [If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

¿Antes de continuar, tiene preguntas acerca de lo que acabo de explicar? [Answer any questions the participant may ask.]

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form (or read it to him/her, depending on respondent preference), answer any questions, and have the respondent sign the form. Leave a separate copy of the form with the respondent.

Fill out receipt of incentive form, have respondent sign and date.

Give the respondent the incentive.

The following is a pre-interview checklist. Please ensure the following are complete before continuing.]

• COLLECT SIGNED CONSENT FORM

• PROVIDE COPY OF CONSENT FORM TO RESPONDENT

• COLLECT SIGNED RECEIPT OF INCENTIVE FORM

• PROVIDE INCENTIVE TO RESPONDENT

• START THE RECORDER (IF THE RESPONDENT HAS CONSENTED)

STEP 2. DISCUSSION OF FAMILY’S EDUCATION EXPERIENCES

Antes de comenzar la encuesta, quiero que usted me ayuda a entender lo que es una semana escolar típica para <este niño/a>. Me puede contar donde está <este niño/a> durante las horas de escuela (Lunes a Viernes)?

STEP 3: COMPLETION OF THE QUESTIONNAIRE

[IF IN-PERSON]: Ahora estamos listos para comenzar. [Hand respondent a copy of the questionnaire and a pen.] Complete la encuesta como si estuviera en casa y hubiese recibido la encuesta por correo. Cuando termine, le voy a hacer algunas preguntas.

[IF REMOTE]: Ahora estamos listos para comenzar. Abra por favor el archivo adjunto de correo electrónico llamado questionnaire.pdf. Si imprimió el archivo adjunto, pon lo delante suyo. Complete la encuesta como si estuviera en casa y hubiese recibido la encuesta por correo. Cuando termine, le haré algunas preguntas.

[NOTE ANY OBSERVATIONS OF RESPONDENT CONFUSION, ETC.

STOP RESPONDENT ANY TIME AFTER ITEM 91 IS COMPLETED.]

Question-by-Question Observation Guides

Sections: CHILD’S SCHOOLING through CHILD’S BACKGROUND
General	Le voy a parar aquí y voy a hacerle algunas preguntas sobre la encuesta. ¿Tiene alguna reacción inicial a la encuesta que me quiere contar? Me fije en algunas cosas cuando usted estaba completando la encuesta. Voy a hacerle preguntas sobre esas cosa primero, y luego revisaremos la encuesta en orden. [For each of the above areas where you noticed hesitance, confusion, or apparently incorrect skip behaviors] Vayamos a la pregunta <number>. [Wait while respondent turns back to that page of the questionnaire.] ¿Me puede explicar como llegó a su respuesta para esa pregunta? [If misunderstanding occurred based on instructions] Cuenteme que le dice esta introducción/instrucción. [If misunderstanding occurred based on apparently incorrect answer choice or skip pattern] Me dí cuenta que usted saltó esta pregunta/marcó esta respuesta. ¿Como llegó a esa respuesta y no esta respuesta?
Q2	[Explore how respondent answers each subpart and note how the answers compare to the description of child’s schooling that the respondent gave at the beginning of the interview.] Cuenteme sobre su respuesta para la Pregunta número 2. ¿Cual pregunta contestó despúes de completar la pregunta número 2?
Q3, Q4	[Explore how respondent answers each of these questions with respect to their description of child’s schooling. Observe whether they understand/do not understand the skip patterns.] Cuenteme sobre sus respuestas para las siguientes preguntas
Q12	[If applicable] ¿En cual escuela estaba pensando cuando contestó a los artículos de la pregunta 12?
Q17	[If applicable] Cuenteme sobre su respuesta para la Pregunta 17.
After Q37	[Explore respondent’s process after the completion of this item (i.e., does s/he go straight to the next item, as intended? Is s/he confused about where to go next?)] ¿Próximo, continuaremos con la pregunta 38. Pero, antes de seguir con la pregunta 38, hay algo que me quiere contar sobre cualquier pregunta antes de la pregunta 38?
Q38	[Explore how respondent answers this question with respect to the description of child’s schooling that the respondent gave at the beginning of the interview.] [Explore whether they understand/do not understand the skip patterns.] Cuenteme sobre su respuesta para Q38.
Q39	[Explore how respondent answers this question with respect to the description of child’s schooling that the respondent gave at the beginning of the interview.] [Explore whether they understand/do not understand the skip patterns.] [If applicable] Cuenteme sobre sus requestas para las próximas preguntas después de la pregunta 38. ¿Cuales eran sus respuestas? ¿Donde ha seguido próximo en la encuesta?
Instructions before Q60	[Explore whether respondent understands the intended skip patterns here and whether s/he goes to the item we would have guessed based on the conversation at the beginning of the interview.] Próximo, continuaremos con la pregunta 60. ¿Pero, antes de seguir con la pregunta 60, hay alo que me quiere contar sobre cualquier pregunta antes de la pregunta 60? ¿Cuando llegó a la sección de Familias y Escuela, que leyó primero? ¿Donde ha seguido próximo en la encuesta?

GENERAL FEEDBACK

¿Hay algo más que me quiere contar sobre esta encuesta que no ha tenido oportunidad para contarme?

[If this is a remote interview] ¿Me puede dar su dirección por favor para que le podamos enviar los $75 de incentivo por su participación hoy?

Name ____________________________________

Street ____________________________________

City ____________________________________

State ___________ Zip ________________

[At the end of the interview] Muchisimas gracias por tomar el tiempo para completar esta encuesta. Agradecemos mucho su tiempo y perspectiva

NHES PFI-C Cognitive Interview Protocol (New Content)

RECORD DATE: _________________

RECORD START TIME: _________________

INTERVIEWER’S INITIALS: _________________

NOTETAKER’S INITIALS: _________________

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Hello, my name is <interviewer name> and I work for <Activate Research/Child Trends/ Sanametrix>. It’s nice to meet you. Thanks for speaking with me today.

We are having a conversation today to help with a survey that we are working on with the U.S. Department of Education. This survey asks about your family’s experiences with your child’s schooling, family activities, and your child’s health and background. A survey is a set of questions about your experiences. I will ask you to take the survey as if you were taking a real survey. There are no right or wrong answers, and it’s ok if you do not know some of the answers. This will help us make the survey better and clearer for other parents just like you.

Because the information you provide is so important, I am going to be taking notes while you are working and while we are talking [IF THERE IS AN OBSERVER: and my colleague will also be observing our session today].

Also, this interview will be recorded so researchers can review the recordings later. Any information that refers to you or your family will not be included in our report. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

As a thank you for your time and effort, you will receive $75.

Remember, you are not being tested; there are no wrong answers and we want you to tell us what you think about the survey. I did not write these survey questions, so I won’t be upset by anything you tell me about them.

If at any time you want to stop, just let me know. [If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

Before we continue, do you have any questions about what I just said? [Answer any questions the participant may ask.]

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form (or read it to him/her, depending on respondent preference), answer any questions, and have the respondent sign the form. Leave a separate copy of the form with the respondent.

Fill out receipt of incentive form, have respondent sign and date.

Give the respondent the incentive.

The following is a pre-interview checklist. Please ensure the following are complete before continuing.]

• COLLECT SIGNED CONSENT FORM

• PROVIDE COPY OF CONSENT FORM TO RESPONDENT

• COLLECT SIGNED RECEIPT OF INCENTIVE FORM

• PROVIDE INCENTIVE TO RESPONDENT

• START THE RECORDER (IF THE RESPONDENT HAS CONSENTED)

STEP 2: THINK-ALOUD PRACTICE

Next I am going to ask you to complete this survey. While you are completing it, I would like you to read out loud everything that you would be reading to yourself if you were completing the survey by yourself at home. I’d also like you to “think aloud” as you answer the questions. Hearing you talk about how you figure out your answers to the questions will help me understand how to make the survey better.

That’s a little different from anything you have probably done before, so we are going to do a practice question. I’ll start just to show you what I am looking for using this sample question.

[Interviewer, to demonstrate an example: Look at the card with the number of windows question. Read aloud. Then answer the question about your own home using think aloud. Include detail about panes of glass in doors, etc.]

Okay, now it’s your turn, I’ll ask you to answer the question and think aloud about your answer.

[If in person, pass the show card to the respondent. If remote, read aloud the question to the respondent. If they just give a number, ask them to do it again, but this time to say more about how they are arriving at their answer so that you can understand how they came up with the number.]

[If needed: What are you including?]

Great, thank you. That’s the kind of detail I am looking for throughout our session. I will remind you to continue to think aloud if you get quiet.

STEP 3: DISCUSSION OF FAMILY’S EDUCATION EXPERIENCES

Before you start the survey, I would like to ask you to help me understand a typical week of school for <this child>. Can you please walk me through where <this child> is for school on each weekday (Monday through Friday)?

STEP 4: COMPLETION OF THE QUESTIONNAIRE

[IF IN-PERSON] Now we’re ready to begin. [Hand respondent a copy of the questionnaire and a pen.]

[IF REMOTE] Now we’re ready to begin. Please open the email attachment named questionnaire.pdf. If you have printed the questionnaire out, please place it in front of you.

Remember that I’d like you to talk aloud while you are thinking of your answer and decide on your answer. Also, remember please to read aloud anything you would have read to yourself if I were not here.

Remember that I am very interested in hearing you talk about what you are thinking as you figure out your answers to the questions.

General Probes
Problem Items	What is this question asking, in your own words?
	What were you thinking when you answered this question?
Problem Instructions	Tell me what this introduction/instruction is telling you.
Problems with Clarity	Can you tell me in your own words what this question is asking?
	What does this word <term> mean to you? [in reference to a particular word of technical term]
	Tell me what you were thinking when I asked you about this <topic, time period, degree, certification, etc.>
Skip and Marking Issues	I noticed you skipped this/checked this. How did you choose this answer and not this other option?
	Can you tell me how you chose this category? How easy or hard was it to find your answer on that list?

Question-by-Question Think Aloud Probes

CHILD’S SCHOOLING
Q2	I’m going to ask you to really think aloud on this next item and walk me through how you are answering it. [Listen for agreement between answer choices and respondent’s description of child’s schooling discussed prior to the interview. Use follow-up probes if necessary to understand how respondent thought through each item.]
Q3	[Navigation comprehension: Be alert to problems here and consider probes like] Tell me what is happening. Can you tell me more about number 3?
Q4	[Navigation comprehension: Be alert to problems here and consider probes like] Tell me what is happening. Can you tell me more about number 4?
Q5	[Listen for difficulties in figuring out what school to answer about.] In your own words, what is this question asking?
Q7	Tell me more about this one. [Listen for comprehension problems and encoding problems. Does respondent know what we are asking? Does respondent know if the school is a magnet school or not?]
Q9.	How did you arrive at your answer for this one? In your own words, what is this question asking? [Are respondents interpreting this item consistently?]
Q12	I’m going to ask you to really think aloud on this next item and walk me through how you are answering it. [a-l are new items; take note of any kind of problem and figure out why it is happening.] [If needed] Are there other reasons for choosing a school that aren’t here in these questions?
Q13	Can you tell me how you chose these categories? How easy or hard was it to find your answers on that list? [For each response marked, make sure to understand why respondent chose it. What is respondent referring to?]
Q16	Can you walk me through how you answered this question?
Q17	How did you arrive at the answer for this question? [Probe for what respondent “counts” as being online, virtual, or cyber courses.] [Is respondent consistent about what they count as an online, virtual, or cyber course?] [Probe for respondent’s understanding of the instruction not to include courses that use the Internet for select assignments.]
Q18	I’m going to ask you to really think aloud on this next item and walk me through how you are answering it. [If needed] Can you tell me how you chose yes or no for each category? How easy or hard was it to determine each answer for the list?
Q19	Tell me more about your answer.
Q20	Can you tell me in your own words what this question is asking? [Probe/listen for respondent’s understanding of who gives credit for courses.] In your own words, what does the response option you chose mean?
Q21		How did you arrive at that answer? [Probe for what sorts of activities respondent included in the hour estimate – receiving instruction, completing assignments, etc.]
Q22		Can you talk me through how you arrived at this answer? [Probe for what respondent counted as a “class.” Listen for consistencies/inconsistencies in answers to items 17, 21, and 22.]
Q33		Can you talk me through how you arrived at your answers for each part? How easy or hard was it for you to figure out each amount?
Q34		What types of activities or care were you thinking about as you answered each part of this question?
Q35		Tell me more about this one. In your own words, what is this question asking? What does the term “Education Savings Account” mean to you?
Q36		In your own words, what is this question asking? What does the term “school voucher” mean to you?
Q37		In your own words, what is this question asking? What does the term “tax credit scholarship” mean to you?

CHILD’S HOMESCHOOLING
Before Q38		[Take note of respondent’s navigation between item 37 and 38. The intention is that all respondents answer item 38.]
Q38	Can you tell me in your own words what this question is asking you? [Probe for understanding of some homeschool/no homeschool distinction, and note respondent’s ability to navigate away from this question.]
Q39		Can you talk me through how you arrived at your answer for this question? [Probe for clarity of choices and compare respondent’s response to conversation at beginning of interview about child’s schooling.] [Note respondent’s ability to navigate away from this question.]
Q40		[If needed] What is the last response option saying, in your own words?
Q41		How did you arrive at the answer for this question? [Probe for what respondent counts as being online, virtual, or cyber courses.] [Probe for respondent’s understanding of the instruction not to include courses that use the Internet for select assignments.] [If needed] In your own words, what are these response options?
Q42		What is this question asking, in your own words? [Probe for respondent’s understanding of question and attempt to understand how easy/hard it is for respondent to come to an answer, given respondent’s relationship to homeschooling.]
Q43		What were you thinking when you answered this question? Tell me more about how you arrived at an answer for this one. [Probe for what sorts of activities respondent included in the hour estimate – receiving instruction, completing assignments, etc.]
Q48		What were you thinking when you answered this question? [Probe for: Ability to answer about only online sources Understanding of each category and what “counts” for each category Understanding of distinction of “Free websites” Understanding of distinction of “organized …resources”]
Q49		What were you thinking when you answered this question? [Probe for: Ability to answer about only physical sources Understanding of each category and what “counts” for each category]
Q52		Can you talk me through how you arrived at your answers for this question? [Pay special attention to parts (h), (i), (j), and (k). If those are chosen] In your own words, what does [h] mean?
Q55		[Note respondent’s approach to this question. Is respondent providing brief or lengthy descriptions? Does respondent hesitate or seem confused? Probe/follow up as necessary. How general or specific are respondent’s responses and why?]
Q56		What does the term “local” mean to you here? Can you talk me through how you arrived at your answer for this question? [Probe for how respondent is thinking about local: geographic region, in-person as opposed to online, other? Listen for what “counts” as an activity or meeting.]
Q59		In your own words, what is this question asking?

FAMILIES AND SCHOOL
Before Q60	[Take note of respondent’s navigation based on the instructions.] [Compare respondent’s navigation to his/her description of child’s schooling at the beginning of the interview.] [Does respondent read the text before Q60?] [Does respondent have difficulty deciding what school to report about?]

CHILD’S HEALTH
Q75	[If respondent chooses a], In your own words, what does a mean?

CHILD’S BACKGROUND
Q91	How did you arrive at your answer to 91?

GENERAL FEEDBACK

Do you have anything else you would like to tell me about this questionnaire that you haven’t had a chance to mention?

[If this is a remote interview] May I please have your mailing address so that we may send you the $75 incentive for your participation today?

Name ____________________________________

Street ____________________________________

City ____________________________________

State ___________ Zip ________________

[At the end of the interview] Thank you so much for taking the time to work through this questionnaire. We appreciate your time and your insight.

NHES PFI-C Cognitive Interview Protocol (New Content)

RECORD DATE: _________________

RECORD START TIME: _________________

INTERVIEWER’S INITIALS: _________________

NOTETAKER’S INITIALS: _________________

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Hola, mi nombre es <interviewer name> y trabajo para <Activate Research/Child Trends/ Sanametrix>. Es un gusto conocerle. Gracias por tomar el tiempo para hablar conmigo hoy.

Nuestra conversación hoy ayudará con una encuesta que estamos realizando con el Departamento de Educación de los Estados Unidos. Esta encuesta busca información sobre las experiencias de su familia con la educación de su hijo/a, actividades de la familia, y el salúd y historia personal del niño/a. Una encuesta es una serie de preguntas sobre sus experiencias. No hay respuestas incorrectas, y no hay problema si no sabe algunas respuestas. Una vez que haya tenido la oportunidad de revisar la encuesta, le voy a hacer una serie de preguntas. Eso nos ayudará a mejorar y hacer más clara la encuesta para otros padres como usted.

Porque esta información es tan importa, voy a tomar notas mientras usted trabaja y mientras nosotros hablamos [IF THERE IS AN OBSERVER: y mi colega también va a observar nuestra entrevista hoy].

Además, esta entrevista se grabará para que los investigadores puedan revisar las grabaciones más tarde. Cualquier información que se refiere a usted o a su familia no será incluida en nuestro reporte. Toda la información que usted proporcione puede ser usada solamente para propósitos estadísticas y no puede ser divulgada, o usada, de manera identificable para cualquier otro propósito excepto como requerido por la ley (20 U.S.C. §9573 y 6 U.S.C. §151).

En agradecimiento por su tiempo y esfuerzo, recibirá $75.

Recuerde, no está siendo probado; no hay respuesta incorrecta y queremos que usted nos diga que piensa acerca de la encuesta. Yo no escribí las preguntas de esta encuesta, así que no me va a molestar por cualquier cosa que me diga acerca de la encuesta.

Me avisa si es que quiere parar en cualquier momento. [If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

¿Antes de continuar, tiene preguntas acerca de lo que acabo de explicar? [Answer any questions the participant may ask.]

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form (or read it to him/her, depending on respondent preference), answer any questions, and have the respondent sign the form. Leave a separate copy of the form with the respondent.

Fill out receipt of incentive form, have respondent sign and date.

Give the respondent the incentive.

The following is a pre-interview checklist. Please ensure the following are complete before continuing.]

• COLLECT SIGNED CONSENT FORM

• PROVIDE COPY OF CONSENT FORM TO RESPONDENT

• COLLECT SIGNED RECEIPT OF INCENTIVE FORM

• PROVIDE INCENTIVE TO RESPONDENT

• START THE RECORDER (IF THE RESPONDENT HAS CONSENTED)

STEP 2: THINK-ALOUD PRACTICE

Ahora, le voy a pedir que complete la encuesta. Mientras lo esta completando, le voy a pedir que lea en voz alta todo lo que leería si usted estuviera completando la encuesta a solas en su casa. También, le voy a pedir que me cuente todo lo que esta pensando mientras responde a las preguntas. Escucharle explicar como llegó a sus respuestas a las preguntas me ayudará a entender como mejorar a la encuesta.

Porque esto tal vez sea algo que no ha hecho anteriormente, vamos a usar una pregunta como práctica. Voy a contestar una pregunta de muestra para demostrarle lo que estoy buscando.

[Interviewer, to demonstrate an example: Look at the card with the number of windows question. Read aloud. Then answer the question about your own home using think aloud. Include detail about panes of glass in doors, etc.]

Ahora le toca a usted, le voy a hacer una pregunta y quiero que me cuente lo que esta pensando mientras responde a esta pregunta.

[If in person, pass the show card to respondent. If remote, read aloud the question to respondent. If they just give a number, ask them to do it again, but this time to say more about how they are arriving at their answer so that you can understand how they came up with the number.]

[If needed: ¿Que está incluyendo?]

Bien, gracias. Ese es nivel de detalle que busco durante esta entrevista. Le voy a recordar al rato si deja de hacer eso.

STEP 3: DISCUSSION OF FAMILY’S EDUCATION EXPERIENCES

Antes de comenzar la encuesta, quiero que usted me ayuda a entender lo que es una semana escolar típica para <este niño/a>. ¿Me puede contar donde está <este niño/a> durante las horas de escuela (Lunes a Viernes)?

STEP 4: COMPLETION OF THE QUESTIONNAIRE

[IF IN-PERSON] Ahora estamos listos para comenzar. [Hand respondent a copy of the questionnaire and a pen.]

[IF REMOTE] Ahora estamos listos para comenzar. Abra por favor el archivo adjunto de correo electrónico llamado questionnaire.pdf. Si imprimió el archivo adjunto, pon lo delante suyo.

Recuerde que quiero que me cuente lo que esta pensando mientras esta pensando en una respuesta y cuando decide en una respuesta. También recuerde por favor a leer en voz alta cualquier cosa que hubiese leído si no estuviera you aquí.

Recuerde que me interesa mucho oírlo hablar de lo que esta pensando mientras esta tratando de decidir sus respuestas a las preguntas.

General Probes
Problem Items	¿En sus propias palabra, que esta preguntando esta pregunta?
	¿Que estaba pensando cuando estaba contestando esta pregunta?
Problem Instructions	Cuenteme que le dice esta introducción/instrucción.
Problems with Clarity	¿En sus propias palabras me puede explicar lo que esta preguntando esta pregunta?
	¿Que significa <term> para usted? [in reference to a particular word of technical term]
	Cuenteme lo que estaba pensando cuando le preguntaba acerca de esto <topic, time period, degree, certification, etc.>
Skip and Marking Issues	Me dí cuenta que usted saltó esta pregunta/marcó esta respuesta. ¿Como llegó a esa respuesta y no esta respuesta?
	¿Me puede explicar como escogió esa categoría? Que tan difícil o fácil fue encontrar su respueta en esa lista?

Question-by-Question Think Aloud Probes

CHILD’S SCHOOLING
Q2	Le voy a pedir que realmente me cuente lo que esta pensando con esta próxima pregunta y me explica como esta decidiendo como contestarlo [Listen for agreement between answer choices and respondent’s description of child’s schooling discussed prior to the interview. Use follow-up probes if necessary to understand how respondent thought through each item.]
Q3	[Navigation comprehension: Be alert to problems here and consider probes like] Cuenteme lo que esta pensando. ¿Me puede contar má acerca del número 3?
Q4	[Navigation comprehension: Be alert to problems here and consider probes like] Cuenteme lo que esta pensando. ¿Me puede contar más acerca del número 4?
Q5	[Listen for difficulties in figuring out what school to answer about.] ¿En sus propias palabras me puede explicar lo que esta preguntando esta pregunta?
Q7	Cuenteme más acerca de esta. [Listen for comprehension problems and encoding problems. Does respondent know what we are asking? Does respondent know if the school is a magnet school or not?]
Q9.	¿Como llegó a su respuesta para esta? ¿En sus propias palabras me puede explicar lo que esta preguntando esta pregunta? [Are respondents interpreting this item consistently?]
Q12	Le voy a pedir que de verdad me cuente lo que esta pensando con esta próxima pregunta y me explica como esta decidiendo como contestarlo. [a-l are new items; take note of any kind of problem and figure out why it is happening.] [If needed] ¿Hay otras razones para escoger a una escuela que no aparecen en estas preguntas?
Q13	Me puede explicar como escogió esas categorías? ¿Que tan difícil o fácil fue encontrar su respueta en esa lista? [For each response marked, make sure to understand why respondent chose it. What is respondent referring to?]
Q16	¿Me puede explicar como llegó a su respuesta para esat pregunta?
Q17	¿Como llegó a la respuesta para esta pregunta? [Probe for what respondent “counts” as being online, virtual, or cyber courses.] [Is respondent consistent about what they count as an online, virtual, or cyber course?] [Probe for respondent’s understanding of the instruction not to include courses that use the Internet for select assignments.]
Q18	Le voy a pedir que de verdad me cuente lo que esta pensando con esta próxima pregunta y me explica como esta decidiendo como contestar. [If needed] ¿Me puede explicar por qué escogió sí o no para cada categoría? ¿Que tan difícil o fácil fue decidir cada respueta para esta lista?
Q19	Cuenteme más acerca de su respuesta.
Q20	¿En sus propias palabras me puede explicar lo que esta preguntando esta pregunta? [Probe/listen for respondent’s understanding of who gives credit for courses.] ¿En sus propias palabra, que significa la opción que escogió?
Q21		¿Como llegó a esa respuesta? [Probe for what sorts of activities respondent included in the hour estimate – receiving instruction, completing assignments, etc.]
Q22		¿Me puede explicar como llegó a su respuesta para esta pregunta? [Probe for what respondent counted as a “class.” Listen for consistencies/inconsistencies in answers to items 17, 21, and 22.]
Q33		¿Me puede explicar como llegó a su respuesta para cada parte? Que tan difícil fue decidir la cantidad para cada uno?
Q34		¿En que clase de actividades o cuidado estaba pensando mientras respondía a cada parte de esta pregunta?
Q35		Cuenteme más acerca de esta. ¿En sus propias palabras me puede explicar lo que esta preguntando esta pregunta? ¿Que significa el término “Cuenta de Ahorros Educativo” para usted?
Q36		¿En sus propias palabras me puede explicar lo que esta preguntando esta pregunta? ¿Que significa el término “vale escolar” para usted?
Q37		¿En sus propias palabras me puede explicar lo que esta preguntando esta pregunta? ¿Que significa el término “beca de crédito de impuestos” significa para usted?

CHILD’S HOMESCHOOLING
Before Q38		[Take note of respondent’s navigation between item 37 and 38. The intention is that all respondents answer item 38.]
Q38	¿En sus propias palabras me puede explicar lo que esta preguntando esta pregunta? [Probe for understanding of some homeschool/no homeschool distinction, and note respondent’s ability to navigate away from this question.]
Q39		¿Me puede explicar como llegó a su respuesta para esta pregunta? [Probe for clarity of choices and compare respondent’s response to conversation at beginning of interview about child’s schooling.] [Note respondent’s ability to navigate away from this question.]
Q40		[If needed] ¿En sus propias palabras me puede explicar lo que significa la última opción?
Q41		¿Me puede explicar como llegó a su respuesta para esta pregunta? [Probe for what respondent counts as being online, virtual, or cyber courses.] [Probe for respondent’s understanding of the instruction not to include courses that use the Internet for select assignments.] [If needed] ¿En sus propias palabras que significan estas opciones?
Q42		En sus propias palabras me puede explicar lo que esta preguntando esta pregunta? [Probe for respondent’s understanding of question and attempt to understand how easy/hard it is for respondent to come to an answer, given respondent’s relationship to homeschooling.]
Q43		¿Que estaba pensando cuando contestaba esta pregunta? Tell me more about how you arrived at an answer for this one. [Probe for what sorts of activities respondent included in the hour estimate – receiving instruction, completing assignments, etc.]
Q48		¿Que estaba pensando cuando contestaba esta pregunta? [Probe for: Ability to answer about only online sources Understanding of each category and what “counts” for each category Understanding of distinction of “Free websites” Understanding of distinction of “organized …resources”]
Q49		¿Que estaba pensando cuando contestaba esta pregunta? [Probe for: Ability to answer about only physical sources Understanding of each category and what “counts” for each category]
Q52		¿Me puede explicar como llegó a sus respuestas para esta pregunta? [Pay special attention to parts (h), (i), (j), and (k). If those are chosen] En sus propias palabras, me puede explicar lo que significa la respuesta para h?
Q55		[Note respondent’s approach to this question. Is respondent providing brief or lengthy descriptions? Does respondent hesitate or seem confused? Probe/follow up as necessary. How general or specific are respondent’s responses and why?]
Q56		¿Que significa el término “local” para usted? ¿Me puede explicar como llegó a su respuesta para esta pregunta? [Probe for how respondent is thinking about local: geographic region, in-person as opposed to online, other? Listen for what “counts” as an activity or meeting.]
Q59		¿En sus propias palabras me puede explicar lo que esta preguntando esta pregunta?

FAMILIES AND SCHOOL
Before Q60	[Take note of respondent’s navigation based on the instructions.] [Compare respondent’s navigation to his/her description of child’s schooling at the beginning of the interview.] [Does respondent read the text before Q60?] [Does respondent have difficulty deciding what school to report about?]

CHILD’S HEALTH
Q75	[If respondent chooses a], ¿En sus propias palabras me puede explicar lo que significa la respuesta para a?

CHILD’S BACKGROUND
Q91	¿Como llegó a su respuesta para el número 91?

GENERAL FEEDBACK

¿Hay algo más que me quiere contar sobre esta encuesta que no ha tenido oportunidad para contarme?

[If this is a remote interview], ¿Me puede dar su dirección por favor para que le podamos enviar los $75 de incentivo por su participación hoy?

Name ____________________________________

Street ____________________________________

City ____________________________________

State ___________ Zip ________________

[At the end of the interview] Muchisimas gracias por tomar el tiempo para completar esta encuesta. Agradecemos mucho su tiempo y perspectiva.

2