Form Attachment 2 & 3 Attachment 2 & 3 Attachment 2 & 3 - NHES 2019 Usability Testing Screener

2019 National Household Education Survey (NHES) Web Usability Testing

Attachment 2 & 3

Attachment 2 Recruitment Screener [English and Spanish]…..p.2

Attachment 3 – Protocol Materials…..p.11

OMB# 1850-0803 v.234

National Center for Education Statistics (NCES)

June 2018

The screener questions, protocol, background questionnaire, any outstanding debriefing questions, and the satisfaction questionnaire that are provided in this document in English will be translated into Spanish language for Spanish speaking participants and will be provided to OMB for approval by mid-September 2018.

Attachment 2 Recruitment Screener [English and Spanish]

NHES: 2019 Usability Testing

Hello. My name is [XXXXXX] and I work for the U.S. Census Bureau. The U.S. Department of Education is conducting a study about children’s care and education. The Census Bureau is assisting with the research.

We are looking for adult volunteers to help us improve the web survey used in this study. We want to know how easy or hard it is to complete the web survey, and we need help from people like you to do that. The information gained from the study will be used for potential revisions to the web survey. Your participation is voluntary, and you will receive no sales pitches or follow-up calls based on your participation. I’d like to ask you a few questions to see if you qualify for the study.

Collect code if offered (English only): _______________________________

1. What is your name? (write in First and Last name in separate fields)

________________________________________________

2. What is your daytime phone number?

(Probe for all 3: mobile, home (if not mobile), and work (if any))

_________________________________________________

3. How many people age 18 or older currently live in your household, INCLUDING yourself?

[ ]

4. How many people younger than 18 currently live in your household?

[ ]

5. Are you a parent or guardian of any children who are in Kindergarten to 12^th grade?

1. Yes

2. No

Name or Initials	Grade (K-12 or no grade)	How old is this child?	Is this child male or female?	Is this child attending a public school? (Y or N)	Is this child attending a private school? (Y or N)	Is this child homeschooled for grades equivalent to K to 12th? (Y or N)	Is this child attending a combination of homeschooling and other schooling? (Y or N) then Please explain the combination of schooling the child receives. (400 character field)

6. Are you a parent or guardian of any children not yet in Kindergarten?

1. Yes (Go to Check)

2. No (Go to Check)

Name or Initials	Is this child NOT YET in kindergarten? (Y or N)	How old is this child?	Is this child male or female?	Is this child in a care arrangement, e.g., family day care, day-care center, preschool, prekindergarten, Head Start or other early childhood program? (Y or N) Please explain the type of caregiving arrangement the child is in. (400 characters)

CHECK: If no children, then thank and terminate call for NHES. If children, determine if they meet the current recruiting needs and then either proceed or thank and terminate call for NHES.

7. ASK/CONFIRM IF NOT OBVIOUS: Are you male or female?

o Male

o Female

8. In what year were you born? (write in)

________

9. Are you Hispanic, Latino, or Spanish origin?

o Yes

o No

o Prefer not to answer

10. What is your race? Please select one or more. White, Black or African American, American Indian or Alaska Native, Asian, or Native Hawaiian or other Pacific Islander?

1. American Indian or Alaska Native

2. Asian

3. Black or African American

4. Native Hawaiian or other Pacific Islander

5. White

6. Prefer not to answer

11. What is the highest grade of school you have completed, or the highest degree you have received? (do not read categories)

o Less than High School

o High School Graduate

o Some College, no degree

o Associate’s degree (AA/AS)

o Bachelor’s degree (BA/BS)

o Post-Bachelor's degree (For example MA, MS, Ph.D., JD, etc.)

12. Are you working for pay full-time, part-time, or are you not working for pay at this time?

o Full-time

o Part-time

o Not working for pay at this time  SKIP TO Q13

a. What is your current job title or job description?

_________________________________________________

2. Are you employed by the Federal Government? (if federal contractor, mark NO)

o Yes

o No

13. In what city, state, and ZIP code do you currently live?

14. Do you own a cell phone?

o Yes

o No  Skip to Q15

1. Some cell phones are called “smartphones” because of certain features they have. Is your cell phone a smartphone, such as an iPhone, Android, Blackberry, or Windows phone, or are you not sure?

o Yes

o No

o Not sure

2. What type of smartphone do you have?

o iPhone

o Android

o Blackberry

o Windows phone

o Not sure

15. Do you own a desktop or laptop computer, or both?

o Desktop computer

o Laptop computer

o Both desktop and laptop computer

o Neither

16. Do you own a tablet computer such as an iPad, Samsung Galaxy Tab, Google Nexus, or Kindle Fire?

o Yes

o No

17. Do you have at least one year of Internet experience?

o Yes

o No  SKIP TO Q18

a. How often do you use the Internet either on a computer or on a mobile device like a smartphone or tablet? Several times a day, about once a day, a few times a week, a few times a month, or a few times a year?

o Several times a day

o About once a day

o A few times a week

o A few times a month

o A few times a year

o Not at all [do not read to respondent]

b. Name two things you do on the Internet besides e-mail.

[ ]

[ ]

Internet Activity #1:

Research

Social media/communication (Facebook, Twitter, Craigslist)

Shopping

Banking or paying bills online

Music / movies / videos

Gaming

News

Travel (directions/reservations)

Employment

Other

Internet Activity #2:

Research

Social media/communication (Facebook, Twitter, Craigslist)

Shopping

Banking or paying bills online

Music / movies / videos

Gaming

News

Travel (directions/reservations)

Employment

Other

18. How did you hear about this research opportunity?

[ ] E-mail link

[ ] Census Bureau employee

[ ] Friend or family member (but not a Census Bureau employee)

[ ] Facebook

[ ] Twitter

[ ] LinkedIn

[ ] Craigslist

[ ] Google advertisement

[ ] Flyer

[ ] Newspaper advertisement

[ ] Other – specify [ ]

19. Is your household’s annual income…?

o Less than $15,000?

o $15,000 to $25,000?

o $25,000 to$50,000?

o $50,000 to $100,000?

o More than $100,000?

20. Are you a U.S. citizen? You do not have to be a U.S. citizen to participate in our studies.

o Yes

o No - cannot come to headquarters

21. Have you participated in a research study or focus group in the past 6 months? (RECRUIT NONE.)

1. Yes (Thank and terminate.)

2. No (Continue)

22. What is the email address we should use when sending directions to our office, should you qualify for a study? (write in)

_________________________________________________

All recruits MUST be articulate, able to answer open ended questions, have at least one year of Internet experience, have a child that fits into the needed recruiting criteria, and able to provide explanations for their responses.

Based on your answers, we would like to invite you to participate in an interview session (to discuss your child’s care and education). You will receive $60 as a thank you for completing the interview. The interview is about 90 minutes in length and will be conducted (at the U.S. Census Bureau headquarters in Suitland, Maryland or at a local library or community center).

English sessions:

Are you willing to come to Suitland, Maryland to the Census Bureau headquarters for the session?

o Yes	 (CONTINUE)
o No	 (participant prefers other location – gather general location that would work for participant and CONTINUE)

English and Spanish:

[If they use a Smartphone, ask] So that we can see how you use the online survey, we would like for you to bring your smartphone with you to the study session.

(if willing to come to Suitland) Will you be willing to use your data on your phone to conduct the survey as the Census Bureau cannot provide WiFi?

o Yes	 (CONTINUE)
o No	 (schedule either offsite at a library or use a laptop)

[If they are in the laptop condition tell them] We will provide a laptop on site for you to use.

We are conducting interviews between __________ and ___________. Are you available to participate in this research study?

o Yes		 (CONTINUE AND GIVE DIRECTIONS)
O No		 (THANK AND TERMINATE)

RECRUITER: Check off all of the criteria this adult meets in order to categorize the interviews we will conduct.

Interview Criteria:

Parent of K-12 child public	Parent of K-12 child private	Parent of child < 6 NOT enrolled in school	Parent of homeschool child (K-12 equivalent)	Parent of homeschool child who also attends another school (K-12 equivalent)	Not eligible for topical (no eligible children)

NHES:2019 Usability Testing [Additional Spanish-only Questions] (OMB# 0607-0725)¹

23. With regard to your culture, do you consider yourself indigenous?

o Yes

o No

24. Were you born in the United States or in another country?

o United States  SKIP TO Q25

o Another country

a. In what country were you born? _____________

b. How long have you been in the United States? ______________

c. Were any of your other household members born in the U.S.?

o Yes

o No

d. Did you complete your schooling in the U.S. or in another country?

o United States  SKIP TO Q25

o Another country

e. In what country did you complete your schooling? ____________

25. For each of the people that live in your household, I’d like to ask you their relationship to you. I don’t need their names – just their relationship to you.

Don’t read

[ ] Opposite sex husband/wife

[ ] Same sex husband/wife

[ ] Opposite sex unmarried partner

[ ] Same sex unmarried partner

[ ] Son or daughter

[ ] Brother or sister

[ ] Father or mother

[ ] Grandchild

[ ] Parent-in-law

[ ] Son-in-law or daughter-in-law

[ ] Other relative (specify: ___________)

[ ] Housemate or roommate

[ ] Roomer or boarder

[ ] Landlord

[ ] Other nonrelative (specify: ____________)

26. What is your native language or languages?

[ ] Spanish

[ ] English

[ ] Other (Specify:______________)

27. [IF NOT MENTIONED AT Q26] Do you speak any indigenous language?

o Yes (Specify:__________)

o No

28. Do either of your parents speak an indigenous language?

o Yes (Specify:__________)

o No

29. We do interviews and focus groups in many different languages at the Census Bureau. I’d like to start by asking you about your Spanish. How well do you speak Spanish?

O Very well

O Well

O Not well

O Not at all

30. How well do you read Spanish?

O Very well

O Well

O Not well

O Not at all

31. [IF NEEDED: It’s not necessary that you speak English to participate in one of our studies, but I need to ask you a few questions about English now.] How well do you speak English?

O Very well

O Well

O Not well

O Not at all

32. How well do you read English?

O Very well

O Well

O Not well

O Not at all

33. Would you say you speak Spanish better than English, English better than Spanish, or both about equally well?

O Spanish better than English

O English better than Spanish

O Both about equally well

34. When you have a choice between English and Spanish, do you typically participate in surveys or fill out forms in…?

O Spanish

O English

O Both equally often

35. What is your daytime telephone number? (SAME as English)

[Probe for all 3 phone numbers]

Mobile ([ ]) [ ] – [ ]

Home (if not mobile) ([ ]) [ ] – [ ]

Work (if any) ([ ]) [ ] – [ ] EXT.[ ]

35a. IF PROVIDED MOBILE NUMBER: If you are eligible to participate in a study, we may send you a text message to remind you of your appointment. Can you receive text messages, or do you prefer not receive them?

36. If you are eligible to participate in a study, we may send you an email confirmation with directions to the interview location. What is the e-mail address we could use when sending directions?

_______________________________________________

Attachment 3 – Protocol Materials

NHES: 2019 Usability Testing

Table of Contents

Usability Testing Protocol 12

Practice Think Aloud 13

Eyetracking 13

Interview – Starting with Invitation letter 13

Interruption – Log out 14

FAQ Interruption 14

Satisfaction Questions 14

Debriefing Questions 14

Additional Debriefing Questions 21

Finishing up session 30

Usability Testing Protocol

MATERIALS NEEDED FOR INTERVIEW

• Interviewer Protocol Script

• Consent Form

• Mailed Invitation Letter with Login ID

• Laptop with the Demographic and Satisfaction Survey and Tobii software

• Tobii x260

• Camera on tripod for mobile device testing

• Incentive

• Payment Receipt

• Pens and Paper

• Guest Wireless Username and Password

• Extra Equipment and Charging Cords

INTERVIEWER USABILITY TESTING SCRIPT

Thank you for your time today. My name is XX and I work at the U.S. Census Bureau. I will be working with you today. In this lab, we evaluate how easy or difficult our various web survey products are to use. What works well, we keep. When potential users, such as you, have difficulty with something, we have an opportunity to fix it before it goes live to a much larger group.

Before we start, there is a form I would like you to read and sign. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to take a video recording of the device as you use it along with the audio from this session to get an accurate record of your feedback. Your name will not be associated with the recording or any of the other data collected during the session.

[Hand NHES consent form; give time to read and sign; sign own name and date. Show participants where the camera is and what part of the screen will be recorded. Then start recording.]

OK great.

Thank you.

There are several parts to today’s session.

The first task is to complete this questionnaire. It collects background information about you.

[Have participant fill out the usability background materials: demographic questionnaire on the laptop (for English) and on paper for Spanish—simply move away from where they are answering so they don’t feel like we are looking over their shoulder.]

Thank you.

The second part of today’s session is for you to help us evaluate the design of a survey that we are working on for the U.S. Department of Education. You will be helping us evaluate the National Household Education Survey. You will do this by completing it using [your smartphone/this laptop].

If you run into any difficulties as you work on the survey, please don’t blame yourself. Any difficulties are the result of the design of the Web survey, not your skills or abilities. We are going to use your comments and experiences as well as comments and experiences of other participants to help improve the survey. I did not create the online survey, so don’t feel like you have to hold back on your thoughts to be polite to me. We appreciate your help so we can make the online survey work well for everyone.

Think Aloud: I want you to respond to the web instrument exactly the way you would if you were at home, but with one major difference. I would like you to think aloud as you work within the instrument. I am interested in your answers, but I am also interested in the process you go through in your mind when you answer the questions. I would like you to tell me everything that you are thinking and feeling as you go about answering each question.

Practice Think Aloud

Let’s do a practice question before we start: Please think aloud as you answer the question, how many windows are in your home?

[Probe if they fall silent; if they just give a number ask them to do it again but this time so that you can understand how they came up with the number; after they finish the think-aloud exercise, explain why we are asking them to do this, e.g., if the Census was interested in the number of windows, which we are not, but if we were, and someone was looking at their sliding glass door and wondered aloud that they didn’t know if they should include it or not, them saying those words would clue us in that we needed to fix our question, make it clear that sliding glass doors should be included, etc.

Probes that will be used in study

• Keep talking

• Um-hum?

• What are you thinking?

Great that’s what I want you to do throughout our session. I will remind you to think aloud if you get quiet.

Eye tracking

We are also going to record where you look on the screen as you complete the survey. Now we will do a simple task that will allow the computer to find your eyes. To calibrate your eyes, please follow the dot with your eyes.

[Calibrate the participants’ eyes and make sure you start on google.com]

Interview – Starting with Invitation letter

[Hand participant the mailing materials (invitation letter with their user name/ID info. Make certain to write down the PIN or any information that user may need later in survey]

This is an example of mailing materials you would receive at your home if you were selected for the survey. Imagine that you received this at your home address. Please take a moment to read it. Notice that the address is not your real address. If you were to receive the survey at your home, the mailing materials you would get would have your real address. Since we cannot replicate that for the lab setting, you will have to pretend that this letter came to your address.

That is the only part of the study that is pretend. Please answer the survey questions as they apply to you in your real life.

Laptop/Mobile: Please go ahead and do what the letter says to get started with the survey. Remember that I am interested in what you have to say as you go about answering the survey questions.

{Spanish-only}

[If respondent tries to complete the survey in English without toggling to the Spanish version, say: “Please wait. This survey is available in Spanish. Can you find a way to change the language to Spanish?” Let the respondent attempt to toggle the language, and assist them if needed to ensure they complete the interview in Spanish.]

Interruption – Log out

[STOP Respondent after they entered initial screener data and ask them to pretend that they had to leave the instrument to do something else.]

Now I’d like you to imagine that you have to leave this right where you’re at as something has come up. For example, imagine that you just remembered you need to pick up something from the grocery store. What would you do?

[Once they are exited, then ask them to resume.]

Okay now I’d like you to imagine that you are ready to continue working on the survey. What would you do?

[Interviewer instruction: Note what participant does if they don’t have their PIN. If they cannot enter with verification questions, give the respondent their PIN.]

FAQ Interruption

[Interruption for FAQ or Help. After they have started into the first topical part (Q60 for ECPP (prior to finding and choosing care or Q58 for PFI)]

Okay now I would like you to imagine you had a question about this survey. You wanted to know how much time it will take to complete this survey. What would you do?

Satisfaction Questions

After final SUBMIT is clicked, have participant answer satisfaction questions.

The next task is for you to answer a short questionnaire about your experience completing the survey you just finished.

[Open the satisfaction survey on the laptop and have participant answer the questions.]

Debriefing Questions

Debriefing: Ask all the probes using a PowerPoint slide with those screens. We will skip the screens that the participant did not see.

The next task is to go over some of the screens you saw.

Each screen will have the following probe:

“Please tell me more about your experience as you answered the question on this screen.” (If there were issues you observed and the person forgets, say something like,) “I thought I saw you /heard you say… , please tell me more about what you were thinking.”

¹Additional probes: Find out more about how it works for parents across different kinds of care arrangements including NO care arrangements. Please tell me more about your child care arrangements in your own words and any comments you had about the questions in this section.

Additional Debriefing Questions

Is there anything else you’d like to mention that we haven’t talked about?

Spanish-specific Debriefing Questions on the Translation

Spanish speakers will be debriefed on the exact same screens as the English speakers, including the same generic probes. These screens are provided in English immediately before this section. Additionally, Spanish speakers will be debriefed on additional screens to determine if there are translation issues.

SCREENER – SPANISH: NHES-11BE(S)(01/17/2018)

1. Q1: Instruction to return the survey if not eligible (p.7)

• Probe: Por favor dígame, en sus propias palabras, lo que las instrucciones al lado derecho de de la “opción no” le están pidiendo hacer.

[In your own words, please tell me more about what the instructions placed on the right of the “no” option are asking you to do].

2. Q2: Roster (p.7)

• Probe: ¿Cómo decidió en sus respuesta a estas pregunta? [Tell me more about how you answered this question. ]

3. Q6: Homeschooled, not in school and vocional (p.7)

• [Researcher note: “vocional” is a typo in the paper instrument]

• Probes: ¿Cómo decidió en sus respuesta a estas pregunta? [Tell me more about how you answered this question. ]

• Por favor dígame, en sus propias palabras, lo que significa “homeschooled” en casa.

[In your own words, please tell me what “homeschooled at home” means to you].

• Por favor dígame, en sus propias palabras, lo que significa “escuela vocional.”

[In your own words, please tell me what “vocational school” means to you]

• Por favor dígame, en sus propias palabras, lo que significa “no está en una escuela.”

[In your own words, please tell me what “not in school?” means to you]

4. Q7: Vocional

• [Researcher note: This is a typo in the paper instrument]

• Probes: Por favor dígame, en sus propias palabras, lo que significa “escuela vocional.”

[In your own words, please tell me what “vocational school” means to you]

5. Mailing out

• Probe: Por favor dígame, en sus propias palabras, lo que significa “franqueo prepagado.”

[In your own words, please tell me what “postage paid” means to you]

NHES-ECPP(S) (03/26/2018) Version 1.

1. Instructions to Q1 (p. 42)

• Probe: Por favor dígame, en sus propias palabras, lo que significa “tutores legales.”

[In your own words, please tell me what “legal guardian” means to you]

2. Daycare arrangements: Q18 (p. 45)

• Probe: Por favor dígame, en sus propias palabras, lo que le pide esta pregunta.

[In your own words, please tell me more about what this question is asking you to do].

3. Q27: Provider (p. 46)

• Probes: Por favor dígame, en sus propias palabras, lo que le pide esta pregunta. ? [In your own words, please tell me more about what this question is asking you to do].

• Por favor dígame, en sus propias palabras, lo que significa “proveedor.” [In your own words, please tell me what “provider” means to you]

4. Q39: Regularity (p. 48)

• [Researcher note: Look out for difficulty operationalizing time.]

• Probes: Por favor dígame, en sus propias palabras, lo que le pide esta pregunta. [In your own words, please tell me more about what this question is asking you to do].

• Por favor dígame, en sus propias palabras, lo que significa “con regularidad.” [In your own words, please tell me what “regularidad” means to you]

5. Q46 & Q61: Early Head Start (p. 49 & p. 52)

• Probes: Por favor dígame, en sus propias palabras, lo que le pide esta pregunta. [In your own words, please tell me more about what this question is asking you to do].

• Por favor dígame, en sus propias palabras, lo que significa “Head Start o de Early Head Start.” [In your own words, please tell me what “Head Start o de Early Head Start” means to you]

6. Q66: Lack of vacancies (p. 53)

• Probes: Por favor dígame, en sus propias palabras, lo que le pide esta pregunta. . [In your own words, please tell me more about what this question is asking you to do].

• Por favor dígame, en sus propias palabras, lo que significa “falta de vacancias.” [In your own words, please tell me what “lack of vacancies” means to you]

7. Q69h: Ranking (p. 55)

• Probes: Por favor dígame, en sus propias palabras, lo que le pide esta pregunta. . [In your own words, please tell me more about what this question is asking you to do].

• Por favor dígame, en sus propias palabras, lo que significa “ranking/clasificación.” [In your own words, please tell me what “ranking/classification” means to you]

Q86: Risk. (Si el niño(a) es menor de tres años) ¿Le ha dicho un profesional de la salud, la educación o de intervención temprana que este niño(a) "está en riesgo" de tener un retraso considerable del desarrollo? (p.58)

• [Researcher note: Look out for misunderstandings regarding magnitude of problems]

• Probe: ¿Cómo decidió en sus respuestas a esta pregunta? [Tell me more about how you answered this question].

8. Q88: Services. ¿Está este niño(a) recibiendo servicios de un Plan de Servicio Familiar Individualizado (IFSP), Programa de Educación Individualizado (IEP) o plan de servicios? (p. 59)

• Probe: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

9. Q95: Reporting age as zero. ¿Qué edad tenía este niño(a) cuando se mudó por primera vez a uno de los 50 estados de los Estados Unidos o al Distrito de Columbia? Si fue menor de un año, por favor escriba "0"

• Probes: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

• Por favor dígame, en sus propias palabras, lo que significa “por favor escriba "0."” [In your own words, please tell me what “please write down zero” means to you]

10. Q102. Education in other languages. Actualmente, ¿asiste este niño(a) a clases de inglés como segundo idioma, educación bilingüe o a un programa de inmersión al inglés?

• Probes: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

• Por favor dígame, en sus propias palabras, lo que significa “programa de inmersión al inglés” [In your own words, please tell me what “English immersion program” means to you]

11. Q108. ¿Es este padre o tutor legal de sexo masculino o femenino?

• [Researcher note: There is no indication for mother so not sure how participant will react. Will they only answer male?]

• Probe: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

12. Q120. Time estimation. ¿Más o menos cuántas horas a la semana trabaja este padre o tutor legal normalmente por pagos o ingresos,contando todos los trabajos? (p. 65) and Q138 (p.68)

• Probe: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

13. Q130. Parental language. ¿Qué idioma habla este padre o tutor legal más en el hogar actualmente? (p. 67) and Q101 (p. 135)

• Probes: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

14. 144. Tenure. Este hogar o apartamento…Marque UNO solo.

¿Es propiedad de alguien en el hogar o la está pagando alguien en este hogar?

¿Es alquilada por alguien en este hogar?

¿Es ocupada por algún otro tipo de acuerdo? (p. 70)

• [Researcher note: The translation is different from the source: “Owned or being bought by someone in this household”]

• Probes: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

• Por favor dígame, en sus propias palabras, lo que significa “propiedad de alguien en el hogar o la está pagando alguien en este hogar” [In your own words, please tell me what “owned or somebody at home is paying for it” means to you]

NHES-PFI(S)

15. Q2. School types (p. 113)

• Probe: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered these questions].

16. Q3. Homeschooling (p. 113)

• [Researcher note: There is not consistency in this term. Sometimes they use “homeschool” in English and here the translated term. Need for a copy editor.]

• Probes: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

• Por favor dígame, en sus propias palabras, lo que significa “Educado en casa”? [In your own words, please tell me what “homeschooled” means to you]

17. Q6. Co-op. ¿Recibe este niño(a) alguna parte de su instrucción en un grupo o cooperativa local para la educación en casa? (P. 114)

• Probe: Por favor dígame, en sus propias palabras, lo que significa “grupo o cooperativa local para la educación en casa” [In your own words, please tell me what “owned or somebody at home is paying for it” means to you]

18. Q9. Cyber. ¿Está inscrito este niño(a) en cursos online, virtuales, o cyber? (P. 114)

• Probe: Por favor dígame, en sus propias palabras, lo que significa “cursos online, virtuales, o cyber” [In your own words, please tell me what “online, virtual and cyber courses” mean to you]

19. Q10a. Advanced placement (p. 115)

• Probes: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

• Por favor dígame, en sus propias palabras, lo que significa “Colocación Avanzado”? [In your own words, please tell me what “advancement placement” means to you]

20. Q16. Curriculum (p. 116)

• Probes: ¿Cómo decidió en sus respuestas a estas preguntas? [Tell me more about how you answered this question].

• Por favor dígame, en sus propias palabras, lo que significa “currículo”? ” [In your own words, please tell me what “curriculum” means to you]

21. Q21 Classes (P. 118): 21. Pensando en los niveles típicos de grados, ¿para qué grados recibió este niño(a) educación en casa, al menos para algunas clases o asignaturas?

• [Researcher Note: Inconsistent translation of classes in Spanish: they use “asignaturas”, clases and sometimes “materias” (see q24)]

• Probe: Por favor dígame, en sus propias palabras, lo que significa “asignaturas”?

.

22. Q24 (p.119): ¿En la semana más reciente en la cual este niño(a) fue educado en casa, cuales materias fueron enseñados durante ese periodo?

• Probe: Por favor dígame, en sus propias palabras, lo que significa “materias”? [In your own words, please tell me what “subjects/classes” means to you]

23. Q33: ¿Es esta escuela una escuela especializada (magnet school) o asiste este niño(a) un programa especializada (magnet program)?

• Probe: Por favor dígame, en sus propias palabras, lo que le pide esta pregunta. [Tell me more about how you answered this question].

24. Q38D. STEM and language immersion (P. 121)

• [Researcher Note: Is this wording intentionally left in English?] Probes: Por favor dígame, en sus propias palabras, lo que significa “language immersion”? ”? [In your own words, please tell me what “language immersion” means to you]

• Probes: Por favor dígame, en sus propias palabras, lo que significa “language immersion”? ”? [In your own words, please tell me what “language immersion” means to you]

• Por favor dígame, en sus propias palabras, lo que significa “STEM”? ” [In your own words, please tell me what “STEM” means to you]

25. Q48. ¿Cúal es la cantidad total de la matrícula y cargos para todas las clases online,

virtuales o cyber que toma este niño(a)? (p. 124)

• Probes: Por favor dígame, en sus propias palabras, lo que le pide esta pregunta. [Tell me more about how you answered this question].

• Por favor dígame, en sus propias palabras, lo que significa “matrícula”? [In your own words, please tell me what “tuition” means to you]

26. Q124 (P.139) ¿Tiene esta escuela intérpretes que hablan la lengua materna de este padre o tutor legal para reuniones en la escuela o conferencias entre padres y maestros?

• Por favor dígame, en sus propias palabras, lo que le pide esta pregunta. [Tell me more about how you answered this question].

Finishing up session

[Give the respondent incentive envelope [$60], and thank them for their time and effort. Collect materials and leave or escort respondent out of the interviewing room.]

Okay, that’s all that I have today. Many thanks for your work on this project. The last thing I need you to do is sign this form that says I am giving you $60.

NOTES PAGES

P#___

PFI or ECPP

Characteristic of Participant:

TA:______________________

Date: _____________________

Location: Headquarters / Other :________________________

PC or Device: _____________________

WiFi or Dataplan

Browser type used:_____________________________________

Other browsers on device:________________________________

Demographics:

Description of household (circle):

Child age is less than 5 years old /

Children 5-18 in Public or Private School /

Children 5-18 in Home schooling

Spontaneous comments:

Any spontaneous comments about length of survey, repetitive questions, etc: Where did it occur?

Time in survey – minus time for vignettes

Background and Demographics Questionnaire

1. In what month and year were you born? _________________ __________________

month year

2. What is the highest grade or year of school you have completed?

Less than high school

Completed high school

Some college, no degree (indicate area of study: ____________________________________)

Associate’s degree (AA/AS) (indicate area of study: __________________________________)

Bachelor’s Degree (BA/BS) (indicate area of study: __________________________________)

Post-Bach degree (indicate degree and area of study: ________________________________)

3. Are you male or female?

Male

Female

4. Are you of Hispanic, Latino, or Spanish origin?

Yes

No

5. What is your race? Choose one or more races.

White

Black or African American

American Indian or Alaska Native

Asian

Native Hawaiian or Other Pacific Islander

Questionnaire for User Interaction Satisfaction (QUIS)

Please Note: The questionnaire will be electronic with word labels on either end of 7 radio buttons. See image below for an idea of what the final format will look like.

1. Overall experience with the survey

Unpleasant to Pleasant

2. The survey screens are visually pleasing

Never to Always

3. The survey screens look professional

Never to Always

4. Characters (words/text) on the screen

Barely legible to Very legible

5. The survey questions follow a logical sequence

Never to Always

6. Words used in the survey questions and answers

Hard to understand to Easy to understand

7. Coming up with answers to the questions

Never bothered me to Always bothered me

8. Instructions for correcting errors

Confusing to Clear

9. Entering your answers into the survey

Difficult to Easy

10. You knew what to do if you needed help answering the question

Never to Always

11. Additional Comments__________________________________________________________________

1 At the Census Bureau, participants are asked Spanish-specific screening questions on topics like indigenous heritage (which makes up the majority of racial diversity seen among participants), language proficiency, household composition (related versus unrelated), etc. because the answers provide demographic diversity context for each project. This screener is a typical, approved Census screener (OMB# 0607-0725) used in every project.  When Spanish respondents call a recruiter, Census does not know in advance which project they are calling about and cannot only ask them screening questions related to a particular project, but need to ask the caller this generic screener and then select them the project for which the caller is most suited.

37