Protocol

Protocol for National Survey of Children’s Health Questionnaire Pretesting

Participant ID #: |___|___|___|___|___| ___|___|___|

Interview Date: |__|__| / |__|__| / |__|__|__|__| (mm/dd/yyyy)

Interviewer initials: |__|__|

Start Time: ____________ AM / PM End Time: ____________ AM / PM

Section 1: ** Read/ Paraphrase the following text**
Greeting: Hello. My name is ________________, and I work for the Census Bureau.

Thank you for agreeing to participate in our study.

What: Let me start by telling you a little about what we will be doing today. The National Survey of Children’s Health, is conducted by the Census Bureau and sponsored by the Health Resources and Services Administration. This survey collects information about the health and well-being of America’s children. We’re working to improve the survey for 2020. Our goal is to design a questionnaire that collects high quality data without placing excessive burden on our respondents. Before we finalize the questions in the survey, we want to test them with people like yourself to make sure the questions are easy to understand and answer.

We’re asking you to read through the survey and complete it as you would if you received it in the mail. We are interested in your feedback so we can know what people think of the questions in the survey and how we can potentially improve them. I’m going to ask you to read through and respond to the survey questions, and then ask you some questions about them: what you think certain questions are asking, and what your reactions to them are. There are no right or wrong answers. We are interested in what people think about the survey. Please give me your honest impressions, whether good or bad. We appreciate your help so we can make the survey work well for everyone.

How: **SHOW PARTICIPANT THE SURVEY**

PAPER MODE:
This is a questionnaire from the National Survey on Children’s Health. If this were the real survey, you would either receive this in the mail or receive an invitation to complete the survey online. In a couple of minutes I am going to ask you to complete the survey just as you would if you had received it at home but with one major difference.

WEB MODE:
You’re going to be seeing the web survey for the National Survey on Children’s Health. If this were the real survey, you would have received an invitation in the mail to complete the survey online. Then, you would log onto the survey with a user ID and passcode. In a couple of minutes I am going to ask you to complete the survey just as you would if you had received it at home but with one major difference.

Think Aloud: I would like you to think aloud as you read and answer the questions in the survey. I am interested in your feedback on the questions, but I am also interested in the process you go through in your mind as you come up with answers to the questions in the survey. I would like you to tell me everything that you are thinking and feeling as you come up with your responses to the survey questions. You might have some questions about the survey that come up as we go. You can still ask these questions, but I will wait until the end to answer them. I want to use this time to get your thoughts and opinions.

Practice: Let’s do a practice before we start. Please think aloud as you answer this question: How many windows are in your home?

Interruptions: I might stop you at a couple of points and ask you some questions about your feedback, or about the materials themselves. I am also going to ask you some questions at the end. There are no right or wrong answers, because only you know what you are thinking.

Do you have any questions before we begin?

Confidentiality: Our session today is completely confidential. Your participation in this study is
completely voluntary, and you can decline to answer any particular question.

Recording: So I don’t have to rely on my memory later on, I’d like to record this interview.
That way, I can focus today on what you’re saying rather than having to concentrate on taking notes. Is that ok with you?

**HAVE PARTICIPANT SIGN CONSENT FORM**

**Begin: **TURN ON RECORDER**

OK, let’s begin. Please remember to think aloud as you go through the survey.

Interviewer: Note any confusion or difficulties participants have with the questionnaire.

If participant is not being talkative despite reminders, ask them to underline things they don’t like or find confusing and circle things they do like in the paper questionnaire. For the web survey, ask them to point out or show you anything they do or do not like.

INTERVIEWER: UNLESS LABELED OTHERWISE, A QUESTION’S PRIORITY LEVEL IS “HIGH”

Section 1 – Topical Questionnaires

**VERIFY THE TOPICAL QUESTIONNAIRE BEING TESTED IS CORRECT, BASED ON THE AGE OF THE REFERENCE CHILD FROM RECRUITMENT**

PAPER:

SET-UP: This is the survey you would receive if you were selected to participate in the National Survey of Children’s Health. You would receive this survey in the mail a few weeks after you received, filled out, and mailed back a short screener survey identifying yourself as a household with one or more children aged 0-17. I’d like for you to go ahead and read through and complete the survey as you would if you received it in the mail. Again, please remember to think aloud as you read the questions and answer them.

ASK CHILD’S NAME AND FILL IT IN. EXPLAIN THAT THE CHILD’S NAME WOULD APPEAR IN THE BOX IF THEY WERE RECEIVING THIS IN THE MAIL.

WEB:

SET-UP: If you were selected to participate in the National Survey of Children’s Health, you would receive a letter in the mail with instructions about how to log into this site. I’d like for you to go ahead and read through and complete the survey as you would if I wasn’t here. Again, please remember to think aloud as you read the questions and answer them.

AFTER PARTICIPANT READS INSTRUCTIONS, ASK:

Standard Probes:

1. Who do you think should fill out the survey?

Section A: This Child’s Health

INTERVIEWER: NOTE ANY PARTICIPANT FEEDBACK TO ITEMS ASKING WHETHER THE REFERENCE CHILD CURRENTLY HAS A CONDITION.

A4. (T2 & T3 only) Does this child have any of the following?

a. Serious difficulty concentrating, remembering, or making decisions because of a physical, mental, or emotional condition

Standard Probes:

1. What kind of things did you consider when answering this question?

2. What do you think is meant by “serious difficulty”? IF NEEDED: Can you give me an example?

A11. Heart Condition?

If yes, was this child born with the condition?

Standard Probes:

1. Did you notice that this question asks whether your child was born with the condition rather than whether they currently have the condition like the other questions in this section?

A29. At any time DURING THE PAST 12 MONTHS, did this child receive behavioral treatment for Autism, ASD, Asperger’s Disorder or PDD, such as training or an intervention that you or this child received to help with their behavior?

Standard Probes:

1. What do you think this question is asking?

ASK IF PARTICIPANT SKIPS THE AUTISM/ASD/ASPERGERS/PDD SERIES (A25-A29), BUT “YES” TO QUESTION THE ADD/ADHD SERIES (A30-A32):

A32. At any time DURING THE PAST 12 MONTHS, did this child receive behavioral treatment for ADD or ADHD, such as training or an intervention that you or this child received to help with their behavior?

Standard Probes:

1. What do you think this question is asking?

A33. (A34 in T2 & T3): Do you think this child has EVER had a concussion or brain injury? A concussion or brain injury is when a blow or jolt to the head causes problems such as headaches, dizziness, being dazed or confused, difficulty remembering or concentrating, vomiting, blurred vision, changes in mood or behavior, or being knocked out.

If yes, did you seek medical care from a doctor or other health care provider?

If yes, did a doctor or other health care provider tell you that your child had a concussion or brain injury?

Standard Probes:

1. What do you think about the definition of concussion or brain injury?

IF NEEDED:

2. Do you think it was clear?

3. In your own words, what types of injuries are we asking about?

4. Are there any injuries you thought of but decided not to include in your answer?

A35. (T2 & T3 ONLY): DURING THE PAST 12 MONTHS, how often have this

child’s health conditions or problems affected their ability to do things other children their age do?

This child does not have any health conditions ➔ SKIP to question B1

Never

Sometimes

Usually

Always

IF RESPONDENT REPORTED A SPECIFIC CONDITION IN SECTION A, BUT CHOOSES “THIS CHILD DOES NOT HAVE ANY HEALTH CONDITIONS” FOR A35, ASK:

Standard Probes:

1. Can you tell me a little bit more about your answer?

2. What do you think is meant by “health condition”?

3. What do you think is meant by “things other children their age do”?

INTERVIEWER: FOR QUESTIONS WITHOUT PROBES, NOTE ANY CONFUSION AND PROBE AS NEEDED

Section C: Health Care Services

C1. DURING THE PAST 12 MONTHS, did this child see a doctor, nurse, or other health care professional for sick-child care, well-child check-ups, physical exams, hospitalizations or any other kind of medical care?

INTERVIEWER: NOTE ANY CONFUSION AND PROBE AS NEEDED

Standard Probes:

1. Who would you consider to be a “health care professional”?

C2. (T3 ONLY) If yes, at their LAST medical care visit, did this child have a chance to speak with a doctor or other health care provider privately, without you or another caregiver in the room?

C7 (T1 ONLY)

NOTE IF PARTICIPANT APPROPRIATELY FOLLOWS THE SKIP INSTRUCTION AT C7

C10 & C11: (QUESTION C11 & C12 in T3)

C10. Is there a place that this child USUALLY goes when they need routine preventive care, such as a physical examination or well-child check-up?

C11. If yes, is this the same place this child goes when they are sick?

ASK PROBES AFTER PARTICIPANT ANSWERS C11

Standard Probes:

1. Tell me in your own words what these two questions are asking. INTERVIEWER: NOTE IF PARTICIPANT TALKS ABOUT WHERE HE/SHE TAKES CHILD IN THESE CIRCUMSTANCES, OR ABOUT WHO THE CHILD THEMSELVES COMES TO WHEN NOT FEELING WELL.

2. Who usually takes this child to routine doctor appointments?

Question C14 &C15 (C15 &C16 in T3):

C14. DURING THE PAST 12 MONTHS, did this child see a dentist or other health care provider for any kind of dental or oral health care?

INTERVIEWER: NOTE ANY CONFUSION AND PROBE AS NEEDED

Standard Probes:

1. When answering this question, did you consider any health care providers other than a dentist?

C15. If yes, DURING THE PAST 12 MONTHS, did this child see a dentist or other health care provider for PREVENTIVE dental care, such as check-ups, dental cleanings, dental sealants, or fluoride treatments?

C19. DURING THE PAST 12 MONTHS, has this child taken any medication because of difficulties with their emotions, concentration, or behavior?

INTERVIEWER: NOTE ANY CONFUSION AND PROBE AS NEEDED

IF NEEDED FOR TIME: SKIP TO SECTION D

INTERVIEWER: FOR QUESTIONS WITHOUT PROBES, NOTE ANY CONFUSION AND PROBE AS NEEDED

Section D: Experience with This Child’s Health Care Providers

D1. Do you have one or more persons you think of as this child’s personal doctor or nurse? A personal doctor or nurse is a health professional who knows this child well and is familiar with this child’s health history. This can be a general doctor, a pediatrician, a specialist doctor, a nurse practitioner, or a physician’s assistant.

Standard Probes:

1. In your own words, what is this question asking?

2. What do you think of the description of “personal doctor”?

D2. DURING THE PAST 12 MONTHS, did this child need a referral to see any doctors or receive any services?

Standard Probes:

1. What do you think is meant by “need a referral”?

D3. How difficult was it to get referrals?

Standard Probes:

1. How easy or hard was it for you to come up with your answer? IF NEEDED: Why?

D4. DURING THE PAST 12 MONTHS, how often did this child’s doctors or other health care providers...

1. Spend enough time with this child?

2. Listen carefully to you?

3. Show sensitivity to your family’s values and customs?

4. Provide the specific information you needed concerning this child?

5. Help you feel like a partner in this child’s care?

D8. DURING THE PAST 12 MONTHS, have you felt that you could have used extra help arranging or coordinating this child’s care among the different health care providers or services?

Standard Probes:

1. What were you thinking of when you were answering this question?

D9. If yes, DURING THE PAST 12 MONTHS, how often did you get as much help as you wanted with arranging or coordinating this child’s health care?

D10. DURING THE PAST 12 MONTHS, how satisfied were you with the communication between this child’s doctors and other health care providers?

D11. DURING THE PAST 12 MONTHS, did this child’s health care provider communicate with the child’s school, child care provider, or special education program?

D12. If yes, during this time, how satisfied were you with the health care provider’s communication with the school, child care provider, or special education program?

D13-D21 (T3 ONLY)

D13. Do any of this child’s doctors or other health care providers treat only children?

D14. If yes, have they talked with you about when this child will need to see doctors or other health care providers who treat adults?

D15. Has this child’s doctor or other health care provider actively worked with this child to:

1. Make positive choices about their health. For example, by eating healthy, getting regular exercise, not using tobacco, alcohol or other drugs, or delaying sexual activity?

2. Gain skills to manage their health and health care. For example, by understanding current health needs, knowing what to do in a medical emergency, or taking medications they may need?

3. Understand the changes in health care that happen at age 18. For example, by understanding changes in privacy, consent, access to information, or decision-making?

D16. Did you and this child receive a summary of your child’s medical history (for example, medical conditions, allergies, medications, immunizations)?

Standard Probe:

1. What do you think this question is asking?

D17. Have this child’s doctors or other health care providers worked with you and this child to create a plan of care to meet their health goals and needs?

Standard Probe:

1. What does a “plan of care” mean to you?

D18. If yes, do you and this child have access to this plan of care?

D19. Does this plan of care address transition to doctors and other health care providers who treat adults?

D20. Eligibility for health insurance often changes in young adulthood. Do you know how this child will be insured as they become an adult?

D21. If no, has anyone discussed with you how to obtain or keep some type of health insurance coverage as this child becomes an adult?

IF THERE IS TIME TO ADMINISTER SECTION E:

Section E: This Child’s Health Insurance Coverage

E7. Thinking specifically about this child’s mental or behavioral health needs, how often does this child’s health insurance offer benefits or cover services that meet these needs?

Standard Probe:

1. What do you think this question is asking?

IF THERE IS TIME TO ADMINISTER SECTION F:

Section F: Providing for This Child’s Health

F1. Including co-pays and amounts reimbursed from Health Savings Accounts (HSA) and Flexible Spending Accounts (FSA), how much money did you pay for this child’s medical, health, dental, and vision care DURING THE PAST 12 MONTHS? Do not include health

insurance premiums or costs that were or will be reimbursed by insurance or another source.

Standard Probe:

1. What do you think this question is asking?

2. What costs did you consider for your answer?

INTERVIEWER: FOR QUESTIONS WITHOUT PROBES, NOTE ANY CONFUSION AND PROBE AS NEEDED

Section G: This Child’s Learning

Questions G8 – G10 (T2 and T3 ONLY)

INTERVIEWER INSTRUCTIONS:

THESE ITEMS WERE MOVED FROM SECTION A. NOTE ANY CONFUSION THAT MIGHT INDICATE THAT THE RESPONDENT FINDS THE CONTEXT OR FLOW OF QUESTIONS UNNATURAL AT THIS POINT.

G8. DURING THE PAST 12 MONTHS, how often was this child bullied, picked on, or excluded by other children? If the frequency changed throughout the year, report the

highest frequency.

G9. DURING THE PAST 12 MONTHS, how often did this child bully others, pick on them, or exclude them? If the frequency changed throughout the year, report the highest frequency.

Standard Probe:

1. How easy or hard was it for you to answer these questions?

2. How certain are you of your answers? IF NEEDED: If I weren’t here would you answer this question? IF SO: how?

G10. How often does this child...

1. Show interest and curiosity in learning new things?

2. Work to finish tasks they start?

3. Stay calm and in control when faced with a challenge?

4. Care about doing well in school?

5. Do all required homework?

6. Argue too much?

INTERIVEWER: NOTE ANY CONFUSION OR FEEDBACK

Section G: This Child’s Learning (T1 ONLY)

INTERVIEWER: NOTE IF THE PARTICIPANT FOUND THE INSTRUCTIONS ABOVE G1 CONFUSING AND PROBE AS NECESSARY

Question G1: Is this child able to do the following...

1. Say at least one word, such as "hi" or "dog"?

2. Use 2 words together, such as "car go"?

3. Use 3 words together in a sentence, such as, "Mommy come now."?

4. Ask questions like "who," "what," "when," "where"?

5. Ask questions like "why" and "how"?

6. Tell a story with a beginning, middle, and end?

7. Understand the meaning of the word "no"?

8. Follow a verbal direction without hand gestures, such as "Wash your hands."?

9. Point to things in a book when asked?

10. Follow 2-step directions, such as "Get your shoes and put them in the basket."?

11. Understand words such as "in," "on," and "under"?

INTERVIEWER: PROBE AS NEEDED

G4. Are you concerned about how this child is learning to do things for themselves?

G5. How confident are you that this child is ready to be in school?

G6: How often can this child recognize the beginning sound of a word? For example, can this child tell you that the word “ball” starts with the “buh” sound?

INTERVIEWER: MEDIUM PRIORITY

Standard Probe:

1. What were you thinking about when you answered this question?

2. Were you thinking specifically about the ‘B’ sound?

3. Was the example helpful?

G7. About how many letters of the alphabet can this child recognize?

INTERVIEWER: MEDIUM PRIORITY

NOTE ANY CONFUSION WITH THIS QUESTION

Standard Probe:

1. How does your child show you they recognize letters?

G8. Can this child rhyme words?

Standard Probe:

1. Can you tell me what you were thinking about when you answered this question?

G9. How often can this child explain things they have seen or done so that you get a very good idea what happened?

INTERVIEWER: LOW PRIORITY

Standard Probe:

1. In your own words, what is this question asking?

2. How did you arrive at your answer?

G10. How often can this child write their first name, even if some of the letters aren’t quite right or are backwards?

INTERVIEWER: HIGH PRIORITY

Standard Probe:

1. Was this question difficult to answer? IF NEEDED: Why?

G11. How high can this child count?

INTERVIEWER: HIGH PRIORITY

Standard Probe:

1. Were you thinking of your child counting objects they see, reciting numbers, or both?

2. How accurate are they? Did you consider accuracy when answering?

G12. How often can this child identify basic shapes such as a triangle, circle, or square?

INTERVIEWER: LOW PRIORITY

Standard Probe:

1. What do you think is meant by “identifying a shape”?

G13. Can this child identify the colors red, yellow, blue, and green by name?

INTERVIEWER: LOW PRIORITY

Standard Probe:

1. Can you tell me in your own words what these response options mean?

2. What does it mean to you to identify each color by name?

G14. How often is this child easily distracted?

INTERVIEWER: HIGH PRIORITY

Standard Probe:

1. What does “easily distracted” mean to you?

G15. How often does this child keep working at something until they are finished?

INTERVIEWER: MEDIUM PRIORITY

Standard Probe:

1. What types of activities were you thinking about?

G16. When this child is paying attention, how often can they follow instructions to complete a simple task?

INTERVIEWER: ANY CONFUSION OR UNCERTAINTY; PROBE AS NEEDED

G17. How does this child usually hold a pencil?

G18. How often does this child play well with others?

INTERVIEWER: MEDIUM PRIORITY

Standard Probe:

1. What does ‘play well’ mean to you?

2. Who do you think is meant by ‘others’?

G19. How often does this child become angry or anxious when going from one activity to another?

INTERVIEWER: LOW PRIORITY

Standard Probe:

1. Were you thinking of the child becoming angry, anxious, or both?

2. What does it mean to you to “go from one activity to another?”

G20. How often does this child show concern when others are hurt or unhappy?

INTERVIEWER: LOW PRIORITY

Standard Probe:

1. What were you thinking as you answered the question?

2. What does “show concern” mean to you?

G21. When excited or all wound up, how often can this child calm down quickly?

INTERVIEWER: HIGH PRIORITY

Standard Probe:

1. Tell me in your own words what ‘all wound up’ means to you. IF NEEDED: Does it mean something different from excited?

2. What does ‘calm down’ mean to you?

G22. How often does this child lose control of their temper when things do not go their way?

INTERVIEWER: LOW PRIORITY

Standard Probe:

1. What does ‘lose control of their temper’ mean to you?

2. What were you thinking about when you answered this question?

G23. Compared to other children their age, how much difficulty does this child have making or keeping friends?

INTERVIEWER: MEDIUM PRIORITY

Standard Probe:

1. Can you tell me about how you decided on your answer? IF NEEDED: What other children were you thinking about?

2. What does ‘making or keeping friends’ mean to you?

G24. Compared to other children their age, how often is this child able to sit still?

INTERVIEWER: HIGH PRIORITY

Standard Probes:

1. How confident are you in your answer? IF NEEDED: What other children are you thinking about?

2. What does ‘sit still’ mean to you?

G25. How often...

INTERVIEWER: HIGH PRIORITY

…

b. Does this child bounce back quickly when things do not go their way?

Standard Probes:

1. What does it mean to “bounce back quickly”?

2. Did you have any trouble choosing a response? Why or why not?

Section H: About You and This Child

Question H6 (T1 ONLY)

Answer the next question only if this child is LESS THAN 12 MONTHS OLD. Otherwise, SKIP to question .

NOTE IF THE PARTICIPANT HAD TROUBLE WITH THE SKIP INSTRUCTION

Section I: About Your Family and Household

Question I12i: (T2 & T3 ONLY)

I12. To the best of your knowledge, has this child EVER experienced any of the following?

9. Treated or judged unfairly because of their sexual orientation or gender identity?

Standard Probes:

1. What do you think is meant by “treated or judged unfairly”?

2. What do you think is meant by “sexual orientation”?

3. What do you think is meant by “gender identity”?

Section J: About You

J13. Does this child have another primary adult caregiver who lives in this household?

INTERVIEWER: NOTE IF PARTICIPANT HAD DIFFICULTY WITH THE SKIP INSTRUCTIONS

Standard Probes:

1. What do you think is meant by “primary adult caregiver”?

2. IF NEEDED: Here’s another way we might ask this question INTERVIEWER: HAND PARTICIPANT ALTERNATE VERSION, WHICH READS: “Does this child have another ‘parent or adult caregiver’ who lives in the household?” Would that make this question easier or more difficult to answer?

3. Are there other adults living in the household?

4. IF NEEDED:How did you decide who to list as Caregiver 2?

Section K: Household Information

Set-Up: This survey will happen in 2020, so this section will ask about the previous year. For our purposes today, lets think of 2018 when answering questions about income.

Question K3f.: Any other sources of income received such as Veterans’ (VA) payments, unemployment compensation, child support, alimony, gifts, prize winnings, etc.

Standard Probes: …

INTERVIEWER: NOTE ANY CONFUSION ABOUT HOW THIS QUESTION DIFFERS FROM THE PREVIOUS SERIES.

1. Was there any income that your family received in 2018 that you did not include?

_____________________________________________________________________________

GENERAL DEBRIEFING PROBES:

Overall, what would you say about the survey you looked at today? If needed: Tell me more about that.

Do you think there are questions some people would find confusing? Which ones?

Were there any questions that you found sensitive? (IF YES) Were there any questions you answered today that you might not feel comfortable answering if you received this survey in the mail? (IF YES) Would you continue the survey?

IF PRONOUN USE HAS NOT YET BEEN MENTIONED BY THE RESPONDENT: Did you notice that we did not use the words “he or she” or “his or her” in the survey and referred to your child as “they” or “their”?

What did you think about that?

IF NEEDED: Was it confusing at all?

IF PARTICIPANT DID NOT PROVIDE FEEDBACK ON SEXUAL ORIENTATION GENDER IDENTITY ITEMS: Some of the questions asked about your child’s experiences based on their sexual orientation or gender identity. How do you feel about that?

Is there anything else you would like to tell us that you haven’t already mentioned?

______________________________________________________________________________

Thank you for your feedback today. Your participation is greatly appreciated and your input has been really helpful.

**PROVIDE INCENTIVE AND HAVE PARTICIPANT SIGN VOUCHER FORM**

J13. Does this child have another parent or adult caregiver who lives in the household?

Yes

No

36