Protocol

Protocol for National Survey of Children’s Health Questionnaire Pretesting

Participant ID #: |___|___|___|___|___| ___|___|___|

Interview Date: |__|__| / |__|__| / |__|__|__|__| (mm/dd/yyyy)

Topical Questionnaire Being Tested (T1, T2, or T3): |__|__|

Interviewer initials: |__|__|

Start Time: ____________ AM / PM End Time: ____________ AM / PM

Section 1: ** Read/ Paraphrase the following text**
Greeting: Hello. My name is ________________, and I work for the Census Bureau.

Thank you for agreeing to participate in our study.

What: Let me start by telling you a little about what we will be doing today. The National Survey of Children’s Health (NSCH), is conducted by the Census Bureau and sponsored by the Health Resources and Services Administration (HRSA). This survey collects information about the health and well-being of America’s children. We’re working to improve the survey for 2018. Our goal is to design a questionnaire that collects high quality data without placing excessive burden on our respondents. Before we finalize the questions in the survey, we want to test them with people like yourself to make sure the questions are easy to understand and answer.

We’re asking you to read through the survey and complete it as you would if you received it in the mail. We are interested in your feedback so we can know what people think of the questions in the survey and how we can potentially improve them. We will be looking at two items. I’ll explain each one to you as we go along. I’m going to ask you to read through and respond to the survey questions, and then ask you some questions about them: what you think certain questions are asking, and what your reactions to them are. There are no right or wrong answers. We are interested in what people think about the survey you will experience today. Please give me your honest impressions, whether good or bad. We appreciate your help so we can make the survey work well for everyone.

How: **SHOW PARTICIPANT THE STACK OF MATERIALS**

These are two questionnaires from the National Survey on Children’s Health. If this were the real survey, you would either receive these in the mail or receive an invitation to complete the surveys online. In a couple of minutes I am going to ask you to look at these just as you would if you had received them at home but with one major difference.

Think Aloud: I would like you to think aloud as you read and answer the questions in the survey. I am interested in your feedback on the questions, but I am also interested in the process you go through in your mind as you come up with answers to the questions in the survey. I would like you to tell me everything that you are thinking and feeling as you read come up with your responses to the survey questions. You might have some questions about the survey that come up as we go. You can still ask these questions, but I will wait until the end to answer them. I want to use this time to get your thoughts and opinions.

Practice: Let’s do a practice before we start. Please think aloud as you answer this question: How many windows are in your home?

Interruptions: I might stop you at a couple of points and ask you some questions about your feedback, or about the materials themselves. I am also going to ask you some questions at the end. There are no right or wrong answers, because only you know what you are thinking.

Do you have any questions before we begin?

Confidentiality: Our session today is completely confidential. Your participation in this study is
completely voluntary, and you can decline to answer any particular question.

Recording: So I don’t have to rely on my memory later on, I’d like to record this interview.
That way, I can focus today on what you’re saying rather than having to concentrate on taking notes. Is that ok with you?

**HAVE PARTICIPANT SIGN CONSENT FORM**

**Begin: **TURN ON RECORDER**

OK, let’s begin. Please remember to think aloud as you go through the survey.

Interviewer: Note any confusion or difficulties participants have with the questionnaires.

If participant is not being talkative despite reminders, ask them to underline things they don’t like or find confusing in the questionnaires and circle things they do like.

______________________________________________________________________________

Section 1 – NSCH S1 Screener Survey

**HAND THE PARTICIPANT THE SCREENER SURVEY**

SET-UP: This is the first survey you would receive if you were selected to participate in the NSCH. I’d like for you to go ahead and read through and complete the survey as you would if you received it in the mail. Please remember to think aloud as you read the questions and answer them.

“START HERE”

INTERVIEWER: NOTE WHETHER THE PARTICIPANT READ THE TEXT UNDER THIS SECTION.

FOCUS THE PARTICPANT ON THE SECOND SENTENCE.

Standard Probes:

1. In your own words, can you tell me what is meant by “please have an adult who is familiar with their health and health care answer all of the questions that apply.”?

2. In your household, who takes the children to healthcare appointments?

“CHILD 1”

Question 8. Does this child need or use more medical care, mental health, or educational services than is usual for most children of the same age?

Standard Probes:

1. In your own words, can you tell me what is meant by “than is usual for most children of the same age”?

2. How would you make this comparison?

3. How confident are you in your understanding of what is usual for children of this age? Why or Why not?

Question 11. Does this child have any kind of emotional, developmental, or behavioral problem for which he or she needs treatment or counseling?

Standard Probes:

4. Can you tell me in your own words what this question is asking?

**HAND THE PARTICIPANT THE ALTERNATIVE WORDING EXAMPLE**

5. ALTERNATIVE VERSION: Does this child have any kind of emotional, developmental, or behavioral problem for which he or she needs or gets treatment or counseling?

Standard Probes:

1. Do you feel that this question is asking about the same thing is the previous version or are they asking about different things?

INTERVIEWER: NOTE WHETHER THE PARTICIPANT NOTICED “OR GETS”

2. Would your response to this question change if the phrase “or gets” had been included in the question?

______________________________________________________________________________

Section 2 – Topical Questionnaires

**VERIFY THE TOPICAL QUESTIONNAIRE BEING TESTED IS CORRECT, BASED ON THE AGE OF THE REFERNCE CHILD FROM RECRUITMENT**

**TRANSCRIBE THE FIRST NAME OF THE CHILD FROM THE SCREENER SURVEY ONTO THE TOPICAL QUESTIONNAIRE – IF THERE IS MORE THAN ONE CHILD LISTED ON THE SCREENER SURVEY, TRANSCRIBE THE FIRST NAME OF A CHILD IN THE AGE RANGE OF THE TOPICAL QUESTIONNAIRE BEING TESTED**

SET-UP: This is the second survey you would receive if you were selected to participate in the NSCH. You would receive this survey in mail a few weeks after you mailed back the survey you just completed. I’d like for you to go ahead and read through and complete the survey as you would if you received it in the mail. Again, please remember to think aloud as you read the questions and answer them.

Section A: This Child’s Health

INTERVIEWER: NOTE ANY PARTICIPANT FEEDBACK TO ITEMS ASKING WHETHER THE REFERENCE CHILD CURRENTLY HAS A CONDITION.

Question A21: Other genetic or inherited condition?

Standard Probes:

1. How confident are you about your answer?

Question A23: Developmental Delay? (QUESTION A24 IN T2 & T3)

Standard Probes:

1. How would you define “developmental delay”?

Question A26: Learning disability? (QUESTION A27 IN T2 & T3)

Standard Probes:

1. What conditions do you think of as learning disabilities?

INTERVIEWER: NOTE ANY CONDITIONS THAT THE PARTICIPANT INCLUDES

Question A27: Has a doctor or other health care provider EVER told you that this child has... Any other mental health condition? (QUESTION A28 IN T2 & T3)

INTERVIEWER: NOTE ANY CONFUSION OR QUESTIONS THE PARTICIPANT MAY ASK.

INTERVIEWER: NOTE ANY CONDITIONS THE PARTICIPANT MAY INCLUDE.

Question A29: How old was this child when doctor or other health care provider FIRST told you that he or she had Autism, ASD, Asperger’s Disorder or PDD?

Standard Probes:

1. UNLESS PARTICIPANT CHECKS THE “DON’T KNOW” BOX, ASK: How confident are you about your answer?

Question A36: DURING THE PAST 12 MONTHS, how often have this child’s health conditions or problems affected his or her ability to do things other children his or her age do? (QUESTION A37 IN T2 & T3)

Standard Probes:

1. What do you think is meant by “ability to do things other children do”

2. What sort of things did you consider when reading the question?

Question A37: To what extent do this child’s health conditions or problems affect his or her ability to do things? (QUESTION A38 IN T2 & T3)

Standard Probes:

1. What do you think is meant by “to what extent”

2. Please tell me how you decided on your response to this survey question?

Section B: This Child as an Infant

Question B1: Was this child born more than 3 weeks before his or her due date? (T2 & T3 ONLY)

Standard Probes:

1. How confident are you in your answer?

Question B2: How much did he or she weigh when born? (T2 & T3 ONLY)

INTERVIEWER: NOTE ANY DIFFICULTY THE PARTICIPANT HAD ANSWERING THE QUESTION/REMEMBERING THE INFORMATION.

Standard Probes:

1. (IF NEEDED) How confident are you in your answer?

Question B3: What was the age of the mother when this child was born? (T2 & T3 ONLY)

INTERVIEWER: NOTE ANY DIFFICULTY THE PARTICIPANT HAD ANSWERING THE QUESTION/REMEMBERING THE INFORMATION.

Standard Probes:

1. (IF NEEDED) How confident are you in your answer?

Section C: Health Care Services

Question C8. DURING THE PAST 12 MONTHS, did a doctor or other health care provider have you or another caregiver fill out a questionnaire about specific concerns or observations you may have about this child’s development, communication, or social behaviors? Sometimes a child’s doctor or other health care provider will ask a parent to do this at home or during a child’s visit. (T1 ONLY)

INTERVIEWER: PAY ATTENTION TO THE WAY THE PARTICIPANT READS AND RESPONDS TO THIS QUESTION. IF PARTICIPANT PAUSES, RE-READS, OR DISPLAYS ANY SINGS OF CONFUSION OR UNCERTAINTY, PROBE ABOUT WHAT HE/SHE MAY BE THINKING.

Questions C9 & C10: (QUESTIONS C7 & C8 IN T2; C8 & C9 IN T3)

C9: Is there a place that this child USUALLY goes when he or she is sick or you or another caregiver needs advice about his or her health?

C10. If yes, where does this child USUALLY go first? Mark (X) ONE box.

ASK PROBES AFTER PARTICIPANT ANSWERS C10

Standard Probes:

1. Tell me in your own words what these two questions are asking. INTERVIEWER: NOTE IF PARTICIPANT TALKS ABOUT WHERE HE/SHE TAKES CHILD IN THESE CIRCUMSTANCES, OR ABOUT WHO THE CHILD THEMSELVES COMES TO WHEN NOT FEELING WELL.

2. Who usually takes child to doctor appointments?

Question C23. DURING THE PAST 12 MONTHS, did this child use any type of alternative health care or treatment? Alternative health care can include acupuncture, chiropractic care, relaxation therapies, herbal supplements, and others. Some therapies involve seeing a health care provider, while others can be done on your own. (QUESTION C21 IN T2; QUESTION C22 IN T3)

Standard Probes:

1. What type of treatment or therapy were you thinking of?

INTERVIEWER: NOTE ANY TREATMENTS OR THERAPIES NOT REFERENCED IN THE QUESTION EXAMPLES.

Questions C24 – C27:

(QUESTIONS C22-C25 IN T2; C23-C26 IN T3)

C24. DURING THE PAST 12 MONTHS, was there any time when this child needed health care but it was not received? By health care, we mean medical care as well as other kinds of care like dental care, vision care, and mental health services.

C25. If yes, which types of care were not received? Mark (X) ALL that apply.

C26. Did any of the following reasons contribute to this child not receiving needed health services? Mark (X) Yes or No for each item.

C27. DURING THE PAST 12 MONTHS, how often were you frustrated in your efforts to get services for this child?

Standard Probes:

1. Was there a specific event or circumstance that you were thinking of when answering C24 – C27?

IF YES, ASK: tell me about it

IF NEEDED: Was it a chronic situation or a single episode/special circumstance?

Section D: Experience with This Child’s Health Care Providers

Question D1: Do you have one or more persons you think of as this child’s personal doctor or nurse? A personal doctor or nurse is a health professional who knows this child well and is familiar with this child’s health history. This can be a general doctor, a pediatrician, a specialist doctor, a nurse practitioner, or a physician’s assistant.

Standard Probes:

1. IF PARTICIPANT CHOOSES “Yes, more than one person”, ASK: In your own words, what is this question asking?

2. What do you think is meant by “personal doctor”?

Question D4: Answer the following questions only if this child had a health care visit IN THE PAST 12 MONTHS.

INTERVIEWER: NOTE WHETHER THE PARTICIPANT SEEMS CONFUSED BY THE INSTRUCTION, WHICH QUESTION THEY SKIP TO IF THE CHILD HAD NO VISIT IN THE 12 MONTHS AND WHETHER THEY ASK FOR HELP WITH NAVIGATION.

Standard Probes:

1. (IF NEEDED) Point to text “Answer the following questions only if this child had a health care visit IN THE PAST 12 MONTHS.” Was this instruction confusing at all?

Question D5: DURING THE PAST 12 MONTHS, did this child need any decisions to be made regarding his or her health care, such as whether to get prescriptions, referrals, or procedures?

Standard Probes:

1. In your own words, what is this question asking?

2. What do you think is meant by “need any decisions to be made”?

Questions D8 & D9:

D8. DURING THE PAST 12 MONTHS, have you felt that you could have used extra help arranging or coordinating this child’s care among the different health care providers or services?

D9. If yes, DURING THE PAST 12 MONTHS, how often did you get as much help as you wanted with arranging or coordinating this child’s health care?

Standard Probes:

1. What types of tasks were you thinking of when you were answering these two questions?

2. Who were you thinking of when you thought about someone that could have helped you?

Section E: This Child’s Health Insurance Coverage

Question E2: Indicate whether any of the following is a reason this child was not covered by health insurance at any time DURING THE PAST 12 MONTHS:

Standard Probes:

1. Can you tell me in your own words what this question is asking?

INTERVIEWER: WE ARE TRYING TO ESTABLISH WHETHER IT IS CLEAR TO THE PARTICIPANT THAT THE QUESTION IS TRYING TO ASK IF THE CHILD WAS UNINSURED FOR ANY LENGTH OF TIME DURING THE PRIOR 12 MONTHES – NOT THAT THE CHILD WAS UNINSURED FOR THE ENTIRETY OF THE PRIOR 12 MONTHS.

2. IF NEEDED: What do you think is meant by “not covered at any time”

Question E4: Is this child covered by any of the following types of health insurance or health coverage plans? Mark (X) Yes or No for EACH item.

INTERVIEWER: NOTE IF PARTICIPANT FOLLOWS THE DIRECTIONS TO MARK “NO” FOR ALL TYPES OF PLANS THAT ARE NOT APPLICABLE RATHER THAN LEAVING THEM BLANK

Questions E5 & E6:

E5. How often does this child’s health insurance offer benefits or cover services that meet this child’s needs?

E6. How often does this child’s health insurance allow him or her to see the health care providers he or she needs?

Standard Probes:

1. In your own words, what is the difference between question E5 and E6?

2. Were there specific experiences that you based your answers to these two questions on?

Section F: Providing for This Child’s Health

Question F1: Including co-pays and amounts from Health Savings Accounts (HSA) and Flexible Spending Accounts (FSA), how much money did you pay for this child’s medical, health, dental, and vision care DURING THE PAST 12 MONTHS? Do not include health insurance premiums or costs that were or will be reimbursed by insurance or another source.

Standard Probes:

1. Did you include your deductible when calculating your response to this question?

2. Which costs were you considering?

3. IF NOT CLEAR FROM PARTICIPANT’S ANSWER TO THE ABOVE, ASK: Were you considering the amounts billed by providers or the amounts you personally paid?

Question F2: How often are these costs reasonable?

Standard Probes:

1. Can you tell me what you think is meant by “reasonable”?

2. Which costs were you considering when answering this question?

INTERVIEWER: NOTE WHETHER THE PARTICIPANT ONLY INCLUDED AMOUNTS HE/SHE PAID OR INCLUDED COSTS COVERED BY INSURANCE.

Question F4: DURING THE PAST 12 MONTHS, have you or other family members:

Standard Probes:

1. IF PARTICIPANT ANSWERS “Yes” TO “A. Stopped working because of this child’s health or health conditions?” ASK: Can you tell me about the situation you were referencing when you chose your answer to this question?

2. IF NOT EXPLICITLY CLEAR FROM PARTICIPANT’S ANSWER TO FIRST PROBE, ASK: Were you thinking about long-term/permanent changes to your work situation? Or something like taking days off and/or working from home more than usual for a period of time?

Section G: This Child’s Schooling and Activities

Question G5: How often can this child recognize the beginning sound of a word? For example, can this child tell you that the word “ball” starts with the “buh” sound? (T1 ONLY)

INTERVIEWER: NOTE ANY CONFUSION WITH THIS QUESTION

Standard Probes:

1. What do you think of this question?

Questions G13 & G14:

G13. How often is this child easily distracted? (T1 ONLY)

G14. How often does this child keep working at something until he or she is finished? (T1 ONLY)

INTERVIEWER: NOTE ANY UNPROMPTED COMMENTS ON THESE TWO QUESTION AND PROBE ON THEM

Questions G22 & G23:

G22. Compared to other children his or her age, how much difficulty does this child have making or keeping friends? (QUESTION G7 IN T2 AND T3)

G23. Compared to other children his or her age, how often is this child able to sit still? (T1 ONLY)

Standard Probes:

1. How confident are you in your answer?

2. How confident are you in your knowledge about how well other kids this age group do these things? Why or why not?

Section H: About You and This Child

Question H4: How often does this child go to bed at about the same time on weeknights?

INTERVIEWER: NOTE ANY PARTICIPANT REACTIONS/CONFUSION

DID PARTICIPANT MENTION “BEDTIME”?

Standard Probes:

1. Were you thinking about when you start a bedtime routine, when the child is in bed, or when the child is asleep?

Question H6: Answer the next question only if this child is LESS THAN 12 MONTHS OLD. Otherwise, SKIP to question H7. (T1 ONLY)

INTERVIEWER: DID PARTICIPANT REREAD THE INSTRUCTION?

Standard Probes:

1. Do you think the wording for this question’s instructions could be more clear?

Question H8: ON AN AVERAGE WEEKDAY, about how much time does this child usually spend with computers, cell phones, handheld video games, and other electronic devices, doing things other than schoolwork? (QUESTION H7 IN T2 AND T3)

INTERVIEWER: NOTE ANY DIFFICULY PARTICIPANT HAS ANSWERING THIS QUESTION

Standard Probes:

1. What do you think is meant by “doing things other than schoolwork”?

Question H11: How well do you think you are handling the day-to-day demands of raising children? (QUESTION H9 IN T2 AND T3)

INTERVIEWER: NOTE ANY QUESTIONS THAT PARTICIPANT MAY ASK

Standard Probes:

1. What do you think is meant by “handling the day-to-day demands”?

INTERVIEWER: NOTE WHETHER PARTICIPANT REFERRED TO HIM/HERSELF OR INTERPRETED THE QUESTION TO BE ASKING HOW THOROUGHLY HE/SHE IS MEETING THE CHILD’S NEEDS.

Section I: About Your Family and Household

Question I4: DURING THE PAST 12 MONTHS, how often were pesticides used inside your residence to control for insects? If the frequency changed throughout the year, report the highest frequency.

Standard Probes:

1. How confident are you in your answer?

2. Do you live in a single or multi-unit home?

3. Did the instruction “If the frequency changed throughout the year, report the highest frequency“ make the question easier or harder to answer?

Question I12: The next questions are about events that may have happened during this child’s life. These things can happen in any family, but some people may feel uncomfortable with these questions. You may skip any questions you do not want to answer. To the best of your knowledge, has this child EVER experienced any of the following? (QUESTION I13 IN T2 AND T3)

NOTE WHETHER P READS ENTIRE TEXT THEN:

** INSTRUCT PARTICIPANT TO SKIP WITHOUT ANSWERING AND MOVE ON TO J1.**

Section J: About You

Complete the questions for each of the two adults in the household who are this child’s primary

caregivers. If there is just one adult, provide answers for that adult.

J1. How are you related to this child?

Standard Probes:

1. Who are you including when you answer this question?

Section K: Household Information

Question K3: Income in 2016

Mark (X) the "Yes" box for each type of income this child’s family received, and give your best estimate of the TOTAL AMOUNT IN THE LAST CALENDAR YEAR. Mark (X) the “No” box to show types of income NOT received.

Standard Probes: …

INTERVIEWER: AFTER PARTICIPANT FINISHES PARTS A. THRU F. ASK:

1. Who did you include income for when you answered this series of questions?

Question K4: The following question is about your 2016 income. Think about your total combined family income IN THE LAST CALENDAR YEAR for all members of the family. What is that amount before taxes? Include money from jobs, child support, social security, retirement income, unemployment payments, public assistance, and so forth. Also, include income from interest, dividends, net income

from businesses, farm, or rent, and any other money income received.

Standard Probes: …

INTERVIEWER: NOTE ANY CONFUSION THAT MAY INDICATE CONFUSION ABOUT HOW THIS QUESTION DIFFERS FROM THE PREVIOUS SERIES.

1. Whose income did you include when you answered this question?

______________________________________________________________________________

GENERAL DEBRIEFING PROBES:

Overall, what would you say about the surveys you looked at today? Tell me more about that.

Do you think most people would find the surveys interesting?

Why/Why not?

Do you think there are questions some people would find confusing? Which ones?

Were there any questions that you found sensitive? (IF YES) If you were taking the survey would you answer those questions?

Do you think there are questions that people would find confusing? Which ones?

Is there anything else you would like to tell us that you haven’t had a chance to mention yet?

______________________________________________________________________________

Thank you for your feedback today. Your participation is greatly appreciated and your input has been really helpful.

**PROVIDE INCENTIVE AND HAVE PARTICIPANT SIGN VOUCHER FORM**

19