TO: Brenda Aguilar, Office of Information and Regulatory Affairs (OIRA); Office of Management and Budget (OMB)
FROM: Office of Planning, Research and Evaluation (OPRE); Administration for Children and Families
RE: Revisions to MIHOPE family follow-up survey
DATE: March 2014
As mentioned in the Supporting Statements A and B for the MIHOPE follow-up data collection (OMB Control No: 0970-0402), a pretest of the follow-up parent survey was conducted. The pretest found that the survey was running about 10 minutes longer than planned, necessitating the elimination of some questions. In addition, the pretest found that we could also save time by streamlining how and who we asked some questions to. Finally, we also discovered that some question wording needed to be revised and that we needed to add a few additional questions. The details about these requested changes are specified below.
Elimination of questions:
To keep the survey to 60 minutes as previously approved by OMB, we identified six groups of questions that could be removed from the survey without affecting the study’s ability to detect impacts on the key outcome domains. These groups of questions are: (1) reasons for not using or receiving services, (2) questions that measure outcomes for which we cannot make a directional hypothesis, (3) some questions on household income, (4) one question on food security, (5) some questions on medical homes, and (6) questions for which we gather key information from other sources.
Reasons for not receiving or using services: We would like to delete questions that asked about the reasons respondents didn’t use services or adhere to certain recommended parenting practices (such as breastfeeding), because we believe that it is more important to focus our time on asking whether or not respondents actually received services or engaged in certain behaviors. Therefore, we recommend deleting:
CH9: What kept [CHILD] from having a well-baby checkup?
CH11: [if child is missing any immunizations] Why is that?
CH43: Why did [CHILD] not get the prescription medications (he/she) needed?
CH53: What is the main reason your child has not visited a dentist in the past year?
P6: What were your reasons for not breastfeeding [CHILD]?
P7: What were your reasons for stopping breastfeeding?
PH8: What is the main reason you don’t have a usual place to go for your general health care?
PH18: Why did you not get medical care or prescription medications you needed?
PH27: What is the main reason you (don’t/didn’t) have a usual place to go for family planning?
PH33: What is the main reason you are not (using birth control now/planning to use birth control)?
Questions that measure outcomes for which we cannot make a directional hypothesis or for which there is no clear evidence that the outcome places a child at risk for later poor outcomes: We also recommend cutting questions for which there is not a clear hypothesis for how home visiting would affect the outcomes (i.e., diagnoses of hypertension or high blood sugar, which home visiting could cause to increase or decrease because it encourages preventive medical care that could uncover more cases as well as help treat them). In addition, we would like to eliminate questions that may be difficult to interpret because they are not applicable to all respondents (i.e., families who do not own a car would not own a car seat), and questions that for which there is no clear evidence that certain outcomes place a 15-month old child at risk (i.e., child is a picky eater at 15 months). The questions we suggest deleting are:
CH17: During the past 12 months, did [CHILD] need a referral to see any doctors or receive any services?
CH18: [if the child needed referrals in the past 12 months] Was getting referrals…
CH49: The next questions are about [CHILD]’s dental health. Does [CHILD] have any teeth yet?
CH67: Do you have a car seat that you can use for your child?
P8: What kind of milk did [CHILD] usually drink during the past 7 days?
P11: How many times do you offer a new food before you decide [CHILD] does not like it?
P12: Do you consider [CHILD] to be… [a picky eater, somewhat picky eater, or not a picky eater]?
PH9: Has a doctor ever told you that you have asthma?
PH11: Has a doctor ever told you that you have high blood pressure?
PH12: Since [CHILD] was born, did a doctor, nurse, or other health care worker tell you that you had diabetes or high blood sugar?
Some questions on household income: After we developed the follow-up parent survey, we finalized an agreement with the Office of Child Support Enforcement to access the National Directory of New Hires. We will be able to access earnings information for most of our sample from the National Directory, and so some of the questions on employment and earnings in the follow-up survey have become duplicative, and we would like to delete them:
FS8: How many jobs do you currently have?
FS9: [if working more than one job] At which job do you work the most hours?
FS12: At [JOB], what is your wage, before taxes? Please include tips, commissions, and regular overtime pay.
FS13: [to clarify unit for wages listed in FS12] Is that…
FS14: What would you estimate you make in a week?
A question on food security: We would like to cut one food security question, in order to align our measure with the six-item module on food security that the USDA deems an acceptable substitute for the longer U.S. Household Food Security Survey and U.S. Adult Food Security Survey modules, to make our measure directly comparable with other surveys.
FS22: Within the past 12 months (I/we) worried whether our food would run out before (I/we) got money to buy more. Was this . . .
Some questions on medical homes: After discussions with experts at Johns Hopkins University and HRSA, we determined that we could assess whether or not a child has a medical home with fewer questions than we originally thought we needed. Therefore, we could cut the following questions:
CH25: Overall, how satisfied are you with communication among [CHILD]’s doctors and other health care providers? Would you say…
CH26: During the past 12 months, how often did [CHILD]’s doctors and other health care providers spend enough time with (him/her)? Would you say…
CH27: During the past 12 months, how often did [CHILD]’s doctors and other health care providers listen carefully to you? Would you say…
CH28: When [CHILD] is seen by doctors or other health care providers, how often are they sensitive to your family’s values and customs? Would you say…
CH29: Information about a child’s health or health care can include things such as causes of any health problems, how to care for a child now, and what changes to expect in the future. During the past 12 months, how often did you get the specific information you needed from [CHILD]’s doctors and other health care providers? Would you say…
CH30: During the past 12 months, how often did [CHILD]’s doctors or other health care providers help you feel like a partner in (his/her) care? Would you say…
CH31: [if the respondent’s primary language is not English] An interpreter is someone who repeats what one person says in a language used by another person. During the past 12 months, did you need an interpreter to speak with [CHILD]’s doctors or other health care providers?
CH32: [if the respondent needed an interpreter in the past 12 months] When you needed an interpreter, how often were you able to get someone other than a family member to help you speak with [CHILD]’s doctors or other health care providers? Would you say…
Questions for which we would gather key information from other sources: In reviewing the survey, we found some questions that were redundant and so we recommend cutting them as well.
PH35: How many more children would you like to have?
CH48: Are you currently receiving help in applying for health insurance for [CHILD]?
PH23: Are you currently receiving help in applying for health insurance for yourself?
PH26: Since [CHILD] was born, how often have you seen someone for family planning or birth control?
PH32: Which of these methods [of birth control], if any, will you use most of the time?
SS10: Overall, how satisfied are you with that communication [between child’s doctors and early intervention program/child care providers]? Would you say…
Questions that we can streamline to reduce administration time:
In reviewing the survey, we found that we could streamline the administration of some questions by being more targeted in who we asked some of the probes or follow-up series questions of. Therefore, we propose the following changes:
Child health insurance. Only ask about the child’s health insurance if the respondent says that the child is not covered under her/his plan or if the respondent has more than one plan. (Originally we were asking all respondents about both their health insurance and the child’s health insurance.)
Birth control. For the question “Since [CHILD] was born, have you used any form of birth control?”, we would like to ask the probe, which lists a multitude of different birth control methods, only of respondents who answer no (instead of asking all respondents).
Conflict Tactics Scale. During the pretest, it was difficult for respondents to use/remember the 8 response categories, especially when the survey was administered over the phone. We consulted with the instrument developers and they said we could reduce the number of response categories from 8 to 6 without affecting the reliability of the measure. Therefore, we would like to change the categories from:
“once in the past year, twice in the past year, 3-5 times, 6-10 times, 11-20 times, more than 20 times, not in the past year but happened before, never happened”
to:
“once in the past year, twice in the past year, 3-5 times in the past year, 6+ times in the past year, this has never happened in the past year”.
Revisions to questions for clarification:
We would also like to revise the following questions based on the feedback we received during the pretest:
Family planning. We would like to ask about source of care for family planning in a more precise way, by adding time periods to the question. Specifically, we would like to change it from:
“Is there a place you go, or have gone, for family planning or birth control?”
to:
For biological mothers: “Since the birth of [CHILD], is there a place you go to, or have gone to, for family planning or birth control?”
For non-biological mothers: “In the past 15 months, is there a place you go to, or have gone to, for family planning or birth control?”
Communication between doctors and early intervention programs. During the pretest we realized that answers to the original question were difficult to interpret, and therefore we would like to make the question clearer by changing it from:
“Do [CHILD]’s doctors or other health care providers need to communicate with (his/her) (child care providers or early intervention program)?” (SS10)
to:
“Do [CHILD]’s doctors or other health care providers and (his/her) early intervention program communicate with one another?”
Home visitor helps connect to services. Since we ask all respondents (male or female) who received home visiting about services the home visitor helped them receive, we would like to ask the more gender-neutral question (i.e., did they receive a “doctor’s visit for care other than when you are sick or injured”?) instead of asking only about maternal preventive care.
Birth spacing. To better be able to estimate the effects of home visiting on birth spacing, we would like to change the question from:
“How many live births have you had?”
to:
“Since [CHILD] was born, have you given birth to another baby?”
Household income. Quality checks of the baseline survey data have revealed that many respondents seem confused about or are unable to answer:
“Some earnings and income come from other sources, like unemployment insurance or help from family and friends. Thinking about these other sources of income, what is the total amount of additional income that you received (that is, in addition to earnings) and the total amount of any income or earnings any other adult members of your household received last month? Was it…”.
We believe respondents would be able to answer a simpler question on total income from other household members, which would also provide the study team with a better sense of the family’s economic status. Therefore we would like to revise this item to ask:
“Now please think about all other members of your household, not including yourself. In [PREVIOUS MONTH], what was the total income of all other members of your household from all sources, including earnings and public benefits?”
Early Intervention probe. When respondents are asked if the child used or received early intervention services (SS7), they are offered the probe “Early Intervention provides services for children with disabilities or delays.” We would like to revise the probe to include examples of such services, so that it reads: “Early intervention refers to services that are provided to infants and toddlers who have or are at risk for developmental delays or who have special needs. Examples include speech and language therapy, occupational and physical therapy, and counseling or other medical services.”
Level of education. To systematically record GED completion as well as the highest grade in school completed, we would like to clarify the response options by allowing for respondents to answer “12th GRADE, BUT NO DIPLOMA” in addition to “12TH GRADE WITH DIPLOMA”.
Receipt of other services. Respondents are asked about their receipt of various services, including mental health, substance abuse, domestic violence shelter and domestic violence. We would like to revise these sets of questions so that if respondents indicate they have used a type of service (domestic violence shelter, for example), we ask them whether they have used services at specific local programs (names of programs will be filled in for each MIHOPE site based on other information we are collecting from the program). We also ask whether they have used services at any other programs. We made these changes so that we will be able to link impacts on service receipt for individual providers with cost information about these providers gathered on the community survey.
Frequency of service receipt. We would like to revise items about frequency of service receipt so that we get more accurate estimates from respondents. Instead of asking how many ‘times’ a respondent used services from particular services, we would like to ask about ‘days’ (or nights in the case of shelter) and include the following probe: “For example, if you got help from service once a week for 10 weeks that would be 10 days.” In addition, we would like to ask about the frequency of services received at specific programs, rather than the frequency of services received at certain types of providers.
Drug use. Respondents are asked a series of items about certain groups of drugs they may use. We would like to revise the first item in the series from “Prescription drugs” to “Prescription painkillers”, because responses to the former question in the initial baseline survey records included many prescription medications that were not considered problematic, such as antibiotics.
Child Care. We would like to edit the following question as follows:
The next questions are about various services you and [CHILD] may receive or use.
Is [CHILD] now receiving care in your home or another home on a regular basis, at least once a week, from someone who is not [child’s] parent? This includes home child care providers or neighbors, but not day care centers or preschools.
Add “home visitor” to list of professionals in following question:
CH45. Next, I’m going to read you a series of topics. For each, please tell me whether a doctor, nurse, health care worker, other professional or home visitor has discussed these topics with you. Please count only discussions, not reading materials or videos.
Revise to parallel question immediately preceding this one:
CH46. Have you ever discussed with a doctor, nurse, health care working, other professional or home visitor what can happen if a baby is shaken?
Delete “bowling” as example of everyday activities:
PH4: Moderate activities, such as moving a table, pushing a vacuum cleaner, or working in the garden?
Add response option “haven’t had it yet” in case the respondent hasn’t yet had her first prenatal visit for her current pregnancy:
PH19: How many weeks or months pregnant were you when you had your first visit for prenatal care? Do not count a visit that was only for a pregnancy test or only for WIC (the Special Supplemental Nutrition Program for Women, Infants, and Children).
Revise the following question to make it accurate since we don’t plan at this time to do a follow up interview with the respondents after the 15-month follow up.
Please tell me the names, addresses, telephone numbers, and e-mail addresses of two people who do not live with you but who will know how to contact you roughly a year from now.
Additions to the survey:
Based on results of the pretest and subsequent re-review of the survey, we would like to add the following questions to gather information on key outcomes, such as breastfeeding, intimate partner violence, household income, and birth spacing. We also propose adding two questions on the child’s race and ethnicity, given the documented flaws in this information from birth certificate data, and a more detailed screening section to ensure that the respondent on the follow-up is the child’s current primary caregiver.
Exclusive breastfeeding. Because exclusive breastfeeding provides significant health benefits for both the child and the mother, the American Academy of Pediatrics recommends it for the first six months of an infant's life. The follow-up survey originally asked about breastfeeding, but not exclusive breastfeeding. Therefore, we would like to add the following question:
“How old was [CHILD] when he/she began drinking formula on a regular basis?”
Intimate partner violence. In the original survey, questions about the respondent’s spouse or partner were limited to those spouses or partners who currently live in the respondent’s household. In the follow-up, we would like to make sure that intimate partner violence questions can be asked of respondents who may not currently have a partner and of respondents whose partner may not currently live in the same household. Therefore, we recommend adding the following questions:
Do you currently have a spouse or partner?
If yes, what is spouse/partner’s name? (This question is in the approved survey, but was previously only asked if the respondent had a spouse or partner living in the home. We would now like to ask of anyone who says they have a spouse/partner, regardless of where that spouse/partner lives)
What is [SPOUSE FIRST NAME]’s relationship to [CHILD]? (This question is in the approved survey, but was previously only asked if the respondent had a spouse or partner living in the home. We would now like to ask of anyone who says they have a spouse/partner, regardless of where that spouse/partner lives)
Does [SPOUSE FIRST NAME] live in this household? (a revision to HH10)
If no current spouse/partner, ask: Did you have any spouse or partner in the past year?
If any in past year, then ask: What was the name of your most recent spouse/partner?
What is [SPOUSE FIRST NAME]’s relationship to [CHILD]?
Home visitor helps connect to services. We would like to ask respondents whether home visitors helped them receive services from a “dentist” and helped them receive “public benefits (TANF, SNAP, WIC)”.
Public benefit levels. The follow-up survey asks respondents whether they have received income or benefits from several sources (cash welfare, food stamp or SNAP benefits, disability insurance, and benefits from WIC; FS12). To gather information about the family’s financial resources, we would like to add the following question to ask about the dollar amounts of each benefit received:
How much [FILL BENEFIT FROM FS17] benefit did you receive in the past month?
Questions from OIRA: Why is this necessary? Could you provide information about why you now need to know the actual dollar amounts? What is the value added to knowing the actual amounts vs. just receipt of benefit?
Response: In the past, programs that help low-income mothers return to work have sometimes found no increases in family income because of cuts in TANF, SNAP, and other forms of public assistance. For that reason, we think it is important to have a measure of total household income. The survey already contains information on the mother’s earnings and income for other household members, but not income from public benefits. Although we think these questions are important at follow-up, they were not on the version already approved by OMB because we had based the public benefits questions on what was on the MIHOPE baseline survey, where income amounts were not considered vital.
Questions from OIRA: Will people likely remember actual amounts accurately?
Response: We have used similar questions on other surveys and obtained reasonable responses. In particular, we used these questions in the Supporting Healthy Marriage Evaluation and found that nearly all responses were in a reasonable range, e.g., of an amount that could have been received through TANF.
Questions from OIRA: Are you already asking about specific dollar amounts for income elsewhere?
Response: As noted above, the survey approved by OMB includes questions on dollar amounts for the respondent’s earnings and income of other household members.
Child race and ethnicity. Originally we planned to gather this information from birth certificates. However, since birth certificates often do not include the race/ethnicity of fathers, biracial children tend to be misclassified. Therefore, to make sure our information about children’s race/ethnicity is accurate, we would like to add two questions (which are identical to the questions asked to determine the adult respondents’ race/ethnicity):
Is [CHILD] of Hispanic, Latino, or Spanish origin?
What is [CHILD]’s race? You may name one or more.
Early Intervention eligibility. The follow-up survey gathers information about the focal child’s receipt or use of early intervention services. To measure the gap between eligibility and service receipt, we would like to add the following question:
Has [CHILD] ever been found eligible to receive Early Intervention services?
Screening section. Biological mothers of focal children completed the baseline interview, but we anticipate that not all of the respondents to the follow-up interview will be biological mothers. During the pretest we realized that this means we need to add a screening section that will identify the focal child’s current caregiver and gather some initial information about any new respondents who are not biological mothers.
Child development. The existing question CH34 asks how old the child is when s/he reaches certain developmental milestones. The question previously asked about when the child sat up, crawled, pulled to standing, or walked holding on to something, but did not ask about when the child took first steps. We propose revising the question as follows:
CH34. How old was [CHILD] in months when (he/she) started to…
Sit alone, steady, without support?
Crawl on hands and knees?
Pull (him/her)self to a standing position?
First walk while holding on to something, such as furniture?
Additional option:
First walk without holding onto anything?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | admin |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |