Appendix RR
National Agriculture Statistics Service Comments
March 13, 2012
Infant and Toddler Feeding Practices Study (ITFPS-2)
Comments Relating to Statistical Methodology for this Proposed Survey
The information presented in Table B1.1. and Table B1.2. From studying Table B1.1. and the accompanying explanations, please clarify whether the sampling units would be the mothers or the infants. The narrative explanation at the beginning of section B.1. (page 10) states that “The respondent universe … will include … pregnant women … and mothers or other representatives”. So should we consider the woman / mother to be the sampling unit? However, the population description in Table B1.1. describes the population components, and one of them is “Infants Enrolled in WIC”. So this is unclear to me … are the sampling units the mothers or the infants?
Response: The infant is the sampling unit. We inserted a sentence in the “Respondent Universe” discussion in Section B.1 to clarify.
Please clarify. The narrative explanations, “already existing” mothers (even if they are not currently pregnant) are eligible for this survey if they have one or more infants that are eligible. As I envision the conduct of this proposed survey, there may be one mother eligible, but with two distinct infants eligible. For example, assume that a woman is 7 months pregnant, and she also has a 9 month old baby that had no prenatal exposure to WIC. I assume that this one woman would have two infants eligible for this study – the infant that will be born in approximately two months and the existing 9 month old baby. I assume that the mother may receive distinct questionnaires to provide responses / information on each infant separately? So this raises the general question: if the mother is the sampling unit and she gets sampled into the survey, are all eligible infants (of this mother) sampled into the survey? Or are only some of the infants sampled into the survey?
Response: You are correct that one eligible mother might have two distinct infants eligible. This doesn’t pertain to the example you gave because the 9-month-old is not age-eligible for the study. (Only infants 2.5 months old or younger are eligible.) However, suppose the 9-month-old was 2 months old instead. In the example you gave, (with this modification) the mother would need to enroll both the 2-month-old and the fetus in WIC during the recruitment window in order for both to be eligible. Yes, the mother will receive distinct questionnaires to provide responses for both; furthermore, the timing of administration of the two sets of questionnaires will differ because the two children will attain various ages at different points in time. Per the response to item 1, the mother is not the sampling unit; the infant is the sampling unit. The exception to the “multiple possible eligibles” rule will be made in the case of multiple births (twins, triplets, etc.); in that case, because of the likelihood of very high intraclass correlation, no more than one of the multiples will be sampled.
In examining Table B1.2., please clarify the four major sections of it. If a woman was not currently pregnant but she had recently enrolled a 6 month old baby in the WIC program, then I assume that this woman (and her 6 month old infant) would be eligible for this study. Would her potential inclusion in the study be reflected in the “cohort” section of Table B1.2. … since this woman was enrolled post-natal?
Response: In your example, the 6-month-old infant (if eligible; i.e., if the 6-month-old infant was enrolled in WIC by age 2.5 months) would be included in the “Postnatal Sampling” counts…”. The “Cohort” counts are sums of the “Consented and Enrolled”counts for “Postnatal Sampling” and the “Live Births”counts for “Prenatal Sampling.”
At the bottom of page 12, there are some brief explanations of nonresponse adjustment methodology. This section states that “Since there is generally no way to directly measure the difference in key survey characteristics between respondents and the population as a whole, various methods have been developed that aim to assess the potential for nonresponse bias.” Should NASS be provided more detail relating to these “various methods (that) have been developed”? Perhaps this is explained in one of the appendices?
Response: Additional detail about the nonresponse adjustment methodology and about the planned methods of nonresponse bias analysis has been inserted into Appendix Y.
Relating to this topic of whole unit nonresponse (NR) and imputation, I hypothesized over how a particular set of circumstances might be handled. Let’s assume, for example, that a woman gives interviews (i.e., fills out the appropriate questionnaire) from the time that she is pregnant to the early months post-partum. If a woman continues to give interviews up through 7 months, but then stops cooperating, we have no responses from her for 9 months, 11 months, 13 months, etc. How do we adjust for this, given these unit NR circumstances? Are the eligible woman put into some type of NR groupings or strata, so that any type of unit NR adjustment may be carried out based on responses from “similar” mothers?
Response: Yes, we will consider earlier wave variables when determining which variables to use to form the cells for nonresponse adjustment. We have inserted a sentence about this in the section on “Calculation of the Survey Weights” in Appendix Y.
Also unclear, how FNS creates this frame relating to the WIC staff. On page 11, it is stated that “We will administer a WIC Staff Survey of all WIC staff members in each service site that provide direct services to pregnant mothers.” For this explanation, I assume that FNS is referring to “each sampled service site”. However, on page 13, it is stated that “ … no national list of service sites exists.”. From reading some of the accompanying explanations, it appears that FNS will first identify a random sample of eligible mothers. And then they will work backwards, in a sense, and identify the WIC service center that service these selected mothers. So using this type of approach, they would hope to identify about 80 WIC service centers that meet certain criteria.
Response: We have inserted text to clarify that the WIC staff sampled for the WIC Staff Survey will only come from sampled service sites. (See the “WIC Staff & State/ Local Personnel” sampling discussion in section B1.) We will actually be sampling service sites directly (and then sampling infants/gravidas within those service sites). Although no national list of service sites exist, we will use the existing list constructed from the PC2010 (which contains a mix of single service sites and composite sites) as the initial sampling frame and will select the sample of service sites in two stages (with the first-stage sample selected in two phases), as described in section B2.
From reading an explanation of the estimation procedures on page 15, I tried to determine whether FNS had planned to primarily publish estimates at the national level. Or did they also plan to publish estimates at some type of regional level? Maybe FNS will want to compare an estimate in a “Southern” region to an estimate in a “Midwestern” region, for example? From some related information presented on pages 16 and 17, it appears that FNS plans to compute estimates by different racial groups and by different demographic groups. What levels of precision does FNS project for some of the important estimates that will be produced at the national level? NASS employs the statistical concept of coefficient of variation (CV), in discussing the precision of survey estimates. What is the magnitude of CVs that FNS anticipates for important estimates at the national level? Perhaps 2%? Perhaps 5%?
Response: The sample for the WIC ITFPS-2 is designed to support national estimates. The precision requirements are expressed in terms of minimum detectable differences, with a test size of 0.05 and power of 0.8. However, at the national level, the expected CVs for key estimates used in this design are 5% or below for feeding practices estimates (percent initiating breastfeeding; percent breastfeeding at six months; percent introduced solid food prior to 6 months) and 13% or below for child weight (normal or low; overweight; obese). (The higher CVs for the child weight estimates are due to the fact that the overweight and obese categories are rare characteristics, with only 5 percent expected to fall in the obese category.)
Near the top of page 16, there is a brief reference to a particular hot deck imputation method. The proposal states “A cyclical n-partition hot deck (an approach analogous to the Gibbs sampler but using the hot deck to generate the imputations) will be used for imputation.” I think of this imputation reference as a way to address item nonresponse. It would be helpful if this proposal provided more details on this approach. Perhaps these details are contained in one of the appendices.
Response: We inserted a sentence in the “Estimation Procedures” discussion in Section B2 to clarify that weighting will be used to adjust for nonresponse to the initial interview and to adjust for attrition. The sentence following the new sentence describes what imputation will adjust for. We also inserted a sentence (at the end of the “Estimation Procedures” discussion) that refers the reader to Appendix Y for further details.
Also, near the top of page 16, it is stated that “The sample size requirements for the WIC ITFPS-2 were determined using power projections.” From this, I am assuming that the sample size projections were arrived at using a statistical basis … as opposed to being driven strictly by budgetary or resource constraints.
Response: Yes, the target sample sizes were arrived at based on consideration of the minimum detectable differences in estimates between key subgroups, as discussed in the ‘Degree of Accuracy Needed for the Purpose Described in the Justification’ discussion in Section B2.
Comments Relating to Other Aspects of this Proposed Survey
As I considered some of the explanations provided on pages 10 and 11, I pondered how a particular circumstance would be handled. If a woman enrolled in the WIC program at the time that her baby was 6 months old (and she was not pregnant at the time), I assume that she would be eligible for this study, and could potentially be interviewed as part of these follow-up interviews – i.e., 7 month, 9 months, … , 24 months. In this particular circumstance, I assume that she would not be asked to recall circumstances from when her baby was 5 months old, 3 months old, etc. On page 15, there is a reference to “catch-up questions”, to obtain information from prior months. But from the explanation given, I assume these catch-up questions would address circumstances where a woman had missed several waves (after responding to an earlier wave). Catch-up questions would not be used to capture events and previous views, for the circumstance where a woman was not previously involved with WIC.
Response: The only infants eligible for enrollment postnatally are those ages 2.5 months of age or younger. That detail has been added to the “Sampling Methods/WIC Participants” discussion in Section B1. The “catch-up questions” are, as you note, to address situations in which a mother misses waves of data collection after responding to an earlier wave.
Also was uncertain about some of the circumstances associated with the third component in our population – see the third row in Table B1.1.; i.e., that row corresponding to “WIC state / local personnel”. An explanation on page 11 states that “We estimate the universe for this interview to be one WIC director in each state agency (91 state agencies) and a site manager in 125,000 provider sites, for an estimated population of 125,091 personnel. “ I was curious about the circumstances by which there were 91 state agencies. Maybe some states have two distinct state agencies for example? Maybe in CA, there is one for northern CA and one for southern CA? But this explanation also states that “We expect to complete interviews in 35 states and 80 WIC sites for a total of 115 interviews.” Why does FNS reference 35 states, and not 50 states? Maybe certain states do not have a heavy WIC presence, so we do not anticipate interviewing WIC staff / personnel in certain states?
Response: The 91 state agencies include the 50 states, DC, Puerto Rico, and each of the Indian Tribal Organizations (ITOs). As discussed in the “Sampling Methods: WIC Staff & State/ Local Personnel” discussion in Section B1, the WIC Staff Survey will be conducted with staff associated with sampled WIC service sites. (This discussion has been modified to clarify this point.) Although there are 91 state agencies in the universe, we expect about 35 of these to be included in the final sample of 80 service sites.
Also had some relatively minor questions with terminology on page 13. Near the top of that page, there is a reference to “sentinel variables”. I have never seen reference to such variables … what are sentinel variables? Reference to these types of variables is made as part of a discussion on nonresponse bias adjustment. Further down on page 13, there is a reference to ITO WIC representatives. What does the acronym ITO stand for?
Response: We have replaced “sentinel variables” with “key variables”. ITO stands for “Indian Tribal Organization”; we have revised the text to spell that out prior to using the acronym.
On pages 18 and 19, there is a section that addresses methods for maximizing response rates. A bullet near the top of page 19 references an introductory video. Will this video be sent to all potential participants? In the very last bullet on page 19, there is a bullet which states that “Participants will receive a total of $270 for enrolling and completing up to 11 interviews.” My primary comment here is just to applaud FNS, in making such a relatively large sum of money available for potential respondents. Hopefully this will be a significant enticement, to motivate many mothers to involve themselves in this important survey.
Response: Regarding the introductory video, we have removed that communication method from our protocol because recruiters will be available on-site to explain the study to potential participants. Thank you for your comment on the incentives. We also hope this will motivate mothers to involve themselves in this important survey.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Susie McNutt |
| File Modified | 0000-00-00 |
| File Created | 2021-01-29 |