Appendix E Response to Comments from the National Agricultural Statistics Service (NASS)

APPENDIX E

RESPONSE TO COMMENTS FROM THE NATIONAL AGRICULTURAL STATISTICS SERVICE

(NASS)

NASS Review

By Andrew Dau

Oct. 14, 2014

FNS - Child and Adult Care Food Program Sponsor and Provider Characteristics Study

Part A

• Looks good. I don’t see any major concerns and it looks like a solid data collection plan has been established.

Part B

B.1

• The overall goal is to produce national level estimates; however, using a cluster design, only 23 states will be selected to meet the precision requirement. One should consider that this is a national program, administered by a state CACFP Administering Agency. If there are significant variations in the administration of this program, using states as PSUs could lead to bias in the estimate. Is it possible that a state CACFP Administering Agency does not follow identical protocol of other states? If that is the case, the 23 states sampled may not be truly reflective of the population. The other thing to keep in mind is the distribution of each sub-group of child care providers. Is there a lot of homogeneity between the states in regards to the distribution of child care centers, family child care centers, head starts, and at risk centers?

Response to Comment: The Kokopelli/Westat proposal originally suggested spreading the sample across all 50 states due to the concern expressed by NASS, but more importantly to improve precision of the estimates.  FNS felt that the burden associated with bringing in each of the states made it more appropriate to limit the number of states in which the study is done.  Many other FNS studies are done in fewer states than has been proposed for this study (e.g., the tiering assessment is done in 14 states).

• The calculations for sample sizes with a 90% response rate and a goal of +- 8.5% margin of error seem appropriate.

• Nesting the provider sample with the sponsor sample will help provide valuable data cross tabulations.

B.2

• Looks good. (See concern over national estimates from section B.1)

B.3

• I recommend expanding upon the non-response adjustments portion of the document. I’m assuming you will use a re-weighting technique at a strata level. If a 90% response rate is attained, the non-response impact should be minimal, but it is still good to have the plan explicitly laid out in advance. It may be worthwhile to investigate a study of bias to determine if the concern in section B.1 is valid.

Response to Comment: We do indeed plan to weight providers (and sponsors) to adjust for nonresponse. The plan is that provider nonresponse is adjusted within sponsors, and sponsors within states, but until observing the actual nonresponse patterns one cannot finalize the adjustments. If the expected 90% response rate is achieved, the impact of nonresponse bias would be minimal. OMB doesn’t require nonresponse bias analyses for these rates. The bias potential in B.1 is not from nonresponse, but rather if the sample of states is not representative of all states. This would not be covered by a nonresponse bias analysis.

B.4

• Looks good.

B.5

• Looks good.

E-1