Appendix E3.1 National Agricultural Statistics Service (NASS) Comments

APPENDIX E3.1

National Agricultural Statistics Service (NASS) Comments

NASS Review of the

Supporting Statement for OMB Clearance for
FNS - National Survey of WIC Participants (NSWP-III)

Reviewer: Chunlin Dong (Mathematical Statistician)

The purpose of this study is very clear and objectives and research questions matched very well. The description in the survey questionnaire is very clear, and it abides by Dillman’s principles:

1. Place the instructions exactly where that information is needed and not at the beginning of the questionnaire, and

2. List answer categories vertically instead of horizontally, etc.

And, the statistic design and method used for this survey are correct. Below are several supporting comments:

1. The seven sampling frames are appropriately defined.

   1. However, it does not explain the differences between the sampling frames. For example, why the State Agency and Local Agency sampling frames include Alaska, Hawaii, and U.S. Territories but, the WIC sampling frame does not.

2. Since the population is concentrated in ‘natural’ clusters (clinics, local WIC offices, streets blocks, etc.), cluster sampling is an effective design for the WIC Participation Certification Survey and Denied Applicant survey.

3. When units have unequal probability of selection, then variance estimation can be very complex. So, it is appropriate to use the high entropy PPS sampling method. The entropy is a measure of randomness and it will be large when the probability mass is well distributed over the set of possible samples. A sampling design with high entropy is more robust than a design with low entropy; a high entropy is usually preferred (Tillé - ‎2016). It is appropriate to use high entropy PPS for the first three stages of the WIC Participation Certification Survey and Denied Applicant Survey and stratified simple random sampling method for the final stage.

4. Clusters of participates located close to each other tend to be correlated, as measured by intraclass correlation. Design decisions have impact on the intraclass correlation and intraclass correlation has an impact on the variability of the sampling estimate.

5. The process to prepare for Spanish speaking respondents is impressive: creating Spanish survey forms, hiring and training Spanish speaking interviewers, etc.

6. Sending a text message reminder is innovative, this may be helpful for participators who check their email infrequently.

Suggestions

1. To improve the questionnaire’s design, consider increasing the font of section titles and section subtitles.

2. Consider using a minimum cluster size of 100.

   • In 1990 the NORC National Frame used a minimum cluster size of 50. However, other NORC surveys that were also conducted in the 1990s often used a minimum cluster size of 100.

   • A sample of 20 for each of the five WIC certification categories will result in a cluster size of 100.

3. Using the finite population correction (FPC) which will result in a smaller standard error of the mean and proportion.

   • FPC is square root of (N – n/N – 1), where N is the number of elements in the population and n is the number of elements in the sample.

     • However, if less than 5% of population is sampled, then FPC factor has little or no practical effect on the value of the standard error of the mean or proportion.

4. If the FPC is used, the design effect formula presented at the bottom of page 24 of the NSWP-III draft OMB package part B document will be:

   • Design effect (first stage) = .