SUPPORTING STATEMENT FOR
WIC Nutrition Services and Administration Costs Study (WIC NSA)
PART B
Tameka Owens
Office of Research and Analysis
Food and Nutrition Service
US Department of Agriculture
3101 Park Center Drive
Alexandria, VA 22302
Phone: 703-305-2321
Fax: 703-305-2576
E-mail: [email protected]
November 5, 2013
Table of Contents
B. Statistical Methods 1
B1. Respondent Universe and Sampling Methods 1
B2. Procedures for the Collection of Information 4
B3. Methods to Maximize Response Rates 5
B4. Tests of Procedures 6
B5. Individuals Consulted on Statistical Aspects of the Design 7
List of Tables
Table B1-1. Respondent Populations 3
Table B4-1. Instrument Pretesting 6
Table B5-1. Statisticians and Researchers 7
List of Appendices
Appendix A: Screen Shots of Web Surveys
A1. WIC State Agency Web Survey
A2. WIC Local Agency Web Survey
A3. WIC Combination Web Survey
A4. Spanish-language WIC Combination Web Survey
Appendix B: Case Study Interview Guides
B1. Case Study Interview Guide for WIC State Agency
B2. Case Study Interview Guide for WIC Local Agency
B3. Case Study Interview Guide for State SNAP/TANF Program
B4. Case Study Interview Guide for County SNAP/TANF Program
Appendix C: Recruitment Materials
C1. Communication Plan
C2. WIC NSA Cost Study Brochure
C3. Recruitment Letter to WIC State Agencies with Case Studies (includes extant data request)
C4. Recruitment Letter to WIC State Agencies without Case Studies (includes extant data request)
C5. Recruitment Letter to WIC Local Agencies with Case Studies
C6. Spanish Recruitment Letter to WIC Local Agencies with Case Studies
C7. Spanish-language Recruitment Letter to WIC State Agencies without Case Studies
C8. Recruitment Letter to State SNAP/TANF Programs
C9. Recruitment Letter to County SNAP/TANF Programs
C10. Optional Email for WIC State Agencies to Encourage Local WIC Agency Participation
Appendix D: Recruitment Materials
D1. Memo on Pretest Results
Appendix E: Burden Estimate Assumptions
E1. Assumptions Used to Estimate Burden
This section of the Supporting Statement addresses the five points outlined in Part B of the OMB guidelines and focuses on statistical methods related to the collection of information for the study.
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
As described in Part A, this study uses both quantitative and qualitative methods to collect data. Specifically, researchers will gather quantitative data from all State and local WIC agencies via Web survey and, additionally, from all WIC State Agencies via an extant data request. By conducting a census of the State and local WIC agencies, concerns about sampling and analytical significance are not relevant.
We expect to obtain a minimum 80% response rate.1 A paper-based alternative to the Web-based survey that can be mailed back is offered in the recruitment letters to WIC local agencies to boost participation rates.
Recognizing that obtaining a response from all of the State and local Agencies may still not be possible, we plan to conduct a nonresponse bias analysis to examine potential bias in the analysis introduced through nonresponse. The first step in conducting the nonresponse bias analysis will be to determine local agency characteristics that are (1) known for all local agencies, and (2) important to the study analysis or will provide insight into potential bias. The Participants Characteristics 2012 file will be used to determine the characteristics that are relevant to include in the nonresponse bias analysis. We will next examine the unweighted distributional differences for the identified local agency characteristics, between the respondents and the frame. This step of the analysis typically utilizes sampling weights, however, since a census was taken for this study, we do not have sampling weights. After completing the analysis weights (which will be adjusted for nonresponse) we will re-examine the distributional differences between respondents (using the analysis weights) and the frame. This latter analysis will indicate whether the analysis weights were successful in reducing bias due to nonresponse. The weighted respondent distributions, adjusted for nonresponse, should be close to the frame distributions.
Qualitative data will be collected during interviews with a subset of the State and local WIC agencies as well as from a number of State and county SNAP and TANF agencies. The purpose of the qualitative interviews is to help better understand the issues that contribute to the quantitative data reported. As such, a nonprobability sampling method will be used to select respondent agencies for this portion of the study.
There are eight basic respondent populations to be surveyed. Divided into quantitative and qualitative data collection sections in the Table B1-1, the respondent populations are listed along with the universe of respondents, the number for planned in the study and the sampling method used.
Table B1-1. Respondent Populations |
||||
Respondent Population |
Universe of Respondents |
Number of Respondents in Study |
Sampling Method |
Appendix cross-reference |
Quantitative Data |
|
|||
|
90 |
90 |
Census |
A1, A3, C3, C4 |
|
1,900 |
1,900 |
Census |
A2 |
Qualitative Data |
|
|||
|
532 |
9 |
Nonprobability. State SNAP officials will be chosen representing the diversity of agency structures and operations, and also based on willingness and agreement of officials to participate. |
B3 |
|
1193 |
9 |
Nonprobability. State TANF officials will be chosen representing the diversity of agency structures and operations, and also based on willingness and agreement of officials to participate. |
B3 |
|
90 |
14 |
Nonprobability. The criteria for selecting 14 State WIC directors are as follows: 3 from large-size WIC State agencies; 3 from medium-size; 3 from small-size; 2 from high cost/geographic-unique areas4; and 3 from Indian Tribal Organizations (ITOs). |
B1 |
(qualitative interview) |
1,900 |
32 |
Nonprobability. Local WIC Directors will be selected from the 14 State WIC agencies listed above with the following distribution:
|
B2 |
|
5,000 |
1 |
Nonprobability. A county SNAP agency official will be selected from one of the 9 state SNAP agencies visited based on the recommendations of a state SNAP official. |
B4 |
|
5,000 |
1 |
Nonprobability. A county TANF agency official will be selected from one of the 9 state TANF agencies visited based on the recommendations of a state TANF official. |
B4 |
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection,
Estimation procedure,
Degree of accuracy needed for the purpose described in the justification,
Unusual problems requiring specialized sampling procedures, and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
Statistical methodology for stratification and sample selection. For the collection of quantitative data, all State and local WIC agencies are being surveyed; thus, stratification and sample selection are not applicable. Qualititative data will be collected from a nonprobability sample of convenience and will be used to help place the quantitative findings in context and to avoid erroneous generalizations about the larger population.
Estimation procedure. For the collection of quantitative data, all State and local WIC agencies are being surveyed; thus, estimation procedures will not be used. Qualititative data will be collected from a nonprobability sample of convenience and will not be used to make inference to the larger population; therefore, estimation procedures will not be employed.
Degree of accuracy needed for the purpose described in the justification. For the collection of quantitative data, all State and local WIC agencies are being surveyed; thus issues related to the degree of accuracy of the data are not applicable to this Study. Qualititative data will be collected from a nonprobability sample of convenience and will not be used to make inference to the larger population; therefore, degree of accuracy is of no concern.
Unusual problems requiring specialized sampling procedures. There are no unusual problems that require specialized sampling.
Any use of periodic (less frequent than annual) data collection cycles to reduce burden. There will be no periodic data collection cycles since this is a one-time examination of annual Federal Fiscal Year (FFY) 2013 data.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.
The following methods will be employed to maximize the response rate for the study:
Minimize the length of time required to complete the Web survey by collecting as much data as possible from other sources (e.g., State Agency size from FNS website, breastfeeding rates from FNS annual report, nutrition education methods/type of staff used from the WIC Nutrition Education Study);
Design an instrument that minimizes respondent burden by being short in length, asking for data the way respondents collect them, and using terms familiar to respondents (Appendices A1, A2, A3, and A4);
Test the draft instrument using cognitive interviews to ensure respondents can properly understand the questions and response options are robust (See Memo on Pretest Results in Appendix D1);
Be flexible by allowing respondents to complete the survey by other (non-Internet) means, if desired, and to submit extant Form 798-A data in the format they currently use;
Have a Help Desk during business hours to answer respondent inquiries (Phone number provided in recruitment letters found in Appendices C3, C4, C5, and C6);
Develop a study brochure to explain the overall survey effort to affected agencies (Appendix C2); and
Contact nonrespondents by email, FedEx letter, and phone calls to encourage participation.
Our expectation is that the planned methods of data collection will result in accurate and reliable data necessary for the planned analyses at acceptable response rates.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.
Pretests of the six data collection instruments were conducted May 7-31, 2013 as shown in the Table B4-1 below:
Table B4-1. Instrument Pretesting |
|
||
Survey Instrument |
State(s) where Pretests Conducted |
Respondent(s) |
Number of Pretests Conducted |
(Appendix A1) |
Kansas and Minnesota |
State WIC directors |
2 |
|
Arizona, Kansas, and California |
Local WIC directors |
3 |
|
South Dakota |
State WIC director |
1 |
|
Maryland and Tennessee |
State WIC directors |
2 |
|
Maryland and Tennessee |
Local WIC directors |
2 |
|
Maine |
State officials |
1 |
As part of the assessment, pretest respondents were provided with either a copy of the instrument or a live Web link by which they could access the Web-based survey. Respondents were then asked to review the questions for ease of comprehension and their ability to answer the questions using FFY 2012 agency data. In addition, they were encouraged to discuss the level of difficulty or ease of obtaining the information needed for answering the questions and to estimate the amount of time required to collect the information. Respondents provided feedback via written comments and in a follow-on debriefing session with Altarum and RTI on May 20, 2013.
Information from the pretest was combined with input previously obtained from the Study’s Peer Advisory Panel (PAP) and revisions to the instruments were made accordingly. FNS was provided with a pretest memorandum summarizing the feedback received and changes made to the instruments based on the pretests and PAP input. The memorandum of pretest results is included as Appendix D.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Altarum and RTI will collect the information and analyze the data on behalf of FNS. Altarum is leading the case study data collection effort (qualitative) and SNAP/TANF comparisons; RTI is leading the Web survey implementation and analysis. Table B5-1 below outlines the project team members associated with these efforts. RTI’s Benjamin Yarnoff, PhD, is the senior economist for the study.
Table B5-1. Statisticians and Researchers |
|||
Name |
Position |
Telephone Number |
|
Altarum Institute |
|||
Loren Bell |
Senior Fellow |
207-358-2793 |
|
Stacy Gleason, MPH |
Senior Policy Associate and Data Analyst |
207-358-2783 |
|
Diane Phillips, MBA, RD |
Senior Policy Associate |
202-603-7142 |
|
RTI International |
|
||
Benjamin Yarnoff, PhD |
Health Economist |
919-541-6640 |
|
Celia Eicheldinger, MA |
Statistician |
919-541-6222 |
|
David Bellard |
Health Economist |
919-541-6598 |
|
USDA |
|
||
Andrew Dau |
NASS |
202-720-6482 |
|
Tameka Owens, PhD |
FNS |
703-305-2321 |
|
1 As noted in Part A, the Study expects to obtain responses from 91% of WIC State Agencies, 86% of WIC Local Agencies and 100% of SNAP and TANF agencies.
250 states, DC, VI and Guam (http://snap.nal.usda.gov/state-contacts )
350 states, DC, 66 ITOs (http://www.acf.hhs.gov/sites/default/files/ofa/2011_caseload_data.pdf ) and 3 territories (http://www.gpo.gov/fdsys/pkg/GPO-CPRT-108WPRT108-6/html/GPO-CPRT-108WPRT108-6-2-12.htm )
4These include all of the Trust Territories, the District of Columbia, Alaska, and Hawaii, which have unique geographic or political governance features that make them different from other state agencies. There are eight state agencies in this group.
| File Type | application/msword |
| File Modified | 2013-11-06 |
| File Created | 2013-09-26 |