OMB Part B_FINAL

Supporting Statement Part B

OMB No. 0584-NEW

Study of WIC Food Package Costs and Cost Containment

September 2017

Project Officer: Ruth Morgan

Office of Policy Support

Food and Nutrition Service

U.S. Department of Agriculture

3101 Park Center Drive

Alexandria, VA 22303

Telephone 703-457-7759

[email protected]

Contents

Part B: Collection of Information Employing Statistical Methods 1

B.1 Respondent Universe and Selection Methods 1

B.2 Procedures for the Collection of Information 7

B.3 Methods To Maximize Response Rates and the Issue of Nonresponse 10

B.4 Tests of Procedures 12

B.5 Consultants 13

Appendices

AA.1. Code of Federal Regulations 7 CFR § 246.10

AA.2 OIG Audit report: State Agencies’ Food Costs for Food and Nutrition Service’s WIC Audit Report 27004-0001-22

AA.3 Additional Authorizing Statutes

A.1. WIC State Agency (58) Advance Letter

A.2. Study of WIC Food Package Costs and Cost Containment Study Information Sheet

A.3. Summary of Cost-Containment Practices

A.4 WIC State Agency Semi-Structured Interview Instrument

A.5 Spreadsheet of WIC State Agency Cost Containment Practices

B.1. WIC State Agency EBT (12) Advance Letter

B.2. Agenda for Conference Call with WIC State Agencies

C.1. Administrative Costs of Cost Containment Practices

D.1. Instruction Sheet for Submitting WIC Participant Certification Data Files

D.2. Sample Reminder Email for WIC Participant Certification Data

D.3. Sample Follow-Up Email for WIC Participant Certification Data

D.4. FTP Site Instructions for State Agencies Submitting Data Files

E.1. Instruction Sheet for Submitting Former WIC Participants Certification Data File

E.2. Sample Reminder Email for Former WIC Certification Data

E.3. Sample Follow-Up Email for Former WIC Participant Certification Data

F.1. Instruction Sheet for Submitting EBT Transaction Data Files

F.2. Sample Reminder Email for WIC EBT Data Request

F.3. Sample Follow-Up Email for WIC EBT Data Submission

G.1.WIC Participant Advance Letter (English)

G.1a. WIC Participant Advance Letter (Spanish)

G.2.Survey of WIC Participants (English)

G.2a. Survey of WIC Participants (Spanish)

G.3.WIC Participant/Former Participant Reminder Postcard (English)

G.3a. WIC Participant/Former Participant Reminder Postcard (Spanish)

G.4 WIC Participant/Former Participant Reminder Letter (English)

G.4a. WIC Participant/Former Participant Reminder Letter (Spanish)

G.5. WIC Participant/Former Participant Refusal Letter (English)

G.5a. WIC Participant/Former Participant Refusal Letter (Spanish)

H.1. Former WIC Participant Advance Letter (English)

H.1a. Former WIC Participant Advance Letter (Spanish)

H.2. Survey of Former WIC Participants (English)

H.2a. Survey of Former WIC Participants (Spanish)

I.1. Insight Policy Research Confidentiality Pledge

J.1. Comment to Federal Register Notice

J.2. FNS Response to Comment

K.1. National Agricultural Statistics Service Comments and Responses to Comments

L.1. Pretest Methods and Summary of Findings

M.1. OMB Burden Table-EXCEL

Tables

Table B.1.1. Estimated Universe and Sample Sizes for the Survey of WIC Participants 5

Table B.1.2. Expected Response Rates 6

Table B.5.1. Individuals Responsible for Statistical Aspects and Data Collection and Analysis 13

Part B: Collection of Information Employing Statistical Methods

1. Respondent Universe and Selection Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

This study has four components: (1) a Survey of WIC Participants, (2) a Survey of Former WIC Participants, (3) analysis of administrative data, and (4) in-depth, semi-structured qualitative interviews with WIC State agency (SA) directors. We plan to collect administrative data from 12 States that have implemented EBT for WIC statewide. See Appendix D.1 for additional information on the certification information being requested on current WIC participants, Appendix E.1 for additional information on the certification information being requested on former WIC participants, Appendix F.1 for additional information on the EBT issuance and redemption information being requested, and Appendix C.1 for additional information on the administrative cost information being collected. The in-depth, semi-structured interviews will be conducted with WIC SA directors in all 50 States, the District of Columbia, the 5 territories, and 14 ITOs. These qualitative interviews will be conducted by phone and will cover the reasons State agencies select some cost containment practices and not others (see Appendix A.4 for the protocol). Statistical procedures will be used to select the ITOs that will be contacted to participate in qualitative interviews; however, the findings from these interviews will not be generalizable to all WIC SAs, given the complexity and type of data collected.

The remainder of this section primarily documents the statistical procedures to be used for the Survey of WIC Participants and the Survey of Former WIC Participants, the components of the study that require statistical sampling and analysis procedures. These surveys will be used to assess the relationships between certain WIC State agency (SA) cost-containment practices and outcomes, such as participant satisfaction with WIC foods, the availability of appropriate foods for participants with special diets, and continued participation in WIC.

These surveys will be based on a randomly selected sample of individuals who currently participate or recently participated in WIC in 12 States that use electronic balance transfer (EBT). The sampling plan is probability based, so study findings can be used to make statistically defensible inferences about the entire populations of WIC participants or former WIC participants in the States from which the samples are drawn. The respondent universes and selection methods for the two surveys are described separately below.

Survey of WIC Participants. The Survey of WIC Participants will be conducted in the 12 States that have implemented EBT statewide as of the beginning of fiscal year (FY) 2016: Florida, Kentucky, Massachusetts, Michigan, Nevada, New Mexico, Ohio, Texas, Virginia, West Virginia, Wisconsin, and Wyoming. Current participants will be selected from a list of all WIC participants in those States with an active certification as of February 1, 2017. These lists will be provided by the WIC SA in each State.

The study team will request the participant name, parent/guardian name (if appropriate), contact information, WIC participant ID, household ID (linking multiple participants in a household), and certain demographic, economic, and nutrition characteristics for all participants (see Appendix D.1 for a complete list of variables). After compiling these participant-level lists, the team will collapse the records into a file of unique household units. Household units in which the only WIC participants are infants who receive only formula¹ or no food package at all will be removed from the sample, as will households in which all WIC participants receive a medical food package. Similarly, households in which the head of household is younger than 18 will be removed from the sample. A few screening questions included at the beginning of the survey will ensure that households that meet these criteria but were not identified in the data file are not included in the survey.

Overall, the study team will interview 250 WIC respondents in each of the 12 States (a total of 3,000 respondents). Cost-containment practices are determined at the State level, so obtaining 250 respondents per State will ensure sufficient statistical power to analyze study outcomes within each of the States. Within the State, estimates of percentages (such as the percentage of WIC participants satisfied with the available choices for a particular food) will have 95 percent, two-tailed confidence intervals of at most 6.2 percentage points.

To examine the effects of a given cost-containment practice across States, States with the same practice will be combined for analysis. Since at least two States have each given cost-containment practice in this study, within a given cost-containment practice, estimates of percentages (such as the percentage of WIC participants satisfied with the available choices for a particular food) will have 95 percent, two-tailed confidence intervals of at most 4.4 percentage points.

We expect to achieve a response rate of 70 percent current WIC participants. We will select a total sample of 4,284 households (or 357 in each State) to achieve 3,000 survey responses (or 250 in each State).

We plan to select a stratified, random sample of WIC participants in each State. Prior to sample selection, we will stratify the State geographically (urban/rural) to ensure sufficient respondents from these areas. Substrata will be defined within each strata by the presence in the household of a WIC participant infant between the ages of 6 and 12 months as of March 2017. This will ensure a sufficient number of responses to questions about infant food fruits and vegetables, meat, and cereal, which are only provided to infants in that age range. Women and children participants receive all the same categories of food, with the exception of canned fish, which is not included in the survey.

Prior to selecting the sample within each State, a sample allocation program will be run to determine the sample sizes within each of the substrata. The sample size for each State will be proportionally allocated to each stratum based on the size of the stratum. The benefits of this procedure include the fact that all weights are exactly the same; as such, there is no “oversampling” of certain strata causing variation in the weights. As a result, the variances of the overall estimates will be smaller than they would be otherwise.

After the appropriate sample size is allocated within each substratum, records in each substratum will be sorted by the number of WIC participants in the family. Then, a systematic random sample will be selected from each stratum. This method involves numbering the WIC households in the population from 1 to N (N = total records in the population). To select a sample of n participating households, we take a household at random from the first k households and every k^th household thereafter until the appropriate number of households is achieved in the stratum. In this way, each household in the sampling frame will be given a known, nonzero probability of selection so that weighted inferences can be made about the entire population of households.

Estimates of the universe presented in Table B.1.1 are based on data from the WIC Participant and Program Characteristics 2014 study (Thorn et al., 2015) and will be updated upon receipt of current participant data in February 2017.² Analysis weights will account for probability of selection and nonresponse; see section B.2, Procedures for the Collection of Information, for more detail on weighting.

Table B.1.1. Estimated Universe and Sample Sizes for the Survey of WIC Participants

Strata	Estimated Number of Participants in Universe	Estimated Number of Households in Universe	Number of Households Selected	Expected Number of Responses
Florida	570,437	N/A	357	250
Kentucky	144,803	N/A	357	250
Massachusetts	129,825	86,318	357	250
Michigan	267,331	176, 730	357	250
Nevada	74,596	48,900	357	250
New Mexico	57,356	38,805	357	250
Ohio	261,805	N/A	357	250
Texas	1,018,175	615,267	357	250
Virginia	155,542	75,503	357	250
West Virginia	43,965	28,456	357	250
Wisconsin	118,487	N/A	357	250
Wyoming	10,675	6,380	357	250
Total	2,852,997	N/A	4,284	3,000

Survey of Former WIC Participants. The Survey of Former WIC Participants will be conducted in 3 of the 12 States that have implemented EBT statewide as of the beginning of FY 2016; Ohio, Texas, and Virginia have been selected because of their relatively restrictive practices. Former participants will be selected from a list of all WIC participants in those States with an active certification as of October 31, 2016, who were due to recertify between November 1, 2016, and January 31, 2017, but have not done so. These lists will be provided by the WIC SA in each State.

The study team will request the same information as for the Survey of WIC Participants, plus the date of the end of the last certification period. After compiling these participant-level lists, the team will identify unique household units. As for the Survey of WIC Participants, household units in which the only WIC participants are infants who received only formula or no food package at all and households in which all WIC participants received a medical food package will be removed from the sample. As in the Survey of Current Participants, a few screening questions will be included at the beginning of the survey to ensure these households are not included in the survey.

We expect to achieve a response rate of 60 percent of the former participants. We will select a total sample of 625 former participants to reach 375 completed responses. The study team will sort the data file by State, urban/rural status, and the number of WIC participants in the family. Then, a systematic random sample of 625 households will be selected. Stratification and sampling will follow the same procedures described for the Survey of WIC Participants above. Analysis weights will account for probability of selection and nonresponse; see section B.2, Procedures for the Collection of Information, for more detail on weighting.

The overall response rate for the entire study is estimated to be 69.3 percent. This estimate includes anticipated response rates of 70.0 percent for the WIC Participant Survey, 60.0 percent for the Former WIC Participant Survey, and 100.0 percent for the WIC State Agency Director Interviews (Table B.1.2).

Table B.1.2. Expected Response Rates

Study Component	Number Selected for Study	Number Expected to Respond	Expected Response Rate
WIC Participant Survey	4,284	3,000	70.0%
Former WIC Participant Survey	625	375	60.0%
WIC State Agency Director Interviews	70	70	100.0%
Total	4,979	3,436	69.0%

See section B.3, Methods to Maximize Response Rates and the Issue of Non-Response, for more information on our strategies for maximizing response rates for these surveys. We expect all WIC State agency directors to participate in the in-depth, semi-structured interviews.

2. Procedures for the Collection of Information

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection

• Estimation procedure

• Degree of accuracy needed for the purpose described in the justification

• Unusual problems requiring specialized sampling procedures

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden

Sample members for the Survey of WIC Participants and the Survey of Former WIC Participants will initially receive an advance letter informing them about the study, describing the types of questions that will be asked, and indicating the incentive (Appendices G.1 and H.1). This letter will inform sample members that they will receive a call from the survey center; sample members may also call the survey center at their convenience if they prefer. The interview will be conducted via CATI by trained interviewers (see section B.3 below for more information on the training received by interviewers and Appendices G.2 and H.2 for the survey instrument). To maximize response rates, non-respondents will receive a reminder postcard (Appendix G.3 , a reminder letter (Appendix G.4 ), and a refusal letter (Appendix G.5 ), each of which will encourage survey participation. See section B.3 for additional information on strategies to increase response rates and quality control procedures used during survey administration. The subsections of B.2 each discuss the topics listed in the bullets above.

B.2.1 Statistical Methodology for Stratification and Sample Selection

Survey of WIC Participants. Within States, the sample will be stratified geographically (urban/rural) to ensure sufficient respondents from these areas. The sample will also be stratified by the presence in the household of a WIC participant infant between the ages of 6 and 12 months as of March 2017. This will ensure that a sufficient number of responses to questions about infant food fruits and vegetables, meat, and cereal, which are only provided to infants in that age range. Women and children participants receive all the same categories of food, with the exception of canned fish, which is not included in the survey. Within each of the four strata, records will be sorted by the number of WIC participants in the family. The sample of 357 records will be proportionally allocated to each stratum. Then, a systematic random sample will be selected from each stratum, for a total sample of 4,284 participants.

Survey of Former WIC Participants. The study team will sort the data file by State, urban/rural status, and the number of WIC participants in the family. Then, a systematic random sample of 625 households will be selected.

In-depth, Semi-structured Interviews with WIC State Agency Directors. The study team will conduct interviews with WIC State agency directors in all 50 States, the District of Columbia, 5 territories, and 14 of the 34 ITOs. The ITO sampling will reduce interview burden by reducing the total number of SA respondents from 90 to 70. The goal of the sample methodology will be to oversample the larger ITOs while ensuring geographic representation across all regions. To meet these goals, the team will first sort the ITOs by the size of the ITO WIC population and create three strata: 1) large ITOs, defined as those with more than 5,149 WIC participants; 2) medium ITOs, defined as those with between 1,043 and 5,149 WIC participants; and 3) small ITOs, defined as those with fewer than 1,951 WIC participants.³ The 14 ITOs will then be randomly selected in each strata using proportional allocation by size. Once the initial sample of ITOs is selected, the team will review the geographic distribution and make minor revisions as needed.

B.2.2 Estimation Procedure

Sample weights for WIC participants and former participants will account for (1) the initial probability of selection, (2) unit nonresponse, and (3) multiple selection opportunities (if needed). The product of these three weights will result in final weights suitable for use in analysis of responses. The final survey response rates will adjust the initial sample weights for nonresponse (and potential duplication) based on relevant variables in the certification records used for sampling. The weighting scheme inflates the respondents’ data to represent the entire universe of WIC participants in EBT States and former WIC participants in the three States included in the survey.

B.2.3 Degree of Accuracy Needed for the Purpose Described in the Justification

Since at least two States have a given cost-containment practice, estimates of percentages (such as the percentage of WIC participants satisfied with the available choices for a particular food) will have 95 percent, two-tailed confidence intervals of at most 4.4 percentage points (e.g., a minimum sample size of n = 500 will be obtained across two States). The confidence interval will be larger for subgroup estimates and will vary with the size of the subgroup analyzed. Estimates of percentages of former participants (such as the percentage of WIC participants who dropped out for a particular reason) will have 95 percent, two-tailed confidence intervals of at most 5.1 percentage points.

B.2.4. Unusual Problems Requiring Specialized Sampling Procedures.

There are no unusual problems that require specialized sampling procedures.

B.2.5. Any Use of Periodic Data Collection Cycles to Reduce Burden.

This is a one-time data collection, so periodic data collection cycles are not applicable.

3. Methods To Maximize Response Rates and the Issue of Non-Response

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Insight’s subcontractor Mathematica Policy Research (Mathematica) will conduct all participant and former participant interviews using computer-assisted telephone interviewing (CATI). Members of the sample frame who are selected for participation in the survey will first receive an advance letter describing the purpose and importance of the survey. Sample members will be called during the 6-month period between March 15, 2017, and September 15, 2017. The letter will include a toll-free number so sample members who prefer not to wait for a call may call the survey center at a time convenient to them.

The study team will make strategic use of the following incentives, refusal conversion techniques, certified bilingual staff, and tools to help achieve the best possible response rate —

• Obtain Complete and Accurate Contact Information. The study team will follow a multipronged approach to obtain current contact information in the case of incomplete or out-of-date records. First, the team will send records with missing contact information to Accurint, which uses a comprehensive database to locate missing contact information. If that is unsuccessful, Mathematica’s locating department will search additional specialized databases.

• Make Early and Efficient Contact With Sample Members. The study team will attempt all interviews according to optimal contact schedules, based on existing data from similar studies. Cases will be delivered to interviewers via the call scheduler. The scheduler reads the call record for the sample cases, determines its current status, sorts them into appropriate queues, schedules calls on active cases, and delivers cases to interviewers in accordance with a predetermined priority system. The scheduler ensures that sample cases are called during the day, in the evening, and on weekends.

• Use a Straightforward, Undemanding Survey. The surveys are designed to be easy to complete. The questions use clear and straightforward language. The CATI instrument will include definitions for terms and examples so interviewers are able to answer respondents’ questions.

• Administer the Surveys Using CATI. Administering the surveys via CATI maximizes the reliability of the data entered by telephone interviewers through skip-pattern logic and checks for consistency and validity.

• Use Trained Interviewers. Respondents will be interviewed by trained members of Mathematica Policy Research’s survey operations center staff, many of whom have significant experience working on similar studies. All survey staff assigned to the study participate in both general training (if they are not already trained) and an extensive project-specific training. Interviewers do not work on the study until they have been certified as prepared. The project-specific training includes role playing with scenarios and other techniques to ensure interviewers are ready to respond effectively to respondents’ questions. They also focus on developing skills for securing respondents’ cooperation and averting and converting refusals.

• Obtain Cooperation. Interviewers will use several strategies to obtain cooperation, including emphasizing the privacy of responses offering to conduct the interview in Spanish, and convincing potential respondents of the importance of the interview.

• Provide Payments for Survey Participants. We offer a $30 payment to survey respondents to increase agreement to participate in the study.

In the event that the response rate is below 70 percent for the Survey of WIC Participants or 60 percent for the Survey of Former WIC Participants, Insight will conduct a nonresponse bias analysis. This analysis will examine any known differences between respondents and nonrespondents to illuminate any potential bias introduced by nonresponse. Results of this analysis will be included in the final report.

It is anticipated that all WIC State agency directors will participate in the in-depth, semi-structured qualitative interviews. Prior experience working with these directors in data collection for other projects indicates that nonresponse is unlikely to be a problem.

4. Tests of Procedures

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Both surveys were pretested during development. The Survey of WIC Participants was pretested with five respondents from WIC households, and the Survey of Former WIC Participants was pretested with four former WIC participants. Pretest respondents were identified by two WIC State agencies (Oregon and Florida). See Appendix L.1 for additional information on participant recruitment. During the pretests, the interviewer asked the questions exactly as worded, followed by specific probes if any questions seemed confusing. The pretest was audio-recorded and included a structured debriefing in which the interviewer discussed questions that appeared difficult to the respondent.

The study team tracked the questionnaire length by topic area to ensure the amount of interview time was proportionate to a topic’s analytical importance. Based on the pretest findings, the study team suggested changes to the survey and the protocol for administering it. See Appendix L.1 for specific details about these changes.

5. Consultants

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Carole Trippe and Betsy Thorn of Insight Policy Research (Insight) will provide consultation on the statistical aspects of the design. Richard Griffiths and Chrystine Tadler (also with Insight) will conduct the sampling procedures. Insight is also responsible for collecting and analyzing all data for this study. Table B.5.1 identifies the individuals responsible for collecting and analyzing the data.

Table B.5.1. Individuals Responsible for Statistical Aspects and Data Collection and Analysis

Name	Title (Project Role)	Organizational Affiliation and Address	Phone Number
Richard Griffiths	Sampling Statistician	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9480
Chrystine Tadler	Sampling Statistician	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9490
Carole Trippe	Project Director	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9498
Betsy Thorn	Primary Investigator	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9488
Betsy Santos	Survey Director	Mathematica Policy Research PO Box 2393 Princeton, NJ 08543	609.750.2018
Nancy Cole	Principal Investigator	Mathematica Policy Research 1100 1st Street, NE, 12th Floor Washington, DC 20002	202.484.9220
Chunlin Dong	Mathematical Statistician	USDA-NASS 1400 Independence Ave., SW Washington, DC 20250	202-720-8951

References

Thorn, B., Tadler, C., Huret, N., Trippe, C., Ayo, E., Mendelson, M., . . . Tran, V. (2015). WIC Participant and Program Characteristics 2014. Prepared by Insight Policy Research. Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition Service.

1 The focus of the survey is participant satisfaction with WIC foods other than infant formula because infant formula is subject to different cost containment laws than other WIC foods.

2 The variable needed to estimate the number of households in the universe was not available for Florida, Kentucky, Ohio, or Wisconsin.

3The recommended cut points for the three strata are based on large gaps in the distribution of ITOs by WIC population size between 6,633 and 3,664 and between 1,236 and 850. We chose the threshold of the cut points to fall at the median of these gaps (5,149 and 1,043).