Appendix A5. NSWP-III Research Questions and Objectives

APPENDIX A5

NSWP-III RESEARCH QUESTIONS AND OBJECTIVES

Study Objectives	Research Questions
I. Calculate a national estimate of annual improper payments (case and dollar error rates and dollar error amount) in WIC due to income eligibility error (one component of certification error) to fulfill USDA-FNS’s obligations under the Improper Payment Elimination and Recovery Act (IPERIA) of 2012.	What is the annual case error rate due to income ineligibility for WIC overall and for each certification category: pregnant women, breastfeeding women, non-breastfeeding postpartum women, infants, and children? Report variation in error rates by FNS region, SA and LA characteristics. Report income data overall and for each certification category for all WIC participants, including income for adjunctively or automatically eligible WIC participants. What are the monthly and annualized dollar error rates and amounts associated with income ineligibility case errors both pre- and post-rebate for infant formula? What is the average monthly value of issued and redeemed food packages? How do these error rates vary by FNS region, SA and LA characteristics, and WIC participant characteristics? Examine possible correlates of erroneous certification due to income ineligibility (e.g., participant characteristics, SA and LA characteristics, SA and LA certification policies and practices). Report proportion of WIC participants (overall and by certification category) certified without providing income documentation.1 Compare error estimates to NSWP–I, NSWP–II, and the FNS WIC Income Verification Study.
II. Provide nationally representative error rates and associated dollar error for all types of case error and for denied applicants.2	Report annual case error rates due to reasons other than income ineligibility for WIC overall and for each certification category: pregnant women, breastfeeding women, non-breastfeeding postpartum women, infants, and children, taking into account Federal, SA, and LA regulations and guidance for certification and variation in SA and LA regulations and guidance for certification. Report distribution of case error by reason: income eligibility error categorical error identity error residence error expired certification error For each type of case error, including expired certification errors and denied applicants, what are the associated dollar error rates overall and by certification category, both pre- and post-rebate, for infant formula (i.e., gross and net improper payments)? Average monthly value of issued and redeemed food packages Annualized dollar cost of redeemed food packages to erroneously and correctly certified participants with expired certifications throughout their certification period Duration of issuing food instruments (or EBT benefits) past expired certification date, duration of post-expiration redemption, dollar value of benefits issued post-expiration Examine possible correlates of each error estimate, including correlates of expired certification issuance of food instruments/EBT benefits (e.g., participant characteristics, SA and LA characteristics, SA and LA certification policies and practices). Report rate of denied applicants, both correctly and erroneously denied, showing distribution of reasons for denial for each group.
III. Conduct a census of State agencies to provide information on their certification-related policies and a description of their caseload and operations.	Report the number of LAs in each SA and number of clinics or sites within each LA. Examine how income eligibility is determined. Report program characteristics used to determine adjunctive/automatic income eligibility. Report criteria used to determine whether individual belongs in family/economic unit. Report policies used to document identity and residency of WIC applicant. Examine guidance provided by the SA on what constitutes a WIC application. Examine policies regarding documentation of WIC applications for certification that are denied. Report which SAs calculate a retention rate for WIC participants, including how often and how they have changed over the last five years. Report rate of terminations in WIC and reasons why. Examine how WIC food benefits distributed to participants (on-site; in the home; EBT). Examine the policies regarding certification periods by categorical group. Examine the policies regarding accepted activities for proxies. Examine record keeping systems SAs have in place for certification, termination, and denial.
IV. Provide nationally representative information on LAs’ and service delivery sites’ certification policies and operations.	Examine SA and participant characteristics of denied applicants. Report procedures used to verify residency. Examine the average distribution of caseload-to-certification staff. Report characteristics of LAs that conduct off-site (i.e., clinic, satellites, mobile units) certifications. Report the ages of infants who are being certified for fully breastfeeding packages and the variation of certification timing across SAs and LAs. Examine process and rates of verifying proof of procedures to determine income eligibility including proportions of documentation provided. Report who administers the LA (i.e., local government, SA, nongovernmental organization). Report the number of LAs and the services they provide. Report the characteristics of staff at LAs. Report caseload of WIC participants at LAs. Examine retention rates at LAs. Report characteristics of the physical space of the LAs. Report types of technology present at LAs. Report regular and extended hours of operations of LAs.
V. Provide nationally representative descriptions and analyses of WIC participants’ experiences with the WIC program.	Report participants’ characteristics. Report participants’ household characteristics. Report participants’ monthly and annual income amounts. Examine participants’ prior experiences with the program. Examine participants’ perceived benefits to participation. Report participants’ satisfaction with food packages, specifically the food instruments, shopping experiences, EBTs, cash-value vouchers, and farmers’ markets. Report participants’ satisfaction with WIC staff, certification and recertification processes, clinic location, provided/referred services, informal benefits (such as savings on groceries or opportunities to socialize), and farmers’ markets. Report participants’ participation in other assistance programs. Examine participants’ level of food security. Examine how participating in WIC affects participants’ food purchases with other food resources.
VI. Conduct a case study of former WIC participants who are still eligible for the program.	Examine the reasons why the former participants discontinued participation in the program. Examine the former participants’ satisfaction with the program. Examine the former participants’ involvement in other assistance programs.
VII. Propose and pilot a new methodology for the NSWP series that provides necessary precision with greater efficiency and reduced cost.	What alternative method can produce estimates that meet IPERIA precision requirements for the case error amounts and rates and associated dollar error amounts and rates for income eligibility, denied applicant, expired certification, identity, residence, and category error? What are the results of a pilot of the method and how do they compare to the estimation model (Objective VIII) to age improper payment estimates annually?
VIII. Develop estimation model(s) for use by FNS staff to age the improper payment estimate annually for IPERIA reporting using easily accessible data.	Propose an evidence-based approach to meet IPERIA required annual estimates of income eligibility and associated dollar error rate and amount) using other extant data (e.g., WIC-PC series, WIC Eligibles annual report) and discuss strengths and limitations. Compare proposed approach to previously used aging process (NSWP–II) and to estimates based on NSWP–III survey data. Describe validity and reliability measures of the proposed approach and how the estimation model’s reliability can be tested in the future. Describe adjustments or alternative approaches to improve estimates in future years.
IX. Provide certification and dollar error rates accounting for rebates for items other than infant formula.	Using information on WIC rebates other than infant formula, produce pre- and post-rebate case and dollar error estimates and total case and dollar error estimates.

1 The proportion certified without providing income documentation includes those certified as adjunctively/automatically income eligible, those given a 30-day certification without providing income documentation, and any participants certified erroneously for a longer period without providing income documentation.

2 Objective II also contributes to USDA’s requirements for IPERIA reporting; that is, error rates for IPERIA include not just error due to income ineligibility (Objective I) but the other types of error delineated under Objective II.

pg. 1