Section A 6-26-08

Contract No.: GS-10F-0050L

MPR Reference No.: 6328-200

SECTION A

OMB Clearance Package

For

CACFP Improper Payments Data Collection Pilot Project

October 23, 2007

Submitted to: U.S. Department of Agriculture Food and Nutrition Service Office of Analysis, Nutrition and Evaluation 3101 Park Center Dr., Rm. 1014 Alexandria, VA 22302-1500 Project Officer: Fred Lesnett 703-605-0811

Submitted by: Mathematica Policy Research, Inc. P.O. Box 2393 Princeton, NJ 08543-2393 Telephone: (609) 799-3535 Facsimile: (609) 799-0005 Project Director: Rhoda Cohen

CONTENTS

Page

INTRODUCTION

A. JUSTIFICATION

1. Circumstances Necessitating the Collection of Information

2. Purposes and Uses of the Information Collection

3. Use of Technology to Reduce Burden

4. Efforts to Avoid Duplication

5. Methods to Minimize Burden on Small Entities

6. Consequences of Not Collecting Data

7. Special Circumstances

8. Federal Register Announcement and Consultation

11. Additional Justification for Sensitive Questions

12. Estimates of Hours Burden

13. Estimates of Cost Burden to Respondents

14. Estimates of Annual Costs to the Federal Government

15. Reasons for Program Changes or Adjustments

16. Plans for Tabulation and Publication of Results

17. Approval to Not Display the OMB Expiration Date

18. Explanation of Exceptions

A. JUSTIFICATION

1. Circumstances Necessitating the Collection of Information

The (CACFP), administered by the U.S. Department of Agriculture (USDA), Food and Nutrition Service (FNS), plays an important role in ensuring that children have adequate access to food. The CACFP is authorized under Section 17 of the National School Lunch Act (42 U.S.C. 1766) to provide funds for meals and snacks served to children in day care centers, children residing in emergency shelters, and youths participating in eligible after-school care programs.

In addition, the CACFP helps improve the quality of day care and makes it more affordable for low-income families by reimbursing family day care homes (FDCHs) for serving nutritious meals and snacks to children participating in these day care facilities. The day care providers are reimbursed at a fixed rate for each qualifying meal they serve to program participants.

However, the USDA has identified the CACFP as one of its programs “susceptible to significant erroneous payments” (U.S. Department of Agriculture n.d.). Erroneous payments in the CACFP arise when program sponsors or day care providers submit improper meal counts and claims for meal and snack reimbursements. Under the Improper Payments Information Act of 2002 (IPIA; Public Law 107-300), the USDA must identify and reduce improper (or erroneous) payments in various food and nutrition programs.

To comply with the requirements of the IPIA, the USDA must estimate the annual rate of erroneous payments in the CACFP. A full assessment of the rate of erroneous payments is a complex undertaking because reimbursement and eligibility requirements vary for different components of the program. To evaluate methods for validating meal claims submitted by the FDCHs for reimbursement, the FNS contracted with MPR to conduct the CACFP Improper Payments Data Collection Pilot Project.

The purpose of the proposed research is to assist the FNS in identifying a reliable, feasible, and cost-effective method for estimating the valid number of reimbursable meals provided to participating children by the FDCHs that participate in the CACFP. In the follow-up study that the FNS will ultimately undertake, the recommended methodology will be used to estimate the number of reimbursable meals actually served. The data collected to document the number of valid meals served will be compared with data on the number of reimbursable meals claimed by the FDCHs and their sponsors to produce a national estimate of erroneous payments in the CACFP and thus meet the reporting requirements of the IPIA.

• The study will address the following research questions:

• What are the strengths of the method for validating the meal reimbursement claims submitted by the FDCHs?

• What are the weaknesses of the method for validating such claims? Which, if any, steps could be taken to overcome these weaknesses?

• What is the level of confidence that the estimates of erroneous meal claims and reimbursements developed from the application of the method will meet the requirements of the IPIA?

• What is the feasibility of administering the method on a national level? Could the method be administered on a national level at this time? Which factors or events must be present for the projected level of feasibility to be met?

• What is the potential cost of implementing the data collection method nationwide?

2. Purposes and Uses of the Information Collection

A comparison of the relative strengths and weaknesses of each data collection methodology will provide the FNS with guidance on selecting the best methodology for validating meal claim reimbursements.

MPR will conduct the pilot study in three states that have sponsors with FDCH locations that are close to one of the MPR offices. MPR will require a total of five sponsors using an average of 6 FDCHs per sponsor for a total of 30 FDCHs for the study.

Method 1: Parent Recall. MPR will use an average of four FDCHs from each of five sponsors for sampling parents/guardians. For half the homes, an average of five parents/guardians will be randomly sampled. For the other half, attempts will be made to reach all parents. Each parent will be assigned to a target week. There will be four target weeks within the month of interest, corresponding to the monthly reporting period. MPR will attempt to complete a 15-minute telephone interview with the sampled parents. Calls to parents will begin on the Sunday following their target week and continue for a total of 4 days. If a parent cannot be contacted during that 4-day period, the parent interview will not be attempted.

Method 2: Data from Direct Observation and Interviews with Providers at FDCHs. In addition to the 20 FDCHs used for parent recalls, MPR will identify 2 additional homes from each of five sponsors for a total of 30 FDCHs for direct observation. MPR will work closely with the sponsors who will be asked to accompany MPR field staff on each visit. Whenever possible, we will try to have our visit count as one of the sponsor’s unannounced monitoring visits. As part of the preparation for the visit, MPR will ask the sponsor about the activities associated with a monitoring visit and will determine how we can work together to achieve both the pilot test goals and the sponsor’s monitoring requirements. One key consideration is our need to observe two eating occasions/ meals and the intention to be on-site for a limited number of hours. We will ask that sponsors introduce MPR field staff to the home providers and explain briefly why we are accompanying them on their monitoring visits and how long we plan to be in the home.

If the sponsor is unable or unwilling to accompany MPR field staff on the visit, we will ask the sponsor to notify the FDCH provider that the sponsor is cooperating in the study and request the provider’s cooperation in granting MPR staff access to the home to observe the meal service. MPR will also ask the sponsor to provide a letter for the field person to take to the visit to explain that the sponsor is cooperating in the study and that the provider should permit MPR field staff to observe meal service. MPR will keep careful note of any problems in gaining cooperation and whether the sponsor was able to accompany us.

Each home selected for observation will be assigned a window for the visit, which will include the two meals scheduled for observation. Each on-site visit will start 30 to 45 minutes before the scheduled time of the first selected meal to be observed. Field staff will have a form on which to record information about whether the home was providing child care on the day of the visit, the number of children present,¹ the presence or absence of any type of attendance sheet, general observations about the space, which meals were provided during the observation, and whether food was brought from home. Field staff will collect information for each meal service observed. During the visit, the MPR field staff person will ask the provider to do a short interview (no more than 15 minutes), with questions about meal preparation, whether the day of the visit is typical, and how the provider tends to complete the required forms.

The scheduling of 30 home visits will occur throughout the entire month of interest. Unlike the parent recall method, there will not be a target week; but, as with the other method, MPR will request three months of administrative records from the sponsor for each home visited. These records include monthly meal claim forms from the providers, child enrollment forms, and attendance forms (if available). All data collected from administrative records and forms completed during the visit will be abstracted, and the data will be entered at MPR for analysis.

3. Use of Technology to Reduce Burden

The data collection plan was designed to obtain reliable information in an efficient way that minimizes respondent burden. Consistent with that goal, information will be gathered from existing data sources, where feasible. Existing data sources will include forms already collected and maintained by sponsors and the FDCHs, such as the CACFP application forms, meal claim forms, and attendance logs. This information will be obtained in the form of computer files if sponsors or the FDCHs prefer this method. If it is too burdensome or not possible for a sponsor or provider to furnish information as a computer file, sponsors and the FDCHs will be asked to provide paper copies of the relevant information that will be coded by the study team.

Parent interviews will be conducted by telephone. All telephone interviews will be brief in duration, ask the minimum number of necessary questions, and take place at times convenient for the parent.

4. Efforts to Avoid Duplication

No equivalent information exists on the best methodology to use to validate meal claims.

5. Methods to Minimize Burden on Small Entities

The primary entities for the study are sponsors, the FDCHs, and parents. Burden is minimized for all respondents by requesting only the minimum data required to meet the study's objectives. The data requirements were determined by careful consideration of the information needed to meet the study's objectives and through a pretest.

6. Consequences of Not Collecting Data

This is a one-time collection effort. The data collection plan described in this submission is necessary for conducting the CACFP Improper Payments Data Collection Pilot Project and, consistent with the goal of the IPIA, will help guide the FNS on a methodology to use to estimate erroneous payments in the CACFP.

7. Special Circumstances

There are no special circumstances associated with this data collection and the request fully complies with the regulations.

8. Federal Register Announcement and Consultation

The 60-day Federal Register Notice was published on February 28, 2007 at 72 FR 8683. Comments (Appendix D) were received from the following parties to the posting of the 60-Day Notice in the Federal Register for the CACFP Improper Payments Data Collection Pilot Project:

Carolyn Morrison

CCDS/Metro CCRR

CACFP National Forum

Jerry Bowers

Executive Director of a Child and Adult Care Food Program (CACFP)

FRAMAX

Michael Morath

Minute Menu Systems

Michael Spevacek

AccuTrak Group LLC

Comments submitted in response to the 60-Day Notice in the Federal Register identify issues related to the reliability, validity and feasibility of the proposed data collection methods for validating meal claims submitted by FDCHs in the CACFP. When conducting the current evaluation of the proposed data collection methodologies and in developing their final recommendations, the contractor will carefully review, evaluate and take under consideration the comments received in response to the 60-Day Notice. The contractor's recommendations at the end of the project will incorporate the comments received in response to the 60-Day Notice and their findings from the pretest and the pilot test phases of the project. Comments received in response to the 60-Day Notice highlight areas of concern in the current evaluation project which is designed to assess the feasibility of developing a data collection methodology that can be used to validate FDCH meal claims.

9. Payments or Gifts

The study is planning to reimburse sponsors and FDCH providers for some of their costs incurred as a result of their participation in the study. The sponsors will receive a total of $75 to defray some of the cost of providing a monitor from the sponsor’s staff to accompany the MPR observers during their visits to six FDCHs and the cost of clerical staff retrieving three months of administrative records for each of their FDCHs included in the study. MPR will give FDCH providers $25 to compensate for any additional staff they might require during the hours of observation and provider interview.

We will use an experiment to determine if a $10 response incentive for parents/guardians to complete a 15-minute telephone interview will achieve a higher response rate during the limited data collection period. Sponsors will be randomly assigned to either the parent incentive stratum or no incentive stratum. Parents of children in sampled FDCHs from sponsors assigned to the parent incentive stratum will receive the $10 incentive payment for completing the telephone interview. Parents in the “no incentive” stratum will not receive the $10 incentive.

10. Assurances of Confidentiality

The data collection efforts that are the focus of this clearance package will be conducted in accordance with all relevant federal regulations and requirements, including the Privacy Act (5 USC 552a). Data to be collected will not be released with individual child, parent, day care provider, or sponsor identifiers. Data will be presented in aggregate statistical form only. A statement to this effect will be read to parents/guardians before administering the telephone survey. All MPR interviewers will be knowledgeable about confidentiality procedures and will be prepared to describe them in full detail, if needed, or to answer related questions raised by respondents. Respondents will be assured that all information identifying them or their day care program will be kept private to the extent permitted by law.

The following safeguards are routinely employed by MPR to carry out confidentiality assurances:

• All employees at MPR sign a confidentiality pledge (Appendix E) that emphasizes the importance of confidentiality and describes their obligations.

• Access to sample selection data is limited to those who have direct responsibility for providing the sample and maintaining sample locating information. At the conclusion of the research, these data are destroyed.

• Identifying information is maintained on separate forms and files, which are linked only by sample identification numbers.

• Access to the file-linking sample identification numbers with the respondents’ identification and contact information is limited to a small number of individuals who have a need to know this information.

• Access to the hard copy documents is strictly limited. Documents are stored in locked files and cabinets. Discarded material is shredded.

• Computer data files are protected with passwords and access is limited to specific users. With especially sensitive data, the data are maintained on removable storage devices that are kept physically secure when not in use.

• MPR will make certain that all surveys are held strictly confidential, as just described, and that in no instance will responses be made available except in tabular form. Under no condition will individual information be made available to sponsors or FNS staff.

11. Additional Justification for Sensitive Questions

• There are no questions of a sensitive nature included in the parental interview questionnaire.

12. Estimates of Hours Burden

Table 1 reports estimates of burden hours for respondents. The study will administer a survey to parents in Method 1. Sponsors will be asked to provide administrative records for the limited number of FDCHs involved in the study. FDCH providers will also be asked to provide copies of any administrative records they maintain.

TABLE 1

ESTIMATES OF BURDEN HOURS FOR RESPONDENTS

Affected Public	Description of Collection Activity	No. of Respon- dents	No. Responses Per Respon- dent	Total Annual Responses	Hrs. Per Response	Total Burden
Sponsors	Administrative records - Method 1	5	4.0	20.0	.75	15.0
	Administrative records - Method 2	5	2.0	10.0	.75	7.5
	Letter to FDCH from sponsor	5	1.0	5.0	.5	2.5
	Sponsor Burden	5	7.0	35.0	.71429	25.0
Individual/ Household	Parent/Guardian interviews -complete	130	1.0	130.0	.25	32.5
	Parent/Guardian interviews -declined/ ineligible	60	1.0	60.0	.08	4.8
	Individual/ Household Burden	190	1.0	190.0	.19632	37.3
FDCH Providers	FDCH Provider Interviews	30	1.0	30.0	.25	7.5
	FDCH Burden	30	1.0	30.0	.25	7.5
		No. of Respon-dents	Estimated Annual Responses Per Respondent	Estimated Total Annual Responses	Estimated Hours Per Response	Est. Total Annual Burden for this ICR
Summary of Burden for this Collection		225	1.1333	255.0	.2737	69.80

A total of 69.8 burden hours are estimated for the study. The data collection effort will take place during 2008.

13. Estimates of Cost Burden to Respondents

There are no additional respondent costs associated with this data collection.

14. Estimates of Annual Costs to the Federal Government

The total estimated cost of the study is $425,000 over a 12-month period which includes the costs associated with the contractor conducting the project and the salary of the assigned FNS project director. This includes site selection, forms development, data collection, analysis, and report writing.

15. Reasons for Program Changes or Adjustments

This is a new project; it will add 69.8 hours to the OMB collection inventory.

16. Plans for Tabulation and Publication of Results

MPR will use the pilot test evaluation to recommend a methodology for a national study of erroneous payments in the CACFP. The recommendation will be based on the quantitative and qualitative assessment of the potential for the methodology to validate meal claim reimbursements reliably and the costs of implementing the methodology. The recommended methodology will be designed such that it has a high probability of validating meal reimbursement claims submitted by day care providers. The recommendation will include a detailed discussion of the procedures necessary to implement the methodology and the practicality and feasibility of following the proposed procedures. In addition, if the pilot evaluation illustrates the need for a policy or regulation change to increase the evaluation’s ability to validate meal claims, the proposed change(s) and reasons for the change will be explained in detail. Finally, it will include a cost projection for implementing the methodology nationally. MPR will communicate its assessment of the pilot test and recommendations for the preferred method through a briefing with the FNS and preparation of a final report.

Study Schedule

Conduct Pilot Test of Data Collection Methods 8/4-10/3/08

Briefing on Results of Pilot Test 11/14/08

Draft Report 12/1/08

Revised Report 1/2/09

Final Report 2/2/09

17. Approval to Not Display the OMB Expiration Date

The study will display the OMB expiration date.

18. Explanation of Exceptions

No exceptions to the certification statement are being sought.

1We will also record the number of infants, since food brought from home was reported mainly by providers with infants in their care: 46 percent of providers with infants reported that some parents sent food, compared with just 8 percent of the providers with no infants enrolled (Zotov et al. 2002). In the same study, 6 percent of parents with children in CACFP family day care reported sending food from home with their child, primarily “to provide something as backup” (Crepinsek et al. 2002).