Section B 6-26-08

Contract No.: GS-10F-0050L

MPR Reference No.: 6328-200

SECTION B

OMB Clearance Package

For

CACFP Improper Payments Data Collection Pilot Project

October 23, 2007

Submitted to: U.S. Department of Agriculture Food and Nutrition Service Office of Analysis, Nutrition and Evaluation 3101 Park Center Dr., Rm. 1014 Alexandria, VA 22302-1500 Project Officer: Fred Lesnett 703-605-0811

Submitted by: Mathematica Policy Research, Inc. P.O. Box 2393 Princeton, NJ 08543-2393 Telephone: (609) 799-3535 Facsimile: (609) 799-0005 Project Director: Rhoda Cohen

CONTENTS

Page

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

2. Statistical Methods for Sample Selection and Degree of Accuracy Needed

3. Methods to Maximize Response Rates

4. Test of Procedures and methods to be Undertaken

5. Individuals Consulted on the Statistical Aspects of the Design

REFERENCES

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

The CACFP Improper Payments Data Collection Pilot Project is designed to have a third-party qualitatively evaluate proposed data collection methods for validating meal claims submitted by family day care homes (FDCHs) in the Child and Adult Care Family Program (CACFP). The study's sample of CACFP sponsors and FDCHs is geographically limited in scope and is designed around a nonprobability sample of sponsors, FDCHs and parents. The contractor will purposively select a sample that includes a maximum of 5 CACFP sponsors and a maximum of 30 FDCHs within 50 miles or a two hour travel radius of MPR offices in New Jersey to minimize travel costs and project costs to FNS.

FNS will identify a State contact in New Jersey, New York and/or Pennsylvania who will inform each CACFP sponsor selected for the study. MPR will call the sponsor to confirm the sponsor’s willingness to participate in all aspects of the study. After the sponsor agrees to participate, the study team will work with the sponsor to identify suitable and cooperative FDCHs to participate in the study. FDCHs will be requested to supply the names, addresses and telephone numbers of children who attend the FDCHs. Parents will be contacted and included in the sample based on their order on the lists from each of the FDCHs. Sampling of parents will be concluded after 130 completed parental interviews have been secured. It is anticipated that a total of 190 parents will need to be contacted with 60 parental interviews not being completed because of refusals or ineligibility.

2. Statistical Methods for Sample Selection and Degree of Accuracy Needed

The study’s goal is to get a preliminary sense of the potential for each of the two data collection methodologies (parental interviews and observation of meals served at FDCHs) to produce valid estimates of meal claim reimbursements to FDCHs in CACFP. The statistical power of the evaluation will be extremely limited and estimates cannot be generated beyond the nonprobability sample of CACFP sponsors, FDCHs and parents. The sample of sponsors, FDCHs and parents will be selected based on their availability and willingness to participate.

3. Methods to Maximize Response Rates

MPR will work to identify and select cooperative sponsors and FDCHs. We, therefore, anticipate high response rates with respect to the administrative data collection (approximately 90 percent). The parent interview is expected to result in a lower response rate of 70 percent. Follow-up phone calls and flexible interview times will be used to ensure that we reach this response rate on the parent interviews during the four-day data collection period. Parents will be assured that all information identifying them or their day care program will be kept private to the extent permitted by law.

In addition, in order to assure a high rate of participation, the study is planning to reimburse sponsors and FDCH providers for some of their costs incurred as a result of their participation in the study. The sponsors will receive a total of $75 to defray some of the cost of providing a monitor from the sponsor’s staff to accompany the MPR observers during their visits to six FDCHs and the cost of clerical staff retrieving enrollment forms and three months of administrative records for each of their FDCHs included in the study. MPR will give FDCH providers $25 to compensate for any additional staff they might require during the hours of observation and provider interview.

We will use an experiment to determine if a $10 response incentive for parents/guardians to complete a 15-minute telephone interview will achieve a higher response rate during the limited data collection period. Sponsors will be randomly assigned to either the parent incentive stratum or no incentive stratum. Parents of children in sampled FDCHs from sponsors assigned to the parent incentive stratum will receive the $10 incentive payment for completing the telephone interview. Parents in the “no incentive” stratum will not receive the $10 incentive.

4. Test of Procedures and methods to be Undertaken

The first phase of the evaluation project consisted of a pretest of the proposed data collection methods which included a preliminary assessment of the methodologies':

• feasibility of being implemented nationwide,

• cost of implementing, and

• reliability and validity in estimating the annual dollar amount of erroneous payments, as required under the Improper Payments Information Act (IPIA).

In April 2007, the pretest involved telephone interviews with 9 parents, on-site observations at 7 family day care homes and the review of parent sign-in/sign-out logs. Interviewers recorded any issues that were raised during the pretest and revisions were considered as appropriate. As more information is obtained about providers and parents, additional revisions may be made as appropriate. Appendix G contains the memo documenting the findings of the initial pretest.

5. Individuals Consulted on the Statistical Aspects of the Design

Dr. Philip Gleason, MPR Senior Researcher, provided feedback on the statistical aspects of the design.

references

Crepinsek, Mary Kay, Linda Ghelfi, and William L.Hamilton. “Households with Children in CACFP Child Care Homes: Effects of Meal Reimbursement Tiering. A Report of the Family Child Care Homes Legislative Changes Study.” U.S. Department of Agriculture, Economic Research Service. E-FAN No. 02-005, 2002. Available on line at
[www.ers.usda.gov/publications/efan02005/efan02005.pdf]

Zotov, Natasha, Shao-shen Keng, and William Hamilton. “Family Child Care Providers in the CACFP—Operational Effects of Reimbursement Tiering: A Report to Congress on the Family Child Care Homes Legislative Changes Study.” U.S. Department of Agriculture, Economic Research Service. E-FAN No. 02-004, April 2002.