Section_A__2-15-2012

SECTION A

Child and Adult Care Food Program (CACFP)

Improper Payment Meal
Claims Assessment

(OMB No.: 0584-NEW)

Project Officer: Fred S. Lesnett

Office: Office of Research and Analysis

Food and Nutrition Service

Room 1014

3101 Park Center Drive

Alexandria, VA 22302

Telephone: 703-605-0811

Fax: 703-305-2576

E-mail: [email protected]

January 26, 2012

Contents

______________________________________________________________________

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary 1

2. Purpose and Use of the Information 7

3. Use of Information Technology and Burden Reduction 8

4. Efforts to Identify Duplication and Use of Similar Information 9

5. Impacts on Small Businesses or Other Small Entities 9

6. Consequences of Collecting the Information Less Frequently 10

7. Special Circumstances Relating to the Guideline of 5 CFR 1320.5 10

8. Comments in Response to the Federal Register Notice and Efforts to

Consult Outside the Agency 11

9. Explanation of Any Payment or Gift to Respondents 13

10. Assurance of Confidentiality Provided to Respondents 13

11. Justification for Sensitive Questions 16

12. Estimates of Hour Burden Including Annualized Hourly Costs 17

13. Estimates of Other Total Annual Cost Burden to Respondents

or Record Keepers 22

14. Annualized Cost to Federal Government 22

15. Explanation for Program Changes or Adjustments 23

16. Plans for Tabulation and Publication and Project Time Schedule 24

17. Reason(s) Display of OMB Expiration Date is Inappropriate 26

18. Exceptions to Certification for Paperwork Reduction Act Submissions 26

Supporting Legislation: IPIA of 2002

Federal Register Notice

Appendix a: State Recruitment and Data Request Materials

Appendix b: Sponsor Recruitment and Data Request Materials

Appendix C: Family Day Care Home Recruitment and Data Request Materials

Appendix D: family day care home Onsite Observation Form

Appendix e: Parent-Recall Interview Questionnaire

Appendix F: Child and adult care food program Provider Child Enrollment Data Abstraction Table

Appendix G: Child and adult care food program Meal Claiming Study Confidentialty Pledge

APPENDIX H: USDA/NASS OBSERVATIONS OF CHILD AND ADULT FOOD CARE PROGRAM (CACFP) IMPROPER PAYMENT MEAL CLAIMS ASSESSMENT......................

Section A: Justification

A.1 Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

This is a new data collection. The Improper Payments Information Act (IPIA) of 2002 (Public Law 107-300) requires the Food and Nutrition Service (FNS), U.S. Department of Agriculture (USDA), to provide estimates of erroneous payments in the Child and Adult Care Food Program (CACFP), and to identify and report corrective actions the agency is taking to reduce them. These measures are necessary to enhance the accuracy and integrity of Federal payments in the CACFP. The CACFP reimburses family day care homes (FDCHs) for serving nutritious meals and snacks to children, especially low-income children who receive day care services in these facilities. To receive Federal funds, each FDCH provider must make reimbursement claims for the meals they serve to eligible children. Meal claiming errors occur if the claim is inaccurate or if the claim is correct but the wrong level of reimbursement is made. USDA widely considers the FDCH component related to meal claiming for reimbursement to have a potential vulnerability to improper payments.

This study uses a two-stage approach, as stipulated by the agency, to determine the feasibility and accuracy of the parent-recall interview method for validating meal claims in a limited study (Appendix E presents the data collection instrument for the parent recall interview). The parent recall method consists of a telephone interview with the parent or primary guardian of a child enrolled in an FDCH to retroactively ascertain whether the child was present for a specific time period (e.g. a week) and what meals the child was served while in attendance at the FDCH. The first component of the two-stage approach is a feasibility study that will be implemented with a subsample of the national sample developed for this study. The feasibility study has two components, a validation of the parent-recall interview method, and a limited testing of the reliability and accuracy of the method for identifying errors in meal claim records. If the methods are found to be reliable, feasible and can be implemented in a cost effective manner, USDA/FNS will then authorize a full national study that will use data collected in the first phase and collect additional data from the balance of the national sample selected for the study. The overview of the data collection activities are presented below.

Feasibility Study Phase 1: Validation of the Parent-Recall Interview Method

The objective of the Phase 1 feasibility study is to validate the parent-recall interview method as a means for indicating the number and type of meals served to a child attending a FDCH. The parent-recall interviews will be validated by conducting onsite observations of meals served at sampled FDCHs. The intent is to use the observations to validate the number and type of meals that are being served to their own children as parents report through the parent-recall interviews.

The parent-recall interview consists of a 12-minute telephone interview conducted with the parent or guardian of a child attending the FDCH, during which the parent or guardian is asked to confirm their child’s attendance and the meals served to the child at the FDCH. The validation of the methodology occurs by comparison of findings based on actual observation conducted in the selected FDCH. In each sampled FDCH, 4 parents, each with 1 child attending the FDCH, will be sampled. The parent recall interview will be conducted within a 5-day window following the onsite observation at the FDCH in order to obtain more accurate recall of the meals the child consumed in the previous week. Conducting the questionnaire via telephone eliminates the extreme lag time in sending a mail version and thus enables more accurate recollections on the part of the parent for the week in question. The ability for parents to respond to the parent recall interview within the 5-day window following observations is critical to enable more accurate cognitive recall of activities that occurred the previous week. Conducting the questionnaire via telephone also positively impacts the likely response rate and minimizes costs for conducting the feasibility study, as mail questionnaires have a very low response rates and follow-up for missing data is expensive. The parent recall interview asks parents to provide information on the sampled child that is enrolled in day care, but to also verbally provide the names, and ages of other children in the household. Parents are being asked this information in the household interview to confirm the de-duplication of the provider lists the contractor will have received from FDCHs and to confirm de-duplication of children in comparing the onsite observation findings. The pretest revealed that the provider list of enrolled children does not provide a clear method for linking children who are siblings but have different last names from each other or even the parent or legal guardian.

Onsite observation at the FDCH will take place over a 2-day period during a target week to observe the children in attendance being served meals that differ by type and reimbursement rate associated with the meals. At the end of the target week, the 4 sampled parents will be contacted and the parent-recall interviews will be conducted. Analysis of the findings will substantiate whether the parent-recall interview method is feasible for detecting erroneous meal claims that have been submitted by the FDCHs.

Feasibility Study Phase 2: Limited Data Collection Testing Use of the Validated Parent Recall to Identify Erroneous Meal Claims

If the parent-recall interview method is validated, FDCH meal claim records will be obtained from sponsors and will be analyzed to compare the parent recall with the submitted claims. The analysis will assess whether the approach produces a relatively reliable estimate of the error rate in meal claims. It will identify the type and source of errors, and will develop preliminary overpayments and underpayments estimates that specify the cost implications of these errors.

Approaches to Determining the Validity of the Parent Recall Method

Assuming that the onsite observation offers the "true" number of meals served, the discrepancies between the parents recall of meals and the observed meals will indicate parent recall errors. We will construct two data files, one for the parent recall records (collected via telephone interview) and another for the observation records. In both files, each record, one for each meal, will uniquely identify the meal with the following variables:

• Sponsor ID

• Provider ID

• Child ID, identification of sampled children served by the provider

• Date of meal served, quantified (MM/DD/YY).

• Meal type, a six-category indicator (breakfast, AM snack, lunch, PM snack, supper, evening snack)

The analysis calls for the use of a binary meal service indicator showing whether the uniquely specified meal is delivered. The two data files will differ, however, in covering the meals served: Onsite observation data will cover only 2 days of meal service, for 2 meals served on each day, whereas the parent recall data will cover 1 week of meal service. Parent recall records of meals not covered by observation will be deleted from the file. The resulting two datasets should contain the same number of meal records with same variables that may have different values on one or more variables.

With the extra records deleted, the parent recall file will be merged with the observation file by sponsor, home, child, date, and meal type. A crosstab of the parent recall meal service indicator and observation meal service indicator (both are dichotomous) will generate the counts and rates of false positive (meals recalled by parents but not observed) and false negative recalls (meals observed but not recalled by parents), as well as the total erroneous recall. Given an acceptable error rate (e.g., 10 percent), a t-test will be used to determine whether the parent recall method produces an error rate that is statistically significantly lower than the given level of acceptability based on the sample data. With results from the parent recall and observation data analysis, the agency will judge the parent recall validity by considering the statistical criteria below:

• The total meal records mismatch (for all meal types) rate is smaller than 10%;

• The total “false positive” rate is lower than 20% (more likely to occur as suggested by anecdotal sources);

• The total “false negative” rate is lower than 5%; and

• Between parent recall and observation, the average meal counts per child (all meal types) are not significantly different at p < .05 level and the average meal counts for lunch and supper (more expensive) per child are not significantly different at p < .01 level.

With the 2X2 contingency table, Phi coefficient will also be generated for the magnitude of the agreement between parent recall and observation. Commonly used in measurement studies, Phi measures inter-rater reliability (in a 50/50 split, values range from −1 to +1) by taking into consideration the fact that by chance the two rates may be in agreement to a fairly large extent. The agency will consider a Phi > .60 as the minimal requirement to accept the parent recall method.¹ McNemar’s test will also be used to decide whether to reject a null hypothesis that the row and column marginal frequencies (followed by false positive and false negative rates in cells b and c) are equal.² If the test does not reject the null hypothesis, then the parent recall measure will be deemed as equivalent to the observation measure.

The final decision on acceptable error level for using the parent recall method in the national study should be based on the agency’s program policy considerations as well. For example, the extent to which sponsors identify errors and reject meal claims in practice may be a reference for establishing an error tolerance level. Measurement reliability benchmarks used in similar government programs evaluations or relevant academic research could be applied as well.

Power Analysis. The feasibility study is essentially a measurement study to compare meal counts by parent recall vs. onsite observation. Since national representativeness is not required, the statistical precision is for detecting between-group differences. Power analysis thus for this analysis is proxy of that used for statistical comparison of means or percentages by groups. The data collected will be in a hierarchical structure where meals are nested within children, children within homes, and homes within sponsor. With a focus on parent (equivalent to child) meal recalls, the study design will use a two-level design for power analysis: meals at level 1 and children at level 2³. As proposed, with 512 parents whose children to be served an average of two meals per day in two days that are covered by the onsite observation, there will be a total of 2,048 meals. With a difference in cell percentage between cell b and c in the 2X2 crosstab assumed as .15 (for McNemar’s test) and with two hypothetic levels of intra-class correlation, the study should be able to achieve sufficient power:

• With an intra-class correlation of .02, the power would be about .90.

• With an intra-class correlation of .04, the power would be about .80.

Full National Data Collection

Based on the findings of this limited study, USDA/FNS will determine whether to move forward with a full national study. This national study will use the balance of the sample frame to continue the parent-recall interview data collection activities and the comparison of additional submitted meals claims from sampled FDCHs. Analysis of these data will be used to develop national meal claim estimates of improper payments for the CACFP.

The sampling approach for the study has been developed to consider the potential ramifications of how the potential for pooling of data could be impacted if the feasibility study results in major changes to study procedures, design or content for the main study. In the event that the study methodology is deemed adequate, but the selected sample is deemed contaminated, either by the observation or by a need to revise the data collection procedures, one of the possible options would be to select a new sample for the main study from within the eight states selected for the feasibility study. In order to facilitate sample selection in that eventuality, a second sample from the same states, but with no overlap with the feasibility study sample will be selected as a shadow sample. This sample will not be contacted, but will be a reserve sample, in that the replacement process for the feasibility study will avoid selecting any homes from the shadow sample.

This request is for clearance of both the feasibility and national study. The agency has considered the possibility that the feasibility study may result in some changes to study procedures even if changes are minor. The agency is requesting approval for this data collection request under the assumption that no major design changes will occur after initial approval, but that minor revisions to data collection protocols will need to be submitted for a 30 day public comment period and OMB approval prior to initiating the main study. If the feasibility study results in major design changes, the agency is aware that a new data collection request will be needed.

A.2 Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

Information gathered from the data collection in Phase 1 of the feasibility study will be used to provide USDA/FNS with a comparison of the relative strengths and weaknesses of the validated parent-recall interview methodology and indicate whether it is feasible to be used to validate meal claim reimbursements at a national level. In Phase 2 of the feasibility study, the information from the parent-recall interview will be compared with sponsor-reviewed meal claims records to provide USDA/FNS with estimates of overpayment and underpayment in the program. This information will then guide the agency in determining whether to execute the full national study if the methodology is found to be sound. Data from the feasibility study and national study will then be used by USDA to fulfill its IPIA reporting requirements for the CACFP. In addition to producing the required estimates of overpayment and underpayment in the program, the data provided will inform the agency about the key areas of potential threat to erroneous payments and document the nature of these administrative errors, thus providing descriptive data that can be used for program improvement.

A.3 Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

FNS is committed to complying with the E-Government Act, 2002 to promote the use of technology. The data collection plan was designed to obtain reliable information in an efficient way that minimizes respondents’ burden. Existing data sources will be requested in electronic format from sponsors when possible, as electronic submissions limit the burden associated with data collection. The data collection also provides automated means for FDCHs to submit their data when feasible, either through the use of e‑mail, fax, mail, or via a telephone abstraction interview.

ICF International also uses automated technologies to collect observation and parent-recall interview data. These methods will allow automatic validation and transmittal of the data collected. Respondents cannot submit data electronically.

A.4 Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in
Item 2 above.

There is no duplication of the data to be collected in this study. Every effort has been made to avoid duplication. FNS has reviewed USDA reporting requirements, state administrative agency reporting requirements, and special studies by other government and private agencies. There is no similar information already available, hence the reason for the study.

A.5 If the collection impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden.

The study gathers information from sponsor organizations, FDCH providers, and parents. The FNS contractor, ICF International, has minimized the participation burden on these entities by designing the data collection procedures to gather only the vital information required to perform the study. Data gathering and transmittal burden have been minimized for sponsors by offering participants alternative methods (e-mail, fax, mail) for submitting the requested data.

Specifically for FDCHs, the data abstraction procedures enable day care providers to choose the transmittal method (e-mail, fax, mail, or telephone) that is most convenient for them. The FDCH in-home observation visits, which will be conducted by specially trained field specialists hired by the contractor, have been structured to minimize burden by following procedures stipulated by USDA for oversight of FDCHs and by collecting the minimal information required for the study. Of the 64 Sponsor organizations participating in the study, it is expected that after sample selection, 21 Sponsor organizations will be considered small entities. All of the 256 FDCHs to be selected in the study sample are considered small entities.

A.6 Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

If these data are not collected, USDA/FNS will be unable to determine if it is feasible to use the parent-recall methodology for identifying meal claiming errors in the CACFP. No assessment of the cost, amount and type of meal claiming errors can be estimated nor can corrective actions be developed and implemented. This is a single time study. This data collection is the only opportunity to determine the relative strengths and weaknesses of the proposed methodology and develop estimates that meet the IPIA requirements.

A.7 Explain any special circumstances that would cause an information collection to be conducted in a manner:

• requiring respondents to report information to the agency more often than quarterly;

  • requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• requiring respondents to submit more than an original and two copies of any document;

• requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

• in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

The data collection requests that sponsoring organizations provide 3 consecutive month’s worth of meal claim data for 4 sampled FDCHs as a part of the study. Section B provides details concerning the sampling approach for the study. The meal claim records are the basis for analysis of erroneous claims that are submitted for reimbursement in the study; without the meal claiming records, erroneous claims cannot be identified or documented. These records are extant forms, which serve as the basis for meal reimbursement under the CACFP. They are already required by the program and submitted monthly by FDCHs. As a part of the CACFP guidelines, FDCHs and sponsoring organizations are in the practice of submitting these monthly meal claims to their State agency for reimbursement. Because of these practices, this additional request will not require any additional effort from FDCHs and will require only a minimal effort from the sponsoring organizations selected for the study.

In the feasibility study after the validation of the parent recall method, the analysis will use the 3 months of meal claim data to check the meal claim records against parent recall records for the target week to identify erroneous claims and generate improper payment error estimates. Another purpose for the 3-month claim data collection is to examine possible impact of the feasibility study on the meal claim behavior of FDCHs. FDCHs awareness of this study may lead to changes in their meal-claiming pattern, (e.g., reducing the number of meals submitted for reimbursement). An assessment of such potential bias is needed to justify the feasibility study where claim data will be checked against parent recall records (if validated).

There are no other special circumstances. The collection of information is conducted in a manner consistent with the guidelines in 5 CFR 1320.5.

A.8 If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported. Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years—even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

A8A. Federal Register Notice: A notice was published on Friday, March 11, 2011 in Volume 76, No. 48, p. 13339.

Public Comments Received on the Notice: No comments were received as of 6/15/11.

A8b. Consultation with Persons outside the Agency: ICF International consulted with 4 sponsoring organizations and several of their respective monitoring staff on several aspects of study data collection procedures (Exhibit A.1). The respective individuals were consulted in regards to the availability of information required for the study at the sponsor organization and FDCH levels, the amount of effort that would need to be expended by FDCH providers to supply the information requested, and the clarity of the instructions given to collect the information at both levels. Sponsoring organizations were selected from States that were excluded from the study sample. The USDAs NASS was also consulted for the Statistical Methodology and comments (Appendix H) were taken into consideration.

Exhibit A.1. Sponsoring Organizations Consulted for the Study

State	Sponsoring Organization	Contact
Washington, DC	United Planning Organization–Family Day Care Homes (UPO-FDCH) 301 Rhode Island Ave, NW Washington, DC 20001	Yasmeen Abdul-Shakur (202) 238-4632
Maryland	Baltimore City Health Department 4 S. Frederick Street, Second Floor Baltimore, MD 21202	Jacqueline Gowans Maultsby (410) 396-4240
Virginia	Fairfax County Office 12011 Government Center Parkway Fairfax, VA 22035	Abeba Tzeggai (703) 324-8018
	Infant Toddler Family Day Care 11166 Fairfax Boulevard, Suite 206 Fairfax, VA 22030	Marisela Morales (703) 352-3449

A.9 Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

The study is planning to offer a stipend up to $75 to FDCH providers, to reimburse them for costs that might be incurred as a result of their participation. The stipend will be offered to defray time and material costs in gathering and providing the documentation required for the study, specifically to offset costs incurred by using personal in-home equipment to make copies and fax the documents required for the study. The stipend is offered with the understanding that copies will be reimbursed at 10 cents each and reasonable faxing charges will be covered. The study had also planned to offer a $10 response incentive to parents/guardians to complete a 12-minute telephone interview to encourage their participation during the limited data collection window. The agency reluctantly chooses not to offer this incentive, which was being offered to encourage response rates within the four day response window, which includes weekends. The incentive was initially offered to support obtaining more accurate cognitive recall among parents of the previous week of meals their child may have consumed.

A.10 Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

Family day care home (FDCH) providers are being provided with written notification of privacy for the data they provide to the study. They are being asked to provided child enrollment data, which includes identifying information about families receiving care. As a part of this data request, the agency is indicating that these data will be handled privately and that the data to be collected will not be released with individual child, parent, day care provider, or sponsor identifiers outside this data collection, except as otherwise required by law. All respondents, including those in the parent-recall surveys, will be informed that information provided is private and held in a secure manner and will not be disclosed, unless otherwise compelled by law. Furthermore, CACFP sponsors and FDCH providers will be assured that participating in the study will not impact their participation in the CACFP or any benefits to which they are entitled.

ICF International has extensive experience in data collection efforts requiring strict procedures for maintaining the confidentiality, security, and integrity of data. Specific data handling and reporting procedures will be employed to maintain the privacy of survey and observations participants and composite electronic files. These data handling and reporting procedures include requiring all project staff, both permanent and temporary, to sign a confidentiality and nondisclosure agreement (Appendix G). In this agreement, staff pledge to maintain the confidentiality of all information collected from the respondents and will not disclose it to anyone other than authorized representatives of the study, except as otherwise required by law. In addition, ICF International has established a number of procedures to ensure the confidentiality and security of electronic data in their offices during the data collection and processing period.

A system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register on March 31, 2000, Volume 65, Number 63, and is located on pages 17251-17252 discusses the terms of protections that will be provided to respondents. Participants in this study will be subject to assurances and safeguards as provided by the Privacy Act of 1974 (5 USC 552a), which requires the safeguarding of individuals against invasion of privacy. The Privacy Act also provides for the confidential treatment of records maintained by a Federal agency according to either the individual’s name or some other identifier.

Individuals participating in this study will be assured that the information they provide will not be published in a form that identifies them. No identifying information will be attached to any reports. Identifying information will not be included in the public use dataset. Names and phone numbers, or any other unique identifier, will not be linked to the data. Interview and observation respondents will be assigned a unique ID number and analysis will only be conducted on data sets that include these unique ID numbers. Records are kept in physically secured rooms and/or cabinets.  Paper records are segregated and physically secured in located cabinets.  Various methods of computer security limit access to records in automated databases.   Access to records is limited to ICF International staff who process the records for the specific uses stated in this Privacy Act notice.  Records in such formats as magnetic tapes and disks are kept in physically secured rooms and/or cabinets.  Various methods of computer security limit access to records in automated databases (such as file encryption/locking too). Names and phone numbers will be destroyed within 12 months of the end of the contract.

FNS does not have any connection to the personal data collected and the only information FNS handles is the aggregated report which contains no personal info and is publicly posted. Data will be presented in aggregate statistical form only.

 Institutional Review Board

ICF International’s Institutional Review Board (IRB) serves as the organization’s administrative body; it conducts prospective reviews of proposed research and monitors continuing research for the purpose of safeguarding research participants’ rights and welfare. All research involving interactions or interventions with human subjects is within the purview of the ICF International’s IRB. ICF International’s IRB is the local agent responsible for ensuring that the organization’s research: 1) meets the highest ethical standards; and 2) receives fair, timely, and collegial review by an external panel. ICF International’s IRB currently holds a federal-wide assurance (FWA) of compliance from the U.S. Department of Health and Human Services’ Office of Human Research Protections (DHHS/OHRP). The FWA covers all federally supported or conducted research involving human subjects. All study materials and instruments were submitted and approved by ICF International’s IRB. Copies of the IRB approval letters are in Appendix G.

Privacy Impact Assessment

Discussions were held with FNS’s FOIA officer (Jennifer Weatherly of FNS’ Information Management Branch) and the Department’s Privacy Office regarding OMB’s question regarding the need for a Privacy Impact Assessment (PIA). Consensus was reached by FNS’ Privacy Office that a PIA was not required for this data collection package request because FNS staff will never handle or see any of the personal data collected and ICF International’s (the contractor) system does not tie into any of FNS’ data management/analysis systems.  Also, ICF International data creation and processing system was not created for this contract agreement. Therefore, in accordance with OMB M-03-022 no PIA is required when collecting non-identifiable information (in this case would be the report) for a discrete purpose.

A.11 Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

This data collection effort does not include any sensitive questions.

A.12 Provide estimates of the hour burden of the collection of information. The statement should:

• Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

• If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

• Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 13.

Estimates of Burden:

The pretest, described further in Section B.4, is the source for the estimates of time required for the data collection. Non-response rates were calculated based on participation rates in prior studies with similar sample populations and on non-response in the pretest.⁴

State Agencies: State agencies are expected to complete a review of the recruitment request and a data abstraction interview. Public burden is estimated at 60 minutes per response for State review and data collection in response to the contact letter for 16 respondents for total burden of 16 hours. It is anticipated that subsequent follow-up activities to clarify data will be conducted with a subset of States. Ten States are expected to complete an initial interview to confirm data that is 10 minutes in length for a total burden of 1.6 hours. Six States are expected to complete the follow up interview, which is estimated at 30 minutes, for a total burden of 3 hours for respondents, while 4 States are expected to complete a supplemental interview to clarify any missing data that is 20 minutes in length for a response burden of approximately 80 minutes or 1.32 hours of response burden. Appendix A includes the data collection request materials for States.

Sponsors: Sponsors will complete a data abstraction interview, a sampling interview, an observation scheduling interview, and provide 3 months of administrative meal claim data. Sixty four sponsors will participation in a data abstraction interview and data request that has duration of 120 minutes or 2 hours, for a total of 128 hours of public burden. As with States, it is expected that a subset of sponsors will participate in follow-up activities to clarify data or obtain missing information. Fifty sponsors are expected to complete a confirmation interview that lasts 9-10 minutes, for 480 minutes or 8 hours of respondent burden. Fourteen sponsors are expected to complete the follow-up interview, which is estimated at 30 minutes, for a total burden of 420 minutes or 7 hours for respondents, while 7 sponsors are expected to complete a supplemental interview to clarify any missing data that is 20 minutes in length for a response burden of approximately 138.6 minutes or 2.31 hours of response burden.

Sixty four sponsors will complete a sample notification and data request interview and related data collection activities that are approximately 120 minutes in duration, for a total burden of 7,680.00 minutes or 128 hours. A subset of sponsor s will receive a follow- up letter for FDCHs that have not complied with the initial data request. It is estimated that 10 sponsors will receive a notification letter that requires 9-10 minutes on average to respond to, for a total burden of 96 minutes or 1.6 hours. Thirty two sponsors will participate in the feasibility study’s onsite observation to determine the validity of the parent recall method. These 32 sponsors will be asked to provide additional data and participate in an observation scheduling interview that is approximately 124 to 125 minutes in duration, for 66.56 hours of total response burden for 32 respondents. Appendix B includes the sponsor data request materials.

FDCHs: Public burden is estimated at 120 minutes or 2 hours per response for 256 FDCHs for completion of the initial data abstraction request, for a total burden of 512 hours for 256 respondents. It is expected that follow-up interviews will be needed to clarify information or obtaining missing information from a subset of FDCHs. 192 FDCHs are expected to complete a data abstraction confirmation interview that is approximately 9 to 10 minutes in length, for a total burden of 1843.2 minutes or 30.72 hours. 5 FDCHs are expected to decline completing the data abstraction request for a total burden of 0.40 hours. A follow-up interview to clarify data received is expected to be conducted with 64 FDCHs; this interview lasts 30 minutes, for a total of 1,920 minutes or 32 hours of response burden. Sixty-four FDCHs are expected to complete a supplemental data interview to follow-up on missing data that is approximately 19-20 minutes in duration for a total of 1,267.2 minutes or 21.12 hours of burden. Appendix C includes the data request instruments for FDCHs.

Parents: Public burden is estimated at 12 minutes for 1 completed interview response, for a total of 9,828 minutes or 163.8 hours per respondent who successfully completes the screener and interview. It is expected that 205 respondents will only complete the interview screener, which is approximately 5 minutes in length for a burden of 984 minutes or 16.4 hours. Appendix E is the Parent Recall Telephone Interview. These are unscheduled interviews and no advance notification letters will be mailed.

Onsite Observation at FDCHs: Public burden is estimated at 2 hours per response, with a total of 4 responses per FDCH, for a total of 128 FDCH respondents, resulting in a total of 1,024 hours of burden. Appendix D includes the Onsite observation form.

Exhibit A.2 presents the Summary Table of response burden for this data collection request.

Exhibit A.2. Estimates of Respondent Burden

Respondent	Data Collection Activity	Estimated Number of Respondents	Responses Annually per Respondent	Total Annual Responses	Estimated Average Number of Hours per Response	Estimated Total Annual Hours of Response Burden
State Agency (overseeing CACFP)	State Agency contact letter[1]	16	1	16	1	16
	State agency Data Abstraction Interview (Initial Completes)	10	1	10	0.16	1.6
	State Agency Follow-up Interview for Missing Data (Attempts to Complete) non-responses	6	1	6	0.5	3
	State Agency Data Interview Supplemental Information	4	1	4	0.33	1.32
SA Subtotal		16		36		21.92
Business (64 Sponsors & 261 FDCH)	Sponsor Recruitment Letter[2]	64	1	64	2	128
	Sponsor Agency Data Abstraction Interview (Initial Completes)	50	1	50	0.16	8
	Sponsor Follow-up Interview for Missing Data (Attempts to Complete)	14	1	14	0.5	7
	Sponsor Data Interview –Supplemental Information	7	1	7	0.33	2.31
	Sponsor FDCH Sample Notification Letter	64	1	64	1	64
	Sponsor FDCH Sample Notification Interview	64	1	64	1	64
	Sponsor Letter for Non-Responsive FDCHs	10	1	10	0.16	1. 60
	Observation Scheduling	32	1	32	2	64
	Sponsor Observation Confirmation Letter	32	1	32	0.08	2.56
	Sponsor 3-Month Meal Claims Data Request	64	1	64	2	128
	FDCH Contact Letter3	261	1	261	2	522
	FDCH Data Abstraction Confirmation Interview	192	1	192	0.16	30.72
	FDCH Follow-up Interview for Missing Data (Attempts to Complete)	64	1	64	0.5	32
	FDCH Data Abstraction Confirmation Interview (Declined) non-responses	5	1	5	0.08	0.4
	FDCH Clarifying Data Interview (Supplemental Information)	64	1	64	0.33	21.12
	FDCH FAQ Letter	256	1	256	0.08	20.48
	Provider Child Enrollment Data Abstraction Table	256	1	256	0.5	128
	On site Observation	128	4	512	2	1,024.00
BUS Subtotal		325		2,011.00		2,246.59
Households	Parent Recall Telephone Interview (Completed)	819	1	819	0.2	163.8
	Parent Recall Telephone Interview (Declined/ineligible) non-responses	205	1	205	0.08	16.4
I/HSubtotal		1,024		1,024.00		180.2
GRAND TOTALS		1,365.00		3,071		2,448.71

The total cost to respondents is $47,787.36. The mean hourly wage rate categories as determined by the Bureau of Labor Statistics, National Industry-Specific Occupational Employment and Wage Estimates, May 2009 release were used for State government, social and community program managers, and child care workers. The cross occupational mean hourly wage rate was used for parents.

Exhibit A.3 Estimated Cost to Respondents

Respondent	Data Collection Activity	Number of Respondents	Number of Responses per Respondent	Estimated Average Number of Hours per Response	Mean Hourly Rate	Cost To Respondent
State Agency (overseeing CACFP)	State Agency contact letter	16	1.00	1.00	$31.57a	$505.12
	State agency Data Abstraction Interview (Initial Completes)	10	1.00	0.16	$31.57	$50.51
	State Agency Follow-up Interview for Missing Data (Attempts to Completes)	6	1.00	0.50	$31.57	$94.71
	State Agency Data Interview –Supplemental Information	4	1.00	0.33	$31.57	$41.67
Sponsors	Sponsor Recruitment Letter	64	1.00	2.00	$24.36 b	$3,118.08
	Sponsor Agency Data Abstraction Interview(Initial Completes)	50	1.00	0.16	$24.36	$194.88
	Sponsor Follow-up Interview for Missing Data ( Attempts to Completes)	14	1.00	0.50	$24.36	$170.52
	Sponsor Data interview (Supplemental Information)	7	1.00	0.33	$24.36	$56.27
	Sponsor FDCH Sample Notification Letter	64	1.00	1.00	$24.36	$1,559.04
	Sponsor FDCH Sample Notification interview	64	1.00	1.00	$24.36	$1,559.04
	Sponsor Letter for Non-Responsive FDCHs	10	1.00	0.16	$24.36	$38.98
	Observation Scheduling	32	1.00	2.0	$24.36	$1,559.04
	Sponsor Observation Confirmation Letter	32	1.00	0.08	$24.36	$62.36
	Sponsor 3-Month Meal Claims Data Request	64	1.00	2.00	$24.36	$3,118.08
FDCH Provider	FDCH contact letter	256	1.00	2.00	$9.33c	$4,776.96
	FDCH Data Abstraction Interview ( Completes)	192	1.00	0.16	$9.33	$286.62
	FDCH Follow-up Interview for Missing Data (Attempts to Completes)	64	1.00	0.50	$9.33	$298.56
	FDCH Follow-up Interview (Declined)	5	1.00	0.08	$9.33	$37.32
	FDCH Data interview (Supplemental Information)	64	1.00	0.33	$9.33	$197.05
	FDCH FAQ letter	256	1.00	0.08	$9.33	$191.08
	Child Enrollment Data Abstraction Table	256	1.00	0.50	$9.33	$1,194.24
Parents	Parent Recall Telephone Interview (Completed)	819	1.00	0.20	$20.90d	$3,423.42
	Parent Recall Telephone Interview Declined/ineligible	205	1.00	0.08	$20.90	$342.76
On site observation	FDCH On site observation	128	4.00	2.00	$24.36	$24,944.64
Total						$47,787.36

^a North American Industry Classification System (NAICS) 999200

^b NAICS 624000

^c NAICS 624000

^d NAICS 00-0000

A.13 Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

• The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life) and (b) a total operation and maintenance and purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and record storage facilities.

• If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of purchasing or contracting out information collections services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

• Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.

There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection

A.14 Provide estimates of annualized costs to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies may also aggregate cost estimates from Items 12, 13, and 14 in a single table.

The annualized cost is $332,075.00 over a 12 month period. The total duration of the study is 3 years. This amount represents the total cost to execute the study and includes the costs for 1) development of instruments, correspondence, and administrative forms; 2) development of the sampling plan and sample selection; 3) development of the evaluation, data collection, and analysis plans; 4) systems programming of the data collection software and tracking systems; 5) study pretest; 6) field interviewer training; 7) sample frame development activities; 8) data collection; 9) data cleaning and processing; 10) data tabulation and analyses; 11) report writing; and 12) overall project management. These costs were estimated by calculating the number of person-hours required conducting the study tasks and by adding the associated other direct costs. The costs presented above represent those associated with the feasibility study for this data collection. Additionally, this information collection also assumes that a total of 20 hours of Federal employee time: for a GS-14, step 5 Program Analyst at $45.32 per hour for a total of $906.40 and Federal staffing cost of $676.87 on an annual basis. Federal employee pay rates are based on the General Schedule of the Office of Personnel Management (OPM) for 2011, for a total cost to the Federal government of $998,254.00.)

A.15 Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-I.

This is new data collection. This program change will add 2,439.91 burden hours to the OMB collection inventory.

A.16 For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The study findings will be presented in a feasibility report and in a final report. The feasibility report will present the viability of the data collection process and the ability of the research design to yield an accurate test of the methodology. The final report will present IPIA estimates of over- and under-payment, as well as the type and severity of errors associated with meal claiming. Detailed technical appendices will be included that fully document all procedures used in the analysis, as well as all data collection forms and instruments, algorithms used for determining error, approaches to estimation, formulas and weighting. In addition to providing information on the strength of the methodology to detect invalid meal claims and subsequently presenting national IPIA estimate of over- and under- payments, the information collected in the feasibility and national studies are intended to inform and assist CACFP program managers at the national, state, and sponsor levels to:

• Assign an appropriate priority level and allocation of resources across areas in CACFP to address improper payments due to invalid meal claims submitted by family day care home providers. The current assessment project will aid in the development of national estimates of the incidence and magnitude of the cost of improper payments as a result of invalid submitted meal claims. Once the magnitude of improper payments due to provider meal claim errors is estimated, the assessment will make information available to assist policy makers and program managers in prioritizing the level of effort for addressing invalid meal claims submitted by providers compared to other sources of improper payment errors, such as tiering level assignment to family day care homes and the nutritional content of meals.

• Assist in identifying specific reasons which result in provider meal claims being invalid. Identifying reasons for invalid meal claims aids in prioritizing and designing monitoring activities by state and sponsor organization staff to focus on sources of provider meal claim errors that can result in invalid claims. State and sponsor organization staff can be provided with information as to how to best allocate their monitoring activities among the potential sources or causes of meal claiming errors which have been associated with family day care home providers.

• Identify distinguishing characteristics of CACFP meal claim providers with a high risk of submitting invalid meal claims and the need to schedule audits by state and sponsor organization monitors, especially if the estimated level of risk exceeds a predetermined level.

• Heighten family day care home providers’ awareness of the parent-recall interview as an effective assessment tool being used by state and sponsor organizations to verify submitted meal claims. Providers will become more cognizant that states and sponsor organizations may employ this method to verify children’s attendance/ receipt of meals and that this will allow for invalid meal claims to be more easily detected. The knowledge that false meal claims can be detected in this manner may lead to less providers submitting false meal claims which will contribute to the Program’s goal of reducing invalid meal claim submissions.

• Focus communications, training efforts and improper payments prevention activities on sponsors which have been identified as having family day care providers with potentially high rates of invalid meal claims.

• Increase awareness among agency and child care association staff, through the use of demonstrated measurements, of the magnitude of the dollar cost of invalid meal claims and its impact on meeting the nutrient needs of children enrolled in family day care homes and CACFP. Heightened awareness may translate into increased support and promotion of education and communication campaigns to lower invalid meal claim submissions.

• Provide information to parents that can be used to generate and strengthen their interest and monitoring of the meals that they expect their children to receive while in family day care homes.

The data collection analysis and reporting study timeline, providing OMB approval is provided when expected, is shown in Exhibit A.4 and the analysis plan is explained in Section B.

Exhibit A.4. Data Collection Analysis and Reporting Study Timeline

Data Collection, Analysis, and Reporting Activities	Timeline
Feasibility Study
Phase 1: Parent Recall Validation Study
Construct Sample Frames and Draw Samples	September 2011–November 2011
State Sample	September 2011
Sponsor Sample	September 2011–October 2011
FDCH Sample	October 2011–November 2011
Schedule FDCH Observations	November 2011–December 2011
Phase 2: Parent-Recall/Meal Claim Data Collection	December 2011–March 2012
Analysis of Validation Study	March 2012–May 2012
Validation Study Feasibility Memo Findings and Reporting	May 2012–June 2012
Full National Data Collection (Required USDA/FNS Approval)
Parent-recall Interview and Meal Claim Abstraction (balance of national sample)	September 2012–November 2012
Meal Claim Analysis	November 2012- January 2013
Final Analysis and Estimation	January 2013–April 2013
Reporting and Presentation	May 2013–June 2013

A.17 If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The agency plans to display the expiration date for OMB approval of the information collection on all instruments.

A.18 Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I. B. Collections of Information Employing Statistical Methods The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results.

There are no exceptions.

1 Conventionally, for highly critical research (life and death), Phi should be above .85; for most social behavioral research, it should be above .70 and not lower than .50. See Criterion-referenced test development: technical and legal guidelines for corporate training. By Sharon A. Shrock, William C. Coscarelli (2007).

2 The null hypothesis of marginal homogeneity states that the two marginal totals for each outcome are the same; i.e., a+b = a + c and c+ d = b + d. Thus the null hypothesis is that b = c. The McNemar test statistic with a continuity correlation is obtained by using the following formula: Chi-square distribution is used in significance test when b and c are large, while binomial distribution is used if b and c are small (Fleiss, 1981).

3 Using software package Optimal Design^?, we performed power analysis to produce the expected sample sizes of parents and meals.

4 U.S. Department of Agriculture, Food and Nutrition Service, Office of Research and Analysis, Child and Adult Care Food Program (CACFP): Improper Payments Data Collection Pilot Project, by Rhoda Cohen, Lara Hulsey, Stacie Feldman, Claudia Gentile and John Hall. Project Officer, Fred Lesnett Alexandria, VA: September 2009.