Appendix L EPICCS Center Sample Selection Memo #2

Appendix L EPICCS Center Sample Selection Memo#2.pdf

Erroneous Payments in Child Care Centers Study (EPICCS)

Appendix L EPICCS Center Sample Selection Memo #2

OMB: 0584-0618

Document [pdf]

Download: pdf | pdf

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

Date:

March 14, 2016

To:

Devin Wallace-Williams

CC:

Chanchalat Chanhatasilpa, Lisa Southworth

From:

Roline Milfort, Ron Klinger, Adam Chu, Mamadou Diallo

Subject:

Selection of Centers for the Erroneous Payments in Child Care Centers Study
(EPICCS) - Contract # AG-3198-C-14-0015

Introduction

This memo is a second follow-up to the memo submitted to FNS on February 22, 2016 on the
selection of centers for EPICCS. The first memo provided an overview of the procedures used to
create the center and primary sampling unit (PSU) frames, and subsequent selection of PSUs and
centers from the universe lists submitted by the States. This prior memo included a summary of a)
the creation of the sample frame for centers and PSUs, b) identification and exclusion out of scope
centers, c) verification or imputation of average daily attendance (ADA), d) creation of PSUs based
on Core Based Statistical Areas (CBSAs) and non-CBSA counties, e) PSU selection, including the
assignment of composite measure of size (MOS) and f) center selection.
This second follow-up memo includes additional details on processes implemented to finalize the
sample frame. Section 2 provides a summary of operational issues and decisions related to data and
logic checks of data files, verification of average daily attendance variable, for profit centers, and
misclassification of center type. Section 3 provides a summary of center selection, with additional
details regarding the process for Tennessee, and a summary of center selection by type of type of
center and PSU.

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

Operational Issues and Decisions

After receiving the data files from the 25 States and the Kokopelli team completed their initial
review, Westat conducted four tiers of additional review as follows:
1.
2.
3.
4.

Project Team Review
Project Team Follow up with States (if needed)
Statistician Review
Project Team 2nd Follow up with States (if needed)

The Westat review included a series of data and logic checks of the data files. In addition, centers
that were out of scope (i.e., at-risk centers, emergency shelters, after school hours) were identified
and flagged. The manual edits were based on additional information received from the State and/or
external sources (e.g., internet searchers, contacting sponsors/centers, etc.), and were tracked in data
edit log files compiled for each State. The most common data edits included:




Adding licensed capacity as a proxy for center ADA when it was missing or equal to zero
Correcting or adding address information
Correcting the center type (sponsored or independent) for misclassified centers



Flagging centers as out of scope

Upon completion of the data checks, other quality control reviews, and manual data edits the State
data files were signed off as being ready for sampling (RS). The biweekly status reports provided a
summary of the status of each State on an ongoing basis. Attachment A includes the final biweekly
status report showing all States as RS. The final report also indicates (by State) the primary and
secondary source of center measure of size and the estimated percent of centers that were deemed
out of scope.
As per the sampling plan, average daily attendance (ADA) was a key variable for sampling as it
provided the measure of size (MOS). States provided different information for this variable.
Attachment B provides a summary of what each State provided for this variable, and what was
eventually used for MOS. In most cases, ADA or licensed capacity was used as the MOS.

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

During our review of the data files received from the States we found that some centers did not
submit any claims during the target month (March 2015). This was primarily due to the fact that
many for-profit centers were not eligible to submit claims in this month because they fall below the
25 percent requirement1. To ensure that the sample was inclusive of the various types of centers that
participate in CACFP, including for-profit centers, we included these centers in the sample frame
even though they did not have claims in the target month. Attachment C is a copy of the memo
submitted to FNS that provided a summary of the issue and the recommendation.
Finally, our review of the data files uncovered that many of the center types (sponsored vs
independent) were misclassified. We found that States varied in their definition of an independent
center. For the purpose of this study, a center is considered a sponsored center if they participate in
the CACFP under the umbrella of a [separate] sponsoring organization that assumes fiscal
responsibility and provides training and monitoring to ensure that its providers comply with all of
the CACFP regulations. This sponsoring organization submits the center’s claims to the State
agency. Independent child care centers act as their own sponsor (i.e., self-sponsor) for the CACFP.
As previously mentioned, a common data edit was correction to the center type. However, it is
likely that we were unable to identify all the misclassified centers. As a result, our recruitment
procedures include verification of the center type, and we update the type if it was misclassified.
Attachment D provides a summary of the States with potentially high number of misclassified
centers due to data discrepancies from the State data files. Recruitment status reports will include a
summary of the misclassified centers.

Center Selection

The PSU and center selections were described in the first memo. This section only provides
additional information with regards to the special situation of the State of Tennessee as well as the
distribution of selected centers by type across the PSUs. As described in the first memo, the centers
across the 50 sampled PSUs were organized into three sub-frames, one for each type of center
(ICCC, SCCC, and HSC). For each type, we selected a total of 300 center-equivalents. In total, 825
distinct centers were selected across 24 States excluding Tennessee. In Tennessee, we selected 10
sponsoring organizations (this step was necessary because the State could not provide the full list of

As per FNS guidelines, for-profit child care centers are eligible to participate in the CACFP program if “…at least 25 percent of the children in care
(enrollment or licensed capacity, whichever is less), are eligible for free or reduced-priced meals or receive benefits from Title XX of the Social Security
Act, and the center receives compensation from funds grated to the States under Title XX. In addition, the center must meet the 25 percent
requirement for every month in which they submit a claim.

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

centers in Tennessee), among which 6 were self-sponsored. We contacted the 10 sponsoring
organizations and obtained the list of all their centers with their associated MOS. Then we selected
one center from each sponsoring organization using PPS method. Table 3.1 provides the counts by
center type and certainty versus non-certainty PSUs including the centers from Tennessee. As
expected, the number of centers in the frame and their measure of size were variable across the 50
PSUs, especially by type of centers. Hence, different numbers of centers were selected per PSU. On
average, the expected number of centers to be selected by PSU for each type was 6. However the
actual number of centers selected per PSU varied from 0 to 19 for ICCC, and 0 to 16 for both
SCCC and HSC. Attachment E provides the number of centers by type for each of the 50 selected
PSUs. Note that this is the total sample including the reserve centers.
Table 3.1 Number of centers by type and certainty status of PSU
Center Type
ICCC
SCCC
HSC
All Types

Certainty PSU
23
25
35

Non-certainty PSU
262
254
236

All PSUs
285
279
271

752

835

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

Attachment A
EPICCS Final Biweekly Status Report on State Data File Request
(Dated: Thursday, 1/21/16)
Overall Summary:
 As of 1/11/16, Westat completed all necessary data checks and other quality control reviews; all
States were thus ready for sampling.
 The next phase involves undergoing a geocoding process to derive the Primary Sampling Units
(PSUs) before obtaining the sampling frame.
 Four (4) States have provided us with licensed capacity values instead of center ADA as their primary
source of the center measure of size.
 Two (2) States have provided us with sponsor ADA instead of center ADA as their primary source
of the center measure of size.
 The table below provides further information on each State.
1. All States are “Ready for Sampling.” After review of the data logic checks from the statisticians, the
project team has approved the data files for sampling.
Table 1. Ready for Sampling
Primary Source of Center
State
Measure of Size
Alabama
Center ADA
Arkansas
Center ADA
California
Licensed Capacity
Colorado
Center ADA
Connecticut
Center ADA
Florida
Licensed Capacity
Georgia
Center ADA
Idaho
Center ADA
Illinois
Center ADA
Kansas
Center ADA
Louisiana
Center ADA
Maryland
Center ADA
Minnesota
Center ADA
Missouri
Center ADA
New Jersey
Licensed Capacity
New York
Center ADA
North Carolina
Center ADA
Ohio
Center ADA
Pennsylvania
Licensed Capacity
South Carolina
Center ADA
Tennessee
Sponsor ADA for March
2015
Texas
Center ADA
Virginia
Sponsor ADA
Washington
Center ADA
Wisconsin
Center ADA

Secondary Source of Center
Measure of Size
N/A
N/A
N/A
N/A
Licensed Capacity
N/A
N/A
N/A
N/A
N/A
N/A
Licensed Capacity
Licensed Capacity
N/A
N/A
N/A
Licensed Capacity
Licensed Capacity
N/A
N/A
Any Sponsor ADA not from
March 2015
N/A
Licensed Capacity
N/A
N/A

A-1

Estimated % of
Centers Out of Scope
21.7%
25.7%
59.8%
36.6%
0%
0%
35.4%
11.8%
32.4%
26.9%
41.8%
36.6%
26.8%
27.3%
40.2%
63.3%
10.8%
25.1%
37.8%
42.2%
19.2%
32.8%
50.0%
26.7%
17.6%

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

Attachment B
AVERAGE DAILY ATTENDANCE VARIABLE


CTATTEND Variable: This column provides clarification on what is contained in the data for the
CTATTEND variable for in-scope centers. The State checklist provides the crosswalk for the
CTATTEND variable name.



Center Measure of Size: This column provides recommendation on what to use for center’s
measure of size.

State
Alabama
Arkansas
California
Colorado
Connecticut
Florida
Georgia
Idaho
Illinois
Kansas
Louisiana
Maryland
Minnesota
Missouri
North Carolina
New Jersey
New York
Ohio
Pennsylvania
South Carolina
Tennessee

CTATTEND Variable
Center ADA
Center ADA
Sponsor ADA
Center ADA
Center ADA
Licensed capacity
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Licensed Capacity
Center ADA
Center ADA
Licensed Capacity
Center ADA
Sponsor ADA

Texas
Virginia

Center ADA
Sponsor ADA

Washington
Wisconsin

Center ADA
Center ADA

B-1

Center Measure of Size
Center ADA
Center ADA
Licensed Capacity
Center ADA
Center ADA
Licensed capacity
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Center ADA
Licensed Capacity
Center ADA
Center ADA
Licensed Capacity
Center ADA
Sponsor ADA /
Number of Centers per Sponsor
Center ADA
Sponsor ADA /
Number of Centers per Sponsor
Center ADA
Center ADA

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

Attachment C
Memorandum on For Profit Centers in EPICCCS Sample
Date:

January 14, 2016

To:

Chanchalat Chanhatasilpa, EPICCS Contracting Officer Representative (COR)

From:

Roline Milfort, EPICCS Project Director

Subject:

For-Profit Centers in the EPICCS Sample

The purpose of this memo is to provide a summary of our discussions, and Westat’s preliminary findings
regarding for-profit centers that participate in the Child and Adult Care Food Program (CACFP).
As per FNS guidelines, for-profit child care centers are eligible to participate in the CACFP program if “…at
least 25 percent of the children in care (enrollment or licensed capacity, whichever is less), are eligible for free
or reduced-priced meals or receive benefits from Title XX of the Social Security Act, and the center receives
compensation from funds grated to the States under Title XX. In addition, the center must meet the 25
percent requirement for every month in which they submit a claim.
During our review of the data files received from the States for EPICCS sampling, we found that some
centers did not submit any claims during the target month (March 2015). This was primarily due to the fact
that many for-profit centers are not eligible to submit claims because they fall below the 25 percent
requirement in certain months. To ensure that the sample was inclusive of the various types of centers that
participate in CACFP, including for-profit centers, we included these centers in the sample frame even
though they did not have claims in the target month.
Consultation with Kindercare Education LLC
Kindercare Education LLC2, one of nation’s largest for-profit early education providers, has three brands that
sponsor for-profit child care centers: KinderCare Learning Centers LLC, Children’s Creative Learning
Centers LLC, and Champions. We consulted with their Director of Subsidy Food Programs for get more
information about for-profit centers’ participation in CACFP. The following is a summary of the information
we learned from our discussions:
1. They have 1,400 centers across the country, and 815 (nearly 60%) are currently approved for
CACFP contracts across 36 States.3 The number that qualifies for CACFP in a given month is
about 800 per month.

Formerly Knowledge Universe until January 3, 2016.

Twenty-two (22) of these States are in the EPICCS sample.

C-1

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

2. For their 815 centers that are currently approved by the State agency to participate in CACFP,
they verify the 25% requirement every month before submitting the claim.
a.

They use the Minute Menu software and scan all attendance and meal count records into
their system. They also manually enter income eligibility information.

b. The system will calculate whether each center meets the 25% threshold.
c. If the center meets the threshold, they move forward with processing the center’s claim.
If it does not, and the State allows for Title XX documentation, they obtain the Title XX
billing for that center’s claim month. If they meet the 25% requirement based on Title
XX, they move forward with the claim.
d. If they do not meet the criteria at all, they do not submit a claim, and try again the
following month.
3. For the centers not currently approved for a CACFP contract, they run an enrollment report to
see which centers are close to the 25% requirement. If they are close, they contact the center
and start an enrollment study based on actual student attendance for the previous month.
4. Typically, 30 – 40 centers are added each year to the CACFP contract (though about 20 centers
are removed each year). Centers are added when they “reliably” meet the 25% requirement, and
thus are unlikely to be become ineligible as a result of small changes in enrollment.
5. Some of the centers participating in CACFP may not be eligible to submit claims for all 12
months of a year. Research was conducted in 2015 to examine this issue. On average, they had
781 centers on CACFP contracts during the twelve month period of this review. During this
review period, 82 different centers (11%) did not qualify over the course of a year. The table
below provides a summary of the 11% who did not qualify for all 12 months:
Table 1. Summary of For-Profit Centers who did not qualify for CACFP for all 12 months of the
review period
Percent
42.68%

Description
35 centers didn’t qualify for 1 month, but they met the 25%
requirement the following month

17.07%

14 centers didn’t qualify for 2 months

14.63%

12 centers didn’t qualify for 3 months

6.10%

5 centers didn’t qualify for 4 months

4.88%

4 centers didn’t qualify for 5 months

14.63%

The remaining 12 centers took more than 6 months to meet the 25%
requirement again.

C-2

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

Preliminary Review of data from EPICCS Sample Frame
Westat’s preliminary review of the EPICCS sample frame found that the majority of the States (23 of 25)
have some type of corporate sponsor(s) for centers in their State. Corporate sponsors typically sponsor forprofit centers, though many for-profit centers are independent centers. Eleven (11) of the corporate
sponsors operate in multiple States. As shown in Table 2, approximately 27% of the centers in the frame are
for-profit centers. Note that this is a rough estimate underestimate because information is not available for
seven States. Table 2 provides a preliminary estimate of the number and percent of for-profit centers in the
sample frame by State and Overall (excluding 7 States in which this data is not currently available). At this
time, the percent that will actually be sampled is unknown as we currently are in the final stages of sampling.
Summary
In summary, we expect that the EPICCS sample will include a reasonable number of for-profit centers, and
some of them will not be eligible to submit claims under CACFP during the month of data collection, and/or
for several months during the study reference year. We recommend including these centers as they represent
the variety of centers participating in CACFP4. However, based on the information gathered from our review
thus far, we expect that a relatively small percentage of for-profit centers sampled for the Study will not be
eligible to submit meal claims during the month of data collection. In these cases, we will request data for the
most recent month in which they were eligible. This information will be documented to track its frequency.

Note that centers that are no longer participating in CACFP at all (i.e. no longer have an active CACFP contract), will
be deemed ineligible.

C-3

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

Table 2 Estimated Number and Percent of For-Profit Centers in the EPICCS frame

State
Alabama
Arkansas
California
Colorado
Connecticut
Florida
Georgia
Idaho
Illinois
Kansas
Louisiana
Maryland
Minnesota
Missouri
New Jersey
New York
North Carolina
Ohio
Pennsylvania
South Carolina
Tennessee
Texas
Virginia
Washington
Wisconsin
Total

Approximate Number
of For-Profit Centers
338
205
unavailable
unavailable
17
2,270
1,152
91
805
unavailable
279
169
122
unavailable
267
612
unavailable
722
1,400
193
unavailable
unavailable
244
369
329
9,584

C-4

Estimated Percent of
For-Profit Centers
45.2%
19.9%
unavailable
unavailable
5.6%
70.0%
88.2%
48.9%
47.7%
unavailable
46.1%
38.8%
25.7%
unavailable
41.5%
29.7%
unavailable
47.4%
47.1%
46.6%
unavailable
unavailable
33.7%
39.0%
40.9%
27.3%

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

Attachment D
STATES WITH POTENTIALLY HIGH NUMBERS OF MISCLASSIFIED CENTERS

State
Alabama

Comments/Notes
All records in the data file had CTSPONSOR = NO.

Arkansas

The State did not distinguish between for-profit independent centers
or sponsored centers.

Georgia

All records in the data file had CTSPONSOR = YES.

Louisiana

The State initially seemed unsure of what sponsored and unsponsored
meant.

Ohio

Nearly all records in the data file had CTSPONSOR = YES. The
State has suggested that this designation is incorrect.

Texas

CTSPONSOR field seemed inconsistent and/or incorrect.

Note: In this table above, we refer to the variable CTSPONSOR. This is the variable in the data file that
indicated whether or not the center had a sponsor. That is, if CTSPONSOR = YES, this indicated it was a
sponsored center. If CTSPONSOR = NO, this indicated it was an independent center.

D-1

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

Attachment E
Number of Sampled Centers by Primary Sampling Units (PSUs) for EPICCS
PSU

STATE

GEOGRAPHICAL AREA

Number of
ICCCs
7

Number of
SCCCs
3

Number of
HSCs
5

Number of
Centers
15

1
2

AL
AL

Birmingham-Hoover, AL
Montgomery, AL

Little Rock-North Little Rock-Conway, AR

Los Angeles-Long Beach-Santa Ana, CA

Riverside-San Bernardino-Ontario, CA

Sacramento--Arden-Arcade--Roseville, CA

San Francisco-Oakland-Fremont, CA

Denver-Aurora-Broomfield, CO

Greeley, CO

Hartford-West Hartford-East Hartford, CT

Crestview-Fort Walton Beach-Destin, FL
Walton Co., FL
Miami-Fort Lauderdale-Pompano Beach, FL

Naples-Marco Island, FL

Palm Bay-Melbourne-Titusville, FL

Tampa-St. Petersburg-Clearwater, FL

Atlanta-Sandy Springs-Marietta, GA

Dalton, GA
Fannin Co. and Gilmer Co., ID (non-CBSA)
Boise City-Nampa, ID
Ontario, OR-ID (ID part)
Washington Co. (non-CBSA)
Pocatello, ID
Caribou Co. and Oneida Co., ID (non-CBSA)
Chicago-Joliet-Naperville, IL-IN-WI

Cape Girardeau-Jackson, MO-IL (IL part)
Paducah, KY-IL (IL part)
Johnson Co. Pulaski Co., and Union Co. (nonCBSA)
McPherson, KS
Massac Co., Ellsworth Co., Marion Co., and
Rice Co.
(non-CBSA)
Baton Rouge, LA

Baltimore-Towson, MD

Duluth, MN-WI
Cook Co., MN (non-CBSA)
Duluth, MN-WI (MN part)
Owatonna, MN
Waseca Co., MN (non-CBSA)
St. Louis, MO-IL (MO part)

E-1

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO#2

PSU

STATE

GEOGRAPHICAL AREA

Number of
ICCCs

Number of
SCCCs

Number of
HSCs

Number of
Centers

Dunn, NC

Lumberton, NC

Winston-Salem, NC

New York-Northern New Jersey-Long Island,
NY-NJ-PA
(NJ part)
New York-Northern New Jersey-Long Island,
NY-NJ-PA
(NY part)
Watertown-Fort Drum, NY
Lewis Co., NY (non-CBSA)
Cleveland-Elyria-Mentor, OH

Springfield, OH

Philadelphia-Camden-Wilmington, PA-NJ-DEMD
(PA part)
Scranton--Wilkes-Barre, PA

Augusta-Richmond County, GA-SC (SC part)
Greenwood, SC
McCormick Co., SC (non-CBSA)
Hilton Head Island-Beaufort, SC

Greeneville, TN
Johnson City, TN
Johnson Co., TN (non-CBSA)
Brownsville-Harlingen, TX

Dallas-Fort Worth-Arlington, TX

Houston-Sugar Land-Baytown, TX

Killeen-Temple-Fort Hood, TX

Midland, TX
Odessa, TX
Pecos, TX
Pecos Co. TX
Waco, TX

Richmond, VA

Kennewick-Pasco-Richland, WA

Spokane, WA

Milwaukee-Waukesha-West Allis, WI

285

279

271

835

All PSUs Combined

E- 2

File Type	application/pdf
File Title	Date:
Author	wilson_e2
File Modified	2016-05-11
File Created	2016-03-14