Supporting Statement – Part B
for
OMB Control Number 0584-####
Family Day Care Home (FDCH) Participation Study
May 2022
Project Officer: Chanchalat Chanhatasilpa
Office of Policy Support
Food and Nutrition Service
United States Department of Agriculture
1320 Braddock Place
Alexandria, VA 22314
Email: [email protected]
Table of Contents
Part B. Statistical Methods 1
B.1 Respondent Universe and Sampling Methods 1
B.2 Procedures for the Collection of Information 4
Statistical Methodology for Stratification and Sample Selection 5
Degree of Accuracy Needed for the Purpose Described in the Justification 9
Unusual Problems Requiring Specialized Sampling Procedures 9
Any Use of Periodic (Less Frequent Than Annual) Data Collection Cycles to Reduce Burden 10
B.3 Methods to Maximize Response Rates and Deal with Nonresponse 10
B.4 Test of Procedures or Methods 12
B.5 Individuals Consulted and Individuals Collecting and Analyzing the Data 13
Appendices
A1 Richard B. Russell National School Lunch Act
B1 Notification Email from FNS National Office to Regional Office
B2 Notification Email from FNS Regional Office to States
B3 Notification from Study Team to States
B4 Template-Notification from States to Sponsors
B5 Study Introductory Letter for Sponsors
B6a Template-Sponsor Notification of Study to Providers-English
B6b Template-Sponsor Notification of Study to Providers-Spanish
B7a Study Introductory Letter for Providers-English
B7b Study Introductory Letter for Providers-Spanish
B8a Provider Experience Survey – First Reminder—English
B8b Provider Experience Survey– First Reminder—Spanish
B9a Provider Experience Survey – Second Reminder—English
B9b Provider Experience Survey – Second Reminder—Spanish
B10 Sponsor Follow-up Email with Request to Email Providers
B11a Encouragement Email from Sponsors to Nonresponding Providers—English
B11b Encouragement Email from Sponsors to Nonresponding Providers—Spanish
B12a Provider Experience Survey, Second Invitation with Paper Survey—English
B12b Provider Experience Survey, Second Invitation with Paper Survey—Spanish
B13a Provider Experience Survey, Third Reminder—English
B13b Provider Experience Survey, Third Reminder—Spanish Provider
B14a Experience Survey, Script for Telephone Follow-up—English
B14b Provider Experience Survey, Script for Telephone Follow-up—Spanish
B15a Study Brochure with FAQs—English
B15b Study Brochure with FAQs—Spanish
B16a USDA Endorsement Letter—English
B16b USDA Endorsement Letter—Spanish
B17a CACFP Organizations Endorsement Letter—English
B17b CACFP Organizations Endorsement Letter—Spanish
B18a Study Website Crosswalk-English
B18b Study Website Crosswalk-Spanish
B19a Provider Experience Survey—English
B19b Provider Experience Survey—Spanish
B20a Provider Experience Survey, Thank you—English
B20b Provider Experience Survey, Thank you—Spanish
B21a Provider Experience Survey Screen Shots—English
B21b Provider Experience Survey Screen Shots—Spanish
C1 Public Comment 1
C2 Public Comment 2
C3 Public Comment 3
C4 Public Comment 4
C5 Public Comment 5
C6 Public Comment 6
C7 Public Comment 7
C8 Public Comment 8
D1 FNS Response to Public Comment 1
D2 FNS Response to Public Comment 2
D3 FNS Response to Public Comment 3
D4 FNS Response to Public Comment 4
D5 FNS Response to Public Comment 5
D6 FNS Response to Public Comment 6
D7 FNS Response to Public Comment 7
D8 FNS Response to Public Comment 8
E National Agricultural Statistics Service (NASS) Comments
F FNS Response to NASS comments
G Westat Confidentiality Code of Conduct and Pledge
H Westat IRB Approval Letter
I Westat Information Technology and Systems Security Policy and Best Practices
J Burden Table
Exhibits
B-1 Respondent universe, samples, and expected response rates B-2
B-3 Individuals consulted and individuals collecting and analyzing the data B-14
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
The respondent universe for the Family Day Care Home (FDCH) Provider Experience Survey includes FDCH providers who are currently participating in the Child and Adult Care Food Program (CACFP) and FDCH providers who formerly participated in the CACFP within the 50 States and the District of Columbia (DC). The study team will create the sampling frame of providers by comparing lists of FDCH providers participating in CACFP in 2019 and 2022. Providers included only on the 2019 list will be the frame for the former CACFP providers and those on the 2022 list will be the frame for the current CACFP providers.
Exhibit B-1 presents a summary of the universe, samples, expected response rates for each respondent type, and overall response rates. We expect differential response rates between current and former participants, with higher response rates for current participants than for former participants. Based on a response rate of 42 percent for FDCH providers in the Study of Nutrition and Activity in Child Care Settings-I (SNACS-I),1 we have assumed conservative response rates of 35 percent for the current participants and 20 percent for the former participants. The expected combined response rate of 27 percent for the combined current and former sample is based on the assumption that 20 percent of current CACFP providers in the 2022 list will have changed to former CACFP providers between the time the list is obtained and the data collection field period. The study team will use multiple strategies to boost response rates , including sending survey reminders (Appendices B8a, B8b, B9a, B9b, B13a, and B13b), offering the survey in multiple formats (web, paper, and telephone), requesting sponsors to follow up with nonresponding providers (Appendices B10 and B11a-b), telephone follow-up with non-respondents (Appendix B14a and B14b), and pre-and post-survey incentives. These strategies are described in detail in Supporting Statement Part A.
Given the expected response rate of 27 percent for the providers, the study team will conduct nonresponse bias analyses, as described in detail in the response to Question B.3 below.
Respondent |
Universea |
Initial sample |
Minimum expected response rate |
Targeted completed cases |
State CACFP Directors |
51 |
51 |
100% |
51 |
CACFP sponsors |
597 |
597 |
90% |
478 |
FDCH Providers |
92,090 |
5,264 |
27%b |
1,340 |
Current CACFP FDCH Providers |
60,512 |
2,393
– 479c |
35% |
670 |
Former CACFP FDCH Providers |
31,578 |
2,871
+ 479c |
20% |
670 |
a Based on FNS National Data Bank (NDB), last accessed on December 12, 2019 and the list of CACFP providers in Program Year 2019 (OMB# 0584-0613; expiration date: 05/31/2024).
b Overall response rate is computed for the combined sample of current and former providers as (0.35 ×2,393 +0.2×2,871)/ (2,393 + 2,871)= 0.27.
c We assume 20 percent of the current providers in the 2022 list will have switched to former providers between the time the list was obtained and the data collection field period.
The current study will also compare results by key subgroups within the current and former CACFP provider stratum. The key subgroups are tier status (Tier I vs. Tier II), program size (small vs. large), and urbanicity (urban vs. rural).
Tier. Tier status for FDCH providers will be included in the list of CACFP providers. collected from the States under a separate information collection (OMB# 0584-0613; expiration date: 05/31/2024). Tier II providers are much less common than Tier I providers.2 For example, only 4 percent of the FDCH providers on the 2019 list were classified as Tier II. Since there are a small number of Tier II providers compared to Tier I providers, the sampling strata for current and former Tier II providers will be oversampled compared to the current and former Tier I (the sample design is stratified with disproportional allocation in Tier I and Tier II strata).
It is possible that a provider’s tier status may have changed from 2019 to 2022, particularly with the expiration or continuation of the area- eligibility waiver in June of 2022.3 If this is the case, we will make the strata assignment based on the 2022 tier status of the FDCH. In addition, eight percent of FDCH providers were missing tier status in the 2019 provider list. Providers with missing tier status likely will not be included in the study sample unless the 2022 list includes the providers’ updated tier status. The final decision to retain or drop providers without a tier status from the sampling frame will be made once we analyze the 2022 list of FDCH providers.
Program Size and Urbanicity. We will determine the definition of a small versus a large program by examining the distribution of the FDCHs based on the average number of children attending the FDCHs across the sample of current and former CACFP providers. Based on State regulations4 and previous research,5 we expect FDCHs serving more than eight children to be considered large. We will geocode provider addresses and classify providers as urban or rural using the Census definition.6
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection;
Estimation procedure;
Degree of accuracy needed for the purpose described in the justification;
Unusual problems requiring specialized sampling procedures; and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
The study team will collect survey data from FDCH providers who participated in the CACFP any time between 2019 and 2022. The study team will first notify the sponsors of the sampled providers that their providers will be contacted for participation in the study survey (Appendix B5). Sponsors will be asked to encourage their providers’ participation using the provided email template (Appendices B6a and B6b). The study team will send an invitation package via mail and/or email with instructions to complete the survey online (Appendix B7a and B7b), following which we will send two postcard reminders (Appendices B8a, B8b, B9a, and B9b), and a second invitation package containing a paper survey (Appendix B12a and B12b) to nonresponding providers. If providers do not respond to the survey by web or paper, the study team will contact their sponsors with a request to review and update the contact information for nonresponding providers and to send a follow-up email, urging nonresponding providers to complete the survey at their earliest convenience (Appendices B10 and B11a-b). The study team will then conduct telephone follow-up with the nonresponding providers with an option to complete the survey over the telephone (Appendix B14).
We will create four sampling strata, based on a) CACFP participant status (current/former) and b) tier (Tier I/II). See Exhibit B-2.
Stratum |
CACFP participant status |
Tier |
1 |
Current |
I |
2 |
Current |
II |
3 |
Former |
I |
4 |
Former |
II |
Prior to selecting the sample, the providers in each of the four sampling stratum will be sorted by State, followed by the sponsor. Sorting by State will create implicit strata to ensure geographic representation across States. Sorting by sponsors will ensure that sponsors with fewer providers are included in the sample and that the sampled providers are not all from the same sponsors.
To implement sorting by sponsor and State, all providers included in the sampling frame must have the corresponding information regarding their sponsor and State. However, an analysis of the 2019 CACFP provider list indicated that approximately 2 percent of providers (n=1,866) were missing the sponsor information and all of these providers were from Minnesota. To avoid excluding Minnesota from sample selection, these providers will be retained in the frame and will be sorted only by State.
The sample will be allocated to produce approximately an equal number of survey responses for current and former participants to estimate differences between these two groups. To obtain 670 completes per participant group—or 1,340 total completes—we will select a sample of 5,264 providers. The proposed sample design will allow us to produce national-level estimates of current and former participants at ± .04 level of precision, with a .95 level of confidence for each group. In addition, the design allows for comparisons between current and former participants at ± .06 level of precision and between the key subgroups (i.e., tier, program size, and urbanicity) within current and former participants at ± .10 level of precision.
The sample size is larger than would be required with simple random sampling for two reasons:
Change in CACFP participation status: Some providers identified as current CACFP participants in 2022 may cease participation between the time the list was obtained and the data collection field period. Therefore, for analytic purposes, the self-reported CACFP participation status will be used to derive the final CACFP participation status. Based on the recruitment and response rates in SNACS I, we have assumed about 20 percent of the sample of current participants will switch to the former status (see Exhibit B-1),7 which increases the variance of the estimates because the current and former participants are sampled at very different rates. Therefore, we assume a design effect of 1.25 to account for the switch from current to former participant and differential weighting adjustments. This design effect requires a 25 percent larger sample than the one needed for simple random sampling.
Differential response rates: We expect differential response rates between current and former participants, with higher response rates for current participants than for former participants. Based on a response rate of 42 percent for FDCH providers in SNACS I,7 we have assumed conservative response rates of 35 percent for the current participants and 20 percent for the former participants. To account for these differences, we will apply weighting adjustments. These adjustments will also decrease the precision of the estimates.
Applying these assumptions to the desired number of completes (i.e. target sample sizes) translates to a starting sample size of 2,393 current participants and 2,871 former participants. A reserve sample of 100 percent of the primary sample will be drawn and set aside for use if needed.
We will design the weights to produce nationally representative estimates of the current CACFP FDCH provider population and the former CACFP FDCH provider population. The weights will reflect the differential probabilities of selection and compensate for survey nonresponse. The base weight for a sampled provider is equal to the reciprocal of the probability of selecting the provider from the sampling frame. Next, for each provider weight, we will adjust the provider base weights for nonresponse. We will make nonresponse adjustments within groups of providers referred to as “weighting classes” that are internally homogeneous with respect to response propensity. To construct the weighting classes, we will conduct a search analysis for cells that are homogeneous in response propensity. Once the weights are adjusted for nonresponse, we will examine if there are weights larger than 3.5 times the median of the nonresponse adjusted weights within the sampling stratum. These large weights will be trimmed to this cut-off value, and the excess weight will be distributed among the weights of the remaining cases within the sampling stratum, so the sum of weights remains the same after trimming. We will carry out poststratification to adjust the provider weights to frame-based control totals, to reduce variability. Poststratification cells will be based on factors such as urbanicity, provider size, etc. Westat has a proprietary SAS macro RAKE-TRIM, which carries out raking and trimming simultaneously.
To adjust for nonresponse bias, we will employ nonresponse adjustments to offset differences in response propensity. This will prevent systematic differences in response propensity that may cause biases in the estimates. We will use a cell-based approach where weighting cells that are heterogeneous in response propensity are developed based on an analysis of response propensity, with weighting adjustments equal to the inverse of weighted response rates within the selected weighting cells.
The goal of the study is to produce estimates of the differences of current and former providers with a margin of error of 6 percentage points. The number of complete interviews required for detecting these differences is 670 completes per participant group or 1,340 total completed interviews. Since the design is stratified with disproportional allocation by type of participant (current/former) and tier with an assumed design effect of 1.25 to account for the switch from current to former participant, 35 percent response rate for current providers and 20 percent response rate for former providers, and differential weighting adjustments, we expect to draw a sample of 5,264 providers. We also expect this sample to produce estimates for subgroups such as tier, urbanicity, and program size and others at ± .10 level of precision.
There are no unusual problems that require specialized sampling procedures for the current and former CACFP participants. However, it is important to note that this is the first study to recruit and survey former CACFP participants and there is uncertainty on the sample size as well as the assumed response rate for this group. If the assumed response rates are not achieved, the reserve sample will be used to achieve the targeted number of completes.
This is a one-time data collection effort during FY 2023. Concern regarding the periodicity of data collection cycles is not applicable.
Describe methods to maximize response rates and deal with issues of nonresponse. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.
As reported in Exhibit B-1, response rates are expected to be low for this study. Specifically, we expect a 35 percent response rate among current CACFP participants and a 20 percent response rate among former CACFP participants. The 35 percent response rate is based on a response rate of 42 percent for FDCH providers in a previous FNS study (Study of Nutrition and Activity in Child Care Settings: OMB #0584-0615, expired 10/31/2019). Due to the additional burdens placed on child care providers as a result of the pandemic,8,9 we have conservatively estimated a response rate of 35 percent for current CACFP FDCH providers in this study. However, no study to date has attempted to recruit and survey FDCH providers who formerly participated in the CACFP. Given that this group has no allegiance to the program, and thus no regulation and sponsor to encourage their participation, we conservatively estimated the response rate for this group at 20 percent.
The recruitment and data collection plans are designed to maximize response rates and address issues of nonresponse. First, we will work with States and sponsors to ensure that they confirm the legitimacy of the study if FDCH providers contact them for verification (Appendix B3-B5). We will also ask sponsors to notify their providers about the study and encourage them to complete the survey (Appendix B6). In addition, we have made the Provider Experience Survey short (approximately 20 minutes) to reduce the burden on providers and encourage provider participation. All communication with sampled participants will be designed to put them at ease, be respectful, be comprehensive to address any concerns or questions they might have, and to encourage them to respond to the data collection request. We will design all provider level communication materials in English and Spanish (Appendix B6, B8a-B9b, B11a-B17b) and use multiple modes—including email, mail, and telephone phone—to reach sampled participants. We will also conduct telephone data collector training, certify them before deploying them to collect data, and monitor their work. These trainings will emphasize strategies for gaining cooperation, answering questions, and administering surveys. Finally, the study team will develop a study website (Appendix B18a and B18b) to serve as a central location of information for study participants. This website will help increase the legitimacy of the study and increase potential respondents’ comprehension of the study.
The study team will monitor daily and weekly response rates as well as mail bounce backs to identify subgroups with lower than expected response rates. We will follow up with program sponsors to get updated contact information and CACFP participation for nonresponding providers, to minimize potential nonresponse bias and develop accurate weights.
Although we will make efforts to achieve as high a response rate as practicable, nontrivial nonresponse losses can occur. If response rates fall below what is expected, we will deploy the reserve sample to achieve the targeted number of survey completes for current and former providers. In addition, the Standards and Guidelines for Statistical Surveys published by the Office of Management and Budget, states that a nonresponse bias analysis (NRBA) is required if the overall unit response rate for a survey is less than 80 percent (Guideline 3.2.9). Therefore, Westat plans to conduct an NRBA to assess the impact of nonresponse on the survey estimates and the effectiveness of the nonresponse adjusted weights to lessen potential nonresponse biases. Specifically, we will compare characteristics of nonrespondents (or the total sample) to those of respondents using information available for both nonrespondents and respondents available in the sampling frame. We will use the data on the 2019 and 2022 lists of CACFP participants to examine differences in characteristics between the responding and nonresponding FDCH providers (e.g., tier, number of children, number of meals served).
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
The Provider Experience Survey was designed specifically for this study, and includes topics that are of interest to FNS. The majority of the survey items are new or modified. In addition, the study team conducted a pretest with nine providers from California and Virginia, representing a variety of current and former CACFP participants, current and former FDCH providers, and English- and Spanish- speaking respondents. These providers gave us feedback on clarity, intent, and comprehensiveness of the survey questions as well as response options. All resulting changes from the pretest are reflected in the current study materials. They include simplifying and rewording questions for clarity, deleting questions that pretesters found to be redundant, adding suggested response options, editing existing response options for clarity, and rearranging question placement to simplify the skip pattern.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
The information will be collected and analyzed by Westat (see Exhibit B-3). The statistical procedures have been reviewed by Jeff Hunt with USDA’s National Agricultural Service (NASS). Comments from NASS are included in Appendix E. Responses are included in Appendix F and incorporated into the supporting statement.
Westat staff (contractor) |
Title |
Contact information |
Tracy Vericker, PhD |
Study Area Associate Director |
|
Alice Ann Gola, PhD |
Senior Study Director |
|
Sujata Dixit-Joshi, PhD |
Senior Study Director |
|
Ismael Flores Cervantes, PhD |
Senior Statistician |
|
FNS Staff |
||
Chanchalat Chanhatasilpa PhD |
Social Science Research Analyst |
703-305-2576 |
Conor McGovern |
Chief, Special Nutrition Evaluation Branch |
703-457-7740 |
NASS Staff |
||
Jeff Hunt |
Mathematical Statistician |
202-720-5359 |
1 Logan C, Connor P, LeClair L, et al. (2021). Study of Nutrition and Activity in Child Care Settings: Appendix A. Methods, Exhibit A.8-1. Prepared by Abt Associates, Contract No. AG-3198-C-14-0017. Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support, Project Officer: Alice Ann H. Gola. Available online at: https://fns-prod.azureedge.net/sites/default/files/resource-files/SNACS-AppendixA.pdf.
2 USDA, FNS National Data Bank (NDB). Accessed on January 21, 2022.
3 U.S. Department of Agriculture, Food and Nutrition Service. Nationwide Waiver of Area Eligibility in the Afterschool Programs and for Family Day Care Home Providers in School Year 2021-2022. COVID-19: Child Nutrition Response # 93. Available online at: https://www.fns.usda.gov/cn/covid-19-child-nutrition-response-93
5Logan
C, Connor P, LeClair L, et al. (2021). Study of Nutrition and
Activity in Child Care Settings: Summary of Findings.
Prepared
by Abt Associates, Contract No. AG-3198-C-14-0017. Alexandria, VA:
U.S. Department of Agriculture, Food and Nutrition Service, Office
of Policy Support, Project Officer: Alice Ann H. Gola. Available
online at:
https://fns-prod.azureedge.net/sites/default/files/resource-files/SNACS-Summary-of-Findings.pdf.
6 United States Census Bureau. Urban and Rural Classification. Available online at: https://www.census.gov/programs-surveys/geography/guidance/geo-areas/urban-rural.html
7Logan, C., Connor, P., LeClair, L., et al. (2021). Study of Nutrition and Activity in Child Care Settings: Appendix A. Methods, Exhibit A.8-1. Prepared by Abt Associates, Contract No. AG-3198-C-14-0017. Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support. Available online at: https://fns-prod.azureedge.net/sites/default/files/resource-files/SNACS-AppendixA.pdf
8 Zero to Three. (2020). How COVID-19 Is Impacting Child Care Providers. Available at: How COVID-19 Is Impacting Child Care Providers.pdf.
9 Straus, V. (2020, May 27). America’s fragile child-care system reported at risk of collapse in covid-19 crisis. Washington Post. Available at: https://www.washingtonpost.com/education/2020/05/27/americas-fragile-child-care-system-reported-risk-collapse-covid-19-crisis/.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | 2020-000: |
| Author | Chantell Atere |
| File Modified | 0000-00-00 |
| File Created | 2022-08-27 |