NeRAED OMB Statement - Part B - Statistical Methods

Nebraska Rural Area Eligibility Determination Pilot

Supporting Statement for Paperwork Reduction Act – Submission to OMB

B. Collections of Information Employing Statistical Methods

B1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

There are approximately 1,380 family day care providers in rural areas of Nebraska – the sample universe. The sampling plan for the Provider Survey is a stratified random selection of respondents from this universe, with CACFP tiering status and location used as stratifiers. Please see Table 4 and the descriptions following.

The Dropout Survey will include family day care providers in the 40-percent threshold areas of Nebraska that elect to leave the CACFP at the end of the pilot, focusing on their reasons for leaving the program. The contractor has no information at this time to estimate the likely number of such providers. Up to 50 providers will be randomly sampled. The contractor plans to make arrangements with the CACFP sponsoring organizations to obtain timely information on which providers leave the program.

B2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection;

• Estimation procedure;

• Degree of accuracy needed for the purpose described in the justification;

• Unusual problems requiring specialized sampling procedures; and

• Any use of period (less frequent than annual) data collection cycles to reduce burden.

The sampling universe for the Provider Survey will be all rural FDCH providers in Nebraska participating in the CACFP. There were about 1,380 of these providers in rural Nebraska as of mid-September 2006.¹ The sampling plan calls for statewide (rural areas only) sample-based estimates to have no more than a ±3-percent error rate at the 95-percent confidence level. To do so, the contractor will sample providers overall, as determined using the following sample formula:

where

N = population size

p = population proportion to be estimated

= permissible error

This sample size computation is based on the chi-square distribution. The 2.706 is the two-tailed chi square value with 1 degree of freedom at the p=.05 level. This computation is more suitable to the categorical nature of the survey data than computations based on a Z value.

In order to generate representative samples for subgroups within the total provider population, the contractor will conduct a stratified sampling based on the number of providers in each appropriate subgroup. Stratification will be based on the following strata:

1. Rural tier II providers, including both tier II and mixed tier II providers (mixed tier II providers have some, but not all, of the children under their care eligible for tier I reimbursement);

2. Rural tier I providers meeting the standard area eligibility criterion of at least 50 percent of children in the area living in households with income below or at 185 percent of the poverty level; and

3. Rural tier I providers not meeting the 50-percent standard eligibility requirement but meeting the expanded 40-percent eligibility requirement due to the pilot.

Using a simple random sampling method, and based on a total population of 1,380 providers, the required sample size is 488. However, because the study is interested in several provider subgroups, the contractor plans to oversample these subgroups to obtain data at a 95-percent confidence level with a ± 5-percent error rate for each group. The study is also interested in examining the difference between tier I 40-percent providers who had previously been tier II providers and those who are new providers. The best method to obtain reliable data on these micro strata is to invite nearly all of the 150 tier I 40-percent providers to participate in the survey. Table 4 shows the expected sample size for each stratum based on random sampling and on the requirements for oversampling. The shaded column tallies these considerations (based on the 95-percent confidence level and the 5-percent error rate) to arrive at the final sample size. Each figure in the shaded column is either the sample size generated from the simple random sample or the sample size determined by the need to oversample to achieve specified precision within the stratum (whichever is larger). Please see the footnote² for a detailed row-by-row, column-by-column explanation of the table.

Table 4 – Projected Sample Size Calculations

Strata	Total Population	Pct of Total	Random Sample Sizes	Sizes Needed to Achieve Confidence Levels	Required Over- Sampling	Final Sample	Projected Response Rate	Need to Invite
Tier I (50 pct)	980	71.0%	346	213	0	346	70%	495
Tier I (40 pct)	150	10.9%	53	97	44	105	70%	150
Tier II	250	18.1%	89	131	42	131	70%	188
TOTAL	1380	100.0%	488	441	86	582	70%	833

B3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield reliable data that can be generalized to the universe studied.

The contractor will employ several techniques to increase the response rates and thus provide USDA/FNS with more useful and reliable results. These techniques include the following:

• Assign each client a unique identification number that will be used to track response rates and to follow up with nonresponders.

• Prepare a letter for each sponsor to send out notifying his or her providers about the upcoming survey and requesting their support. For sponsors that send newsletters to their providers, write a short description of the study and planned survey for inclusion in the newsletter.

• Ask the NDE to endorse the survey plans, both in its quarterly meetings with sponsors and in its correspondence to providers.

• Send out a cover letter (and an advance e-mail where possible), with the survey instrument attached, to each sampled provider with an initial request to participate signed by either the Contracting Officer’s Representative (COR) or another senior USDA/FNS staff member. If an e-mail, it will include the survey’s focus, estimated length of the survey, the timeframe for completing the interviews, and the location of the survey on the Internet. These letters will describe the importance of the survey and the respondent’s participation. The legitimacy advance letters give to surveys generally increases response rates. Provide the telephone number of the contractor’s Project Manager (PM) and the FNS COR so that respondents can call to further confirm the legitimacy of the survey.

• Specify survey length at the beginning of the survey.

• Incorporate both e-mail and telephone followup reminders.

• Follow up with nonresponders:

  • The contractor will send providers who do not complete the survey a second letter or e-mail reminding them of the importance of the survey.

  • Providers who do not complete the survey after the second contact will be called on the telephone by survey staff and encouraged to do so. The survey staff will utilize experienced Refusal Conversion techniques, such as reiterating the criticality of the survey, to convince those who ignore or refuse the requests to comply.

• The contractor will send follow-up reminders beginning 1 week after the original notification in order to maintain proper sample management (e.g., tracking response rates and reasons for refusals).

B4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The contractor pretested a draft of the Provider Survey with five family day care providers in Nebraska. Information from the pretest was used to refine question wording to increase readability and validity. Because many of the questions in the Dropout Survey repeat those from the Provider Survey (for providers who did not participate in the Provider Survey), a separate pretest is not needed. Furthermore, because the new questions seek to learn about the possible influence of the pilot’s termination on provider behavior, it is not possible to test these few questions while the pilot continues.

B5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

John Kirlin USDA/FNS 703-305-2128

Barbara Milton King McFarland & Associates 301-589-0780

Joe Kirchner Exceed Corporation 301-731-3790

Mitch Wang McFarland & Associates 301-589-0780

1According to the State of Nebraska’s Department of Education administrative database. http://cnp.nde.state.ne.us/Splash.asp

Click on Reports, then FDCH Reports, then DchRpt404, and then Submit.

2 Table 4 presents the results of our sample size calculations based on our sampling formula. Our objective is a 95-percent confidence level with no more than a ±3-percent error rate for the full sample and ±5-percent for each stratum. The first three rows list the figures for each of the principal strata. These figures are based on the 5-percent criterion. The Subtotal row sums the figures for the strata. The Full Population stratum row lists the calculations for the sample of the full population and is based on the 3-percent criterion.

The focus of the study is on the tier I 40-percent providers. The micro strata, including those that had been tier II and those who are new providers, are not sufficiently large enough to yield sample sizes meeting our sampling requirements. The best we can do is to invite all tier I (40-percent) providers.

The first column lists the strata or full sample for the row. The second column lists the population size for that stratum or full population. The third column lists the stratum's percent of the full population. The fourth column lists the calculated sample size of the full population in the second-to-last row and multiplies that figure by the percent for each stratum from column three. This column represents the number of sampled providers who would come from each stratum based on a random probability of selection. The sample sizes thus generated for tier I 40 percent and tier II would not be large enough to satisfy our 95-percent confidence level with no more than a ±5-percent error rate criterion for strata; hence we need to oversample these strata. Column five lists the sample size required to meet our confidence level and error rate criteria. Column six shows how much oversampling would be required. Column seven (shaded) shows the maximum of columns four and five. The tier I 40-percent stratum sample size was increased above that maximum to include all providers who could be obtained at a 70-percent response rate in order to obtain the best confidence level and error rate possible for the micro strata discussed above. The shaded column is our selected sampling size for each stratum and the full population according to our sampling plan. The last column lists the number of providers to be invited to participate in the survey. These numbers are large enough to yield our selected sample size based on our targeted response rate of 70-percent.

20

Part B – Statistical Methods