TANF Sampling and Statistical Methods Manual Part 265.5

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN:

Through this information collection, ACF is gathering information to assess and evaluate whether a State TANF program meets statutorily required participation rates. Public reporting burden for this collection of information is estimated to average 192 hours per grantee per year, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a mandatory collection of information (42 U.S.C. § 611). An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the

Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0338 and the expiration date is XX/XX/XXXX. If you have any comments on this collection of information, please contact the Office of Family Assistance by email at [email protected].

TANF

SAMPLING AND STATISTICAL

METHODS MANUAL

May 2007

Amended May 2023

Contents

1100. INTRODUCTION 4

1110. Purpose of the Manual 4

1200. BASIC STATISTICAL CONCEPTS 6

1210. Sampling Errors 6

1220. Common Types of Scientific Sampling Techniques 6

1221. Simple Random Sampling 6

1222. Systematic Random Sampling 6

1223. Stratified Random Sampling 7

1224. Allocation of Stratified Sample 8

1230. Validity and Reliability of Statistical Data 8

1300. SAMPLING PLAN REQUIREMENTS (apply to both TANF and SSP) 9

1310. Criteria for Plan Approval 9

1320. Sample Frame 10

1330. Sample Selection Procedures 11

1400. SAMPLE SIZES AND PROCEDURES FOR SELECTING SAMPLE CASES 12

1410. Annual Sample Size Requirements 12

1411. Sample Size Requirements for the TANF/SSP Active Sample 12

1412. Sample Size Requirements for the TANF (and/or SSP) Sample of Closed Cases 13

1413. Average Monthly Sample Size 14

1420. Sample Frame 14

1421. Sampling Frame for the TANF (and/or SSP) Active Case Sample 14

1422. The Treatment of Special Groups with Respect to TANF (or SSP) Reporting 16

1422.1 Newly Approved Applicant (aka, Initial Assistance Cases, only in TANF population) 16

1422.2. Non-Custodial Parents 17

1422.3. Members of Indian Tribes Not Eligible under a Tribal Family Assistance Plan 17

1422.4. Members of Indian Tribes Receiving Assistance under a Tribal Family Assistance Plan 17

1422.5. Cases Selected for More Than One Sample Month 17

1422.6. Cases Receiving Assistance Under the STT's TANF and SSP programs for the Same Month 17

1422.7. Cases for Which STT Changes Funding Stream 18

1423. Sample Frame for the Sample of Closed TANF (and/ or SSP) Cases 18

1430. Procedures for Selecting Sample Cases (Apply to both TANF and SSP samplings) 18

1440. Procedures for Selecting Sample Cases Using a Simple Random Sample 22

1450. Retention of Sampling Records 23

1500. CORRECTION FOR UNDERSAMPLING 25

1510. Standard Method to Correction for Undersampling when Sample Selected Using Systematic Random Sampling 25

1520. Alternate Method of Correcting for Undersampling When Sample Selected Using Systematic Random Sampling 27

1530. Correcting for Undersampling Using a Reserve Sample Pool 29

1531. Procedure for Setting Up a Reserve Sample Pool 29

1532. Procedure for Obtaining Cases from a Reserve Sample Pool 31

1540. Correction for Undersampling When Sample Was Selected Using Simple Random Sampling 32

1541. Correcting for Undersampling 32

1100. INTRODUCTION

Title I of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA) establishes the Block Grants for Temporary Assistance for Needy Families (TANF) Program by amending Titles IV-A and IV-F of the Social Security Act. The purpose of this welfare reform initiative, which replaced the Aid to Families with Dependent Children Program (AFDC) and the Job Opportunities and Basic Skills Training (JOBS) program, is to increase the flexibility of allow States, District of Columbia, Territories, and Tribal grantees (STTs) in operating a program designed to:

1. Provide assistance to needy families so that children may be cared for in their own homes or in the homes of relatives.

2. End the dependence of needy parents on government benefits by promoting job preparation, work, and marriage.

3. Prevent and reduce the incidence of out-of-wedlock pregnancies and establish annual numerical goals for preventing and reducing the incidence of these pregnancies; and

4. Encourage the formation and maintenance of two-parent families.

While the TANF provisions allow STTs discretion as to the mechanisms used in meeting these goals, they place on STTs a responsibility for measuring, tracking, and reporting on their reform initiatives.

The PRWORA requires STTs to collect on a monthly basis and report to the Secretary of the Department of Health and Human Services (DHHS) on a quarterly basis a wide variety of disaggregated case record information on the families receiving assistance, families no longer receiving assistance, and families applying for assistance from programs funded under the TANF program (and/or Separate State Program - Maintenance of Effort (SSP-MOE)). STTs may comply with this requirement by collecting and submitting case record information for its entire caseload or by collecting and submitting the case record information for a portion of the caseload which is obtained by scientifically acceptable sampling methods.

1110. Purpose of the Manual

Sampling is the selection of a part of a whole for the purpose of drawing conclusions about the population, or universe. The manual explains statistical techniques in sufficient detail for careful observance of sound sampling procedures and other basic statistical principles. Theory is included to the extent necessary to provide working rules for application of the more commonly used techniques as well as for recognizing the limitation of such techniques. Because many users of the Manual are not statisticians, mathematical exposition and technical language have been kept to a minimum.

This sampling manual contains the broad framework and procedures to be used by each STT, that opts to file its TANF Data Report, SSP-MOE Data Report (or Tribal TANF Data Report) based on a sample of its caseload. In developing its more specific sampling plans, STTs have considerable latitude in designing samples that are consistent with the principles described herein. The manual is designed to provide the user with a basic understanding of the TANF program sample requirements and statistically valid sampling methods, which are essential to the successful reporting on the TANF program.

Section 1200 describes common types of sample designs (e.g., simple random sampling and systematic random sampling) and basic statistical concepts, which are applicable in any sample survey setting. It is intended to provide a general background to non- statisticians who use the manual. Section 1300 contains sampling plan requirements: a STT sampling plan must include a detailed description of the sample frame and the procedures that are to be employed in constructing the sample frame, i.e., the list from which the sample is to be selected. Also, the plan must describe in detail the sample selection procedures for identifying the sample families (cases) for which data are to be reported. Section 1400 contains the sample size requirements, sample selection procedures for systematic random sampling and simple random sampling, and special sampling problems associated with the TANF program. Section 1500 describes procedures to be using in the event an adjustment to the sample size is needed. If STTs need further assistance to develop sampling plans, they can contact the Administration for Children and Families (ACF) Office of Family Assistance (OFA) Regional Program Manager (RPM) for assistance.

1200. BASIC STATISTICAL CONCEPTS

Probability sampling is an acceptable alternative to providing 100 percent counts of the TANF/SSP caseload each month. Probability sampling has two properties: (1) every unit in the entire population has a known, non-zero chance (called a probability) of being selected in the sample, and (2) there is an element of "randomness" used to select the sample for which data are to be collected. These two principles 1) measurability and randomness and 2) distinguishing probability samples from haphazard, judgment, or quota samples.

1210. Sampling Errors

When a sample is selected through a random procedure, the estimates of a population characteristic from that sample will generally be different from the true value of the population characteristic simply because the estimates are based on a sample. Thus, a sampling error may be defined as the difference between the value of the characteristic as estimated from the sample and the true population value of the characteristic. Although such errors cannot be avoided, they can be controlled and measured (in probability samples).

1220. Common Types of Scientific Sampling Techniques

It is impossible to specify a single sampling procedure that would be best suited to all STTs for all samples. There are many ways of selecting scientific (probability) samples from populations with items of equal importance. The simplest and most widely used methods are: simple random sampling, systematic random sampling, stratified simple random sampling, and stratified systematic random sampling. These four widely used methods are acceptable methods of sampling for the purpose of collecting and reporting the disaggregated TANF and SSP-MOE data.

1221. Simple Random Sampling

Simple random sampling is a method of selecting a sample in such a way that each unit of the frame has an equal and independent chance of being included in the sample. For samples of any given size (n) from a population of size N, all possible combinations of n units that could form samples of that size must have the same probability of selection. A computer program with a random number generator is generally used to choose the sample units. This method is relatively easy to administer and is responsive to variations in caseload size over the course of the sample period.

1222. Systematic Random Sampling

Systematic random sampling method provides a system or pattern of selection of individual units from a sample frame (which may be a hardcopy list or computer file of all the individual units in the population) at equally spaced intervals (such as every 10th, 140th, 850th, etc., as required to obtain the total of a given sample size) with the starting point within the first interval being determined by random selection.

In using the systematic random sampling method, one needs to be aware of a major pitfall that exists when the cases on the sample frame are arranged in repetitive or cyclical pattern. In such an ordered list, the sample interval might sometimes be the same as the cycle and could, therefore, yield a sample of cases with similar characteristics which may not be typical of the caseload. It is, therefore, important not to use a systematic sample with a listing that is cyclical in nature.

1223. Stratified Random Sampling

Stratified random sampling is random sampling of a population that is divided into several sub-populations according to some pre-determined criterion (geographic location, characteristic, etc.). To produce estimates with a given precision while minimizing the total sample size required, the population is divided into several homogeneous groups so that the units in the same group are more alike than the units in different groups. Each group is called a "stratum" and the process of dividing the population into groups is referred to as “stratification". The strata do not overlap and together comprise the entire population. Sample cases can be selected independently from each stratum using either systematic random sampling, simple random sampling, or an alternative approved sampling procedure. If the percent of the sub-population selected from each sub-population are equal, i.e., proportional sampling, no weighting is required. The sample is "self-weighting". Otherwise, individual weighting factors for each sub- population must be considered before the sub-population sample results can be combined.

There are various purposes for stratification. It may be that information is desired on the strata separately; that more accurate estimates of the population parameters are needed than can be obtained by a non-stratified sample; or that costs and administrative constraints must be considered. To achieve these purposes, optimum allocation of the sample size among the strata is usually required. Because a disproportionate number of cases can be drawn from particular strata, some strata may be sampled more intensively than others. For example, a STT may find it administratively efficient to give a higher probability of being sampled to urban areas than to rural areas.

The following points should be considered in using the stratified sampling method:

1. Stratified sampling requires advance knowledge of the proportion of the population in each stratum.

2. Stratification by one characteristic does not ensure an efficient stratification by other characteristics that may be of interest.

3. Gains in precision for population estimates will be negligible unless it is known that there are substantial differences between the strata and relatively small differences within each stratum.

4. The cost and effort of creating the strata may outweigh the potential gains in precision.

5. The weighting procedures required for calculating population estimates and confidence levels for stratified samples in which the strata units are disproportionately allocated can be complex and time consuming, and

6. Over stratification (i.e., creating too many strata) for a given size of sample can result in some small strata that may adversely affect the precision of estimates.

1224. Allocation of Stratified Sample

If a STT selects a stratified sample, the STT must decide how to allocate the sample among the strata and describe the allocation procedures in the sampling plan.

1230. Validity and Reliability of Statistical Data

Sampling and statistical procedures, by themselves, cannot assure validity (or freedom from bias) of the collected data -- that is, that case record information is correct and is reported correctly.

If sampling procedures are soundly based, the results obtained from one sample taken from the total caseload will be the approximate results obtained if the whole caseload was reviewed. These results can be made more reliable through proper application of statistical methods, as well as through an increase in sample size.

By algebraic rearrangement, it is possible to compute the minimum sample size needed to obtain a desired precision. For example, to obtain the sample size required for 95 percent confidence, that a sample proportion "p" will be within plus or minus 2 percent of the true proportion

"p" when is assumed to be 50 percent. The computation is as follows (where "e” is the desired precision level. 2 percent in this example), so n = 2,401 or approximately 2,400 cases.

1300. SAMPLING PLAN REQUIREMENTS (apply to both TANF and SSP)

The sampling plan serves as the foundation for ACF review of the integrity of the STTs TANF and SSP sampling procedures. The STT that elects to submit case record information for a sample of cases must select its TANF (or SSP) sample for data reporting purposes under a sampling plan approved by the ACF OFA Director. All sampling procedures used by the STT, including frame composition and construction, must be fully documented and available for review by the Division of Data Collection and Analysis (DDCA) at ACF OFA. This requirement includes all data processing specifications and automated routines used to select the samples.

The sampling plan documentation must describe the list(s) of families from which the samples are selected, the sample selection procedures, and the methodology for estimating caseload characteristics and sampling errors. Referencing sub-sections of this manual in the sampling plan does not constitute acceptable compliance with the requirements set forth for sampling plan documentation without further explication of the specific procedures the STT will use. Detailed descriptions of the sample frames, sample selection, and estimation procedures used by the STT must be included in the sampling plan documentation.

If a STT opts to report the required case record information for a sample of families (as opposed to for the entire caseload), it shall have an approved sampling plan in effect for a full sample period. A STT may not implement a new sample design without prior approval. A revised sampling plan must be submitted to the ACF OFA Regional Office with specific documentation of any substantive modification of a previously approved sample design at least 60 days before the start of the annual sample period, which is no later than August 1. The STT is not required to resubmit the sampling plan if it is unchanged from the previous year. Changes in random start numbers, sample intervals, or caseload estimates are not to be submitted as a revision of the sampling plan. They should, however, be sent to the ACF OFA Regional Office.

1310. Criteria for Plan Approval

The sampling plan must meet the following criteria:

1. Conformance to principles of probability sampling, i.e., each family (case) in the population must have a known, non-zero probability of selection and computational methods of estimation must lead to a unique estimate.

2. Documentation of methods for constructing and maintaining the sample frame(s), including assessment of frame completeness and any potential problems associated with using the sample frame(s).

3. Documentation of methods for selecting the sample cases from the sample frame(s), and

4. Documentation of methods for estimating case characteristics and their sampling errors, including the computation of weights, where appropriate.

1320. Sample Frame

Samples are selected from a list of families called a "sample frame". The sampling plan must describe in detail the master file, or other list(s) from which the sample of families is selected. The plan must explicitly describe the following sample frame characteristics:

1. Date(s) when the sample cases (both regular and supplemental, if applicable) for the sample month are selected, e.g., first workday of the month following the sample month).

2. Source, components, accuracy, and completeness of the sample frame in relation to the total caseload; if not accurate or complete, explanation of why not and how STT plans to correct for the problems with the sample frame.

3. Procedures for ensuring that the sample frame contains complete coverage of the applicable caseload (e.g., the active TANF sample frame include all families receiving assistance under the STT’s TANF Program, including all newly approved applicants for the sample month and the closed TANF sample frame includes all families no longer receiving assistance under the STT’s TANF Program, i.e., assistance terminated effective for the sample month).

4. Whether or not the frame is constructed by combining more than one list (if more than one list, explanation of how lists are identified and how duplication of cases on lists are prevented).

5. Whether the frame is compiled entirely in the STT’s office, entirely in local offices, in the STT’s office based on information supplied by local offices, etc.

6. Form of the frame, e.g., a computer file, if parts of the frame are in different forms, specifications for each part.

7. Frequency and length of delays and method used in updating the frame or its sources.

8. Procedures for estimating the proportion of sample cases for which the STT will not be able to collect and report case record information (e.g., dropped as "listed-in-error" because the case did not receive TANF assistance for the reporting month).

9. Methods of locating and deleting "listed-in-error" cases from the frame.

10. Structure of the frame, i.e., the order of cases within each list and the data elements on the frame, including definitions of coded values.

11. Treatment of special populations under TANF (e.g., individuals under a tribal family assistance plan, a non-custodial parent who participates in work activities), and

12. Criteria for stratifying sample (if applicable).

1330. Sample Selection Procedures

The sampling plan must describe in detail the procedures for selecting the sample cases. The plan must explicitly describe the following characteristics:

1. Procedures for estimation of caseload size, if applicable to sampling method.

2. Procedures for determination of an appropriate allowance for sample cases for which the review may not be complete because the sample case family did not receive TANF assistance for the sample month and therefore was "listed-in-error".

3. Procedures for determining the required monthly sample size and indication of the sample size.

4. If stratified sample design is used, procedures for sample allocation.

5. Procedures for the computation of sample intervals and the determination of

random starts if the STT used systematic random sampling or stratified systematic random sampling.

6. Application of selection procedures to identify sample cases.

7. Procedures to compensate for undersampling, and

8. Time schedule for each step in the sampling procedure.

1400. SAMPLE SIZES AND PROCEDURES FOR SELECTING SAMPLE CASES

1410. Annual Sample Size Requirements

STTs should consider their own management information needs relative to desired reliability of characteristic data broken out for specific groupings, geographic areas, or by monthly or quarterly time periods in deriving the TANF and/or the SSP-MOE sample sizes. While this section of the manual specifies the minimum required annual sample sizes for completed case reviews, STTs are encouraged to select larger size samples to increase the precision of the resulting estimates and to meet their own information needs.

For TANF (and /or SSP-MOE) data collection and reporting purposes, there are two sampling frames from which cases are to be sampled. The sampling frames are for families receiving assistance (i.e., active cases, including all newly approved applicants for TANF) and families no longer receiving assistance (i.e., closed cases).

1411. Sample Size Requirements for the TANF/SSP Active Sample

The minimum required annual sample size for the active TANF/SSP sample is 3,000 cases.

Among the 3,000 TANF samples, at least 2,400 are ongoing cases and 600 are newly approved applicants, if the STT has two-parent program of the 2,400 ongoing cases approximately 600 cases are two-parent TANF cases; Of the 3,000 ongoing SSP-MOE cases approximately 600 cases are two-parent families if the STT has two-parent program.

Approximately, one-twelfth of the annual sample must be selected each month of the annual sample period. The minimum required sample sizes are designed to provide reasonably precise estimates for such proportions as the work participation rates for all families (e.g., a precision of about plus or minus 2 percentage points at a 95 percent confidence level) and for two-parent families (e.g., a precision of about plus or minus 2.3 percentage points at a 95 percent confidence level), as well as for demographic and case characteristics of newly approved TANF families and all TANF families. In addition, these sample sizes will permit us to detect real changes in certain proportions over time (e.g., changes in the proportion of child-only cases).

The midpoint estimate (from which the confidence limits are constructed) of the overall and two-parent work participation rates will be used in determining if STTs have met the statutory requirements. If the STT is unwilling to accept the precision levels obtained from the minimum required annual sample sizes for the purpose of assessing penalties for failing to meet the work participation rates, it is the STT's responsibility to increase its sample size to what the STT determines is an acceptable level of precision for this purpose.

To meet these sample size requirements, STTs may select one of the following options:

1. Use a simple or systematic random sampling methodology (or other acceptable method) and use an overall sample size that is sufficiently large enough to obtain the 600 cases needed to meet the two-parent family required sample size, the 600 required to meet the newly approved applicant sample size (this only applies to TANF), and the 3,000 cases required to meet the overall sample size.

2. Stratify the sample by newly approved applications; two-parent families; and all other families and use a random sampling method within each stratum to select the sample. Taking into consideration the fact that two- parent families are included in the calculation of the all-family work participation rate, compute the sample size for each stratum based on the 600-case requirement for the two-parent stratum and 600 for the newly approved applicant stratum, and 1,800 cases for the remaining families.

Under option 2, each stratum is sampled separately, and the monthly all families work participation rate is a weighted rate, reflecting the representation of two-parent families and other families with at least one adult or a minor child head-of-household to the total all family population. If a STT uses a stratified sample design, it must submit the monthly caseload for each stratum. These monthly caseload sizes by stratum are due 45 days after the close of each quarter (i.e., the same due dates as for the quarterly TANF and SSP Data Report, Sections one, two and three).

If a STT does not have enough newly approved applicants or two-parent families to meet the required annual sample sizes of 600 families (i.e., the average monthly sample size of approximately 50 newly approved applicant families or 50 two-parent families), the STT must select 100 percent of such families and select from the other ongoing stratum enough additional cases to meet the overall required annual sample size of 3,000 families. If a STT does not have enough families to meet the overall sample requirement (i.e., 3,000 families for the active TANF sample for an average monthly sample of 250 families), the STT must report on 100 percent of their families each month.

STTs are not limited to these two methods for meeting the sample size requirements. However, alternative methods should be discussed with ACF OFA DDCA to ensure the reliability of the work participation rates and any other statistic used to assess a penalty is not severely affected.

1412. Sample Size Requirements for the TANF (and/or SSP) Sample of Closed Cases

The minimum required annual sample size for the sample of closed cases is 800 cases. Approximately one-twelfth of the annual sample must be selected each month of the annual sample period. An 800-case sample will permit us to obtain a precision of plus or minus 3.5 percentage points for an attribute of 0.50 at a 95 percent confidence level. This result is obtained from the formula in Section 1230 of this manual.

If a STT does not have enough closed cases to meet the required minimum annual sample size of 800 families (i.e., an average monthly sample size of approximately 67 families), the STTs must collect data for and report on 100 percent of the closed cases.

1413. Average Monthly Sample Size

A STT must select approximately one-twelfth of its annual sample size each sample month. The average monthly sample size is determined by dividing the required annual sample size by 12 and rounding the result up to the nearest whole number. For the active TANF sample and SSP sample, the average monthly sample sizes are 250 cases, of which 50 are two-parent families (if STT has two-parent program), or 50 are newly approved applicants for TANF. For TANF and SSP samples of closed cases, the average monthly sample sizes are about 67 cases. The following additional procedures apply to the TANF samples and to the SSP samples:

1. STTs should select additional cases (use the rate for "listed-in-error" cases based on historical data or, if unknown, use five percent) of each sample to compensate for cases that may be reported as "listed-in-error".

2. A STT is encouraged to increase its sample size above the minimum but cannot reduce its sample size below the minimum; and

3. A STT has the option of collecting and reporting data for the entire TANF and/or SSP-MOE population.

1420. Sample Frame

Creating a frame or list of cases from which the monthly samples are to be selected and determining the sample size are preliminary steps applicable to any probability sample design. Careful study of the structure of the sample frame is always essential in probability sampling, especially in systematic random sampling. The choice of a frame depends upon the criteria of timeliness, completeness, and administrative burden. The structure of the sample frame should provide for an unduplicated list of cases comprising the target population or otherwise allow for all units to have a known, non-zero chance of selection into the sample. In systematic random sampling, cases should be randomly ordered with respect to the variables being measured, e.g., case characteristics data, earnings, participation in work activities, etc. This random order is usually achieved if cases are arranged by case number or by county and then alphabetically within county, or by any other file organization that is not directly related to the measurement of critical variables. In stratified sampling, each family must be assigned to one (and only one) stratum. The structure of the sampling frame must be fully documented in the sampling plan and may not be changed without an approved revision of the sampling plan.

1421. Sampling Frame for the TANF (and/or SSP) Active Case Sample

The monthly TANF (or SSP) sample frame consists of all families who receive assistance under the STT’s TANF (or SSP) Program for the sample month by the end of the sample month. The term "assistance", defined in §260.31 of the final rule, includes cash, payments, vouchers, and other forms of benefits designed to meet a family's ongoing basic needs (i.e., for food, clothing, shelter, utilities, household goods, personal care items, and general incidental expenses). It includes such benefits even when they are provided in the form of payments by a TANF agency, or other agency on its behalf, to individual recipients and conditioned on their participation in work experience, community service, or other work activities (i.e., under §261.30).

Except where excluded as indicated in the following paragraph, it also includes supportive services such as transportation and childcare provided to families who are not employed.

The term "assistance" excludes:

1. Nonrecurrent, short-term benefits (such as payments for rent deposits or appliance repairs) that:

1. Are designed to deal with a specific crisis situation or episode of need;

2. Are not intended to meet recurrent or ongoing needs; and

3. Will not extend beyond four months.

2. Work subsidies (i.e., payments to employers or third parties to help cover the costs of employee wages, benefits, supervision, and training).

3. Supportive services such as childcare and transportation provided to families who are employed.

4. Refundable earned income tax credits.

5. Contributions to, and distributions from, Individual Development Accounts.

6. Services such as counseling, case management, peer support, childcare information and referral, transitional services, job retention, job advancement, and other employment-related services that do not provide basic income support, and

7. Transportation benefits provided under an Access to Jobs or Reverse Commute project, pursuant to section 404(k) of the Act, to an individual who is not otherwise receiving assistance.

The exclusion of nonrecurrent, short-term benefits under (1) of this paragraph also covers supportive services for recently employed families, for temporary periods of unemployment, to enable continuity in their service arrangements.

The TANF (or SSP) active case sample frame could be a master file; a payroll file; an eligibility, activity, or other caseload file; or a combination of such files depending on how the STT defines its range of benefits/assistance. If such a list cannot be constructed based on the above definition of the sampling universe, it may be necessary to use a special procedure to ensure that all families receiving assistance have a known, non-zero chance of being included in the sample. The sampling plan should contain the STT’s objective criteria for the delivery of assistance and determination of eligibility as set forth in the STT’s family assistance plan. STTs should verify the receipt of assistance for all selected cases, and all such cases discovered not to have received assistance for the reporting month should be reported as "listed-in-error" (should not be reported to ACF). For all other cases selected into the sample, the data collection must be completed, and the data must be submitted to ACF by the specified time frames.

STTs that use regular first-of-the-month payroll or eligibility listings as the frame for selection of sample cases must extend that frame at the end of the report month and continue sampling all cases for which assistance was initiated during the report month that were not on the first-of-the-month payroll/eligibility listing. Care must be taken to ensure that the sample frame consists of unduplicated cases. A distinction is made between cases already receiving TANF (or SSP) and cases in which assistance is initiated during the month. For example, a case receiving a regular payment on October 1 and a supplemental payment on October 12 should only be subject to selection once for the month of October. Procedures for accomplishing this must be specified in the sampling plan. Normally, this will be accomplished by running a computer sort/merge routine at the end of the report month to establish the list of supplemental cases to be added to the frame.

STTs that use simple random sampling should form the sample frame at the end of the sample month, ensuring all families that received assistance for the month by the end of the month are on the sample frame. Then the sample is selected after the end of the sample month.

1422. The Treatment of Special Groups with Respect to TANF (or SSP) Reporting

There are several family circumstances that merit special attention. These are described below.

1422.1 Newly Approved Applicant (aka, Initial Assistance Cases, only in TANF population)

A newly-approved applicant or an "initial payment/assistance" case for a sample month means the family is newly added to the TANF caseload and the current reporting month is the first month in which the TANF family receives TANF assistance (and thus has had a chance to be selected into the TANF sample). This may be either the first month that the TANF family has ever received assistance or the first month of a new spell on assistance. The initial payment/assistance case should be included on the sample frame for the initial month in which it received assistance and for all subsequent months for which assistance is issued. For STTs that provide assistance back to the date of application, these cases may, at STT option, be included on the frames for prior months, as assistance was not received by the end of such months.

A family that moves back and forth between receipt of assistance to receipt of only non- assistance in a subsequent month while remaining in the TANF program will be a newly approved applicant each time it moves to receipt of assistance for a reporting month.

1422.2. Non-Custodial Parents

A non-custodial parent is defined in §260.30 as a parent of a minor child who: (1) lives in the STT and (2) does not live does not live in the same household as the minor child. The STT must report information on the non-custodial parent if the non-custodial parent: (1) is receiving assistance as defined in §260.31; (2) is participating in work activities as defined in section 407(d) of the Act; or (3) has been designated by the STT as a member of a family receiving assistance. In reporting non-custodial parents, STTs should not treat the non-custodial parent as a separate case. Rather, when the family unit containing his/her child(ren) is selected into the sample, code the type and amount of assistance received by the non-custodial parent as part of that case. The non-custodial parent's person level data must also be provided. STTs have the option to include or exclude the non-custodial parent from the work participation rate on a case-by- case basis. If an individual is both a custodial parent for a TANF family receiving assistance and a non-custodial parent for another TANF family receiving assistance, the STT should report the individual only with the family for which (s)he is the custodial parent.

1422.3. Members of Indian Tribes Not Eligible under a Tribal Family Assistance Plan

The STT sample frame must include each member of an Indian tribe otherwise meeting the definition of the sampling unit who is domiciled in the STT and is not eligible for assistance under a Tribal family assistance plan.

1422.4. Members of Indian Tribes Receiving Assistance under a Tribal Family Assistance Plan

The STT should not include members of an Indian tribe receiving assistance under a Tribal family assistance plan, even if the STT selected the option to include such families in the calculation of its participation rate as provided for in section 407(b)(4) of the Social Security Act.

1422.5. Cases Selected for More Than One Sample Month

If a family is selected into the sample for more than one month during the annual reporting period, the STT should collect data for and report on the family for each month for which it is selected.

1422.6. Cases Receiving Assistance Under the STT's TANF and SSP programs for the Same Month

An eligible family may receive some form of assistance under both the STT's TANF and its SSP programs during the reporting month. If this occurs, the family should be included on the active sample frame for both the TANF and the SSP. If such a family is selected into the sample, the STT should collect data for and report on the family for each program for which it was selected.

1422.7. Cases for Which STT Changes Funding Stream

STTs must make all changes in funding streams to cases for a report month prior to formation of the sample frame(s) and sample selection for the report month. Changes in funding stream after sample selection are not permitted because such changes will destroy the representativeness of the sample and result in invalid samples. This would make the STT liable for a data reporting penalty.

1423. Sample Frame for the Sample of Closed TANF (and/ or SSP) Cases

For closed cases, the monthly sample frame must consist of all families whose assistance under the STT’s TANF (or SSP) Program was terminated for the reporting month (do not include families whose assistance was temporarily suspended) but received assistance under the STT’s TANF (or SSP) Program in the prior month. A family that moves from receipt of assistance under the TANF (or SSP) program for a month to receipt of benefits that are not assistance under the TANF program for the subsequent month is a closed case for reporting purposes. Also, a TANF family that is transferred to the STT's SSP or Tribal TANF program is closed for the STT’s TANF Program, the same is true for a SSP family that is transferred to TANF Program is a closed case for the STT’s SSP program.

1430. Procedures for Selecting Sample Cases (Apply to both TANF and SSP samplings)

STTs have flexibility to choose from a wide variety of sampling methods, including systematic random sampling, simple random sampling, and stratified (systematic or simple) random sampling. For illustrative purposes, the following procedures are based on the systematic random sampling design and, if used, are repeated each month during the annual sample period. In illustrating the procedures, a STT with an estimated average monthly active TANF caseload of 42,600 is used. These same procedures could be used to select the sample of closed TANF cases or the sample of the active and closed SSP cases. Note, these procedures could be used to select a sample within each stratum for a stratified systematic random sample design.

1. Estimate Caseload Size

The TANF average caseload is an estimate of the average monthly number of cases that will receive assistance for the forthcoming annual sample period. The average caseload size should be estimated based on the past caseload sizes and trends. Any known circumstances, such as policy changes that would appreciably change caseload sizes, also should be considered in making the estimate.

Since the average monthly caseload must be estimated before the beginning of the annual sample period, unanticipated changes can result in the need for adjusting the sample interval. Recognizing the difficulty of forecasting caseloads over a 12-month period, STTs should re-evaluate the estimated caseload before the end of each quarterly reporting period. If the caseload estimate is changed, a new sample interval for the 12-month period and adjustments to the number of sample cases already selected may be needed. The procedures in Section 1510 can be used depending on whether the sample requires correction for undersampling. If no correction is required for the remaining quarterly reporting period(s), using these procedures will result in a self-weighting annual sample.

2. Determine Sample Size

The minimum required annual sample sizes of completed cases are shown in Section 1410. In our illustration, the minimum sample size is used.

n = 3,000

An estimate of the percent of cases that may possibly be reported as listed- in-error during the TANF (or SSP) data collection process will need to be made to arrive at the required completed sample size. For example, assuming that 5 percent of the selected cases will be reported as listed-in-error, the number of cases to be selected can be computed as follows:

3,000 ÷ 1.05 = 3,158

3. Establish Frame

As mentioned in Section 1421, a listing of all TANF (or SSP) cases that received assistance for the sample month by the end of the month (including initial assistance cases and cases that are reinstated) comprises the frame from which the sample is selected.

4. Establish Average Monthly Sample Size

The average monthly sample size is obtained by dividing the sample size for the sample period by the number of months in the period. In our illustration, the average monthly sample size is 3,158 ÷ 12, which is 263.167 cases.

5. Compute Sample Interval

The sample interval is obtained by dividing the estimated average caseload in the annual sample period (Step 1) by the unrounded average monthly sample size (Step 4). In our example, the sample interval is 42,600 ÷ 263.167, or 161 (rounded down). This means that each month, data will be collected for 1 out of every 161 TANF cases.

6. Select Random Start Number

The random start number can be as large as the number of cases contained in the sample interval and is used only to determine the first selected sample case for each month's sample. Since the sample interval in our example is 161, we must select a random start number between 001 and 161 (inclusive). Assume the number selected is 103.

7. Select Monthly Sample

The sequential position of the first selected sample case on the frame is the starting point for selection of all subsequent cases. (If the frame is in several parts, it will be desirable to assemble the parts so that one continuous list is created. Every "kth" case will then be selected from a list in which all cases are present.)

It is important in selecting the monthly sample to apply the same sample interval to the entire list of cases each month. This is an important part of the sampling design and should not be violated to obtain a specific number of cases each month.

In our illustration, if the sample interval was a whole number, the 103rd case on the list would be selected and every 161st case thereafter, i.e., 103rd, 264th, 425th, etc. In each of the remaining eleven months of the sample period, assuming no adjustment in estimated caseload size is necessary after the sixth month, new random starts would be obtained as the first case of each month and then multiples of 161 added to obtain the other cases for data collection and reporting.

The numbers selected for the sample cases relate to specific cases; substitutions or approximations are not acceptable. For example, only the 103rd case must be selected, not the 102nd, or 104th, etc. Once the random start and sample interval are determined, the specific cases to be selected are identified.

There are several methods of selecting sample cases when the sample interval is not a whole number. In one method, the sample case to be selected is determined by rounding the number obtained after the sample interval is added to the previous sample interval. For example, since in our illustration the sample interval number was actually 161.87 instead of 161, the following sample cases would be selected from the sample frame of eligible cases (assuming a random start number of 103):

Selected Cases	Selection Procedure
# 103	- random start case
# 265	103 + 161.87 = 264.87 = 265	- random start case - interval = total rounded
# 427	264.87 + 161.87 = 426.74 = 427	- previous total - interval = total rounded
# 589	426.74 + 161.87 = 588.61 = 589	- previous total - interval = total rounded
# 750	588.61 + 161.87 = 750.48 = 750	- previous total - interval = total rounded
etc.	etc.

For TANF (or SSP) purposes, an acceptable method for selecting sample cases when the sample interval is not a whole number is to round down to the next lower whole number and use that number in selecting the sample cases. For example, using the same sample interval of 161.87 and random start number of 103, the interval would be rounded down to 161 and the sample cases selected would be the 103rd, the 264th, the 425th, the 586th, 747th, etc.

8. Submission of Caseload Size, Sample Interval and Sample Cases Selected

If a STT opts to use systematic random sampling or stratified systematic random sampling, it should send the estimated average monthly caseload and the computed sample interval(s) to be used for the 12-month sample period to the ACF OFA RPM thirty (30) calendar days before the October sample selection.

If a STT uses a stratified sample design, it must submit the monthly caseload sizes by stratum (see the TANF Data Report - Section four and the SSP-MOE Data Report - Section four) for each month of the quarter within 45 days after the end of the quarter. These data are needed for weighting purposes.

Regardless of the method used to select the sample cases, each STT that opts to report on a sample of cases must submit the monthly list of selected sample cases (including reserve pool cases, if applicable, under Section 1531), within 10 days of the date of selection specified in the STT’s plan.

1440. Procedures for Selecting Sample Cases Using a Simple Random Sample

STTs may want to use simple random sampling or stratified simple random sampling because there are several computer software packages that contain programs that use this method of sampling. For illustrative purposes, the following procedures are based on the simple random sampling design and, if used, are repeated each month during the annual sample period. These same procedures could be used to select the sample of closed TANF (or SSP) cases or the sample of the active and closed SSP cases. Note, these procedures could be used to select a sample within each stratum for a stratified simple random sample design.

1. Establish the Monthly Sample Frame

As mentioned in Section 1421, a listing of all TANF (or SSP) cases that received assistance for the sample month by the end of the month (including initial assistance cases and cases that are reinstated) comprises the frame from which the sample is selected.

2. Determine the Number of Families on the Sample Frame

Many automated simple random sampling routines need to know the number of sampling units on the sample frame and the number of units to be selected prior to execution of the sample selection routine. For the TANF (or SSP) active sample, the sampling units are the families receiving TANF assistance. If a stratified simple random sample is used, the STT must determine the number of families in each stratum for the sample month.

3. Determine Sample Size

The minimum required annual sample sizes of completed cases are shown in Section 1410. In our illustration, the minimum sample size 3,000 is used.

n = 3,000

An estimate of the percent of cases that may possibly be reported as "listed- in-error" during the TANF (or SSP) data collection process will need to be made in order to arrive at the required completed sample size. For example, assuming that 5 percent of the selected cases will be reported as "listed-in- error", the number of cases to be selected can be computed as follows:

3,000 ÷ (1 - .05) = 3,158

4. Establish Average Monthly Sample Size

The average monthly sample size is obtained by dividing the sample size for the sample period by the number of months in the period. In our illustration, the average monthly sample size is 3,158 ÷ 12, which is 263.167 cases or 263 cases.

5. Select Monthly Sample

The most practical way of selecting a sample of TANF (or SSP) cases using a simple random sample is with the use of automated routines. These routines use a random number generator to select n (the number of units to be selected) out of N (the number of units on the sample frame). The n sample cases should be selected without replacement. To illustrate using a monthly sample frame with 42,600 families and a monthly sample size of 263 sample cases, the automated sampling routine would select 263 numbers between 1 and 42,600 inclusive. If the random numbers generated include 20, 175, 183. 500, etc., then the 20th, 175th, 183rd, and 500th case on the sample frame would be drawn into the sample.

6. Submission of Caseload Size, Sample Interval, and Sample Cases Selected

If a STT uses a stratified sample design, it must submit the monthly caseload sizes by stratum for each month of the quarter within 45 days after the end of the quarter. These data are needed for weighting purposes. STT that use non-stratified sample designs report their total monthly caseload numbers on the TANF (or SSP) Data Report - Section Three. These figures are used to weight the STT data.

Regardless of the method used to select the sample cases, each STT that opts to collect data for and report on a sample of cases must submit the monthly list of selected sample cases (including reserve pool cases, if applicable) within 10 days of the date of selection specified in the STT sampling plan.

1450. Retention of Sampling Records

The regulations at 45 CFR 92.42 set forth record retention and access requirements applicable to all financial and programmatic records, supporting documents, statistical records, and other records of grantees or subgrantees. Regarding record retention, 45 CFR 94.42(b) requires a 3-year period or longer, if any litigation, claim, negotiation, audit, or other action involving the records has been started before the expiration of the 3- year period. When one of the enumerated events occurs, the retention period extends “until completion of the action and resolution of all issues which arise from it, or until the end of the regular 3-year period, whichever is later.”

Each STT shall retain all sampling records for an annual sample period in accordance with the policy stated in the preceding paragraph. These materials shall include the

1. original monthly sample frames from which the sample was selected.

2. computer programs used to construct the sample frames and select the sample cases.

3. caseload estimate worksheets.

4. sample intervals and random start numbers.

5. sample size.

6. lists of selected cases, including supplemental and reserve pool cases if applicable.

7. audit trail tracking logs.

8. the quarterly TANF Data Reports and/or SSP-MOE Data Reports, and

9. the annual report containing information on the TANF program and, if applicable, the SSP-MOE program(s).

In addition, the STTs shall retain the approved sampling plan until a revised plan is approved and implemented. When the revised approved sampling plan is implemented, the previously approved sampling plan should be retained for three years. These materials are to be made available to ACF TANF Regional staff upon request.

1500. CORRECTION FOR UNDERSAMPLING

When using systematic random sampling, imprecise caseload projections or an unexpected drop rate will result in the STT not obtaining its target sample size. If the actual universe is larger than the estimated size, oversampling may occur. If the actual universe is smaller than the estimated size, undersampling may occur. A STT must correct for undersampling to the extent necessary to meet sample size requirements for TANF or SSP reporting. Oversampling does not need to be corrected, because we encourage STTs to select larger than the minimum required annual sample size to increase the precision of statistics that are estimated from the sample data.

In correcting the TANF or SSP sample size, care must be taken to assure that the statistical principles of "randomness" and measurability are not violated. The selection of additional families for the TANF and SSP samples or deletion of units from the samples must be done in a manner that assures all cases in the population have a known, non-zero probability of selection into the final sample. In addition, techniques of stratification should not be employed in such a way that small additional strata are created for which computed estimates may be unreliable, resulting in a loss of precision in population estimates.

The procedures that a STT uses to correct for undersampling will depend partly on the procedures the STT used to select its original sample cases. STTs may choose from a wide variety of sampling methods. STTs that select their TANF or SSP samples using the systematic sampling method can use the procedures in Sections 1510, 1520, or 1530 of this manual to adjust sample sizes. STTs that select their TANF or SSP samples using the simple random sampling method can use the procedures in Sections 1540 to adjust sample sizes. For STTs that use another method to select their TANF or SSP samples, ACF OFA Regional Office and DDCA staff will be happy to provide technical guidance on procedures to correct for undersampling to ensure that the principles of probability sampling are retained.

Monthly sample sizes should be monitored throughout the reporting period and correction should be made only when it becomes clear that target samples will not be met. It is good practice to re-estimate caseloads at the end of each quarterly reporting period. Waiting to the end of the annual period to make necessary corrections could create difficulties in collecting the information and adversely affect the STT’s ability to submit data in a timely manner.

The following procedures allow STTs to make corrections in all months starting with the first month of the reporting period. A consideration for a STT in selecting this method is that, in certain circumstances, it may be difficult to obtain accurate information for past months. This method does not involve the creation of additional strata.

1510. Standard Method to Correction for Undersampling when Sample Selected Using Systematic Random Sampling

1. Using the procedure described in Section 1430, Step 1, re-estimate the caseload size, adding on the expected number of cases to be dropped as listed-in-error, and compute a revised sample interval.

For each month in which the sample cases have already been selected:

2. Divide the size of the monthly sample frame by the revised sample interval (Step 1) to obtain the revised estimate of the number of sample cases that should have been selected.

3. Subtract the number of sample cases already selected from the number obtained in Step 2. This is the number of additional sample cases to be selected from the monthly frame.

4. Divide the total monthly sample frame size by the number identified in Step 3 to obtain the secondary sample interval to be used in selecting additional cases from the monthly sample frame.

5. Use a random start and apply the secondary sample interval obtained in Step 4 to the monthly sample frame from which cases have already been selected. (If correction for undersampling is required only for the third and/or fourth quarters of the annual period, the STT has the option of applying the secondary interval either to the first month of the sample period (October) or the first month of the applicable quarter (April or July)). Add the specific cases identified to the cases already selected for the same month as the month of the sample frame from which they were selected. If a case previously selected in the sample is again selected and identified for the same month as previously selected, an alternate case is to be selected by using a table of random numbers.

For months in the annual period for which sample cases have not yet been selected:

6. Use the corrected sample interval for the period obtained in Step 1 to select sample cases from the monthly frames.

1520. Alternate Method of Correcting for Undersampling When Sample Selected Using Systematic Random Sampling

An alternate method involves no adjustment for the months for which cases were already selected, however it does result in stratification of the sample by time. The alternative method entails the computation of a new sample interval that will oversample the remaining months of the annual period to meet sample size requirements if the earlier months had been undersampled.

Because two different sample intervals will have been used, results of cases selected by each sample interval cannot be directly added to obtain STT-wide estimates as the proportions of the monthly frames sampled are different, i.e., the total sample is not a self-weighting sample. 1/ The alternate method will require all data to be weighted at the end of the 12-month period. The procedure involves inflating the various frequencies (e.g., number of families with an adult working, the number of families with a minor parent head of household, cases with earned income, etc.) in cases obtained using each sample interval, to their representation in the caseload and dividing the result by the caseload. 2/ This gives the weighted rate for the STT. In order to make each of the frequencies (number of families with an adult that working, the number of families with a minor parent head of household, etc.) comparable with those of other STT, it is necessary to multiply the weighted rate by the total sample size. The equation for this procedure is as follows:

1/ It should be noted that a self-weighting sample, except for rounding, must possess the following characteristic:

Sample cases selected in a specific

month

Total sample cases selected in sample period

Cases in sample frame for same

month

=

Total cases in all sample frames in

sample period

2/ "Caseload", for the purpose, is defined as the completed sample size multiplied by the sample interval.

where:

= the sum of . . . all strata ("stratum" is defined as part of the annual period using the same sample interval).

= the stratum (m is the stratum index).

= "characteristic of interest" in the stratum.

= completed sample size in the stratum, and

= sample interval used in the stratum.

For example, assume that a STT originally had estimated that its caseload would average 80,000 cases for the annual sample period. Assuming a 5 percent drop rate, the STT used a sample interval of 303. Actual experience after 10 months resulted in the STT revising its average caseload to 75,000, making no change in its drop rate. If the STT made no corrections, the final completed sample size for the period would be short approximately 188 cases.

Assume that the STT decides to obtain the additional 188 cases by using a revised sample interval of 219 for the last 2 months of the sample period. Also assume for the first 10 months of the sample period (m=1) that the --

Number of cases completed ( n₁ ) = 2,350

Number of cases with "characteristic of interest" ( x₁ ) = 112 and

for the last 2 months of the sample period ( m = 2 ) that the --

Number of cases completed ( n₂ ) = 650

Number of cases with "characteristic of interest" ( x₂ ) = 37

Using the definition of "caseload" as defined earlier, i.e., sample cases completed multiplied by the sample interval, the weighted proportion of the case with the characteristic of interest would be computed as follows:

The STT case proportion for the "characteristic of interest" would be 0.0492. The reported number of cases with the characteristic of interest for the 12-month period, for comparability with other STTs, would be 148, i.e., 0.0492 x 3,000.

Note that each frequency of occurrence or proportion of the total sample must be calculated in the same way, e.g., number of families with an adult working, the number of families with a minor parent head of household, the number of child only cases, the number of cases with earned income, etc. Caseload weights are to be used in computing STT-wide characteristics.

Note, it is important that the appropriate code be entered on the coding schedule to identify the stratum from which the case was selected.

1530. Correcting for Undersampling Using a Reserve Sample Pool

Correcting for undersampling using the sample interval (see Section 1520) involves resampling the original frame using a new sample interval. A STT may find this to be difficult and/or costly. The same result can be achieved by selecting a reserve sample pool at the time of original sample selection. The designated reserve sample cases are to be used only if correction for undersampling is required. Properly selected reserve pool cases retain the self-weighting property of the final sample. However, careful attention to the controls is necessary to ensure that cases are properly selected. Any number of cases may be designated as a reserve pool - a good number could be 10 or 15 percent of the required sample size.

The STT sampling plan must describe in detail the procedures for setting up a reserve sample pool. If a random number generator is used, the type of generator and seed number is to be specified.

1531. Procedure for Setting Up a Reserve Sample Pool

Procedures for setting up a reserve sample pool are like those outlined in Section 1430. To illustrate the procedures, the example in Section 1430, is used, i.e., a STT uses the systematic random sampling method, elects the standard sample size, has an estimated average monthly caseload of 42,600 and estimates a 5 percent drop rate for the 12-month sample period. In addition, the STT specifies 15 percent of its selected sample as reserve pool cases each month.

1. Determine Average Monthly Sample Size

Divide the number of sample cases for which data is to be collected in the annual sample period by (1 - 0.15) to obtain the estimated total number of sample cases to be selected. In our example, according to Section 1430, Step 2, the number of sample cases (completed and dropped cases) is 3,158. The number of cases to be selected would be 3158 ÷ (0.85), or 3,715, or an average of 309 cases per month. The average number to be placed in a reserve pool each month is 15 percent of 309 cases, or 46 cases (rounded down).

Note that the reserve pool is only to be used to correct for undersampling; it is not to be used to replace dropped cases.

2. Select Monthly Sample

Using the monthly sample size from Step 1, 309 cases, and the procedures outlined in Section 1430, Steps 5, 6 and 7, compute the sample interval, determine a random start and select monthly sample cases from the sample frame.

3. Compute Secondary Interval for Selection of Reserve Pool Cases

Compute a secondary sample interval to be applied to the list of sample cases selected each month. This is obtained by dividing the estimated average monthly sample size by the average estimated number of cases designated for the reserve pool. In our illustration, the sample interval is 309 ÷ 46, or 6.72.

4. Select and Identify Monthly Reserve Pool Cases

Since the interval obtained in Step 3 above is not a whole number, the acceptable method is to round up to the next higher number. (Note that rounding up is recommended to ensure that the basic sample will have a sufficient number of cases.) In our example, 1 out of every 7 cases on the monthly list of selected sample cases would be identified for the reserve pool, using a random start number between 1 and 7 inclusive. It is important in selecting monthly reserve pool cases to apply the same sample interval to the entire list of selected cases each month. This is an important part of the sample design and should not be violated to obtain a specific number of reserve pool cases each month.

5. Submission of Sample Cases Selected

The estimated average caseload, the specified percentage of monthly selected sample cases for the reserve pool, the computed sample intervals, manually generated random start and seed numbers to be used in the 12-month sample period for selection of total sample cases and reserve pool cases should be sent to the ACF OFA Regional Office thirty (30) calendar days before the October sample selection. The monthly list of selected sample cases, with reserve pool cases identified, and computer-generated random start and seed numbers should be submitted within 10 days of the date of selection specified in the STT sampling plan.

If random numbers are used to identify cases for the reserve pool, it is essential that the total number of sample cases selected each month is known.

The following procedures are to be used and repeated each month if random numbers are used.

6. Determine the total number of sample cases selected. In our example, assume that the number selected for October was 309.

7. Multiply the number obtained in Step 1 by the percentage of selected sample cases specified for the reserve pool. In our illustration, the number of cases to be placed in a reserve pool for October is 309 x .15, or 46 cases (rounded down). Note that the same percentage must be applied each month.

8. Randomly select and identify reserve pool cases. In our example, 46 random numbers between 1 and 309, inclusive, would be selected.

1532. Procedure for Obtaining Cases from a Reserve Sample Pool

STTs with reserve pools must use the same procedures in correcting for undersampling as outlined in Section 1520 and 1530. A revised estimate of the number of sample cases that should have been selected (excluding reserve pool cases) is to be computed. The difference between the number that should have been selected and the number that was selected is the number of additional sample cases that will need to be selected from the reserve pool.

If a STT uses a disproportionate stratified sample design, it must maintain a separate reserve sample pool for each stratum. If a STT with a disproportionate stratified sample design undersamples, it must use the allocation procedures specified in its sampling plan to determine in which stratum (or strata) the STT has undersampled. The STT must correct for undersampling in each stratum in which undersampling occurred.

The same primary sample interval as determined in Section 1430, Step 5, Step 2, must be used to select sample cases for months in the annual period for which such cases have not yet been selected. However, a new secondary sample interval to be applied to the monthly lists of selected sample cases must be used in identifying cases for the reserve pool.

The revised secondary sample interval is the product of the original secondary sample interval and the number of cases in the reserve sample pool prior to selection of cases from the reserve sample pool divided by the number of cases remaining in the reserve sample pool after selection of cases from the reserve sample pool.

1540. Correction for Undersampling When Sample Was Selected Using Simple Random Sampling

As described in Sections 1510 and 1520, there are two basic approaches to correcting the annual samples. The first approach is to correct the sample for both the months for which the sample has already been selected and the months for which the sample has not been selected. This approach provides an annual sample with approximately one-twelfth of the sample selected each month. The second approach is to make the entire adjustment in the months for which the sample has not yet been selected. Monthly samples selected using simple random sampling are less likely to need large adjustments for undersampling than for samples selected using systematic random sampling. This is true because under simple random sampling a fix number of sampling units is selected each month regardless of the monthly caseload. Under systematic random sampling, a fixed proportion of the caseload is selected each month. However, caseloads can vary from month to month. This variation of the monthly caseload results in variation in the monthly sample size.

1541. Correcting for Undersampling

If a small correction (e.g., less than 50 cases) is needed to ensure the STT will meet its minimum required annual sample size and no month is substantially short of the approximate one-twelfth of the annual sample, then the STT should correct for undersamping by adjusting the sample size in months for which the sample has not yet been selected. If the sample for all months have been selected, then the adjustments should be made for the months in the last quarter of the fiscal year. On the other hand, if a large correction is needed for months in which the sample have already been selected, the STT should consider adjusting all monthly samples. To correct a monthly sample for undersampling, use the following procedures:

1. Retrieve the Original Monthly Sample Frame

As required under Section 1450, STTs must retain their original monthly sample frames. The STTs should locate the original month sample frame for use in selecting the additional sample cases.

2. Review Original Determinations of Total Monthly Caseload and Average Monthly Sample Size

Review the original application of the sample selection procedures from Section 1440 Steps 2, 3, and 4 to identify the reason for undersampling (e.g., underestimated the number of listed-in-error cases).

3. Determine the Number of Additional Sample Cases Needed

Subtract the number of completed cases for the sample month from the required number of sample cases for the month to determine the short fall. Allowing for some additional listed-in-error cases (using the procedures in Section 1440, Step 4, determine the number of additional sample cases to be selected from the original sample frame.

4. Select the Additional Sample Cases

Using the same procedures as in Section 1440, Step 5, select the addition sample cases from the sample frame and forward the sample selection list to the ACF Regional Office.

9