1205-0332 Supporting Statement Part B

Tax Performance System (TPS)

OMB Control No. 1205-0332

Supporting Statement

Part B

Collections of Information Involving Statistical Methods

Introductory Note.

Social Security Act section 303(a)(1) gives the Secretary of Labor several responsibilities toward the Unemployment Insurance (UI) system. Among these is to oversee the performance of the system, and to ensure that it provides “full payment of unemployment compensation when due.” In general, this includes ensuring that states are in substantial compliance with their laws, which must embody the requirements of Federal law. The Department is reviewing the timeliness, accuracy, and completeness of certain tax collection (revenue) operations in the states utilizing the Tax Performance System (TPS) (formerly known as the Revenue Quality Control (RQC) program.) This request is for an extension of the TPS program.

The design combines four different evaluative approaches. Some involve statistical sampling. As part of the "Program Review" methodology, small Acceptance Samples (AS) are drawn from various processes' outputs to confirm that the processes' internal controls work as intended to yield accurate results. The objective of the Program Review methodology, as of the program audits upon which it is based, is to make a judgment of reasonable assurance of accuracy--not to produce a point estimate of the accuracy/inaccuracy rate.

Acceptance Sampling differs considerably in concept from the more common estimation sampling. Estimation (or enumerative) sampling seeks to infer the size or rate of occurrence of something--in this case, some measurement of an attribute such as accuracy--within a universe or population. It usually implies a null hypothesis that the population value equals or exceeds a desired value for the attribute. For example, if the standard is that a program function be at least 95% accurate, a sample would be drawn with the objective of estimating the accuracy rate (percentage) for the population and specifying the lower limit of the confidence interval that includes the universe value at the given level of probability. The probability specified is the ability to avoid rejecting the null hypothesis when the hypothesis is true (statistically, this known as making a Type I error). The assumed population value, the estimated variance, the precision desired and degree of confidence determine the sample size. Estimation samples often form the beginning of a process of further investigation of causes of errors or accuracy rates for population subgroups.

The objective of Acceptance Sampling is to indicate economically whether or not certain events (usually, errors or exceptions) occur at or below some specified frequency referred to as the "acceptable quality level" (AQL). An initial step is to examine the process and assess its risk of producing errors. An AQL is set to represent the upper level of the rate of exceptions produced by the process that can be tolerated. Sample size is determined by the size of the population being inspected; the AQL (e.g., error rate or exception rate); and the degree of confidence desired. The design of Acceptance Samples balances the risk of rejecting (failing) a process that meets the AQL (Type I error) and accepting (passing) a process that produces exceptions above the AQL (Type II error).

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

Data are being collected from 52 states (50 states plus the District of Columbia and Puerto Rico). Samples are drawn from universes of completed actions (e.g., new employer status determinations, field audits, and benefit charging). The potential respondent universe and size for each AS appears in Table 1. The range is based on data from two states, Montana, and California, which contain some of the smallest and largest employer populations, and so indicate the upper and lower limits for each.

Table 1: Potential Respondent Universe*

Type of Completed Action	Universe Minimum (Montana)	Universe Maximum (California)	Sample Size	Exception Rate
Status - New Determinations	8,381 / year	209,436/ year	60 per year	2 percent
Status - Successor Determinations	621 / year	910/ year	60 per year	2 percent
Status - Inactive/ Termination Determinations	6,134/ year	138,071/ year	60 per year	2 percent
Report Delinquency - Delinquent Accounts	3,683 / quarter	77,077/ quarter	60 in one quarter	2 percent
Collections - Accounts Receivable	529 / at given point in time	46,931/ at given point in time	60 at the point in time	2 percent
Field Audit – Audits	547/ year	4,920 / year	60 per year	2 percent
Contribution Reports	48,815 / quarter	992,105 / quarter	60 in one quarter	2 percent
Billings - Contributory Employers	987 / quarter	15,571/ quarter	60 in one quarter	2 percent
Billings - Reimbursing Employers	19 / quarter	370 / quarter	up to 60 in one quarter	2 percent
Credits / Refunds	851 / quarter	11,600 / quarter	up to 60 in one quarter	2 percent
Benefit Charging - Statements	5,081/ quarter	307,735/ quarter	60 in one quarter	2 percent
Tax Rates – Notices	31,731 / year	1,468,439 / year	60 in a year	2 percent

*Based on state data reported in calendar year (CY) 2022. Exception: Montana reviewed tax rates in CY 2019 and California reviewed tax rates in CY 2020. Tax rates are reviewed once every four years.

Response rates per se are not relevant, because verification merely involves retrieving information relevant to a determination from primary source records, which are maintained by the state agencies. Occasionally, however, a sampled case cannot be verified because documentation cannot be located. Under such circumstances, the missing case will be treated as a nonresponse, and the results of the AS will be evaluated from the non-missing cases. Only one such missing case is allowed; if additional cases cannot be evaluated because of missing documents, the state cannot claim a reasonable assurance of accuracy, and the AS will fail. The state must document the reason for the missing case materials and provide those actions it will take to ensure the availability of case documentation for future samples.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

a. Methodology for Acceptance Sample selection. States are given instructions on how to assemble the "transactions" (universe) files for each AS. If the sampling is to occur in an automated environment, the state has options for proceeding. It can use the software provided as part of the TPS software, or the state can select the sample in the same way using the state’s application software or a commercial statistical package. In both cases, the samples are drawn using a balanced systematic (interval) sampling method. The universe is arrayed according to a prescribed key (in most cases, employer account number). A sampling interval is obtained by dividing the universe by the number of cases to be selected; and a random start number is applied to pick the first case. The remaining cases are picked by applying the interval. Instructions are also provided for selecting samples manually, however, all states have an automated process.

b. Methodology for Estimation Sample Selection. For the Cashiering tax function, data are collected for the sole purpose of determining whether the state has met timely deposit requirement of 90% or more remittances deposited into the clearing account within three days or less of receipt. This is the only part of the TPS program which uses an estimation sample rather than an acceptance sample. This consists of computing a skip interval, k, which equals N*/n (where N* is the estimated population size and n is the sample size), rounded to the nearest integer. The first selection, i, is randomly selected between 1 and k. Subsequent selections are: i + k, i + 2k, ..., i + (n-1)k. Because the population size is unknown, the skip interval must be estimated. For example, a state estimates that the number of checks that will be received is 50,000. A sample of 500 checks will be selected, and the skip interval is computed: k=50,000/500,k=100.

Because it is unlikely that the actual population is 50,000, the sample size will not be exactly 500, but will vary according to the actual size of the population. The true population size is estimated by k*n', where n' is the sample produced by the estimated skip interval k. For example, if the actual population is 52,000, the skip interval will produce a sample of 520, not the targeted 500, and k*n'=100*520 or 52,000.

Several states separate large remittances, for example through separate post office boxes. Different cutoff points are set for large remittances, and states must ensure that the sample is representative with respect to these separately collected remittances.

Estimation Procedure for Cashiering

1. Ratio Estimate of Timeliness Rate

The parameter to be estimated, R_o, is the ratio of Unemployment Insurance (UI) tax remittances deposited into bank clearing accounts within 3 days to total UI tax remittances: R_o = Y/X, where Y = Total dollars in the population deposited into bank clearing accounts within 3 days and X = Total UI tax remittances in the population.

R_o is estimated by the sample ratio:

r_o =

where:

n = Number of tax remittances sampled.

x_i = Amount of UI tax submitted for the i^th case.

y_i = Amount of UI tax deposited into the bank clearing account within 3 days for the i^th case.

2. Sampling Variance of Ratio Estimate of Timeliness Rate

The following formula will be used to estimate the sampling variance of the ratio estimate of UI tax remittances deposited into bank clearing accounts within 3 days to total UI tax remittances.

Note: Because the sampling fractions, f=n/N, are negligible, the term (1-f) has been omitted from the equations.

estVar(r_o)=

where:

is the sample variance of the amount of UI tax deposited into the bank clearing account within 3 days;

is the sample variance of the amount of UI tax submitted; and

is the sample covariance of the amount of UI tax submitted and the amount of UI tax deposited into the bank clearing account within 3 days.

x = Total amount of UI tax submitted.

3. Confidence Intervals

The 90% and 95% confidence intervals for the estimated ratio (r_o) are:

r_o ± 1.645 * sqrt (estVar(r_o))

r_o ± 1.96 * sqrt (estVar(r_o))

The sampling instructions indicate that the sampling selection period will cover the time during which the state receives the peak mail for the quarter’s reports. Whether this is a span of weeks or days, the reviewer is to identify, based on historical data, the five peak days and to sample a total of 500 remittances from those five days. The instructions also include a chart that gives the critical values to pass for various sample sizes for the percentages estimated from the samples. Unless the population estimate is grossly inaccurate, the samples will fall within the ranges shown in table 2 below, and the appropriate critical values are used to determine if the state has met the 90 percent standard (P).

Table 2: Value to Pass for Various Sample Sizes

Sample Is Value

Between To Pass

----------- -------

375 and 405 87.5

406 and 441 87.6

442 and 481 87.7

482 and 527 87.8

528 and 579 87.9

580 and 640 88.0

Value to pass (p*):

p* = 90 - [100 * (1.645 * sqrt (var (P)/n)],

where:

var (P) = P * (1-P) = .9 * .1 = .09,

n = sample size, and

P = 90 percent standard

1.645 is the value of the standard normal deviate (z), appropriate for 90 percent of the cumulative standard normal distribution.

Effect of Electronic Remittances

Employers may submit their tax payments electronically. Because these electronic fund transfers (EFT) are deposited directly to the clearing account, 100% of them meet the 3-day deposit standard. As more employers adopt EFT, the percentage of the non-EFT remittances that must be deposited within three days (in order to meet the 90 percent standard for all remittances) decreases.

The following tables show the effect of the percentage of employers using EFT on the value to pass for the non-EFT cashiering sample. Type I and Type II errors are shown for the various EFT percentages and sample sizes.

Type I Error:

If the state has no EFT deposits, the Cashiering sample cases must meet the 90 percent deposit standard.

Let:

p = the proportion of sampled tax remittances deposited within three days; and

SE(p) = the sampling error of the estimated proportion, p.

Assuming that the sampling distribution is normal, we compute a standard normal variable, z:

z = (p - .90) / SE(p).

Because values of p equal to or greater than .90 (90 percent) will “meet” the standard, we are only interested in determining if values of p less than .90 are unlikely, assuming a population value of .90 and taking sampling error into account. We define unlikely as a less than 5 percent probability (p < .05) for samples of 500 and less than 10 percent probability (p < .10) for samples of 400, 300, or 200.

From the normal distribution, the probability that z is less than or equal to -1.645 is 5 percent. So, values of z greater than or equal to -1.645 will “pass” (that is, meet the 90 percent standard) and values of z less than -1.645 will “fail”, because there is a less than 5 percent probability of observing a rate this low, if the true (population) rate is 90 percent or more. The corresponding z value for samples of 400, 300, or 200 (p < .10) is -1.282.

If the state has EFTs, the percentage of the non-EFT remittances that must be deposited within three days (in order to meet the 90 percent standard for all remittances) decreases, according to values in table 3. For example, if 50 percent of the tax remittances are EFTs, the minimum percentage of non-EFT remittances that must be deposited within three days in order to meet the deposit standard is reduced to 80 percent.

For Type I error calculations, the EFT-adjusted value to pass (p*) is used in the formula z = (p - p*) / SE(p) instead of .90. For example, if 50 percent of the remittances are EFTs, 80 percent of the non-EFT remittances for a sample of 500 must be deposited within three days. (z = (p - .80) / SE(p))

Percentage of EFTs	Min. Pct. of Non-EFTs	Minimum Value To Pass With Sample Size Of:*
	Needed For 90% Timely	500	400	300	200
50.0%	80.0%	77.1%	77.4%	77.0%	76.4%
55.0%	77.8%	74.7%	75.1%	74.7%	74.0%
60.0%	75.0%	71.8%	72.2%	71.8%	71.1%
65.0%	71.4%	68.1%	68.5%	68.1%	67.3%
70.0%	66.7%	63.2%	63.6%	63.2%	62.4%
75.0%	60.0%	56.4%	56.9%	56.4%	55.6%
80.0%	50.0%	46.3%	46.8%	46.3%	45.5%
85.0%	33.3%	29.9%	30.3%	29.8%	29.1%
90.0%	0.00%
* Minimum values to pass are set so that the probability of failing when the true value is greater than or equal to the minimum percentage of non-EFTs needed to meet the 90% timeliness standard (col. 2) is less than or equal to 5% (Type I error p=.05) for samples of 500, and is less than or equal to 10% (Type I error p=.10) for samples of 400, 300 or 200. Table 4: Minimum Values to Pass for Various Percentages of EFTs and Sample Size 500

Table 3: Minimum Percent of Non-EFT Timely Deposits Needed to Pass the 90% Timeliness Standard

Percentage of EFTs	Min. Pct. of Non-EFTs	Minimum Value To Pass With Sample Size Of: 500*
	Needed For 90% Timely	Type I = 5%
50.0%	80.0%	77.1%
55.0%	77.8%	74.7%
60.0%	75.0%	71.8%
65.0%	71.4%	68.1%
70.0%	66.7%	63.2%
75.0%	60.0%	56.4%
80.0%	50.0%	46.3%
85.0%	33.3%	29.9%
* Minimum values to pass are set so that the probability of failing when the true value is greater than or equal to the minimum percentage of non-EFTs needed to meet the 90% timeliness standard (col.2) is less than or equal to 5% (Type 1 error p=.05).
Table 5: Sampling Error Sensitivity Analysis for Various Levels of Passing
Percentage of EFTs	Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
	Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
		2.5		5		7.5	10
50.0%	80.0%	75.0%		70.0%		65.0%	60.0%
55.0%	77.8%	72.2%		66.7%		61.1%	55.6%
60.0%	75.0%	68.8%		62.5%		56.3%	50.0%
65.0%	71.4%	64.3%		57.1%		50.0%	42.9%
70.0%	66.7%	58.3%		50.0%		41.7%	33.3%
75.0%	60.0%	50.0%		40.0%		30.0%	20.0%
80.0%	50.0%	37.5%		25.0%		12.5%
85.0%	33.3%	16.7%

Table 6: Sampling Error Sensitivity Analysis for Various Levels of Passing

Sampling Errors*	Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
	Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
		2.5	5	7.5	10
	1.79%	1.94%	2.05%	2.13%	2.19%
	1.86%	2.00%	2.11%	2.18%	2.22%
	1.94%	2.07%	2.17%	2.22%	2.24%
	2.02%	2.14%	2.21%	2.24%	2.21%
	2.11%	2.20%	2.24%	2.20%	2.11%
	2.19%	2.24%	2.19%	2.05%	1.79%
	2.24%	2.17%	1.94%	1.48%
	2.11%	1.67%
*where the sampling error SE(p) = sqrt[(p×(1-p))/499] and each cell’s sampling error uses p from the prior table above. Table 7: Type II Errors - Probability of passing if combined EFT-Non-EFT result is x percentage points below 90%.
Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
	0	2.5	5	7.5	10
80.0%	0.9500	0.1440	0.0003	0.0000	0.0000
77.8%	0.9500	0.1063	0.0001	0.0000	0.0000
75.0%	0.9500	0.0697	0.0000	0.0000	0.0000
71.4%	0.9500	0.0373	0.0000	0.0000	0.0000
66.7%	0.9500	0.0137	0.0000	0.0000	0.0000
60.0%	0.9500	0.0021	0.0000	0.0000	0.0000
50.0%	0.9500	0.0000	0.0000	0.0000
33.3%	0.9500	0.0000

Type II Error:

Let:

p = the proportion of sampled tax remittances deposited within three days; and

SE(p) = the sampling error of the estimated proportion, p.

P* = the minimum percentage of non-EFTs needed to pass the three-day deposit requirement in order to meet the 90 percent timeliness standard.

Assuming that the sampling distribution is normal, we compute a standard normal variable, z:

z = (p - p*) / SE(p).

The cumulative probability of z is displayed in the Type II error table, above.

Example: If 50 percent of the remittances are EFTs, the minimum percentage of the non-EFT sample needed to pass is 80%. If the combined EFT-Non-EFT result is 85 percent (5 percentage points below the 90% standard, then:

.85 = .5 + .5p

p = (.85 - .5) / .5

p = .35 / .5

p = .70.

SE(p) = sqrt[(.7 * (1 - .7)) / 499]

SE(p) = .0205

From the table above, the minimum value to pass for an EFT rate of 50 percent and a sample of 500 is 77.1 percent (.771).

Therefore:

z = (.70 - .771) / .0205

z = -3.46

Prob. (z) < -3.46 = .0003

The probability of passing (Type II error) is .0003.

Table 8: Minimum Values to Pass for Various Percentages of EFTs and Sample Size 400
Percentage of EFTs	Min. Pct. of Non-EFTs	Minimum Value To Pass With Sample Size Of: 400*
	Needed For 90% Timely	Type I = 10%
50.0%	80.0%	77.4%
55.0%	77.8%	75.1%
60.0%	75.0%	72.2%
65.0%	71.4%	68.5%
70.0%	66.7%	63.6%
75.0%	60.0%	56.9%
80.0%	50.0%	46.8%
85.0%	33.3%	30.3%
* Minimum values to pass are set so that the probability of failing when the true value is greater than or equal to the minimum percentage of non-EFTs needed to meet the 90% timeliness standard (col. 2) is less than or equal to 10% (Type I error p=.10).
Table 9: Sampling Error Sensitivity Analysis for Various Levels of Passing
Percentage of EFTs	Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
	Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
		2.5	5	7.5	10
50.0%	80.0%	75.0%	70.0%	65.0%	60.0%
55.0%	77.8%	72.2%	66.7%	61.1%	55.6%
60.0%	75.0%	68.8%	62.5%	56.3%	50.0%
65.0%	71.4%	64.3%	57.1%	50.0%	42.9%
70.0%	66.7%	58.3%	50.0%	41.7%	33.3%
75.0%	60.0%	50.0%	40.0%	30.0%	20.0%
80.0%	50.0%	37.5%	25.0%	12.5%
85.0%	33.3%	16.7%

Table 10: Sampling Error Sensitivity Analysis for Various Levels of Passing
Sampling Errors*	Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
	Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
		2.5	5	7.5	10
50.0%	2.00%	2.17%	2.29%	2.38%	2.45%
55.0%	2.08%	2.24%	2.36%	2.44%	2.48%
60.0%	2.17%	2.32%	2.42%	2.48%	2.50%
65.0%	2.26%	2.40%	2.47%	2.50%	2.47%
70.0%	2.36%	2.47%	2.50%	2.47%	2.36%
75.0%	2.45%	2.50%	2.45%	2.29%	2.00%
80.0%	2.50%	2.42%	2.17%	1.65%
85.0%	2.36%	1.86%
*where the sampling error SE(p) = sqrt[(p×(1-p))/399] and each cell’s sampling error uses p from the prior table above. Table 11: Type II Errors - Probability of passing if combined EFT-Non-EFT result is x percentage points below 90%
Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
	0	2.5	5	7.5	10
80.0%	0.9001	0.1303	0.0006	0.0000	0.0000
77.8%	0.9001	0.0984	0.0002	0.0000	0.0000
75.0%	0.9001	0.0669	0.0000	0.0000	0.0000
71.4%	0.9001	0.0381	0.0000	0.0000	0.0000
66.7%	0.9001	0.0156	0.0000	0.0000	0.0000
60.0%	0.9001	0.0030	0.0000	0.0000	0.0000
50.0%	0.9001	0.0001	0.0000	0.0000
33.3%	0.9001	0.0000
Example: If the minimum pct. needed to pass is 80% and the combined EFT-Non-EFT Result is 5 pct. pts. below the 90% standard (85%) the probability of passing is .0006.

Table 12: Minimum Values to Pass for Various Percentages of EFTs and Sample Size 300

Percentage of EFTs	Min. Pct. of Non-EFTs	Minimum Value To Pass With Sample Size Of: 300*
	Needed For 90% Timely	Type I = 10%
50.0%	80.0%	77.0%
55.0%	77.8%	74.7%
60.0%	75.0%	71.8%
65.0%	71.4%	68.1%
70.0%	66.7%	63.2%
75.0%	60.0%	56.4%
80.0%	50.0%	46.3%
85.0%	33.3%	29.8%
* Minimum values to pass are set so that the probability of failing when the true value is greater than or equal to the minimum percentage of non-EFTs needed to meet the 90% timeliness standard (col. 2) is less than or equal to 10% (Type I error p=.10).
Table 13: Sampling Error Sensitivity Analysis for Various Levels of Passing
Percentage of EFTs	Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
	Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
		2.5	5	7.5	10
50.0%	80.0%	75.0%	70.0%	65.0%	60.0%
55.0%	77.8%	72.2%	66.7%	61.1%	55.6%
60.0%	75.0%	68.8%	62.5%	56.3%	50.0%
65.0%	71.4%	64.3%	57.1%	50.0%	42.9%
70.0%	66.7%	58.3%	50.0%	41.7%	33.3%
75.0%	60.0%	50.0%	40.0%	30.0%	20.0%
80.0%	50.0%	37.5%	25.0%	12.5%
85.0%	33.3%	16.7%

Table 14: Sampling Error Sensitivity Analysis for Various Levels of Passing
Sampling Errors*	Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
	Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
		2.5	5	7.5	10
50.0%	2.31%	2.50%	2.65%	2.75%	2.83%
55.0%	2.40%	2.59%	2.72%	2.81%	2.87%
60.0%	2.50%	2.68%	2.80%	2.86%	2.89%
65.0%	2.61%	2.77%	2.86%	2.89%	2.86%
70.0%	2.72%	2.85%	2.89%	2.85%	2.72%
75.0%	2.83%	2.89%	2.83%	2.65%	2.31%
80.0%	2.89%	2.80%	2.50%	1.91%
85.0%	2.72%	2.15%
*where the sampling error SE(p) = sqrt[(p×(1-p))/299] and each cell’s sampling error uses p from the prior table above. Table 15: Type II Errors - Probability of passing if combined EFT-Non-EFT result is x percentage points below 90%
Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
	0	2.5	5	7.5	10
80.0%	0.9001	0.2073	0.0039	0.0000	0.0000
77.8%	0.9001	0.1689	0.0016	0.0000	0.0000
75.0%	0.9001	0.1276	0.0004	0.0000	0.0000
71.4%	0.9001	0.0848	0.0001	0.0000	0.0000
66.7%	0.9001	0.0444	0.0000	0.0000	0.0000
60.0%	0.9001	0.0136	0.0000	0.0000	0.0000
50.0%	0.9001	0.0008	0.0000	0.0000
33.3%	0.9001	0.0000
Example: If the minimum pct. needed to pass is 80% and the combined EFT-Non-EFT Result is 5 pct. pts. below the 90% standard (85%) the probability of passing is .0039.

Table 16: Minimum Values to Pass for Various Percentages of EFTs and Sample Size 200

Percentage of EFTs	Min. Pct. of Non-EFTs	Minimum Value To Pass With Sample Size Of: 200*
	Needed For 90% Timely	Type I = 10%
50.0%	80.0%	76.4%
55.0%	77.8%	74.0%
60.0%	75.0%	71.1%
65.0%	71.4%	67.3%
70.0%	66.7%	62.4%
75.0%	60.0%	55.6%
80.0%	50.0%	45.5%
85.0%	33.3%	29.1%
* Minimum values to pass are set so that the probability of failing when the true value is greater than or equal to the minimum percentage of non-EFTs needed to meet the 90% timeliness standard (col.2) is less than or equal to 10% (Type I error p=.10). Table 17: Sampling Error Sensitivity Analysis for Various Levels of Passing
Percentage of EFTs	Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
	Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
		2.5	5	7.5	10
50.0%	80.0%	75.0%	70.0%	65.0%	60.0%
55.0%	77.8%	72.2%	66.7%	61.1%	55.6%
60.0%	75.0%	68.8%	62.5%	56.3%	50.0%
65.0%	71.4%	64.3%	57.1%	50.0%	42.9%
70.0%	66.7%	58.3%	50.0%	41.7%	33.3%
75.0%	60.0%	50.0%	40.0%	30.0%	20.0%
80.0%	50.0%	37.5%	25.0%	12.5%
85.0%	33.3%	16.7%
Table 18: Sampling Error Sensitivity Analysis for Various Levels of Passing
Sampling Errors*	Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
	Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
		2.5	5	7.5	10
50.0%	2.84%	3.07%	3.25%	3.38%	3.47%
55.0%	2.95%	3.18%	3.34%	3.46%	3.52%
60.0%	3.07%	3.28%	3.43%	3.52%	3.54%
65.0%	3.20%	3.40%	3.51%	3.54%	3.51%
70.0%	3.34%	3.50%	3.54%	3.50%	3.34%
75.0%	3.47%	3.54%	3.47%	3.25%	2.84%
80.0%	3.54%	3.43%	3.07%	2.34%
85.0%	3.34%	2.64%
*where the sampling error SE(p) = sqrt[(p×(1-p))/199] and each cell’s sampling error uses p from the prior table above.
Table 19: Type II Errors - Probability of passing if combined EFT-Non-EFT result is x percentage points below 90%.
Min. Pct. of Non-EFTs	Non-EFT Sample Producing Combined EFT-Non-EFT Result
Needed For 90% Timely	X Percentage Points Below Min. Pct. Needed To Pass:
	0	2.5	5	7.5	10
80.0%	0.9001	0.3283	0.0250	0.0004	0.0000
77.8%	0.9001	0.2878	0.0141	0.0001	0.0000
75.0%	0.9001	0.2406	0.0063	0.0000	0.0000
71.4%	0.9001	0.1856	0.0019	0.0000	0.0000
66.7%	0.9001	0.1233	0.0002	0.0000	0.0000
60.0%	0.9001	0.0588	0.0000	0.0000	0.0000
50.0%	0.9001	0.0102	0.0000	0.0000
33.3%	0.9001	0.0000
Example: If the minimum pct. needed to pass is 80% and the combined EFT-Non-EFT Result is 5 pct. pts. below the 90% standard (85%) the probability of passing is .0250.

c. Degree of accuracy needed for the purpose described in the justification. As noted, the objective of the TPS investigations is to establish reasonable assurance of accuracy, taking into account findings of both the reviews of system controls ("Systems Review") and the AS. The level of "reasonable assurance" was determined through discussions with state tax staff, particularly top-level tax administrators, who were asked what level of inaccuracy in a given tax function would induce them to take corrective action. As a result of these discussions, the Department decided to use an AQL of 98% for all functions except remittances and accounts of active contributory employers. Samples of 60 cases, with up to 2 exceptions allowed, will be used to balance the risks of penalizing states with acceptable systems (Type I error) or passing states with unacceptably high exception rates (Type II error). The following chart shows the probability of passing for various population error rates.

Chart 1: Probability of Acceptance Sample Passing

Note: Probabilities are based on the binomial distribution. It is assumed that the sampling fractions are small and that differences from the probabilities for a hypergeometric distribution, which assumes sampling from a finite population without replacement, are minor.

For the Cashiering sampling process, the following table shows the critical values for the test of the null hypothesis that the population percentage is greater than or equal to 90 percent (H₀: P > .9), with the risk of a type I error of 5 percent and the risk of a type II error of 10 percent. The results are stated as percentages (below).

Value Minimum

Sample To Pass Pct. Passed

400 87.5 85.3

500 87.8 85.8

600 88.0 86.2

Value to pass (p*):

p* = 90 - [100 * (1.645 * sqrt (var (P)/n))],

where:

var (P) = P * (1-P) = .9 * .1 = .09,

n = sample size, and 1.645 is the value of the standard normal deviate (z), appropriate for 95 percent of the cumulative standard normal distribution.

Ninety-five percent of the samples of the indicated size selected from a population in which timeliness is equal to or greater than 90 percent will be equal to or greater than the percentage in the "Value To Pass" column. These samples will pass the test.

Five percent of the samples will be below the value to pass and will fail the test, even though the actual percentage is 90 percent or greater.

Ten percent of the samples of the indicated size selected from a population in which timeliness is equal to the percentage in the "Minimum Percent Passed" column will be equal to or greater than the percentage in the "Value To Pass" column. These samples will pass the test. Ninety percent of the samples will be below the value to pass and will fail the test.

The minimum percent passed (p’) is the minimum value that satisfies the condition:

p’ + [100 * (1.282 * sqrt (var (p’)/n))] > p*

where:

var (p’) = p’ * (1-p’),

n = sample size, and 1.282 is the value of the standard normal deviate (z), appropriate for 90 percent of the cumulative standard normal distribution.

d. Unusual problems requiring specialized sampling procedures. Not applicable.

e. Use of less frequent sampling to reduce burden. It has been decided that AS need to be drawn annually to monitor the health of the various tax functions, since systems reviews will only be done every 4 years, unless a problem was discovered in the year before or the state introduced a system change.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

The acceptance samples will be drawn from existing agency records; therefore, nonresponse is not an issue. Should documentation for an entire employer’s file be missing, the missing case will be treated as a nonresponse, and the results of the AS will be evaluated from the non-missing cases. Acceptance samples with more than one missing case will fail because the state has not demonstrated a reasonable assurance of accuracy.

Nonresponse is not an issue for the Cashiering Estimation Sample. As discussed in section B-2, the sampling selection period covers the time during which the state receives the peak mail for the quarter’s tax remittances. The state reviewer selects a target sample of 500 remittances from the peak mail days identified. The deposit times for the samples are determined from bank and agency records to determine whether the state has met timely deposit requirement of 90% or more remittances deposited into the clearing account within three days or less of receipt.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

Various parts of the design have been tested at least once. The systems reviews were pretested in 6 States in 1990; their comments on the workability of the design led to considerable modification of the questions. (No AS were drawn, nor data results submitted to the Department during the pretest). A full-scale pilot test, including AS and computed measures, was conducted in 8 other states. This test gathered data on the results of systems reviews and AS, the degree that they confirmed one another, and the time required to program and collect the various kinds of information. The test also refined the questions further. These pilot studies were conducted from 1991 through 1992. Pilot evaluations were part of the original PRA Supporting Statement submitted to OMB as part of the initial approval process.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Mr. Daniel Sommers, U.S. Department of Labor, Office of Unemployment Insurance, Division of Performance Management, 200 Constitution Avenue, NW, Room S-4519, Washington, DC 20210, (202) 693-3197.

B - 29