CMS - 10794 - Supporting Statement B Final

Supporting Statement B

B. Collections of Information Employing Statistical Methods.

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When Item 17 on the OMB Form 83-I is marked "Yes," the following documentation should be included in the Supporting Statement to the extent that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, state and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The respondent universe consists of all policies in State Exchanges that have received an APTC greater than $0 during the benefit year under consideration. There are 18 State Exchanges at this point in time.

State Exchange	Number of Policies Receiving APTC (BY 2019)
CA	1,065,504
CO	98,768
CT	69,382
DC	1,241
ID	56,741
MA	356,707
MD	108,291
ME	46,298
MN	unavailable*
NJ	144,119
NM	29,336
NV	52,859
NY	154,115
OR	93,092
PA	250,638
RI	32,023
VT	24,575
WA	121,918
*At the time we retrieved the data, no records for Minnesota were available for BY 2019.

2. Describe the procedures for the collection of information including:

- Statistical methodology for stratification and sample selection.

- Estimation procedure.

- Degree of accuracy needed for the purpose described in the

justification.

- Unusual problems requiring specialized sampling procedures, and

- Any use of periodic (less frequent than annual) data collection cycles

to reduce burden.

The SEIPM program is the process for determining estimated improper payments and other information statutorily required under the Payment Integrity Information Act of 2019 (PIIA), and implementing guidance, for APTC.

Planned Sample Confidence Level and Margin of Error

The SEIPM will be designed to produce a point estimate of the improper payment rate in accordance with the guidance provided in Appendix C to OMB Circular A-123, and an associated confidence level and margin of error. CMS is targeting a confidence level of 95% and a margin of error of 3 percentage points for each State Exchange (SE) based on consultation with statisticians and in consideration of the current and historical OMB guidance, the parameters of other improper payment measurement programs administered by CMS, program characteristics and available resources. The error rate as reported in aggregate across all the SEs would have a margin of error less than 3 percentage points with high probability.

Stratification

Within the context of SEIPM, the goals for stratification include:

1. Controlling the sample size for individual Exchanges

2. Ensuring precise estimates for individual Exchanges

3. Ensuring the cost of review is predictable and reasonable, and

4. Improving the precision of the overall estimate after aggregating across strata

18 strata would be produced – one for each of the 18 expected SEs. Once the 18 stratum-level improper payment rates and their MOEs have been estimated, estimating the improper payment rate of any aggregation of strata and its MOE can be done using the statistical properties of the variances of random variables. Estimates for individual SE improper payment rates will be combined to produce an estimate of an aggregated SE improper payment rate. The estimate of the FFE improper payment rate will be reported separately.

Individual SE improper payment rate estimates and MOEs will be provided to the SEs and not be published in public medium. The aggregate SE improper payment rate estimate will be reported in the AFR.

Sampling Unit and Sampling Frame

CMS intends to use tax households as the sampling unit, and all tax households for which net positive APTC payments were made for the subject benefit year as the sampling frame. Note that “negative cases” (that is, instances where no net positive APTC payments were made, for example, where eligibility for the APTC program was denied) are not included within the sampling frame and are outside of the scope of measurement due to the nature of the APTC program. Specifically, recognizing any such improper payments would require significant speculation about the monthly payment amount the consumer would have elected and the duration of monthly payments during a benefit year.

Expected Sample Size and Methodology to Estimate

CMS currently estimates a sample size of approximately 100 tax households for each SE. This sample size was determined in part by calculating the variance of the expected improper payment rate at 2%, based on FFE pilot outcomes, as a function of sample size and determining the smallest sample size for which the margin of error is less than three percentage points at a 95% confidence level in the aggregate. Ratio estimation will be used to estimate the improper payment rate. The complete methodology was created by CMS.

Implications of Population Size on Expected Sample Size

For the sampling rate being considered, population size is not a major influence on the expected margin of error. This can be explained based on the finite population correction factor, which is multiplied by the usual variance formula when the sampling rate is high. The SE with the smallest population size in plan year 2019 was Washington, DC. Its population size (1,241) is still much larger than the proposed sample size for this exchange (100). Under simple random sampling, the finite population correction factor would be equal to,

,

multiplying the uncorrected margin of error by . So, the finite population correction would reduce the margin of error by only 4% if it were applied for DC. For the other SEs, the impact on the margin of error would be even smaller. See table below. The small impact led CMS to decide upon a constant sample size across exchanges for the sake of simplicity.

State Exchange	Number of Policies Receiving APTC (Benefit Year 2019)	Sampling Rate
CA	1,065,504	0.01%
CO	98,768	0.10%
CT	69,382	0.14%
DC	1,241	8.06%
ID	56,741	0.18%
MA	356,707	0.03%
MD	108,291	0.09%
ME	46,298	0.22%
MN*	Not available
NJ	144,119	0.07%
NM	29,336	0.34%
NV	52,859	0.19%
NY	154,115	0.06%
OR	93,092	0.11%
PA	250,638	0.04%
RI	32,023	0.31%
VT	24,575	0.41%
WA	121,918	0.08%
* At the time we retrieved the data, no records for Minnesota were available for BY 2019.

Overpayments

Due to unique characteristics of the Advance payment of Premium Tax Credit program, there is not currently a mechanism to recover overpayments identified by the SEIPM program.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Responses from State Exchanges will be required by the proposed regulation. CMS expects a 100% response rate.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

CMS has completed pilot testing with the Federally-Facilitated Exchanges (FFE) and has recently begun the first cycle of improper payment measurements. The knowledge that has been gained through the FFE will be used as a foundation for the State Exchange Improper Payment Measurement. Additionally, CMS is currently engaged in different levels of pilot testing with several states to test improper payment measurement procedures and data sharing methods. All testing will be complete prior to beginning collection of data as described in this form.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The contract for the statistical contractor who will perform the statistical work associated with this contract has not yet been awarded.