CMS-10166.Supporting Statement Part B

Supporting Statement Part B

1. To measure improper payments for PERM, 17 States from a total of 50 States plus the District of Columbia (i.e., ‘51’ States) were selected each year to create a three year rotation cycle. The States were rank-ordered by their past Federal fee-for-service (FFS) expenditures and grouped into the four strata of 17 States each for three PERM cycles (3x17=51). This distribution of States is shown in Table 1.

Table 1: State Strata Defined

Stratum	Cycle 1	Cycle 2	Cycle 3
1A	3	3	3
1B	3	3	2
2	6	5	6
3	5	6	6
Total	17	17	17

Claims are grouped into strata by service type before sampling. The FFS annual sample size for each State is state specific but the base sample size is 500.

The sample design for PERM is typically referred to as a two-phase sampling approach, where the first stage refers to the sampling of states and the second stage refers to the sampling of line items within a state.

All sampled FFS claims receive a data processing review; sampled FFS claims that are not denied or Medicare crossover claims receive medical review. The final PERM payment error rate reports will contain national Medicaid and CHIP rates that include FFS, managed care, and eligibility components, as prescribed by Public Law 107-300.

The anticipated response rate for all facets of PERM should approach 100% due to the regulatory requirement under Final Rule CMS 6062-F 42 CFR 431.970. Previous periods of performance of PERM have shown that most States comply at the 100% level for their programs.

2. The PERM program must estimate a national Medicaid error rate that covers the fifty states and District of Columbia. According to IPIA requirements, the estimated national error rate must be bound by a 90% confidence interval of 2.5 percentage points in either direction of the estimate. To achieve this goal, the PERM program will sample 17 states, without replacement, each year. Sampling a different 17 states each year ensures that all states are sampled only once in a span of 3 years.

The variance of the estimator for this design is quite complex, and it is difficult to numerically solve for the sample size. The equations and assumptions used are described below.

2i

From Cochran (1977), the variance for a total from a two stage sampling is¹:

)S

(1)Var( Y^ˆ

) ^N ( 1 f

_Y_i Y ²

• N _

M_i( 1 

^f2i ²

2

)

u _n 1

N 1 n m_i

where:

N=total states in universe n=total states sampled ^f1^{=proportion of states sampled}

^Yi^{=projected dollars in error for a state M}i^{=total units in state i}

^mi^{=total units sampled in state i f}2i^{=proportion of units sampled from state i S}2i^{2=sample variance of errors within state i}

The goal is to determine the expected value of the variance of the projected dollars in error.

Letting

2. ^σYB^{2 =}

_Y_i Y ²

N 1

^{denote the variance in projected error between states, using the identity Y}i^=Ri^Pi ^{, and assuming that all R}i ^{are equal, then σ}YB^{2=R2* σ}PB^{2, where σ}PB^{2 is the variance in payments} between states.

2. Next, for each state-level estimate, the variance of the error rate is given by:

2i

M_i( 1 f_2i )S ²

2. Var( R^ˆ _i )

² 

R_i

m_{i Pi}2

_2

_ EW

_Pi2

^{where P}i ^{is the total payments for the state, and σ}EW^{2 is the variance of the projected} dollars in error within a state.

¹ Note that in practice, PERM will utilize a slightly different estimator that utilizes the error rate for each state, as opposed to the projected dollars in error for each state. However, the above formula could be used as well, and the resulting sample size calculation of states would yield similar results.

The goal is to determine the expected variation in projected dollars in error within a state, which depends on the state selected (the first stage). Hence, we want:

(5) E(²

) E(²

* P² ) ²

E( P² ) ²

(² ² )

iw _Ri i

_Ri i

_Ri P P

which is attained only when assuming independence between the error rate and payments, which appears to be a reasonable assumption.

^{Additionally, note that σ}Ri^{2 is currently designed such that the precision met is .03, with} 95% confidence. For generality, we will denote the desired state level precision as d, then ^{note that σ}Ri^{2=d2/z2 where z is the standard normal of a designated level of confidence.}

Combining this, equation (1) can be rewritten as:

2

 ² 

(6) Var( Y^ˆ

N

) 

( 1 f

)R²² ^N n^{d }(² ² )

u _n 1

p _{n z 2 } P P

 

All that remains is to find the variance of the error rate, which for the Office of Inspector General (OIG) “difference” estimator would simply be:

ˆ

(7)V ( R^ˆ _u )

_V( Y_u ) _P2

Note that the sample size calculations in these derivations were for simple random sampling schemes. For stratified random sampling schemes, the same procedure is used, partitioned into the respective strata, and each individual result combined in a standard fashion.²

Sampling with strata is done as follows:

1. Sorted the data first by paid amount

2. Calculated the total payments for universe

3. Defined strata: sorted claims in descending order, such that each stratum represents 10 percent of expenditures

4. Determined the skip factor for each stratum (denoted by k_i). Let

th

N_i denoted the universe number of claims for the i

k ^Ni

i _n

stratum in a State

i

5. Determined a random start value for each stratum (denoted by start_i), such that

th

1 start_i

k_i

(i denotes the i

strata)

² A stratified sample is simply a series of simple random samples, combined together.

6. th

   Sampled every k_i

item within the i^th

stratum

The estimation procedure thus accounts for the nesting of claims within payment methods within program types within States. The error rate calculations utilize the Intra-class Correlation Coefficient to properly adjust for similarities within the nested structures in the data. In doing so, the PERM SC has chosen to use a Separate, Separate, Combined Estimator (SSC). This method represents a mixture of two methods: the combined ratio estimator and the separate ratio estimator. It is not documented in standard sampling textbooks, but the estimator and its standard error are straightforward to formulate and have been used for PERM in FY 2006 and FY 2007. The discussion is divided into the three steps for the estimator.

Stage 1: SRE for Combining Stratified Results

First, the PERM sample design has four State strata, determined by their expenditure amounts. The estimates and standard errors can be assumed to be produced for each State stratum.

The SR estimator is given by:

a

10. SSC  i i

    R^ˆ  S R^ˆ

i1

where

_tu

_p_i

share of expenditures for State stratum i (sum across all strata equals 1)

S_{i u}

t _p

i

R^ˆ 

later

error rate for stratum i, as determined by a ratio estimator, to be described

i denotes the State stratum (i=1 to 4)

The variance for the SSC is given by:

a

10. SSC  i i

    Var(R^ˆ )  S ²Var(R^ˆ )

i1

Note the variance of the stratum specific error rate will be derived in later steps.

Stage 2: SR estimator for State Stratums

Within each State stratum, individual rates are estimated. The application of the separate ratio estimator occurs again when creating these State stratum rates. The State stratum rate will be the weighted combination of the State specific rates, with the weights being the relative shares of expenditures. Therefore,

n_i

10. i  ij ij

    R^ˆ  S R^ˆ

j 1

where

^Sij



t

u ^pij

t

u p_i

share of expenditures for State j in State stratum i (sum of all strata

equals 1)

R



ij

^ˆ error rate for State j in stratum i, as determined by a ratio estimator, to be

described later

i denotes the State stratum (i=1 to 4) j denotes the State (i=1 to n_i)

The variance follows the properties of a three stage sample design, where the selection of States is the first stage, the selection of program type is the second stage, and the selection of the sampling units (claims) within payment method is the third stage. The variance of this portion of the estimator is given by:

ˆ

(13)²

R_i

VarR^ˆ VarER^ˆ

EVarR^ˆ 

i

i

i

i

i

Let

n_i n_i N_i

i  ij ij  ij ij  ij ij

(14) R^ˆ  S R^ˆ  S R^ˆ  S ^/ R^ˆ

j 1

j 1

j 1

such that

/

^S_i ^_i

15. S_ij

_

_0 _i

Then continuing from (13),

ⁿⁱ 

^Ni 

15. i  ij ij  ij ^ˆ

    Var R^ˆ Var_ S R _E_ S _{R }



_j 1

 

_{ }j 1

/ 2 2 _

^eij _

_n n_i

n

S R

• _S_ij  _R

2

ij ij

j 1

2 2

N

ˆ

^eij

The estimated variance is given by

2

 ⁿⁱ _ˆ 



_n ^ ^Sij ^Rij ^

ⁱ j 1 ^



_^Sij

^Rˆij





^ n ^

j 1 _



n_{i }



_n n_i

(17)V^ˆarR^ˆ

n ^_1 ^{i   }

_S ²^{ˆ 2}



N

i i

 i 

n_i 1

ij

N

j 1

^Rˆeij

Stage 3: Combined estimator for State Stratums

Where

^{ˆ 2}

^Rˆeij

can vary based on the estimator employed for estimating rates at the State

level. For the combined ratio estimator, the State level error rates are estimated by:

^{a m}k

_tˆ ^Wk ^ekl

(18) R^ˆ 

f (t^ˆ , t^ˆ

_{) }e _k 1 l 1

e p _tˆ

^{a m}k

p

^Wk  ^pkl

k 1

l 1

where:

^a _M ^mk

^{a m}k

t

e

k

ˆ _{m}

^ekl

^ ^Wk ^ekl

k 1

k ^{l 1}

k 1

l 1

^a _M ^mk

^{a m}k

ˆ

k

t _p _m

^pkl

^ ^Wk  ^pkl

^{k } k ^{l }

k  l 

1 1 1 1

m_k are the number of claims sampled from strata k

M_k are the number of claims or line items in the universe from strata k e_kl represents the error on the lth claim in the kth stratum

p_kl represents the payment on the lth claim in the kth stratum

Then estimated variance is given by:

^nk 

2

_{a a} ^ _(e_kl  R^ˆp_kl  (e_k  R^ˆp_k )) ^

k

kl

1 ₂

1

^{2 }l 1 ^



(19) V^ˆar(R^ˆ ) 

t

_ˆ2

p

^Wk

k 1

n V^ˆar(e

R^ˆp ) 

t

kl

_ˆ2

p

^Wk

k 1

n_{k }





n 1 ^

k





The needed accuracy is provided by the IPIA and should be no more than an anticipated

+/- 3% margin of error at a 95% confidence level for payment error rates at the State program level, and no more than an anticipated +/- 2.5% margin of error at a 90% confidence level for payment error rates at the national program level.

In order to meet the requirements of IPIA, all selected States must fully participate.

3. Most States have been quite responsive, so non-response is a minimal issue for PERM. The accuracy and the reliability for PERM are specified by federal regulations and supported by appropriate sample sizes. For these reasons, the information collected should be appropriate for its intended purposes. Reliable data are expected because the PERM SC compares the States’ data with their CMS 64 and CMS 21 submissions for Medicaid and CHIP, respectively. Further, States are subject to an OIG audit on their PERM submissions.

3. Not applicable.

3. Contact information

The Lewin Group

3130 Fairview Park Drive Falls Church, VA

2