CHIPRA-10 Sample Design Memo_for OMB

P.O. Box 2393
Princeton, NJ 08543-2393
Telephone (609) 799-3535
Fax (609) 799-0005
www.mathematica-mpr.com

MEMORANDUM

TO:

CHIPRA 10-State Evaluation Team

FROM:

Kimberly Smith and Christopher Trenholm

SUBJECT:

Sample Design for the Survey of Enrollees and Disenrollees

DATE: 10/25/2011

This memorandum describes the sample design for the household survey of CHIP and
Medicaid enrollees and disenrollees to be conducted as part of the CHIPRA 10-state evaluation.
It serves as a follow on to the design report by providing further specification of several key
aspects of the sample design previously described. First, it presents refined definitions of the
three sample domains that comprise the target population for the survey. Second, it specifies the
target sample sizes for each state and sample domain, based on the “compromise allocation”
approach to sample allocation discussed in the design report. Finally, it presents the level of
precision provided by the specified sample design.
The household survey will be administered to parents and guardians of current and former
CHIP enrollees in 10 states: Alabama, California, Florida, Louisiana, Michigan, New York,
Ohio, Texas, Utah, and Virginia. In 3 of these states—California, Florida, and Texas—the survey
will also be administered to parents and guardians of Medicaid enrollees. Information will be
collected on the characteristics of these children, their movement in and out of the programs, and
their experiences accessing and using health care. The sample will be designed so that survey
data can be used for three purposes: (1) to make inferences about the three enrollment domains
for each state; (2) to make comparisons among comparable domains across states; and (3) to
make comparisons between CHIP and Medicaid enrollment domains within the three states
selected for the Medicaid survey.
In the remainder of this memorandum, we specify the target population for the survey, how
the sample will be allocated across states and domains, and the resulting precision of the
parameter estimates generated from the survey data.
TARGET POPULATION
The target population for the CHIP and Medicaid samples will be drawn from three domains
(or sub-populations) of enrollees in each program: (1) new enrollees, (2) established enrollees,
and (3) disenrollees. Two exclusion criteria will be used to further limit the target population:
age of the child at the time of sampling and the child’s basis of eligibility for CHIP or Medicaid

An Affirmative Action/Equal Opportunity Employer

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
2
(as discussed below). 1 For each sample domain, the start of enrollment will be based on the date
of eligibility determination and the end of enrollment will be based on the date of disenrollment.
These two dates are chosen because they best approximate the time when a parent would
consider the child to be enrolled or to be disenrolled. Because these dates may be among several
shown in the administrative files, they may need to be identified with the assistance of the state.
Depending on the state, CHIP may be administered either (1) separately from Medicaid, (2)
as an expansion to Medicaid, or (3) as a combination of these two. For the CHIP survey, the
target population includes both possible administrative models; that is, it includes both
enrollees/disenrollees in the separately-administered (S-CHIP) component and the Medicaidexpansion (M-CHIP) component. Note that for the states in the third group—the so-called
“combination states” that administer both components—the sample size for each component will
be proportional to the respective population size in the domain. Having included all M-CHIP
enrollees/disenrollees in the target population for the CHIP survey, the target population for the
Medicaid survey excludes these children. Thus, for the Medicaid survey, the target population
includes only enrollees/disenrollees in the “traditional’ (Title XIX-funded) Medicaid program
and NOT those in the M-CHIP component of the program.
Below, for both the CHIP survey and the Medicaid survey, we present definitions and
stratification schemes for each sample domain.
1.

New Enrollees
Sample Definition: A child enrolled in the specified program (CHIP or Medicaid) during
two consecutive months, preceded by a gap in coverage of at least one month. This
corresponds to a monthly coverage spell of “011”, where 0 equals a month without coverage
in the specified program and 1 equals a month with coverage.
Note that, in some study states, children may be newly enrolled in CHIP or Medicaid under
a “presumptive eligibility” policy, which provides temporary coverage while a final
eligibility determination is made. This period of temporary coverage can typically last for up
to two months, requiring a slightly modified definition of new enrollment for these children
(see below).
Sample Restrictions and Refinements:

1

Sample eligibility determinations will be made based on the state administrative data at the time of sample
frame construction and not based on respondent self-reports at the time of the interview, with one exception.
Children who are reported as “deceased” at the time of the interview will be excluded from the sample.

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
3

• Age range: The child must be at least one year (12 months) of age and less than 19 years
of age.
• Excluded eligibility groups: The child’s basis of eligibility (BOE) must be household
income; any child with an alternate BOE at the time of sampling will be excluded. This
restriction pertains largely to Medicaid and spans a large number of eligibility groups,
including: blind/disabled, SSI, institutionalized, foster care, qualified as Medically
Needy; or received partial benefits because of dual eligibility for Medicare, immigrant
status, or other reasons. Some assistance may be needed from the state to identify all of
these restricted groups.
•

Treatment of temporary (“presumptive”) eligibility: If a sample child has an eligibility
code reflecting presumptive eligibility, s/he will be held until the next data extract is
provided by the state and a final eligibility determination is made. If that determination
results in continued enrollment, the sample child remains in the target population and is
released to the SOC for interview. Otherwise, the sample child is dropped from the target
population. As with the eligibility codes for exclusion, some assistance from states may
be needed to identify the relevant code(s) for this temporary eligibility status.

Stratification for Sampling:
•

Pre-enrollment coverage: The new enrollee population will be stratified into three
groups that characterize a child’s recent coverage transition:
1. “Churners”: New enrollees in CHIP/Medicaid who are returning to the same
program after a gap in coverage of just one, two, to three months (i.e., 1011,
10011, or 100011).
2. “Transfers”: New enrollees in one program (CHIP/Medicaid) who are
transferring from the other program (Medicaid/CHIP) following a gap in
coverage of zero, one, two or three months.
3. “Clean cases”: New enrollees who are neither churners nor transfers, as defined
above.
As described further in the design report, we will only sample for interview children
in the third “clean cases” group in order to maximize the analytic value of the

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
4
overall study sample. 2 Note that transfers and churners will be retained for eventual
analysis; for example, drawing on their public coverage histories from the
administrative data, we can pool these two groups with the “clean cases” and
construct an accurate estimate of prior insurance coverage among all new CHIP (or
Medicaid) enrollees.
• Income: In California and New York, new enrollees will be further divided into two
income groups: (1) upper-income and (2) lower-income. Children in households
above 200 percent of the federal poverty line (FPL) are considered upper-income. As
specified below, the purpose of this stratification is to oversample upper-income new
enrollees—a population that is relatively small in both California and New York but
that holds considerable interest given the anticipated expansion of public coverage
under health reform. 3
2. Established Enrollees
Sample Definition: A child enrolled in the specified program (CHIP or Medicaid) for 12
consecutive months. 4
Sample Restrictions and Refinements:
•

Age range: The child must be at least one year (12 months) of age and less than 19 years
of age.

2

Survey data on new enrollees who either churn or transfer have limited value to our analysis—for two
reasons. First, based on our experience from the prior CHIP survey, few parent(s) of these children even recognize a
new enrollment has taken place because their short gap in coverage and/or their transfer between programs goes
unnoticed. Thus, these parent(s) are unable to report reliably on anything related to their new enrollment experience.
Second, because these children’s prior coverage history reflects a period of public coverage, it does not serve as a
valid counterfactual for measuring the impacts of CHIP. Thus, we would make little use of any data we might
collect on their health or health care outcomes prior to enrolling.
3

Two other states, Alabama and Louisiana, also provide CHIP coverage to children above 200 percent FPL.
However, the relative size of these populations is too small to permit oversampling at the level necessary to obtain
meaningful estimates for this subgroup.
4

The enrollment period for established enrollees in the prior survey was 5 months. The period was extended to
12 months to facilitate comparisons of key survey outcomes—such as health care access and use—to benchmark
measures from validated national and state surveys.

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
5
•

Excluded eligibility groups: All children whose BOE is not income-based will be
excluded from the CHIP and Medicaid samples (see new enrollee definition for list of
excluded groups).

Stratification for Sampling:
• Income: In California and New York, established enrollees will be stratified into
upper- and lower-income groups, where upper-income is defined as above 200
percent of the FPL.
3.

Recent Disenrollees
Sample Definition: A child who has been disenrolled in the specified program (CHIP or
Medicaid) for at least one month and who was previously enrolled for at least two months
prior to their month of disenrollment. This corresponds to a monthly coverage spell of
“110”, where 0 equals a month without coverage in the specified program and 1 equals a
month with coverage.

.
Sample Restrictions and Refinements:
•

Age range: The child must be at least one year (12 months) of age and less than 20 years
of age.

•

Excluded eligibility groups: All children whose BOE is not income-based will be
excluded from the CHIP and Medicaid samples (see new enrollee definition for list of
excluded groups).

Stratification for Sampling:
•

Post-disenrollment coverage: The recent disenrollee domain will be stratified into
three groups that characterize the child’s coverage status after disenrollment:
1. “Churners”: Children recently disenrolled from CHIP/Medicaid who return to
the same program after a gap in coverage of just one month (i.e., 1101).
2. “Transfers”: Children recently disenrolled from one program (CHIP/Medicaid)
who transfer to the other program (Medicaid/CHIP) following a gap in coverage
of zero months or one month.
3. “Clean cases”: Recent disenrollees who are neither churners nor transfers, as
defined above.

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
6
As with the new enrollee domain, we will only sample for interview recent
disenrollees in the third “clean cases” group. Relevant data on the characteristics of
these sample members and their coverage transitions will be obtained from
administrative records.
• Income: In California and New York, recent disenrollees will be stratified by income
using the same stratification scheme described above for new enrollees.
SAMPLE ALLOCATION AND SAMPLE SIZES
The most basic allocation of the study sample across the CHIP and Medicaid states is an
“equal allocation”, whereby we aim to complete interviews with the parent(s)/guardian(s) of 500
children in each of the three sample domains across the ten CHIP states and three Medicaid
states. This would yield a total survey sample of 15,000 CHIP children and 4,500 Medicaid
children, equal to the total sample specified in the RFP for the evaluation. As described in the
design report, however, two factors lead us to employ an alternate approach to allocating the
survey sample across states and domains. First, to better understand the characteristics and
experiences of enrolled children (particularly subgroups), we want to obtain a larger sample of
established enrollees. Second, to minimize the design effects associated with pooling data, we
want to increase the sample sizes in larger states. 5
In addition, to better understand the experiences of children from households with relatively
high incomes—a key group targeted by upcoming health reforms—we will sample within each
sample domain a disproportionate share of children in households above 200 percent of the
FPL. 6 While four survey states—Alabama, California, Louisiana, and New York—have CHIP
eligibility limits exceeding 200 percent of the FPL, we will restrict our analyses of this sub-group
to the two states with relatively sizable populations of upper-income enrollees: California and
New York. Given the small proportion of enrollees in income bands above 200 percent of the
FPL, oversampling is essential for obtaining precise estimates of this group.
Our recommended sample allocation is, therefore, a function of four interrelated constraints
and considerations: (1) the total sample size for the study, (2) the sample size required to produce

5

The sample design assumes that we will use sample weights to account for differences in the size of the
CHIP/Medicaid populations across states and to obtain pooled estimates that are representative at the 10-state level.
6

As mentioned previously, in the new enrollees and disenrollee domains, all transfers or churners will be
included in the final analysis sample, but not interviewed. Therefore, oversampling will only occur with the third
(“All other children”) stratum in these two sample domains.

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
7
sufficiently precise within-state descriptive statistics on key sub-groups of established enrollees,
(3) the minimum sample size needed to obtain reasonably precise within-state estimates in small
states (and within-stratum estimates of upper-income enrollees in California and New York), and
(4) the maximum number of sample members that can be allocated to the largest states. The
closer the sample size “ceiling” is to the sample size that would be allocated to the largest states
under proportional allocation, the lower the design effect and greater the precision of our pooled
estimates. This final parameter is determined by the first three.
In order to examine the precision of various sample sizes and sample allocations across
domains and states, we first estimated the likely design effects associated with clustering and
non-response adjustments, and the unequal weighting arising from various sample allocations.
Next, using these design effects, we analyzed the confidence interval (CI) half widths for a series
of descriptive statistics, calculated for different combinations of states, domains and subgroups.
Finally, to assess the available precision when comparing outcomes among samples (for
example, between new and established enrollees) or among sub-groups (for example, defined by
race and ethnicity or other demographics), we estimated minimum detectable differences, or
“MDDs,” for alternative sample sizes. 7 Based on these calculations, we determined that a target
sample size of 5800 established enrollees would provide sufficient precision for anticipated subgroup outcomes at the state and 10-state level. This also allows for precise estimates of outcomes
in the new enrollees and disenrollee domains, which will each have a target sample size of 4600
under this allocation. The minimum sample size required to generate reasonable precise within
state/stratum estimates was determined to be 400.
To account for the design effects of oversampling upper-income children in California and
New York to achieve a pooled sample size of 400 for separate analysis of this sub-group, we
made a slight modification to the compromise allocation method used for the new enrollee and
disenrollee domains. To ensure that the effective sample sizes for these two sample domains in
California and New York are sufficient to meet our analysis objectives after taking into account
these larger design effects, we based the compromise allocation on a total sample size of 4500
(rather than 4600) for each of these domains. In doing so, we reserved 200 sample members who
we then allocated to the new enrollee and disenrollee samples in California and New York (50
sample members per domain in each state).

7

We calculated all MDDs with powers of 80 percent for two-tailed tests of significance with 95 percent
confidence.

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
8
Table 1 presents the resulting target sample sizes for each state and sample domain for the
CHIP survey, using the compromise allocation approach described above. 8 In the established
enrollee domain, this allocation allows for roughly 1025 sample members in states with large
enrollee populations (California and Texas) and 400 sample members in the six states whose
sample size would fall below that minimum under proportional allocation: Alabama, Louisiana,
Michigan, Ohio, Utah, and Virginia. In the new enrollee and disenrollee domains, the maximum
number of sample members allocated to large states is significant lower at 590, while the
minimum is the same. The allocation of 50 additional sample members to each of these domains
in California and New York results in New York having a larger sample than Texas, even though
Texas has a larger CHIP population. The bottom panel of Table 1 shows target sample sizes by
income stratum for California and New York.

Table 1: Target Sample Sizes (Completed Interviews) for the CHIP Survey, by State and Domain
Established Enrollees

New Enrollees

9

Recent Disenrollees

Total Survey Sample
Alabama
California

400
1025

400
590

400
590

Florida
Louisiana

621
400

482
400

482
400

Michigan
New York

400
752

400
590

400
590

Ohio
Texas

400
1025

400
540

400
540

Utah
Virginia

400
400

400
400

400
400

5823

4602

4602

Total

Upper-Income/Lower-Income Sub-Sample
California
New York

200/825
200/552

200/390
200/390

200/390
200/390

8

The target sample sizes for the Medicaid survey are presented in Appendix Table A1. We used the same
compromise allocation approach described for CHIP survey to allocate the total Medicaid sample of 4,500
households across the three Medicaid states and sample domains.
9

Note that the final analysis sample will exceed the sample sizes shown in Table 1 in cases where transfers and
churners are included in the new enrollee and disenrollee analysis samples.

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
9
PRECISION OF SAMPLE ALLOCATION AND SAMPLE SIZES
In this section, we present the precision and power provided by the specified sample design.
First, in Table 2, we present confidence interval-half width estimates for a range of descriptive
statistics. 10 We focus on three illustrative proportional outcomes having the following sample
means: (1) 50 percent; (2) 25 percent (or, equivalently, 75 percent); and (3) 10 percent (or, again
equivalently, 90 percent). In each row of the table, we display for each illustrative outcome the
associated CI half width for a specified sample size and sample composition of interest. The
results in Table 2 show that there is clearly sufficient precision for anticipated outcomes across
all three domains when pooled across states. This is true whether the outcomes focus on the full
population or on subgroups. For example, for outcomes measured for a full established enrollee
sample (shown in the first panel of Table 1), the half widths are 2.0 percentage points for a 50
percent proportion, 1.7 points for a 25/75 percent proportion, and 1.2 points for a 10/90 percent
proportion. Half widths naturally rise when focusing on subgroups. However, even for a 25
percent subgroup within this domain, the half widths for the illustrative outcomes are less than 4
percentage points. 11 The second panel of Table 2 shows half-widths for the new enrollee and
disenrollee estimates. While the half-widths are larger due to the smaller size of these domains,
we will still be able to obtain a sufficient level of precision for full sample and sub-group pooled
estimates.
While the primary focus of the evaluation will be on outcomes and subgroups defined across
states—an approach we adopted successfully for the prior study—we also plan to explore these
outcomes at the state level as well. As seen in the lower rows of the two panels in Table 2,
precision falls when focusing on state-specific outcomes. For a full sample established enrollee
domain in the largest states (California and Texas), the largest half width shown (for a proportion
of 50 percent) is 4.0 percentage points. This number increases to 5.9 percentage points for the
smallest states in the sample. For the recent enrollee and disenrollee samples, the half-width for a
50 percent proportion in the largest states is 5.0 and in the smallest states 5.9.

10

Appendix Table A2 presents confidence interval half-widths estimates for outcomes based on the Medicaid

sample.
11

Based on findings from the earlier study, a 25 percent subgroup approximates many of the focal subgroups
for the evaluation, including children with elevated health care needs, children in low-education households, and
children in Spanish-speaking households.

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
10

Table 2. Confidence Interval (CI) Half Widths for Illustrative CHIP Outcomes
Estimated CI Half Widths for Illustrative Proportions
(shown in percentage points)
Mean=50%

Mean=25% (or 75%)

Mean=10% (or 90%)

(E.g., Had Recent
Preventive Visit)

(E.g., Has Elevated
Health Care Need)

(E.g., Has Unmet Dental
Need)

5,800 [full sample domain]

2.0

1.7

1.2

2,900 [50% domain subgroup]
1,450 [25% domain subgroup]
Individual State

2.6
3.6

2.3
3.2

1.6
2.2

1025 [domain in largest state: CA]

4.0

3.5

2.4

400 [domain in smaller state; e.g. UT]
Upper Income

5.9

5.1

3.6

400 [CA and NY Pooled]

5.9

5.1

3.6

Sample Size [Composition]
Established Enrollees
Ten States Pooled (CHIP Sample)

New Enrollees and Recent Disenrollees
Ten States Pooled (CHIP Sample)

`

4,600 [full sample domain]
2,300 [50% domain subgroup]

2.4
3.2

2.0
2.8

1.4
1.9

1,150 [25% domain subgroup]
Individual State

4.5

3.9

2.7

590 [domain in largest state: CA]
400 [domain in smaller state; e.g. UT]
Upper Income

5.0
5.9

4.3
5.1

3.0
3.6

400 [CA and NY Pooled]

5.9

5.1

3.6

Notes: The confidence interval half width is equal to the standard error of an outcome multiplied by the standard normal deviate used in a
95% confidence interval, 1.96. Standard errors have been adjusted to reflect the expected design effect under a compromise allocation of
sample members to states (see text for details).

Next, in Table 3, we present MDDs for comparisons of two sample domains for illustrative
proportions given the sample allocation discussed above. 12 When pooling the 10 states’ data and
comparing outcomes between the established enrollee and new enrollee domains (top panel; row

12

Appendix Table A3 presents MDDs for comparisons of Medicaid sample domains.

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
11
one), we have sufficient statistical power to detect differences of 4.8 percentage points for a
proportion of 50 percent and differences of 4.1 percentage points for a proportion of 25 (or 75)
percent. These differences are relatively modest—both equivalent to effect sizes of just over 10
percent (not shown), which is commonly considered “small” in social science research (Cohen
1988). MDDs naturally increase for comparisons of subgroups, but they remain around levels
that can detect meaningful differences at desired power. For example, for a comparison between
domains for a 50 percent subgroup (top panel; row two), the MDD on a 50 percent proportion is
6.3 percentage points, again equivalent to a “small” effect size. Perhaps not surprisingly,
comparisons between domains or other subgroups within a single state have relatively weak
statistical power, particularly for smaller states (not shown). We assume that the study of such
within-state differences will be a relatively low priority for this study, as it was for the prior
evaluation.
Table 3. Minimum Detectable Differences for Illustrative CHIP Outcomes
Estimated CI Half Widths for Illustrative Proportions
(shown in percentage points)

Sample Size [Composition]

Mean=50%

Mean=25% (or 75%)

Mean=10% (or 90%)

(E.g., Had Recent
Preventive Visit)

(E.g., Has Elevated
Health Care Need)

(E.g., Has Unmet Dental
Need)

Comparisons of Established and New Enrollee Sample Domains
Ten States Pooled
5,800 : 4,600 [full domain vs. full domain]
2,900 : 2,300 [50% subgroup comparison]

4.8
6.3

4.1
5.5

2.9
3.8

1,450 : 1,150 [25% subgroup comparison]

8.8

7.6

5.3

Notes: The MDD is equal to the smallest difference between two samples that can be detected for a specified level of power and statistical
significance. (We calculated the MDD above under standard assumptions of 80% power and 95% statistical significance, two-tailed test).
Standard errors for calculating the MDD have been adjusted to reflect design effect that we expect for the different sample compositions
shown, based on the results from the prior CHIP survey (see text for details).

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
12

APPENDIX
MEDICAID SURVEY: SAMPLE SIZES AND PRECISION

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
13

Table A1. Target Sample Sizes (Completed Interviews) for the Medicaid Survey, by State and Domain
Established Enrollees

New Enrollees

Recent Disenrollees

California

800

550

550

Florida

400

400

400

Texas

550

425

425

Total

1750

1375

1375

Table A2. Confidence Interval (CI) Half Widths for Illustrative Medicaid Outcomes
Estimated CI Half Widths for Illustrative Proportions
(shown in percentage points)
Mean=50%

Mean=25% (or 75%)

Mean=10% (or 90%)

(E.g., Had Recent
Preventive Visit)

(E.g., Has Elevated
Health Care Need)

(E.g., Has Unmet
Dental Need)

1,740 [full sample domain]
870 [50% domain subgroup]

3.0
4.0

2.6
3.5

1.8
2.4

435 [25% domain subgroup]
Individual State

5.6

4.9

3.4

800 [domain in largest state: CA]
400 [domain in smaller state: FL]

4.4
5.9

3.8
5.1

2.6
3.6

Sample Size [Composition]
Established Enrollees
Three States Pooled (Medicaid Sample)

New Enrollees and Recent Disenrollees
Three States Pooled (Medicaid Sample)

`

1,375 [full sample domain]
688 [50% domain subgroup]

3.3
4.5

2.9
3.9

2.0
2.7

344 [25% domain subgroup]
Individual State

6.3

5.5

3.8

550 [domain in largest state: CA]
400 [domain in smaller state: FL]

5.2
5.9

4.5
5.1

3.1
3.6

Notes: The confidence interval half width is equal to the standard error of an outcome multiplied by the standard normal deviate
used in a 95% confidence interval, 1.96. Standard errors have been adjusted to reflect the expected design effect under a
compromise allocation of sample members to states.

MEMO TO: CHIRPA-10 Evaluation Team
FROM:
Kimberly Smith and Christopher Trenholm
DATE:
10/25/2011
PAGE:
14

Table A3. Minimum Detectable Differences for Illustrative Medicaid Outcomes
Estimated CI Half Widths for Illustrative Proportions
(shown in percentage points)

Sample Size [Composition]

Mean=50%

Mean=25% (or 75%)

Mean=10% (or 90%)

(E.g., Had Recent
Preventive Visit)

(E.g., Has Elevated
Health Care Need)

(E.g., Has Unmet
Dental Need)

Comparisons of Established and New Enrollee Sample Domains
Three States Pooled
1,750 : 1,375 [full domain vs. full domain]
875 : 688 [50% subgroup comparison]

6.6
8.8

5.7
7.6

3.9
5.3

438 : 344 [25% subgroup comparison]

12.3

10.6

7.4

Notes: The MDD is equal to the smallest difference between two samples that can be detected for a specified level of power and
statistical significance. (We calculated the MDD above under standard assumptions of 80% power and 95% statistical significance,
two-tailed test). Standard errors for calculating the MDD have been adjusted to reflect design effect that we expect for the different
sample compositions shown, based on the results from the prior CHIP survey (see text for details).