ASPE sample size calculations

CPS Test Questions on Health Insurance: Power Calculations

March 27, 2012

Ben Sommers (ASPE) and Colleen Carey (CEA)

Assumptions:

• The number of individuals per household is fixed at 2.7 (the value among respondents in 2011).

• Desired power is 0.8 and alpha is set at 0.1.

• The total CPS sample is 204,000, but we consider both the possibility of comparison to the remaining sample in March 2013 and the full sample in March 2014 (i.e., the pretest may be useful both to show how response changes and to set a baseline).

CENSUS CALCULATION:

For binary outcome with a mean of 3.7%, and a difference of 0.45%, Test Sample needs to be approximately 13,600 (n ~ 5,000 households).

ALTERNATIVES:

However, the power falls quickly with the prevalence of the binary outcome:

For binary outcome with a mean of 15% (e.g. uninsured), and a difference of 0.5%, Census Test Sample of 13600 has only approximately 47% power. Test Sample needs to be almost 40,000 (n ~15,000 households) to have 80% power. If we are willing to settle for a full percentage point difference, the proposed sample is adequate.

For binary outcome with a mean of 65% (e.g. ESI), and a difference of 0.5%, Census Test Sample of 13600 has only approximately 32% power. Test Sample X needs to be almost 80,000 (n ~29,000 households, a large proportion of the full sample) to have 80% power. If we’re willing to settle for a full percentage point difference, Census Sample of 13,600 has 77% power, and a few hundred more households (about 600) would give us adequate power.

Source: Lenth, R. V. (2006-9).  Java Applets for Power and Sample Size [Computer software].  Retrieved 23 March 2011, from http://www.stat.uiowa.edu/~rlenth/Power .