Attachment E--Power Analysis and Simulation Study

ATTACHMENT E.

POWER ANALYSIS AND SIMULATION STUDY

1. Power Analysis

A base sample size is computed based on the following conditions.

• Binary measure of accessibility or variance in population (10%, 20% and 25%), which is an expected proportion of accessible AJC;

• +/- 3 and +/- 5 percent margin of error or precision;

• Confidence level of 95%;

• Expected response rate (50%, 60%, 70%, 80% and 90%); and

• Population size of 2,700.

A required sample size, n, is computed by an equation:

where N is a population size, V is a binary measure of accessibility, P is a precision (margin of error), and Z is a confidence level, and R is an estimated response rate. Table 1 shows the required sample size based on different conditions.

Table 1.

Sample Size		Binary Measure of Accessibility
Precision	Response rate	90%	80%	75%	70%	60%	50%
+/- 5	0.9	146	250	289	320	361	374
	0.8	164	282	325	360	406	420
	0.7	188	322	372	412	464	480
	0.6	219	376	434	480	541	561
	0.5	263	451	521	576	649	673
+/- 3	0.9	374	606	686	748	825	850
	0.8	420	681	772	841	928	956
	0.7	480	779	882	961	1061	1093
	0.6	561	908	1029	1122	1238	1275
	0.5	673	1090	1235	1346	1485	1530

2. Simulation Study

The IMPAQ team has conducted simulation studies to evaluate the impact of the number of site visits based on the proposed analytical framework, Facet model, in terms of the bias and precision of the latent construct measure and the respondent effects (as facet) of AJC survey.

Survey responses are generated over 2,000 respondents, which is approximately 75% of total AJCs (including both comprehensive and satellite centers) in addition to varied number (i.e. 10-100) of site visits. The parameters of interests, accessibility level was generated from the standard normal distribution and the SDR effect (or respondent effect) had -0.5 for 2,000 respondent simulating the expected SDR bias by center directors and +0.5 by site visitors.

For both simulation studies, data was generated based on a Facet Model with a total of 32 questions (16 binary questions and 16 rating scale items) based on the specifications of communication domain of current AJC surveys. A simulation condition of three step parameters, , for survey item with rating scale is:

• 

• 

• 

and for a dichotomous survey item is:

• 

The latent construct score, , (e.g AJC accessibility level) used a following condition:

• 

A bias of the latent construct estimates is defined as:

where is the estimated latent construct score.

Table 2 shows the mean, standard deviation, 2.5^th percentile, and 97.5^th percentile bias values for the latent construct and respondent effect estimates.

Table 2: Bias and Standard error of the latent construct estimates

	Accessibility Estimates Bias				Respondent Effect Estimates Bias
# of site visits	Mean	SD	P2.5	P97.5	Mean	SD	P2.5	P97.5
10	0.055	0.022	0.017	0.097	0.000	0.104	-0.148	0.148
20	0.064	0.024	0.023	0.112	0.000	0.114	-0.157	0.157
30	0.047	0.027	0.000	0.100	0.000	0.116	-0.165	0.165
40	0.042	0.023	-0.005	0.093	0.000	0.102	-0.158	0.158
50	0.040	0.024	-0.004	0.084	0.000	0.086	-0.121	0.121
60	0.038	0.025	-0.001	0.085	0.000	0.086	-0.121	0.121
70	0.039	0.024	-0.001	0.087	0.000	0.080	-0.114	0.114
80	0.043	0.022	-0.001	0.088	0.000	0.077	-0.108	0.108
90	0.045	0.022	0.004	0.089	0.000	0.074	-0.102	0.102
100	0.043	0.023	-0.004	0.085	0.000	0.074	-0.097	0.097

OMB Part B Attachment D – Page 4 February 4, 2014