(State Gov't) External Quality Review of Medicaid MCOs and Supporting Regulations in 42 CFR 438.360, 438.362, and 438.364

2012 EXTERNAL QUALITY REVIEW (EQR) PROTOCOLS
APPENDIX II: SAMPLING APPROACHES

TABLE OF CONTENTS
PURPOSE OF THE APPENDIX ................................................................................................ 1
PROBABILITY SAMPLING ........................................................................................................ 1
NON-PROBABILITY SAMPLING ............................................................................................... 2

PURPOSE OF THE APPENDIX
This Appendix provides an overview of potential sampling methods that may be used in
Protocols 3, 5, 7, and 8. A statistician or staff with expertise in the design and implementation of
sampling should advise the State and/ or EQRO of the most appropriate sampling strategy.

PROBABILITY SAMPLING
Probability (or random) sampling methods leave selection of population units totally to chance
and not to preference on the part of the individuals conducting or otherwise participating in the
study. Biases are removed in these methods. There are several types of probability (or
random) sampling:

Simple Random Sampling
Simple random sampling is used when members of the study population have an equal chance
of being selected for the sample. Population members are numbered and random numbers
generated by a computer select units from the population. This sampling approach ensures that
all members of the target population have an equal chance of selection and assure the sample
is fully representative of the population.

Systematic Random Sampling
Systematic random sampling is used when the nth unit in a list is selected. This can be used
when a sampling frame is organized in a way that does not bias the sample.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it
displays a valid OMB control number. The valid OMB control number for this information collection is 0938-0786. The time required
to complete this information collection is estimated to average 1,591 hours per response for all activities, including the time to review
instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you
have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500
Security Boulevard, Attn: PRA Reports Clearance Officer, Baltimore, Maryland 21244-1850

EQR Protocol: Appendix II
Sampling Approaches
September 2012

1

Steps to organize and select a systematic sample are:
1. Construct a comprehensive sampling frame (e.g., list of all beneficiaries);
2. Divide the size of the sampling frame by the required sample size to produce a
sampling interval or skip interval (e.g., if there are 250 beneficiaries and a sample of
25 is needed, then divide 250/25 = 10);
3. From a random number table select a random number between 1 and 10;
4. Count down the list to get the Nth name (i.e., the # identified in step 3);
5. Skip down 10 names on the list and select a second name. Repeat the process as
many times as needed until the required sample size has been reached.

Stratified Random Sampling
Stratified random sampling is used when the target population consists of independent subgroups or strata. This technique divides the population into specific, strata or subgroups that
are homogeneous (same) within a strata and heterogeneous (different) between strata with
respect to certain characteristics such as ethnicity (e.g., Hispanic, non-Hispanic), age (e.g.,
under 30, over 30, or diagnosis (e.g., diabetic, non-diabetic). Stratification is done both to
improve the accuracy of estimating the total population’s characteristics and to provide
information about the characteristics of interest within subgroups. Stratified random sampling
requires more information about the population and requires a larger overall sample size than
simple random sampling. Once strata are identified and selected, sampling must be conducted
within each strata using probability (or random) sampling. As a result, it is typically more
expensive than simple random sampling. Stratified sampling may also involve “weighting” the
sample. In this process, a survey selects a disproportionately larger number of units of analysis
from one or more of the strata to allow the survey to produce information on that particular
stratum (e.g., individuals dually receiving both Medicare and Medicaid).

Cluster Sampling
Cluster sampling is used when a comprehensive sampling frame is NOT available. Units in the
population are gathered or classified into groups, similar to stratified sampling. Unlike the
stratified sampling method, the groups must be heterogeneous with respect to the measured
characteristic. This method requires prior knowledge about the population. Once clusters are
identified, a random sample of clusters is selected.

NON-PROBABILITY SAMPLING
Non-probability sampling methods are used when subjects are scarce and the study relies on
volunteers, or for comparisons of a subset of the population with a large population or
comparisons of non-stratified groups. They are based on the choice of those administering the
survey rather than chance; therefore, some bias can be expected. Non-random sampling
methods do not lend themselves to statistical analysis. Considering the risk of biased results
and the obstacles to statistical analysis, non-probability sampling is discouraged. However, at
times it can be an appropriate and efficient way of collecting needed information. The following
are types of non-probability sampling:

EQR Protocol: Appendix II
Sampling Approaches
September 2012

2

a. Judgment sampling- units are selected based on whether they are judged to be
representative of the population. By doing so, the sample is constructed to be a subpopulation.
b. Convenience sampling- uses readily available or convenient units. For example, if the
objective was beneficiary opinions regarding a group practice, patients in the office on
any given day or during a specific month could be interviewed.
c. Quota sampling- ensures that units in the sample appear in the same proportion as in
the population. For instance, if a certain target population is 55 percent female and 45
percent male, the quota sample requires a similar female/male distribution.

EQR Protocol: Appendix II
Sampling Approaches
September 2012

3