MIPCDSupportStatementPartB

Revised December 2013

Supporting Statement B: Collections of
Information Employing Statistical Methods
Medicaid Incentives for Prevention of Chronic
Diseases Evaluation
Request for OMB Approval

Prepared for
Jean Scott, DrPH
Center for Medicare and Medicaid Innovation
Centers for Medicare & Medicaid Services
7500 Security Boulevard, Mail Stop WB-06-05
Baltimore, MD 21244-1850
E-mail: [email protected]
Prepared by

Thomas Hoerger, PhD
Rebecca Perry, MSc
Linda Chamiec-Case, BA
Katherine Treiman, PhD
Julia Kish Doto, PhD
Stephanie Teixeira-Poit, MS

RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
RTI Project Number 0212790.004.000.003

Supporting Statement B: Collections of Information Employing Statistical Methods
Medicaid Incentives for Prevention of Chronic Diseases Evaluation

Submitted by:
RTI International

Contact Person:
Rebecca Perry, MSc
Linda Chamiec-Case, BA

TABLE OF CONTENTS
Section

Page

BENEFICIARY SATISFACTION SURVEY.......................................................................... 1
1. Respondent Universe and Respondent Selection Method .................................................. 1
2. Procedures for the collection of information ...................................................................... 2
2.1

Statistical Methodology for Sample Selection ......................................................... 2

2.2

Estimation Procedure ............................................................................................... 2

2.3

Degree of Accuracy .................................................................................................. 6

2.4

Unusual Problems..................................................................................................... 6

2.5

Periodic Data Collection Cycles............................................................................... 6

3. Response Rates and Non-Response .................................................................................... 6
4. Tests of Procedures or Methods .......................................................................................... 7
5. Individuals Consulted on Statistical Aspects of Design ..................................................... 7
FOCUS GROUP DISCUSSIONS: ROUND 1 ......................................................................... 9
1. Respondent Universe and Respondent Selection Method .................................................. 9
2. Procedures for the collection of information .................................................................... 13
2.1

Recruitment ............................................................................................................ 13

2.2

Recruitment Alternatives ........................................................................................ 13

2.3

Implementation....................................................................................................... 15

3. Response Rates and NonResponse ................................................................................... 16
3.1

Methods to Maximize Response Rates and to Deal With Issues of
Nonresponse ........................................................................................................... 16

4. Tests of Procedures or Methods ........................................................................................ 16
5. Individuals Consulted on Statistical Aspects of Design ................................................... 16
FOCUS GROUP DISCUSSIONS: ROUND 2 ....................................................................... 18
1. Respondent Universe and Respondent Selection Method ................................................ 18
2. Procedures for the collection of information .................................................................... 19
2.1

Recruitment ............................................................................................................ 19

iii

2.2

Recruitment Alternatives ........................................................................................ 19

2.3

Implementation....................................................................................................... 19

3. Response Rates and NonResponse ................................................................................... 20
3.1

Methods to Maximize Response Rates and to Deal With Issues of
Nonresponse ........................................................................................................... 20

4. Tests of Procedures or Methods ........................................................................................ 20
5. Individuals Consulted on Statistical Aspects of Design ................................................... 20
References ............................................................................................................................... 21

TABLES
Number

Page

Table 1. Samples Sizes and Expected Number of Completed Interviews by State ....................... 2
Table 2. Overview of Data Collection Steps .................................................................................. 6
Table 3. Number of Focus Groups and Percent of Participants as of January 2013..................... 12

iv

BENEFICIARY SATISFACTION SURVEY
1.

RESPONDENT UNIVERSE AND RESPONDENT SELECTION METHOD
The study design calls for a beneficiary satisfaction survey, two rounds of focus groups,

stakeholder interviews, and site visits. This portion of the supporting statement describes the
beneficiary satisfaction survey. The respondent universe will consist of all Medicaid
beneficiaries who are at least 18 years of age who participated in the 10 State MIPCD program
within at least the past 6 months of when the survey is fielded in Year 3 of the evaluation and
were assigned to the experimental group (in States with experimental and control groups).
We will construct a sampling frame based on data obtained from the 10 State programs.
The sample design will be stratified systematic sampling by State. A probabilistic sample of
participants on the sampling frame will be drawn in each State. With a target response rate of 58
percent, a sample size of 6,139 participants will be needed to yield State-level estimates with a 5
percent margin of error. Table 1 presents the sample sizes and expected number of completed
interviews by State to support State-level estimation for 80 percent statistical power and
detectable State-differences as small as 10 percent. Across all States combined, these sample
sizes will be sufficient to detect significant differences of 5 percent between subgroups (e.g.,
gender, targeted health condition, incentive type) at 0.05 level of probability. Once the
beneficiary data have been received, we will develop an optimal allocation of the full sample to
the individual States to minimize the overall design effect of unequally weighting the sample.
Hence, Table 1 is an example of the sample distribution across the States based on acceptable
levels of statistical power and the limited information currently available for design purposes.
Refinements will be made based on the most recent beneficiary data. For example, we
will maximize precision on estimates for Montana and Texas due to the relatively small number
of program beneficiary participants. The current design selects all eligible individuals from these
two States and assumes a 58 percent response rate. Statistical power for each State will be at
least 80 percent with the exception of Montana due to the small number of eligible Medicaid
beneficiaries expected at only 362.

1

Table 1. Samples Sizes and Expected Number of Completed Interviews by State

State

Total of No. of Program
Participants (in experimental
condition)

California

Sample Size

Expected Number
of Completes

6,000

652

375

32,686

687

395

Hawaii

4,521

639

368

Minnesota

2,160

587

338

362

362

210

Nevada

4,000

632

364

New Hampshire

3,875

631

363

13,842

676

389

625

625

363

5,500

648

373

73,571

6,139

3,561

—

—

58%

Connecticut

Montana

New York
Texas
Wisconsin
Total
Target Response Rate

Note: Assumes true prevalence rate of 50%, in ±5% range of the true rate.

2.

PROCEDURES FOR THE COLLECTION OF INFORMATION

2.1

Statistical Methodology for Sample Selection
In an effort to control the distribution of the beneficiary sample and to assist in

minimizing sampling variance, the sampling frame from each State will be implicitly stratified or
sorted by key covariates such as, race/ethnicity, gender, and targeted health condition. Following
implicit stratification, a systematic probability sample of 652 beneficiaries will be selected out of
the 6,000 within California. Similarly, 687 beneficiaries out of 32,686 will be selected from
Connecticut. Sample sizes for the other State programs are also shown in Table 1. Since SAS
PROC SURVEYSELECT generates many input variables for sample weighting and contains
several sampling algorithms developed at RTI, such as Chromy’s probability proportional to size
sequential sampling procedure, we will use PROC SURVEYSELECT to select the beneficiaries
from each State sampling frame.
2.2

Estimation Procedure
We will adjust the sample design weights indexed by predictors of response propensity,

such as race/ethnicity and gender. Following traditional weighting practices, the weight

2

adjustments will be applied to the design weight of each responding beneficiary within the cells.
The sum of the product of the adjustment times the design-based weights will equal the
population characteristics of any group they were selected from. This algorithm forces each
responding beneficiary to represent a portion of the nonrespondents within the cell. These weight
adjustments will be calculated using sample weighting procedures in SAS that were developed
by RTI as part of the SUDAAN software package.
The sample survey data analysis procedures in SUDAAN (Shah, Barnwell and Bieler,
1997) or SAS (SAS Institute 2012) will be used to generate weighted sample estimates of means
or proportions for the domains of interest, and to conduct statistical tests of differences among
these estimates.
We will develop a detailed analysis plan for review and approval by CMS. We will
conduct the following types of analyses using statistical tests and logistic regression models for
weighted sample survey data:


Across States, compare satisfaction (overall, accessibility, quality) by beneficiary
characteristics, including sociodemographic characteristics (e.g., race/ethnicity,
education) and health conditions.



Across States, compare satisfaction by program characteristics, including the
disease focus areas (e.g., diabetes, smoking cessation) and incentive (e.g., cash vs.
noncash, value). For incentives, we will also investigate whether a program’s
incentives can be classified as “front-loaded” (with more frequent and larger
incentives paid early in a beneficiary’s participation) or “back-loaded.” If so, we
will consider whether there is sufficient variation between programs to test
whether front-loaded incentives lead to better outcomes than back-loaded
incentives. We will also examine whether participation-based incentives (e.g.,
participation in a weight loss class, attendance at a smoking cessation program)
produce better outcomes than outcome-based incentives (e.g., losing weight,
stopping smoking).



Across States, examine satisfaction among special populations of interest (adults
with disabilities, adults with chronic illness, children with special health care

3

needs [response by proxies]) and how satisfaction compares among beneficiaries
generally.


Across States, examine differences in satisfaction between beneficiaries with
different levels of participation in the program (e.g., completed program,
discontinued program).



Examine the predictors of overall satisfaction and of satisfaction with accessibility
and with quality of care specifically.

Our statistical analysis will include the development and testing of models to determine
state and individual-level predictors of various outcomes and of changes over time in outcomes.
The models will be derived based on theoretical considerations and relevant findings from prior
research.
Below is the conventional linear regression model for infinite populations.

Y = Xβ + e ,
X = the matrix of independent variables in the beneficiary population (namely;
demographic characteristics, health conditions, and disease focus areas).
Y = the vector of dependent variables in the beneficiary population (namely;
accessibility, quality, differences in satisfaction, and overall satisfaction).
The vector β = the vector of regression coefficients to be estimated, and e is the vector of
random errors.
Since our sample is a finite population, we will estimate

β = [X ' X ] X 'Y .
For our beneficiary sample, we will use the Horvitz-Thompson estimators as follows:
−1

x'wx

n

=∑

x ' x w = Horvitz-Thompson estimator for

X'X

(2.1)

x'wy = ∑ x ' y w = Horvitz-Thompson estimator for

X 'Y

(2.2)

i =1

i

i

i

and
n

i =1

i

i

i

where
xi = row vector of independent variable values for sample beneficiary (i),

w = the nonresponse adjusted analysis weight for sample beneficiary (i), and
y = the dependent variable values for sample beneficiary (i).
i

i

4

Our regression coefficients will be estimated by b as below:

b = ( x'wx) ( x'wy)
−1

(2.3)

We will test our hypothesis that the regression coefficient associated with each variable
in the model is equal to zero, namely:

H : b = 0 vs
H :b ≠ 0
o

i

A

i

The test for overall model significance will also be computed as:
H o : b = 0 vs

H :b
A

≠0

Linear regression will be used to model continuous outcome variables. This procedure is
found in SUDAAN (software developed by RTI for statistical analyses of cluster-correlated
data).
Logistic regression, also found in SUDAAN, will be used to model binary outcome
variables. The model is provided below:

log ( p / 1− p) = X 'β
where,

p = prob(Y =1 / X )
To predict the probability of binary outcomes, we will define Y as follows:
Y = a vector set to 1 for responses of yes, 0 if no.
Similarly, descriptive statistics will include weighted estimates for means, proportions,
and percentages as well.
We will use SUDAAN to estimate the β’s by solving the weighted score equations
presented in Section 19.9 of SUDAAN 11 (Research Triangle Institute, 2012). Variances will be
estimated either via implicit Taylor linearization (a form of Generalized Estimating Equations,
GEE) or via replication methods (BRR and Jackknife). Likewise, descriptive statistics will
include weighted estimates for means, proportions, and percentages as well.

5

2.3

Degree of Accuracy
Our sample size was determined to enable us to generate weighted sample estimates of

proportions of interest in each State in the ±5% range of the true proportion with a 95%
probability.
2.4

Unusual Problems
None anticipated.

2.5

Periodic Data Collection Cycles
The cross-sectional survey will be conducted one time only in Year 3 of the study.

3.

RESPONSE RATES AND NON-RESPONSE
We will employ a multiple-contact multiple-mode approach to maximize survey

response. The protocol uses proven methods for increasing response rates, which is based on
methodological research conducted by Jenkins and Dillman (1997) and Dillman (2000). This
approach consists of steps as presented in Table 2.
Table 2. Overview of Data Collection Steps
Step

Schedule

Mail pre-notification letter to all sample members

—

Mail the first questionnaire to all sample members

Approximately 1 week after pre-notification letter
mailed

Mail second questionnaire to sample members who
do not respond to the survey

Approximately 3 weeks after first questionnaire
mailed

Follow up by telephone with all sample members for
whom a telephone number can be obtained who do
not respond to the mail survey

Approximately 3 weeks after second questionnaire
mailed

A toll-free number and contact name will be listed on the survey instrument and in all
correspondence for individuals to call if they have questions or if they prefer to complete the
survey by phone. Interviewers will return all calls to the toll-free number within 1 business day.
We will conduct a nonresponse analysis to assess the potential impact of nonresponse on
bias in estimation. The sampling weights will be adjusted for nonresponses and used in

6

estimation to correct for bias due to nonresponses. The weighting class adjustment method will
be used to compensate for differential response rates in terms of respondent characteristics. The
Generalized Exponential Model method for sampling weight adjustment may be used if the
nature of nonresponses turns out to be more complicated.
4.

TESTS OF PROCEDURES OR METHODS
We conducted two rounds of testing of the draft survey instrument using a cognitive

interviewing approach. The purpose of the testing was to assess respondents’ understanding of
questions, consistency in interpreting questions and response options, ability to recall necessary
information, how well the items reflect the measurement domains, and the flow of the
instrument. We conducted the testing in two rounds, with nine participants in Round 1 and six
participants in Round 2. We revised the questionnaire following Round 1 based on the findings
of the cognitive interviews. We recruited participants through two of the State programs.
We also conducted time tests with nine respondents to determine the time required to
complete the questionnaire. On average, they required 8 minutes to complete the survey. For
burden estimates, we have made the more conservative assumption that the survey will require
15 minutes to complete when self-administered and 20 minutes by telephone. This assumption
accommodates respondents with multiple health conditions who will answer more questions.
5.

INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS OF DESIGN
The tables below provide the names, affiliation, and contact information for those

consulted on the statistical aspects of the design and who will actually collect or analyze the
information.
Individuals Consulted on Statistical Aspects/Data Collection & Analysis
Name

Affiliation

Contact Information

Katherine Treiman, PhD

RTI International

919-541-6648

Lei Li, PhD

RTI International

919-541-8821

Shelton Jones, MS

RTI International

919-541-5879

Anne Kenyon, MBA

RTI International

919-541-6574

Stephanie Teixeira-Poit, MS

RTI International

919-541-5915

Thomas Hoerger, PhD

RTI International

919-541-7146

7

CMS Staff who Advised on Design
Name

CMS

Contact Information

Jean Scott, DrPH

Rapid Cycle Evaluation Group,
Center for Medicare and Medicaid Innovation

410-786-6327

Jean Gaines, PhD, RN

Rapid Cycle Evaluation Group,
Center for Medicare and Medicaid Innovation

410-786-4933

8

FOCUS GROUP DISCUSSIONS: ROUND 1
1.

RESPONDENT UNIVERSE AND RESPONDENT SELECTION METHOD
The study design calls for a beneficiary satisfaction survey, two rounds of focus groups,

stakeholder interviews, and site visits. This portion of the supporting statement describes the first
round of focus groups. The focus groups by their qualitative design are descriptive and
exploratory in nature. Statistical methods will not be used to select respondents. The Round 1
beneficiary satisfaction study design calls for a total of 30 focus groups with Medicaid
beneficiaries. Up to 4 of the 30 groups may be conducted in Spanish with the remaining groups
conducted in English. Each focus group will last for 90 minutes. The focus group segments will
be based on three health prevention program types—tobacco-use cessation, diabetes
management, and weight loss (which includes diabetes prevention)—and primary incentive type.
The distribution of focus groups across these three health prevention program types would
include the following:


9 focus group discussions among tobacco-use cessation participants in three States—
California, Connecticut, and Wisconsin;



8 focus group discussions among diabetes management participants in three States—
Hawaii, Nevada, and New York;



10 focus group discussions among weight loss program participants in four States—
Minnesota, Montana, New Hampshire, and New York; and



3 focus group discussions among program participants in Texas where one prevention
program addresses multiple conditions).
The table below shows the number of focus groups we will conduct in each State by

health program focus (i.e., chronic disease prevention focus). The percentage of enrolled
participants who are included and excluded based on this segmentation strategy is also included.
The percentage is calculated based on the number of estimated beneficiaries targeted to
participate in the intervention arms of programs as of Fall 2012.
Our aim is to include six to nine beneficiaries for each focus group for a total of 270
beneficiaries. Given concerns about last-minute cancellations, a total of 330 beneficiaries will be
recruited for each focus group (i.e., 9 beneficiaries and 2 alternates). Based on past experience,
we anticipate that 50% of those contacted will meet the eligibility criteria and agree to
participate. Therefore, we anticipate that 22 beneficiaries will be contacted and screened for each

9

focus group to recruit 11 for participation and ensure 9 actual focus group participants. This
suggests that a total of 660 beneficiaries (i.e., 30 focus groups x 22 beneficiaries) will be
contacted and screened using the Screener in either English or Spanish depending on the
participant’s primary language (Supporting Statement A- Attachments 4.a. & 8.a.) for the focus
groups.
We will work with the States for focus group recruitment and general logistics (e.g.,
identifying a location to hold the focus groups). To facilitate working with each State program,
we will identify a liaison(s) to assist with planning and implementation of the focus groups.
Effective recruitment strategies may vary from program to program; we will work with the
liaisons to identify the most appropriate recruitment methods (see Section 2 Procedures for the
collection of information for more detail on recruitment).
We will conduct a telephone screener in English or Spanish (Supporting Statement AAttachment 4.a. & 8.a.) with potential participants to assess their eligibility and interest for
participation in the study. Participants will call a toll-free number with a recorded greeting that
will direct callers to either an English or Spanish-speaking focus group recruitment staff member
based on their selections (e.g. “Thank you for calling the MIPCD focus group screening line. For
English, please press 1. Para Español, por favor presione 2.”). Once participants select a
language, they will be redirected to the focus group recruitment staff member who will conduct
their screening in their preferred language, as indicated initially on the call. If focus group
recruitment staff are unavailable to answer a participant’s call, the participant will be connected
to a recorded message in both English and Spanish that will instruct the participant to leave
his/her contact information and a focus group recruiter will call the participant back as soon as
possible.

Eligible respondents will be Medicaid beneficiaries who have actively participated in the
program of interest within the past 6 months. Participants will be screened and selected based on
the following eligibility criteria:


Inclusion Criteria
– Age 18 or older
– Having English or Spanish as a primary language

10

– Self-describing participation in the program as very frequent or somewhat frequent
– Participated in the program within the past 6 months
– Receiving or received rewards or incentives for participating in the program


Exclusion Criteria
– Age 17 or younger
– Not having English or Spanish as a primary language
– Self-describing participation in the program as not frequent or not at all
– Not participated in the program within the past 6 months
– Not receiving or received rewards or incentives for participating in the program

11

Table 3. Number of Focus Groups and Percent of Participants as of January 2013

State (Not listed
alphabetically)

Estimated percent of
beneficiaries included
in the sample (%) 1

Number of Focus Groups (focus group) by type health
prevention program focus

California

≈50 2

3 focus groups with tobacco-use cessation participants

Connecticut

100

3 focus groups with tobacco-use cessation participants

Wisconsin

≈73

3 focus groups with tobacco-use cessation participants

Hawaii

100

3 focus groups with diabetes management participants

Nevada

≈54 4

3 focus groups with diabetes management participants

New York

≈464

2 focus groups with diabetes management participants and
2 focus groups with weight loss participants

Minnesota

100

2 focus groups with weight loss participants

Montana

100

3 focus groups with weight loss participants

New Hampshire

≈724

3 focus groups with weight loss participants

Texas

100

3 focus groups with program participants 5

Total

≈74 6

30 focus groups across three health prevention
program foci

3

1

Based on the number of estimated beneficiaries participating in the intervention arms of programs as of January
2013.
2
In the intervention arm, 100 percent of California’s participants receive nicotine replacement therapy which the
State considers to be a noncash incentive. Fifty percent of these participants who receive NRT will also receive a
gift card. We plan to include the 50 percent of participants who are receiving NRT and gift cards.
3
Twenty- seven percent of program participants are pregnant women and will be excluded from our segmentation
strategy. The remaining 73 percent of program participants who are not pregnant will be included in our
segmentation strategy.
4
This percent sampled is less than 100 because we will only include participants from one or two intervention arms
and not the entire State program. To determine the percent that will be sampled in this State, we used previous
estimated enrollment numbers from the proposals and operational protocols for MIPCD States, because the most
recent estimated enrollment numbers from the State data synthesis spreadsheet as of January 2013, were not
broken down by program arm.
5
Texas’ program does not have a specific health condition focus; instead, participants identify health goals and
work with a navigator to accomplish chronic conditions health prevention-related behavior changes.
6
For States with 100 percent included in one condition, the numbers to calculate the total percentage included and
excluded were based on the most recent estimated enrollment numbers from the State data synthesis spreadsheet
as of January 2013. For the three remaining States, we had to reference previous estimated enrollment numbers
from the proposals and operational protocols for MIPCD States, because the most recent estimated enrollment
numbers from the State data synthesis spreadsheet as of January 2013, were not broken down by program arm.

12

2.

PROCEDURES FOR THE COLLECTION OF INFORMATION

2.1

Recruitment
We will recruit a mix of beneficiaries in terms of their age, sex, race, ethnicity, and

education. As mentioned above, effective recruitment strategies may vary from program to
program; our core approach is to follow up with each grantee implementation site to determine
its willingness to assist with recruitment and identify a point of contact who we will partner with
to publicize the study. As needed, we will provide each site with recruitment flyers in English
and Spanish and IRB/OMB approval notices and assist with any necessary IRB approvals at the
sites. We will work with the points of contact to identify recruitment methods, which may
include posting materials (see Supporting Statement A- Attachments 4.c. in English & 8.c. in
Spanish) at the program sites, distributing flyers to beneficiaries during program activities, and
announcing the focus groups through other communication channels (e.g., program newsletters).
The recruiting materials will direct interested individuals to call a toll-free hotline that will direct
callers to either an English or Spanish-speaking focus group recruitment staff member based on
their preferred language. The focus group recruitment staff members will then screen callers,
determine their eligibility, and—as appropriate—schedule their participation in a group.

2.2

Recruitment Alternatives
To supplement efforts with the State grantees, we may employ a series of alternate

recruitment strategies as needed.
We may work with the States to obtain lists of program participants that include
beneficiary characteristics and information about their program participation and outcomes (if
available). Because of HIPAA requirements, program staff will make initial contact with
potential participants assessing their interest; if individuals are interested in the focus group,
program staff will obtain their consent to provide their names and contact information (see
Supporting Statement A- Attachments 4.d. in English & 8.d. in Spanish). Then staff can screen
the participants, determine their eligibility, and—as appropriate—schedule their participation in
a group.

13

Selection bias is a potential concern with focus group recruitment and could occur if
program staff are more likely to recruit certain types of beneficiaries (e.g., individuals who are
more active in the program, more successful in achieving outcomes) or if certain types of
beneficiaries (e.g., more actively involved, more satisfied) are more likely to respond to
recruitment efforts. To minimize bias, we will explore the feasibility of States providing deidentified lists of program participants that include beneficiary characteristics and information
about their program participation and outcomes (if available). Participants would be categorized
into groups of interest—such as beneficiaries who successfully completed the program and those
who discontinued—randomly select identifiers from each category, and provide this pool of
names to the program staff for recruitment. This approach would be more labor-intensive, but it
could minimize selection bias.
To further publicize the focus group discussions and assist with participant recruitment,
we may work with staff at local government and community-based organizations and other
places that provide health care (e.g., YMCA, community health clinics, and State health
departments) to share and display focus group recruitment materials with State program
participants. Interested individuals who contact the hotline to join the focus groups will be
screened to ensure that they are program participants or have participated in the program within
the past 6 months prior to the focus group date.
Second, if necessary, we may work with CMS to identify and provide contacts for local
and national organizations that provide health care or other services to the audience. Our
experience has shown that it is helpful to have a government organization make initial contact
with the sites to explain the study, the types of individuals eligible to participate, and how the
sites may help in our recruitment efforts (e.g., posting or distributing flyers, contacting interested
individuals, posting information to an e-mail listserv or Web site). We will provide the necessary
materials to facilitate this process (e.g., flyers, outreach e-mails and letters).
As a third alternative strategy, we may use targeted discussion groups, blogs, or
communities that reach out to State program participants. In some cases, States are using social
media outlets to advertise or even communicate with program participants. We will draw on
these outlets to provide additional advertisements about the focus group discussions and to
recruit participants.

14

2.3

Implementation
As noted above, 11 beneficiaries will be recruited for each focus group to ensure a target

number of nine respondents in each group. Participants will be reminded of the date and time of
their focus group via telephone 1 to 2 days beforehand. Consequently, if more than nine of those
recruited arrive at the focus group, the first nine will be asked to participate and the others will
be paid the $75 incentive and excused from participation.
The focus groups will be conducted at the program facilities by a professionally trained
moderator. Each focus group will be scheduled for 90 minutes. Staff will attend the focus groups
to take notes on a laptop computer and coordinate logistics of checking in participants and
obtaining informed consent. All focus groups will be audio recorded. A professional
transcriptionist will prepare verbatim transcriptions of all focus groups. Transcripts and notes
will not include participants’ names.
Signed informed consent forms will be used for the focus groups (see Supporting
Statement A- Attachments 4.e. in English & 8.e. in Spanish ). Participants will be asked to
provide their written informed consent upon arrival at the focus group prior to attending the
group. Staff will explain the process and distribute and collect the consent forms. Before each
group, the moderator will review the consent form with the participants to ensure that they
understand their rights and are participating voluntarily.
The consent forms will be printed in duplicate, with one copy retained by focus group
staff and the other provided to the project participants. The consent form retained by focus group
staff will be stored in a locked filing cabinet. Only select project staff will have access to the
project files. Focus group participants will receive $75 at the conclusion of the focus group as
incentive for participation.
Following each focus groups, team members will prepare brief “topline” summaries to
capture themes and highlights from the group. Together with the notes and audio-recordings,
these topline summaries will be used for the analysis.

15

3.

RESPONSE RATES AND NONRESPONSE

3.1

Methods to Maximize Response Rates and to Deal With Issues of Nonresponse

The recruitment plan includes a $75 incentive for beneficiaries for their participation and
transportation expenses to promote efficient enrollment in this voluntary study.
All focus groups will be conducted at the program facilities to eliminate beneficiaries’
need for travel to a separate focus group facility location that may be unfamiliar or inconvenient
for them, potentially increasing their response rate. We will provide each program $500 to
compensate them for their use of their facility (i.e., room rental) and to thank the staff for the
assistance provided. Sample locations for the facilities may include YMCAs, American Diabetes
Association offices, Public Health Departments, Schools of Medicine, and health care facilities.
4.

TESTS OF PROCEDURES OR METHODS
The proposed project is formative in nature and involves the collection of qualitative

information. The data collection instruments have not previously been fielded.
5.

INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS OF DESIGN
There are no statistical aspects of this project, but the individuals collecting or analyzing

data are listed below.
Individuals Consulted on Data Collection & Analysis
Name

Affiliation

Contact Information

Julia Kish Doto, PhD

RTI International

301-468-8280

Katherine Treiman, PhD

RTI International

919-541-6648

Stephanie Teixeira-Poit, MS

RTI International

919-541-5915

Rebecca Perry, MSc

RTI International

202-974-7818

Thomas Hoerger, PhD

RTI International

919-541-7146

16

CMS Staff who Advised on Design
Name

CMS

Contact Information

Jean Scott, DrPH

Rapid Cycle Evaluation Group,
Center for Medicare and Medicaid Innovation

410-786-6327

Jean Gaines, PhD, RN

Rapid Cycle Evaluation Group,
Center for Medicare and Medicaid Innovation

410-786-4933

17

FOCUS GROUP DISCUSSIONS: ROUND 2
1.

RESPONDENT UNIVERSE AND RESPONDENT SELECTION METHOD
The study design calls for a beneficiary satisfaction survey, two rounds of focus groups,

stakeholder interviews, and site visits. This portion of the supporting statement describes the
second round of focus groups. The focus groups by their qualitative design are descriptive and
exploratory in nature. Statistical methods will not be used to select respondents. The Round 2
beneficiary satisfaction study design calls for a total of 10 focus groups with Medicaid
beneficiaries over an approximately nine month timeframe (two groups per State in five States).
Each focus group will last for 90 minutes. The focus group segments will be determined based
on the Round 1 findings but may encompass up to three health prevention program types—
tobacco-use cessation, diabetes management, and weight loss (which includes diabetes
prevention)—and primary incentive type.
Our aim is to include six to nine beneficiaries for each focus group for a total of 90
beneficiaries. Given concerns about last-minute cancellations, a total of 110 beneficiaries will be
recruited for each focus group (i.e., 9 beneficiaries and 2 alternates). Based on past experience,
we anticipate that 50% of those contacted will meet the eligibility criteria and agree to
participate. Therefore, we anticipate that 22 beneficiaries will be contacted and screened for each
focus group to recruit 11 for participation and ensure 9 actual focus group participants. This
suggests that a total of 220 beneficiaries (i.e., 10 focus groups x 22 beneficiaries) will be
contacted and screened using the Screener (Supporting Statement A- Attachment 4.a.) for the
focus groups.
We will work with the States for focus group recruitment and general logistics (e.g.,
identifying a location to hold the focus groups). To facilitate working with each State program,
we will identify a liaison(s) to assist with planning and implementation of the focus groups.
Effective recruitment strategies may vary from program to program; and although we will have
lessons learned from recruiting from Round 1, we will still work with the liaisons to identify the
most appropriate recruitment methods (see Section 2 Procedures for the collection of information
for more detail on recruitment).
We will conduct a telephone screener (Supporting Statement A- Attachment 4.a.) with
potential participants to assess their eligibility and interest for participation in the study. Eligible

18

respondents will be Medicaid beneficiaries who have actively participated in the program of
interest within the past 6 months. Participants will be screened and selected based on the
following eligibility criteria:


Inclusion Criteria
– Age 18 or older
– Having English as a primary language
– Self-describing participation in the program as very frequent or somewhat frequent
– Participated in the program within the past 6 months
– Receiving or received rewards or incentives for participating in the program



Exclusion Criteria
– Age 17 or younger
– Not having English as a primary language
– Self-describing participation in the program as not frequent or not at all
– Not participated in the program within the past 6 months
– Not receiving or received rewards or incentives for participating in the program

2.

PROCEDURES FOR THE COLLECTION OF INFORMATION

2.1

Recruitment
As mentioned earlier, effective recruitment strategies may vary from program to

program; and although we will have lessons learned from recruiting from Round 1, we may still
use a variety of recruitment methods to enhance recruitment as needed (see Section 2 Procedures
for the collection of information for more detail on recruitment).
For further details, please see Focus Group Discussion: Round 1 Text.
2.2

Recruitment Alternatives
Please see Focus Group Discussion: Round 1 Text.

2.3

Implementation
Please see Focus Group Discussion: Round 1 Text.

19

3.

RESPONSE RATES AND NONRESPONSE

3.1

Methods to Maximize Response Rates and to Deal With Issues of Nonresponse

Please see Focus Group Discussion: Round 1 Text.
4.

TESTS OF PROCEDURES OR METHODS
Please see Focus Group Discussion: Round 1 Text.

5.

INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS OF DESIGN
There are no statistical aspects of this project, but the individuals collecting or analyzing

data are listed in the Focus Group Discussion: Round 1 Text.

20

REFERENCES
Dillman, D. A. (2000). Mail and telephone surveys: The tailored design method. New York.
Jenkins, C. R. & Dillman, D.A. (1997). Towards a theory of self-administered questionnaire
design. In L. Lyberg, P. Biemer, M. Collins, L. Decker, E. deLeeuw, C. Dippo, N.
Schwarz & D. Trewin (Eds.), Survey measurement and process quality (pp. 165-196).
New York: Wiley-Interscience.
Research Triangle Institute (2012). SUDAAN Language Manual, Volumes 1 and 2, Release 11.
Research Triangle Park, NC: Research Triangle Institute.
Shah, B. V., Barnwell, B. G., & Bieler, G. S. (1997). SUDAAN user’s manual. Research Triangle
Park, NC: Research Triangle Institute.
SAS Institute. (2012). SAS/STAT 9.3 user’s guide. SAS Institute: Cary, NC.

21