CMS-10477 State evaluator interview protocol and consent

Attachment 5.f. Site Visit Evaluator Interview Protocol and Consent Form

MEDICAID INCENTIVES FOR PREVENTION OF CHRONIC DISEASES EVALUATION
INTERVIEW GUIDE
Evaluator Protocol
(Individuals who collect and track data, and oversee state program evaluation)
Roles may include the following:
• State program evaluators
• Medicaid claims/encounter technical staff
• Data collection and tracking staff
Questions in blue are designed to be probes that will be asked when needed.

1

Introduction
Thank you for making time to speak with us today. We are researchers from Research Triangle Institute
International (RTI) and the National Academy for State Health Policy (NASHP), evaluating the
Medicaid Incentives for Prevention of Chronic Diseases demonstration.
In this discussion, we are interested in gathering your perspective on implementation progress and
challenges, program details such as the types of special populations involved or targeted by your state
initiative, your data collection processes, and the project’s evaluation progress. We are also particularly
interested in your perspective on the effectiveness and sustainability of this initiative on reducing
Medicaid and health care costs.
Our evaluation is being funded by the Centers for Medicare & Medicaid Services (CMS). As a condition
of participation, CMS expects state staff and program partners to cooperate with the evaluation team, but
ultimately your decision to participate in this aspect of the study is voluntary. If you do not wish to
participate in this interview or answer specific questions, please let us know immediately.
We believe there are minimal risks to you from participation, and every effort will be made to protect
your confidentiality. We want to assure you that we will not quote you by name. We will use some
quotes in reports, but quotes will not be attributed to an individual or his or her organization.
There are no direct benefits to you from participating in this study. But your insights will be used by
federal and state policymakers as well as other Medicaid programs to improve the impact that incentives
have on chronic disease prevention and management among Medicaid beneficiaries and address the
challenges involved in implementing an incentivized health prevention program. CMS and RTI’s
Institutional Review Board (IRB) have reviewed and approved this research protocol.
We would like to record our conversation, to ensure that our notes from today are complete. Do I have
your permission to audio record our conversation today?
Finally, we would like to follow up with you via e-mail to ask a few detailed questions related to your
annual state evaluation costs. Because these questions may involve reference materials, we thought it
would be helpful to share them via e-mail to provide you with time to review any relevant materials
before responding.
Are the remaining interview conditions OK with you? Do you have any questions before we begin?
Background
About the Respondent
1. What is your role in [name of state program]?
2. How long have you been in this role?
3. [For partner organization staff only] How did your [organization/clinic] become involved
in [name of state program]?
a. Did your [organization] respond to a solicitation such as an RFP to participate in
[name of state program]?

2

Implementation Progress
Overall Program
4. To your knowledge, has implementation of [name of state program] followed the originally
proposed timeline?
a. What are the reasons implementation has not followed the original timeline?
5. In your opinion, what have been the key successes of the [name of state program]?
a. What factors helped [name of state program] succeed?
b. What stakeholders were involved in each success?
6. In your opinion, what challenges or barriers has [name of state program] faced
implementing the program? These challenges may not impact your role as evaluator, but
rather may have had an impact on the project.
a. Did [name of state program] face any challenges obtaining IRB approval?
b. How has the [name of state program] addressed this challenge(s)?
c. Are there any challenges that remain?
i. In your opinion, what could help [name of state program] overcome these
remaining challenge(s)?
d. What impact, if any, have these challenges had on [name of state program]’s
evaluation plan?
i. Short term? Long term?
Special Populations
7. As part of Affordable Care Act, the MIPCD national demonstration is required to focus on
the extent to which special populations (such as adults with disabilities, children with
special health care needs) are able to participate in the program. What, if any, State
evaluation efforts focus on special populations?
a. What special populations are included in these efforts?
b. How were these populations selected?
8. Are there special populations that are not included in your evaluation efforts that you feel
should be?
a. Who are these groups?
b. In your opinion, what needs to be done to include them in your evaluation efforts?
3

9. Will any of your analyses examine how study outcomes vary among specific sub-groups of
participants such as:
a. Sex?
b. Racial or ethnic group?
c. Geographic location?
d. Other characteristics?
10. What impact do participants who speak languages other than English have on your
evaluation efforts?
a. Have you translated your evaluation instruments into other languages or used
translators to assist participants in completing these instruments?
State Grant Evaluation Progress
11. Based on your state operational protocol, we know that you are conducting a [include
evaluation type here]. What changes, if any, have been made to this evaluation plan over
time?
a. Tell us about the changes and the current plan that is in place.
i. Why were these changes made?
12. How is this current evaluation progressing?
a. Is it occurring according to the originally proposed timeline?
i. If not, why?
13. What are the major evaluation challenges you’ve encountered?
a. Tell us about these challenges.
b. How are these challenges being addressed?
14. The MIPCD grant program is designed to test the effectiveness of providing incentives
directly to Medicaid beneficiaries of all ages to try to change their health risks and
outcomes by adopting healthy behaviors. Are there any results to date?
a. What has [name of state program] learned so far from this evaluation?
15. If evaluation results are currently available, to what extent are these results being shared
with state staff and recruiters and providers working directly with participants?
a. Please describe how these results are being shared.

4

i. Through what methods?
ii. How often?
b. What impact have these results had on program implementation or design?
c. [IF NOT SHARED] Are there plans to share the evaluation results in the future?
i. If so, please tell us how, when, and with whom.
16. When do you plan to share the final results with [name of state program] management?
a. How long after the final incentives are disbursed to participants at the end of
December 2015?
Data Collection
17. We are aware that the [name of state program] has implemented [list the data tracking
systems] to track data on participation, health behavior change, and changes in health
outcomes. Are there other tracking systems the program uses to collect data?
18. In some states, evaluators are receiving data collected by providers or educators. What
data, if any, do you receive from other program staff?
19. Are there any issues you have encountered collecting, receiving, or tracking participant
data?
a. In your opinion, do you have the data you need for this evaluation?
b. In your opinion, are the systems used able to adequately monitor and track
participants?
20. How often do you collect or receive the data necessary to conduct your evaluation?
21. Are there any differences collecting, receiving, or tracking control group participant data
compared to experimental group data?
22. The state indicated that you were planning to include [not include] individuals who are
enrolled in both Medicaid and Medicare, dual eligibles. How are these participants
identified within [name of state program] system?
23. What analyses, if any, are you conducting specifically on dual eligibles in your evaluation?
Overall
24. What advice would you give other evaluators assessing similar types of prevention
programs?
25. Is there anything else about [name of state program], that we haven’t covered but that would
be important for our team to know?
5