CMS-10398 (#39) 39 Protocol for DA States

OMB Control Number 0938-1148

Expiration date: { mm/dd/2015 } CMS staff to fill this in.

Data Analytics Learning Collaborative

State participants

2015 Individual Discussion Guide

(45 minutes)

INTRODUCTION

Hello, I’m [NAME] from Mathematica Policy Research. Thank you for taking the time to talk with me. As part of the MAC LC project, Mathematica is assessing the LC experience. Through this assessment we will provide feedback that may be used to help CMS learn about the features of effective LCs and to improve the LC experience for state participants in the future. We’re collecting feedback from participants from the states, as well as from LC facilitators and the CMS subject matter experts associated with each LC. We’re interested in talking with you about your experiences with the Data Analytics LC. [For states only involved with performance indicators: Specifically, we are interested in your experience with developing and/or reporting the new eligibility and enrollment performance indicators for Medicaid and CHIP, work that the Data Analytics LC has supported over the past two years].

This interview will cover the last (four/two) years of the LC. We will not share interview notes with the LC facilitation teams, or other CMS staff, and we will not use direct quotes in any reports or briefing documents that result from the assessment activities. If it is okay with you, we’d like to record this interview for the purpose of completing our notes only. Do you have any objection to our taping of the discussion?

The discussion will last up to 45 minutes. We have a series of questions we would like to ask you, so to make sure we don’t take up too much of your time, we may re-direct the conversation at some points.

Do you have any questions before we begin?

A. Respondent Background and Role in the Data Analytics LC

Let’s do quick introductions.

1. Please tell us your name, organization, job title, and how long you have been with that organization.

2. When did you become involved in [or aware of] the Data Analytics LC?

B. Goals and Objectives

Now, let’s talk about your goals and objectives for participating in the Data Analytics LC.

1. [Skip for states only involved in the performance indicators] Thinking back to when you first decided to participate in the Data Analytics LC, what did you hope to gain from participating?

1. For example, were you interested in the Data Analytics LC primarily as an opportunity to collaborate with other states to improve data analytics capacity through peer-to-peer sharing; to seek technical assistance on using CMS Medicaid data sources; to collaborate with CMS on its priority projects related to state data analytics (e.g., T-MSIS)?

2. How were the goals and objectives of the LC communicated to you? How well did they align with your own motivation for participating?

   1. Do you think the goals and objectives of the LC shifted over time? How did that affect the value of the LC experience for your state?

C. State Experience of LC Activities

Now we have a few questions about your engagement with and experience of the various LC activities.

1) How would you describe your state’s level of participation in the webinars and meetings convened by the Data Analytics LC?

a) There were 19 webinars in the first two years of the LC, and then 5 in the latter two years. About what proportion of these webinars did your state attend (e.g., a third, half, three-quarters). [Interviewer note: States only involved in the performance indicators would have been exposed to only the webinars in the latter two years]

i) [If attendance is less than three-quarters] What prevented you from attending the other webinars (i.e., scheduling, content was not perceived as valuable, etc)?

b) Have you, or someone else from your state, presented on your state’s experiences with data analytics at an LC meeting?

i) If not, have you wanted to present material or felt that you had insights to share? What prevented you from sharing your insights?

c) Have you asked questions during discussions?

i) How helpful were the responses you received?

ii) If you have not asked questions, what prevented you from asking questions?

d) Have you ever contacted another state, CMS, or the LC facilitators to follow up after an LC session? How helpful was that interaction?

2) In the last two years, the Data Analytics LC has focused on the Medicaid and CHIP performance indicators. How would you describe your experience of the work the LC supported on performance indicators?

a) How did you participate in the development of the indicators? What did or did not work about the process of obtaining state feedback on indicators?

b) How would you describe your experience with the technical assistance you received as part of the work on performance indicator data? Has it improved your state’s data reporting capacity, the quality of the data, or your ability to use the reported data?

3) [For states only involved with performance indicators] Are you aware of the Medicaid.gov site? Have you accessed any of the materials produced by the DA LC, such as the following issue briefs: “Findings from an Analysis of Publicly Available Reports on Medicaid and CHIP Performance Measures” and “Strategies to Enhance Stakeholders’ Use of Publicly Available Medicaid and CHIP Performance Measures”? Have you used any of the webinar slide decks?

a) How have you used the materials?

b) How appropriate was the level of detail for materials? Did it strike the right balance between being too basic and getting too detailed?

4) [For states involved in the small group DA LC] Have you used documents produced or disseminated by the LC? For example, the “Strategies to Enhance Stakeholders’ Use of Publicly Available Medicaid and CHIP Performance Measures” issue brief, background resources on issues related to data analytics (e.g. measuring ED use, measuring quality of care outcomes), or any of the webinar slide decks that have been made available?

a) How have you used the materials?

b) How appropriate was the level of detail for materials shared in the Data Analytics LC? Did it strike the right balance between being too basic and getting too detailed?

5) What could be done to enhance your level of engagement in the LC?

a) Do you feel that you had sufficient notice and explanation about upcoming LC sessions so that appropriate staff from your state could be available to attend?

i) Thinking about specific webinars, were you able to identify who should attend from the state, given the information provided in advance about the topics covered? [Interviewer note: Examples include the Measuring Avoidable ED Use webinar in September 2012, Measuring Beneficiaries’ Experiences with Care webinar in March 2013, and Overview of Eligibility & Enrollment Performance Indicators webinar in June 2014.]

b) Do you feel like you had sufficient opportunity to suggest topics that should be covered in LC sessions or via technical assistance?

6) How would you describe the interaction between Data Analytics LC states’ and CMS subject matter experts? Is it sufficient and productive?

a) What supports/hinders this interaction?

7) How would you describe the interaction between LC states? What worked or did not work to promote a peer-to-peer learning environment?

a) Would you be interested in engaging with other states through an online forum or document sharing tool? What aspects of such a forum or tool would make your active participation more likely (for example, a forum facilitator, e-mailed updates when a participant posts something)?

D. Benefits for States

1. In what ways did you benefit from participation in the LC? How well did the LC meet your goals and objectives? Please give specific examples.

2. What specific role can LCs play, in contrast to other kinds of technical assistance offered to states? How best can LCs complement and not duplicate the assistance provided by state-led organizations, think tanks or other non-profits, such as the National Association of Medicaid Directors or the Robert Wood Johnson Foundation?

3. Can you think of any ways the LC could be modified so that your state reaps additional benefits?

4. Looking back on the LC overall, what features or aspects worked well and would be important to incorporate in future LCs?

E. Wrap Up

Now, let’s wrap up.

1. Other than what you’ve already mentioned, are there any ideas you would like to share with me for improving the effectiveness of future LCs?

2. Thank you for taking the time to talk with us today. Is there anything else someone would like to add before we end the discussion?

PRA Disclosure Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-1148. The time required to complete this information collection is estimated to average 45 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.