ATTACHMENT N.2. Pretest report on quarterly calls

Attachment N.2.

pre-test Memo from quarterly interview guides

Pretest Summary: Quarterly Interview Guides

A. Overview of the Quarterly Interview Guide Pretest

The quarterly interview guide instrument collects information about Medicaid and CHIP experiences with defining and implementing enrollment and renewal policies. In March 2012, we invited two respondents to complete the interview guide and discuss their responses and their experience of the instrument with us. One was a former state Medicaid official who had multiple years of experience with Medicaid and CHIP enrollment and renewal. The second is a current CHIP director from a state with a separate CHIP program (and who is also a member of the project’s Technical Advisory Group). The purpose of the pretest was to test the language, question flow, respondent comprehension, and to determine the overall burden of completing the survey.

Each of the respondents completed the interview by phone (these are designed to be administered in this manner). The average time to complete the survey was 45 minutes, but this time includes the debriefing and discussion we conducted with them about the questions and related edits. With this part of the conversation removed, we expect response times will average 25 minutes, ranging from 20 to 30 minutes for each interview.

Both respondents agreed that their agency would have found the survey of interest and relevant, and that their agency would have responded to our request (even given its quarterly nature). Both found the survey length acceptable.

Respondents noted that the Medicaid Director or CHIP Director (if the state has a separate CHIP program) are the best persons to contact initially to schedule the interviews, even though he or she may refer it on for completion. We will work with the Medicaid and CHIP Directors to identify the most appropriate informant for these interviews.

Both Respondents identified a few general revisions that will assist in obtaining more precise answers from participants, but did not suggest substantive changes in question content. Their main revisions are summarized as follows:

• Because we are not including any currently approved ELE states in our group of 30 (although some of them may administer an un-approved or ELE “like” program), we revised the ELE-specific questions to be directed to any “newly identified” ELE states, or in some cases, “newly considering” ELE.

• We modified language that presumed an answer, such as “how are you tracking…” to “Are you tracking…if so, how?”

• When asking about costs/savings related to various policy options, we clarified that we were interested in both administrative and benefit/program costs and savings.

• We added a statement in the beginning of each interview guide assuring respondents that they will not be individually identified.

Additional clarifying and simplifying edits made in response to respondent feedback are noted in the attached complete copy of the revised interview guide. New language is shown in yellow highlight, and language removed is shown in strikethrough font. (Attachment G, an attachment to the Part A Statement, contains a clean version of the final protocol.)

Interview Guide for the 30-State Key Informant Interviews

Questions for the 1st quarterly call

Introduction: This study is being conducted for the U.S. Department of Health and Human Services (HHS) by its contractor, Health Management Associates (HMA). The state interviews are part of a larger evaluation being conducted by HMA, Mathematica Policy Research, and the Urban Institute to understand the impact of states’ activities to enroll and retain eligible children in Medicaid and CHIP.

CHIPRA authorized an evaluation of Express Lane Eligibility (ELE), creating an exceptional opportunity to document ELE implementation across states and to assess the changes to coverage or administrative costs that may have resulted. The evaluation also provides an opportunity to understand other methods of simplified enrollment that states have been pursuing and to assess the benefits and potential costs of these methods compared to those of ELE.

The following questions seek your perspective on implementing and operating enrollment simplification policies in [Interviewee’s State]. As you are aware, we will conduct follow up interviews with you on a quarterly basis to track how these efforts are unfolding over the next year. Information you provide on state policies will be reported at the state level, but your individual views on progress and impacts will be reported in aggregate narrative and will not be attributed to you or your state.

1. Can you provide a brief summary of the enrollment simplification policies you have in place currently? PROMPT: Are the following [summarize currently reported simplifications for enrollment] still in place?

2. Are you working on developing any new activities and/or policies that will improve identification and enrollment of children in Medicaid, CHIP, and or other publicly subsidized health insurance programs?

1. If not, why?

2. If so, what are they?

3. Where are you in the development process?

   1. Data analysis (enrollment/fiscal)?

   2. IT needs assessment?

   3. CMS negotiation/SPA development?

   4. Other?

4. Why did you choose to explore the approach(es)? (NOTE: legislative/Gubernatorial health coverage champion, administrative savings arguments, political will to ensure all children are covered, etc.)

3. What efforts to identify or enroll children have been recently implemented? (In the past year.)

1. For each, what were the effective or “go live” dates?

2. What administrative and/or policy barriers have you encountered?

3. How are you overcoming them?

4. How Are you tracking the impact of these policies/operational approaches in terms of coverage and costs? If so what have you found? We are interested in both Administrative and Benefit costs.

5. Do you have any early indicators of success/not a success?

6. For the long-standing identification and enrollment activities that have been in place for over a year, do you have any newly available findings related to measuring the impact?

1. Are you happy with how these policies are operating today? If so, how did you get to that point?

2. Are you considering any changes or improvements?

3. What administrative and/or policy barriers did you encountered?

4. How are you overcoming them?

7. We’re interested in your perspective on the value of this evaluation. What would you consider to be most important to learn from this study?

8. Anything else you’d like to tell me about your states simplifications in CHIP or Medicaid that we haven’t covered?

Interview Guide for the 30-State Key Informant Interviews

Questions for the 2^nd quarterly call

Introduction: The first two questions below are follow-up questions on progress reported in your previous quarterly interview in (insert month/year). The remaining questions seek your perspective on issues related to [your state’s] process for developing enrollment simplification policies. Information you provide on state policies will be reported at the state level, but your individual views on progress and impacts will be reported in aggregate narrative and will not be attributed to you or your state.

1. Are the following simplification policies still in place? [List state-specific simplification policies]

2. Is there anything new that we haven’t previously discussed?

3. When we talked last quarter, you reported the following simplifications were newly implemented and/or in development [List based on previous quarter response]. Can you provide a progress update?

   1. Where are you in the development process?

      1. Data analysis (enrollment/fiscal)?

      2. IT needs assessment?

      3. CMS negotiation/SPA development?

      4. Other?

   2. What administrative and/or policy barriers have you encountered?

   3. How are you overcoming them?

4. What are the goals of ELE [or Non-ELE] application and renewal simplification policies? intervention?

   1. How Have these goals changed over time? If so, how?

5. What Did you get input did you get from families and consumers and advocates prior to design? If so, what?

6. How did [your state] decide which ELE/non-ELE simplification policy option to adopt [PROMPT from list provided in Question #1]?

7. For For ELE state:States newly considering Express Lane Eligibility (ELE) - what changes were required have you identified to be needed to use ELE? [PROMPT: staffing, organizational/ cultural issues, IT, legislative, SPA negotiation etc.?] Describe the level of effort it will required.

1. Which programs are you considering for matching up eligibility information?

2. Why those and not the others?

8. For states opting not to pursue ELE: What are the reasons you opted to not pursue ELE? [PROMPT: staffing, organizational/ cultural issues, IT, legislative, SPA negotiation etc.?]

9. For Non-ELE states: How Do you believe does your streamlined approach to finding and enrolling eligible children compare is as effective compared to ELE? If so, why? PROMPT: What are the advantages of this approach in your view? Can you provide examples of outcomes of this approach; state satisfaction with approach

10. Anything else you’d like to tell me about your states simplifications in CHIP or Medicaid that we haven’t covered?

Interview Guide for the 30-State Key Informant Interviews

Questions for the 3rd quarterly call

Introduction: The first two questions below are follow-up questions on progress reported in your previous quarterly interview (in month/year). The remaining questions seek your perspective on issues related to [your state’s] process for implementing enrollment simplification policies. Information you provide on state policies will be reported at the state level, but your individual views on progress and impacts will be reported in aggregate narrative and will not be attributed to you or your state.

1). Are the following simplification policies still in place? [List state-specific simplification policies]

2) When we talked last quarter, you reported the following simplifications were newly implemented and/or in development [List based on previous quarter response]. Are there any other new policies? Can you provide a progress update?

1. Where are you in the development process?

   1. Data analysis (enrollment/fiscal)?

   2. IT needs assessment?

   3. CMS negotiation/SPA development?

   4. Other?

2. What Are there any administrative and/or policy barriers have you encountered?

3. How Are you overcoming them? If so, how?

3. For ELE states: what pieces needed to be in place that made it successful?? For each category below, please describe the necessary conditions for implementation:

   1. Relationships between sister agencies/identifying partner agencies.

   2. Political will/Policy environment for implementing or not implementing ELE (emphasis on budget, coverage of uninsured children, program integrity)

   3. Technological readiness pre-intervention

   4. Implementation at the field level: issues for field workers, outreach workers, providers doing enrollment etc.

4. How did you determine the policy decision related to allowing families to opt in vs. opt-out of ELE enrollment?

5. What is your process for ensuring enrollees (and their parent/guardians) understand their options related to providing consent to participate in ELE determination?

3. For Non-ELE states: Which of your simplification policies has been most challenging to implement and why?For each category below, please describe the necessary conditions for implementation:

   1. Relationships between sister agencies/identifying partner agencies.

   2. Policy environment (emphasis on budget, coverage of uninsured children, program integrity)

   3. Technological readiness pre-intervention

   4. Implementation at the field level: issues for field workers, outreach workers, providers doing enrollment etc.

4. For any of these simplifications Ddo you do post implementation outcome analysis? If so:

   1. Do you review denials to determine why people were denied?

      1. If so, what are you finding? How are you addressing?

   2. Do you review approvals to evaluate accuracy?

      1. If so, what are you finding?

5. Anything else you’d like to tell me about your states simplifications in CHIP or Medicaid that we haven’t covered?

Interview Guide for the 30-State Key Informant Interviews

Questions for the 4th quarterly call

Introduction: The first two questions below are follow-up questions on progress reported in your previous quarterly interview (in month/year). The remaining questions seek your perspective on the financial impact (costs vs. savings) related to implementing enrollment simplification policies. Information you provide on state policies will be reported at the state level, but your individual views on progress and impacts will be reported in aggregate narrative and will not be attributed to you or your state.

1. Are the following simplification policies still in place? [List state-specific simplification policies]

2. When we talked last quarter, you reported the following simplifications were newly implemented and/or in development [List based on previous quarter response]. Can you provide a progress update?

   1. Where are you in the development process?

      1. Data analysis (enrollment/fiscal)?

      2. IT needs assessment?

      3. CMS negotiation/SPA development?

      4. Other?

   2. What administrative and/or policy barriers have you encountered?

   3. How are you overcoming them?

3. For newly identified ELE State: what kind and level of financial investment was required to implement policy?

   1. IT infrastructure?

   2. Administrative costs related to staffing?

   3. Program costs? What percentage of enrollment increase do you attribute to these policies?

   4. Other?

4. How were these costs covered? How do you track costs for this?

   1. Private and/or Public Grant?

   2. State administrative funding?

5. Were costs more/less than you expected/budgeted?

6. How Has your state realized are you tracking any administrative savings?

   1. What are the findings?

7. For Non-ELE states: how much were implementation costs a factor in opting to not pursue ELE?

8. For Non-ELE states: how Are you tracking programs/administrative savings related to other simplification strategies? [PROMPT: paperless verification, electronic reminders, retention strategies that decrease processing new applications, etc.]

   1. What are the findings?

9. Anything else you’d like to tell me about your states simplifications in CHIP or Medicaid that we haven’t covered?

Interview Guide for the 30-State Key Informant Interviews

Questions for the 5th quarterly call

Introduction: The first four questions below are follow-up questions on progress reported in your previous quarterly interview in (month/year). The remaining questions seek your perspective on how your state’s enrollment simplification policies are facilitating readiness with the upcoming 2014 Medicaid Expansion and Health Insurance Exchanges. Information you provide on state policies will be reported at the state level, but your individual views on progress and impacts will be reported in aggregate narrative and will not be attributed to you or your state.

1. Are the following simplification policies still in place? [List state-specific simplification policies]

2. When we talked last quarter, you reported the following simplifications were newly implemented and/or in development [List based on previous quarter response]. Can you provide a progress update?

   1. Where are you in the development process?

      1. Data analysis (enrollment/fiscal)?

      2. IT needs assessment?

      3. CMS negotiation/SPA development?

      4. Other?

   2. What administrative and/or policy barriers have you encountered?

   3. How are you overcoming them?

3. How are you tracking the impact of these policies/operational approaches in terms of coverage and administrative costs?

4. Do you have any early indicators of success/not a success?

5. Are Have your states simplification activities contributed contributing to activities to prepare for Medicaid Expansion and Health Insurance Exchanges?

   1. If yes, how so?

6. What are some key lessons learned about simplifying enrollment/renewal that could be helpful to ACA implementation?

7. For ELE States: Do you envision potentially expanding ELE to adults, if Federally authorized? Other simplification strategies?

8. For Non-ELE States: Do you envision potentially expanding some or all of these simplification policies to adults, if Federally authorized?

   1. If so, which ones?

9) Anything else you’d like to tell me about your states simplifications in CHIP or Medicaid that we haven’t covered?