State, Local, and Tribal Government (CN Agency Staff, State Medicaid Agency Staff, Other State Agency Staff, and School District Staff)

APPENDIX A-3c

DETAILED SITE VISIT INTERVIEW PROTOCOL:
DISTRICT STAFF

This page has been left blank for double-sided copying.

Detailed Site Visit Interview Protocol: District Staff

Evaluation of Demonstrations of Direct Certification with Medicaid for Free and Reduced-Price Meals (DCM-F/RP)

State:	Date:
Interviewee(s)/position(s):	Start time:
	End time:
Permission to record:	Interviewer:

Introduction

The purpose of this interview is to gather information about [State’s] demonstration of direct certification of students for free or reduced-price meals using Medicaid data. This new demonstration is known by the acronym DCM-F/RP [for States that participated in the previous demonstration, add: to distinguish it from the earlier DCM demonstration].

The information that we collect in this interview will be used to describe the experiences of States participating in the demonstration. Because each State’s project is unique, a description of a particular State’s experiences could identify that State. We will not use your name in our reports, but your identity might be inferred from the identity of your State and the nature of the information that you provide. If there is something that you want to say in confidence that you would not otherwise mention, let us know and we’ll use it to enhance our understanding, but we will keep the details private. Although participation in the data collection is a requirement of the demonstration, your individual participation in this interview is voluntary, and you do not have to answer any questions that you don’t want to answer.

The interview will last approximately one hour. With your permission, we’d like to make an audio recording of the interview to fill in gaps in our notes. Do I have your permission to record the interview?

[If yes:] Thank you.

[If no:] You have my assurance that we will keep anything private that you wish. If you’d prefer, we will not cite anything that you say verbatim from the recording. Wait for response; if yes, record: Thank you. If no: That’s OK. Just bear with us/me as I take detailed notes.

Background

Before we discuss the details of your direct certification process, I’d like to learn more about your district, your role here, and your direct certification efforts more broadly.

1. What is your position here at [district]?

2. What is your role in the DCM-F/RP process?

3. [Moderator note: Because we’ll be interviewing almost everyone who plays a key role, the responses to this question will collectively summarize the DCM-F/RP process. Later sections of the protocols delve into the details of many activities, but if a response here mentions aspects not fully covered by later questions, probe for details here.]

   1. [After each person in the interview describes his or her role:] Are there any aspects of the process conducted by your district that haven’t been covered in the description you just gave me? [If so:]

      1. Tell me about that part of the process.

      2. Who conducts that step of the process? [If someone not on the site visit schedule (or who was scheduled but unable to attend the interview):] Could I contact [that person] by telephone later to learn more about [his or her] role?

   2. [If any key steps (retrieving the Medicaid file, conducting the match[in local-matching states], updating certification status) haven’t been mentioned, ask:] What about [steps]? Who is involved in that part of the process? [If they’re not on the site visit schedule:] Could I contact them by telephone later to learn about their role?

4. Can you tell me a little bit about your district and student population?

   1. [If district characteristics unknown, ask:] We pulled statistics on your district, but there are some data I could not find.

Planning and preparation

Let’s move on to how you prepared for DCM-F/RP.

5. When did your district first learn about DCM-F/RP?

   1. How did you learn about it?

   2. Was any guidance or training provided to your district on DCM-F/RP? If so, please describe.

6. How did your district plan and prepare for DCM-F/RP?

   1. What concerns or challenges did you encounter when planning and preparing for DCM-F/RP?

   2. [Florida and Massachusetts, ask:] How did participation in the first demonstration help you prepare for DCM-F/RP?

      1. What are some of the key changes you made to your approach for DCM-F/RP?

7. Was your district involved in any testing of the DCM-F/RP implementation process? If so, please tell me about that.

8. [In local-level matching States, ask:] What data systems do you use to conduct direct certification matching? (Probe: Do you match by hand (Excel) or by using point-of-sale [POS] software?)

   1. [If matching using POS system, ask:] Who is your point-of-sale vendor?

      1. Was your point-of-sale vendor aware of DCM-F/RP when you first discussed it?
         [If aware, ask:] How did the vendor prepare the software to conduct DCM‑F/RP?
         [If unaware, ask:] How is the vendor accommodating the DCM-F/RP match since learning about it?

9. In what ways has the implementation of DCM-F/RP changed the direct certification process in your district?

   1. Does the direct certification process this year differ from last year’s process in any other ways (not related to DCM-F/RP)?

Obtaining the data

In order to directly certify students under Medicaid, you must first receive a list of [matches/eligible students] from the [CN agency]. Let’s talk about that process a bit.

[Moderator note: No need to ask questions that have been fully answered in response to Question 2 above.]

10. How is the list of [matches/eligible Medicaid students] made available to your district?

    1. Does the State agency provide a single direct certification list that combines students who are eligible based on different programs (SNAP, Medicaid, etc.), or is the list of [DCM-F/RP matches/eligible Medicaid children] provided separately?

    2. What is the geographic scope of the file? Does your district receive a list of [matches/eligible children] for the entire State, just your district, or some other area?
       [If just your district, ask:] How does [State] identify your district boundaries (e.g., based on county, a district indicator in the statewide student information system, etc.)?

11. Are duplicates included in the file(s) you receive? For instance, if a child receives both SNAP and Medicaid, does he or she appear only once in the file(s)? If once, does the file indicate all programs or just one? Which one?

12. [In central-level matching States, ask]: What data does [State] need from your district to conduct the direct certification match and create your file?
    [Probe:] Do you have to upload an enrollment list for matching, or does the [State] maintain student enrollment data that it matches against?

    1. [If upload enrollment lists to State, ask]: How do you provide these data to [State]? How often do you upload your enrollment? Please describe in detail.

13. [Central-level matching, ask]: Do you receive a list of partial matches or nonmatches from [State]? If so:

    1. Are any additional details provided with these matches (e.g., match probability, what variables failed to match)?

14. What challenges, if any, have you encountered in obtaining the DCM-F/RP file? Describe how you overcame them or, if ongoing, how you plan to do so.

    1. What technical assistance or resources were most helpful to you in resolving each of these challenges?

Timing

15. [Central-level matching, ask:] How often does [State CN agency] send you a list of DCM-F/RP matches?

[Local-level matching, ask:] How often does [State CN agency] send you a list of DCM-F/RP– eligible children?

16. [Central-level matching, ask:] How often do you download and use the list of DCM‑F/RP matches?

[Local-level matching, ask:] How often do you conduct DCM-F/RP matching during the school year?

17. Is the timing the same for DCM-F/RP as for direct certification with SNAP?

DCM-F/RP matching and certification process

Now I’d like to discuss what happens after you receive the list of [matches/eligible Medicaid children].

[Moderator note: Respondents may jump around when discussing their matching process due to the many components involved. Be prepared to jump between questions in this section.]

18. [Local-level matching using POS system, ask:] Once you receive the list of eligible Medicaid children, do you have to modify or format the list in order to conduct your match?

    1. [Probe:] Do you have to reformat or reorganize any variables to conduct an automated match using your point-of-sale system? If so, what do you have to do? How much effort is required to prepare the file?

19. Can you describe the next step in the DCM-F/RP process in your district after you receive [and format] the list of [matches/eligible children]?

    1. [Local-level matching probe:] How do you match eligible Medicaid students to your enrollment lists? How do you update their direct certification status?

    2. [State-level matching probe:] How do you update students’ direct certification status after you receive the list of DCM-F/RP matches?

    3. How do you track what program (e.g., SNAP, TANF, Medicaid, etc.) a student matches under?

    4. What updates, if any, were needed to your data systems to track Medicaid and direct certification for reduced-price meals?

20. [Central-level matching and receives DCM-F/RP data for partial matches, ask]: Do you attempt to match any of the partial matches that the [State CN agency] sends you? If so, please describe how you do this.

21. [Central-level matching and receives DCM-F/RP data for nonmatches, ask:] Do you attempt to match any of the nonmatches that the [State CN agency] sends you? If so, please describe how you do this.

22. [Local-level matching, ask:] What data elements do you use to conduct your match?

    1. Were the data elements or matching algorithm dictated by the State or determined by your district?

    2. Does your [point-of-sale software] use probabilistic matching?

       1. What results in a positive versus partial match? Please describe in detail.

    3. [Local-level matching using point-of-sale software, ask:] Does your district’s software use phonetic matching, string matching, date-of-birth accommodation, or other approaches to improve matching success? If so, please describe in detail.

       1. How does this facilitate DCM-F/RP matching?

    4. [Moderator note: Ask if the district have a reference guide that details the matching algorithms and if you may obtain a copy of it.]

23. To what extent do the direct certification results the State [CN agency] provides differ from the certification outcomes in your district?

    1. [Probe:] Does the list of direct certification matches the State sends you ever include students who do are not actually enrolled in your district? If so, how common is this?

    2. What do you do when a student listed as directly certified for reduced-price meals on the file the State sends you is already certified for free meals based on application? How common is this? [Probe:] Can you provide an estimate of the proportion of students listed as reduced-price on the direct certification list that are already certified for free meals?

24. When a student identified as eligible (though direct certification or application) has already been certified, do you overwrite the older information to reflect the change? Under what circumstances? [Probe: Which methods or bases of certification take precedence over which others?] Is this an automated or manual process?

    1. [Local-level matching or State-level matching that provides duplicates, ask:] How [do you/does your point-of-sale system] ensure that students who could be directly certified through both Medicaid and another program (e.g., SNAP, TANF, or FDPIR) are recorded as certified based on the other program?

       1. How easy or difficult was it for [you/your point-of-sale system] to ensure that matches with other program data take priority over Medicaid?

       2. What updates, if any, were required to your systems?

    2. If a student is directly certified by Medicaid and later enrolls in SNAP, do you change the student to DC-SNAP? Please explain the process.

25. How do you extend eligibility to other children who are in the same household as a child directly certified through Medicaid? Please describe in detail.

Time and resources

26. How much additional time and resources have been needed to implement DCM-F/RP?

    1. Which activities associated with DCM-F/RP have been the most time-consuming? The least time-consuming?

    2. Has DCM-F/RP saved your district or schools time in any way? How so?

       1. [Probe:] Has it reduced the number of paper applications?

       2. [Probe:] Has it reduced the amount of time you spend on verification?
          [If not, ask:] Why do you think that is? Was your district already conducting direct verification by using Medicaid data?

       3. [For districts that conducted their first DCM-F/RP match after the beginning of the school year, probe:] Do you think it would have saved your district or schools [more] time if the match had been conducted at the beginning of the school year? Please explain.

    3. Which activities associated with DCM-F/RP have been the most difficult? The least difficult?

Outcomes

Let’s discuss some of the early outcomes you’ve seen thus far.

27. [Local-level matching, ask:] Think about the results of DCM-F/RP matching. Approximately how many children were on the DCM-F/RP lists provided to you by [agency]? Roughly what proportion of these cases did you successfully match?

    1. How does this compare to the proportion of SNAP cases successfully matched from the list of SNAP recipients?

       1. [Probe, if applicable:] What are some reasons that you think the Medicaid match was more/less successful than the DC-SNAP match?

28. Has Medicaid matching success varied by student or household characteristics?

    1. Are there any differences by:

       1. Race/ethnicity?

       2. Family/household size?

       3. Family/household composition?

       4. Families/households with name differences?

       5. Split households (refers to children who live in two separate households if divorced parents split custody)?

       6. Grade level?

       7. Other student or household characteristics?

29. Are there any aspects of your district that may have influenced the success of DCM‑F/RP? If so, what are they?

Challenges and resolutions

Now I’d like you to think about any difficulties you experienced in implementing the demonstration.

30. What challenges have you encountered in implementing DCM-F/RP?
    [Probe:] Any other serious challenges?

    1. What problems have these challenges caused?

    2. To what extent have you been able to resolve each of these challenges? How?

    3. What technical assistance or resources were most helpful to you in resolving each of these challenges?

Best practices and lessons learned

[State] is one of the first states to implement DCM-F/RP. We’re interested in what you’ve learned from your experience.

31. What aspects of your State’s or your district’s direct certification processes helped facilitate DCM-F/RP?

    1. [Local-level matching, ask:]Are there specific features of your systems that helped you conduct the match?

    2. Are there other activities your district conducted that helped facilitate implementation?

    3. Did [CN agency] provide any resources, training, or technical assistance that helped you implement DCM-F/RP?

32. What aspects of your district’s direct certification processes made implementation more difficult?

    1. Are there specific features of your district’s systems that hindered implementation?

33. What changes, if any, have been planned to improve the matching success of DCM‑F/RP?

    1. Are there other changes you think your district—or the State—could implement to improve matching success?

       1. [Probe:] Are there any changes to your matching approach or point-of-sale systems that could improve your direct certification matching rates?

       2. [Probe:] Has your district or point-of-sale vendor explored phonetic matching, string matching, date-of-birth accommodation, or other approaches that are designed to improve matching success rates? What are the advantages and disadvantages of these approaches?

    2. What resources, training, or technical assistance would you find most helpful to help you implement DCM-F/RP?

Closing

34. Is there anything else you would like to add about your experience in implementing DCM-F/RP so far?

Thank you for meeting with us today. We’ll be contacting you [primary respondent for each agency] again in [month] to conduct a follow-up telephone interview to reflect on your experiences near the end of this DCM-F/RP school year.

This page has been left blank for double-sided copying.