Touchpoints for Addressing Substance Use Issues in Home Visiting: Performance Measurement Pilot

^{A nchor}

Touchpoints for Addressing Substance Use Issues in Home Visiting

Pilot Study Interview Protocol: State Awardees

February 2021

CHARLOTTE CABILI, STACIE FELDMAN, AND MELISSA AZUR

This page has been left blank for double-sided copying.

Interview protocol: STATE AWARDEES

As you may remember, my firm, Mathematica, has been contracted to conduct a pilot study of two performance measures, called Screening for Substance Misuse (SUD-1) and Follow up for Caregivers At Risk of Substance Misuse (SUD-2). In advance of today’s interview, we emailed you a reference document that shows the data used to derive these measures.

The purpose of the pilot study is to assess the usefulness of the measures to Maternal, Infant, and Early Childhood Home Visiting (MIECHV) awardees and local implementing agencies for improving home visiting services and identify facilitators and challenges with using the measures. Mathematica is conducting this evaluation on behalf of the Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF) in collaboration with the Health Resources and Services Administration (HRSA). There are no direct benefits or risks for your participation, but your input will help to determine the feasibility and usefulness of these two performance measures that could be adopted into the MIECHV performance system.

To date, Mathematica has been providing several local implementing agencies with technical assistance on the use of these performance measures. Today, Mathematica is going to ask you to reflect on practices in your state along three broad topics: existing substance use screening and follow up care before the pilot; substance use screening measurement before the pilot; and exploring the usefulness of SUD-1 and SUD-2. We expect our discussion to last about 90 minutes. Your participation is voluntary, and you may choose to skip any question. Mathematica will compile findings from this interview, other state MIECHV awardees, local implementing agencies, and home visiting model representatives. We will provide our aggregated pilot study findings to OPRE and HRSA. Your information will be kept private to the extent permitted by law. We will not name individual staff in our pilot study report. Our main goal is to learn from you!

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0355 and the expiration date is 05/31/2021. If you have any comments on this collection of information, please contact Melissa Azur.

Would it be okay if we recorded the interview? We take notes to capture everything you say but sometimes we miss something important. We listen to recordings in case we want to go back and check something in our notes. We do not share recordings outside of the study team and will destroy recordings when our project ends.

[If the respondent does not want to be recorded] That’s no problem.

Do you have any questions before we begin?

I. Existing substance use screening and follow up care

1. I would like to begin by hearing about your role at [STATE AGENCY NAME]. What is your formal title and your key responsibilities?

1. How long have you worked in your current role?

Now I would like to talk through your agency’s formal documentation and other materials and supports that you make available to local implementing agencies. We will be focusing on documentation and supports related to substance use screenings and the follow up care with a behavioral health specialist after positive screens.

3. What kinds of written policies or procedures does your agency provide to local implementing agencies on substance use screenings and follow up care with a behavioral health specialist?

4. What kinds of professional development materials, trainings, or other guidance resources does your agency provide to local implementing agencies on substance use screening and follow up care with a behavioral health specialist?

Following responses to questions 3 and 4, probe for policies, procedures, and supports on these topics:

1. Purpose or benefits of screening and follow up care

1. Tracking data on screenings, referrals, follow up care appointments and attendance

2. Recommended substance use screening tools

3. How and when screening is conducted

4. For whom screening is conducted

5. Scoring methods

6. Interpreting the screening results

7. Approaching and engaging in discussions about screening results with primary caregivers when substance misuse is identified

8. Explaining availability and benefits of follow up care

9. Identifying appropriate follow up care providers

10. Conveying positive screening results to behavioral health specialist

11. Timing of meetings with behavioral health specialists, for all primary caregivers and for pregnant mothers (if different)

12. Monitoring and use of screening results

5. What additional supports might help your agency to support local implementing agencies?

6. What kinds of professional development materials, trainings, or other guidance resources does your agency make available to local implementing agency staff to help them support primary caregivers with substance use issues?

Probe for primary caregiver supports:

1. Education on the effects of substance use on themselves and their children’s development

2. Positive social support

3. Support for behavior change promotion

7. What additional materials and resources might be beneficial for your agency to provide to home visiting staff to build their knowledge base and improve their practices related to supporting primary caregivers with substance use issues?

Use probes (a) through (c) above.

II. Substance use screening measurement

I am going to ask about substance use screening data that you collect from local implementing agencies.

1. Does your agency routinely collect substance use screening data from local implementing agencies? If so, do you do this through an online portal or some other method?

If no, go to question 7.

2. How often are substance use screening data collected?

3. What substance use screening measures do local implementing agencies report to your agency?

Probe separately for measures on completed screenings and follow up care with a behavioral health specialist.

4. Now let’s talk about the requirements for each screening measure.

Probe for each screening measure:

1. Reference period (that is, time frame of interest)

2. Types of substances screened

3. Numerator definition (that is, who in your program is supposed to be screened for substance use)

4. Denominator definition (that is, the pool of eligible primary caregivers)

5. Exclusions from the measure and reporting missing data

6. How requirements are guided by [HOME VISITING MODEL NAMES USED IN STATE] and how requirements differ

5. Please describe the recordkeeping system that your agency uses to manage data on substance use screening and follow up care with a behavioral health specialist.

6. [Refer the respondent to SUD-1 and SUD-2 measure data specifications that were shared in advance of the interview] Does your recordkeeping system have the specifications and structure for collecting the data necessary to calculate SUD-1 and SUD-2?

Probe for individual data elements that are required:

1. Record identifier

2. Primary caregiver enrollment date in home visiting program

3. If primary caregiver has been enrolled for at least 30 days

4. Screening completed for unhealthy alcohol use or unhealthy drug use with a validated tool

5. Name of screening tool used

6. If screening completed for unhealthy alcohol use or unhealthy drug use occurred within 30 days of enrollment

7. If screen was positive for unhealthy alcohol use or unhealthy drug use

8. Date of positive screen for unhealthy alcohol use or unhealthy drug use

9. Status of primary caregiver participation in substance use treatment program

10. Follow up care with behavioral health specialist completed within 14 days of positive screen

11. Date of follow up care with behavioral health specialist

7. [Ask if the state agency does not collect substance use screening data from local implementing agencies. Refer the respondent to SUD-1 and SUD-2 measure data elements that were shared in advance of the interview] How would your recordkeeping system need to be modified to allow you to collect the data necessary to calculate SUD-1 and SUD-2?

Provide examples if needed:

1. Addition of data elements

2. Different data entry requirements, such as replacing open-ended with closed data fields

8. What might be the key barriers to collecting the required data?

Probe:

1. Local implementing agency staff capacity and training needs

2. IT or technology barriers or costs

3. Developing guidance resources

4. Conducting monitoring and technical assistance

9. Are these expected barriers similar or different than those you encounter when collecting other MIEHCV performance measures?

10. To what extent would the SUD-1 and SUD-2 collection add difficulty and preparation time to your typical MIECHV performance measure reporting process?

11. What kinds of resources and supports might help overcome these barriers or would generally improve your experience with measures collection overall?

Probe:

1. Forms and/or systems for monitoring and tracking

2. Policies and procedures

3. Organizational leadership

4. Staffing or training

5. Other

We would like to turn now to the costs your agency would expect to incur for SUD-1 and SUD-2 data collection, with respect to staff time, resources, and technology needs. Please consider the costs of both the initial data collection setup and ongoing data collection and management.

12. What purchased expenses would be involved in your agency’s collection of SUD-1 and SUD-2 from local implementing agencies?

Prompt for the estimated amount of each expense.

13. How much state agency staff time might be dedicated to the collection of SUD-1 and SUD-2? What kinds of staff at your agency would need to dedicate time to set up and ongoing collection?

III. Using information from SUD-1 and SUD-2 measurement

Now we will discuss how your agency might use the results of SUD-1 and SUD-2, that is, the proportion of positive screens and visits with a behavioral health provider, relative to primary caregiver enrollment.

1. What SUD-1 and SUD-2 data would you provide to various stakeholders? Why?

Probe for type of results by stakeholder group (for example, local implementing agency staff, other staff within your state agency, partner agencies and organizations).

Probe for the purpose of information sharing (for example, technical assistance or decision or policy making).

2. Think for a moment about ways your agency might use SUD-1 and SUD-2 measures data. How might this compare to ways your agency uses other MIECHV performance measures?

3. Please describe any reservations or cautions you have associated with your agency moving forward with the collection of SUD-1 and SUD-2 measures.

Thank you! Those are all the questions I have for you today.

[If the interview was not recorded, ask] Would it be okay to contact you after today with a follow up question or two, in case we missed something in our notes?