Attachment B17 - Pilot Testing Feedback

Attachment B17: Behavioral Health Consultation Provider Semi-Structured Interview Pilot Test Results

Health Resources and Services Administration (HRSA) Evaluation of the Maternal and Child Health Bureau (MCHB) Pediatric Mental Health Care Access (PMHCA) and Screening and Treatment for Maternal Mental Health and Substance Use Disorders (MMHSUD) Programs Project

Note: The instruments included in this OMB package have been revised and incorporate the pilot test results.

Feedback	Changes Made to Instrument, if Applicable
General Feedback
Time to complete: 1. If you can use plain language, you will get it done in 30 minutes. As it is currently worded interviewees may be slow to respond while they try to figure out what is being asked or have to ask for clarification. 2. Depending on the respondent, I think it will take about 40 minutes to conduct this interview. 3. 60 minutes 4. 45 minutes max; the Clinical Behavioral Health Consultation Process question is a critical one primarily because it lets you know how knowledgeable a provider is about their program, its processes and overall operation. 10 to 15 minutes is needed. While I think the Clinical Behavioral Health Consultation Requests portion of the interview will require only 3 to 4 minutes, the Health Equity section may require additional guidance from the interviewer.	Adjusted the interview time to 45 minutes.
How clear and easy to understand are the survey instructions and questions? 1. The audience for the “instructions to interviewers” isn’t clear despite the section head. From the interviewer’s perspective the guide isn’t an overview it is a script. Presumably JBS will have already worked with HRSA to identify the interviewee. There aren’t really any actual instructions in this section. The instructions to the interviewee start out very bureaucratic. Could it start with “You have been invited… because your program… is funded by HRSA to implement ….. JBS is also funded by HRSA but our role is to….. We are not part of HRSA or any other ….. You were selected to participate because of your level of involvement/experience providing bh [behavioral health] consultation through [program]. The purpose of the interview is to….” It might be worth defining not just PMHCA program (Is it really necessary to say “PMHCA program” throughout? Why can’t you just say “your program”) but also client and telehealth up front. The interview might flow better and then you don’t have to go through and swap out wording for the maternal version. Is it necessary to say “clinical behavioral health consultation” over and over? Could it not be shortened to “consultation” since you will have set up the context? To be extra sure you could say something reminding the interviewee that while they may have multiple roles in different programs to confine their answers to just program X. 2. Most are clear and easy to understand; I did have a couple suggestions. • In the interview introduction, consider removing the term “semi-structured interview” and simply replacing with “interview”. • Consider revising the guidance for the Clinical Behavioral Health Consultation process to the following: First, we would like to discuss your PMHCA program’s clinical behavioral health consultation processes, including the use of telehealth. For this interview, the term “client” refers to the children and/or adolescents in need of services and their families or caregivers. 3. Easy 4. Instructions are clear	1. We reviewed this section to streamline the text, including separating background information for the interviewers from the introductory remarks to interview participants. 2. Agree, suggest changing to interview. Added suggested language.
Were there any issues with the flow, formatting, layout, or appearance of the survey? If so, please explain. 1. I made a suggestion below about shifting the order of 9 and 10. 2. Question 4 is general barriers and question 3 is barriers to using telehealth. May flow better if the general barriers question is first. 3. I see no problem with the order of the questions.	1. Revised as recommended. 2. Revised as recommended.
Do you foresee any challenges or barriers to conducting an online (via Zoom or Microsoft Teams) SSI? If so, please explain. 1. No 2. There may be some loss of contextual cues or nonverbal communication, when conducting an online interview. Identification of cultural mannerisms that might necessitate a shift in response from the interviewer might also be more difficult to identify during online interactions. Potential connectivity challenges should always be considered and planned for accordingly. This may mean building in additional time. 3. I foresee no problem conducting the interview with Zoom or Microsoft Teams. What’s important is the interviewer can address provider’s questions.	2. We acknowledge the comment, but in the interest of being inclusive of a variety of regional areas, the interviews need to be conducted virtually. A call-in option will be available as well. This may require building in additional time so we extended the duration of the interview to 45 minutes.
Please provide any additional comments or suggestions to improve the SSI guide (e.g., unnecessary or missing questions). 1. Overall, the SSI guide flows well and most of the questions are clear. I have added comments in table 2 about those that may benefit from additional clarity. 2. I’ve included additional comments and suggestions in the Question-Specific Feedback below	Addressed below.
SSI Question Feedback
What is your job title with the PMHCA program?  PROBE: How long have you been involved in the PMHCA program?  PROBE: Besides you, what other types of program staff or behavioral health professionals provide behavioral health consultation services?  1. Doesn’t seem to be asking about what they actually do (how they are involved) maybe just say “We’d like to start by asking a little about you.” Second probe asks about types of health professionals, but it isn’t clear to me whether the job title of the interviewee will reveal the type of BH professional. Also not sure what is meant by “types of program staff.” Could you say “What are the job titles and professional disciplines of others who provide consultation for program…”	1. Clarified wording. Note: Consultation providers may not have assigned job titles or be employed for the program’s organization, so “job role” may be more applicable across individual programs.
Please describe the PMHCA program’s behavioral health consultation process. For example, how do requests for consultations come in, how are consultation requests triaged, and how long does it take to address consultation requests. PROBE: Do you provide behavioral health consultation directly with health professionals, clients, or both? 1. This is the second question, but the transition starts with “first.” Consider “Next I would like to ask…” Only one of the 2 is supposed to be doing the interview. It is not a discussion because the interviewer is not going to engage with the information. What does it mean to address a consultation? Is that the first response or when it is considered fully resolved. 2. The time it takes to address consultation requests most likely varies so that may be better worded as what is the average length of time for a consultation request. For the probe question, suggest saying “Do you provide behavioral health consultation directly with health professionals, directly with clients or both?”	1. Revised per first recommendation. Reworded/Clarified “address” language. 2. Reworded/clarified “address” language.
How are telehealth mechanisms being used for clinical behavioral health consultations? Telehealth includes the use of technology, such as telephone, email, text messaging, video conferencing, and screensharing, for provider-to-provider or provider-to-client communication. PROBE: Please describe any changes in how your program has used telehealth mechanisms for clinical behavioral health consultation. PROBE: What barriers do your program, health professionals, and/or clients experience related to using telehealth? SUB-PROBE: What strategies has your program implemented to reduce these barriers? 1. How is telehealth used for consultations? Please include what kinds of telehealth you use and any differences between provider to provider and provider to client communication. Question: is provider here the consultant? 2. This question, probes and sub-probe are relatively clear. The term “telehealth mechanisms” is a little confusing and seems redundant. I am unclear what is meant by “mechanisms” in this context. Does this mean technologies by which to deliver telehealth? Telehealth is a mechanism/modality of care. 3. For the first probe question, should state changes since when. The question could say “Please describe any changes, since participating in the PMHCA program, in how your program has used telehealth mechanisms….	1. Agree with changes; kept examples of telehealth technology in original question. 2. Revised. 3. Made change; started probe with “Since your involvement…”
What barriers or challenges have you and/or your program experienced to providing behavioral health consultation? PROBE: What approaches have you and/or your program used to mitigate or overcome these challenges? 1. Aside from any telehealth related issues, what barriers….Could the probe be “how has your program reduced or overcome these barriers?”	1. Revised question order, so will not make first change. Agree with second change and revised.
What factors have supported successful delivery of behavioral health consultation? 1. “What has been most important to successful delivery of …” For example…. 2. Specify what factors related to their program, have supported successful…..	1. Did not agree with recommendation. Revised question order. 2. Did not agree with recommendation. We want to know any factors (internal or external to the program).
What are the most common topics or requests for behavioral health consultation? PROBE: How have the topics or requests for clinical behavioral health consultation changed over time? PROBE: Have you noticed any trends regarding the types of consultation requests, for example, increases or decreases in the severity or complexity of requests? If yes, please describe. 1. Now that we’ve gone over the consultation process, I’m going to ask some questions about the nature of the consultations. What are the most common reasons for consultations and what is most often requested? Have you observed any changes in the consultations you have made? For example, the clinical concern, severity or needs of the providers or clients?	1. Added introduction. Did not change question but include examples.
How, if at all, has the frequency of clinical behavioral health consultation requests changed over the course of your PMHCA program? PROBE: Have you observed a change in the frequency of calls for repeat users of the consultation line? Please explain. 1. Frequency? From individual providers, frequency of contacts with clients? Volume? Not sure what the ask is here.	1. Revised wording.
How do you think your PMHCA program is addressing SDOH, including supporting access to behavioral health care? PROBE: Do you observe differences in behavioral health consultation requests based on SDOH or health disparities? Please explain. 1. Is it a goal or a focus? Is it the interviewee’s program or HRSA? Who is the “we” here? HRSA? Could the introductory bit be broken up or simplified a bit? “A goal of … is to achieve health equity in access to behavioral health care by addressing SDOH for racial and ethnic groups as well as people in rural and other underserved areas. 2. This question could be expanded to get more pointed and useful responses. The existing probe is useful, clear, and appropriate. Consider probing for any of the following: • Which strategies have been most effective in addressing SDOH in your PMHCA program, including supporting access to behavioral health care? • Have inequities decreased, increased, or remained the same? • Describe your PMHCA program’s current strategy, activities, and goals to achieve health equity related to the SDOH. • Which barriers has your PMHCA program faced in promoting health equity and addressing health disparities related to the SDOH? 3. I suggest providing clear and definitions of SDOH and health equity. Of all the topics being addressed, health equity is one where some providers may not know the definition and/or be aware of the relationship to SDOH. I also suggest separating this question into two questions: 1) "How do you think your PMHCA program is addressing SDOH to achieve health equity?" 2) How do you think your PMHCA program supports accessing behavioral health care by addressing SDOH?	1. Reworded introduction. 2. Did not add probes as they are not within the interviewee’s purview to answer them. Revised current probe. 3. Added SDOH and health equity language. Did not separate question but revised language.
To what extent do you think providing behavioral health consultation has affected health professionals’ capacity to address behavioral health? 1. Now I’d like to hear from you about how your program is promoting behavioral health integration into pediatric primary care. I think question 9 might be better after question 10. Establish the level of receptivity before asking about effect? 2. This question is unclear. Is this question asking about addressing behavioral health needs? Improving behavioral health?	1. Revised question order. Added introduction. 2. Revised.
Based on your interactions with pediatric primary care health professionals, how receptive are they to providing behavioral health care services?  PROBE: To what extent do you think the behavioral health consultation line has affected their willingness to provide behavioral health services? 1. Whichever question comes second, If you specify “pediatric primary care professionals” in the preceding question, is it really necessary to repeat it all? “How receptive do you think they are to providing….”	1. Removed “pediatric”.
Do you have any additional questions, comments, or feedback at this time? 1. Consider “additional comments, feedback, or questions?” They’ve definitely been giving comments and feedback so anything they say here will be additional but we don’t know if they will have asked any questions.	1. Changed the order per the suggestion.