Form SSA KII Administra SSA KII Administra SSA KII Administrator Version

Substance Abuse Mental Health Services Administration

SSA KII—Administrator Version

OMB No: XXXXX

Expiration Date: XXXX

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB control number for this project is XXXX-XXXX.  Public reporting burden for this collection of information is estimated to average 60 minutes per respondent, per year, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 2-1057, Rockville, Maryland, 20857.

Community Support Evaluation: Supported Employment

Scalability/sustainability assessment key informant interview

Administrator Version (year 2)

Introduction

Thank you for agreeing to participate. We are going to begin by talking a bit about you and your agency. After that, I’d like to talk about the outcomes of the Supported Employment project, and its infrastructure. We are also going to talk about both the scalability and the sustainability of the SE project. While I have separate questions to ask about scale-up and sustainability, I understand that these topics are closely related and you may need to talk about them simultaneously in some cases.

1. To start off, I’d like to know a bit about you. How would you describe your role within your state/local mental health system?

   1. How long have you worked in this role?

   2. What are your key responsibilities overall?

   3. What are your key responsibilities related to the SE project?

   4. How has your role changed since implementing the SE project?

2. What kind of employment programming, if any, did your state/agency offer prior to receiving this grant?

   1. In what ways, if any, was the previous programming similar to the principles of the Individual Placement and Support (IPS) Supported Employment (SE) model?

      1. Eligibility by consumer choice/non-exclusivity

      2. Employment services coordinated with mental health treatment

      3. Competitive employment focused

      4. Provision of personalized benefits counseling

      5. Rapid commencement of job search

      6. Unlimited follow-along support

      7. Emphasis on consumer preference

3. How would you describe the state of the workforce and job availability prior to receiving the grant?

4. How did your state decide to apply for this grant? [Skip question for Agency Director interview.]

   1. What factors motivated the state to pursue this grant?

   2. What were you hoping to accomplish by implementing SE in your state?

Outcomes

Now I’d like to talk about the outcomes of the program.

5. What changes in outcomes have you seen since implementing the SE project, if any?

   1. [Insert list of intended outcomes from grantee’s proposal to be used as probes if necessary]

   2. What outcomes have changed? How have they changed?

   3. How has the SAMHSA SE project affected these outcomes?

   4. What other factors may have played a role in this change?

6. How successful have the state’s efforts to develop a statewide SE workforce been?

   1. What evidence do you have/see for this?

   2. What percentage of communities/providers have you reached so far?

   3. What activities under this project have had the most impact on the SE workforce?

   4. What improvements would you like to make in the future?

7. How successful have the state’s efforts to increase collaboration been?

   1. What evidence do you have/see for this?

   2. What activities have had the most impact on collaboration?

   3. What improvements would you like to make in the future?

Infrastructure, Planning, and Development

Let’s talk a bit more about infrastructure and planning.

8. What staffing changes have been made in relation to the SE project at the state/agency level? [Skip question for SECC Member interview.]

   1. How many staff manage/oversee the SE project for your state/agency?

   2. Have you hired any staff members specifically to manage the SE project? If so, please describe.

   3. What expertise do these staff members have related to SE?

   4. What roles does each staff member fill related to SE?

   5. Do these staff members also have roles unrelated to SE? If so, approximately what percentage of their time is dedicated to SE?

9. Please tell me about the Supported Employment Coordinating Committee. [Skip probes a, b, c, and f for Agency Director interview.]

   1. How were the members of the committee selected?

   2. What is your role on/involvement with the committee?

   3. How do key stakeholders communicate for collaboration?

   4. What barriers, if any, have you encountered in collaboration?

   5. What facilitators have enabled collaboration?

   6. How is the committee involved in the following activities:

      1. Policy development supporting the SE project

      2. Identifying funding sources

      3. Identifying employers to participate in SE efforts

      4. Consulting on performance standards and quality assurance

      5. Engaging in workforce development

10. Other than the SECC, who are the key stakeholders for the SE project in your state?

    1. What are their roles?

    2. In what ways do they support SE management and/or implementation?

11. What existing funding sources have been leveraged to support SE in your state at the state, organizational, and individual level?

    1. In what ways, if any, has the SE project made it easier for employment services to be funded?

    2. What challenges, if any, has the SE project created for funding employment services?

    3. Has SE in the state lost any outside funding throughout the time it has been implemented? If so, why did those funders stop supporting the project?

    4. What efforts are you implementing to secure funding after the grant?

12. How are resources allocated for the SE project?

    1. What stakeholders are involved in decisions about resource allocation?

Scalability

Moving beyond infrastructure, let’s talk a bit about scalability, or the capacity of the SE project to increase in size and reach.

13. How scalable is the state’s current approach to SE?

    1. What factors currently affect the scalability of SE?

    2. How do you expect the scalability of the project to change over the next two years?

14. How has the scale of SE in your state changed since receiving the grant? [Skip probes a and b for SECC Member interview; Skip probe c for Project Director interview.]

    1. Beyond the two community implementation sites funded by the SAMHSA project, has there been an increase in the number of communities/counties with SE programs?

    2. Has there been an increase in the number of clients served? If so, please describe.

    3. What role has your agency/organization played in scaling up SE?

    4. What facilitators have improved the scalability of SE?

    5. What barriers have inhibited the scalability of SE?

15. How have agency operations changed at the state level since engaging in the SE project? [Skip question for SECC Member interview.}

    1. In what ways have agency operations changed?

    2. In what ways have agency operations remained the same?

16. What policy changes does your state/agency anticipate for the future?

    1. How might these changes affect the SE program?

    2. What role, if any, does your agency/organization intend to play with regard to this change in policy? In implementation of the policy?

Sustainability

For the final section of our interview, I’d like to ask you a few questions about sustaining the SE program going forward.

17. How sustainable is the state’s current approach to SE?

    1. What factors currently affect the sustainability of SE?

    2. How do you expect the sustainability of SE to change over the next two years?

    3. What changes do you anticipate to the sustainability of SE once the grant has concluded?

18. What efforts, if any, are currently underway to increase the sustainability of SE?

    1. Is your agency/organization making changes to any of the following factors to increase SE sustainability?

       1. Funding

       2. Policies

       3. Workforce development

       4. Monitoring and Evaluation

    2. What challenges are you currently facing in these activities?

19. What additional activities, if any, are planned to increase the sustainability of SE?

    1. Does your agency/organization have plans in place to make changes to any of the following factors to increase SE sustainability?

       1. Funding

       2. Policies

       3. Workforce development

       4. Monitoring and Evaluation

    2. What factors would enable you to engage in these activities?

    3. What challenges might prevent you from engaging in these activities?

20. Please tell me about how funding sources may affect the sustainability of SE.

    1. Does your agency/organization have plans to devote current funding sources to future implementation of SE? If so, please describe.

    2. Does your agency/organization have plans to secure additional funding for future implementation of SE? If so, please describe

21. What additional infrastructure components impact sustainability of SE in your state?

    1. Would sustainability be increased by support in any of the following areas? How?

       1. Funding

       2. Policies

       3. Workforce development

       4. Monitoring and Evaluation

    2. Do you expect to receive this kind of support? Why or why not?

Conclusion

As we conclude, I have a few final questions.

22. If you could change anything about your SE program since receiving this grant, what would you change?

    1. Would these changes affect the scalability of SE? How?

    2. Would these changes affect the sustainability of SE? How?

23. [Time permitting] What advice would you give to a new state that is just starting to scale up SE?

24. Do you have anything else you would like to add about the SE project that we have not covered?

Thank you for taking the time to speak with me today. Please do not hesitate to contact me if you have further insights you would like to add or questions about this study.

OMB No: XXXXX

Expiration Date: XXXX

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB control number for this project is XXXX-XXXX.  Public reporting burden for this collection of information is estimated to average 60 minutes per respondent, per year, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 2-1057, Rockville, Maryland, 20857.

Community Support Evaluation: Supported Employment

Scalability/sustainability assessment key informant interview

Administrator Version (Year 4)

Introduction

Thank you for agreeing to participate. We are going to begin by talking a bit about you and your agency. After that, I’d like to talk about the outcomes of the Supported Employment project, and its infrastructure. We are also going to talk about both the scalability and the sustainability of the SE project. While I have separate questions to ask about scale-up and sustainability, I understand that these topics are closely related and you may need to talk about them simultaneously in some cases.

1. To start off, I’d like to know a bit about you. How would you describe your role within your state/local mental health system?

   1. How long have you worked in this role?

   2. What are your key responsibilities overall?

   3. What are your key responsibilities related to the SE project?

   4. How has your role changed over the final two years of the SE grant?

2. Does the state plan to continue the SE project now that the grant has concluded?

   1. Why/why not?

Outcomes

Now I’d like to talk about the outcomes of the program.

3. What changes in outcomes have you seen since implementing the SE project, if any?

   1. [Insert list of intended outcomes from grantee’s proposal to be used as probes if necessary]

   2. What outcomes have changed? How have they changed?

   3. How has the SAMHSA SE project affected these outcomes?

   4. What other factors may have played a role in this change?

4. How successful have the state’s efforts to develop a statewide SE workforce been?

   1. What evidence do you have/see for this?

   2. What percentage of communities/providers have you reached so far?

   3. What activities under this project have had the most impact on the SE workforce?

   4. What improvements would you like to make in the future?

   5. [Insert relevant probes based on data from Year 2 as needed]

5. How successful have the state’s efforts to increase collaboration been?

   1. What evidence do you have/see for this?

   2. What activities have had the most impact on collaboration?

   3. What improvements would you like to make in the future?

   4. [Insert relevant probes based on data from Year 2 as needed]

Infrastructure, Planning, and Development

Let’s talk a bit more about infrastructure and planning.

6. What staffing changes have been made in relation to the SE project at the state/agency level? [Skip question for SECC Member interview.]

   1. How many staff manage/oversee the SE project for your state/agency?

   2. Have you hired any staff members specifically to manage the SE project? If so, please describe.

   3. What expertise do these staff members have related to SE?

   4. What roles does each staff member fill related to SE?

   5. Do these staff members also have roles unrelated to SE? If so, approximately what percentage of their time is dedicated to SE?

7. Please tell me about the Supported Employment Coordinating Committee. [Skip probes a, b, and e for Agency Director interview.]

   1. What is your role on/involvement with the committee?

   2. How do key stakeholders communicate for collaboration?

   3. What barriers, if any, have you encountered in collaboration?

   4. What facilitators have enabled collaboration?

   5. How is the committee involved in the following activities:

      1. Policy development supporting the SE project

      2. Identifying funding sources

      3. Identifying employers to participate in SE efforts

      4. Consulting on performance standards and quality assurance

      5. Engaging in workforce development

8. Other than the SECC, who are the key stakeholders for the SE project in your state?

   1. What are their roles?

   2. In what ways do they support SE management and/or implementation?

9. What existing funding sources have been leveraged to support SE in your state at the state, organizational, and individual level?

   1. In what ways, if any, has the SE project made it easier for employment services to be funded?

   2. What challenges, if any, has the SE project created for funding employment services?

   3. Has SE in the state lost any outside funding throughout the time it has been implemented? If so, why did those funders stop supporting the project?

   4. What efforts are you implementing to secure funding after the grant?

10. How are resources allocated for the SE project?

    1. What stakeholders are involved in decisions about resource allocation?

Scalability

Moving beyond infrastructure, let’s talk a bit about scalability, or the capacity of the SE project to increase in size and reach.

11. How scalable is the state’s current approach to SE?

    1. What factors currently affect the scalability of SE?

    2. What changes do you anticipate to the scalability of SE now that the grant has concluded?

    3. [Insert relevant probes based on data from Year 2 as needed]

12. Do you intend to secure funding to scale this program up across the state? [Skip question for Agency Director and SECC Member interview.]

    1. Why/why not?

    2. What factors affect these plans?

13. How has the scale of SE in your state changed since receiving the grant? [Skip probes a and b for SECC Member interview; skip probe c in Project Director interview.]

    1. Beyond the two community implementation sites funded by the SAMHSA project, has there been an increase in the number of communities/counties with SE programs?

    2. Has there been an increase in the number of clients served? If so, please describe.

    3. What role has your agency/organization played in scaling up SE?

    4. What facilitators have improved the scalability of SE?

    5. What barriers have inhibited the scalability of SE?

14. How have agency operations changed at the state level since engaging in the SE project? [Skip question for SECC Member interview.]

    1. In what ways have agency operations changed?

    2. In what ways have agency operations remained the same?

15. Broadly speaking, how has receiving this grant changed local programming? [Skip question for SECC Member interview.]

    1. How, if at all, have the changes looked different across the implementation sites?

    2. How, if at all, has the SE project changed local programming at any other local community behavioral health agencies besides the implementation sites?

16. What state/agency policies have changed as a result of the SE project (both legislative and non-legislative)?

    1. How have these policies changed?

    2. How have these policy changes impacted the implementation and scale-up of SE?

    3. How will these policy changes affect the sustainability of SE?

    4. Are there any policies you hoped would change that have remained the same? If so, please describe.

17. What policy changes does your state/agency anticipate for the future?

    1. How might these changes affect the SE program?

    2. What role, if any, does your agency/organization intend to play with regard to this change in policy? In implementation of the policy?

Sustainability

For the final section of our interview, I’d like to ask you a few questions about sustaining the SE program going forward.

18. How sustainable is the state’s current approach to SE?

    1. What factors currently affect the sustainability of SE?

    2. What changes do you anticipate to the sustainability of SE once the grant has concluded?

    3. [Insert relevant probes based on data from Year 2 as needed]

19. What efforts, if any, are currently underway to increase the sustainability of SE?

    1. Is your agency/organization making changes to any of the following factors to increase SE sustainability?

       1. Funding

       2. Policies

       3. Workforce development

       4. Monitoring and Evaluation

    2. What challenges are you currently facing in these activities?

20. What additional activities, if any, are planned to increase the sustainability of SE?

    1. Does your agency/organization have plans in place to make changes to any of the following factors to increase SE sustainability?

       1. Funding

       2. Policies

       3. Workforce development

       4. Monitoring and Evaluation

    2. What factors would enable you to engage in these activities?

    3. What challenges might prevent you from engaging in these activities?

21. Please tell me about how funding sources may affect the sustainability of SE.

    1. Does your agency/organization have plans to devote current funding sources to future implementation of SE? If so, please describe.

    2. Does your agency/organization have plans to secure additional funding for future implementation of SE? If so, please describe

22. What additional infrastructure components impact sustainability of SE in your state?

    1. Would sustainability be increased by support in any of the following areas? How?

       1. Funding

       2. Policies

       3. Workforce development

       4. Monitoring and Evaluation

    2. Do you expect to receive this kind of support? Why or why not?

Conclusion

As we conclude, I have a few final questions.

23. [If continuing SE] In what ways will SE in the future look different from what was implemented under the grant?

    1. Will the scale of SE change? If so, how?

    2. Will the changes in the future affect the sustainability of SE? If so, how?

24. If you could change anything about your SE program since receiving this grant, what would you change?

    1. Would these changes affect the scalability of SE? How?

    2. Would these changes affect the sustainability of SE? How?

25. [Time permitting] What advice would you give to a new state that is just starting to scale up SE?

26. Do you have anything else you would like to add about the SE project that we have not covered?

Thank you for taking the time to speak with me today. Please do not hesitate to contact me if you have further insights you would like to add or questions about this study.

14