Form Case Study Protoco Case Study Protoco Case Study Protocol/Focus Groups

OMB No. 0930-0197

Expiration Date: 1/31/17

Appendix B

Case Study Protocol for TA Network CQI Process

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB control number for this project is 0930-0197.  Public reporting burden for this collection of information is estimated to average 90 minutes per respondent, per year including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 2-1057, Rockville, Maryland, 20857.

This interview protocol will initially be used on a pilot basis with two grantees, and after the pilot is completed and appropriate modifications are made, it will be used with nine grantees per year. The grantees in the sample will include three from each of these three categories: local implementation grant; planning grant for statewide expansion; or implementation grant for statewide implementation. Case studies will be initiated at the rate of one per month beginning in January of each year. Grant communities will be selected at random within each of these categories (Note: If there is a particular reason to select more sites from a particular category of grantees, this can certainly be done).

For each of the grantees participating in the Case Study, three to five informants will be identified to be interviewed individually on the phone. The informants will include the Project Director, the TA coordinator, and at least one other person centrally involved in receiving TA or implementing the grantee’s system change strategy.

Prior to the beginning of the interviews, the TA reports for the sample sites will be reviewed. This will include the TARS and satisfaction and impact surveys. This will provide the interviewer with important context and background information.

The findings from the interviews and from the reviews of the surveys will be summarized by main themes by two reviewers. The main themes, as shown below, are in the form of questions:

• What was the decision-making process about whether or not to request TA assistance from the Network, and what were the rationales for decisions that were made?

• What were the purposes of the TA and how well were they achieved?

• What preparation was done for the TA that was provided, and how useful was the preparation?

• How closely did the TA that was provided fit the values, goals and strategies of the recipients?

• If multiple TA providers were used, how consistent were they in the information they provided and the methods they used?

• How consistent was the TA with the values, goals, logic model, and theory of change of the TA Network?

• What is the likely impact of the TA both in meeting a short-term need, and in enhancing the overall capacity of the recipient?

• What lessons can be learned from the experience of providing TA to this recipient either about improving the TA process or strengthening system change efforts?

Reviewers will select specific statements from the interviews that best address each of these themes/questions. They will also rate, on a seven point scale, the following:

• Overall degree of fit of the TA with the recipients values, goals and strategies;

• Overall degree of consistency of the TA with the Network’s values, goals, logic model, and theory of change;

• Overall quality of the TA experience and satisfaction with it, as reflected by the recipient;

• Overall quality of the TA experience, as judged by the reviewers;

• Magnitude of positive change likely to occur as a result of the TA.

The two reviewers will discuss their ratings, and reach agreement on a score for each of these five items.

The narrative information gathered from the interviews and the ratings provided by the reviewers will be shared with the CQI team, which will examine them particularly to identify any lessons to be learned. The aggregated findings will also be shared with the Network’s leadership team to support continuous quality improvement.

Semi-Structured Interview

Date:

Name of Interviewer:

Name of Interviewee:

Role of Interviewee:

Name of SOC Grant:

State:

Year Funded:

Initial Individual Interview Questions

1. What is the goal and population of focus for your grant/cooperative agreement?

2. What are the major strategies that you are using to try to achieve that goal?

3. How often do you call on assistance from individuals/groups outside your grant community to help you implement your strategies and achieve your goal?

4. What is the process of deciding whether to call on outside help?

5. What have been the purposes of outside help that you have sought? (select as many as apply):

   1. Provide information on an issue/topic;

   2. Provide training to enhance capacity of community;

   3. Provide support/credibility for a decision already made, or about to be made;

   4. Help in planning/problem solving;

   5. Serve as independent/neutral voice at time of conflict;

   6. Other

6. If you decide that you would benefit from outside help, how do you decide who to turn to for that help? What role do the following factors play in your decision on a seven point scale ranging from no role at all to highly important:

   1. Prior relationship with possible TA provider

   2. Word of mouth reputation about responsiveness and helpfulness

   3. Ease of access

   4. Cost

   5. Credibility

   6. Perceived expectations/requirements/preferences of funding agency (CMHS)

   7. Other factors

7. Have you used the technical assistance services of the TA Network? If so, which ones have you used?

   1. On-going individual consultation on the phone;

   2. On-going individual consultation on site;

   3. One time phone consultation;

   4. One time on site consultation;

   5. Retrieval of information on a specific issue/topic;

   6. On-site training;

   7. Webinar;

   8. Rapid response;

   9. Other

8. How many individuals from the TA Network or its group of consultants have been involved with your grant?

9. Some consultants offer general expertise on systems of care and others offer specific expertise on particular topics, such as financing, cultural and linguistic competence, family involvement, or CQI. Were the consultants that you used generalists or specialists, or both?

10. How well have the consultants understood your goals and your strategies for achieving your goals?

11. Have the services provided by the consultants been consistent with your goals and strategies?

12. Have the services been data-based and field-driven?

13. If you have used more than one consultant, have they been consistent in their input?

14. Has their input helped you achieve your goals and implement your strategies? If so, in what way? If not, what were the reasons for failing to do so? In general, what factors affected the effectiveness of the service—for example:

    1. Degree of preparation for the TA by the providers/trainers

    2. Degree of preparation for the TA by you/your team/your community

    3. Level of expertise of the TA providers

    4. Readiness of the community to receive outside input

    5. Credibility of the TA providers

    6. Complexity/difficulty of the issue(s) addressed in the TA

    7. Other

15. Do you have anything else you would like to add about the services that you have been provided by the TA network, and their helpfulness?

4