Form Clinical Superviso Clinical Superviso Clinical Supervisors Survey

OMB No. 0930-xxxx

Expiration Date: xx/xx/xxxx

Assertive Adolescent &

Family Treatment (AAFT) Program

AAFT Implementation Survey

Clinical Supervisors

&

Clinicians

Public reporting burden for this collection of information is estimated to average 45 minutes per response per year, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Thank you for agreeing to participate in the

Program Evaluation for Assertive Adolescent & Family Treatment (AAFT) Program!

This survey is being conducted by Advocates for Human Potential, Inc. (AHP). We are a research and consulting firm based in Sudbury, MA and Albany, NY. We’re conducting this study as part of our contract to assist the Center for Substance Abuse Treatment (CSAT) in the national evaluation of the implementation of AAFT, funded by the Substance Abuse and Mental Health Services Administration (SAMHSA).

At the end of each project year, we are asking program administrators, clinical, and research staff across the entire AAFT3 grantee cohort to provide information about their professional background as well as opinions/thoughts on a variety of topics, including substance abuse treatment. We are gathering data from many sources and believe it is important to collect information from many perspectives as well. As you complete this survey, PLEASE RESPOND TO THE QUESTIONS FROM YOUR OWN PERSPECTIVE—choosing an answer that best describes your experience or opinion.

Your responses to the survey will be kept in a private record. Your responses will come directly to AHP staff. Any reports generated as part of this evaluation will contain only aggregate responses (for example, “50% of the program staff had been working for this program for less than two years”).

If you have any questions, concerns or comments about the questionnaire or the study, please feel free to contact us by phone or email:

Dr. Terri Tobin Denise Lang

Evaluation Director Research Associate

1-800-795-5486 x418 401-323-9678

[email protected] [email protected]

Thank you again for your participation!

Section A

Respondent ID: Please provide your First, Middle, & Last initials, and the Month & Day of your birthday. |               | (For example: ALA0415|)

What is your primary role in the project (select one):

Principal Investigator

Clinician/Counselor

Other, please specify      

Project Director

Researcher/Evaluator

Clinical Supervisor

Data Manager

What is the highest degree that you have obtained?

No high school diploma or equivalent

Bachelor’s degree

High School Diploma or Equivalent

Master’s degree

Some college, but no degree

Doctoral degree or equivalent

Associate’s degree

Other, please specify      

What is your professional background? (check all that apply)

Addictions Counseling

Social Work/Human Services

Nurse Practitioner

Other Counseling

Physician Assistant

Administration

Education

Medicine: Primary Care

None, unemployed

Vocational Rehabilitation

Medicine: Psychiatry

None, student

Criminal Justice

Medicine: Other

Other, please describe      

Psychology

Nurse

Which licenses, credentials, or certificates do you currently hold? (check all that apply)

ACDP/ACPS

LMHC

MD/DO

None

LCDP/LCDCS

LSW/LCSW/LICSW

LMFT

Other, please describe      

How many years of experience do you have in the substance use treatment field?      

Do you have experience providing substance use treatment services to:

Adolescents (ages 12-17)

YES, # years      

NO

Transition Age Youth (TAY; ages 18-24)

YES, # years      

NO

Families

YES, # years      

NO

Please list on what date you began working at your current agency (MONTH/YEAR):      

Please list on what date you began working on this AAFT project (MONTH/YEAR):      

Grantee experience
This grant may not be your first experience working with A-CRA/ACC, Chestnut Health Systems or research/evaluation. The following questions ask about your experiences with A-CRA/ACC, Chestnut Health Systems, data collection and research/evaluation projects.
Prior to this project, did you participate in a CSAT-funded adolescent project?		Yes				No (If NO, skip to next question)
IF YES, please indicate each type(s).	AAFT	EAT		YORP	TCE/HIV		OJJDP	SCY	Other      	DK
Please indicate what your level of A-CRA/ACC/GAIN certification was prior to working on this AAFT project. [check all that apply]:
I was a certified GAIN local trainer.			I was a certified GAIN administrator.
I was an A-CRA-certified clinical supervisor.			I was an A-CRA-certified clinician.
I was an ACC-certified clinical supervisor.			I was an ACC-certified clinician.
None of the above

Section B: Please indicate…

1. To what extent have each of these components become part of the program’s normal, day-to-day routine?

2. How well do each of these components work within your program (e.g., how well do they “fit” with your program)?

	To what extent have each of these components become part of the program’s normal, day-to-day routine?				How well do each of these components work within your program (e.g., how well do they “fit” with your program)?
	Great change	Moderate change	Slight change	No change	Fits extremely well	Fits moderately	Fits a little	Not a fit
A-CRA approach
ACC approach
GAIN assessment at Intake
Treatment Satisfaction Index (TxSI)
EBTx website
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
Follow-up interviews (3, 6, 12 months)
Data management activities (e.g., monthly data submission to CHS, responding to GAIN Edits)
GAIN certification
GAIN Clinical Interpretation Certification (GCIC)
ACRA/ACC certification
Monitoring & compliance reports (e.g., follow-up rates, enrollment rates, DSRs, 13+week reports)

Adolescent/TAY reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
Family reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
Clinician reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
Use of EBTx website
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
The use of monitoring & compliance reports (e.g., follow-up rates, enrollment rates, DSRs, 13+week reports)
Project leadership (e.g., project director, supervisor) reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
Use of EBTx website
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
The use of monitoring & compliance reports (e.g., follow-up rates, enrollment rates, DSRs, 13+week reports)

Section C: Based on your experience, please rate the reactions to each of the components of this project. If you feel that the question does not apply to you or you do not know the answer, please mark “N/A” or “DK” (respectively).

Section D: The following questions ask about your experiences with the support provided for the AAFT grant from Chestnut Health Systems, CSAT and other outside sources. For each item, please indicate how helpful each support has been. Lastly, please add any comments or recommendations for improvement for each.

Have you had involvement with A-CRA/ACC, EBTx, and/or DSRs?			YES		NO		IF “NO”—SKIP questions below, continue on next page
How helpful has it been in helping you implement the AAFT Program?	Extremely	Moderately		Slightly		Not at all	Not enough participation to rate this item	Any comments or recommendations for improvement?
A-CRA/ACC Training								     
A-CRA/ACC certification								     
A-CRA/ACC coaching calls								     
A-CRA/ACC Training manual (e.g., refer back to procedures, certification process)								     
A-CRA/ACC Materials (e.g., Happiness Scale, Functional Analysis worksheet)								     
EBTx website & DSRs								     
EBTx Support team staff								     
Program reports (e.g., 13+weeks report)								     
A-CRA/ACC team staff								     

Have you had involvement with the GAIN, GAIN data, and/or ABS?			YES		NO		IF “NO”—SKIP questions below, continue on the next table
How helpful has it been in helping you implement the AAFT Program?	Extremely	Moderately		Slightly		Not at all	Not enough participation to rate this item	Any comments or recommendations for improvement?
GAIN Training								     
GAIN certification								     
GAIN coaching calls								     
GAIN Training manual (e.g., intent of questions, certification process, etc.)								     
GAIN Clinical Interpretation Training								     
GAIN Clinical Interpretation Certification Individual-level reports (e.g., GRRS, PFR)								     
ABS web-based system								     
Reports (e.g. Site Profiles Report)								     
GAIN data (e.g., analytical files)								     
GAIN Support team staff								     

How helpful has it been in helping you implement the AAFT Program?	Extremely	Moderately	Slightly	Not at all	Not enough participation to rate this item	Any comments or recommendations for improvement?
Implementation calls/site visits to monitor/address progress at grantee sites						     
Individualized coaching calls to address areas for improvement (e.g., low recruitment/follow-up rates, DSR uploads, increase TxSI completion, etc.)						     
List Serve Information						     
Technical assistance available through NIATx						     
Special topic calls (e.g., Cultural Responsiveness)						     
Monthly calls with CSAT project officer/Chestnut						     
Interactions with other grantees at meetings/ calls						     
Initial/Annual Grantee Meetings						     

Section E: To meet the needs of your target population, you may have felt the need to modify or adapt the A-CRA/ACC treatment model. Please tell us about any changes your program has made to the A-CRA/ACC model for this project.

Has your program made any modification/adaptations to the A-CRA/ACC treatment model?
YES (If YES, proceed to next question below)			NO (If NO, skip to next page)
Has well have these modifications worked for your program?
Extremely well	Quite well	Not too well		Not well at all

Section F: Based on your experience, please indicate if the issues listed below have been barriers to implementation at your site. Then, describe the strategies you have used to overcome the MOST CHALLENGING BARRIER at your site.

POSSIBLE BARRIERS	Was this a barrier to implementation at your site?
Program/Organizational Issues	YES	NO
Internal communication (e.g., program staff)
External communication (e.g., CSAT, Chestnut Health Systems)
Staff attitudes (e.g., morale, enthusiasm, resistance)
Leadership/management attitudes
Recruiting clinical staff
Recruiting supervisory staff
Turnover; significant loss of staff
Budget issues
Service Delivery	YES	NO
Enrolling clients
Client engagement
Family engagement
Client retention
Grant-related Activities	YES	NO
A-CRA/ACC training
GAIN training
Collecting GPRA data
Collecting GAIN data
Collecting follow-up data
Using ABS
Using the SAIS system for GPRA
Recording treatment sessions (DSRs) & using EBTx website
Other Grant requirements (e.g., certification, coaching calls, data management)
Research/Evaluation Activities	YES	NO
Working with the local evaluator
Use of data, monitoring, compliance reports (e.g., Site Profile Data, follow-up/enrollment rates, 13+week reports)
Any other barriers not included above (specify):     
Please describe the strategies you have used to overcome the MOST CHALLENGING BARRIER at your site.      

Section G: Staff turnover is common in this field. Please tell us about any staff turnover and effects it may have had or is having on your program.

During the last project year, did the project hire new…			If YES, how would you rate the overall effect of this change on the grant program?
Program Manager(s)	YES	NO	Positive	Neutral	Negative	Don’t know
Clinical Supervisor(s)	YES	NO	Positive	Neutral	Negative	Don’t know
Clinician(s)/direct care staff	YES	NO	Positive	Neutral	Negative	Don’t know
Evaluator(s)	YES	NO	Positive	Neutral	Negative	Don’t know
Data manager(s)	YES	NO	Positive	Neutral	Negative	Don’t know

Section H: To better understand barriers adolescents and families may have experienced when starting treatment prior to the AAFT grant, please tell us about the barriers adolescents and family may have encountered prior to starting the AAFT grant and after implementation of the AAFT grant.

	Barrier for adolescents/families before the grant?				Barrier for adolescents/families now?
	An extreme barrier	Very much a barrier	A slight barrier	Not a barrier at all	An extreme barrier	Very much a barrier	A slight barrier	Not a barrier at all
Transportation
Child care
Need for mental health treatment
Neighborhood safety (e.g., home visits, gang turf)
Insurance/cost of treatment
Family issues (e.g., lack of family involvement)
Language/cultural issues
Treatment resistance (e.g., low readiness for change, not understanding substance abuse disorders/treatment, stigma, shame)
Other (specify):      

Section I: We would like to know about your program’s needs.

Your program needs additional guidance in…	Agree Strongly	Agree	Uncertain	Disagree	Disagree Strongly
Assessing client needs.
Matching needs with services.
Increasing program participation by clients.
Measuring client performance.
Developing more effective group sessions.
Raising overall quality of counseling.
Using client assessments to guide clinical and program decisions.
Using client assessments to document program effectiveness.

Section J: The next section asks about staffing and staff time.

	Agree Strongly	Agree	Uncertain	Disagree	Disagree Strongly
There are enough counselors here to meet current client needs.
A larger support staff is needed to help meet program needs.
Frequent staff turnover is a problem for this program.
Counselors here are able to spend enough time with clients.
Support staff here have the skills they need to do their jobs.
Clinical staff here are well-trained.
You are under too many pressures to do your job effectively.
Staff members often show signs of stress and strain.
The heavy workload here reduces program effectiveness.
Staff frustration is common here.

Section K: We would like to know more about your time management and flexibility.

	Agree Strongly	Agree	Uncertain	Disagree	Disagree Strongly
You consistently plan ahead and carry out your plans.
You usually accomplish whatever you set your mind on.
You are effective and confident in doing your job.
You have the skills needed to conduct effective individual counseling.
You are willing to try new ideas even if some staff members are reluctant.
Learning and using new procedures are easy for you.
You are sometimes too cautious or slow to make changes.
You are able to adapt quickly when you have to shift focus.
You consistently plan ahead and carry out your plans.

Section L: Communication is an important part of any organization. Please tell us about how information and ideas are communicated within your organization.

	Agree Strongly	Agree	Uncertain	Disagree	Disagree Strongly
Ideas and suggestions from staff get fair consideration by program management.
The formal and informal communication channels here work very well.
Program staff are always kept well informed.
More open discussions about program issues are needed here.
Staff members always feel free to ask questions and express concerns in this program.

Section M: Please rate how much you agree or disagree with the following statements.

	Agree Strongly	Agree	Uncertain	Disagree	Disagree Strongly
Novel treatment ideas by staff are discouraged.
You can change procedures here quickly to meet new conditions.
You frequently hear good staff ideas for improving treatment.
The general attitude here is to use new and changing technology.
You encourage counselors to try new and different techniques.
Manuals make therapists more like technicians than caring human beings.
Using a treatment manual makes a therapist think more about sticking to the manual than the needs of the individual client.
Treatment manuals are appropriate for research clients but not “real world” clients.
Using a treatment manual keeps therapists from using his or her intuition in responding to a client.
Following a treatment manual will enhance therapeutic outcomes by insuring that the treatment being used is supported by research.
Treatment manuals, if used appropriately, will enhance the average outcomes of clients treated in psychotherapy.
Treatment manuals can help keep therapists on track during therapy.

Section N: These next questions ask about your opinions regarding substance abuse treatment.

	Agree Strongly	Agree	Uncertain	Disagree	Disagree Strongly
Addiction is really a disease.
12-step programs should be used more in substance abuse treatment.
Confrontational approaches should be used more in substance abuse treatment.
Recovering counselors make the best therapists in substance abuse treatment.
Substance abuse treatment services should routinely include the patient’s family members.
Engaging family members in treatment has a negative impact on the youth’s outcomes.
Family-based treatment is effective with substance-abusing youth.

Section O: Please read each statement carefully and indicate how much you agree with each of the following statements related to your grantee site’s experience with research and evaluation activities.

	Agree Strongly	Agree	Uncertain	Disagree	Disagree Strongly
Policies and procedures are in place to ensure integrity of data collected.
Staff capacity is adequate to meet the demands of data collection efforts.
Our program regularly conducts surveys with consumers to identify program strengths and weaknesses.
Program staff use data to evaluate program services and consider opportunities for improvement.
Our agency relies on data to set goals and measure success.
Data collected is analyzed and used to evaluate/enhance program implementation.
Management shares data with staff about program effectiveness in meeting the needs of clients.
Data are provided to/discussed with staff.

P

Section P. Sustainability: Please tell us about any plans your program may have to sustain this program after CSAT funding has ended.
Are you currently engaged in any activities aimed toward sustainability of your program?	YES (if YES, proceed to next question)	NO (if NO, skip to FINAL THOUGHTS below)
If YES, please describe the activities.
What parts/components are likely to continue and why? FINAL THOUGHTS Given your experience implementing this project, what do you think you would have done differently if you could turn back the clock to the time when you first got involved with this grant?       Please tell us about your biggest success(es) with this project. This is your opportunity to brag about the work you are doing to help adolescents, transition-age youth, and their families. To what would you attribute the successes (e.g., strong staff, additional funding, morale…)?

Thank you for your time and contribution!

Please e-mail to [email protected]

or FAX to (978) 443-4722, Attn: Terri Tobin

4