Form Data Manager Surve Data Manager Surve Data Manager Survey

OMB No. 0930-xxxx

Expiration Date: xx/xx/xxxx

Assertive Adolescent &

Family Treatment (AAFT) Program

AAFT Implementation Survey

Evaluators

&

Data Managers

Public reporting burden for this collection of information is estimated to average 45 minutes per response per year, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Thank you for agreeing to participate in the

Program Evaluation for Assertive Adolescent & Family Treatment (AAFT) Program!

This survey is being conducted by Advocates for Human Potential, Inc. (AHP). We are a research and consulting firm based in Sudbury, MA and Albany, NY. We’re conducting this study as part of our contract to assist the Center for Substance Abuse Treatment (CSAT) in the national evaluation of the implementation of AAFT, funded by the Substance Abuse and Mental Health Services Administration (SAMHSA).

At the end of each project year, we are asking program administrators, clinical, and research staff across the entire AAFT3 grantee cohort to provide information about their professional background as well as opinions/thoughts on a variety of topics, including substance abuse treatment. We are gathering data from many sources and believe it is important to collect information from many perspectives as well. As you complete this survey, PLEASE RESPOND TO THE QUESTIONS FROM YOUR OWN PERSPECTIVE—choosing an answer that best describes your experience or opinion.

Your responses to the survey will be kept in a private record. Your responses will come directly to AHP staff. Any reports generated as part of this evaluation will contain only aggregate responses (for example, “50% of the program staff had been working for this program for less than two years”).

If you have any questions, concerns or comments about the questionnaire or the study, please feel free to contact us by phone or email:

Dr. Terri Tobin Denise Lang

Evaluation Director Research Associate

1-800-795-5486 x418 401-323-9678

[email protected] [email protected]

Thank you again for your participation!

Section A

Respondent ID: Please provide your First, Middle, & Last initials, and the Month & Day of your birthday. |               | (For example: ALA0415|)

What is your primary role in the project (select one):

Principal Investigator

Clinician/Counselor

Other, please describe      

Project Director

Researcher/Evaluator

Clinical Supervisor

Data Manager

What is the highest degree that you have obtained?

No high school diploma or equivalent

Bachelor’s degree

High School Diploma or Equivalent

Master’s degree

Some college, but no degree

Doctoral degree or equivalent

Associate’s degree

Other, please describe      

Please list on what date you began working at your current agency (MONTH/YEAR):      

Please list on what date you began working on this AAFT project (MONTH/YEAR):      

Grantee experience

This grant may not be your first experience working with A-CRA/ACC, Chestnut Health Systems or research/evaluation. The following questions ask about your experiences with A-CRA/ACC, Chestnut Health Systems, data collection and research/evaluation projects.

Prior to this project, did you participate in a CSAT-funded adolescent project?

Yes

No (If NO, skip to next question)

IF YES, please indicate each type(s).

AAFT

EAT

YORP

TCE/HIV

OJJDP

SCY

Other      

DK

Please indicate what your level of A-CRA/ACC/GAIN certification was prior to working on this AAFT project. [check all that apply]:

I was a certified GAIN local trainer.

I was a certified GAIN administrator.

I was an A-CRA-certified clinical supervisor.

I was an A-CRA-certified clinician.

I was an ACC-certified clinical supervisor.

I was an ACC-certified clinician.

None of the above

Section B: Please indicate…

1. To what extent have each of these components become part of the program’s normal, day-to-day routine?

2. How well do each of these components work within your program (e.g., how well do they “fit” with your program)?

	To what extent have each of these components become part of the program’s normal, day-to-day routine?				How well do each of these components work within your program (e.g., how well do they “fit” with your program)?
	Great change	Moderate change	Slight change	No change	Fits extremely well	Fits moderately	Fits a little	Not a fit
A-CRA approach
ACC approach
GAIN assessment at Intake
Treatment Satisfaction Index (TxSI)
EBTx website
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
Follow-up interviews (3, 6, 12 months)
Data management activities (e.g., monthly data submission to CHS, responding to GAIN Edits)
GAIN certification
GAIN Clinical Interpretation Certification (GCIC)
ACRA/ACC certification
Monitoring & compliance reports (e.g., follow-up rates, enrollment rates, DSRs, 13+week reports)

Section C: Based on your experience, please rate the reactions to each of the components of this project. If you feel that the question does not apply to you or you do not know the answer, please mark “N/A” or “DK” (respectively).

Your reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12, months)
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
Clinician reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
Use of EBTx website
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
Project leadership (e.g., project director, supervisor) reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
Use of EBTx website
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
The use of monitoring & compliance reports (e.g., follow-up rates, enrollment rates, DSRs, 13+week reports)

Section D: The following questions ask about your experiences with the support provided for the AAFT grant from Chestnut Health Systems, CSAT and other outside sources. For each item, please indicate how helpful each support has been. Lastly, please add any comments or recommendations for improvement for each.

Have you had involvement with A-CRA/ACC, EBTx, and/or DSRs?			YES		NO		IF “NO”—SKIP questions below, continue on next page
How helpful has it been in helping you implement the AAFT Program?	Extremely	Moderately		Slightly		Not at all	Not enough participation to rate this item	Any comments or recommendations for improvement?
EBTx website & DSRs								     
EBTx Support team staff								     
Program reports (e.g., 13+weeks report)								     
A-CRA/ACC team staff								     

Have you had involvement with the GAIN, GAIN data, and/or ABS?			YES		NO		IF “NO”—SKIP questions below, continue on the next table
How helpful has it been in helping you implement the AAFT Program?	Extremely	Moderately		Slightly		Not at all	Not enough participation to rate this item	Any comments or recommendations for improvement?
GAIN Training								     
GAIN certification								     
GAIN coaching calls								     
GAIN Training manual (e.g., intent of questions, certification process, etc.)								     
GAIN Clinical Interpretation Training								     
GAIN Clinical Interpretation Certification Individual-level reports (e.g., GRRS, PFR)								     
ABS web-based system								     
Reports (e.g. Site Profiles Report)								     
GAIN data (e.g., analytical files)								     
GAIN Support team staff								     
Data manager training call								     
Data manager manual								     
Responsiveness of Data Support team								     

How helpful has it been in helping you implement the AAFT Program?	Extremely	Moderately	Slightly	Not at all	Not enough participation to rate this item	Any comments or recommendations for improvement?
Evaluator Listserv						     
Implementation calls/site visits to monitor/address progress at grantee sites						     
Individualized coaching calls to address areas for improvement (e.g., low recruitment/follow-up rates, DSR uploads, increase TxSI completion, etc.)						     
Listserve Information						     
Technical assistance available through NIATx						     
Special topic calls (e.g., Cultural Responsiveness)						     
Monthly calls with CSAT project officer/Chestnut						     
Interactions with other grantees at meetings/calls						     
Initial/Annual Grantee Meetings						     

Section E: Based on your experience, please indicate if the issues listed below have been barriers to implementation at your site. Then, describe the strategies you have used to overcome the MOST CHALLENGING BARRIER at your site.

POSSIBLE BARRIERS	Was this a barrier to implementation at your site?
Program/Organizational Issues	YES	NO
Internal communication (e.g., program staff)
External communication (e.g., CSAT, Chestnut Health Systems)
Staff attitudes (e.g., morale, enthusiasm, resistance)
Leadership/management attitudes
Turnover; significant loss of staff
Service Delivery	YES	NO
Enrolling clients
Client retention
Grant-related Activities	YES	NO
GAIN training
Collecting GPRA data
Collecting GAIN data
Collecting follow-up data
Using ABS
Using the SAIS system for GPRA
Other Grant requirements (e.g., certification, coaching calls, data management)
Any other barriers not included above (specify):     
Please describe the strategies you have used to overcome the MOST CHALLENGING BARRIER at your site.      

Section I: Please read each statement carefully and indicate how much you agree with each of the following statements related to your grantee site’s experience with research and evaluation activities.

	Agree Strongly	Agree	Uncertain	Disagree	Disagree Strongly
Policies and procedures are in place to ensure integrity of data collected.
Staff capacity is adequate to meet the demands of data collection efforts.
I do not have time to complete data collection in a standardized way
I find structured interviews easy to use.
Research is an important part of my job.
I feel comfortable with structured data collection processes.
I have had adequate training in how to collect data in a standardized way.
When conducting a semi-structured interview, it is acceptable to deviate a little bit from the interview script.
I am experienced in collecting data using computer-assisted interviewing techniques.
Program staff use data to evaluate program services and consider opportunities for improvement.
Our agency relies on data to set goals and measure success.
Data collected is analyzed and used to evaluate/enhance program implementation.
Management shares data with staff about program effectiveness in meeting the needs of clients.
Data are provided to/discussed with staff.
Our program regularly conducts surveys with consumers to identify program strengths and weaknesses.

FINAL THOUGHTS

Given your experience implementing this project, what do you think you would have done differently if you could turn back the clock to the time when you first got involved with this grant?

     

Please tell us about your biggest success(es) with this project. This is your opportunity to brag about the work you are doing to help adolescents, transition-age youth, and their families. To what would you attribute the successes (e.g., strong staff, additional funding, morale…)?

Thank you for your time and contribution!

Please e-mail to [email protected]

or FAX to (978) 443-4722, Attn: Terri Tobin

3