Supporting Statement
The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Treatment (CSAT) is requesting approval from the Office of Management and Budget (OMB) for 13 data collection instruments for the National Evaluation of the Addiction Technology Transfer Centers (ATTCs):
Site Visit Protocol and Interview Guide
Focus Group Protocol
Key Informant Interview Protocol
Collaborative Functioning Survey
Customer Satisfaction and Benefit Survey
Evidence-based Critical Action Survey for Clinical Supervision
Evidence-based Critical Action Survey for Motivational Interviewing
Evidence-based Critical Action Survey for Treatment Planning M.A.T.R.S.
Success Case Interview Protocol for Clinical Supervision
Success Case Interview Protocol for Motivational Interviewing
Success Case Interview Protocol for Treatment Planning M.A.T.R.S.
Organizational Readiness for Change Survey
Clinician Self-Assessment Form on Motivational Interviewing
The ATTC program was established by SAMHSA/CSAT in 1993 to enhance clinical practice and improve the provision of addictions treatment for the 23 million Americans age 12 and older who need treatment for alcohol or illicit drug problems by (1) promoting the adoption of culturally appropriate, evidence-based, and promising practices; (2) disseminating relevant research findings; and (3) translating the latest science on treatment services into information that states and clinicians can use in service delivery. ATTCs deliver training and technology transfer activities in partnership with Single State Authorities (SSAs), treatment provider associations, addictions counselors, multidisciplinary professionals, faith and recovery community leaders, family members of those in recovery, and other stakeholders. Particular emphasis is on raising awareness of and improving skills in using evidence-based and promising treatment/recovery practices in recovery-oriented systems of care (ATTC Program Announcement, 2007). The ultimate vision of the ATTC Network is to unify science, education, and services to transform the lives of individuals and families affected by alcohol and drug addiction (National ATTC, 2006). The evaluation will address the research questions shown in the table below. For a more detailed description of the ATTC program, please refer to the ATTC Initial Program Announcement, which is appended as Attachment A.
CSAT awarded grants to 14 regional ATTCs and 1 National Coordinating Center in the most recent award cycle in September 2007. The grants are authorized under Section 509 of the Public Health Service Act, as amended. With awards between $500,000 and $550,000 per year for five years, the 14 regional Centers serve between 2 and 6 States, the District of Columbia, and U.S. territories (i.e., Puerto Rico, Virgin Islands, and Pacific Islands). The National Coordinating Center focuses primarily on nation-wide initiatives involving the entire ATTC Network.
There has not been an evaluation of the ATTC Network since it was first funded in 1993. Consequently, an evaluation of the Network is necessary to identify the effectiveness of the ATTCs’ technology transfer activities, and to build upon the successes of the Network in the future. The evaluation will also distinguish between processes and outcomes that are region- and need-specific and those that are common to the Network as a whole.
The evaluation questions are shown below in Table 1. The evaluation questions will be answered by conducting three coordinated studies:
A Planning and Partnering Study to determine the processes and partnerships ATTCs undertake to meet the needs of their constituencies.
A Customer Satisfaction and Benefit Study to assess the degree of satisfaction and self-reported cumulative benefit from ATTC activities and products among the range of customers and stakeholders important to the ATTC Network.
Three Change in Practice Studies to assess the extent to which desired changes in clinical practice occur in the substance use disorder workforce in association with ATTC activities and products. These studies will focus on three evidence-based practices: Motivational Interviewing (MI), Clinical Supervision, and Treatment Planning M.A.T.R.S. Each ATTC will participate in one Change in Practice Study.
Table 1 - Evaluation Questions for the National Evaluation of the ATTCs
Process Evaluation Questions |
|
1. |
How do ATTCs set priorities within their regions? |
2. |
How and to what extent are ATTCs collaborating with each other and with local, State, and Federal partners to address regional and/or national needs? |
3. |
What does the technology translation and transfer process look like across the ATTC Network? |
4. |
To what extent do ATTC Network activities meet States’, treatment professionals’, and recovery support staff needs? |
5. |
With what frequency is each technology transfer objective type – awareness raising, skill building, changing practice – addressed across the Network? |
6. |
What are the key characteristics of effective ATTC activity planning and delivery that can be shared and implemented across the Network? |
Outcome Evaluation Questions |
|
7. |
To what extent has the ATTC Network enhanced the competencies, including cultural competencies, of specialty addictions treatment practitioners, paraprofessionals, and multidisciplinary professionals to strengthen the workforce? |
8. |
What changes in substance use disorder treatment systems are associated with activities of the ATTC Network? |
Data will be obtained from several data collection instruments, including a Site Visit Protocol and Interview Guide, Key Informant Interview Protocol, Focus Group Protocol, Collaborative Functioning Survey, Customer Satisfaction and Benefit Survey, Evidence-based Critical Action Surveys (for Clinical Supervision, Motivational Interviewing and Treatment Planning M.A.T.R.S.), Success Case Interview Protocols (also for Clinical Supervision, Motivational Interviewing and Treatment Planning M.A.T.R.S.), an Organizational Readiness for Change Survey, and a Clinician Self-Assessment Form on Motivational Interviewing.
The primary purpose of the data collection is program evaluation. The data will enable CSAT to document and assess the accomplishments and lessons learned of the ATTCs in transferring culturally appropriate, evidence-based, and promising practices in addictions treatment and recovery support to States, clinicians and others for use in providing treatment and recovery support services. The data will also inform ATTC program operations by identifying best practices for technology transfer that may be adopted by individual ATTCs, as appropriate, or across the Network.
One site visit will be conducted to each ATTC. The purpose of the site visits will be to obtain in-depth information on the regional environment in which they are operating and how ATTCs plan and deliver services within their regions. During the site visits, data will also be collected on the organizations and agencies ATTCs partner with to plan and provide these services and the array of stakeholders and stakeholder needs within each ATTC region and how these influence planning activities. During each site visit, we will conduct semi-structured interviews with the ATTC Director, the ATTC evaluator, and other ATTC staff. A Site Visit Protocol and Interview Guide (Attachment B) will be used for this data collection activity.
The data collected through the site visits will be used to answer evaluation questions 1, 2, 3, 5, and 6 (see Table 1).
The evaluation will include focus groups with technology transfer specialists at each ATTC. These staff may be located in the main office of the ATTC, or may operate from offices in other parts of the ATTC region. These staff will be asked about their involvement in service delivery, the components of the technology transfer process, and how the process is adapted when the audiences for services are ethnically, racially, or culturally diverse. A Focus Group Protocol
Telephone interviews will be conducted with a sample of ATTC program stakeholders. Stakeholders will include SSA directors, ATTC Advisory Board members, directors of provider associations, treatment agency directors, addiction educators, and cultural leaders. The purpose of the interviews will be to understand how ATTCs engage stakeholders in their planning processes and the partnerships they develop with stakeholders for service delivery. The key informant interviews will also provide information about the types of services for which stakeholders contact the ATTCs, their general satisfaction with these services, and their perceptions of the benefit and impact of the ATTCs within the addictions treatment and recovery field. A Key Informant Interview Protocol (Attachment D) will be used for this data collection activity.
The data collected through the key informant interviews will be used to answer evaluation questions 1, 2, 3, 4, and 6 (see Table 1).
The Collaborative Functioning Survey (Attachment E) will be used to survey the Advisory Board members of each ATTC. The initial survey and follow-up survey will be administered in person via a paper form at an Advisory Board meeting. The National Evaluation team will seek the assistance of the ATTCs’ local evaluators in conducting this survey. The purpose of the survey is to gather Board members’ opinions about the planning processes they are involved in with their regional ATTC and the perceived outcomes of their participation and collaboration with the ATTC.
The evaluation will conduct a survey of a representative sample of individuals who have participated in services from the ATTCs, as well as individuals who are among the ATTCs’ potential customers but may not have received services from an ATTC. The Customer Satisfaction and Benefit Survey (Attachment F) is a mail survey. The purpose is to collect data about individuals’ experiences with the ATTCs, including individuals’ motivations to seek services from the ATTCs, the services that have been most useful, and how information obtained from the ATTCs has been used by the ATTCs’ customers.
The data from the Customer Satisfaction and Benefit Survey will be used to answer evaluation questions 3, 4, 6, 7, and 8 (see Table 1).
The evaluation will field Evidence-based Critical Action Surveys on three topics (or interventions): Clinical Supervision (Attachment G), Motivational Interviewing (MI) (Attachment H), and Treatment Planning M.A.T.R.S. (Attachment I). Each ATTC will participate in one of these topics, or Change in Practice Studies. Consistent with the Success Case Method (SCM) developed by Robert Brinkeroff (2003), these surveys will assess training participants’ prior familiarity and competence in each topic and current implementation experience. Each survey is structured around the critical actions associated with the intervention and will be used to identify those participants reporting the greatest and least amount of success implementing the specified critical actions related to the intervention. Reviewing trainer notes will validate identified successes and nonsuccesses and training participants will then be targeted for in-depth interviews (see Success Case Interview Protocols below). The survey will assess the value of each training initiative and the impact of the training on adoption and implementation of the intervention.
The data from the Evidence-Based Critical Action Surveys will be used to answer evaluation questions 6, 7, and 8 (see Table 1).
The SCM approach takes the top and bottom adopters (those who use the new knowledge/techniques vs. those who do not) of a particular intervention and compares differences in the two extremes. The evaluation will interview the top and bottom adopters of the three practices that were the focus of the Critical Action Surveys (Clinical Supervision [Attachment J], MI [Attachment K], and Treatment Planning M.A.T.R.S. [Attachment L]) using the Success Case Interview Protocol developed for each practice. The information gathered through the interviews will provide qualitative insights on facilitators and barriers to usage of the practices/interventions.
The data from the Success Case Interviews will be used to answer evaluation questions 6, 7, and 8 (see Table 1).
The Organizational Readiness for Change (ORC) Survey (Attachment M) is a validated instrument that assesses organizational traits known to predict program change (Lehman et al., 2002).1 It includes scales from four major domains—motivation, resources, staff attributes, and climate. The National Evaluation will administer the ORC to administrators of the agencies participating in ATTC services on Treatment Planning M.A.T.R.S.
The data from the Organizational Readiness for Change Survey will be used to answer evaluation questions 7 and 8 (see Table 1).
The Motivational Interviewing Adherence, Clinician Self-Assessment Form (Attachment N) is an instrument that clinicians use to assess the degree to which they have incorporated Motivational Interviewing strategies or techniques into their client sessions. The instrument was developed as part of the NIDA/SAMHSA Blending Initiative.2 The National Evaluation will ask 72 clinicians—36 who are most successful at implementing Motivational Interviewing based on the results of the Evidence-Based Critical Action Survey and 36 randomly selected from all survey respondents—to complete the form.
The data from the Clinician Self-Assessment Form will be used to answer evaluation questions 7 and 8 (see Table 1).
The data collection plan reflects sensitivity to issues of efficiency, accuracy, and respondent burden. Where feasible, information will be gathered from existing data sources, rather than imposing additional burden by collecting primary data. For example, the evaluation will obtain data from ATTC grant applications, ATTC websites, ATTC publications, databases maintained by the National Coordinating Center, and GPRA data that are collected on selected ATTC events (trainings, conferences and meetings, and some technical assistance activities). Minimizing evaluation costs and reducing respondent burden were also key considerations in the decision to answer outcome evaluation questions (see Introduction) by focusing on three topics and evidence-based practices (Clinical Supervision, Motivational Interviewing, Treatment Planning M.A.T.R.S.) that are of high priority to SAMHSA and the addictions treatment field, as opposed to collecting data on changes in practice resulting from all ATTC services.
The National Evaluation will also use information technology where appropriate. The manner in which each instrument will be administered is indicated below:
Site Visit Protocol and Interview Guide: Face-to-face interviews
Focus Group Protocol: Face-to-face group discussion
Key Informant Interview Protocol: Telephone interviews
Collaborative Functioning Survey: Paper-and-pencil survey, distributed to respondents at an Advisory Board Meeting of each ATTC
Customer Satisfaction and Benefit Survey: Mail survey
Evidence-based Critical Action Survey for Clinical Supervision: Web-based survey
Evidence-based Critical Action Survey for Motivational Interviewing: Web-based survey
Evidence-based Critical Action Survey for Treatment Planning M.A.T.R.S.: Web-based survey
Success Case Interview Protocol for Clinical Supervision: Telephone interview
Success Case Interview Protocol for Motivational Interviewing: Telephone interview
Success Case Interview Protocol for Treatment Planning M.A.T.R.S.: Telephone interview
Organizational Readiness for Change Survey: Mail survey
Clinician Self-Assessment Form on Motivational Interviewing: Paper-and-pencil survey
As noted above, the Critical Action Surveys will be web-based. A decision was made to ask the ATTC evaluators to distribute and administer the Collaborative Functioning Survey at a meeting of each ATTC’s Advisory Board, which will help ensure a high response rate and efficiency in data collection. The ATTC grant announcement specifically requested each ATTC to budget .25 FTE for providing support to the National Evaluation, and ATTC evaluators will use some of these resources to assist with this data collection activity.
A National Evaluation of the ATTCs has never been undertaken. Therefore, the proposed data collection does not duplicate other efforts.
The primary entities for this study are university-based or non-profit Addiction Technology Transfer Centers and participants in ATTC activities—e.g., SSA directors, ATTC Advisory Board members, provider association directors, addiction educators, clinicians, clinical supervisors, and treatment agency directors. Burden is reduced for all respondents by requesting only the minimum information required to meet the evaluation objectives. The burden on respondents has been minimized through the careful specification of information needs, restricting questions to generally available information, and designing the data collection strategy—particularly the survey methods—to minimize burden on respondents. All data collection will be coordinated and scheduled, so as to minimize burden on the ATTCs, State agencies, treatment providers, and other recipients of the ATTCs’ services.
Because the ATTCs and most other respondents are employees of state agencies, universities, or other large organizations, the evaluation will have no significant impact on small entities or small businesses.
Not conducting this data collection would significantly impede CSAT’s ability to assess the impacts of the ATTC Network on the addictions treatment and recovery field and States and provider organizations who offer critical services. Moreover, the processes regional ATTCs use to plan their services, whom they partner with to deliver services, and the outcomes of the ATTC Network on the provision of addictions treatment and recovery support services cannot be assessed solely from extant databases. Without the National Evaluation, this information will not be available for dissemination to interested stakeholders at Federal, regional, State, and local levels, or for enhancement of the ATTC program.
The proposed data collection fully complies with all guidelines of 5 CFR 1320.5.
A 60-day notice to solicit public comments was published in the Federal Register on Wednesday, December 19, 2007 (Volume 72, No. 243, pp. 71932 - 71935). One comment was received during the comment period (see Attachment O). CSAT’s response to this comment is provided below:
Request to “review the surveys for the Customer Satisfaction and Benefit Study prior to their distribution.” To ensure the independence and objectivity of the national evaluation, the ATTCs were not invited to review the data collection instruments. However, to create a participatory environment, the evaluation contractor has provided multiple opportunities for the ATTCs to have input to the evaluation design, including the evaluation questions, data sources, and topics to be emphasized in the data collection activities. These opportunities occurred during ATTC Directors’ meetings (in November 2005, June 2006, November 2006), a 1-1/2 day face-to-face meeting (in February 2007), a limited number of stakeholder interviews, and several conference calls during the design phase of the evaluation. The input received and information that was shared during these events was used extensively in developing the evaluation design and the data collection instruments.
Request that the national evaluation plan measure not just “the dissemination of NIDA Blending Products” through the Change Study, but also “capture the other important ways that ATTCs impact their regions.” The Change Study that is proposed as one of several components of the national evaluation will measure the extent to which the ATTCs have been successful in disseminating best practices and upgrading the skills of practitioners, which is a major objective of the ATTCs. The Change Study will focus on three specific practices—Clinical Supervision, Motivational Interviewing, and Treatment Planning M.A.T.R.S., of which only one (Treatment Planning M.A.T.R.S.) is a NIDA Blending topic. These practices were selected after examining GPRA data submitted by the ATTCs, inventories of ATTC activities compiled by each ATTC during the evaluation design phase, and a review of activities mandated of the ATTCs in the Spring 2007 RFA. In addition to the Change Study, the national evaluation will collect data through site visits, focus groups, and key informant interviews to examine and describe the contributions ATTCs have made to organizational- and system-level (e.g., policy) changes. Thus, the national evaluation will obtain data from a broad array of stakeholders to capture the many ways that the ATTCs impact their regions.
Request that the evaluators “formally assess the regional differences in disseminating NIDA Blending Products.” As noted above, the national evaluation will assess far more than the dissemination of NIDA Blending Products and the differences across regions. The evaluation’s multiple data collection activities will enable the evaluation to answer questions related to the potentially broad array of changes that occur within and across the states and regions served by the ATTCs. The data collection instruments also include items that will assess regional differences in the demand for activities on specific topics (e.g., those addressed by the Change Studies). Additionally, the evaluation will examine how service delivery decisions are made (e.g., whether based on regional requests, or coordinated based on regional needs), assuring sensitivity to regional differences in service delivery patterns.
Comment that data from the national evaluation not be used to “compare one ATTC against another.” The national evaluation will not compare one ATTC against another. Instead, an important purpose of the evaluation is to identify “best practices” that can be shared across regions for the enhancement of the overall ATTC program. To do this, it will be necessary to examine practices within regions and to determine whether these practices are tied to unique conditions within a region or could be replicated more broadly. However, examining practices within regions does not imply that ATTCs will be compared with each other. Further, practices and all other findings will be discussed in evaluation reports without attribution.
Consultations on the evaluation design, sample design, data sources, and planned evaluation reports have occurred during the evaluation’s design phase and have continued to take place during the early months of the implementation phase of the evaluation. During the evaluation design phase, Abt Associates Inc. (Abt) and RMC Research Corporation (RMC) convened an Expert Advisory Panel on April 18, 2006 to discuss the purpose of the evaluation and possible evaluation design options. The panel offered valuable suggestions on the evaluation questions and design.
Since the Expert Panel Meeting, staff from Abt, RMC, and MANILA Consulting Group have had regular meetings with the SAMHSA/CSAT project officer and other CSAT staff, who have reviewed the evaluation design and data collection plan, including all data collection instruments. RMC and Abt also met with the ATTC directors and ATTC evaluators in February 2007 to develop an inventory of ATTC activities, to identify the technology transfer objectives of these activities, and to determine the relative emphasis each ATTC places on a broad array of addiction treatment and technology transfer topics. The purpose of these consultations was to ensure the technical soundness of the evaluation and the relevance of its findings, and to verify the importance, relevance, and accessibility of the information sought in the evaluation.
SAMHSA/CSAT staff, Expert Panel members, and others who have provided guidance on the evaluation are listed below:
Deepa Avula
Public Health Advisor and Project Officer
Division of State and Community Assistance
SAMHSA/CSAT
Deni Carise, Ph.D.
Director
Treatment Systems Research
Mady Chalk, Ph.D.
Director of the Center for Performance-based Policy
Treatment Research Institute
Donna M. Cotter, M.B.A.
Partners For Recovery Coordinator (former)
Office of Program Analysis and Coordination
SAMHSA/CSAT
Charlotte Chapman, M.S., LPC
Director of Counseling Services
University of Virginia Women’s
Center
Karen Ingersoll, Ph.D.
Department of Psychiatric Medicine
Center
for Addiction Research and Education
University of Virginia
Andrea Kopstein, M.P.H., Ph.D.
Chief, Practice Improvement Branch
Division of Services Improvement
SAMHSA/CSAT
Hannah Knudsen, Ph.D.
Assistant Research Scientist
Center for Research on Behavioral Health and Human Services Delivery
University of Georgia
Meghan Love
Section Coordinator
Treatment Research Institute
Pamela Mattel, LCSW-R CASAC
Executive Deputy Director
BASICS,
Inc.
Dennis McCarty, Ph.D.
Professor
Department of Public Health and Preventive Medicine
Oregon Health & Science University
Thomas McLellan, Ph.D.
Chief Executive Officer
Treatment Research Institute
William R. Miller, Ph.D.
Department of Psychology
University
of New Mexico
Catherine D. Nugent, M.A.
Public Health Advisor
Division of Services Improvement
SAMHSA/CSAT
Denise Pintello, M.S.W., Ph.D.
Special Assistant to the Deputy Director
National Institute on Drug Abuse
John Porter
Technology Transfer Specialist
PEGASUS Training and Consulting
Christopher Wagner, Ph.D.
Departments of Rehabilitation
Counseling, Psychology and Psychiatry
Virginia Commonwealth
University
The national evaluation of the ATTCs includes surveys of clinicians in the addictions treatment field who have participated in ATTC services. These clinicians are a “hard to reach” population, for data collection purposes, as “high turnover rates . . . and inadequate compensation”3 plague this group of workers overall and limit their time and inclination to participate in data collection activities. As part of CSAT’s strategy to get a high response from clinicians who will be given the Critical Action Surveys, we will send each person who completes a survey a nominal incentive of $5 (or equivalent non-monetary incentive—i.e., a gift card). A high response rate on these surveys is important to the evaluation, because a sample of the highest scorers (most successful implementers of the evidence-based practice, or critical actions) and the lowest scorers (least successful implementers) will be selected for a follow-up interview. Without a high response, the selection of the most and least successful implementers will be less representative, and there will be a potential for non-response bias. We will offer an additional $5 incentive to those survey respondents who are selected for a follow-up interview.
We base the use of incentives on a review of the literature, which shows that incentives, even when small in monetary terms, are effective in increasing response rates (Armstrong, 1975; Church, 1993; Goyder, 1994).4 Studies on the use of incentives also show that rather than negatively affecting data quality, the quality of the data is improved, because there are fewer instances of item non-response and more comments to open-ended questions (James and Bolstein, 1990; Brennan, 1992; Shettle and Mooney, 1999).5
No other respondents for these data collection activities will be paid for participating in the evaluation. Participation in the National Evaluation is completely voluntary.
All individual data collected by the evaluation will be conducted in accordance with the Privacy Act of 1974 (5 U.S.C. 552a), SAMHSA Participant Protection requirements, and other Federal and Department of Health and Human Services (DHHS) regulations on the protection of human subjects (e.g., 5 U.S.C. 301; 42 U.S.C. 289(a)).
The evaluation team will work closely with the ATTCs’ staffs and their regional and State Institutional Review Boards (IRBs) to ensure that the necessary IRB approvals are obtained, and that human subject protections are assured. For some data collection activities—i.e., the Critical Action Surveys—respondents will be asked to provide their name and telephone number, because the evaluation will conduct follow-up interviews with some participants. When provided, individual identifying information will be maintained separately from completed data collection forms and from computerized data files used for analysis. No respondent identifiers will be contained in public use files made available from the evaluation. The evaluation team will utilize personal codes, not names, when reporting data. No respondents or specific comments will be individually identified to anyone or any organization or agency.
Each respondent will be given an assurance of privacy and the project will protect the privacy of respondents. The privacy statement will state that participation in the evaluation is strictly voluntary and individuals have the right to refuse to complete the survey or participate in the interview or focus group discussion. Additionally, they will be assured that information will be reported only in aggregate form in reports, that their names will not be associated with their answers, and that no one will have access to this information except as may be required by law, regulation, or subpoena or unless permission is given by the respondent.
Hard-copy data collection forms will be held in a locked area for receipt and processing. All data files on multi-user systems will be under the control of a database manager, with access limited to project staff on a “need-to-know” basis only.
There are no questions of a sensitive nature in the data collection instruments.
The estimated burden for data collection is 4,118 hours. Using May 2006 National Occupational Employment and Wage Estimates from the Bureau of Labor Statistics, U.S. Department of Labor (http://www.bls.gov/oes/current/oes_nat.htm#b11-0000), the estimated total cost to respondents is $101,940.
Respondent Burden and Total Cost to Respondents
Name of Instrument / Respondent |
No. of Respondents |
Responses per Respondent |
Total Responses |
Average Time per Response (Hours) |
Total Respondent Time (Hours) |
Estimated Hourly Wage |
Cost to Respondents* |
||||||
Site Visit Interview Protocol |
|
|
|
|
|
|
|
||||||
ATTC Directors |
15 |
1 |
15 |
1.5 |
22.5 |
$41.67 |
$938 |
||||||
ATTC Staff |
38 |
1 |
38 |
1.5 |
57 |
$21.81 |
$1,243 |
||||||
Focus Group Protocol |
|
|
|
|
|
|
|
||||||
ATTC Field Staff |
35 |
1 |
35 |
2 |
70 |
$21.81 |
$1,527 |
||||||
Key Informant Interview Protocol |
|
|
|
|
|
|
|
||||||
ATTC Advisory Board Members |
45 |
1 |
45 |
1 |
45 |
$39.02 |
$1,756 |
||||||
SSA Directors |
55 |
1 |
55 |
1 |
55 |
$39.02 |
$2,146 |
||||||
Provider Association Directors |
43 |
1 |
43 |
1 |
43 |
$39.02 |
$1,678 |
||||||
Addiction Educators |
70 |
1 |
70 |
1 |
70 |
$21.81 |
$1,527 |
||||||
Treatment Agency Directors |
42 |
1 |
42 |
1 |
42 |
$39.02 |
$1,639 |
||||||
Other Key Advisors |
42 |
1 |
42 |
1 |
42 |
$39.02 |
$1,639 |
||||||
Collaborative Functioning Survey |
|
|
|
|
|
|
|
||||||
ATTC Advisory Board Members |
450 |
2 |
900 |
0.5 |
450 |
$39.02 |
$17,559 |
||||||
ATTC Staff |
15 |
2 |
30 |
0.5 |
15 |
$21.81 |
$327 |
||||||
Customer Satisfaction and Benefit Survey |
|
|
|
|
|
|
|
||||||
SSA Directors |
55 |
1 |
55 |
0.5 |
27.5 |
$39.02 |
$1,073 |
||||||
Provider Association Directors |
43 |
1 |
43 |
0.5 |
21.5 |
$39.02 |
$839 |
||||||
Addiction Educators |
158 |
1 |
158 |
0.5 |
79 |
$21.81 |
$1,723 |
||||||
Treatment Agency Directors |
700 |
1 |
700 |
0.5 |
350 |
$39.02 |
$13,657 |
||||||
ATTC Activity Participants |
3,000 |
1 |
3,000 |
0.5 |
1,500 |
$17.28 |
$25,920 |
||||||
Other Unique Regional Partners |
168 |
1 |
168 |
0.5 |
84 |
$39.02 |
$3,278 |
||||||
Evidence-Based Critical Action Surveys |
|
|
|
|
|
|
|
||||||
ATTC Clinical Supervision Training Participants |
240 |
1 |
240 |
0.5 |
120 |
$18.26 |
$2,191 |
||||||
ATTC Motivational Interviewing Training Participants |
360 |
1 |
360 |
0.5 |
180 |
$17.28 |
$3,110 |
||||||
ATTC Treatment Planning M.A.T.R.S. Treatment Participants |
240 |
1 |
240 |
0.5 |
120 |
$17.28 |
$2,074 |
||||||
Success Case Interview Protocols |
|
|
|
|
|
|
|
||||||
ATTC Clinical Supervision Training Participants |
48 |
1 |
48 |
1 |
48 |
$18.26 |
$876 |
||||||
ATTC Motivational Interviewing Training Participants |
72 |
1 |
72 |
1 |
72 |
$17.28 |
$1,244 |
||||||
ATTC Treatment Planning M.A.T.R.S. Training Participants |
48 |
1 |
48 |
1 |
48 |
$17.28 |
$829 |
||||||
Clinician Self-Assessment Form on Motivational Interviewing |
72 |
12 |
864 |
0.5 |
432 |
$17.28 |
$7,465 |
||||||
Survey of Organizational Readiness |
|
|
|
|
|
|
|
||||||
Treatment Agency Directors |
240 |
1 |
240 |
0.5 |
120 |
$39.02 |
$4,682 |
||||||
TOTAL |
6,294 |
----------------- |
7,551 |
--------------- |
4,118 |
-------------- |
$101,904 |
||||||
*Cost to respondents = Total Respondent Hours x Estimated Hourly Wage
SAMHSA/CSAT’s 2007 Program Announcement for the ATTCs stated the following:
“Grantees are expected to participate in the evaluation. It is expected that approximately .25 FTE will be required to carry out some data collection as part of the independent evaluation. Applicants should budget for this [cost].” The ATTCs will use these budgeted resources to assist the National Evaluation team in identifying participants in the focus groups and key informant interviews as well as Customer Satisfaction and Benefit Survey and Critical Action Survey respondents. They will also use some of these funds to assist the evaluation team in administering the Collaborative Functioning Survey and in coordinating with treatment providers for their participation in the evaluation (e.g., obtaining a high response to the Survey of Organizational Readiness). Other costs are the opportunity costs of respondents’ time required to provide information as explained in A12, above.
The National Evaluation of ATTCs does not place any capital equipment, start-up, or record maintenance requirements on respondents.
The estimated cost to the Federal government of conducting the National Evaluation of the Addiction Technology Transfer Centers is based on the government’s contracted cost of the data collection and related study activities along with the personnel cost of government employees involved in oversight and/or analysis. For the data collection activities for which OMB approval is currently being requested, the overall cost to the government is $1,964,817. Most of these costs will be incurred after August 2008, when data collection is expected to be underway. When annualized, the cost to the government amounts to $654,939 per year. The personnel cost of government employees is included in this figure.
This is a new data collection.
The evaluation team will write annual and final reports that include a synthesis of the evaluation findings. The reports will include qualitative and quantitative analyses of data collected and graphic and tabular displays of the key findings.
The qualitative analysis will investigate the plausible arguments that can be made regarding the relationship between ATTC planning and processes for service delivery, customer satisfaction with ATTC services, and individuals’ successful implementation of knowledge acquired and skills learned as a result of participation in these services. This analysis will include identifying themes that emerge across ATTC regions and across respondent types. The quantitative analysis will include presenting descriptive statistics of survey results (e.g., measures of central tendency [mean or median], dispersion, maximum and minimum values, and frequencies) and inferential statistics (e.g., cross-tabulations, t-tests, regression analyses) where appropriate to assess relationships among key variables (e.g., type of respondent, utilization of ATTC services, implementation of skills learned as a result of participation in ATTC services).
The primary products of the evaluation will be annual reports delivered in September 2008 and September 2009 and a final report in July 2010. These reports will cover findings from the data collection activities conducted each year, and for all data collection activities at the end of the three-year contract.
In addition, the evaluation will produce case studies on four special topics: (1) planning and partnering processes of the ATTCs, (2) relationship building processes, (3) cultural adaptation in delivering ATTC services, and (4) use of technology in service delivery. These case studies will draw upon data collected through site visits, key informant interviews, document reviews, website reviews, and focus groups with ATTC field staff.
The full timeline for the evaluation is shown below.
Evaluation Timeline
Evaluation Activity |
Schedule |
Evaluation Design |
September 2005 – December 2007 |
Database Design |
October 2007 – March 2008 |
Assignment of ATTCs to Change Studies |
March – May 2008 |
ATTC Site Visits |
July – October 2008 |
Focus Groups with ATTC Field Staff |
July – October 2008 |
Survey of ATTC Advisory Board Members |
July-September 2008; October – December 2009 |
First Annual Report |
September 2008 |
Key Informant Interviews |
October 2008 – September 2009 |
Survey of ATTC Customers |
October 2008 – September 2009 |
Critical Action Surveys |
March – June 2009 |
Success Case Interviews |
August – November 2009 |
Clinician Self Assessments |
August – November 2009 |
Survey of Organizational Readiness |
August – November 2009 |
Second Annual Report |
September 2009 |
Case Study: Relationship Building |
April – September 2009 |
Case Study: Planning and Partnering |
July – December 2009 |
Case Study: Use of Technology |
October 2009 – March 2010 |
Case Study: Cultural Adaptation |
January – May 2010 |
Final Report |
July 2010 |
The expiration date will be displayed.
The certifications are included in this submission.
The evaluation design, and subsequent sampling plan for the National Evaluation of the ATTCs, is driven by four key considerations:
The necessity to collect evaluation data from multiple stakeholder groups and partners, as well as a broad array of target populations and customers. The ATTCs work closely with key stakeholders and partners in their regions to carry out their work (stakeholder groups include Single State Authorities for Alcohol and Drug Services [SSAs], treatment provider and counseling associations, faith-based and recovery organizations, academic institutions and addiction educators, credentialing/licensing bodies, and Regional Indian Health Boards). Formal relationships (e.g., National Institute on Drug Abuse [NIDA] Blending Initiative) and informal relationships (e.g., Centers for the Application of Prevention Technologies [CAPTs]; Network for the Improvement of Addiction Treatment [NIATx]) also have been forged between the ATTCs and other Federal programs and offices to consolidate efforts and avoid duplication. The majority of ATTC resources are directed toward front-line addiction counselors, clinical supervisors, and other key treatment organization and recovery support personnel, as well as students preparing for roles in the substance use disorder treatment field. However, the ATTCs also provide services for professionals in other settings (e.g., criminal justice, child welfare, primary health care) and nonprofessionals (e.g., faith community members, peer recovery community leaders). In addition, ATTCs are expected to address the needs of several special population groups including women who abuse substances, racial and ethnic minorities, culturally distinct members of rural and remote communities, the recovery community, criminal justice populations, individuals with co-occurring disorders, and individuals in welfare-to-work environments. To ensure as many voices are heard as possible, sampling will be employed across all data collection methods.
The importance of balancing burden across a large and diverse population of potential respondents. The sampling plan has taken special consideration to spreading the burden of data collection across multiple groups as evenly as possible.
The availability of limited resources to complete the evaluation. Despite the large number of potential respondents, and the overall size and reach of the ATTC Network, resources for the evaluation are limited. This has made deliberate sampling a necessary feature of CSAT’s design.
The stated purpose of the evaluation, which is to identify best practices and lessons learned. This purpose relies upon learning and understanding what is occurring within the ATTC Network by engaging a broad array of customers and stakeholders across diverse settings. Conversely, the evaluation purpose is not being driven by specific research hypotheses or analytic processes that would require more specific sampling frames.
The evaluation design phase interviews with ATTC Directors, SSA Directors, and CSAT staff helped define these considerations, and provided feedback valuable in ensuring that they were reflected in the final evaluation design. In addition, the Expert Panel commented on and reviewed the feasibility of CSAT’s data collection activities and sampling estimates.
Given these considerations, multiple data collection strategies will be employed. The sampling strategy is described for each below (if applicable), with overall sampling numbers provided in the table below.
Site Visits: To ensure best use of the limited time available during site visits to each ATTC (14 regional centers plus the National Coordinating Center), individual interviews will be conducted with the ATTC Director and (if applicable) Co-Director or Deputy Director, the ATTC evaluator, the ATTC technology specialist, NIDA Blending Initiative liaison (if applicable), and ATTC staff who specialize in cultural adaptations related to the cultural diversity of their particular region. ATTC field staff who deliver services to the regional customers and, where applicable, ATTC staff stationed in specific States in the region will be invited to participate in a focus group.
Key Informant Interviews: A sample of SSA directors, addiction educators, and presidents of treatment provider associations will be interviewed within each ATTC region to gain their perspectives on the ATTCs’ planning and partnering processes. In some regions, cultural leaders (e.g., American Indian tribal leaders, officials from the Historically Black Colleges and University network), collaborators from other addiction-related disciplines (e.g., primary care physicians, criminal justice administrators), and leaders of recovery associations may be identified as key stakeholders by senior ATTC staff. Across these respondent groups, we expect to conduct an average of 20 key informant interviews in each region. We will select those key informants who are most engaged and influential in the planning and implementation of ATTC services and those who represent the diversity of the customer base in the region.
Data Collection Method |
Respondents (Population Estimate) |
||||||||||
ATTC Directors1 N= 15
|
ATTC Staff2 N≈ 60 |
ATTC Field Staff3 N≈ 35 |
ATTC Advisory Board Members4 N≈ 450 |
SSA Directors5 N= 55 |
Provider Association Directors6 N= 43 |
Addiction Educators7 N= 536 |
Treatment Agency Directors8 N≈ 10,000 |
Other Key Advisors |
ATTC Activity Participants9 N≈ 30,000 |
Other Unique Regional Partners |
|
Site Visits |
15 |
38 |
35 |
|
|
|
|
|
|
|
|
Key Informant Interviews |
|
|
|
45 |
55 |
43 |
70 |
42 |
42 |
|
|
Collaborative Functioning Survey |
|
15 |
|
450 |
|
|
|
|
|
|
|
Customer Satisfaction & Benefit Survey |
|
|
|
|
55 |
43 |
158 |
700 |
|
3000 |
168 |
Evidence-Based Critical Action Surveys |
|
|
|
|
|
|
|
|
|
840 |
|
Success Case Interviews |
|
|
|
|
|
|
|
|
|
168 |
|
Clinician Self Assessment Forms |
|
|
|
|
|
|
|
|
|
72 |
|
Organizational Readiness for Change Survey |
|
|
|
|
|
|
|
240 |
|
|
|
1: Each of the 15 ATTCs (14 Regional Centers and 1 National Coordinating Center) has a Director.
2: Estimated that each ATTC has on average 4 core staff, and that each ATTC advisory board has an ATTC representative.
3: We are estimating that each of the 14 ATTCs has 2- 3 field staff that work outside of the ATTC home office.
4: We are estimating an average Advisory Board membership of 30 people.
5. Each state, as well as Puerto Rico and the Pacific Island jurisdictions, has a Single State Authority.
6. There are currently 43 Provider Associations that are members of the national association, SAAS (List of SAAS Member State Provider Associations, 2007).
7. There are currently 536 identified Addiction Education College and University Programs in the United States (Source: /www.nattc.org/degrees/search.asp).
8. There are currently over 10,000 addiction treatment centers in the United States (Source: www.findtreatment.samhsa.gov/).
9. We are estimating that each ATTC serves approximately 1000 people per year (Source: www.nattc.org ).
Collaborative Functioning Survey: A survey of Collaborative Functioning will be administered to members of the regional planning advisory groups of each ATTC region. It is expected that many members of these groups will also be among the key informants to be interviewed for their perceptions of the ATTC’s planning processes. This overlap in respondents will provide useful opportunities to validate the survey results and further elaborate findings. We will administer the survey at the time of each ATTC’s regional advisory board meeting, thus helping to ensure a high response rate. CSAT’s understanding is that these advisory groups can include 20–40 professionals, so CSAT’s overall sample size is expected to include approximately 450 addiction related professionals. In addition, we expect that each advisory group will have at least one ATTC staff member who would also take the survey.
Customer Satisfaction and Benefit Survey: Some important stakeholders are common among all regions; others are involved in ATTC planning and activities in only some regions. For this survey, census sampling within each region will be employed with those stakeholders that are common among all regions (SSA personnel (n= 55) and senior leadership of State treatment provider associations (n= 43)). Region-specific stakeholders (e.g., cultural leaders, other professional associations) and addiction educators will be included on a sampling basis (n=12 per regional center, for each of these two groups). Furthermore, unlike the key informant interviews, local treatment agency staff will be included as respondents to this Satisfaction and Benefit survey. Treatment agency administrators (n=50 per regional center, N=700 total)—whose staff have attended ATTC activities—and activity participants themselves (e.g., clinical/counseling staff, n=200 per regional center and the national office, N=3,000 total) will be included on a sampling basis within each region.
Evidence-based Critical Action Surveys: Each ATTC will be required to participate in one of the three Change Studies (Clinical Supervision, Motivational Interviewing, or Treatment Planning M.A.T.R.S.). The National Evaluation team will work closely with the directors and evaluators of the ATTCs selected to participate in each of the Change Studies to devise a suitable sample of ATTC activities and participants from which to select a sample for this survey activity. CSAT’s estimates indicate that we will have a sample size of 840 participants across the 3 Change Studies. Each participating center will need to include approximately 40-60 training participants depending on which Change Study they participate in. For example, if six ATTCs are selected to participate in a given Change Study, we will plan for N=240 participants to be surveyed across all regions, averaging n=40 participants per region. Prior consent for this survey and potentially a follow-up interview will be an explicit part of this process, in concert with IRB requirements in the selected regions.
Success Case Interviews: This methodology calls for selection of a sample of successful and less successful implementers following exposure to ATTC training or technical assistance in a given practice. Based on the results of the Critical Action Surveys, the evaluation team will select a 10% sample representing the highest scorers (most successful implementers) and a 10% sample representing the lowest scorers (less successful) on the implementation scales of the Critical Action Surveys. Based on this approach, for each region participating in a Change Study, 20% of all participants will be selected for these in-depth qualitative interviews. It is estimated that approximately N=168 participants will be interviewed across the Network, an average of n=12 per region.
Organizational Readiness for Change (ORC) Survey: For the Assessment-based Treatment Planning Change Study, the evaluation team will administer the ORC to administrators from all the agencies that participate in ATTC services relating to this activity emerging from the NIDA/SAMHSA Blending Initiative. CSAT’s estimated sample size (n= 240) is based on the assumption that each training participant will represent a different treatment agency, and subsequently one agency director.
For data collected through each of these data collection methods, we will conduct sub-group analysis where sample sizes allow, such as reporting findings by type of respondent (e.g., ATTC staff, state agency staff, providers, and other stakeholders; ATTC customers vs. stakeholders who have not participated in ATTC services), longevity of participation in the ATTC Network, and so on. The analysis of site visit data, key informant interview data, and focus group data will rely on qualitative analytic techniques, such as coding data to extract themes across ATTC regions.
The list of instruments, above, does not include the collection and analysis of quantitative and qualitative archival data, which will help to answer evaluation questions related to customer satisfaction, the extent to which ATTC services meet regional and national needs, and the process of technology transfer. (These archival data collection activities are not included in the burden table, because they do not impose a burden on respondents.) The National Evaluation will also conduct ongoing reviews of existing documents that illuminate ATTC regions’ approaches to planning their services and engaging key stakeholders as partners in carrying out technology transfer activities. The National Evaluation will also conduct secondary analyses of GPRA data for each region and the Network as a whole. As data on the characteristics of the regional workforce in all regions becomes available via the ATTC workforce surveys (which CSAT now requires all ATTCs to implement), the characteristics of participants in ATTC activities obtained via GPRA data analysis will be compared to characteristics of the entire State or regional workforce to determine whether ATTC activity participants are representative of the workforce as a whole. Finally, the National Evaluation will analyze ATTC website statistics to determine what products, reports, and other resources are most frequently viewed and downloaded, the length of time visitors are engaged on the websites, and where they are from (e.g., State agencies, local provider organizations).
The analysis of survey data (e.g., Customer Satisfaction and Benefit Survey, Collaborative Functioning Survey, Critical Action Surveys) will be both descriptive and correlational. For example, item and scale-level results will be summarized using descriptive statistics such as sample proportions, means, and standard deviations. In addition, because we will be administering the Collaborative Functioning Survey twice during the evaluation period, it will be possible to analyze change over time using statistical correlation and analysis of variance (ANOVA).
Survey items related to satisfaction and benefit will be summarized for each region and be correlated with participant characteristics to determine statistically significant relationships. For example, analyses may focus on whether self-perceived benefit from ATTC services relate to such characteristics as the number of years the participant has been in the field, or the number of services that have touched the participant, or by education level of the participant. We also may conduct statistical tests (e.g., t-tests and chi squares) to determine whether there are significant differences between groups on certain process and outcome variables.
The data collection activities of the evaluation are shown in the table below. The table identifies the data collection activities, respondents, and the key data to be collected. The evaluation team will work with the ATTCs to coordinate data collection involved with site visits, the Collaborative Functioning Survey, and all measurement related to the evidence-based critical action Change Studies.
Respondent |
Mode |
Timeline |
Key Data |
ATTC Directors and Staff
|
Site Visit Protocol and Interview Guide |
Summer/Fall 2008 |
ATTC goals and objectives Regional priorities National priorities Planning processes Collaborative/partnership relationships ATTC organizational structure Funding Coordination of services Tech transfer strategies Cultural adaptation in service delivery |
ATTC Technology Transfer Specialists
|
Focus Group Protocol |
Summer/Fall 2008 |
Regional priorities Planning processes Coordination of services with other providers Cultural adaptation in service delivery |
SSA Directors ATTC Advisory Board Members Provider Association Directors Addiction Educators Treatment Agency Directors |
Key Informant Interview Protocol |
October 2008 – September 2009 |
Collaboration with ATTCs for activity planning and service delivery Awareness of ATTC services Utilization of ATTC services Service quality Changes in awareness, skill or systems Gaps in ATTC services |
ATTC Regional Advisory Board Members |
Collaborative Functioning Survey |
July – September 2008 Follow-Up: October – December 2009 |
Characteristics of communications with ATTC Satisfaction with planning processes |
ATTC Customers |
Customer Satisfaction and Benefit Survey |
October 2008 – September 2009 |
Participation in ATTC activities Satisfaction with activities Changes in awareness, skills, and practices |
Participants in ATTC Activities |
Critical Action Surveys:
|
Spring 2009 |
Prior training on evidence-based practice (EBP) Prior use of EBP Changes in practice Proficiency level Factors affecting ability to implement practice |
Participants in ATTC Activities |
Success Case Interview Guides:
|
Summer/Fall 2009 |
Application of what was learned Characteristics of training that enabled application Barriers to application |
Participants in Training on Motivational Interviewing |
Clinician Self-Assessment Form |
Summer/Fall 2009 |
Motivational Interviewing (MI) style Implementation of MI |
Provider Association Directors |
Organizational Readiness Survey |
Summer/Fall 2009 |
Characteristics of provider organization Technical assistance needs |
The timeline for the data collection procedures is provided in the above table, and also under A16c, above.
Compliance with the National Evaluation of the ATTCs is a condition of the ATTC grants, which have been awarded as cooperative agreements. The program announcement under which applications were awarded states:
An independent evaluation of the ATTC program is being designed through a CSAT contract and will be implemented in FY 2008. Grantees are expected to participate in the evaluation. It is expected that approximately .25 FTE will be required to carry out some data collection as part of the independent evaluation. Applicants should budget for this.
In addition, the evaluation team will use the following techniques to ensure a high response rate:
Coordination with Regional ATTCs When Scheduling Data Collection Activities: As referenced in B2, all data collection related to site visits, the Collaborative Functioning Survey, and all measurement related to the evidence-based critical action Change Studies will require coordination between the evaluation team and the ATTCs. All such coordination will be done well in advance so ATTCs have time to prepare for data collection, and so that they and their local evaluators fully understand what is being asked. This process will be aided greatly by the participatory nature of the project to this point, as the ATTC Network already understands that coordination will be required. In fact, the ATTC Program Announcement required each regional center to set aside .25 FTE for such coordination with the National Evaluation.
Ensuring Good Data Collection Practice: The National Evaluation team will establish sound data collection procedures that conform to sound research principles. All data collection will, of course, be guided by data collection protocols; all data collectors will be trained; data collection timelines will be coordinated with respondents in flexible manner; and adequate follow-up procedures will be in place.
Use of Incentives: see A9.
A limited number of key informant interviews with ATTC directors and other program stakeholders were conducted during the evaluation design phase. These stakeholders were eager to participate in the National Evaluation and confirmed CSAT’s ability to obtain data about ATTC processes and outcomes through similar types of interviews. During these earlier interviews, we did not identify any problems with questions being answered inappropriately and did not receive negative feedback from respondents about the interview process or specific questions.
To obtain a true estimate of the burden associated with these interviews and to assess the efficacy of specific questions, many questions included in the Site Visit Protocol, Focus Group Protocol, and Key Informant Interview Protocol were pre-tested with ATTC program stakeholders. These prior telephone interviews, lasting one hour or less, verified that most stakeholder interviews could be conducted in person or by phone in a one-hour period. It was determined that two hours would be required for interviews with ATTC directors.
Two of the instruments—the Organizational Readiness for Change (ORC) Survey and the Motivational Interviewing Adherence, Clinician Self-Assessment Form—are validated instruments that have been used numerous times for other studies. In addition, Evidence-Based Critical Action Surveys, as well as Success Case Interviews, have been used by the evaluation team in other related evaluation projects (e.g., a local evaluation of the Northwest Frontier ATTC). Therefore, it is proven that these instruments can be used successfully with the types of respondents that will participate in the National Evaluation of the ATTCs. The Critical Action Surveys have also been reviewed by experts in the evidence-based practices, which are the focus of these surveys.
Several individuals from MANILA Consulting Group, and its subcontractors RMC Research Corporation and Abt Associates Inc., participated in the development of the evaluation plan. The evaluation team from these three organizations includes persons with knowledge of statistical methods and expertise in the evaluation of national substance abuse treatment and prevention programs. Roy Gabriel, Ph.D., of RMC Research Corporation, is the Principal Investigator of the National Evaluation. Richard Finkbiner, Ph.D., of MANILA Consulting Group, is the Project Director and point of contact for this project. He can be contacted at:
Richard Finkbiner
MANILA Consulting Group
6707 Old Dominion Road, Suite 315
McLean, VA 22101
Telephone: 571-633-0335, ext. 206
Fax: 703-356-0975
The Government Project Officer for the national evaluation is:
Deepa Avula
Center for Substance Abuse Treatment
Substance Abuse and Mental Health Services Administration
1 Choke Cherry Road, Room 5-1148
Rockville, MD 20850
Telephone: 240-276-2961
Fax: 240-276-2960
LIST OF ATTACHMENTS
A. Addiction Technology Transfer Centers Initial Program Announcement
B. Site Visit Protocol and Interview Guide
C. Focus Group Protocol
D. Key Informant Interview Protocol
E. Collaborative Functioning Survey
F. Customer Satisfaction and Benefit Survey
G. Evidence-Based Critical Action Survey on Clinical Supervision
H. Evidence-Based Critical Action Survey on Motivational Interviewing
I. Evidence-Based Critical Action Survey on Treatment Planning M.A.T.R.S.
J. Success Case Interview Protocol on Clinical Supervision
K. Success Case Interview Protocol on Motivational Interviewing
L. Success Case Interview Protocol on Treatment Planning M.A.T.R.S.
M. Organizational Readiness for Change Survey
N. Motivational Interviewing Adherence, Clinician Self-Assessment Form
O. Comment Received from the 60-day Federal Register Notice
1 Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22(4), 197-209.
2 Martino, S., Ball, S.A., Gallon, S.L., Hall, D., Garcia, M., Ceperich, S., Farentinos, C., Hamilton, J., and Hausotter, W. (2006) Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency. Salem, OR: Northwest Frontier Addiction Technology Transfer Center, Oregon Health and Science University.
3 U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration (2006), Report to Congress: Addictions Treatment Workforce Development, Rockville, MD.
4 Armstrong, J. (1975) “Monetary Incentives in Mail Surveys,” Public Opinion Quarterly, 39(1): 111-116; Church, A. (1993) “Estimating the Effect of Incentives on Mail Survey Response Rates: a Meta-analysis,” Public Opinion Quarterly, 57(1): 62-79; Goyder, J. (1994) “An Experiment with Cash Incentives on a Personal Interview Survey,” Journal of the Market Research Society, 36(4): 360-366.
5 James, T. and Bolstein, R. (1990) “The Effect of Monetary Incentives and Follow-up Mailings on the Response Rate and Response Quality in Mail Surveys,” Public Opinion Quarterly, 54(3): 346-361; Brennan, M. (1992) “The Effect of Monetary Incentives on Mail Survey Response Rates: New Data,” Journal of the Market Research Society, 34(2): 173-177; and Shettle, C. and Mooney, G. (1999) “Monetary Incentives in US Government Surveys,” Journal of Official Statistics, 15(2): 231-250.
| File Type | application/msword |
| File Title | MEMORANDUM |
| Author | Monica Capizzi |
| Last Modified By | SKING |
| File Modified | 2008-03-28 |
| File Created | 2008-02-25 |