Final_VERSION_SS_OMB_8_31_11

S

UPPORTING STATEMENT FOR THE ADDICTION TECHNOLOGY TRANSFER CENTERS (ATTC) NETWORK NATIONAL ADDICTION TREATMENT

WORKFORCE SURVEY

A. JUSTIFICATION

1. Circumstances of Information Collection

1. The Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) is requesting the Office of Management and Budget’s (OMB) approval for a new data collection entitled, “Vital Signs - Taking the Pulse of the Addiction Treatment Profession: An ATTC Workforce Survey Sponsored by SAMHSA-CSAT.” SAMHSA’s CSAT funds this program under legislative authority of Section 509, Priority Substance Abuse Treatment Needs of Regional and National Significance, of the Public Health Service Act, as amended. CSAT is requesting approval to implement this new workforce data collection effort through the ATTC Network. CSAT intends to use three (3) instruments to collect original data related to understanding and guiding America’s substance abuse treatment workforce efforts. This data collection effort will provide guidance to organizations, programs, states, and regions in improving their own workforce efforts and should also provide comprehensive, benchmark data to supplement existing N-SSATS and other national data sets. These three (3) instruments will be outlined in section two (2) and include:

• A Clinical Director Survey (in both a paper and online format)

• Key Informant Telephone Interviews of Clinical Directors

• Key Informant Telephone Interviews of Thought Leaders

The survey instrument can be seen as attachment 1. The key informant telephone interview questionnaires can be seen as attachment 3. In addition, CSAT will also utilize existent data sets to model the projected growth or retraction and characteristics of the substance use disorders workforce. These data sets are listed as attachment 4.

Estimates of annual counseling staff turnover in substance use disorders treatment programs range from 18.5 percent (Johnson, Knudsen, & Roman, 2002; Gallon, Gabriel, & Knudsen, 2003) to as high as 50 percent (Carise, McLellan, & Gifford, 2000; McLellan, Carise, & Kleber, 2003). High turnover rates add to provider training and recruitment costs and potentially threaten the quality of care received by clients entering substance use disorder treatment (Mor Barak, Nissly, & Levin, 2001; Lum, Kervin, Clark, Reid, & Sirola, 1998; McLellan, Carise, & Kleber, 2003). The Institute of Medicine (2001) highlights continuity of care as a critical element of primary care. The assumption is that continuity is associated with improved quality and decreased costs. There is, however, limited data to support these assumptions or to develop interventions to improve workforce retention. Both the SAMHSA “Strengthening Professional Identity” (Abt Associates, 2007) and the 2007 Annapolis Coalition “An Action Plan for Behavioral Health Workforce Development” reports repeatedly point to the lack of valid data to inform behavioral health workforce practices and initiatives. Both reports conclude it is imperative to build a strong workforce knowledge base, especially in relation to the effectiveness of existing strategies and practices that enable retention of qualified professionals, leading to improved treatment outcomes for clients in substance use disorder treatment programs.

The ATTC Network, a nationwide, multidisciplinary resource that draws upon the knowledge, experience and latest research of recognized experts in the field of addictions, is a CSAT initiative formed in 1993 in response to a shortage of well-trained addiction professionals in the public sector. The ATTC Network works to enhance the knowledge, skills and aptitudes of the addiction treatment and recovery services workforce by disseminating current health services research from the National Institute on Drug Abuse, National Institute on Alcohol Abuse and Alcoholism, National Institute of Mental Health, Agency for Health Care Policy and Research, National Institute of Justice, and other sources, as well as other SAMHSA programs. To accomplish this, the ATTC Network (1) develops and updates state-of-the-art research based curricula and professional development training, (2) coordinates and facilitates meetings between Single State Authorities, Provider Associations and other key stakeholders, and (3) provides ongoing technical assistance to individuals and organizations at the local, regional and national levels.

Currently, CSAT funds a network of fourteen Regional Centers and a National Office, which serve all 50 states, the District of Columbia, Puerto Rico, the U.S. Virgin Islands, and the Pacific Islands (Attachment 5). Of the fifteen sites, twelve are located at academic institutions, two are located within nonprofit institutes, and one is a project of a state certification board. Although the individual sites vary in the number of states served and areas of emphasis, each is charged, as is the Network as a whole, with seven key objectives:

• Building and maintaining collaborations and facilitating systems change to improve the provision of substance use disorders treatment and recovery services among key organizations and groups;

• Using innovative technologies to create partnerships with, and disseminate current behavioral health research from various federal sources;

• Developing and updating state-of-the-art research-based curricula;

• Enhancing the clinical and cultural competencies of professionals in a variety of disciplines to address the treatment/recovery needs of individuals with substance use disorders;

• Upgrading standards of professional practice for addictions workers in a variety of settings;

• Serving as a technical resource on state-of-the-art treatment and recovery services for community-based and faith-based organizations, consumers and recovery organizations, and other stakeholders; and

• P

  
  

  roviding feedback from the field to CSAT regarding the development of a comprehensive agenda for learning about and applying state-of-the-art treatment approaches.

2. Purpose and Use of Information

In response to the emerging shortages of qualified addiction treatment and recovery services professionals, SAMHSA/CSAT instructed the ATTC National Office to lead the ATTC Network in the development and implementation of a national addiction treatment workforce data collection effort. The purpose of the survey and key informant interviews is to gather information to guide the formation of effective national, regional, state, and organizational policies and strategies aimed at successfully recruiting and retaining a sufficient number of adequately prepared providers who are able to respond to the growing needs of those affected by substance use problems and disorders. This data collection effort will offer a national and regional perspective on the clinical treatment field so that CSAT and the ATTC Network can better understand current successful strategies and methodologies being used in the workforce and develop appropriate training for emerging trends in the field.

Although SAMHSA/CSAT is the primary target audience for data collection findings, it is expected that the data collected and resulting reports will also be useful to the ATTC Network, as well as to Single State Agencies, provider organizations, professional organizations, training and education entities, and individuals in the workforce.

Overview of Data Collection and Purposes

Data will be collected from two main sources: 1. A random sample of clinical directors or a designated direct care supervisor from facilities listed in the I-SATS database. 2. A national sample of key thought leaders, identified by CSAT in conjunction with the ATTC network, in the substance use disorders treatment field. Clinical Directors will be asked to participate in at least one of the following distinct methods:

• A web-based Clinical Director Survey (also available in paper format)

• Key Informant Telephone Interviews

Thought Leaders in the field of substance abuse treatment will be invited to participate in only one method of data collection, Key Informant Interviews.

In addition to this original data collection, existing national data sets will also be utilized (Attachment 4). Such data systems will include:

· Census 2000 datasets
· National Survey of Substance Abuse Treatment Services (N-SSATS)
· SAMHSA Treatment Gap Projection Analysis
· Treatment Episode Data
· Bureau of Labor datasets such as Current Employment Statistics
· Annapolis Coalition Data

The original design of the study included two online focus groups, one for clinical directors and one for thought leaders. These focus groups were going to be carried out using a web-based software called IdeaScale.com. Upon further investigation into the feasibility of this approach, and based upon the recommendation of the Office of Management and Budget, the online focus group aspect of the study has been eliminated. Instead, the ATTC will concentrate efforts to collect qualitative data on the key informant interviews (see B.2). Please note that due to the deletion of the online focus groups, attachment 2, which originally included a web site screen shot of the web-based idea management software that was going to be used for the online focus groups, has been deleted as well.

Clinical Director Survey: The Clinical Director Survey (Attachment 1) asks 57 questions of the clinical director or a designated direct care supervisor (direct care refers to staff members who spend a majority of their time providing clinical care for clients with substance use disorders as their primary diagnosis). For the purpose of this survey, clinical director is defined as the person whose role it is to oversee direct clinical service delivery for this facility. The instrument asks respondents to report demographic information about both themselves and the direct care staff they supervise, information about the facility at which they currently work, as well as information about their job satisfaction, recruitment and retention strategies, clinician training and preparation, and staff turnover (Table 1).

Key Informant Interviews of Clinical Directors: Based on responses to questions 44, 47 and 51 in the quantitative survey (Attachment 1), respondents will be asked to participate in a key informant interview in order to provide qualitative data that will inform on key strategies to prepare, maintain and retain the workforce. While the quantitative survey will collect some basic information on potentially effective strategies used to prepare and recruit individuals to enter the workforce and encourage them to remain in the workforce, the qualitative data will further enrich our understanding of how treatment facilities are implementing those strategies. An interview script has been developed to guide the question formation for the interviews. This script was updated after the deletion of the online focus group aspect of the study (Attachment 3).

Key Informant Telephone Interviews of Thought Leaders: Based on the ATTC Network’s knowledge and experience with recognized experts in the field of addictions, a list will be compiled of relevant Thought Leaders to include Single State Authorities (SSAs), addiction treatment agency directors, academics, and policy makers. The purpose of these interviews is to enrich understanding surrounding current and future trends in substance use disorders treatment (Table 1). An interview script has been developed to guide the question formation for the interviews. This script was updated after the deletion of the online focus group aspect of the study (Attachment 3).

Overview of Questions Related to Data Collection

The objectives of the national addiction treatment workforce data collection effort are to understand the national demographics of the current workforce and how this differs across regions and states, in addition to exploring issues related to workforce development: 1. Staff training, recruitment and retention; 2. Professional development; and 3. Support for strategies and methodologies to prepare, recruit, retain, and sustain the workforce. To accomplish these objectives, CSAT outlined three primary questions to be addressed by the workforce data collection effort:

1. What are the basic demographics of the workforce?

For the purposes of the ATTC data collection effort, this means that we will comprehensively describe the workforce comprised of direct care staff, clinical supervisors, and administrators in agencies represented in the Inventory of Substance Abuse Treatment Services (I-SATS).

2. What are the anticipated workforce development needs in the next five years?

For the purposes of this data collection effort, the ATTC Network will identify the growth and capacity-building needs over the next five years of direct care staff, clinical supervisors, and administrators in agencies represented in the I-SATS registry.

3. What are the common strategies and methodologies to prepare, retain, and maintain the workforce?

Identification of potentially effective strategies used to prepare and recruit individuals to enter the workforce (as previously defined), and encourage them to remain in the workforce and stay current on clinical and other job related skills (e.g., evidence based practices).

This will be the first national survey of the substance use disorders treatment workforce. The quantitative survey and the qualitative interviews and analysis will be used to provide a snapshot of the current state of the addiction treatment workforce as it relates to demographics, workforce development needs, and retention and maintenance of a strong workforce. These data will provide national benchmark data that can be used to inform ongoing policy and practice.

Information collected from this workforce data collection effort will help CSAT and the ATTC Network to better understand the needs of the workforce and categorize some best practices for providing support to the field now and in the future. Emerging trends in addiction treatment will be identified and shared with those in the addiction treatment field so appropriate training and funding can be allocated. The information from this data collection effort will also help CSAT identify areas where deficiencies in substance use disorder treatment exist and provide assistance to regions (and states) to help them develop and adopt strategies for addressing this.

T

able 1: Data Collection Methods

Method	Timeline	Sample	Type of Information
Secondary data analysis of existing substance use disorders treatment workforce data sets (quantitative)	July, 2011 through October, 2011	No unique data collection	Utilize existent data sets to model the projected growth or retraction and characteristics of the substance use disorders workforce. This will provide a comprehensive projection of the demographics of the workforce and treatment program needs over the next five years. (See Attachment 4 for more detail on the secondary data sets included). Answers part of question 2.
Survey (quantitative): Clinical Director Survey (available in paper and web-based formats)	October, 2011 through January 2012	Clinical directors or a designated direct care supervisor	The survey will ask participants to report demographic information about both themselves as well as those they supervise. Questions will also cover information about the current facility/agency for which they work, professional development, job satisfaction, retention strategies, clinical supervision practices and use of technology (Attachment 1). Answers question 1.
Key Informant Telephone Interviews (qualitative)	October, 2011 through February, 2012	Thought Leaders & Clinical Directors	Enriched understanding of the quantitative data on current and future trends in substance use disorders treatment as well as effective workforce development, recruitment, and retention strategies (Attachment 3). Answers part of questions 2 and 3.

3. Use of Information Technology

This workforce data collection effort will utilize a web-based version of the survey (Clinical Director Survey) to eliminate paperwork. This web based application will comply with the requirements of Section 508 of the Rehabilitation Act to permit accessibility to people with disabilities. All data collected will also be submitted and managed in electronic databases.

Web-based Version of the Clinical Director Survey

The Clinical Director Survey will be administered using the on-line survey software, Survey Monkey. All facilities listed in the ATTC database currently submit web-based data and initial consultation suggests that 100% of respondents will have access to the Internet to submit surveys electronically. Respondents who do not have access to the Internet will be sent a paper copy to mail back to the ATTC. We anticipate approximately 100% of our respondents will use Survey Monkey to complete the survey (Attachment 1).

Electronic Data Management

The survey data will be managed in electronic databases and while each ATTC Regional Center is responsible for solicitation of responses from the respondents in their region, all data will be sent to the ATTC National Office. Data collected on the survey instrument will be entered into the on-line database maintained by the ATTC National Office. Once data are entered into the system, it will be available to CSAT for review. These data can also be downloaded by the Regional Centers for their use. The ATTC National Office will keep a record of the data collected, and merge files for data examination as requested.

4. Efforts to Identify Duplication

The data to be collected are unique and are not otherwise available. In 2009, a recent literature review (2003-2008) relevant to workforce issues in the substance use disorders treatment field was conducted (Attachment 6). This report utilized all the workforce materials from a variety of sources with a focus on 2003-2008. This included surveys and reports from the Addiction Technology Transfer Center (ATTC) Network in addition to government-funded reports, studies, and white papers from myriad professional groups and coalitions. Initially, this included a computerized bibliographic search of databases including EBSCO, LexisNexis Academic, MEDLINE, Web of Science (Social Sciences Citation Index), PubMed and PsycINFO.

While the workforce literature is clearly growing due to a renewed focus by SAMHSA/CSAT, there is still a dearth of standardized data on substance use disorders treatment agencies. The workforce materials currently in existence demonstrate inconsistent methodologies, poor response rates, and lack the scope necessary to draw conclusions and/or comparisons across states, regions, or even provide a snapshot of what the pertinent workforce issues are nationally.

5. Involvement of Small Entities

Participation in the CSAT/ATTC workforce data collection will not be a significant burden on small businesses, small entities, states, local governments, or on their workforces.

6. Consequences If Information Collected Less Frequently

Participation is voluntary. Each participant is asked to respond only once to the survey or interview data.

7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

This information collection fully complies with the guidelines in 5 CFR 1320.5(d)(2).

8. Consultation Outside the Agency

The notice required by 5 CFR 1320.8(d) was published in the Federal Register on March 31^st, 2011, Volume 76, page 17931.

In developing the survey instrument a small group of 9 potential respondents were chosen to consult and give feedback on the survey response burden, the quality of the questions, the quality of the response choices and general thoughts about the information being gathered by the survey. The following people (separated by ATTC region) offered feedback:

Pacific Southwest Region:

Danise Leveille, LPC.

Education & Training Director

Community Bridges, Inc.
310-267-5519

[email protected]

www.communitybridgesaz.org

Tim Davis, PSW

Chief Clinical Officer

West Yavapai Guidance Center

310-267-5519

[email protected]

Prairielands ATTC:

Kurt Snyder                             

Executive Director

Heartview Foundation
[email protected]
701-222-0386

Andi Johnson
Director of Operations

Sharehouse                    

[email protected]
701-282-6561

Michaelene Spence
Clinical Director

Waysidehouse
952-405-7644
[email protected]           

Jay Hanson
Executive Director

Prairieridge

[email protected]

641-424-2391

Rick Swanson
Executive Director

Alcohol and Drug Dependency Services of Southeast Iowa

[email protected]

319-753-6567

Central East ATTC:

Cydney, Savage
Director, Adult Outpatient Counseling Services

Atlanticare Behavioral Health
[email protected]

609-646-5142

Mid-Atlantic ATTC:

Hannah K. Knudsen, Ph.D.
Assistant Professor
University of Kentucky
College of Medicine
859-323-3947
[email protected]

9. Payment to Respondents

There will be no payment or incentives to clinical directors or designated direct care supervisors for completing the survey or telephone interviews.

10. Assurance of Confidentiality

Each survey response will have a unique identification code,

which will allow respondents to be tracked and data to be sorted by state and region.

11. Questions of a Sensitive Nature

There are no questions of a sensitive nature.

12. Estimates of Annualized Hour Burden

The total annualized burden to an estimated 619 total respondents for the national addiction treatment workforce data collection is estimated to be 594 hours. Burden estimates are based on initial outside survey consultation with 9 potential respondents (see section A.8), along with previous use of related data collection instruments (surveys, on-line focus groups, and telephone interviews) by the ATTC Network. The annualized hourly costs to respondents are estimated to total $13,727.39. As no data is available specifically for clinical directors or direct care supervisors, hourly wage information is based on estimated 75^th percentile median hourly wages of $23.11 an hour for substance abuse and behavioral disorder counselors as reported in the May 2008 Occupational Employment Statistics available from the Bureau of Labor Statistics, U.S. Department of Labor (Attachment 7; http://www.bls.gov/oes/current/oes211011.htm). There are no direct costs to respondents other than their time to participate. Burden estimates are detailed in Table 2.

Table 2: Annualized Burden Estimates

Type of Respondent	Number of Respondents	Responses per Respondent	Hours per Response	Total Annual Burden Hours	Hourly Wage Cost	Total Hour Cost
Clinical directors or supervisors Web-based version of survey	569	1	1	569	$23.11	$13,149.59
Clinical directors or supervisors Telephone interviews	25	1	.5	12.5	$23.11	$288.90
Thought leaders Telephone interviews	25	1	.5	12.5	$23.11	$288.90
TOTAL	619			594		$13,727.39

13. Estimates of Annualized Cost Burden to Respondents

There are neither capital or startup costs nor are there any operation and maintenance costs.

14. Estimates of Annualized Cost to the Government

The annual estimated cost to the government for the national addiction treatment workforce data collection is $40,870.  Approximately $20,000 represents ATTC staff time to implement the survey. The 15 ATTCs are funded entirely through SAMHSA cooperative agreements.  The other approximately $20,000, represents consulting fees to employ a statistical consultant.  $870 per year represents SAMHSA costs to oversee the project for 1% of one employee (GS-13). Adding in the $32,000 in costs related to the sampling among regions, this brings the total estimated costs to $72,870.  These costs include both the national and regional survey costs, telephone interview costs, and regional costs of collating data.  Estimated costs for the survey data are based on the conservative assumption that at least half of the national sample would fulfill the sampling needs of the regional sample and we are focused on observing a small to medium effect size (.15-.25).  These costs may fluctuate and decrease by approximately $3,000 if the national sample is diverse; reducing the need for a sample larger than 437 (Table 6; B1).  These costs may also increase by approximately $5,000 if the national sample lacks regional representation.

1

5. Changes in Burden

This is a new project.

16. Time Schedule. Publication and Analysis Plans

Time Schedule

Data collection is projected to take place from October 2011 to February 2012. Table 3 outlines the timeline for each activity and how this relates to the workforce data collection questions.

T

able 3: Schedule of Key Activities and Relation to Questions

Activity & Participants	Timeline	Relation to questions (Q)
Secondary data analysis of existing substance use disorders treatment workforce data sets (quantitative)	July, 2011 through October, 2011	Related to Q2: What are the anticipated workforce development needs in the next five years? Utilize existent data sets (Attachment 4) to model the projected growth or retraction and characteristics of the substance use disorders workforce. This will provide a comprehensive projection of the demographics of the workforce and treatment program needs over the next five years.
Web-based version of Clinical Director Survey (quantitative): clinical directors or a designated direct care supervisor (also available in paper)	October, 2011 through January, 2012	Related to Q1: What are the basic demographics of the workforce? The survey will ask participants to report demographic information about both themselves as well as those they supervise. Questions will also cover information about the current facility/agency for which they work, professional development, job satisfaction, retention strategies, clinical supervision practices and use of technology (Attachment 1).
Key Informant Telephone Interviews (qualitative): Clinical Directors & Thought Leaders	October, 2011 through February, 2012	Related to Q2: What are the anticipated workforce development needs in the next five years? Also Q3: What are the common strategies and methodologies to prepare, retain, and maintain the workforce? Enriched understanding of the quantitative data on current and future trends in substance use disorders treatment as well as effective workforce development, recruitment, and retention strategies (Attachment 3).

Publication Plan

Before the end of September 2012, several publications will emerge based on these data:

• Each ATTC regional center will produce a regional report for CSAT, which outlines the results from the quantitative survey.

• The ATTC Network will produce a national report for CSAT that aggregates all of the regional data and integrates the qualitative results with the survey data.

• All of these reports will be disseminated through the ATTC Network web site.

Analysis Plan

This workforce data collection uses a mixed methods approach and will primarily focus on using the quantitative (survey) and qualitative (telephone interviews) to answer the three main questions:

1. What are the basic demographics of the workforce?

2. What are the anticipated workforce development needs in the next five years?

3. What are the common strategies and methodologies to prepare, retain, and maintain the workforce?

Quantitative Analysis

The purpose of the quantitative (survey) analysis is to provide a description of the current status of the direct care staff in the addiction field workforce (question 1). The statistics presented will be descriptive and aggregated at the national and regional ATTC level. These data will allow for comparisons between the ATTCs and the national averages on multiple variables, including items such as: the number of direct care staff in recovery, number of licensed direct care staff, recruitment and retention strategies, turnover rates, and training needs. Table 4 provides an example of how the data could be presented; this table would allow for comparisons to be drawn between the national average and the various ATTC regions, disaggregating the data by the size of the direct care staff.

Table 4: Example of a Table that Could Be Included in the Analysis

Specific Variables	ATTC: Pacific Southwest			National
	0-2 direct care staff	2-4 direct care staff	5+ direct care staff	0-2 direct care staff	2-4 direct care staff	5+ direct care staff
Number of facilities Staff characteristics Female Male In recovery Licensed in substance abuse counseling Program characteristics Average caseload (<10 clients) Average caseload (10-20 clients) Average caseload (20-30 clients) Average caseload (20-30 clients) Staff turnover Less than 10% 10-20% 20-30% Recruitment strategies Has formalized relationships with community colleges and universities Makes concerted effort to recruit from under-represented groups

Qualitative Analysis

Key-informant telephone interviews guided by a semi-structured interview format will be used to gather qualitative data from two samples (attachment 3). The first set of telephone interviews will gather particularly insightful or novel ideas from experts in the field of substance abuse treatment. These interviews will provide a national perspective on what mega trends are expected to affect the substance abuse treatment workforce in the next five years (see Thought Leader interview questionnaire in attachment 3). The second set of interviews will gather qualitative data on the key strategies used by substance abuse treatment facilities to prepare, maintain and retain the workforce from clinical directors (see Clinical Director interview questionnaire in attachment 3).

As Key informant interviews are conducted, they will be audio recorded and transcribed. As the ATTC Network has prior experience using QSR NVivo, this qualitative software will be used to code the data and divide it into meaningful segments of data. The coding scheme will be emergent as information is collected from Key Informants. Finally, the data will be divided in major themes or categories representing the findings of our qualitative research.

Integration of Qualitative and Quantitative Data

The data from the survey and key informant interviews will supplement each other to refine answers to the three questions. The qualitative data will be used to aid in the interpretation of the quantitative data by providing a narrative and commentary to support the findings. While the quantitative data predominantly focuses on “what is” in regards to the addiction field workforce, the qualitative data takes a more visionary approach to enable practical implementation of development strategies to meet the needs of the future addiction workforce.

17. Display of Expiration Date

The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.

18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

Overview of Facilities in Respondent Universe

As instructed by CSAT, the workforce data collection will sample facilities used in the Inventory of Substance Abuse Treatment Services (I-SATS) for the National Survey of Substance Abuse Treatment Services (N-SSATS). The N-SSATS is designed to collect data on the location, characteristics, services offered, and number of clients in treatment at alcohol and drug abuse facilities (both public and private) throughout the 50 States, the District of Columbia, and other U.S. jurisdictions. In 2008, N-SSATS data included approximately 13,500 facilities with about 1.1 million clients in treatment. The map (Attachment 8) displays the break down of these facilities across the 14 ATTC regions.

N-SSATS collects data from each physical location where treatment services are provided. Accordingly, a “facility” is defined as the point of delivery of substance abuse treatment services (i.e., physical location). Treatment facilities that are licensed, certified, or otherwise approved by the State substance abuse agency to provide substance abuse treatment make up the largest group of facilities. The survey also includes programs operated by Federal agencies—the Department of Veterans Affairs (VA), the Department of Defense, and the Indian Health Service. Together, these facilities represent about 80 percent of the total. The remaining facilities included in N-SSATS are those that are not licensed or certified through the State substance abuse agencies or Federal agencies. These facilities are usually hospital-based or private-for-profit facilities. N-SSATS does not include treatment programs in facilities that have solo practitioners or in jails or prisons (p.89, National Survey of Substance Abuse Treatment Services (N-SSATS): 2008). Each year, new facilities are added to the Inventory of Substance Abuse Treatment Services (I-SATS) by State agencies or when they are identified by examination of databases such as the one maintained by the American Hospital Association (http://www.oas.samhsa.gov/2k3/NSSATS/NSSATS.pdf).

Overview of Sampling Methods

The purpose of this workforce data collection is to collect original data related to understanding and guiding America’s substance abuse workforce efforts. The intent of the survey data is that it will be useful at both a national and regional (ATTC center) level. While a national dataset could show us how effective staff perceive specific recruitment and retention strategies to be, a regional and national database could show us which strategies work for specific populations and what professional development needs are both across the U.S. but also in specific areas. This would allow for more targeted training and recruitment approaches that meet the needs of not only the current workforce, but enable the ATTC Network to prepare for future workforce needs identified by regions.  In addition, having data that is useful to the regions and enables strategic decision-making enhances buy-in, making it much more likely that facilities will be encouraged to respond at higher rates and that results will be used (and not shelved). 

Consequently, the sampling method for the workforce survey will be a multi-phased approach, designed to ensure we have a dataset that is representative both nationally (Phase 1) and regionally (Phase 2).  

Phase 1: National Sample

Overview

The Phase 1 sample will be a simple random national sample of 450 substance use disorder facilities. The simple random sample will ensure we have a representative sample of U.S. substance use disorders organizations so data from the survey can be used to provide a snapshot of the current state of the workforce across the U.S. The power of the sample survey is its ability to estimate the distribution of different characteristics in the substance use disorders workforce population by obtaining information from relatively few organizations. This nationally representative sample will only allow the ATTC National Office to make descriptive statements regarding the national workforce population. As this is a random sample, it will not allow for any regional or state-level comparisons that could provide insights into future workforce needs in specific areas or differences across regions; this is the reason to include the Phase 2 data.

Determining Sample Size

To determine the sample size needed to ensure we have a representative sample of the approximately 14,056 substance use disorder organizations currently listed in I-SATS (Attachment 9), we conducted an a priori power analysis to take into account sampling error, confidence level, and any variance within the substance use disorders workforce across the U.S. Sampling error is the type of error that occurs due to not collecting information from all substance use disorders organizations. This equates to how precise the estimates should be and generally ranges from plus or minus three to ten percent. Confidence level describes the amount of confidence one wishes to have in the estimates made from the sample for the entire population. This data collection will assume a confidence level of 95%. To account for any variance across the workforce, we have used the most conservative value possible to assume maximum variation (Cohen, 1988; Dillman, 2000). As can be seen from the table below, a range of sample estimates have been developed at the 95% confidence level from least precise (plus or minus 10 percent) to most precise (plus or minus 3 percent) using the following equation:

Ns=_(Np)(p)(1-p)/_(Np-1)(B/C)² _{+ (p)(1-p)}.

Where:

Ns=completed sample size needed for desired level of precision

Np=size of population

P=proportion of population expected to vary on different characteristics

B=acceptable amount of sampling error; .03=plus or minus 3% of the true population value

C=Z statistic associated with the confidence level; 1.96 corresponds to the 95% level

These estimates can be seen in Table 5. This table shows the completed sample sizes needed to gain a nationally representative sample at three levels of precision.

Table 5: Estimates of Sample Size Needed for Nationally Representative Sample

	Sample size for the 95% confidence level with maximum variance (50/50 split)
Population size	 10% sampling error	 5% sampling error	 3% sampling error
14,056	96	374	989

National Sample Size

Decisions on final sample size to acquire a nationally representative sample were based on level of precision, anticipated response rates, and estimates of costs at different levels of precision (Attachment 10). The ATTC Network anticipates an 80% overall response rate for this workforce data collection. In 2008 the National Survey of Substance Abuse Treatment Services (N-SSATS) received a 94.1 % response rate, with 32 states or jurisdictions having response levels that equaled or surpassed the overall response rate. As the ATTC Network is targeting the same facilities as those in the N-SSATS, compliance is expected to be high. Furthermore the ATTC Network has a history of a strong positive relationship with participants who are receptive to the request for participation with a traditional single e-mailing methodology.

A national sample size of 450 substance use disorder facilities will provide a safety net to account for any issues of non-response or potentially unusable data. This will provide a high level of precision ( 5%) at a 95% confidence level allowing for maximum variance within the responses.

Phase 2: Regionally Representative Sample

Overview

The Phase 2 sample will allow for targeted sampling of 569 facilities across the 14 ATTC regions (minimum of 41 per region) to enable comparisons across regions (and the majority of states) on specific variables, such as workforce turnover rates, success in recruitment strategies, and direct care staff demographics. This will significantly supplement the national data by allowing for national averages to be compared with specific regions in addition to allowing for comparisons across regions. Unlike the national database created in phase 1, the phase 2 data will be more comprehensive and can be disaggregated to ensure cultural and geographic differences are accounted for in any analyses.

Determining Sample Size

As this dataset supplements the national sample, sample sizes needed for each of the ATTC regions will vary based on the initial respondents to the simple random sample necessary for the national sample. In contrast to the national random sample that allows every substance use disorders organization equal weighting to create a nationally representative sample, the regional sample will seek equal variance across regions to allow for comparisons to be made. Creating equal sample sizes will allow regions and the national office to conduct their own analyses with limited statistical skills. Unequal sample sizes across regions would require the ongoing assistance of statistical consultants. Consequently, once the nationally representative sample has been determined, phase 2 data will supplement the existing national data set to ensure we have sufficient numbers to allow us to make comparisons across regions (and states).

To determine sample size, a conservative analysis of covariance (ANCOVA) model with fixed effects, main effects, and interactions was utilized using G*Power 3.1 (Faul, Erdfelder, Lang, & Buchner, 2007). This model was selected above a basic ANOVA as it allows for a more conservative estimate of the sample size needed. As questions regarding interventions and other strategies may arise post data collection, it is prudent to allow for a conservative sample so we have the statistical power to defend comparisons at the regional level (Cohen, 1988).

Based on this ANCOVA model and using the same variance (50/50), sampling error range (10% to 3%), and confidence level as in the national sample, we created a range of sample estimates to demonstrate the sample size needed to detect small to medium effect sizes (0.15 to 0.25). These sample sizes are shown in Table 6.

Table 6: Estimates of Sample Size Needed for Regionally Representative Sample at Small and Medium Effect Sizes (Total Number Shown with Region in Parentheses)

	Sample size for the 95% confidence level with maximum variance (50/50 split)
Effect size	 10% sampling error	 5% sampling error	 3% sampling error
0.25	316 (23)	437 (31)	523 (37)
0.15	858 (61)	1192 (85)	1430 (102)

Regional Sample Size

Decisions on final sample size to acquire a regionally representative sample were based on level of precision, anticipated response rates, and estimates of costs at different levels of precision (Attachment 10). As previously noted, the ATTC Network anticipates an 80% overall response rate for this workforce data collection based on previous data collection efforts and existing relationships with facilities. A regional sample size of 569 substance use disorder facilities (41 per region) will provide a safety net to account for any issues of non-response or potentially unusable data. This will provide a high level of precision ( 5%) at a 95% confidence level allowing for maximum variance within the responses. To determine the regional sample size, a conservative analysis of covariance (ANCOVA) model with fixed effects, main effects, and interactions was utilized using G*Power 3.1 (Faul, Erdfelder, Lang, & Buchner, 2007). This model was selected above a basic ANOVA as it allows for a more conservative estimate of the sample size needed.

As questions regarding interventions and other strategies may arise post data collection, it is prudent to allow for a conservative sample so we have the statistical power to defend comparisons at the regional level (Cohen, 1988). Decisions on final sample size to acquire a regionally representative sample were based on level of precision, anticipated response rates, and estimates of costs at different levels of precision. As an effect of .15 would be considered small and result in significant cost increases, the regional sample size was focused toward detecting small (0.15) to medium (0.25) effects. The notes of caution added in Attachment 10 outlined potential fluctuations in the costs of sampling if we wished to consider small effect sizes such as .15. The extra costs for including the phase 2 regional data will be a minimum of $13,635 due to the fixed costs of 140 hours of ATTC regional work (10 hours per region for collating data), plus the statistical consulting needed for analysis. The costs associated with this study cannot simply be assessed by adding the costs for the national sample with the costs for the regional sample. It is anticipated that there will be significant overlap between the two samples, but as phase 1 (the national data) will be random, it is unlikely it will provide the individual regional sample numbers needed for phase 2 data. Looking at Attachment 9, one example for this is that the Pacific Southwest region has approximately 2082 substance use disorder organizations (15% of the total population) whereas the Southeast has 376 substance use disorder organizations (3% of the total population). If a random national sample of 989 selects 148 organizations (or 15%) from the Pacific Southwest region and only 30 (3%) from the Southeast region, the Southeast region may need to be supplemented in the Phase 2 data collection effort.

A fair and conservative estimate of extra costs associated with a regional data collection effort that focused on varying effect sizes could then approximate a range of $13,635 (for fixed costs) to a conservative estimate of $31,777 (based on the small effect size of 0.15, the 3% margin of error, and the conservative assumption that at least half of the national sample would fulfill the sampling needs of the regional sample).

2. Information Collection Procedures

Sample Selection

Quantitative Clinical Director Survey

T

he sampling design for the survey including power analysis and compensating for issues of non-response or potentially unusable data was outlined in B1. The first phase of this data collection will involve a simple random sample of 450 substance use disorder facilities for a nationally representative sample. In phase 2, this sample will be supplemented by a stratified random sample that is regionally representative to ensure that there is a minimum of 41 facilities included for each region (approximately 569 facilities in total). As instructed by CSAT, the workforce data collection will sample from 14,056 current facilities used in the Inventory of Substance Abuse Treatment Services (I-SATS) for the National Survey of Substance Abuse Treatment Services (N-SSATS).

All facilities will be given independent identifiers and included in a general sampling frame that also denotes region and state. The national (Phase1) sample will be a simple random sample that allows every substance use disorder organization equal weighting to create a nationally representative sample. Once these facilities have been drawn, the Phase 2 sample will stratify facilities by region. Each region will need a minimum of 41 facilities to create a regional sample with equal sample sizes that will allow for regional comparisons (without using complex statistical techniques). These facilities will be selected at random from within each regional stratification.

The sample sizes selected will allow sufficient power for detection of an effect size of .25 and will provide a high level of precision ( 5%) at a 95% confidence level allowing for maximum variance (50/50 split) within the responses. In addition to these conservative power estimates, we employed a +20% safety net on power estimate sample numbers to compensate for any issues of non-response or potentially unusable data. This will ensure we obtain sufficient sample sizes to meet the needs of any national and regional analysis.

Key Informant Telephone Interviews: Thought Leaders

For the qualitative data, the emphasis is on a deeper and more contextualized understanding of the workforce and their experiences. Therefore, there will be less restrictions surrounding sampling, as the emphasis will be on encouraging as much response as possible to build a more comprehensive picture of the workforce and gain enriched insights. Thought leaders in addiction treatment, which could include: Single State Authorities (SSAs), addiction treatment agency directors, academics, and policy makers, will be invited to participate in key informant interviews. These thought leaders would be identified by CSAT in conjunction with the ATTC Network. All 14 ATTC Regional Centers and the National Office will be invited to provide the contact information for 5 national and/ or regional experts in the field. This invitation will also be extended to ATTC Network liaisons at CSAT. With this list of approximately 80 individuals, the ATTC National Office will use a purposive sampling strategy to identify relevant respondents resulting in an expanded understanding and explanation of the research question. In this sense, the researcher will focus on selecting a range of leaders in the field of substance abuse and addictions treatment that will best inform the focus of this study. In order to provide diversity in responses, thought leaders will be invited to take part in a telephone interview based on their areas of expertise as it relates to the research question posed, and their geographic location. The invitation will be extended to approximately 45 thought leaders with an expected response rate of 80% (36 key informants). While we estimate 36 potential respondents, due to resource limitations, key informant interviews will cease once we reach saturation and it is deemed that no further insightful observations are being made. It is expected that saturation will be reached once 25 interviews have been conducted.

Key Informant Telephone Interviews: Clinical Directors

For the key informant interviews of Clinical Directors, an extreme case, purposive sampling strategy will be utilized.  Extreme case sampling is used to focus on cases that are special or unusual, typically in the sense that the cases highlight notable outcomes. These extreme cases are useful because they often provide significant insight into a particular phenomenon, which can act as lessons that guide future research and practice.  One of the prime objectives of this project is to identify potentially effective strategies used to prepare and recruit individuals to enter the workforce, and encourage them to remain in the workforce and stay current on clinical and other job related skills. The Clinical Director Survey will provide basic information relative to this goal (see questions 41 – 51 in Attachment 1).  In order to provide a richer understanding of the survey data, extreme cases will be systematically selected to serve as informants for the Clinical Director Interviews.  Extreme cases will be selected from the respondents to the clinical director survey that fall within the nationally representative sample of substance use disorder facilities.   Extreme cases will be selected by analyzing the responses to questions 44, 47 and 51 of the survey (Attachment 1).  Respondents who score an average of 4 or greater to all three of those questions will be invited to participate in the key informant interviews of Clinical Directors.  An average score of 4 or greater on all three of these questions will suggest that these respondents are highly satisfied with their agency’s recruitment and retention strategies.  Sampling and interviewing will continue until either informational saturation is reached, or, due to resource limitations, until 25 interviews have been conducted.

Information Collection Procedures

As outlined in A.2, the ATTC workforce data collection will collect original data using three distinct methods (Table 7): a web-based Clinical Director Survey (Attachment 1; for clinical directors or a designated direct care supervisor), and two Key Informant Telephone Interviews (Attachment 3).

Table 7: Data Collection Methods

Activity	Data Collection Method	Timeline
Quantitative
Clinical Director Survey (or a designated direct care supervisor)	Web-based survey (Survey Monkey); paper form available	October, 2011 through January 2012
Qualitative
Key Informant Interviews: 1.Clinical Directors 2. Selected participants recognized by CSAT and the ATTC Network as thought leaders in the addiction treatment field	Telephone interviews	October, 2011 through February, 2012

Data Collection Strategies

Three protocols have been established in order to guide the data collection process (Attachment 2): the executive director contact protocol for the survey, the clinical director contact protocol for the survey, and the key informant telephone interview protocols. The first two protocols relate to the quantitative (survey) portion of the data collection and the process for encouraging Clinical Directors (or their designated direct care supervisors) to complete the survey instrument. The final protocols relate to the qualitative portion of the data collection and describe how key informants will be contacted for a brief interview to provide an enriched understanding of the quantitative data on current and future trends in substance use disorders treatment as well as effective workforce development, recruitment, and retention strategies (Attachment 3).

Executive Director Contact Protocol (Survey)

The first step in the process will be for the ATTC Regional Center (RC) to make contact with the Executive Director (ED) of the treatment center in their region. Initially, an e-mail will be sent to all ED’s with an invitation for them to respond with the contact details for each of the Clinical Directors (CD) at their institution. If EDs do not respond to that e-mail contact will be made by phone requesting contact information. If the ED does not consent to provide contact information a phone call will be made to their SSA asking that they encourage the ED to consent. If the initial attempt to contact the ED fails, an attempt at contact will be made every two days on up to four occasions. If contact is not successful at that point a phone call will be made to their SSA encouraging them to respond.

Clinical Director Contact Protocol (Survey)

Initially an e-mail will be sent out to all CD’s asking them to respond with their preferred method of survey completion. If they do not respond a phone call will be made instead, asking what their preferred method of survey completion would be. An attempt will be made every two days for up to four occasions (TIER 1) to contact them by phone. If they cannot be contacted a call will be made to their ED asking them to encourage the CD to respond (TIER2). If they respond but do not complete the survey, TIER 1 and TIER 2 will be repeated. In addition to the initial e-mail to participate, a reminder e-mail will be sent to the CD. If the CD is unresponsive to e-mail initially, reminders will be made over the phone.

Thought Leader Contact Protocol (Key Informant Interviews)

An initial e-mail will be sent to each Thought Leader (TL) inviting them to participate in a telephone interview. If they do not respond, the e-mail will be resent once. If they respond and agree to participate they will be contacted by phone for a brief interview to discuss future trends in workforce development. If they respond and do not agree to participate, a follow up e-mail will be sent, once again providing them with the opportunity to participate. If they respond to the follow up e-mail and agree to participate, they will be contacted by phone for a brief interview. If they respond to the follow up e-mail and do not agree to participate, no further contact will be made. If they do not respond to any correspondence, an e-mail will be sent from CSAT encouraging them to participate

Clinical Director Contact Protocol (Key Informant Interview)

Once Clinical Directors complete the online survey (and fall within the sampling frame) an email will be sent inviting them to participate in a telephone interview. If they respond and agree to participate, they will be contacted by phone for a brief interview to discuss key strategies that effectively prepare, maintain and retain the workforce. If they respond and do not agree to participate, a follow up e-mail will be sent, once again providing them with the opportunity to participate. If they respond to the follow up e-mail and agree to participate, they will be contacted by phone for a brief interview. If they respond to the follow up e-mail and do not agree to participate, no further contact will be made. If they do not respond to any correspondence, an e-mail will be sent to their ED encouraging them to participate.

3. Methods to Maximize Response Rate

The ATTC Network anticipates an 80% overall response rate for this workforce data collection. In 2008 the National Survey of Substance Abuse Treatment Services (N-SSATS) received a 94.1% response rate, with 32 states or jurisdictions having response levels that equaled or surpassed the overall response rate. As the ATTC Network is targeting the same facilities as those included in the N-SSATS, compliance is expected to be high. Furthermore the ATTC Network has a history of a strong positive relationship with participants who are receptive to the request for participation with a traditional single e-mailing methodology.

Methods to maximize response rates include:

• Brief and easy to complete instruments that have been piloted with potential respondents;

• Clear reports summarizing findings to ATTC regional directors and CSAT;

• Summary reports of findings for broader dissemination to addiction treatment field;

• All ATTC regions will emphasize the importance of completing these instruments in order to improve the workforce situation in the future;

• E

  
  

  ase and convenience of electronic reporting; and

• The ATTC Network has a history of a strong positive relationship with participants who are receptive to the request for participation with a traditional single e-mailing methodology.

As outlined in the data collection strategies in B2, the ATTC Network has devised a number of protocols to ensure respondents receive adequate follow up in the event that the first attempt fails (Attachment 2). These strategies are also supported by clearly outlined contact scripts for both the quantitative (survey) data (Attachment 11) and the qualitative (telephone interviews) data (Attachment 12). As outlined in these attachments, a number of response methods will be available to each participant. The survey instrument will be available both by mail and on-line. Invitations to participate will be sent over the phone and by e-mail. The survey instrument is designed to be brief and easy to complete.

Respondents will not be offered payment for involvement in the data collection. It is expected that having emphasized the importance of completing these instruments, respondents will understand the data collection’s projected positive impact on the workforce in the future. These results will provide strategies that are useful to all substance use disorder facilities and a focus will be put on promoting the dissemination on these strategies to all facilities in the ATTC Network.

With help from CSAT and the Single State Directors (SSD), marketing of the data collection will be extensive. Respondents will receive postcards by e-mail informing them of the data collection and a letter will be sent to each ED from their SSD. Marketing materials will also be distributed at major conferences prior to the data collection.

The findings gathered by the workforce data collection will be summarized in a report that will be widely available to the addictions field. This report will also be submitted to CSAT and used by the ATTC Network in understanding and guiding America’s substance abuse treatment workforce efforts. Each ATTC region will also receive data specific to their region, allowing them to make informed decisions about future workforce strategies that will enhance recruitment and retention efforts.

4. Tests of Procedures

As listed in section A8, a small group of 9 potential respondents were chosen to consult and pre-test the survey instrument. These individuals provided feedback on the survey response burden, the quality of the questions, the quality of the response choices, and general thoughts about the information being gathered by the survey.

Once the survey and questionnaire instruments have been developed on-line, a group of nine or less individuals will be asked to pilot the instruments to ensure there are no logistical issues. It is not anticipated there will be any major changes to the instruments at that time. Should any changes occur, OMB would be informed of them expediently.

5. Statistical Consultants

The individuals contracted on statistical aspects of the design include:

Scott Churchill

Professor and Chair of the Psychology Department, University of Dallas

(972) 721-5348

[email protected]

Hannah K. Knudsen

Assistant Professor, Department of Behavioral Science, University of Kentucky

(859) 323-3947

[email protected]

 

Deena Murphy

Assistant Professor, Department of Interdisciplinary Studies, NC State University

919-349-4596

[email protected]

Shevaun Neupert

Assistant Professor, Department of Psychology, NC State University

[email protected]

919-513-7952

REFERENCES

Abt Associates. (2007). Strengthening Professional Identity: Challenges of the Addictions Treatment Workforce, Rockville: Substance Abuse and Mental Health Services Administration (SAMHSA)/DHHS.

The Annapolis Coalition on the Behavioral Health Workforce. (2007) Full report: An action plan for behavioral health workforce development. Retrieved March 13, 2007 at www.annapoliscoalition.org/files/Strategic_Planning/WorkforceActionPlan.pdf.

Carise, D., McLellan, T., & Gifford, L. (2000). Development of a ‘treatment program’ descriptor: The Addiction Treatment Inventory. Substance Use and Misuse, 35, 1797-1818.

Charmez, K. (1995). Grounded theory. In Smith, J., Harre, R., & van Langenhove, L. (eds.), Rethinking methods in psychology. London: Sage Publications.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

Dillman, D.A. (2000). Mail and Internet Surveys. NY: John Wiley & Sons.

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.   

Gallon, S., Gabriel, R., & Knudsen, J. (2003). The toughest job you'll ever love: A Pacific Northwest Treatment Workforce Survey. Journal of Substance Abuse Treatment, 24(3), 183-196.

Glaser, B. G. & Strauss, A. L. (1967). The discovery of grounded theory. Chicago: Aldine.

Johnson, J.A., Knudsen, H.K., & Roman, P.M. (2002) Counselor Turnover in Private Facilities. Frontlines: Linking Alcohol Services Research and Practice. NIAAA.

Lum, L., Kervin, J., Clark, K., Reid, F., & Sirola, W. (1998). Explaining nursing turnover intent: Job satisfaction, pay satisfaction, or organizational commitment? Journal of Organizational Behavior, 19(3), 305-320.

McLellan, T., Carise, D., & Kleber, H. (2003). Can the national addiction treatment infrastructure support the public’s demand for quality care? Journal of Substance Abuse Treatment, 25, 117-121.

Mor Barak, M., Nissly, J., & Levin, A. (2001). Antecedents to retention and turnover among child welfare, social work, and other human service employees: What can we learn from past research? A review and meta-analysis. Social Science Review, 75, 625-661.

Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. National Academy Press: Washington, DC.

LIST OF ATTACHMENTS

Attachment 1: Clinical Director Survey

Attachment 2: Data Collection Protocols

Attachment 3: Key Informant Telephone Interview Questionnaires

Attachment 4: List of Secondary Data Sets

Attachment 5: List of ATTC Network Regional Centers

Attachment 6: ATTC Literature Review: Understanding America’s Substance Use Disorders Treatment Workforce: A Summary Report

Attachment 7: Estimated Hourly Wages for Clinical Directors and Thought Leaders

Attachment 8: Map of N-SSATS Facilities by ATTC Region

Attachment 9: Table of Substance Use Disorder Facilities by ATTC Region

Attachment 10: Estimated Costs for National and Regional Sample

Attachment 11: Quantitative Data Collection Contact Script

Attachment 12: Qualitative Data Collection Contact Script

25