Justification

SAMHSA SBIRT Organizational Readiness and Satisfaction Survey

A. Product/Activity to be Assessed

The Substance Abuse and Mental Health Services Administration’s (SAMHSA), Center for

Substance Abuse Services’ (CSAT) mission is to promote community-based substance abuse (SA) treatment and recovery services for individuals and families in every community. The Screening, Brief Intervention, and Referral to Treatment (SBIRT) program is central to CSAT’s goal of supporting the treatment system and recovery services. In 2013/2014, SAMHSA funded six grantees to implement SBIRT services: Maryland, Ohio, New Mexico, New York, South Carolina, and Vermont. This new cohort of grantees is required to utilize health information technology (HIT) in implementing SBIRT. Each grantee is funded to provide SBIRT services in performance sites, which may include inpatient hospitals, emergency departments, outpatient health centers, etc. Implementing SBIRT in these medical settings requires support and buy-in from performance site administrators and providers.

Grantees and their participating sites receive implementation support from SAMHSA program staff, from a technical assistance (TA) contractor, and from trainings provided by SAMHSA or by independent trainers. However, implementation feedback and evaluation of grantee satisfaction with TA or training is limited in scope, assessing primarily whether or not the grantee felt that the topic area was addressed in an understandable format. Measures also focuses only on single episodes of TA or training when multiple episodes are often required to provide the comprehensive support required by a grantee to reach their goals. CSAT is requesting to conduct a survey of SBIRT staff at all participating sites to garner information on their organization’s readiness to implement SBIRT services. Organizational readiness measures implementation and satisfaction within participating sites related to organizational climate, staff capacity, and organizational infrastructure, including HIT. By measuring organizational readiness, CSAT will receive feedback on areas of satisfaction and dissatisfaction, areas in which grantee services can be improved and ways to improve TA requests.

B. Brief Statement of Objectives

The survey’s objective is to assess grantee readiness to implement SBIRT services and their satisfaction with SAMHSA program staff support, SAMHSA TA, and grantee management support regarding the implementation and provision of SBIRT services. The data collected from this customer satisfaction survey will include the following components:

• Overall SBIRT Feedback

• Screening Feedback

• Brief Intervention Feedback

• Brief Treatment Feedback

The Branch and the contractor will use the data gathered by these forms to continuously improve TA services, the provision of training, and to better assess the needs of their grantees. To analyze the data, descriptive statistics (e.g., mean, median, and frequency) will be computed for the survey items.

C. Overview of Methods to Collect the Information

Data collection method. The survey will be collected from a convenience sample of SBIRT grantee staff at participating performance sites around a planned site visit. Online survey will be the preferred format of administration. The survey is anonymous and will not collect any personally identifiable information.

Method for identifying respondents. Participants may include administrators, clinical management staff and direct service providers. The contractor will work with each grantee to obtain a list of SBIRT staff at each performance site and identify in collaboration with the site administrator the most efficient way to announce the survey to grantee staff. It is anticipated that each site will have on average 8 staff available to participate in the survey.

Proposed sample size and rationale. It is assumed that approximately 410 SBIRT staff (assuming 64 performance sites, 8 staff at each site and an 80% response rate). CSAT expects a significant level of grantee interest in providing this feedback.

Planned frequency of information collection. This will be a one-time survey conducted in the summer of 2015, prior to the completion of the base year of the evaluation.

Time period over which the information will be collected. This will be a one-time survey conducted over a four week period with each grantee to accommodate staff schedules

Expected response rate and plan for follow-up, if any, of non-respondents. CSAT assumes that 80% of SBIRT staff will respond to the survey. They will be contacted via email and receive reminders about the survey as coordinated with the performance site administrator.

Expected ability to assess non-response bias using existing information. The contractor will be able to compare the participation rates by grantee and by performance site.

Methods used to maintain customer privacy. The contractor will maintain strict control of survey results, sharing them only in aggregate form with appropriate contractor and Branch staff.

D. Annual Response Burden Estimate

The following table provides information about the annual response burden estimated for this survey.

Respondent	Number of Respondents (a)	Number of Responses/ Respondent (b)	Total Number of Responses	Hours/ Response in Minutes (c)	Annual Burden Hours	Hourly Wage (d)	Annual Cost ($)
Performance Site Administrator	52	1	52	0.15	7.8	46.36	361.61
Clinical Supervisor	102	1	102	0.15	15.3	43.72	668.92
Clinical Service Provider	256	1	256	0.15	38.4	29.38	1,128.19
TOTAL	410		410	61.5			2,158.72

(a) The maximum number of annual respondents has been based on an estimated 8 staff at each of 64 performance sites.

(b) Each staff will be asked to complete the survey once annually.

(c) The average burden per response was estimated based on independent review of the instrument by contractor staff.

(d) The median hourly wage is estimated at $46.36 for performance site administrators, $43.72 for clinical supervisors and $29.38 for clinical service providers. Median wages estimates were obtained salary estimates of related professions from the US Department of Labor’s Bureau of Labor Statistics http://www.bls.gov/oes/current/oes_nat.htm).

E. Methods Used to Develop and Test the Questions

The SBIRT Organizational Readiness and Satisfaction Survey was developed based on reviewing the extant implementation research literature (e.g., Chaudoir, Dugan, & Barr, 2013; Damschroder et al., 2009; Garner, 2009; Greenhalgh, MacFarlane, & Kyriakidou, 2004; Weiner, 2009; Weiner, Belden, Bergmire, & Johnston, 2011). Based on this review, we identified the Organizational Readiness for Implementation Change (ORIC; Shea, Jacobs, Esserman, Bruce, & Weiner, 2014) and the Implementation Climate Scale (ICS; Jacobs, Weiner, & Bunger, 2014) as the two most appropriate instruments for the current project due to each of them being theoretically-grounded, having established psychometric properties, and being composed of relatively few questions (i.e., 6-12 items ). In addition to these two instruments, the survey includes questions to assess satisfaction, capacity, and infrastructure to implement SBIRT screening, BI, and BT. Pilot testing the survey suggests the survey takes 9 minutes or less to complete.

F. Federal Project Officer and Contractor

Sarah Ndiangui

Contracting Officer Representative

SAMHSA/CSAT

1 Choke Cherry Lane

Room 5-1062

Rockville, MD 20857

Phone: 240.276.2918

Email: [email protected]

Kathryn Wetherby

Assistant Contracting Officer Representative

SAMHSA/CSAT

1 Choke Cherry Lane

Room 5-1062

Rockville, MD 20857

Phone: 240.276.2899

Email: [email protected]

Georgia Karuntzos

Sr. Director, Business Strategy and Research

RTI International

3040 Cornwallis Road

PO Box 12194

Research Triangle Park

NC 27709-2194

Phone: 919.541.6159

Email: [email protected]

G. Project Statistician and Data Collection Entity

The Project Statistician is Arnie Aldridge at RTI International. The contractor will collect and compile data using Survey Gizmo. The use of compiled and analyzed results is at the sole discretion of the Branch.

List of Attachments

Attachment 1: Survey

3