1 OMB 30 Day Training-TA Survey Supporting Statement

Center for Substance Abuse Treatment’s

Services Accountability Improvement System (SAIS)

CSAT GPRA 30 Day Post-Training and 30 Day Post-Technical Assistance Feedback Surveys

1. Product/Activity to be Assessed

Under the Government Performance and Results Act (GPRA), the Substance Abuse and Mental Health Services Administration (SAMHSA) Center for Substance Abuse Treatment (CSAT) is responsible for monitoring program performance across all of its budgetary lines. The Services Accountability Improvement System (SAIS) consolidates GPRA reporting and related activities for CSAT’s entire discretionary program of over 500 Targeted Capacity Expansion (TCE) services grantees and 125 Best Practices (BP) grantees under one umbrella. Specifically, SAIS provides support for accountability and performance assessments of these programs and makes such information available to CSAT Project Officers for improved project oversight and to CSAT management for improved staff and grantee portfolio oversight.

SAIS is a web-based data repository for the programs mentioned above and they were approved by OMB as follows:

• Targeted Capacity Expansion (TCE) Services grantees – Approved as the “Government Performance and Results Act (GPRA) Client Participant Outcome Measures” – OMB No. 0930-0208.

• Best Practices (BP) grantees (formerly known as Knowledge Application [KA]) – Approved under the generic Voluntary Customer Surveys to Implement Executive Order 12862 in the Substance Abuse and Mental Health Services Administration as “Government Performance and Results Act Customer Satisfaction Surveys for the Center for Substance Abuse Treatment Knowledge Application Programs” – OMB No. 0930-0197.

• In addition, SAIS has assumed responsibility for gathering and reporting GPRA data for the Access to Treatment (ATR) portfolio – OMB No. 0930-0266.

CSAT is interested in conducting feedback surveys for those grantee staff who receive GPRA Training and Technical Assistance (TA). Feedback will be about the content and materials provided during training and technical assistance, and their utility to grantee staff 30 days after they return to performing their day-to-day duties at grantee project sites.

B. Statement of Objectives

Results of the feedback surveys, the CSAT GPRA 30 Day Post-Training and 30 Day Post-Technical Assistance Feedback Surveys, will be used by SAMHSA/CSAT to assess the training and technical assistance provided to grantee staff. Findings from the survey will allow SAMHSA/CSAT to make improvements to the training and technical assistance provided, including content and materials in order to better meet the needs of grantee personnel and to enhance program outcomes.

The feedback surveys will assess content and materials provided using both close-ended quantitative items and open-ended qualitative items (see Attachment A).

• Content/Information received at training or TA – These questions provide information about the level of satisfaction with the content/information received during the training or TA and reasons for satisfaction or dissatisfaction.

• Materials received at training or TA – These questions provide information about the level of satisfaction with the materials provided at the training or TA and reasons for satisfaction or dissatisfaction. In addition, the survey asks if the respondent has shared the information or materials with other co-workers and staff.

• Suggestions for improvement – This open ended question asks for ways to improve the training or TA.

C. Methodology

Data Collection Method

Training

1. GPRA Regional trainings are conducted twice a month on average.

2. A list of training participants’ email addresses will be generated.

3. 30 days after the training is completed, an email will be sent to the training participants asking them to complete the survey. This email will include a link to the survey (see Attachment B). When users click on this link, a window will open with the survey website, and users will be directed to complete the survey online. Instructions at the beginning of the survey will explain the purpose of the survey, how the information collected will be used, and a statement that all responses will be kept confidential.

Technical Assistance

1. Technical assistance is provided to a grantee.

2. 30 days after the technical assistance is provided, an email will be sent to the grantee staff member who requested the TA asking him/her to complete the survey. This email will include a link to the survey (see Attachment B). When users click on this link, a window will open with the survey website, and users will be directed to complete the survey online. Instructions at the beginning of the survey will explain the purpose of the survey, how the information collected will be used, and a statement that all responses will be kept confidential.

Respondents

Training

Respondents to the training survey will be all grantee staff who attend the GPRA Regional Trainings, including those who attend any or all of the three topical training days - data entry, tool administration and follow-up training. All attendees will be encouraged to participate in the survey; however, they will also be told that their participation in the survey is voluntary.

Technical Assistance

Respondents to the TA survey will be the grantee staff member who requested the GPRA TA, typically the Project Director or Evaluator. All potential respondents will be encouraged to participate in the survey; however, they will also be told that their participation in the survey is voluntary.

Sample Size and Rationale

Training

For training, 482 people were trained in calendar year 2006. It is expected that training attendance will remain consistent in upcoming years and all participants will be surveyed. Therefore, it is estimated that the sample size for the training survey will be 482.

Technical Assistance

For TA, 22, 22 and 27 GPRA TA visits were conducted in the years 2004, 2005 and 2006 respectively. On average, 24 GPRA TA visits were conducted over the past three years and the grantee staff person requesting the TA will be surveyed. Therefore, it is estimated that the sample size for the TA survey will be 24.

Timeline of Data Collection

The training and TA surveys will begin after completion of the first training and TA sessions after OMB approval is obtained. The surveys will continue until the contract ends. At this time, the SAIS contract is scheduled to end in September of 2008.

Response Rate

Based on past experience of collecting similar data, SAMHSA anticipates receiving responses from approximately 80-85% of respondents. In order to maximize the number of responses, respondents will be sent a reminder email and strongly encouraged to complete the feedback survey (see Attachment C).

Confidentiality

All responses to the surveys will be kept confidential. When respondents complete the online training or TA survey, their responses will be transmitted directly to a data file stored on a server that is only accessible by SAIS staff, and access to the area where the data file is stored is password protected. In addition to these precautions, respondents will be assured that results of this survey will only be presented in aggregate form so that individual responses cannot be identified.

D. Annual Response Burden Estimate

This data collection effort will not involve start-up or operational/maintenance costs to respondents. The maximum number of responses has been estimated for each potential contact with the respondents.

Survey	Number of Respondents(a)	Responses per Respondent	Hours per Respondent(b)	Annual Burden Hours	Hourly Wage(c)	Total Hourly Cost
30-Day Post-Training Feedback Survey	482	1	.08	38.56	$10.00	$385.60
30 Day-Post-TA Feedback Survey	24	1	.08	1.92	$15.00	$28.80
Total	506			40		$414.40

1. The number of training responses has been estimated from the number of training attendees in calendar year 2006. The number of TA responses has been estimated by averaging the number of completed GPRA TA visits completed in calendar years 2004, 2005, and 2006.

2. The average burden per response was estimated from a pilot test of the training surveys. On average, they each took approximately 5 minutes to complete.

3. The hourly wage of grantee staff attending training is estimated at $10.00. The hourly wage of project director or evaluator staff for the TA survey is estimated at $15.00.

E. Methods Used to Develop and Test the Survey Questions

The proposed training and TA feedback surveys were developed by research staff on the SAIS team. Members of the SAIS team reviewed and provided input for the surveys.

An initial version of the training survey was piloted among five training attendees who attended the January 23 through 25, 2007 Regional GPRA Training.

7. Federal Project Officers

Ms. Deepa Avula

SAIS Project Officer

SAMHSA/CSAT

1 Choke Cherry Road

Rockville, MD 20857

Phone: 240-276-2961

Fax: 240-276-2960

Email: [email protected]

Ms. Patrice Roth

SAIS Alternate Project Officer

SAMHSA/CSAT

1 Choke Cherry Road

Rockville, MD 20857

Phone: 240-276-2973

Fax: 240-276-2960

Email: [email protected]

List of Attachments

Attachment A – CSAT GPRA 30 Day Post-Training and 30 Day Post-TA Feedback Surveys

Attachment B – Initial Prompt for CSAT GPRA 30 Day Post-Training and 30 Day Post-TA Feedback Surveys

Attachment C – Follow-Up Prompt for CSAT GPRA 30 Day Post-Training and 30 Day Post-TA Feedback Surveys

6