Supporting Statement

Recovery to Practice Resource Center

Technical Assistance and Webinar Evaluation Forms

A. Product/Activity to Be Assessed: Technical Assistance and Training

The Substance Abuse and Mental Health Services Administration (SAMHSA)/Center for Mental Health Services (CMHS) regards customer feedback about services provided as a critical step in assessing and improving customer service. The Technical Assistance Evaluation Form (Attachment A) and the Webinar Evaluation Form (Attachment B) are presented immediately following a technical assistance (TA) or Webinar training event, so the customer can give quantitative and qualitative assessments of the customer service. Domains measured in the survey are responsiveness, utility, new information provided, customer service provider knowledge, and customer service provider communication skills.

Through the Recovery to Practice (RTP) Resource Center, Development Services Group, Inc. (DSG, the contractor), staff will provide TA and training to support learning strategies for implementing Recovery-Oriented Care in practical and sustainable ways.

Over the past 15 to 20 years, the concept of recovery has grown from an “idea whose time has come” to a full-fledged model. Developed by people who use mental health services, Recovery-Oriented Care has grown to include recovery-based values, relationships, services, goals, cultures, programs, systems, accountability tools, and even community development. The recovery movement and the recovery model together have brought new people, new energy, and new ideas into the mental health field. The RTP Resource Center seeks to advance a recovery approach in mental health care by developing, promoting, and disseminating standardized training manuals on what recovery is and how to implement it into practice. The project will support the expansion and integration of recovery-oriented care delivered by mental health providers.

SAMHSA/CMHS begins by understanding the “customer” who will be requesting TA. Each of the professional disciplines is unique in terms of academic training, theoretical approach, and delivery of mental health care.

Users will know that the assistance exists and how to access it in at least the following ways. The contractor will provide an overview of SAMHSA/CMHS’s TA services on the RTP Resource Center’s Web site, with links from the contractor’s team members’ Web sites, in emails from RTP and SAMHSA ListServs, and in quarterly informational updates, along with clear instructions for knowing where to download the request form, the person to contact with questions, and the timing of delivery.

The Resource Center will process TA requests through a series of steps that are user-friendly, that document and track actions taken for each request, and that generate useful data for SAMHSA in understanding needs submitted and in analyzing trends. To facilitate access to TA, users can request TA through the Web site, call the toll-free 800/888 number, or email their request. The contractor will post a downloadable TA request form on the Resource Center’s Web site. This form will include the following intake information: date of request; name of requestor, and affiliation; type of request (i.e., written materials, referral to resource, request for onsite training); and topic of request. Each request will be responded to within 48 hours of receipt. In addition to the contractor’s project staff, a committed group of nationally recognized expert consultants will also provide content-specific TA. Technical assistance will begin no later than March 31, 2010.

To help SAMHSA/CMHS understand emerging needs, widespread barriers, and patterns of requests, it is important to track and analyze TA requests. The contractor’s TA tracker, used to log and follow TA requests, provides an easy way for the contractor to ensure that each task pertaining to each TA request is completed in a timely manner and for the Task Order Officer (TOO) to “check in” and see what kinds of requests are being received, and what types of TA are being provided. The database will contain flag fields that will allow the project director and other key staff to know at any point in time all open requests for technical assistance, which contractor staff person or persons have been assigned to the request, and its current status. The database system will be secured, encrypted, and password protected, but will be remotely accessible by the TOO, key staff of the contractor, and others as determined by the TOO.

The RTP Resource Center will feature quarterly training presentations on relevant topics each year. Steering Committee members will recommend ideas for the topics, and as the project continues, the TA tracking system will provide data about high priority topics and patterns of requests. Project staff will generate activity reports from the TA Tracker, analyze activities by topic and audience, and disseminate status reports. For each training presentation, the contractor will present a draft concept proposal for each teleconference 2 months prior to the event. Upon approval from the TOO, the contractor will secure three speakers for each presentation.

The training sessions will be delivered through interactive Webinars that include teleconferencing capability and presentation of visuals (e.g., PowerPoint presentations). For large audiences of 250–350 participants, the contractor will use SAMHSA’s teleconference and Web-based conference services to organize and execute the sessions, and will employ mechanisms for muting lines and inviting Q&A. All Webinars will be recorded and posted on the Resource Center Web site for easy download following the live event. The contractor anticipates up to 350 respondents for each one of these events.

B. Statement of Objectives

Beginning in April 2010, the contractor expects to handle 25 to 40 TA requests a month. Webinars will occur twice in the first year of the contract and four times a year afterward. At the conclusion of each technical assistance event and each Webinar, customers will be asked whether they would be willing to take a few moments to fill out a customer feedback form. Each consenting customer will be emailed the Recovery to Practice Resource Center Technical Assistance Evaluation Form or Recovery to Practice Resource Center Webinar Evaluation Form (see Attachments A and B).

The purposes of these forms are to collect data that will inform the contractor about what is working and to analyze trends.

C. Methodology

Data Collection

Using the RTP Resource Center Technical Assistance Evaluation Form or the RTP Webinar Evaluation Form, customers will be asked to evaluate the technical assistance or Webinar training received on the basis of responsiveness, utility, clarity, and depth (among other things). Each element is rated on a scale of 1 (poor) to 5 (excellent). Customers are invited to comment on any suggested improvements.

Data will be collected by email.

Respondents

Respondents will be RTP Resource Center technical assistance requestors and RTP Resource Center Webinar training participants. Participation in evaluation of TA or training will be voluntary.

Respondents will consist primarily of psychiatrists, psychologists, psychiatric nurses, social workers, and other mental health professionals. The balance of participants will be people from the general public interested in mental health recovery. They will learn of the RTP Resource Center’s TA and training services from various sources, for example, the RTP Web site (still under construction) and assorted ListServs (e.g. RTP, CMHS, CMHS’s Office of Consumer Affairs, SAMHSA eNews).

Sample Size and Rationale

Twenty-five to 40 TA requests will be made monthly (for a range of 300 to 480 in a year), and all requestors will be invited to fill out an evaluation form following the fulfillment of a TA request. Each quarterly Webinar training will be attended by 250 to 350 persons, for an annual range of 1,000 to 1,400.

Timeline of Data Collection

Responses will be tallied and averaged and reported in monthly reports to the government project officer.

The contractor will close out completed TA requests, evaluate and report the activities accomplished, resources used, products developed, and recommendations regarding further work.

Response Rate

Although the total TA and training participants will range from 1,300 to 1,880 annually, SAMHSA/CMHS estimates a response rate of 60 percent.

Privacy

The RTP TA Evaluation Form and the RTP Webinar Evaluation Form contain no identifying information, and all responses will be kept private to the extent of the law. Although responses will be sent by email, responses will be printed out and placed in a hardcopy file at the contractor’s place of business before they are tabulated by the contractor’s data collection/analysis staff. The surveys will be accessible only to contractor staff and data analysis staff. Results of the survey will be presented only in aggregate form so that individual responses cannot be identified.

D. Annual Response Burden Estimate

This data collection effort will not involve start-up or operational/maintenance costs to respondents. The maximum number of responses has been estimated for each potential contact with the respondents.

 

Survey	Number of Respondentsa	Responses per Respondent	Hours per Respondentb	Annual Burden Hours	Hourly Wage	Total Hourly Cost
Technical Assistance Evaluation Form	480c	1	.083	40.0	$31.00	$1,240.00
Webinar Evaluation Form	1,400d	1	.083	116.2	$31.00	$3,602.20e
Total	1,880	—	—	156.2	—	$4,842.20
aThe number of responses has been based on information from the Task Order Officer and CMHS Government Project Officers. Please note that these are maximums. This table is based on a response rate of 100 percent, although SAMHSA/CMHS considers a 60 percent response rate to more likely. bThe burden per response is estimated, from experience, at up to 5 minutes (.083 hours) to complete the survey. cThe number is based on a maximum of 40 respondents a month. dThe number is based on a maximum of 350 respondents per Webinar, for four Webinars a year. eThere will be only two Webinars during the contract’s first year, thus number of respondents, annual burden hours, and total hourly cost will be halved that first year.

E. Methods Used to Develop and Test the Survey Questions

The proposed customer satisfaction survey is a variant of the TA Evaluation Form the contractor has used in its national training and technical assistance contracts for the past decade.

F. Federal Project Officer/Task Order Officer and Consultants

Stacey Lesko

Recovery to Practice Project Officer

SAMHSA/CMHS

One Choke Cherry Road

Rockville, MD 20857

Phone: 240.276.1390

Email: [email protected]

Contractor

Development Services Group, Inc. (DSG)

7315 Wisconsin Avenue, Suite 800E

Bethesda, MD 20814

Phone: 301.951.0056

Fax: 301.951.3324

Web site: http://www.dsgonline.com

4