Justification

SAMHSA Customer Satisfaction Survey

Child, Adolescent and Family Branch

New Community Technical Assistance Visit Survey

A. Product/Activity to be Assessed

The Substance Abuse and Mental Health Services Administration’s (SAMHSA), Center for

Mental Health Services’ (CMHS) Child Adolescent and Family Branch (Branch) promotes and

ensures that the mental health needs of children and their families are addressed by a

community-based system of care. A component of the Branch’s work involves a range of activities designed to provide comprehensive technical assistance (TA) to CMHS system of care grantees. The feedback of these grantees is vital to the improvement of CMHS grantee services. For this reason, the Branch is requesting OMB approval for a customer satisfaction feedback form designed to provide input for one the Branch’s TA contractors. The New Community Technical Assistance Visit (NCTAV) Survey (Attachment 2) assesses participants’ satisfaction with the new community TA visits to recently funded system of care grantees.

Key new community TA visit community participants will be asked to complete the NCTAV Survey via Survey Monkey. The survey will assess participant satisfaction with the new community TA visit planning process as well as the visit itself. The contractor will email a link to the survey to new community TA visit participants within one week following each visit, using the email message included in Attachment 1 (or a similarly worded message).

B. Brief Statement of Objectives

The objective of the NCTAV Survey is to assess participant satisfaction with the TA Partnership’s new system of care grantee visits. The data collected from the evaluations will include the following components:

Specific satisfaction – responses will determine mean satisfaction across multiple items, indicating the extent to which participants are satisfied with the new community TA visit planning process, team, and visit activities.

Personal satisfaction – participants’ comments and recommendations will provide rich,

contextualized information further explaining the benefits of the visits and opportunities for the contractor to improve the planning and implementation of the visits.

The Branch and the contractor will use the data gathered by these forms to continuously improve TA services and to better assess the needs of their grantees. To analyze the data, descriptive statistics (e.g., mean, median, and frequency) will be computed for the Likert scale and yes/no responses. Open-ended comments will be summarized.

C. Overview of Methods to Collect the Information

Data collection method. The evaluations will be administered electronically using Survey Monkey. Potential respondents will receive an email requesting their completion of the survey and a link to it in Survey Monkey (see Attachment 1). The NCTAV Survey uses commonly asked questions to solicit feedback regarding participant satisfaction with TA services.

Method for identifying respondents. All key new community TA visit participants will be eligible and encouraged to complete the evaluations. Though no one is required to do so, it is the hoped that enough evaluations will be completed so as to have a representative sample of all participants in the new community TA visits.

Proposed sample size and rationale. It is assumed that approximately 450 key participants (30 per community) will participate in approximately 15 new community TA visits annually. All new community TA visit participants will be asked to complete the NCTAV Survey. With an estimated completion rate of 40%, approximately 180 respondents (15 communities x 12 respondents per community) will complete these forms annually.

Planned frequency of information collection. The NCTAV Survey will be emailed following each new community TA visit to SAMHSA-funded system of care grantee communities.

Time period over which the information will be collected. The information will be collected

approximately 15 times annually. Each grantee community will receive the survey only once, following their new community TA visit.

Expected response rate and plan for follow-up, if any, of non-respondents. Based on past

experience collecting similar customer feedback forms, the Branch anticipates receiving

responses from approximately 40% of the new community TA visit participants. Participants will be told about the survey at the end of the new community TA visits, and encouraged to complete it. Within one week of the NCTAV Survey due date, they will receive an email from the contractor asking them to complete the survey. This email will include a survey link. The contractor will send a reminder email to all potential respondents approximately on the survey due date (see Attachment 3). No further follow-up activities are proposed.

Expected ability to assess non-response bias using existing information. The contractor does not anticipate any non-response bias based on existing information.

Methods used to maintain customer privacy. Responses will be anonymous. Respondents will reply to the survey using a common survey link, so it will not be possible to identify respondents unless they self-identify in the survey’s open-ended comments. The contractor will send the follow-up email (Attachment 3) to all potential respondents because the contractor will not able to differentiate between non-respondents and respondents.

D. Annual Response Burden Estimate

The following table provides information about the annual response burden estimated for this survey.

Respondent	Number of Respondents (a)	Number of Responses/ Respondent (b)	Hours/ Response in Minutes (c)	Annual Burden Hours (d)	Hourly Wage (e)	Annual Cost ($)
Grantee Communities	180	1	0.167	30	24.26	729

(a) The maximum number of respondents has been estimated for the total of approximately 15 new system of care communities annually (30 potential respondents/site x 40% response rate).

(b) The number of responses has been given based on the length of the NCTAV Survey.

(c) The average burden per response was estimated based on independent review of the

instrument by contractor staff.

(d) The mean hourly wage is estimated at $24.26 for the types of respondents based on a mean

yearly salary of $50,470 (assessed through a review of average salary estimates of related

professions from the US Department of Labor’s Bureau of Labor Statistics

http://www.bls.gov/oes/current/oes_nat.htm).

E. Methods Used to Develop and Test the Questions

The contractor prepared the proposed NCTAV Survey in collaboration with Branch staff. Several draft versions were circulated and critiqued by the contractor’s management team and other staff for content and clarity.

F. Federal Project Officer and Contractor

Susan Stromberg

Federal Project Officer

SAMHSA/CMHS

Child Adolescent and Family Branch

1 Choke Cherry Road, Room 6-1048

Rockville, MD 20857

Phone: 240.276.1929

Fax: 240.276.1990

Email: [email protected]

Regenia Hicks

Project Director

American Institutes for Research

1000 Thomas Jefferson St., NW

Washington, DC 20007

Phone: 202.403.5381

Fax: 202.403.5007

Email: [email protected]

G. Project Statistician and Data Collection Entity

The contractor will collect and compile data using Survey Monkey. The use of compiled

and analyzed results is at the sole discretion of the Branch.

List of Attachments

Attachment 1: Initial Invitation

Attachment 2: Survey

Attachment 3: Reminder Email

3