Supporting Statement

SAMHSA Customer Satisfaction Survey
Child, Adolescent and Family Branch
Technical Assistance Feedback Surveys
A.

Product/Activity to be Assessed

The Substance Abuse and Mental Health Services Administration’s (SAMHSA), Center for
Mental Health Services’ (CMHS) Child Adolescent and Family Branch (Branch) promotes and
ensures that the mental health needs of children and their families are addressed by a
community-based system of care. A component of the Branch’s work involves a range of
activities designed to provide comprehensive technical assistance (TA) to CMHS system of care
grantees. The feedback of these grantees is vital to the improvement of CMHS grantee services.
For this reason, the Branch is requesting OMB approval for three customer satisfaction feedback
surveys designed to provide input for one the Branch’s TA contractors:


The Annual Webinar Survey (Attachment 1), which will assess grantee satisfaction
with the subcontractor’s webinars on an annual basis. It will include fill-in-the-blank
items, Likert scale items (strongly disagree, disagree, neither agree nor disagree, agree,
strongly agree), and checkbox responses.



The Specific Webinar Survey (Attachment 2), which will assess webinar participant
satisfaction with onoing TA events. It will include fill-in-the-blank items, Likert scale
items (strongly disagree, disagree, neither agree nor disagree, agree, strongly agree), and
checkbox responses. The contractor will tailor questions in section 5 of each Specific
Webinar Survey based on each webinar’s focus and what participants should learn from
the event.



The Cultural Competence Action Team Survey (Attachment 3), which will assess
satisfaction of grantee participants in the Cultural and Linguistic Competence
Community of Practice on a biannual basis. It will include fill-in-the-blank items, Likert
scale items (strongly disagree, disagree, neither agree nor disagree, agree, strongly agree),
and checkbox responses.

B.

Brief Statement of Objectives

The three surveys will assess participant satisfaction with the contractor’s TA services. The data
collected from the evaluations will include the following components:


Annual Webinar Survey:
o Specific satisfaction – responses will determine mean satisfaction across multiple
items, indicating the extent to which participants are satisfied with the webinars
and Cultural Competence Action Team’s individualized TA, webinars and calls,
resources, and website.
o Personal satisfaction – participants’ comments and recommendations will
provide rich, contextualized information on the webinars and Cultural
Competence Action Team TA services.
o The Branch and the contractor will use the data gathered by the survey to
continuously improve TA services and to better assess the needs of grantees. To
1

analyze the data, the contractor will compute descriptive statistics (e.g., mean,
median, and frequency) for the Likert scale and yes/no responses, and summarize
the open-ended comments.


Specific Webinar Survey:
o Specific satisfaction – responses will determine mean satisfaction across multiple
items, indicating the extent to which participants are satisfied with the webinars
and Cultural Competence Action Team’s individualized TA, webinars and calls,
resources, and website.
o Personal satisfaction – participants’ comments and recommendations will
provide rich, contextualized information on the webinars and Cultural
Competence Action Team TA services.
o The Branch and the contractor will use the data gathered by the survey to
continuously improve TA services and to better assess the needs of grantees. To
analyze the data, the contractor will compute descriptive statistics (e.g., mean,
median, and frequency) for the Likert scale and yes/no responses, and summarize
the open-ended comments.



Cultural Competence Action Team Survey:
o Specific satisfaction – responses will determine mean satisfaction across multiple
items, indicating the extent to which participants are satisfied with the webinars
and Cultural Competence Action Team’s individualized TA, webinars and calls,
resources, and website.
o Personal satisfaction – participants’ comments and recommendations will
provide rich, contextualized information on the webinars and Cultural
Competence Action Team TA services.
o The Branch and the contractor will use the data gathered by the survey to
continuously improve TA services and to better assess the needs of grantees. To
analyze the data, the contractor will compute descriptive statistics (e.g., mean,
median, and frequency) for the Likert scale and yes/no responses, and summarize
the open-ended comments.

C.

Overview of Methods to Collect the Information

Data collection method.
 Annual Webinar Survey: The contractor will administer the Annual Webinar Survey
electronically using Survey Monkey. Grantee communities will receive an email
requesting their completion of the survey and an electronic link to it in Survey Monkey.
The survey uses commonly asked questions to solicit feedback regarding participant
satisfaction with TA services.


Specific Webinar Survey: The contractor will administer the Specific Webinar Survey
electronically using Survey Monkey. As part of each webinar event, webinar participants
will receive an electronic link to the survey in Survey Monkey. The contractor will ask
webinar participants to complete the survey at the end of each webinar. The survey uses
commonly asked questions to solicit feedback regarding participant satisfaction with TA
services.

2



Cultural Competence Action Team Survey: The contractor will administer the Cultural
Competence Action Team Survey electronically using Survey Monkey. Grantee
communities will receive an email requesting their completion of the survey and an
electronic link to it in Survey Monkey. The survey uses commonly asked questions to
solicit feedback regarding participant satisfaction with TA services.

Method for identifying respondents.
 Annual Webinar Survey: The All grantee community members will be eligible and
encouraged to complete this survey.


Specific Webinar Survey: The contractor will ask participants in each webinar event to
complete this survey.



Cultural Competence Action Team Survey: All members of the Cultural and
Linguistic Competence Community of Practice will be eligible and encouraged to
complete this survey.

Proposed sample size and rationale.
 Annual Webinar Survey: The contractor’s listserv of grantee community members has
approximately 2,000 members. The contractor will ask all of these individuals to
complete this survey, to help obtain representative feedback of all community members.
Assuming that 20% of these individuals complete this survey, 400 grantee community
members will complete the survey annually.


Specific Webinar Survey: For this survey, the contractor assumes an average of 25
participants in each webinar event (based on historical participation rates). With
approximately 24 webinars annually and an estimated completion rate of 30%,
approximately 180 respondents will complete this survey annually.



Cultural Competence Action Team Survey: The contractor’s Cultural and Linguistic
Competence Community of Practice listserv has approximately 500 members. The
contractor will ask all of these individuals to complete the Cultural and Linguistic
Competence Action Team Survey, to help obtain representative feedback of all members.
The contractor assumes that 30% of these participants will complete this survey twice
annually (300 total respondents annually).

Planned frequency of information collection.
 Annual Webinar Survey: The contractor will administer the Annual Webinar Survey
once annually.


Specific Webinar Survey: The contractor will administer the Specific Webinar Survey
following each TA webinar.



Cultural Competence Action Team Survey: The contractor will administer the Cultural
Competence Action Team Survey twice annually.

Time period over which the information will be collected.
 Annual Webinar Survey: This survey will remain active for approximately four weeks.
3



Specific Webinar Survey: This will remain active for approximately one week
following each TA webinar.



Cultural Competence Action Team Survey: This survey will remain active for
approximately four weeks.

Expected response rate and plan for follow-up, if any, of non-respondents.
 Annual Webinar Survey: Based on past experience collecting similar feedback, the
contractor anticipates that approximately 20% of grantee community members will
complete the Annual Webinar Survey, The contractor will send to grantee community
members an email with a Survey Monkey link to the Annual Webinar Survey. Within
one week of the survey due date, the contractor will send an email asking grantee
community members to complete the survey. This email will again include a survey link.
The contractor will send an additional reminder email to all potential respondents around
the survey due date. No further follow-up activities are proposed.


Specific Webinar Survey: Based on past experience collecting similar feedback after
webinar events, the contractor anticipates that approximately 30% of grantee community
members will complete the Specific Webinar Survey. The contractor will tell webinar
participants about the Specific Webinar Survey at the end of each webinar event. No
follow-up is planned for this survey.



Cultural Competence Action Team Survey: Based on past experience collecting
similar feedback, the contractor anticipates that approximately 30% of Cultural and
Linguistic Competence Community of Practice members will complete the Cultural
Competence Action Team Survey. The contractor will send to members of its Cultural
and Linguistic Competence Community of Practice an email with a Survey Monkey link
to the survey. Within one week of the survey due date, the contractor will send an email
asking grantee community members to complete the survey. This email will again
include a survey link. The contractor will send an additional reminder email to all
potential respondents around the survey due date. No further follow-up activities are
proposed.

Expected ability to assess non-response bias using existing information.
 Annual Webinar Survey: The contractor does not anticipate any non-response bias
based on existing information.


Specific Webinar Survey: The contractor does not anticipate any non-response bias
based on existing information.



Cultural Competence Action Team Survey: The contractor does not anticipate any
non-response bias based on existing information.

Methods used to maintain customer privacy.
 Annual Webinar Survey: Responses will be anonymous unless respondents provide
their contact information. The introductory text for the survey explains this for
respondents. Respondents will reply to the surveys using a common survey link, so it will
not be possible to identify respondents unless they self-identify in the open-ended
comments. Since we use the same link for everyone, we assume the possibility of
4

duplicate responses from the same person. We don’t anticipate duplicate responses,
though.


Specific Webinar Survey: Responses will be anonymous unless respondents provide
their contact information. The introductory text for the survey explains this for
respondents. Respondents will reply to the surveys using a common survey link, so it will
not be possible to identify respondents unless they self-identify in the open-ended
comments. Since we use the same link for everyone, we assume the possibility of
duplicate responses from the same person. We don’t anticipate duplicate responses,
though.



Cultural Competence Action Team Survey: Responses will be anonymous.
Respondents will reply to the surveys using a common survey link, so it will not be
possible to identify respondents unless they self-identify in the open-ended comments.
Since we use the same link for everyone, we assume the possibility of duplicate
responses from the same person. We don’t anticipate duplicate responses, though.

D.

Annual Response Burden Estimate
The following table provides information about the annual response burden estimated for
these consumer feedback surveys.

Number of
Respondents
(a)

Number of
Responses/
Respondent
(b)

Hours/
Response in
Minutes (c)

Annual
Burden
Hours

Hourly
Wage (d)

Annual
Cost ($)

Grantee Communities (Annual
Webinar Survey)

400

1

0.13

52

24.26

1,262

Grantee Communities
(Specific Webinar Survey)

180

1

0.08

14.4

24.26

349

Grantee Communities
(Cultural Competence Action
Team Survey)

300

1

0.20

60

24.26

1,456

Total

880

Respondent

126

3,067

(a) The number of respondents has been estimated using the number of grantee community
members on the contractor’s listservs and using the average number of participants in each
webinar.
(b) For the purposes of estimating annual cost, the contractor assumes that community members
will complete each survey only once. For the Special Webinar Survey, it is likely that some
respondents will participate in more than one webinar event and complete a related survey—but
not will not complete this survey more than once for the same webinar event. It is also possible
that some members of the Cultural and Linguistic Competence Community of Practice will
complete the Cultural Competence Action Team Survey both times it is administered.

(c) The average burden per response was estimated based on independent review of the
surveys by contractor staff.

5

(d) The mean hourly wage is estimated at $24.26 for the types of respondents based on a mean
yearly salary of $50,470 (assessed through a review of average salary estimates of related
professions from the US Department of Labor’s Bureau of Labor Statistics
http://www.bls.gov/oes/current/oes_nat.htm).

E.

Methods Used to Develop and Test the Questions

The contractor prepared the consumer feedback surveys. The contactor circulated several draft
versions and critiqued by the contractor’s management team and other staff for appropriate
content and clarity of questions.
F.

Federal Project Officer and Contractor
Susan Stromberg
Federal Project Officer
SAMHSA/CMHS
Child Adolescent and Family Branch
1 Choke Cherry Road, Room 6-1048
Rockville, MD 20857
Phone: 240.276.1929
Fax: 240.276.1990
Email: [email protected]
Regenia Hicks
Project Director
American Institutes for Research
1000 Thomas Jefferson St., NW
Washington, DC 20007
Phone: 202.403.5381
Fax: 202.403.5007
Email: [email protected]
Jeffrey Poirier
Senior Research Analyst
American Institutes for Research
1000 Thomas Jefferson St., NW
Washington, DC 20007
Phone: 202.403.5368/202.329.5440
Fax: 202.403.5007
Email: [email protected]

G.

Project Statistician and Data Collection Entity


Annual Webinar Survey: The contractor will collect and compile data using Survey
Monkey. The use of compiled and analyzed results is at the sole discretion of the Branch.
No data will be shared publicly beyond the Branch or the contractor.

6



Specific Webinar Survey: The contractor will collect and compile data using Survey
Monkey. The use of compiled and analyzed results is at the sole discretion of the Branch.
No data will be shared publicly beyond the Branch or the contractor.



Cultural Competence Action Team Survey: The contractor will collect and compile
data using Survey Monkey. The use of compiled and analyzed results is at the sole
discretion of the Branch. No data will be shared publicly beyond the Branch or the
contractor.

List of Attachments
Attachment 1: Overall Webinar Survey
Attachment 2: Specific Webinar Survey
Attachment 3: Cultural Competence Action Team Survey

7