Supporting Statement - Justification

Customer Satisfaction Survey for

The Office of Program Services (OPS)

Grant Applicant Survey and Grant Reviewer Survey

A. Product/Activity to be Assessed:

SAMHSA’s mission within the Nation’s health system is to build resilience and facilitate recovery for those with or at risk for substance abuse and mental illness. SAMHSA accomplishes this through block grants to States and discretionary and formula grants and contracts. SAMHSA is required by statute to conduct peer reviews of all discretionary grant application that are $100,000 or more. This constitutes approximately 85 percent of all of SAMHSA’s discretionary grant applications.

The grant review process is extremely important to SAMHSA because it determines which grantees will be awarded available grants. The grantees contribute to SAMHSA’s well being; therefore, SAMHSA would like to ensure they are receiving the most qualified and best grantees.

The provision and coordination of these services entail frequent interaction with non-Federal customers, including grant applicants and grant reviewers. SAMHSA desires to assess the effectiveness and efficiency of its current review process. SAMHSA also desires to more thoroughly meet our non-Federal customers’ requests and needs in the areas identified below:

• Grants Application

• Grants Review

• Grants Management

Two customer satisfaction surveys will be administered. The first survey will be administered to Applicants that have applied for SAMHSA grants, titled “Grant Applicant Survey”. The second survey will be administered to individuals responsible for reviewing the grant applications, titled “Grant Reviewer Survey”. There have been no prior efforts to assess the satisfaction levels of grant applicants or grant reviewers in the above areas.

B. Brief statement of objectives of the proposed customer satisfaction effort, including specific identification of who will use the resulting information and how they will use it:

The purpose of the customer satisfaction surveys are to solicit feedback and suggestions on how to improve the quality and efficiency of the SAMHSA grants review process. In keeping with the notion of creating a customer-focused and customer-driven organization, the objectives of the proposed customer satisfaction survey effort are:

• to find out how well SAMHSA staff are presently meeting applicants’ and reviewers’ needs;

• to determine if the current review process produces the most qualified grantees selected;

• to determine if the current grant review process is effective for applicants and reviewers;

• to identify additional information on how to improve the grant process from the customer perspective.

Trained analysts will be designated to analyze survey responses. They will compile quantitative and qualitative data from the responses. The quantitative analyses will include frequency and percentage distributions. They will: 1) organize and prepare data for statistical analysis, 2) generate descriptive statistics for close-ended data and conduct a thematic analysis of the open-ended questions, and 3) generate numerical and graphical outputs.

A report will be provided to Elaine Parry, Executive Officer and Director of the Office of Program Services (OPS), Substance Abuse and Mental Health Services Administration, that will include tables of quantitative data and a narrative that highlights important quantitative findings and analyses of qualitative data, such as suggestions for improving service. Below are samples of a table shell (bar graph).

Overall Agreement with the statement “My questions on the RFA and/or the process were sufficiently answered”:

	Strongly Agree	Agree	Disagree	Strongly Disagree	N/A
Percent (n= )	20	65	8	5	2

The OPS Director will give feedback from the survey results to SAMHSA staff members. Staff may also work together to determine how to enhance those strengths identified by the surveys and conversely how to overcome or improve identified weaknesses. Should systemic changes need to be made due to the survey results, SAMHSA Senior Staff will work to design any appropriate changes.

C. Overview of methods to be used to collect the information:

OPS will select specific Requests for Application (RFAs) that have been recently published. Grant reviewers and grant applicants that participated (reviewed the grant application or submitted an application) in the selected RFAs will receive an email from OPS staff soliciting their participation in one of the customer satisfaction survey (Grant Applicant Survey or Grant Reviewer Survey). The email will thank users for their participation in the grant application process. Participants (non-Federal recipients) will have the opportunity to click on a hot link embedded in the thank-you e-mail and will be invited to provide feedback on the on the application process by completing either the Grant Reviewer Survey or Grant Applicant Survey. The hot link will be introduced with the following message: “Thank you for recently serving as a grant reviewer for the Substance Abuse and Mental Health Services Administration (SAMHSA)” (Grant Reviewers) or “Thank you for recently submitting a grant application to the Substance Abuse and Mental Health Administration (SAMHSA) (Grant Applicants)”. It will continue by saying “SAMHSA is committed to continuous improvement. We take pride in our work and value your feedback to ensure we maintain an excellent level of performance. We would like to get your feedback on different aspects of the grant review process to identify any issues which need to be addressed. Please take a few moments to complete the survey in the link below. Thank you in advance for your feedback.

We anticipate receiving a response rate of approximately thirty (30) percent. A reminder email will be sent one (1) week after the initial email.

The responses of those who choose to take part in the survey will go from the respondent via the Web directly to a database file. The Web site will keep no record of where the responses came from, thereby maintaining the discretion of customers who do not self-disclose. Additionally, name, phone number and e-mail address are designated as optional information on the survey.

D. Annual response burden estimate:

Grant Applicant Survey

The numbers below are based on the sample population of grant applicants. Grant applicants have an estimated average salary of $60,000. The estimated time to complete the survey is approximately 10 minutes.

(1) 153 respondents x 10 minutes = 26 hours yearly.

(2) 26 hours x $29.00 per hr. = $754.00 yearly.

3. This will involve no startup or operational/maintenance costs to respondents.

Grant Reviewer Survey

The numbers below are based on the sample population of grant applicants and reviewers. Grant applicants and reviewers have an estimated average salary of $50,000. The estimated time to complete the survey is approximately 10 minutes.

(1) 83 respondents x 10 minutes = 14 hours yearly.

(2) 14 hours x $24.00 per hr. = $336.00 yearly.

(3) This will involve no startup or operational/maintenance costs to respondents.

Respondents	# of respondents	# responses/ respondent	Hours/ response	Total burden hours	Hourly wage rate ($)	Total cost/ year ($)
Non-Federal Customers – Grant Applicants	153	1	.170	26	$29	$754
Non-Federal Customers – Grant Reviewer	83	1	.169	14	$24	$336
Non-Federal Customers - Total	236			40		$1090.00

E. Methods used to develop and test the questions (attach a copy of the questionnaire):

To develop the questions for the surveys, internal stakeholder interviews were conducted with six Subject Matter Experts (SMEs) designated by SAMHSA, from various departments and Centers to gain a thorough understanding of the internal processes, interactions between departments/divisions, and potential areas of improvement within the process. Insight garnered from stakeholder interviews with SAMHSA representatives were used to develop the survey questions. The OPS Director and SAMHSA SMEs then reviewed and refined the surveys.

Two analysts will generate descriptive statistics for closed-ended data and conduct a thematic analysis of the one open-ended question on each survey. Quantitative and qualitative data will be compiled and reported to Elaine Parry, Executive Officer and Director of the Office of Program Services (OPS).

4