Supporting Statement MHFA Part A 1 25 17

Evaluation of the Mental Health First Aid Program

OMB Supporting Statement A

1. Justification

1. Circumstances of Data Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Mental Health Services (CMHS), is requesting approval from the Office of Management and Budget (OMB) for a new data collection needed to conduct the Mental Health First Aid Evaluation.

SAMHSA is requesting clearance for four data collection instruments:

1) Mental Health First Aid/Youth Mental Health First Aid (MHFA/YMHFA) Pre-Training Survey (Attachment 1)

2) MHFA/YMHFA Post-Training Survey (Attachment 2)

3) MHFA/YMHFA 3-Month and 6-Month Follow-Up Survey (Attachment 3)

4) Qualitative protocol for interviews with site coordinators (Attachment 4)

SAMHSA has made an investment in beginning to scale up Mental Health First Aid (MHFA) and Youth Mental Health First Aid (YMHFA). These mental health literacy curricula advance our mission to reduce the impact of substance abuse and mental illness on America’s communities. The evaluation compare state education agency-based, school district-based, or community-based grant programs through surveys intended to measure program goals and outcomes.

SAMHSA’s widespread dissemination of MHFA/YMHFA programs is perhaps the most significant federal effort to break down social barriers to accessing care. These mental health literacy curricula teach community members how to identify, understand, and respond to signs of mental health problems and substance use disorders before problems become crises. MHFA, which is for those whose lives or work bring them in contact with many adults, covers signs of suicidality, depression, self-injury, panic attacks, traumatic events, anxiety, psychosis, disruptive or aggressive behavior, and substance abuse. YMHFA is a parallel curriculum for those who work with youth ages 12–18; this curriculum covers all the MHFA topics, plus adolescent development, eating disorders, disruptive disorders, and ADHD.

SAMHSA leads federal efforts toward prevention of and early intervention in mental, emotional, and behavioral problems. SAMHSA’s mission is to reduce the impact of substance abuse and mental illness on America’s communities). MHFA and YMHFA activities directly address this objective. SAMHSA has coordinated a significant federal investment in state, school district, and community implementation of MHFA and YMHFA. As more specifically described below, SAMHSA integrated MHFA/YMHFA into three Project AWARE grant programs starting in late 2014. Project AWARE grants are part of the Now is the Time Initiative, which was launched in response to the Newtown school shooting.

This data collection is covered under the requirements of P.L. 103-62, the Government Performance and Results Act (GPRA) of 1993; Title 38, section 527, Evaluation and Data Collection, as well as 38 CFR section 1.15, Standards for Program Evaluation.

2. Purpose and Use of Information

The purpose of the proposed data collection activities is to evaluate implementation of MHFA and YMHFA in three distinct grant programs: Project AWARE State Education Agency (SEA) Cooperative Agreements which provide funding to support MHFA and YMHFA training to state education agencies, Project AWARE Local Education Agency (LEA) Grants, which provide funding to school districts, and Project AWARE Community (C), a new funding opportunity in FY2015 that is intended to support MHFA and YMHFA training through a wide range of community organizations. The MHFA/YMHFA evaluation will address both overarching and program-specific questions related to the implementation and widespread dissemination of these mental health literacy programs through these three distinct funding mechanisms and increase SAMHSA’ s understanding of training, referral benefits and issues in varied milieu (e.g., implementation climate, leadership). These evaluation questions are essential to address because, although MHFA/YMHFA has a track record and well-articulated theory of action (see next page), it is vital for SAMHSA to be able to identify factors that are related to the extent MHFA/YMHFA is disseminated and implemented with quality.

To describe how this evaluation will be carried out to provide SAMHSA with the necessary information, process and outcome evaluation questions are presented below along with data sources/instruments, and frequency of data collection. After providing this information (in Tables 1 and 2), SAMHSA further specify each of the data sources, how the data will be used, and by whom.

Table 1. Implementation Evaluation Questions, Data Sources, Collection Methods, and Collection Frequency

Process and implementation evaluation questions	Data sources/ Instruments	Frequency of data collection
To what extent have the grantees charged with implementing MHFA and YMHFA met their targets for recruiting and training instructors and first aiders? To what extent has the program content matched the national model, and to what extent have grantees adapted or modified the programs?	Qualitative Interviews	Interviews conducted one time per respondent
What were the barriers/facilitators to implementing MHFA and YMHFA? How were they addressed for the different grant programs? How are barriers/facilitators related to saturation, trainees’ mental health literacy and behaviors, and actual mental health referrals, within and across grantees?	Qualitative Interviews Mental health literacy items in MHFA/YMHFA Pre-Training Survey, Post-Training Survey, 3-Month and 6-Month Follow-Up Survey	Interviews conducted one time per respondent MHFA/YMHFA Trainee Surveys: Baseline (before training) After training 3 months after training 6 months after training
What role do systems-level factors, such as leadership, planning, time and staff commitment, collaboration, cultural competence, and partnership, play in implementation of MHFA and YMHFA for the grant programs? How are factors at the systems level related to saturation, trainees’ mental health literacy and behaviors, and actual mental health referrals, within and across grantees?	Qualitative Interviews Mental health literacy items in MHFA/YMHFA trainee surveys	Interviews conducted one time per respondent Four waves of MHFA/YMHFA Trainee Surveys (described above)
What policies at the state and/or community level facilitate/hinder implementation of MHFA and YMHFA for the grant programs? What differences/benefits were noted between state-level implementation and local level? How are policies at the state and community level related to saturation, trainees’ mental health literacy and behaviors, and actual mental health referrals, within and across grantees?	Interviews with site coordinators Mental health literacy items in MHFA/YMHFA trainee surveys	Interviews conducted one time per respondent
What developments with respect to the mental health workforce facilitated/hindered implementation for the grant programs?	Interviews with site coordinators	Interviews conducted one time per respondent
How does implementation of MHFA or YMHFA interact with the local organizational climate and supports to affect practices including promoting awareness of mental health and engagement with individuals showing signs of distress?	Interviews with site coordinators	Interviews conducted one time per respondent
How have program activities related to outputs, such as the numbers of: instructors trained and credentialed; first aid trainings provided and people trained; agencies and organizations participating in training; and youth served by trained first aiders? How is the relationship between activities and outcomes moderated by grantee characteristics, including the structure and delivery of grantee training, community context (community mental health data), trainer characteristics, and fidelity to the MHFA/YMHFA model, within and across grantees?	Mental health literacy items in MHFA/YMHFA trainee surveys	Four waves of MHFA/YMHFA Trainee Surveys (described above)

Table 2. Outcome Evaluation Questions, Data Sources, Collection Methods, and Collection Frequency

Outcome Evaluation questions	Data sources/ Instruments	Frequency of data collection
How are changes in outcomes related to grantee characteristics, including the structure and delivery of grantee training, community context (community mental health data), trainer characteristics, and fidelity to the MHFA/YMHFA model, within and across grantees?	MHFA/YMHFA trainee surveys	Four waves of MHFA/YMHFA Trainee Surveys (described above)
How do different milieus (e.g., urban communities, rural school districts) correspond to the identification and referral of individuals in need of behavioral health services to appropriate care?	MHFA/YMHFA trainee surveys	Four waves of MHFA/YMHFA Trainee Surveys (described above)

SAMHSA is requesting clearance for four data collection instruments. Each is described below:

(a) MHFA/YMHFA pre-training survey: This survey will be completed by MHFA/YMHFA trainees immediately prior to training and will include demographic questions, along with items that operationalize the mental health literacy constructs in Table 3 (which will recur in the post-training and follow-up surveys.)

Table 3. MHFA/YMHFA Mental Health Literacy Constructs (Trainee Outcomes) and Definitions

MHFA/YMHFA mental health literacy construct	Definition
Behavioral intention	Motivation to take action to help someone address his/her mental health problem through MHFA
Attitudes about behaviors	Beliefs about the rewards, difficulty, usefulness, and positivity of performing MHFA actions
Social beliefs	Personal and normative (i.e., important others’) beliefs about mental health problems, people with mental health problems, and performing MHFA actions
Self-efficacy	Confidence in ability to carry out MHFA actions and control their success
Knowledge and skills	Understanding of mental health problems and methods to address them
Cues to action	Exposure to those with risk factors and warning signs for mental health problems
Behavior	Actual performance of MHFA-related actions

(b) MHFA/YMHFA post-training survey: This survey will be completed by MHFA/YMHFA trainees immediately following training and will include training satisfaction questions along with items that operationalize the mental health literacy constructs in Table 3.

(c) MHFA/YMHFA follow-up survey: This survey will be completed by MHFA/YMHFA trainees at 3- and 6-months after training and will consist of items that operationalize the mental health literacy constructs in Table 3.

Information from these surveys will be used by SAMHSA to compare across grant programs the extent to which MHFA/YMHFA trainings are improving mental health literacy and associated behaviors.

(d) Qualitative interview protocol: Site coordinators in ten percent of grantee sites will be interviewed one time by telephone subsequent to OMB approval. Variables that will be explored via interviews include implementation leadership, implementation climate (positive perceptions of MHFA/YMHFA, and implementation citizenship (activities of staff that help to mainstream successful implementation). Other general factors that will be assessed include access to and use of resources, partnerships, data-driven infrastructure, and change management processes. The qualitative interview protocol will also ask about formal and informal support that trainees are receiving subsequent to training (e.g., in-house professional development, communities of practice). The protocol will ask how those factors influenced the implementation of the program and, in turn, how the program influenced climate and supports. Qualitative interview responses will be used by SAMHSA to understand contextual factors associated with grantee implementation of MHFA/YMHFA, including supportive climate and leadership. This information will be used along with type of grant program to identify optimal conditions for disseminating and implementing MHFA/YMHFA in future SAMHSA opportunities.

3. Use of Information Technology

MHFA/YMHFA trainee surveys will be administered to grantees electronically. SAMHSA will provide an online portal that respondents will use to complete and submit MHFA/YMHFA trainee surveys. The data portal will be configured to allow respondents to conveniently complete the surveys on their smart phones or personal computers.

4. Effort to Identify Duplication

This data collection is unique and necessary, representing an unprecedented effort by SAMHSA to systematically evaluate the dissemination and implementation factors that may facilitate high-quality use of MHFA/YMHFA in school districts and communities. Although there are other relevant data collection efforts, none of these related efforts have an explicit focus on comparing implementation of MHFA/YMHFA across grant programs. In cases where there are potential points of overlap, SAMHSA has taken steps to prevent any possible duplication.

SAMHSA’s NITT evaluation (pending OMB approval) also includes a focus on MHFA/YMHFA (limited to AWARE-SEA grantees), strategic steps were taken to prevent duplication of effort. In order to reduce burden on respondents, all data collection activities for the AWARE-C grantees will take place as part of the MHFA/YMHFA evaluation, and are outlined in this request. A standard data sharing agreement will allow the MHFA/YMHFA contractor to share information with the NITT evaluation contractor, thus preventing duplication and undue burden for grantees.

5. Involvement of Small Entities

AWARE-LEA and AWARE-C grantees may include small entities. Efforts to minimize burden to small entities will include messaging about the importance of the data collection, and, to the extent possible, engaging small entities in TTA and vetting so that they have more ownership over the process and motivation to participate. Moreover, the use of technology rather than paper-and-pencil surveys will serve to minimize burden for grantees, including small entities.

6. Consequences If Information Collected Less Frequently

Each data collection activity (MHFA/YMHFA trainee surveys, grantee interviews) is necessary in order to provide SAMHSA with sufficient information to compare the three AWARE grant programs and determine how MHFA/YMHFA is best implemented.

• The MHFA/YMHFA trainee surveys are essential aspects of this data collection. With less frequent data collection, SAMHSA will not be able to systematically measure of mental health literacy changes (attitudes, knowledge, skills, and behaviors) associated with MHFA/YMHFA training. When used in conjunction with process evaluation, this level of frequency of data collection will allow SAMHSA to draw conclusions about how MHFA/YMHFA is best implemented and how its impacts persist over time. All trainees in grantee sites will be asked to complete the MHFA/YMHA instrument (over a defined period, roughly October 2016 to August 2017.

• Qualitative interviews are essential to elucidate contextual factors, including facilitators and barriers, which need to be considered when SAMHSA disseminates and supports the implementation of MHFA/YMHFA in future funding decisions. Ten percent of grantees is a reasonable amount of interviews that is not overly burdensome; with fewer than 10% of grantees SAMHSA risk not having a sufficiently representative picture of implementation factors in grantee sites.

7. Consistency with Guidelines in 5 CFR 1320.5(d)(2)

The data collection fully complies with the requirements of 5 CFR 1320.5(d) (2).

8. Consultation Outside the Agency

The notice required by 5 CFR1320.8(d) was published in the Federal Register on May 17, 2016 (81FR30545). No comments were received.

a. Consultations Outside the Agency

As part of a separate project, Georgetown University conducted cognitive testing to improve the items in the MHFA/YMHFA trainee survey, including wording of items to ensure that respondents understand what is being asked. Findings from these cognitive testing activities have been used to estimate participant burden outlined in Table 5.

In addition, a Technical Work Group (TWG) for the Mental Health First Aid Evaluation has provided valuable input into our data collection activities and design. As indicated in Table 4, the TWG includes two stakeholders with lived experience who provide input into the needs being addressed by the evaluation, perceptions of burden, and design of the evaluation and (in the final year of the evaluation) will participate in the review of findings and provide input into recommendations.

Table 4. MHFA/YMHFA Technical Work Group

Name	Affiliation
Gregory Aarons, PhD	Professor, Department of Psychiatry University of California, San Diego
Will Aldridge, PhD	Independent
Alfiee M. Breland-Noble, PhD	Associate Professor, Department of Psychiatry Georgetown University
Diana Fishbein, PhD	Professor, Human Development and Family Studies Pennsylvania State University
Ann Mahling Geddes, PhD	Director of Public Policy, Maryland Coalition of Families for Children's Mental Health
Jill Kluesner	Independent
Mark Weist, PhD	Professor, Department of Psychology University of South Carolina
Peter Wyman, PhD	Professor, Department of Psychiatry University of Rochester

9. Payment to Respondents

Payments or gifts will not be provided to respondents.

10. Assurance of Confidentiality

The MHFA evaluation team will ensure that personal identifying information is maintained on a secure, password-protected and encrypted server. All data are the property of SAMHSA and will be securely transmitted to SAMHSA at the conclusion of the project. Assurances regarding confidentiality will be reflective of HHS policies.

The MHFA evaluation team will be responsible for securing initial and annual IRB approval through the American Institutes for Research’s federally registered IRB (FWA-00003952) as the IRB of record. All data collected will be kept private to the extent determined by the laws applicable in each state. For each of the data collection activities, every effort will be made to ensure that all data are protected and will be used for evaluation purposes only by the MHFA evaluation team.

MHFA/YMHFA trainee surveys: An electronic written consent process will accompany the pre-training, post-training, and follow-up versions of the MHFA/YMHFA trainee survey (see attachments). Participants will be notified that all of their responses will be protected and that responses will be combined with others’.

Qualitative interviews: A verbal consent process will be used for qualitative interviews (see attachment). The MHFA evaluation team will store the names and contact information of respondents separately from the transcript of interviews and recordings, with a code key linking the two. Only team members involved in data collection will have access to the code keys, which will be destroyed as soon as data collection is complete. Standard procedures include limiting access to identifying information, using locked files to store completed hard-copy tools; assigning unique code numbers to participants; and following minimal data requirements when reporting findings.

11. Questions of a Sensitive Nature

Information collected as part of the MHFA/YMHFA trainee surveys on stigma and attitudes towards mental illness is of a potentially sensitive nature. Despite the potentially sensitive nature of these questions, this information is essential to the evaluation because an important part of MHFA/YMHFA training involves improving attitudes, and, to the extent that trainings are implemented with quality, SAMHSA would expect to see improvements in attitudes. This information can then be tied back to the type of grant program in order to inform future grant programs for disseminating MHFA/YMHFA. As mentioned above, a consent form will provide respondents with sufficient information about the risks and benefits in order to make an informed decision about participation.

Questions of a sensitive nature are not included in the other data sources.

12. Estimates of Annualized Hour Burden

a. Estimates of Annualized Hour Burden

Table 5 provides estimates of the average annual burden for collecting the proposed information. Detailed estimates for each data collection instrument are provided below. Hourly rates for respondents are calculated based on data published by the Bureau of Labor Statistics.

MHFA/YMHFA Pre-Training Survey: Respondents will be recruited to complete the pre-training survey between October 2016 and August 2017, which is approximately one year. The pre-training survey is estimated to take 20 minutes to complete, which is slightly longer than the other trainee surveys because it includes demographic questions. As shown in the detailed table in Supporting Statement B, SAMHSA will be collecting data from three types of grantees: AWARE-SEA, AWARE- LEA, and AWARE-C. Based on prior experiences with data collection in schools and school districts, in conjunction with planned steps to monitor and enhance response rates, SAMHSA is estimating up to 80% response rates for trainee surveys. ^1,2 For example, in an evaluation of social and emotional learning that had over 25,000 survey respondents across eight districts, average response rates on a staff survey were 61% (in 2012–13) and 63% (in 2013–14), with a range by district of 22% to 78%. This was a comparable evaluation in terms of survey audience and survey length, although for the prior study respondents had no obligation to participate. Planned steps in the MHFA evaluation to monitor and enhance response rates include providing a regular schedule of survey reminders, delivering proactive technical assistance, and monitoring response rates over time with course corrections in the technical assistance approach as needed. SAMHSA recognizes that response rates may be lower than 80%; an estimation of 80% helps to ensure that burden estimates will not be exceeded.

Key data related to annualized response burden and cost are summarized in Table 5 below.

• AWARE SEA: There are 20 AWARE-SEA grantees, each of which works with three LEAs. The estimated annualized number of trainees is 125 per grantee. It is estimated that 80% of trainees will complete the MHFA/YMHFA pre-training survey, resulting in a total estimate of 6,000 respondents from the SEA grant program. 60 LEAs  125 trainees per LEA  .80 response rate = 6,000 responses.

• AWARE LEA: There are 31 AWARE-LEA grantees participating in no-cost-extensions extended through the third or fourth quarter of FY 2017. The estimated annualized number of trainees is 125 per grantee. It is estimated that 80% of trainees will complete the MHFA/YMHFA pre-training survey, resulting in a total estimate of 3,100 respondents from the LEA grant program. 31 grantees  125 trainees per grantee  .80 response rate = 3,100 responses.

• AWARE C: There are 71 AWARE-C grantees. The estimated annualized number of trainees is 125 per grantee. It is estimated that 80% of trainees will complete the MHFA/YMHFA pre-training survey, resulting in a total estimate of 6,900 respondents from the C grant program. 71 grantees  125 trainees per grantee  .80 response rate = 7,100 responses.

• Across the three grant programs, the estimated annualized number of respondents to the MHFA/YMHFA pre-training survey is 16,200.

MHFA/YMHFA Post-Training Survey: Respondents will be recruited to complete the post-training survey between October 2016 and August 2017, which is approximately one year. The post-training survey is estimated to take 15 minutes to complete, which is slightly shorter than the pre-training survey because it excludes demographic questions, but slightly longer than the follow-up survey because it includes training satisfaction questions. As shown in the detailed table in Supporting Statement B, SAMHSA will be collecting data from three types of grantees: AWARE-SEA, AWARE- LEA, and AWARE-C. SAMHSA are estimating 80% response rates for trainee surveys. Key data related to annualized response burden and cost are summarized in Table 5 below.

• AWARE SEA: It is estimated that 80% of trainees will complete the MHFA/YMHFA post-training survey, resulting in a total estimate of 6,000 respondents from the SEA grant program. 60 LEAs  125 trainees per LEA  .80 response rate = 6,000 responses.

• AWARE LEA: It is estimated that 80% of trainees will complete the MHFA/YMHFA post-training survey, resulting in a total estimate of 3,100 respondents from the LEA grant program. 31 grantees  125 trainees per grantee  .80 response rate = 3,100 responses.

• AWARE C: It is estimated that 80% of trainees will complete the MHFA/YMHFA post-training survey, resulting in a total estimate of 6,900 respondents from the C grant program. 71 grantees  125 trainees per grantee  .80 response rate = 7,100 responses.

• Across the three grant programs, the estimated annualized number of respondents to the MHFA/YMHFA post-training survey is 16,200.

MHFA/YMHFA 3-Month Follow-Up Survey: Respondents will complete the 3-month follow-up survey between January 2017 and October 2017, which is approximately one year. The follow-up survey is estimated to take 10 minutes to complete, which is shorter than the other surveys because it excludes demographic and satisfaction questions. As shown in the detailed table in Supporting Statement B, SAMHSA will be collecting data from three types of grantees: AWARE-SEA, AWARE- LEA, and AWARE-C.S AMHSA anticipate 15% sample loss at 3-month follow up due to attrition. SAMHSA are estimating 80% response rates for trainee surveys. Key data related to annualized response burden and cost are summarized in Table 5 below.

• AWARE SEA: SAMHSA anticipates 15% sample loss at 3-month follow up due to attrition. It is estimated that 80% of trainees will complete the MHFA/YMHFA follow-up survey, resulting in a total estimate of 60 LEAs  125 trainees per LEA  .85 sample retention  .80 response rate = 5,100 responses.

• AWARE-LEA: SAMHSA anticipates 15% sample loss at 3-month follow up due to attrition. It is estimated that 80% of trainees will complete the MHFA/YMHFA follow-up survey, resulting in a total estimate of 31 grantees  125 trainees per grantee  .85 sample retention  .80 response rate = 2,635 responses.

• AWARE-C: SAMHSA anticipates 15% sample loss at 3-month follow up due to attrition. It is estimated that 80% of trainees will complete the MHFA/YMHFA follow-up survey, resulting in a total estimate of 71 grantees  125 trainees per grantee  .85 sample retention  .80 response rate = 6,035 responses.

• Across the three grant programs, the estimated annualized number of respondents to the MHFA/YMHFA 3-month follow-up survey is 13,770.

MHFA/YMHFA 6-Month Follow-Up Survey: Respondents will complete the 6-month follow-up survey between April 2017 and February 2018, which is approximately one year. The follow-up survey is estimated to take 10 minutes to complete, which is shorter than the other surveys because it excludes demographic and satisfaction questions. As shown in the detailed table in Supporting Statement B, SAMHSA will be collecting data from three types of grantees: AWARE-SEA, AWARE- LEA, and AWARE-C. SAMHSA anticipate 25% total sample loss at 6-month follow up due to attrition. SAMHSA are estimating 80% response rates for trainee surveys. Key data related to annualized response burden and cost are summarized in Table 5 below.

• AWARE SEA: SAMHSA anticipate 25% sample loss at 6-month follow up due to attrition. It is estimated that 80% of trainees will complete the MHFA/YMHFA follow-up survey, resulting in a total estimate of 60 LEAs  125 trainees per LEA  .75 sample retention  .80 response rate = 4,500 responses.

• AWARE-LEA: SAMHSA anticipate 25% sample loss at 6-month follow up due to attrition. It is estimated that 80% of trainees will complete the MHFA/YMHFA follow-up survey, resulting in a total estimate of 31 grantees  125 trainees per grantee  .75 sample retention  .80 response rate = 2,325 responses.

• AWARE-C: SAMHSA anticipate 25% sample loss at 6-month follow up due to attrition. It is estimated that 80% of trainees will complete the MHFA/YMHFA follow-up survey, resulting in a total estimate of 71 grantees  125 trainees per grantee  .75 sample retention  .80 response rate = 5,325 responses.

• Across the three grant programs, the estimated annualized number of respondents to the MHFA/YMHFA 6-month follow-up survey is 12,150.

Qualitative interviews: Interviews about MHFA/YMHFA implementation factors will be conducted with site coordinators across the three grant program. SAMHSA will schedule 45-minute interviews with approximately 10% of the grantees in the interview study (i.e., at least 16 school districts from the SEA and LEA programs and seven community grantees from the community program). This estimate is based on prior experience in conducting qualitative interviews and the number of questions in the semi-structured protocol. The total time burden will be 17.25 hours.

Totals for all of the data collection activities are provided in Table 5 and in Support Statement B.

Table 5. Estimated Annualized Burden Hours

Instrument/ Activity	Number of Respondents	Responses per Respondent	Total Number of Responses	Hours per Response	Total Burden Hours	Hourly Wage3	Total Respondent Cost
MHFA/ YMHFA Pre-Training Survey	16,200	1	16,200	.33	5,346	$27.28	$145,839
MHFA/ YMHFA Post-Training Survey	16,200	1	16,200	.25	4,050	$27.28	$110,484
MHFA/ YMHFA 3-Month Follow-Up Survey	13,770	1	13,770	.17	2,341	$27.28	$63,862
MHFA/ YMHFA 6-Month Follow-Up Survey	12,150	1	12,150	.17	2,066	$27.28	$56,360
Qualitative Interviews	23	1	23	.75	17.25	$27.28	$471
Total	16,223		58,343		13,820		$377,016

13. Estimates of Annualized Cost Burden to Respondents

There are no respondent costs for capital or start-up or for operation or maintenance.

14. Estimates of Annualized Cost to the Government

It is estimated that two SAMHSA employees will be involved for 10% of their time, at an estimated annualized cost of $25,000 to the government. Additional costs are 100 percent of the contract awarded for the MHFA evaluation by SAMHSA ($2,128,731 over 3 years or an annualized cost of $709,577). The total estimated average cost to the government per year is $734,577.

15. Changes in Burden

This is a new data collection.

16. Time Schedule, Publication, Analysis Plans

a. Time Schedule

The time schedule for implementing the MHFA/YMHFA evaluation is summarized in Table 6. A 3-year clearance is requested for this project.

Table 6. Time Schedule

Task	Timeline
MHFA/YMHFA Pre-Training Survey Administration	October 2016 –August 2017
MHFA/YMHFA Post-Training Survey Administration	October 2016 –August 2017
MHFA/YMHFA Follow-Up Survey Administration	January 2017 –February 2018
Qualitative Interview with Site Coordinators	October 2016 –October 2017

b. Publication Plans

Information from the proposed data collection instruments will be used to help guide program development and management, and inform future SAMHSA funding announcements. Data from the evaluation will also be used to develop the following publications and presentations:

1. Annual evaluation reports (due July 25, 2016; July 25, 2017; July 25, 2018)

2. Final evaluation report (due July 26, 2018)

3. Monthly and quarterly reports

Evaluation data may also be used to inform journal articles, scholarly publications, and other products (e.g., conference posters).

c. Data Analysis Plan

Qualitative analyses: Qualitative analyses will characterize the frequency and prevalence of responses, examine differences among groups, and identify key findings or themes. Baseline and follow-up trainee-level implementation data will be analyzed at the trainee, grantee, and program level to create a comprehensive picture of MHFA/YMHFA implementation at the grantee and program levels.

Quantitative analyses: Based on frequency and prevalence of responses identified via qualitative analyses, SAMHSA will use hierarchical linear modeling in SAS or SPSS to assess relationships between trainee-perceived MHFA/YMHFA training implementation (e.g., structure, content, delivery, engagement; Level 1) and grantee-level characteristics (e.g., grantee program type, MHFA/YMHFA saturation type) and mental health literacy at the various time points. Additional moderating factors at the trainee level (e.g., teacher vs. law enforcement) and grantee level (e.g., number of other grantees in the same state) will be investigated.

17. Display of Expiration Date

The expiration date will be displayed on all data instruments.

18. Exceptions to the Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certification is included in this submission.

1 The MHFA surveys have not been previously administered at this scale within or outside of SAMHSA grant programs. A pilot of the MHFA survey was conducted by Georgetown University, but the pilot was of a different scope and size and does not provide a strong point of comparison for the planned evaluation effort.

2 In the planned evaluation effort, non-respondents may be less motivated to participate in MHFA classes and associated data collection activities compared to respondents. There may hence be a selection bias, such that the decision to respond to a survey is correlated with how individuals respond to survey items. We will ensure that this is mentioned as a possible limitation in our final report.

3 A significant majority of respondents will be teachers. The Bureau of Labor Statistics reports that the average hourly wage for secondary school teachers in the U.S. is $27.28 (http://work.chron.com/hourly-wages-teachers-2044.html).

1