GLS Evaluation Part B 7-24_23 final

Garrett Lee Smith (GLS) State/Tribal Youth Suicide Prevention

and Early Intervention Evaluation

Supporting Statement B

2. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

Exhibit 1 displays the expected number of respondents to each data collection activity across the three-year OMB period.

Exhibit 1. Total Number of Respondents by Data Collection Activity

Instrument	Participating Grantees	Respondents per Grantee	Total Respondents
PSI	31	1	31
TASP	31	1	31
EIRFT Individual Form	31	1	31
EIRFT Screening Form	31	1	31
TSA-P & Consent to Contact	31	Approximately 320	30,000
TSA-F 6-Month Version	31	Approximately 6	561
TSA-F 12-Month Version	31	Approximately 5	420
TSA-PS	31	Approximately 3	303
YORS Baseline Version	31	Approximately 30	900
YORS 3-Month Version	31	Approximately 23	720
YORS 6-Month Version	31	Approximately 19	576
YORS 12-Month Version	31	Approximately 11	345
YER Journal	31	Approximately 2	75

The respondent universe and sampling methods are described below for the following data collection activities: Training Skills Assessment Follow-up (TSA-F 6- and 12-months), the Training Skills Assessment Phone Simulation (TSA-PS) and the Youth Outcomes and Resiliency Survey (YORS). The following data collection activities are reports on grant activities or existing data abstractions required from every grantee, so no sampling is required: PSI, TASP, EIRF-I, and EIRF-S. Respondents to these activities will be program staff and/or project evaluators. Recent response rates to appropriate activities, along with psychometric analyses, are presented in Exhibit 3 in Section B.4.

Training Skills Assessment- Follow-up (TSA): the TSA-F will be conducted with a random sample of adults participating in training. Respondents to the TSA-F at 6 months are also asked to consent to be contacted at 12 months.

Recruitment of the sample will continue until a total of 187 trainees per year are enrolled to participate in the TSA-F in years 2-5. Results of a power analysis for the TSA-F assuming the use of a repeated measures analysis of variance (ANOVA) with two groups of trainees (based on trainee characteristics like gender, age group, or training type [gatekeeper vs. clinical]) and three measurement timepoints (baseline, 6 months, and 12 months) to test for a group X time interaction effect indicated a total sample size of 140 (70 per group) with an effect size of f = 0.30, alpha = 0.05, and correction for non-sphericity = 0.75 would have power >= 0.80. Assuming a 25% attrition rate, a total sample of 187 trainees will be enrolled per year for the 6-month TSA-F to have a final sample of 140 at 12 months.

Training Skills Assessment- Phone Simulation: For the TSA-PS subsample, results of a power analysis assuming the use of a repeated measures ANOVA with two groups of trainees (low vs. high skills retention at 3 months) and three timepoints (baseline, 6 months and 12 months) to test a group X time interaction effect indicated a total sample of 76 (38 per group) with an effect size of f = 0.38, α = 0.05, and correction for non-sphericity = 0.75 would have power = 0.80. Again, assuming 25% attrition, a total subsample of 101 trainees will need to be enrolled per year (years 2-4) for the 3-month TSA-PS to have a final sample of 76 at 12 months.

To ensure diverse representation, participants for the TSA-PS and TSA-F will be selected with consideration for the following stratification criteria: training from the best practices registry¹ (yes/no); training type (gatekeeper or clinical); grantee type (state/tribal); geographic region (e.g., Northeast, Southeast, Southwest. Northwest), planned frequency of trainings; and planned numbers of trainees.

Youth Outcomes and Resiliency Survey (YORS) and Youth Experience Reflective (YER) Journal

Once grantees are identified and agree to participate, participants between the ages of 14–24 years who receive a positive screening result from a participating grantee and receive a referral to a mental health provider (mental health service) or youths who attend skills-based training will be eligible for participation. The YORS will be administered at enrollment, 3-, 6-, and 12-months post enrollment, with enrollment occurring no later than 1 month following referral to a behavioral health service. Participants in the YORS will also be eligible to participate in the YER Journal. Results of a power analysis for the YORS assuming the use of a repeated measures analysis of variance (ANOVA) with three groups (school, juvenile justice, and community coalition settings) across four measurement timepoints (baseline, 3, 6, and 12 months) to test for a group X time interaction effect indicated a total sample size of 115 (approximately 38 per group²) with an effect size of f = 0.34, α = 0.05, and correction for non-sphericity = 0.75 would have power = 0.80. Assuming a 25% attrition rate, a total sample of 300 youths will be enrolled to have a final sample of 115 at 12 months.

Information Collection Procedures

Information collection procedures for the GLS State/Tribal Evaluation instruments are described in Exhibit 2.

Exhibit 2. Information Collection Procedures

Instrument	Procedures
PSI	Quarterly PSI: One day after the end of each FY quarter, the grantee will receive a link to the PSI (via the SPDC) by email. Before logging in initially, the grantee will reset their password and then proceed to enter program strategy and budget information relevant for that quarter. The grantee must finalize the submission by the PSI deadline, which is the last day of the month after the end of each FY quarter. The PSI instrument lead will review PSI data submitted each quarter, following the deadline, to ensure data quality. PSI respondents will be provided with technical assistance via email or telephone to address any questions on how to categorize or enter prevention strategies implemented through their GLS program. PSI Behavioral Health Equity (BHE) Module (annual): The PSI-BHE module will be administered once annually directly following grantees’ submission of the PSI in quarter 4 (see process overview above). This module is designed to assess grantee efforts to reduce behavioral health disparities and promote behavioral health equity as part of their strategy implementation. Questions focus on cultural adaptations, efforts to address social determinants of health, progress highlights, and lessons learned. To support the grantee in completing the PSI-BHE, the SPDC will generate a list of the strategies/sub-strategies that grantees have entered throughout the year (in Q1, Q2, Q3, and Q4) for reference. Grantee staff completing the PSI will be prompted to consider the list of strategies and respond to several questions with broad consideration of strategy implementation over the year. In addition, grantees will be asked to respond to strategy-specific questions as relevant to their project implementation activity. PSI Sustainability Module: This module will be administered twice during the grant period for each grantee to assess grantee activity related to planning for program sustainability. For currently funded grantees, this module will be administered in quarter 1 of FY2025 (all grantees) and again in quarter 3 of FY2027 (Cohort 17 grantees only). Any newly funded grantees will participate in quarter 1 of the first year of the grant and in quarter 3 of the last year of the grant. In each case, the module will be administered following the relevant quarterly PSI (see process overview above). To support grantees in completing this module, the SPDC will generate a list of the strategies/sub-strategies grantees have reported implementing in the quarterly PSI for reference. Grantee staff will respond to one strategy-specific question at each administration timepoint and a larger set of questions regarding program sustainability overall at the second administration timepoint.
TASP	Grantees will receive a link to the TASP (via the SPDC) by email. Before logging in initially, the grantee will reset their password and then proceed to submit TASP data via a web-based form on the SPDC. ICF will provide training for grantees on entering the data and will monitor participation. A TASP is completed by grantee program staff within 2 weeks of each in-person training activity and quarterly for virtual training activities.
TSA-P, TSA-F, and TSA-PS	During training events, all participants will be asked to complete the TSA-P. The survey will be administered electronically (via URL or QR code at the time of the training). If a trainer is unable to administer the survey or consent-to-contact form electronically, or a trainee does not have access to a mobile device or computer, they may also complete the survey and consent-to-contact form on paper. The grantee will submit this information to ICF, through direct data entry into the SPDC, within 2 weeks of the training event. The TSA-P includes a consent to contact form indicating their willingness to be contacted to participate in the TSA-F and the TSA-PS. Once consent to contact has been received, ICF will contact a random sample of respondents for the TSA-PS and TSA-F(6). Following each data collection, trainee respondents will be asked for consent to recontact.
EIRFT-I	Grantee staff upload EIRFT-I data or enter EIRFT-I data via an online survey function each quarter to the SPDC. Initial follow-up information is entered during the same month of identification. Additional referral and service receipt is added for 6 months following the initial identification. Data are extracted from case records or other existing data sources, including any organizational staff, community members, or family members who make a mental health identification and referral. For grantees that do not have access to an existing tracking system, they should contact their technical assistance liaison, prevention specialist, and SAMHSA Government Project Officer to discuss approaches for adequately tracking and monitoring youth identified and referred for services. Grantee program staff enter EIRFT-I data into the SPDC on an ongoing basis throughout their grant period.
EIRFT-S	The grantee will submit EIRFT–S forms each quarter via upload or an online survey function. EIRFT-S forms are completed once per each implementation of a screening tool in a group setting, once per month for clinical screenings, and once per month for one-on-one screenings. For each screening event at which multiple youth are screened at a given time, one EIRFT–S should be completed for the event. For one-on-one screenings in a clinical or other setting, one aggregated EIRFT–S is completed per month to reflect screening outcomes of all youth screened during the month. Grantees develop systems locally to gather screening data, including extracting data from existing electronic health records or forms. Grantee program staff enter EIRFT-S data into the SPDC on an ongoing basis throughout their grant period.
YORS	Youth participants will be invited to participate the YORS via a web-based survey completed on the SPDC. Youth will receive an email invitation to complete the survey at approximately 3-, 6-, and 12-months after enrollment, with enrollment occurring no later than 1 month following referral to a behavioral health service.
YER Journal	Youths will be asked to respond to a weekly journal prompt with a photo and a corresponding narrative interpretation of the photo. For example, youths may be asked to reflect on a recent experience receiving services. The youth would be asked to submit a photo that represents that experience, followed by a prompt that asks: “What words come to mind? How did it make you feel?” The narrative description of what the photo represents will be analyzed using qualitative methodologies as described below. Up to 25 youths will be recruited to participate in the YER Journal each year.

2. Methods to Maximize Response Rates

Participation in the GLS State/Tribal Evaluation is a requirement of the GLS Suicide Prevention Program. Therefore, completion of the PSI, EIRFT-I, EIRFT-S, and TASP by program staff will be a requirement. However, the evaluation team has taken several steps to minimize the burden on local programs to ensure that completion is timely. These steps include developing a web-based data collection system, using updated technology, and providing training and technical assistance to grantees. The evaluation team also will provide training and technical assistance to maximize response rates for the other data collection activities by hosting web trainings, and distributing procedural manuals, Specific methods to increase the response rates are provided in Exhibit 3 below.

Exhibit 3. Methods to Maximize Response Rates

Instrument	Methods to Maximize Response Rates
TSA-F and TSA-PS	The TSA-P link will be generated within the SPDC or emailed to an alternative contact3. Following the training, the trainer will distribute this link to all participants who completed the entire training. This link may be shared via email, a QR code, or text message. Participants will be encouraged to complete the survey at the time of the training. If a participant starts the survey but does not complete it, they will receive up to 2 reminders via email or text message to encourage participation. A $20 gift card is provided to all respondents who complete the follow-up survey (for a total of $40) and a $50 gift card is provided to all respondents who complete the phone simulation.
YORS	For youths enrolled in the YORS, we will employ a graduated incentive scheme to encourage participation and ensure retention. Youths will receive $20 for data collection timepoint 1, $25 for timepoint 2, $25 for timepoint 3, and $30 for timepoint 4, for a total of $100 for study participation for 12 months.
YER Journal	Our team will leverage innovative data collection technology to engage youth. Weekly prompts will be sent to youths for 6 weeks post enrollment to discover, for example, which components of what youths are receiving are meaningful and helpful, and how youths may be utilizing skills or services following the initial screening, both in the short and long terms. For the YER Journal, youth will receive $20 for participation.

3. Tests of Procedures

Drawing on a 14-year experience collecting data through the evaluation of the GLS program, as well as the findings from the evaluation, improvements have been made to the administration protocols and content of data collection instruments. As new measures were developed, standard instrument development procedures, including review of the literature, item development, and content review by experts in the field were used. All instruments underwent procedures to enhance question accuracy and determine administration times. In addition, web-enabled instruments will undergo usability testing prior to fielding. Usability testing refers to pilot testing of the interface for administering questionnaires to determine the most efficient and understandable presentation. Typically, this is completed with a prototype and modifications are made before final fielding.

Drafts of the instruments were developed and reviewed by evaluation team members, survey methodologists, representatives from SAMHSA, and content experts in the field of suicide prevention and two youth with lived experience. Item analyses were conducted across instruments to be sure that key critical items were assessed similarly across all questionnaires. To enhance question accuracy and determine administration time, instruments underwent cognitive and/or pilot testing or expert review.

Exhibit 4 below outlines the response rates and psychometric analyses associated with GLS State/Tribal Evaluation data collection activities, as well as revisions to existing protocols to maximize response rates.

Exhibit 4. Data Collection Activity Revisions and Response Rates

Instrument	Response Rate & Psychometric Analyses Information4	Revisions to Proposed Protocol 5
PSI Revised	Among previously funded grantees, 95.5% of states and tribes participated in the last PSI. Psychometric analyses are not appropriate.	No revisions related to response rates proposed; content changes only.
TASP Revised	TASP is not a sample survey, but a brief form that program staff are required to complete for every GLS- sponsored training. Because of the simplicity and its required nature, coverage is presumed to be close to complete.	None
EIRFT-I & EIRFT-S Revised	The EIRFT-I and EIRFT-S do not have identified samples and therefore response rate information is not applicable. However, we monitor the participation of grantees in each activity. Based on analysis of the previous GLS evaluation data, overall, 86.6% of currently funded grantees participate in the EIRF (92.3% of cohort 9, 83.3% of cohort 10, 100% of cohort 11 and 33.3% of cohort 12 grantees are participating in the EIRF). Psychometric analyses are not appropriate.	The EIRFT-I and EIRFT-S will continue to collect information about youth identified at-risk by gatekeepers and/or via screening tools. Initial referral follow-up information and details about second appointments will be obtained within 3 months. No revisions related to response rates are proposed; content changes only.

4. Methods Used

ICF has full responsibility for the development of the overall statistical design and assumes oversight responsibility for data collection and analysis. Training, technical assistance, and monitoring of data collection will be provided by the GLS State/Tribal Evaluation team.

5. Consultants

The individuals responsible for overseeing data collection and analysis are:

Christine M. Walrath, PhD

ICF

630 3rd Avenue 11th floor

New York, NY 10017

Phone: (212) 941-5555

Email: [email protected]

Sophia I. Zanakos, PhD

ICF

530 Gaither Road

Rockville, MD 20850

Phone: (301) 572-0239

Email: [email protected]

The following individuals will serve as statistical consultants to this project:

Bob Stephens, PhD

ICF

2635 Century Parkway Suite 1000

Atlanta GA 30345

Lucas Godoy-Garraza, MA

Los Nardos 444 PH5

La Paz, Bolivia

The agency staff person responsible for receiving and approving contract deliverables is:

Portland Ridley, Public Health Advisor

Suicide Prevention Branch

SAMHSA/CMHS/DPTSSP

5600 Fishers Lane, 14E89D

Rockville, MD 20857

Phone: (240)-276-1848

Email: [email protected]

References

American College Health Association. (2016). National College Health Assessment, Reference Group Executive Summary, Fall 2016. Retrieved from http://www.acha-ncha.org/docs/NCHA-II_FALL_2016_REFERENCE_GROUP_EXECUTIVE_SUMMARY.pdf

Biden, J. (2023, April 28). A Proclamation on National Mental Health Awareness Month, 2023 [Review of A Proclamation on National Mental Health Awareness Month, 2023]. https://www.whitehouse.gov/briefing-room/presidential-actions/2023/04/28/a-proclamation-on-national-mental-health-awareness-month-2023/

Cash, J. A., & Bridge, J. A. (2009) Epidemiology of youth suicide and suicidal behavior. Current Opinions in Pediatrics, 21(5), 613-9.

Centers for Disease Control and Prevention. (2023). #BeThere to Help Prevent Suicide. https://www.cdc.gov/injury/features/be-there-prevent-suicide/index.html

Centers for Disease Control and Prevention. (2019, September 9). #BeThere to Help Prevent Suicide. Centers for Disease Control and Prevention. https://www.cdc.gov/injury/features/be-there-prevent-suicide/index.html

Centers for Disease Control and Prevention. (2018). Web-based Injury Statistics Query and Reporting System (WISQARS). Atlanta, GA: U.S. Department of Health and Human Services [HHS], CDC, National Center for Injury Prevention and Control. Retrieved from https://www.cdc.gov/injury/wisqars/index.html

Centers for Disease Control and Prevention. (2016). 10 leading causes of death by age group, United States -2016. [Graphic depicting cause of death by age in tabular format]. Retrieved from https://www.cdc.gov/injury/wisqars/pdf/leading_causes_of_death_by_age_group_2016-508.pdf

Centers for Disease Control and Prevention. (2012). Suicide facts at a glance. Retrieved April 15, 2014, from http://www.cdc.gov/violenceprevention/pdf/Suicide-DataSheet-a.pdf

Condron, S., Godoy-Garraza, L., Kuiper, N., Sukumar, B., Walrath, C., & McKeon, R. (2018). Comparing the effectiveness of brief versus in-depth gatekeeper training on behavioral outcomes for trainees. Crisis. doi: 10.1027/0227-5910/a000539. [Epub ahead of print]

Condron, D. S., Godoy-Garraza, L., Walrath, C. M., McKeon, R., Goldston, D. B., & Heilbron, N. S. (2015). Identifying and referring youth at risk for suicide following participation in school-based gatekeeper training. Suicide and Life-Threatening Behavior, 45, 461-476.

Condron, D. S., Godoy-Garraza, L., Walrath, C., Mckeon, R., & Heilbron, N. (2014). Referral and service receipt for youths identified as being at risk for suicide: Relationships between training type, professional role, and referral behavior in school-based trainees. Suicide and Life-Threatening Behavior. doi:10.1111/sltb.12142

Crosby, A. E., Han, B., Ortega, L. A., Parks, S. E., & Gfoerer, J. (2011). Suicidal thoughts and behaviors among adults aged ≥18 years—United States, 2008–2009. MMWR Surveillance Summaries, 60(13), 1–22. Retrieved March 10, 2014, from http://www.cdc.gov/mmwr/preview/mmwrhtml/ss6013a1.htm

Cross, W. F., Seaburn, D., Gibbs, D., Schmeelk-Cone, K., White, A.M., & Caine, E. D. (2011) Does practice make perfect? A randomized control trial of behavioral rehearsal on suicide prevention gatekeeper skills. Journal of Primary Prevention, 32(3-4).

Finn, J. L. & Checkoway, B. (1998). Young people as competent community builders: A challenge to social work. Social Work, 43(335-345).

Godoy-Garraza, L., Kuiper, N., Goldston, D., McKeon, R., & Walrath, C. (2019). Long‐term impact of the Garrett Lee Smith Youth Suicide Prevention Program on youth suicide mortality, 2006–2015. Journal of Child Psychology and Psychiatry, 60(10), 1142–1147. https://doi.org/10.1111/jcpp.13058

Godoy-Garraza, L., Kuiper, N., Cross, W. F., Hicks, B., & Walrath, C. (2020). The Effectiveness of Active Learning Strategies in Gatekeeper Training on Behavioral Outcomes. Crisis, 42(5), 360–368. https://doi.org/10.1027/0227-5910/a000733

Godoy-Garraza, L., Peart-Boyce, S., Walrath, C., Goldston, D. B., & McKeon, R. (2018a). An economic evaluation of the Garrett Lee Smith Memorial Suicide Prevention Program. Suicide and Life-Threatening Behavior, 48(1), 3-11.

Godoy-Garraza, L., Walrath, C., Kuiper, N., Goldston, D., & McKeon, R. (2018b). The impact of GLS prevention activities on youth suicide mortality from 2006 to 2015. Presented at the 51^st American Suicidology Conference, Washington, D.C.

Goldston, D. B., & Walrath, C. (2023). The Garrett Lee Smith Memorial Act: A Description and Review of the Suicide Prevention Initiative. Annual Review of Clinical Psychology, 19(1), 261–275. https://doi.org/10.1146/annurev-clinpsy-080921-082634

Goldston, D. B., Molock, S. D., Whitbeck, L. B., Murakami, J. L., Zayas, L. H., & Hall, G. C. (2008). Cultural considerations in adolescent suicide prevention and psychosocial treatment. American Psychologist, 63(1), 14–31.

Gould, M. S., Cross, W., Pisani, A. R., Munfakh, J. L., & Kleinman, M. (2013). Impact of applied suicide intervention skills training on the National Suicide Prevention Lifeline. Suicide and Life Threatening Behavior, 43(6), 676–691.

Gould, M. S., & Kalafat, J. (2009). Role of crisis hotlines in suicide prevention. In D. Wasserman & C. Wasserman (Eds.), The Oxford textbook of suicidology—The five continents perspective (pp. 459–462). Oxford, UK: Oxford University Press.

Gould, M. S., Kalafat, J., Munfakh, J. L. H., & Kleinman, M. (2007). An evaluation of crisis hotline outcomes, Part II: Suicidal callers. Suicide and Life Threatening Behavior, 37(3), 338–352.

Gould, M. S., Lake, A. M., Munfakh, J. L., Galfalvy, H., Kleinman, M., Williams, C., Glass, A., & McKeon, R. (2016). Helping Callers to the National Suicide Prevention Lifeline Who Are at Imminent Risk of Suicide: Evaluation of Caller Risk Profiles and Interventions Implemented. Suicide and Life-Threatening Behavior, 46(2), 172-90.

Gould, M. S., Munfakh, J. L. H., Kleinman, M., & Lake, A. M. (2012). National Suicide Prevention Lifeline: Enhancing mental health care for suicidal individuals and other people in crisis. Suicide and Life-Threatening Behavior, 42(1), 22–35.

Heilbron, N., Goldston, D., Walrath, C., Rodi, M., & McKeon, R. (2013). Suicide risk protocols: Addressing the needs of high risk youths identified through suicide prevention efforts and in clinical settings. Suicide and Life-Threatening Behavior, 43(2), 150–160.

ICF International. (2018). Garrett Lee Smith Youth Suicide Prevention and Early Intervention Cross-site Evaluation: Report to Congress (Report to the Substance Abuse and Mental Health Services Administration). Atlanta, GA: Author.

Institute of Medicine. (2002). Reducing Suicide: A National Imperative. National Academy Press, Washington, D.C. https://doi.org/10.17226/10398

Isaac, M., Elias, B., Katz, L. Y., Belik, S., Deane, F. P., Enns, M. W., et al. (2009). Gatekeeper training as a preventative intervention for suicide: A systematic review. Canadian Journal of Psychiatry-Revue Canadienne de Psychiatrie, 54, 260–268.

Kalafat, J., Gould, M. S., Harris-Munfakh, J. L., & Kleinman, M. (2007). An evaluation of crisis hotline outcomes part 1: Nonsuicidal crisis callers. Suicide and Life-Threatening Behavior, 37(3), 322-37.

Keller, D. P., Schut, L. J., Puddy, R. W., Williams, L., Stephens, R. L., McKeon, R., et. al. (2009). Tennessee lives count: Statewide gatekeeper training for youth suicide prevention. Professional Psychology: Research and Practice, 40(2), 126–133.

King, K. A. & Smith, J. (2000). Project Soar: A training program to increase school counselors’ knowledge and confidence regarding suicide prevention and intervention. Journal of School Health, 70(10), 402–407.

King, R., Nurcombe, R., Bickman, L., Hides, L., & Reid, W. (2003). Telephone counseling for adolescent suicide prevention: Changes in suicidality and mental state from beginning to end of a counseling session. Suicide and Life Threatening Behavior, 33(4), 400–411.

King, K. A., Vidourek, R. A., & Strader, J. L. (2008). University students’ perceived self-efficacy in identifying suicidal warning signs and helping suicidal friends find campus intervention resources. Suicide and Life Threatening Behavior, 38(5), 608–617.

Knox, K. L., Kemp, J., McKeon, R., & Katz, I. R. (2012). Implementation and early utilization of a suicide hotline for veterans. American Journal of Public Health, 102(Suppl. 1), S29–S32.

Leung, C., Bender, M. & Kwok, S. (2020). A comparison of positive youth development against depression and suicidal ideation in youth from Hong Kong and the Netherlands. International Journal of Adolescent Medicine and Health, 32(2), 20170105. https://doi.org/10.1515/ijamh-2017-0105

LeViness, P., Bershad, C., & Gorman, K. (2017). The Association for University and College Counseling Center Directors Annual Survey: Reporting period: July 1, 2016 through June 30, 2017. Retrieved from https://www.aucccd.org/assets/2017%20aucccd%20survey-public-apr17.pdf.

Lynch, F. L. (2014). Population health outcome models in suicide prevention policy. American Journal of Preventive Medicine, 47(3S2), S137–S143.

Matthieu, M. M., Cross, W., Batres, A. R., Flora, C. M., & Knox, K. L. (2008). Evaluation of gatekeeper training for suicide prevention in veterans. Archives of Suicide Research, 12(2), 148–154.

Mishara, B. L., Chagnon, F., Daigle, M., Balan, M., Raymond, S., Marcoux, I., et al. (2007a). Which helper behaviors and intervention styles are related to better short term outcomes in telephone crisis intervention? Results from a silent monitoring study of calls to the U.S. 1-800-SUICIDE Network. Suicide and Life Threatening Behavior, 37(3), 291–307.

Mishara, B. L., Chagnon, F., Daigle, M., Balan, M., Raymond, S., Marcoux, I., et al. (2007b). Comparing models of helper behavior to actual practice in telephone crisis intervention: A silent monitoring study of calls to the U.S. 1-800-SUICIDE Network. Suicide and Life Threatening Behavior, 37(3), 308–321.

National Action Alliance for Suicide Prevention. (2014). A prioritized research agenda for suicide prevention: An action plan to save lives. Rockville, MD: National Institute of Mental Health and the Research Prioritization Task Force.

Native Connections | SAMHSA - Substance Abuse and Mental Health Services Administration. (n.d.). Www.samhsa.gov. https://www.samhsa.gov/native-connections

New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America: Final report (DHHS Pub. No. SMA-03-3832). Rockville, MD: Author.

Onyeka, O., Richards, M., Tyson McCrea, K., Miller, K., Matthews, C., Donnelly, W., Sarna, V., Kessler, J., & Swint, K. (2022). The role of positive youth development on mental health for youth of color living in high-stress communities: A strengths-based approach. Psychological Services, 19(Suppl 1), 72–83. https://doi.org/10.1037/ser0000593

Reis, C., & Cornell, D. (2008). An evaluation of suicide gatekeeper training for school counselors and teachers. Professional School Counselling, 11(6), 386–394.

Robins, J. M., Hernán, M. A., & Brumback, B. (2000). Marginal structural models and causal inference in epidemiology. Epidemiology, 11(5), 550-60.

Rodi, M. S., Godoy-Garraza, L., Walrath, C., Stephens, R., Condron, D. S. Hicks, B. B., et al. (2012). Referral patterns for youth identified at risk for suicide by trained gatekeepers. Crisis, 33(2), 113–119.

Rosenbaum, P. R., & Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79(387), 516-24.

Rosenbaum, P. R. & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41-55.

Shepard, D. S., Gurewich, D., Lwin, A. K., Reed, G. A., Jr., & Silverman, M. M. (2015). Suicide and suicidal attempts in the United States: Costs and policy implications. Suicide and Life-Threatening Behavior, 46(3): 352-62.

SPAN USA, Inc. (2001). Suicide Prevention: Prevention Effectiveness and Evaluation. SPAN USA, Washington, D.C.

Sprague Martinez, L., Richards-Schuster, K., Teixeira, S., Augsberger, A. (2018). The power of prevention and youth voice: a strategy for social work to ensure youths' healthy development. Social Work, 63:2: 135-143.

Substance Abuse and Mental Health Services Administration. (2023a). 988 Suicide & Crisis Lifeline. https://988lifeline.org/?ref=w3use

Substance Abuse and Mental Health Services Administration. (2023b). Native Connections. https://www.samhsa.gov/native-connections

Substance Abuse and Mental Health Services Administration. (2019). Garrett Lee Smith Youth Suicide Prevention and Early Intervention national Outcomes Evaluation Fiscal Year 2019 Annual Report. Evaluation Findings: Report to Congress. September, 2019.

Substance Abuse and Mental Health Services Administration. (2017). National Strategy for Suicide Prevention Implementation Assessment Report. HHS Publication No. SMA17–5051. Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental Health Services Administration.

Substance Abuse and Mental Health Services Administration. (2013). Results from the 2012 National Survey on Drug Use and Health: Mental health findings. NSDUH Series H-47, HHS Publication No. (SMA) 13–4805. Rockville, MD: Author. Available at http://www.samhsa.gov/data/NSDUH/2k12MH_FindingsandDetTables/2K12MHF/NSDUHmhfr2012.htm#sec3–1.

Substance Abuse and Mental Health Services Administration. (2011). Leading change: A plan for SAMHSA’s roles and actions 2011–2014—Executive summary and introduction (HHS Publication No. (SMA) 11-4629). Rockville, MD: Author.

Substance Abuse and Mental Health Services Administration. (2005). Results from the 2004 National Survey on Drug Use and Health: National findings (DHHS Publication No. SMA 05-4062, NSDUH Series: H-28). Rockville, MD: U.S. Department of Health and Human Services.

Suicide Prevention Resource Center. (2008). Suicide risk and prevention for lesbian, gay, bisexual, and transgender youth. Newton, MA: Education Development Center, Inc. Retrieved January 7, 2014from http://www.sprc.org/sites/sprc.org/files/library/SPRC_LGBT_Youth.pdf

The White House. (2023). A Proclamation on National Mental Health Awareness Month, 2023. https://www.whitehouse.gov/briefing-room/presidential-actions/2023/04/28/a-proclamation-on-national-mental-health-awareness-month-2023/

U.S. Bureau of Labor Statistics. (2017). National Occupation Employment Statistics Survey. Retrieved August 2018 from https://www.bls.gov/oes/

U.S. Department of Health and Human Services (HHS) Office of the Surgeon General and National Action Alliance for Suicide Prevention. (2012). 2012 national strategy for suicide prevention: Goals and objectives for action. Washington, DC: U.S. Department of Health & Human Services. Retrieved January 7, 2014, from http://www.surgeongeneral.gov/library/reports/national-strategy-suicide-prevention/full_report-rev.pdf

U.S Public Health Service. (2001). National strategy for suicide prevention: Goals and objectives for action. Washington, D.C.: U.S. Department of Health and Human Services.

U.S. Public Health Service. (1999). The Surgeon General’s call to action to prevent suicide. Washington, DC: U.S. Department of Health and Human Services.

Walrath, C., Godoy-Garraza, L., Reid, H., Goldston, D., & McKeon, R. (2015). The impact of the Garrett Lee Smith (GLS) Suicide Prevention Program on suicide mortality. American Journal of Public Health, 105(5), 986-993.

Wyman, P. A., Hendricks Brown, C., Inman, J., Cross, W., Schmeelk-Cone, K., Guo, J., et al. (2008). Randomized trial of a gatekeeper program for suicide prevention: 1-year impact on secondary school staff. Journal of Consulting Clinical Psychology, 76, 104–115. https://doi.org/10.1037/0022-006X.76.1.104

988 Suicide and Crisis Lifeline. (2022.). 988lifeline.org. https://988lifeline.org/about/

Attachments

1. Garrett Lee Smith Memorial Act

2. Prevention Strategies Inventory (PSI)

1. Clean Version

2. Changes Marked Version

3. Training Activity Summary Page (TASP)

1. Clean Version

2. Changes Marked Version

4. Training Skills Assessment – Post Training (TSA-P)

5. Training Skills Assessment – Follow-up (TSA-F)

6. Training Skills Assessment – Phone Simulation (TSA-PS)

7. Early Identification, Referral, Follow-up, and Treatment – Individual Form (EIRFT-I)

1. Clean Version

2. Changes Marked Version

8. Early Identification, Referral, Follow-up, and Treatment – Screening Form (EIRFT-S)

1. Clean Version

2. Changes Marked Version

9. Youth Outcomes and Resiliency Survey (YORS)

10. Youth Experience Reflective Journal (YER Journal)

11. SPDC Data Use and Access Agreement

1 SPRC Best Practices Registry, https://sprc.org/bpr-archive

2 The number of youths enrolled from the three settings may vary and cannot be determined a priori.

3 For example, an administrative person may register the trainings in the system in advance and the TSA-P can be emailed to the trainer.

4 Psychometric analyses are indicated for data collection instruments that apply scales and generate analyzable data. If the activity doesn’t collect scaled data it is not considered appropriate for psychometric analysis and is thereafter indicated as “not appropriate”.

5 The approaches to address response rates and implementation challenges through protocol revision have been included in the last column of the exhibit.

10