0930-0333 Iminent Risk SS-A_5.31.18

the networking suicide prevention hotlines—Evaluation of imminent risk

Supporting Statement

1. JUSTIFICATION

A1. Circumstances of Information Collection

Background

The Substance Abuse and Mental Health Services Administration’s (SAMHSA), Center for Mental Health Services (CMHS) is requesting approval from the Office of Management and Budget (OMB) for the revision of the Networking Suicide Prevention Hotlines – Evaluation of Imminent Risk (OMB No. 0930-0333; Expiration, 04/30/2018) data collection. The Evaluation of Imminent Risk data collection is part of SAMHSA’s Networking and Certifying Suicide Prevention Hotlines grant program, which established the National Suicide Prevention Lifeline (“Lifeline”). This grant program is operated under authorization of Section 520A of the Public Health Service Act (42USC290bb-32.) Each year, beginning with the 2001 appropriations bill, Congress directed that funding be provided for the Suicide Prevention Hotline program. In addition to the Suicide Prevention Hotline program, funds have been continually allocated for the evaluation of the program.

The Evaluation of Imminent Risk was first implemented to evaluate the management of imminent risk callers by hotline counselors, assess counselor adherence to the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide, and identify the types of interventions implemented with imminent risk callers. Eight centers participated in the first phase of data collection from October 2011 to September 2013. Data analysis from Phase I was complete in April 2013 and an evaluation report was released in September 2013. A manuscript of these findings was published in April 2014 (Gould et al., 2016).

In 2015, OMB approved the use of the Imminent Risk Form–Revised and eight new centers were recruited for the second phase of this evaluation. Data collection for Phase II was delayed due to several factors, including development of a center recruitment strategy, the actual recruitment of participating centers, the development and training of the data collection system REDCap (discussed below), and the training of data collectors. The implementation of the Lifeline Simulation Training System was also delayed; as the effectiveness of this simulation training is one of the research questions driving Phase II of this evaluation, this delay also impacted researchers’ ability to collect enough post-training responses to assess the effectiveness of this training. Therefore, while SAMHSA completed the planned Phase II data collection during the previous OMB-approved data collection (OMB No. 0930-0275), additional data collection is sought to collect the required number of post-training responses necessary for this evaluation.

This revision requests OMB approval for the continuation of the second phase of this effort. A total of seven centers will continue their participation in Phase II of this work. One of the original centers was withdrawn from the evaluation due to insufficient participation. The withdrawal of this center should not impact the overall evaluation due to sufficient data collection at the remaining seven centers to date. The effort described above builds on a series of data collection efforts previously reviewed and approved by OMB (Phase I, OMB No. 0930–0274; Phase II, OMB No. 0930–0275) to evaluate crisis hotline practices, protocols and outcomes.

About the National Suicide Prevention Lifeline Network and Imminent Risk

As noted above, SAMHSA funds a National Suicide Prevention Lifeline (“Lifeline”) Network, consisting of toll–free telephone numbers that route calls from anywhere in the United States to a network of local crisis centers. Since its inception, the Lifeline has received more than ten million calls.

The crisis centers answering these calls provide invaluable services for callers who are and are not at imminent risk. Evidence to support the value of crisis hotlines to suicide prevention has grown (King et al., 2003; Gould et al., 2007; Kalafat et al., 2007; Mishara et al., 2007a & 2007b; Gould & Kalafat, 2009; Gould et al., 2012; Knox et al., 2012; Gould et al., 2013; Gould et al. 2016). Based on the evidence, the Lifeline has emerged as a vital resource for a range of suicide prevention initiatives and programming, to include becoming central in public awareness messaging campaigns on a federal, community and advocacy level.

Previous hotline evaluations have shown that large numbers of callers have significant histories of suicidal ideation and attempts (Kalafat et al., 2007). While not every caller is at imminent risk for suicide, crisis hotlines will typically provide referrals to mental health and other services, and also will advise the caller that they may call back if they are in crisis or have additional needs. For those at imminent risk for suicide, emergency intervention may be initiated and may result in a psychiatric hospitalization or other acute mental health service provision.

The Lifeline developed the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide in 2010 and completed implementation of these guidelines across the Lifeline network in 2012. The guidelines are comprised of two policies: (1) telephonic practices and (2) establishing and maintaining collaborative relationships with local crisis and emergency services. In addition there are nine supporting guidelines to assist crisis centers. These guidelines focus on three core areas:

• The use of active engagement, which requires that callers are actively engaged in the process of ensuring their own safety, that there is collaboration between the caller and hotline staff, and that the least invasive approach is taken to ensure a positive outcome;

• The use of active rescue, which requires that staff take all action necessary to secure the safety of a caller and initiate emergency response without the callers consent if they are unwilling or unable to take action on their own behalf; and

• A focus on collaboration with other community crisis and emergency services and the establishment of working relationships with entities that can serve to assist in the ongoing safety of the caller.

The goals of the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide are to enhance collaborative intervention (active engagement) and to work toward the least invasive intervention with callers at imminent risk; that is, to use active rescue only when necessary.

Following the dissemination of the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide across the network in 2012, Lifeline staff verified that each center’s policy documents were modified accordingly. It was left to the centers to determine how best to incorporate the guidelines into the centers’ trainings for crisis helpers (which often include but are never limited to ASIST). In collaboration with SIMmersion, the Lifeline also developed a web-based Simulation Training System designed to improve crisis counselor’s abilities to accurately identify a caller’s risk level and to choose an intervention appropriate to the identified level of risk. Phase II of this evaluation was designed to assess the impact of this simulation training.

The Need for Evaluation

As noted above, crisis counselors from seven Lifeline centers will complete the Imminent Risk Form-Revised in a continuing effort aimed at providing a profile of imminent risk callers and assessing the interventions used with these callers. The purpose of this ongoing evaluation is to inform the network’s knowledge of the extent to which counselors are aware of and being guided by Lifeline’s imminent risk guidelines; counselors’ definitions of imminent risk; the rates of active rescue of imminent risk callers; the types of rescue and non-rescue interventions used; barriers to intervention; and the circumstances in which active rescue is initiated, including the caller’s agreement to receive the intervention. To capture differences across centers, the form also collects information on counselors’ employment status and hours worked/volunteered, level of education, license status, training status, source of safety planning protocols, and responsibility for follow up.

Data collected from the Imminent Risk Form-Revised will be used to address the following evaluation questions:

1. Does simulation training related to the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide change the risk profile of callers identified as being at imminent risk by crisis counselors?

2. Does simulation training change the risk profile of callers for whom the crisis counselors initiate an active rescue?

Evaluation data provide the information necessary for shaping and influencing program and policy development through the systematic analysis and aggregation of information across the components of large-scale initiatives, thus contributing to an understanding of overall program effectiveness. With a comprehensive assessment of counselor implementation of imminent risk and active rescue protocols, counselor effectiveness can be monitored and adapted as needed, and ways in which program activities can be improved or differentially targeted can be identified.

A2. Purpose and Use of Information

The Lifeline seeks to instill hope; sustain living; and promote the health, safety, and well-being of the callers and community members it serves. Preventing the suicide of callers is the primary mission of the Lifeline; thus, all staff must act to secure the safety of callers determined to be attempting suicide or at imminent risk for suicide.

The Imminent Risk Form-Revised is completed by the crisis center counselors following the completion of a call where the counselor has determined that the caller is at imminent risk of suicide based on their understanding of the definition of imminent risk. The form captures information about the interventions utilized during the call, including whether they involved the active collaboration of the caller or not, as well as information about the counselor who handled the call. The form also assesses caller outcomes based on knowledge related to the outcome of imminent risk calls that the crisis centers may gain in the days or weeks following the crisis call. For example, if known and/or applicable, counselors reported on whether rescue resulted in the caller’s hospitalization, as well as on whether the caller was successfully reached for follow-up. No direct data collection will occur from imminent risk callers. Counselors will continue to use the Imminent Risk Form-Revised throughout this revision period.

Preliminary analysis of the data collected during Phase I of this work found that volunteers who worked as counselors answering calls to the Lifeline were less likely to actively engage callers that were determined to be at imminent risk of suicide and were more likely to implement a non-collaborative active rescue compared to employees of the Lifeline network (non-volunteers). Volunteers differed from non-volunteers in several important ways, including that they were less likely to be licensed clinicians/mental health professionals, they had less experience working as crisis helpers, and were less likely to have completed ASIST training related to individuals at imminent risk. Volunteers also worked fewer average hours answering Lifeline calls per week, which appears to be the primary difference underlying the differences in the types of interventions implemented. This association between hours spent answering calls and the types of interventions implemented was also true for non-volunteers, suggesting that counselors with greater training and/or experience may develop a greater familiarity with imminent risk callers and more confidence in implementing less invasive approaches that are better aligned with the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide (Gould et al., 2016). Given these lessons learned from Phase I, Phase II will be focused on using counselor training and experience, including use of the Simulation Training System, to predict the likelihood of three potential outcomes for imminent risk caller interactions: implementation of voluntary (collaborative) rescue (a preferred outcome), the implementation of involuntary (non-collaborative) rescue (only recommended as a last resort), and the reduction of risk during the call (such that rescue was not needed, a preferred outcome).

Counselor adherence to the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide will be reflected in counselors’ assessing the four dimensions of a caller’s suicide risk, and implementing an intervention which is consistent with the caller’s risk level. For example, in accordance with the Lifeline’s imminent risk guidelines, counselors should seek to actively engage all callers in actions to help themselves, regardless of level of risk; counselors should refrain from initiating active rescues in the event that a caller’s risk can be reduced using collaborative means; and counselors should initiate active rescues when the caller’s risk is not successfully reduced using collaborative means.

The data to be collected will contribute to the evidence-base of suicide prevention hotlines. Through this effort, SAMHSA will enhance the efficacy and accountability of crisis intervention services, and ultimately optimize public health efforts that prevent suicidal behavior. More immediately, this effort will provide a risk profile of callers who are determined to be at imminent risk for suicide and who may require active rescue and assess the types of interventions counselors used with them. The evaluation will also assess whether a center’s follow-up practices have an impact on rates of active rescue. By collecting additional responses from counselors who have completed the online simulation training, the evaluation findings from Phase II will provide sufficient statistical power to accurately inform future program practices and policy recommendations, as well as refine the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide. The information will be compiled in a report for SAMHSA, which it may choose to disseminate. The specific areas of contribution for the Evaluation of Imminent Risk efforts are detailed below.

• SAMHSA can use the results from the evaluation to develop policies and provide guidance regarding the handling of imminent risk callers to the Lifeline. Information and findings from the evaluation also can help SAMHSA refine the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide, if deemed necessary, to promote the systematic implementation of guidelines across crisis centers.

• Findings from the evaluation can be used by crisis centers to improve their services, assess the ability of counselors to implement the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide, train crisis counselors in center processes and functions related to imminent risk, and guide the use of voluntary and involuntary rescue. Centers also can use the information gathered to better identify imminent risk callers and improve their services and outcomes.

• The research community, particularly the field of mental health services research, will continue to benefit in a number of ways from the information gathered. First, evaluation of the implementation of the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide adds significantly to the developing research base about the use of hotline services. Second, the focus on imminent risk callers allows researchers to examine and understand the actions taken by counselors to aid imminent risk callers, assess the need for active rescue, determine caller risk and protective factors, and identify the types of interventions used. Finally, the analysis of evaluation data helps both researchers and service providers improve the delivery of crisis hotline services to imminent risk callers.

The Imminent Risk Form-Revised will remain unchanged from the earlier Phase II data collection with the exception of the addition of one question. As noted, the Imminent Risk Form-Revised examines whether the crisis counselor is following the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide for helping callers at imminent risk of suicide, the counselor’s experience and training, the criteria for counselors to identify a caller as being at imminent risk, and the interventions implemented with and without caller consent.

A3. Use of Information Technology

The Imminent Risk Form-Revised will be completed by trained crisis workers via REDCap, a secure web application commonly used in academic research. Counselors will complete the form for imminent risk callers after the call based on information provided by the caller. There is no direct data collection involved and callers will not be asked to answer the questions on the form.

The evaluation team has direct access to the data in real time and can export it into Excel or SPSS. With the exception of the dates of crisis calls, all data entered into REDCap are de-identified, with centers identified only by state and counselors only by initials.

A4. Efforts to Identify Duplication

While other assessments have been conducted related to suicide callers at imminent risk of suicide, the purpose of this evaluation is to assess whether the interventions used by Lifeline crisis center callers are in line with the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide, a policy that was specifically designed for the National Suicide Prevention Lifeline. Moreover, Phase II of this evaluation is focused on addressing two research questions related to the effectiveness of the web-based Simulation Training System designed to improve crisis counselor’s abilities to accurately identify a caller’s risk level and to choose an intervention appropriate to the identified level of risk. This simulation training was developed based on the Lifeline Policies and Guidelines for Helping Callers at Imminent Risk of Suicide and as such, is specific to the Lifeline network and its counselors. Data collected during earlier imminent risk evaluations of the Lifeline preceded the release of the simulation training being examined here. Therefore, this data collection is not duplicative of other efforts, as data collected as part of other studies would not be applicable to this specific evaluation.

A5. Involvement of Small Entities

There are over 160 Lifeline centers, some of which may meet the definition of “small entity” as defined by OMB. The Imminent Risk Form-Revised was developed prior to the identification of the specific centers that would be participating in Phase II of this evaluation. Given the large number of centers of varying sizes and status, and the potential the participating centers to meet the small entity definition, the Imminent Risk Form-Revised was developed to minimize the burden for all centers, whether they meet the definition or not.

A6. Consequences if Information Is Collected Less Frequently

The current application represents a revision of a previous data collection effort. This revision is being sought to ensure that sufficient data are collected to address the evaluation questions.

A7. Consistency With Guidelines of 5 CFR 1320.5

There are no special circumstances that would require more frequent or more burdensome data collection or retention than that outlined in 5 CFR 1320.5. Moreover, this data collection is for evaluation purposes and meets the statistical requirements for OMB-approved data collections. No confidential data will be collected. Therefore, this information collection fully complies with 5 CFR 1320.5 (d) (2).

A8. Consultation Outside the Agency

A 60-day notice was published in the Federal Register on February 2, 2018 (83 FRN 4918). No public comments were received from the 60-day notice.

Directors and representatives to the National Suicide Prevention Lifeline Steering Committee provided feedback to the evaluation design and data collection instrument. These steering committee members have been involved in related hotline evaluations.

A9. Payment to Respondents

There will be no payment to respondents.

A10. Assurance of Privacy

All reports and publications from data collected on imminent risk callers will include only group-level analyses that fully protect the privacy of individual participants. No data have been or will be stored with identifying respondent information. Due to the anonymity of the callers and the nature of the data collected, a certificate of confidentiality was deemed unnecessary by the evaluation team in collaboration with the IRB of record.

All data entered into REDCap are de-identified, with centers identified only by state and counselors only by initials. Initials will be replaced with an ID number, following routine practice recommended by the IRB of record. The initials are included temporarily so that the evaluation team is able to contact counselors if information is missing or internally inconsistent. Because the forms include information already available to supervisors through their own routine quality control monitoring, do not request personal information about counselors, and do not identify imminent risk callers, the provision of privacy has been deemed unnecessary. Nevertheless, SAMHSA will maintain the privacy of participants through the privacy protocol described (e.g., removing names or initials and replacing with an ID number). All files will be destroyed at the end of the project.

New York State Psychiatric Institute, Department of Psychiatry of Columbia University serves as the Institutional Review Board of record for the Evaluation of Imminent Risk.

A11. Questions of a Sensitive Nature

The items included on the Imminent Risk Form-Revised, while related to a sensitive topic, are not asked directly of callers, but filled in by counselors after the completion of the call. Therefore, the counselor will be discussing sensitive issues with the caller as a function of the crisis call. Counselors will not be asking sensitive questions as a function of the evaluation. The content of the form includes dimensions such as suicidal desire, intent, capability, protective factors, interventions, barriers to getting help, and steps taken with a person at risk. The answers to these questions will be used to understand and assess the actions taken by counselors in response to imminent risk callers.

A12. Estimates of Annualized Hour Burden

Table 1 shows the annualized burden associated with the evaluation, which will occur across two years, the period for which renewed OMB approval is being sought.

An average of 16 to 17 counselors at each of seven centers will interact with imminent risk callers for a total of 116 respondents per year of data collection. Each counselor will complete one Imminent Risk Form-Revised for each call they receive from a caller who is identified as being at imminent risk of suicide. It is expected that a total of 440 imminent risk forms will be completed across the two year data collection period, which is equal to 220 annual responses from the 116 respondents, or on average 1.9 per respondent annually. The respondent indicated in the estimate of burden is the counselor. The response represents the imminent risk call/form.

The number of respondents per Lifeline center and the number of centers for Phase I of the Evaluation of Imminent Risk was determined based on working with the centers during the planning stage of the study and understanding their call flow, as well as the total number of responses needed to detect statistical significance. The number of responses for Phase II was adjusted based on experience from Phase I and the number needed to detect statistical significance for the new study.

During the first completion of the Imminent Risk Form-Revised only, counselors will complete 11 questions about their experience and training in addition to information about the person at imminent risk. Therefore, over the two years, the burden associated with the first imminent risk form completion is 17 minutes, while the remaining 2.8 completions of the form are estimated at 15 minute burden. Together, when averaged across the 3.8 form completions (estimated as 1.9 forms/calls per year per counselor), the imminent risk form burden is 15.5 minutes. The time to complete the Imminent Risk Form-Revised was determined based on a pilot test of the form conducted prior to Phase I. Four questions about the center will be completed once by one respondent per center. SAMHSA did not think this will increase burden to a measurable degree.

Table 1. Evaluation of Imminent Risk—Estimated Annualized Burden for the Two Year Evaluation Period

Instrument	Number of Respondents	Responses / Respondent		Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hourly Cost
National Suicide Prevention Lifeline— Imminent Risk Form-Revised	116	1.9	220		.26	57	$23.02*	$1,312

*Assuming mean hourly wage of mental health counselors taken from U.S. Department of Labor, Bureau of Labor Statistics, May 2016 National Occupational Employment and Wage Estimates. http://www.bls.gov/oes/current/oes_nat.htm#21-0000

A13. Estimates of Annualized Cost Burden to Respondents

Respondents do not need a license to use the REDCap data collection system. Therefore, respondents will not incur any capital, startup, operational, or maintenance costs.

A14. Estimates of Annualized Costs to the Government

SAMHSA has planned and allocated resources for the management, processing, and use of the collected information in a manner that enhances its utility to agencies and the public. The contract for this evaluation, including staff salary, has an annualized cost of $231,910. An estimated 72 hours per year of a senior GS-14 level federal staff member will also be required for oversight to the evaluation efforts at an annualized cost of $3,497. Therefore, the total annualized cost to the Government is estimated at $235,407.

A15. Changes in Burden

Currently there are 65 annual burden hours in the OMB inventory. CMHS is requesting 57 annual hours for this submission. The decrease adjustment of 8 hours is due to a reduced number of participating centers and therefore, a reduced number of respondents.

A16. Time Schedule, Publication, and Analysis Plans

Time Schedule

The time schedule for the evaluation is summarized in Tables 2.

Table 2. Time Schedule

Activity	Timeline
Receive OMB approval for study	June 15, 2018
Data collection period	June 15, 2018 – September 14, 2018
Analysis complete	March 2019
Final report written	September 14, 2019

Publication Plan

A final report will be submitted to SAMHSA with anticipated subsequent dissemination to other interested parties, such as researchers, policymakers, and program administrators at the Federal, State, and local levels. Although not required under the evaluation contract, it is also anticipated that results from this data collection will be published and disseminated in peer-reviewed publications such as Suicide and Life Threatening Behavior.

Data Analysis Plan

SAMHSA expects to be able to answer the following questions from this evaluation:

• What is the extent to which counselors are aware of and being guided by Lifeline’s imminent risk guidelines?

• How do counselors across and within centers define imminent risk? Are counselors’ definitions of imminent risk impacted by their training histories?

• What are the rates of active rescue of imminent risk callers and the types of rescue?

• What are the circumstances in which active rescue is initiated, including the caller’s agreement to receive the intervention and the extent to which counselors’ experience, including their training histories, influences the rates of active rescue among callers at imminent risk?

• What is the risk profile(s) of callers identified by counselors as being at imminent risk?

• How do counselor training and experience affect the types of callers identified as being at imminent risk and the types of interventions implemented with these callers?

• How does exposure to the Lifeline Simulation Training impact the interventions implemented by counselors with callers at imminent risk?

Statistical Analyses

Analyses will be modeled after those employed in our previous Imminent Risk Evaluation data collection effort. Mixed effect logistic regression model will be used with random effects for counselors nested into the random center effects. Counselor training and experience will be used to predict outcomes including the implementation of voluntary rescue, the implementation of involuntary rescue, and the reduction of risk during the call such that rescue was not needed. In analyses conducted for our earlier evaluation, counselors’ having completed safety planning training was a marginal protective factor against voluntary rescue (OR = 0.54, t359 = -1.73, p = 0.08); yet counselors who competed safety planning training had about half the odds of asking for a voluntary rescue compared to those who did not complete safety planning training. Another important trend to emerge was the finding that the average number of suicide calls a counselor handled each week was a marginal predictor of voluntary rescue (b = -0.045, t356 = -1.83, p = 0.068). For every number increase in the average number of suicide calls handled each week by a counselor, SAMHSA expects to see about a 4 percent reduction in the odds of asking for a voluntary rescue. Combining the data collected during the first phase of this evaluation with the data collected during both the current ongoing data collection and the proposed revision period should provide the increase in statistical power needed to achieve statistical significance. For example, assuming 79% of all counselors complete safety planning training, SAMHSA will have 80% power to detect a voluntary rescue rate difference between 25% (without safety planning training) and 17% (with safety planning training). SAMHSA also intends to perform qualitative analyses of open-ended responses to further understand how counselors are interpreting “imminent risk”.

A17. Display of Expiration Date

The expiration date for OMB approval will be displayed.

A18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

Page 10