Supporting Statement for THE evaluation of The networking suicide prevention hotlines
Supporting Statement
JUSTIFICATION
A1. Circumstances of Information Collection
Background
The Center for Mental Health Services (CMHS), Substance Abuse and Mental Health Services Administration (SAMHSA), is requesting approval for the revised data collection associated with the Evaluation of the Networking Suicide Prevention Hotline (OMB No. 0930–0274) which expires on May 31, 2009. The purpose of this program is to promote systematic follow up of suicidal persons who call the National Suicide Prevention Lifeline.
Crisis hotlines are one of the oldest suicide prevention resources in the United States and are now ubiquitous sources of help worldwide. Yet, pervasive concerns exist about the clinical effectiveness of these services and the extent to which high-risk individuals are utilizing these resources. As such, we – from federal agencies to local communities – are relying on a relatively untested method of help for those individuals considered most at risk for suicide death.
Telephone crisis services are resources for individuals at imminent risk for suicide and are an important component of suicide prevention efforts in the United States. The Substance Abuse and Mental Health Services Administration’s (SAMHSA), Center for Mental Health Services (CMHS) funds a National Suicide Prevention Lifeline Network (NSPL), which consists of two toll–free telephone numbers (1–800–273–TALK and 1–800–SUICIDE) that route calls from anywhere in the United States to a network of local crisis centers. In turn, the local centers link callers to local emergency, mental health, and social service resources. These two national hotline numbers provide back-up resources to a number of suicide prevention programs, including public awareness messaging campaigns, school-based suicide prevention programs, and federal-, community- and advocacy-information/referral documents and internet sites. Every month, almost 44,000 calls are answered through the NSPL. Not every caller is at acute risk for suicide but past SAMHSA-funded hotline evaluations have shown that a large number of callers have significant histories of suicidal ideation and attempts (Kalafat et al., 2007). Crisis centers provide invaluable services for those at imminent risk for suicide; emergency intervention is frequently initiated and may result in a psychiatric hospitalization or other acute mental health services may be provided. For those not at imminent risk, crisis hotlines will typically provide referrals to mental health and other services, and will also advise the caller that they may call back if they are in crisis or have additional needs. The SAMHSA evaluations demonstrated that callers experienced a reduction in hopelessness and suicidal intent, but also showed that 43% of suicidal callers, who completed follow-up assessments, experienced some recurrence of suicidality (ideation, plan, or attempt) since their crisis call (Gould et al., 2007).
This recurrence of suicidal thinking underscores the importance of receiving follow up behavioral healthcare or other appropriate services or interventions. However, the hotline evaluations found that, upon follow-up, 12.6% of callers had scheduled an appointment but not be seen at the time of follow-up, and 22.5% of suicidal callers did not make an appointment after receiving a referral to the behavioral health care system (Gould et al., 2007 & Kalafat et al., 2007). Clearly, there is a critical need to promote more systematic follow up of callers at risk of suicide.
In addition to the ongoing need to monitor and ensure quality of calls and gather post-crisis call information from the callers themselves, there is an additional need to identify the effective practices that facilitate caller follow-up and referral to the behavior health system. The proposed data collection activity will continue previously cleared efforts to monitor and ensure call quality and will add additional evaluation efforts for a pilot project that systematically follows-up with NSPL callers. The evaluation efforts associated with the pilot project will focus on a new cooperative agreement between SAMHSA and five crisis centers in the National Suicide Prevention Lifeline (NSPL) network that were funded at the beginning of federal fiscal year 2009.
The pilot project will evaluate the new initiative to offer and provide follow up to all NSPL callers who have reported suicidal desire during or within 48 hours before making a call to NSPL. Centers involved in this project will receive training in motivational interviewing/safety planning and case management techniques (MI/SP Intervention), designed to enhance follow- up and assist in keeping callers safe after the call and before they are seen by a health care provider.
The National Suicide Prevention Lifeline Crisis Center Follow Up grants are authorized under Section 520A of the Public Health Service Act, as amended. This announcement addresses Healthy People 2010 focus area 18 (Mental Health and Mental Disorders).
Previously Approved Clearance
The previously submitted OMB clearance request was approved for the first 3 years of the evaluation of the crisis hotline. The attachments associated with the original clearance request are:
National Suicide Prevention Lifeline—Call Monitor Form (Attachment A)
Crisis Hotline Telephone Initial Script (Attachment B)
Crisis Hotline Telephone Consent Script (Attachment C)
Crisis Hotline Telephone Follow-up Assessment (Attachment D)
The evaluation’s design includes silent monitoring of crisis callers utilizing the National Suicide Prevention Lifeline—Call Monitoring Form. Silent monitors code the type of problem presented by the caller, the elements of a suicide risk assessment that are completed by the crisis worker, as well as what action plan is developed with the caller and what referrals are provided to the caller. Additionally, the evaluation includes follow-up data collection utilizing the Crisis Hotline Telephone Follow-up Assessment. The purpose of this assessment is to collect data regarding: (1) suicide risk status at the time of and since the call, (2) depressive symptoms at follow-up, (3) service utilization since the call, (4) barriers to service access, and (5) the caller’s perception of the efficacy of the hotline intervention.
Clearance Request
SAMHSA is requesting approval for the continuation of the original evaluation package (OMB No. 0930–0274). This request also includes an expanded effort to better understand the process and impact of motivational interviewing and safety planning training with callers in crisis.
The program is authorized under Section 520A of the Public Health Service Act (42USC290bb-32). Each year, beginning with the 2001 appropriations bill, Congress has directed that funding be provided for the Suicide Prevention Hotline program. In addition to the Suicide Prevention Hotline Program, funds have been continually allocated for the evaluation.
Crisis hotlines in the NSPL network must be accredited by recognized bodies. To ensure quality, many crisis centers conduct periodic on-site monitoring of selected calls using unobtrusive listening devices, but this information is generally used for supervisory purposes and does not contribute to the evidence-base of hotline services. SAMHSA monitors the quality of calls through the Evaluation of the Networking Suicide Prevention Hotlines. The goal of the evaluation is to improve standards and methods of service delivery.
SAMHSA uses the Evaluation of the Networking Suicide Prevention Hotlines to collect data on follow-up assessments of individuals calling the Lifeline network. This effort will provide an empirical evaluation of crisis hotline services, which is necessary to add to the evidence base and optimize public health efforts to prevent suicidal behavior. The evaluation will contribute to two of SAMSHA/CMHS’s National Outcome Measures (NOMs) – “Perception of Care” and “Access to Care.” The “Perception of Care” NOM is addressed through questions to participants about the outcomes of the help they received during calls to the Network, including their emotional state and suicidal risk before, during and after the call. The “Access to Care” NOM is addressed by determining whether the caller followed up with provided referrals.
The components of the original evaluation collect data by monitoring calls routed to 18 crisis centers (selected on the geographic region(s) they serve) during shifts of consenting staff. It will assess the degree to which crisis staff complete suicide risk assessments on appropriate (i.e., severe crisis and suicide) calls, and determine what intervention was provided, and what type of referral was provided to the caller. This work collects follow-up information from the callers themselves. Additionally, this work compares the types of callers calling into the two separate hotlines to assess the phone number usage and reach.
The expanded components of the evaluation will (Evaluation Expansion) evaluate additional but related activities recently funded through a new cooperative agreement between SAMHSA and five crisis hotline centers in the NSPL network. The data collection will evaluate the process and impact of motivational interviewing and safety planning (MI/SP) with callers who have expressed suicidal desire. The five grant-funded centers will train counselors to implement an intervention with callers during their initial call to a center, which incorporates aspects of MI/SP and utilizes an evidence-based practice model to provide follow-up to callers who have expressed a suicidal desire.
The requested changes for the new clearance request are described below.
The original OMB clearance was requested and approved for the first 3 years of the evaluation. Respondent burden for the revised clearance is calculated for the next 3 years of data collection.
The number of participating crisis centers for the continued evaluation remains at 18. For the expanded evaluation component, five new centers will participate.
Four new data collection instruments and three new consents have been added to this revised approval submission. Estimates in this package are based on the participation of the five new crisis centers.
SAMHSA is requesting OMB review and approval of the following items:
National Suicide Prevention Lifeline—Call Monitor Form (see Attachment A)
Crisis Hotline Telephone Initial Script (see Attachment B)
Crisis Hotline Telephone Consent Script (see Attachment C)
Crisis Hotline Telephone Follow-up Assessment (see Attachment D)
MI/SP Silent Monitoring Form (see Attachment E)
MI/SP Caller Initial Script (see Attachment F)
MI/SP Caller Follow-up Consent Script (see Attachment G)
MI/SP Caller Follow-up Interview (see Attachment H)
MI/SP Counselor Consent (see Attachment I)
MI/SP Counselor Attitudes Questionnaire (see Attachment J)
MI/SP Counselor Follow-up Questionnaire (see Attachment K)
A2. Purpose and Use of Information
The data collection will contribute to the evidence-base of suicide prevention hotlines and will improve standards and methods of service delivery to suicidal callers by informing the development of staff training in networked crisis centers. The information will be compiled in a report for SAMHSA who may choose to disseminate it.
Information and findings from the evaluation can help SAMHSA plan and implement other efforts related to suicide prevention. SAMHSA can also use the findings from the evaluation to provide objective measures of its progress toward meeting targets of key performance indicators put forward in its annual performance plans as required by law under GPRA.
Findings from the evaluation can be used by crisis centers to improve their services, processes, and functions. Centers can use the information gathered to better identify their target populations and improve their services and outcomes.
The research community, particularly the field of mental health services research, will benefit in a number of ways from the information gathered through this evaluation. First, the crisis hotline evaluation will significantly add to the developing research base about the use of hotline services. Second, the focus on suicidal callers allows researchers to examine and understand who is being served with hotline services and the outcomes of receiving these services. Third, the evaluation examines the reach of populations served by the two different hotlines. Finally, the analysis of evaluation data aids researchers in formulating new questions about the NSPL network and helps both service providers and researchers improve the delivery of crisis hotline services.
The specific data collection activities from the original evaluation are:
National Suicide Prevention Lifeline–Call Monitor Form; and
Crisis Hotline Telephone Follow-up Assessment Interview Form
All questions in Section I of the National Suicide Prevention Lifeline–Call Monitor Form examine whether the crisis worker is following “best practice” protocol for suicide assessment which is a prerequisite for being able to provide appropriate referrals. This instrument is completed by a trained counselor monitoring the crisis call. In addition, Section II of the National Suicide Prevention Lifeline–Call Monitor Form directs the monitor to note:
Whether the crisis worker provided the caller with a referral;
The source of the referral; and
The kind of referral that was made.
Using Sections III and IV, the evaluators will determine whether the referrals made were appropriate. The responses to these questions will help CMHS:
Ensure that appropriate referrals are provided;
Build and enhance its central database to provide the information crisis centers need to provide appropriate referrals to suicidal callers; and
Develop staff training to improve access to care and the overall quality of services provided.
The telephone scripts associated with the data collection instruments are intended to provide potential participants with standardized information to inform their consent decision. Trained crisis counselors will use the Crisis Hotline Telephone Initial Script to ask for permission to have the evaluation staff re-contact the caller. The Crisis Hotline Telephone Consent Script, used at the time of re-contact, incorporates the required elements of a written consent form, such as:
A statement that the study involves research, an explanation of the purposes of the research and the expected duration of the subject’s participation, a description of the procedures to be followed, and identification of any procedures which are experimental;
A description of any reasonably foreseeable risks or discomforts to the subject;
A description of any benefits to the subject or to others which may reasonably be expected from the research;
A disclosure of appropriate alternative procedures or courses of treatment, if any, that might be advantageous to the subject;
A statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained; and
A statement that participation is voluntary.
The Crisis Hotline Telephone Follow-up Assessment Interview Form is focused on client-centered outcomes and will be conducted one time with each consenting caller. It will include an evaluation of the caller’s suicide risk status at the time of and since the call, depressive symptoms at follow-up, service utilization since the call, barriers to access, and the client’s perception of the efficacy of the hotline intervention.
New York State Psychiatric Institute is the Institutional Review Board of record for the original Evaluation.
Clearance is also being requested for four new activities that represent the necessary next steps for hotline evaluation – steps that were deemed necessary through prior research (Kalafat et al., 2007 & Gould et al., 2007). The new data collection activities are:
MI/SP Silent Monitoring Form;
MI/SP Caller Follow-up Interview;
MI/SP Counselor Attitudes Questionnaire; and
MI/SP Counselor Follow-up Questionnaire
These activities have been designed to evaluate the process and impact of motivational interviewing and safety planning (MI/SP) with callers who have expressed suicidal desire. The MI/SP Counselor Attitude Questionnaire will be administered to hotline counselors at the conclusion of the MI/SP training and will be used as a predictor of fidelity of the MI/SP intervention. The MI/SP Silent Monitoring Form will be completed by research monitors who will access a remote “real time” monitoring system through the internet to conduct silent monitoring. The purpose of the form is to collect the following information:
Call specifics for each call, such as date, time, and length;
Suicide risk status of the caller;
Information on elements of safety planning, such as making the environment safe and identifying triggers that led to the caller’s suicidality;
Types and referrals the counselor gave and to what services;
Ratings of counselors behaviors and caller behavioral changes that occurred; and
Re-contact permission status
Crisis counselors must read and sign a MI/SP Counselor Consent before monitoring and collecting the data. This form explains the purpose of the research, privacy, risks and benefits, what the study entails, and participant rights.
The MI/SP Counselor Follow-up Questionnaire will be completed by counselors that follow-up with or attempt to contact crisis callers after the initial call. This instrument provides an assessment of the outreach, whether the caller followed-up with referrals or resources provided during the initial call, whether MI/SP was implemented, and challenges and benefits to MI/SP.
The MI/SP Caller Follow-up Interview will be conducted with the caller approximately six weeks after the initial monitored call to the center. The purpose of the interview is to collect demographic data, caller feedback on the initial call made to the center, suicide risk status at the time of and since the call, current depressive symptomology, follow through with the safety plan and referrals made by the crisis counselor, and barriers to service.
A3. Use of Information Technology
The National Suicide Prevention Lifeline–Call Monitoring Form and MI/SP Silent Monitoring Form are hard copy forms so that the monitors listening to the crisis calls will be able to easily turn pages to locate items on the form. This is important because the calls may not proceed in the same sequence as the form itself. It is easier for monitors to scan and turn pages than to scroll up and down computer screens. Thus, call monitors will be on the phone, listening to the hotline call, while completing the form in hard copy. The forms are computerized for the purpose of data entry with the data entry interface mirroring the hard copy form.
The Crisis Hotline Telephone Follow-up Assessment and will be conducted by a trained crisis worker and will include an assessment of suicidal ideation (thoughts) and behavior. This data collection involves human subjects (i.e., crisis hotline callers) will be administered through telephone and computer assisted interviews. The MI/SP Caller Follow-up Interview also involves human subjects and will be administered in the same manner. The key rationale behind this decision is that participants in the evaluation are considered high-risk individuals, in that they will have called a crisis center within the past 30 days and expressed suicidal thoughts or behavior, or revealed that they were in a crisis situation. Protocols will be in place that will enable immediate access to professional mental health services, should this be warranted. CMHS believes that computer-assisted telephone interviews are the most professional, simplest, and convenient, and the least time-consuming collection method. Given the possibility that a respondent could require professional mental health assistance, use of the telephone is also the method that would be most responsive to the public’s needs.
The MI/SP Counselor Attitudes Questionnaire and MI/SP Counselor Follow-up Questionnaire will be administered in hard copy to the crisis counselors at the participating crisis centers.
A4. Efforts to Identify Duplication
The information will be collected only for the purposes of this program and is not available elsewhere.
A5. Involvement of Small Entities
The information collected will not have a significant impact on small entities.
A6. Consequences if Information Is Collected Less Frequently
The current application represents a one-time data collection effort.
A7. Consistency With Guidelines of 5 CFR 1320.5
This information collection fully complies with 5 CFR 1320.5 (d) (2).
A8. Consultation Outside the Agency
SAMHSA published a 60-day notice in the Federal Register on Friday, November 21, 2008 (FRN 70-663), soliciting public comment on this study. No comments were received.
Consultation on the design, instrumentation, data availability and products, and statistical aspects of the evaluation occurred throughout the development of the evaluation design process and throughout the first 3 years of the evaluation. Directors and representatives to the National Suicide Prevention Lifeline Steering Committee also provided feedback to the evaluation design and data collection instruments. Steering committee members have been regularly updated and apprised of milestones and accomplishments of the evaluation. Although this evaluation does not affect current initiatives in any other Federal agency, a number of Federal agencies are concerned about suicide prevention. CMHS briefed representatives from the following agencies on the evaluation’s design and goals:
Centers for Disease Control and Prevention
Indian Health Service
National Institute of Mental Health
Health Resources and Services Administration
Veterans Administration
A9. Payment to Respondents
The 18 participating crisis centers will receive an incentive of $5,000 per year ($10,000 total) for their participation and support of the project which will be distributed quarterly in equal installments. As previously approved (OMB No. 0930-0274), crisis center callers will receive $50 for the follow-up interview.
Respondents will be offered $50 remuneration for participating in the MI/SP follow-up telephone assessments. Based on experience in the previous evaluation, if callers on this project are not offered at least $50 for participating in these lengthier assessments, it is anticipated that only 25–30% of callers would agree to receive a follow-up call and that only 50% of that group would actually participate in the follow-up interviews. This level of participation would constitute a biased sample, which would be unrepresentative crisis and suicide callers.
The five participating crisis centers will not receive a financial incentive through the evaluation as they are currently receiving SAMHSA funding through their cooperative agreements.
A10. Assurance of Confidentiality
For all studies in the evaluation, all reports and publications from these data include only group-level analyses that fully protect the privacy of individual participants, and no data have been or will be stored with identifying respondent information.
Strict measures to ensure privacy will continue to be followed. Only if re-contact permission is granted will the caller’s identifying information be coded temporarily on the silent monitor abstract form until the linkage is completed. Once the linkage has been completed, the identifying information will be replaced by matching codes on the monitor and follow-up forms for consenting callers. Neither the consenting counselor nor the caller will be identified by the monitor on the National Suicide Prevention Lifeline—Call Monitoring Form or the MI/SP Silent Monitoring Form. These forms will include a code for the crisis center, but no crisis center will be identified in any reports. Privacy would only be lost in the rare case of a monitor hearing what is judged to be a clear and imminent threat of harm to the caller or to someone with whom the caller has contact, which does not appear to have been responded to by the crisis counselor. Their procedure will consist of immediately contacting the supervisor on duty at the time of the call to apprise them of the situation and inquire if an emergency response such as a call trace or rescue protocol has been implemented.
A recording will inform callers who call during the shifts of consenting crisis staff that some calls may be monitored for quality assurance purposes. Previous reports from crisis centers that had this message found no difference in rates of hang ups before the calls were answered or within the first 15 seconds of the calls as compared to centers with similar call volumes that did not have the message.
Respecting participants’ preferences for contact method
At the end of the call, appropriate callers will be asked for permission to be contacted by the evaluation team (see Crisis Hotline Initial Script, Attachment C & MI/SP Caller Initial Script, Appendix G). The Crisis Hotline Initial Script and the MI/SP Caller Initial Script protect the privacy of callers by asking the caller how and when they want to be contacted, and what type of message (if any) can be left on an answering machine or with the person picking up the telephone. The caller also has the option of not providing contact information to the crisis center if he/she prefers to call the evaluation team back directly.
Participants preferring to call back directly will be given telephone numbers for the Project Director’s (PD) home landline and one for her cell phone, with instructions to call collect. (This will meet the needs of most participants, who will call the project in the evening or weekends, as well as callers from the West Coast whose evening calls often result in late evening calls [Eastern Time].) Participants will also be given the office number, but will be informed that they cannot call collect on this number. Callers using this number will be called back immediately so that they will not incur costs.
After an initial message is left, unreachable potential participants will be called back at a later time. Similar procedures to the ones described above will be employed, leaving the potential respondent the PD or interviewers’ home or cell phone numbers, for which they alone have access. As described above, they will also be given the office number and will be called back immediately if they use it.
Maintaining secure procedures for confidential information
Secure procedures will be maintained for personal identifiers/call back information provided by potential participants. Crisis centers will transfer this confidential contact information to the evaluation PD by telephone (speaking directly to the PD) or by secure faxes. A fax machine devoted to the project has been set up in a locked room that is only accessible to research staff. In turn, the PD will provide contact information to the follow-up interviewers in person or by telephone.
All hard-copy forms containing personal identifiers will be stored under lock and key in the PD’s office; only the PI, PD, and Database Administrator/Data Analyst will have access to those files. All files containing personal identifiers will be destroyed at the end of the project.
A case number, rather than a caller’s name, will be used for the computerized follow-up assessment instruments. The instruments will be on a computer that is password protected and kept secured at all times.
All computerized identifying information on participants (i.e., name, address, telephone number, and telephone consent scripts) will be stored by the PI in password-protected administrative files maintained on the Columbia University Child Psychiatry server, which is behind a “firewall.” Only the PI, PD, and Database Administrator/Data Analyst will know the password. This is the only linkage between specific individuals and the assessment instruments to be collected. All project staff will sign a confidentiality statement saying that they will keep the participants’ answers private.
Once the assessment instruments have been gathered, they will be stored in separate locked files at the evaluation headquarters at the New York State Psychiatric Institute.
For quality assurance purposes, 10 percent of the interviews will be selected for taping, after participants provide informed consent. The tapes will have no identifying information on them. They will be locked in file cabinets and destroyed at the end of the project.
Statement to respondents
The telephone script used when the evaluation team contacts the participant for their follow-up interview (see Crisis Hotline Telephone Consent Script, Appendix D & MI/SP Caller Follow-up Consent Script, Attachment H)) includes (1) the fact that the information collection is sponsored by an agency of the Federal Government, (2) the purpose of the information collection and the uses which will be made of the results, (3) the voluntary nature of participation, and (4) the extent to which responses will be held confidential.
A11. Questions of a Sensitive Nature
Because this project concerns suicide prevention, it is necessary to ask callers questions that are potentially sensitive. However, only information that is central to the study is being sought. Questions address dimensions such as suicidality and other self-injurious behaviors, drug and alcohol use at the time of the call, and access to lethal means. The answers to these questions are used to understand who is being served by the hotlines, correlates of help-seeking after the initial crisis intervention, and hotline intervention outcomes.
Because the monitor never actually speaks to the caller, the monitor will never ask the caller any sensitive questions. The counselor will be discussing sensitive issues with the caller as a function of the crisis call. Counselors will not be asking sensitive questions as a function of the evaluation.
Additionally, the purpose of this evaluation is to collect follow-up information on participants’ mental health status after their call to a hotline. This information is sensitive, but important to expanding the evidence base for suicide prevention hotlines.
The crisis counselors at participating centers are trained to obtain informed consent. They will only use the Crisis Hotline Telephone Initial Script or the MI/SP Caller Initial Script if, at the end of the telephone crisis counseling intervention (approximately 20 minutes) they believe that the caller has the capacity (emotionally and intellectually) to understand the script/request. During the crisis counseling, the counselor will be able to decide whether the caller is able to follow the conversation and respond in a meaningful manner, and whether they are sufficiently calm at the end of the call to consider the request. Only then would the caller be approached for a follow-up contact. (Note that callers who are under 18 years old are screened out at the beginning of the script. Non-English speakers will also be screened out).
Approximately one week after a caller consents to be re-contacted, they will talk to a trained crisis counselor, who will use the Crisis Hotline Telephone Consent Script, which incorporates all elements normally included in a written informed consent form. The script will ask for consent to participate in the Crisis Hotline Telephone Follow-up Assessment, as well as permission for the evaluation staff to obtain baseline information by the counselor who helped them during their crisis call. The caller’s consent will be audio taped. At that point, ten percent of the callers will also be asked whether they would agree to the audio taping of their actual Crisis Hotline Telephone Follow-up Assessment; the counselor will explain that this will be done for quality control purposes and that it is not a requirement for their participation. The caller’s response to this request will also be audio taped.
The demographic information, caller feedback on the crisis call, a depression scale, and whether the caller followed through with the safety plan and referral will be collected. Additionally, a retrospective assessment of suicidal ideation, plans, preparatory and suicidal behavior, and intent to die at the time of the original call to the center is also included.
Approximately six weeks after a caller consents to be re-contacted, they will talk to a trained crisis counselor, who will use the MI/SP Caller Follow-up Consent Script, which incorporates all elements normally included in a written informed consent form. The script will ask for consent to participate in the MI/SP Caller Follow-up Interview, as well as permission for the evaluation staff to obtain baseline information on referral recommendations by the counselor who helped them during their crisis call. The caller’s consent will be audio taped. At that point, ten percent of the callers will also be asked whether they would agree to the audio taping of their actual MI/SP Caller Follow-up Interview; the counselor will explain that this will be done for quality control purposes and that it is not a requirement for their participation. The caller’s response to this request will also be audio taped.
A12. Estimates of Annualized Hour Burden
Table 1 shows the burden associated with the cross-site evaluation during years 4–6 of the evaluation, the period for which OMB clearance is being sought. Burden estimates presented in Table 1 are based on information supplied by various sources. Measures that were developed for the original evaluation were piloted by the evaluator to determine average burden estimates and have since been implemented in the field, allowing for updated burden estimates. These measures include the National Suicide Prevention Lifeline—Call Monitoring Form, Crisis Hotline Telephone Initial Script, Crisis Hotline Telephone Consent Script, and the Crisis Hotline Telephone Follow-up Assessment.
A total of 365 calls will be monitored, utilizing the National Suicide Prevention Lifeline—Call Monitoring Form, during the first 5-month period following OMB clearance; calls at each of the 18 sites will be monitored. Follow-up assessments will be implemented utilizing the Crisis Hotline Telephone Follow-up Assessment.
A total of 75 counselors per year will be trained to implement motivational interviewing and safety planning (MI/SP) with callers and complete the MI/SP Counselor Attitude Questionnaire. Two monitors across the five funded centers will monitor 37 calls for a total of 370 monitored calls utilizing the MI/SP Silent Monitoring Form. Approximately 6 weeks after the initial call, and follow-up consent is obtained, interview will be conducted with 295 callers throughout the study period utilizing the MI/SP Caller Follow-up Interview.
Table 1
Evaluation of Networking Suicide Prevention Hotlines—Revision
Estimated Annual Burden
Note: Total burden is annualized over the 3-year clearance period.
Instrument |
Number of respondents |
Responses / Respondent |
Burden/ response (hours) |
Annual burden (hours) |
Hourly Wage |
Total Hourly Cost |
Evaluation Continuation |
||||||
National Suicide Prevention Lifeline–Call Monitoring Form |
10 |
44 |
.58 |
255 |
$33.522 |
$8,548 |
Crisis Hotline Telephone Initial Script |
365 |
1 |
.08 |
29 |
$19.883 |
$577 |
Crisis Hotline Telephone Consent Script |
365 |
1 |
.17 |
62 |
$19.883 |
$1,233 |
Crisis Hotline Telephone Follow-up Assessment |
365 |
1 |
.67 |
245 |
$19.883 |
$4,871 |
Evaluation Expansion |
||||||
MI/SP Silent Monitoring Form |
10 |
37 |
.58 |
215 |
$33.522 |
$7,207 |
MI/SP Caller Initial Script |
368 |
1 |
.08 |
29 |
$19.883 |
$577 |
MI/SP Caller Follow-up Consent Script |
368 |
1 |
.17 |
63 |
$19.883 |
$1,252.44 |
MI/SP Caller Follow-up Interview |
295 |
1 |
.67 |
198 |
$19.883 |
$3,936 |
MI/SP Counselor Consent |
75 |
1 |
.08 |
6 |
$18.974 |
$114 |
MI/SP Counselor Attitudes Questionnaire |
75 |
1 |
.25 |
19 |
$18.974 |
$360 |
MI/SP Counselor Follow-up Questionnaire |
175 |
2 |
.17 |
60 |
$18.974 |
$1,138 |
Total |
||||||
Total |
2,471 |
– |
– |
1,181 |
– |
$29,813 |
Rounded to the nearest whole number.
Assuming mean hourly wage of social scientists and other labor workers taken from Bureau of Labor Statistics, May 2007 National Occupational Employment and Wage Estimates. http://www.bls.gov/oes/current/oes_nat.htm#b21-0000
Assuming mean hourly wage of mental health counselors taken from Bureau of Labor Statistics, May 2007 National Occupational Employment and Wage Estimates. http://www.bls.gov/oes/current/oes_nat.htm#b21-0000
Assuming mean hourly wage of civilian workers taken from Bureau of Labor Statistics, National Compensation Survey: Occupational Earnings in the United States, 2007, Summary. http://www.bls.gov/ncs/ocs/sp/nctb0298.pdf
A13. Estimates of Annualized Cost Burden to Respondents
The respondents will not incur any capital, startup, operational, or maintenance costs.
A14. Estimates of Annualized Costs to the Government
SAMHSA has planned and allocated resources for the management, processing, and use of the collected information in a manner that enhances its utility to agencies and the public. Including the Federal contribution that funds the grantees participating in the expanded evaluation efforts (i.e., MI/SP evaluation), the contract with the evaluator and Government staff to oversee the evaluation, the annualized cost to the Government is estimated at $446,623. These costs are described below.
Approximately $350,000 per federal fiscal year for each of three years has been planned to fund grantees participating in the MI/SP evaluation, of which 10% can be dedicated to evaluation which results in an estimated annualized grantee evaluation cost of $35,000. Awards or plans for future awards have been made to cover the continuation and enhanced evaluation in the annualized cost of $409,223. An estimated 72 hours per year of a senior GS-14 level federal staff member will be required for oversight to the evaluation efforts for an annualized cost of $2,400.
A15. Changes in Burden
The estimate of total burden hours associated with the original 3-year approval period was 2,586 hours. SAMHSA is requesting 3,543 hours for this submission, an increase of 957 hours. The additional data collection activities to assess the process and impact of motivational interviewing and safety planning (MI/SP) have resulted in increased burden.
The estimate of total burden hours associated with the Evaluation Continuation for the 3-year approval period is 1,773 hours.
The estimate of total burden hours associated with the 3-year approval period for the Evaluation Expansion is 1,770.
A16. Time Schedule, Publication, and Analysis Plans
Time Schedule
The time schedule for continuing the evaluation is summarized in Tables 2 and 3. A 3-year clearance is requested for this project.
TABLE 2
Time Schedule
Activity |
Timeline |
Receive OMB re-approval for study |
March 2009 |
Continue call monitoring and data collection |
March 2009–February 2010 |
Data collection complete |
February 2010 |
Analysis complete |
April 2010 |
Final report written |
September 2012 |
TABLE 3
Time Schedule – Evaluation Continuation
Activity |
Timeline |
Receive OMB approval |
March 2009 |
MI/SP Procedures implemented |
Ongoing after OMB approval |
Call monitoring/Data collection |
March 2009–February 2012 |
Analysis completed |
June 2012 |
Final report written |
September 2012 |
Publication Plan
A final report will be submitted to SAMHSA with anticipated subsequent dissemination to other interested parties, such as researchers, policymakers, and program administrators at the Federal, State, and local levels. Although not required under the evaluation contract, it is also anticipated that results from this data collection will be published and disseminated in peer-reviewed publications such as Suicide and Life Threatening Behavior, similar to the published articles from prior phases of the hotline evaluation efforts (i.e., Kalafat et al., 2007 and Gould et al., 2007).
Data Analysis Plan
All of the data collection and analytic strategies detailed in this package are linked to the evaluation questions.
CMHS expects to be able to answer the following questions from this evaluation:
Are there baseline differences in the extent to which crisis counselors at different types of centers complete suicide risk assessments on appropriate (i.e., severe crisis and suicide) calls?
Are there baseline differences in the extent to which crisis counselors at different types of centers provide intervention to callers in crisis?
Are there baseline differences in the extent to which crisis counselors at different types of centers provide appropriate referrals to callers in crisis?
What are the usage patterns associated with 1-800-SUICIDE and 1-800-273-TALK? Are the two 800-numbers reaching callers with different crisis- and suicide-profiles? Are there gender, age, and ethnic differences between callers to each 800 number? Is one number superior in reaching seriously suicidal individuals?
CMHS expects to be able to answer the following questions from this evaluation:
What is the impact of MI/SP training?
What is the reach of the follow-up strategies?
What types of follow-up strategies are used?
What kings of referrals are made?
What types of linkages are made as a result of the referral mechanism used?
What are the process measures?
How closely did site plans match actual implementation (i.e., fidelity to plan)?
What type of deviation from the plans occurred?
What led to the deviations?
What impact did the deviations have on referral follow-up?
What are the mediators associated with changes in outcomes of the pilot follow-up?
The statistical analyses and power calculations take into account the matched, hierarchical structure, and stratification strategy of our evaluation. Mixed effects linear models will be estimated. Individual centers and unexplained error are random effects; time (first and second time periods, i.e., pre- and post-training and resource data base) and center type (i.e., local, within state, outside state) are fixed effects. To examine dichotomous variables, such as referral successfully accessed, risk assessment conducted, callers’ at-suicide-risk status, a logistic linear regression model will be evaluated. To examine the effects of continuous variables, such as distance between caller and center, and caller and accessed service, the same set of linear models as above will be estimated using the identity link function (appropriate for normal distributions). PROC MIXED in SAS will be used to estimate these models. The analyses will address our two main research questions: (1) Are there baseline differences in outcomes (i.e., “pre” period) between the types of centers? This will involve three pairwise contrasts among the different types of centers, using the contrast statements in PROC MIXED and in PROC GENMOD. (2) The descriptive analyses of the monitor forms will primarily use frequency distributions and counts from items in the monitor form, including type of call (suicide or non-suicide crisis); presenting problem; whether the crisis worker explored suicide; and whether the crisis worker inquired about current ideation, plans, intent to die, hopelessness, depression, and previous suicide attempts. Frequencies will also be derived as to type of action plans for callers, whether a rescue protocol was initiated, and whether/what type of referral was provided. Changes in frequencies of these interventions between the two data collection periods will also be assessed in a pre-post no-comparison group design. These analyses will provide information as to the need for further risk assessment training and referral resources for crisis centers.
The aim of comparing callers to 1-800-SUICIDE and 1-800-273-TALK will largely be addressed by descriptive comparisons of the calls that come in to the two lines, and the demographics and psychiatric histories of the callers using the two lines. Information with respect to the calls will come from the Silent Monitoring Form. Information on the characteristics of the callers will come from the follow-up interview on the sample of callers who agree to be re-contacted. Two separate sets of tables will be generated. The first will divide all monitored, eligible calls into those that came in on the 1-800-SUICIDE line and those that came in on 1-800-273-TALK. Calls from each of the two lines will be categorized in the tables on the following variables: gender of the caller (inferred by the silent monitor); nature of the crisis (or crises) that precipitated the call; degree of suicidality of the caller (determined from the suicide risk assessment portion of the silent monitoring form or the impression of the call monitor if the risk assessment was incomplete). This set of tables will include one table for all monitored calls across the sites, as well as separate tables for each site.
The second set of tables will divide all followed callers into those who called on 1-800-SUICIDE and those who called on 1-800-273-TALK. By design, the numbers of callers in each group will be roughly equal. In this set of tables, the two groups will be categorized on the following variables: age, gender, race/ethnic group, marital status, and educational attainment level of the caller; the composition of the caller’s household; the caller’s residential and migration history; veteran status, the caller’s self-report of the crisis (or crises) that precipitated the call; the caller’s history of suicidal behaviors; and the caller’s current level of depressive symptoms. This set of tables will include one table for all followed callers over the course of the study, as well as separate tables for the followed callers from each site. It is expected that these two sets of tables will provide the primary basis for decisions regarding the future of the separate lines.
It will also be important to compare both the monitored calls and the followed callers from the two different lines in a multivariate framework. This is because certain patterns across the lines may actually be spurious associations resulting from the action of a third variable. For example, an overrepresentation of a particular racial or ethnic group on one or the other of the two lines could possibly be an artifact of the demographic character of the population served by a particular site, and the fact the line in question received greater publicity in that particular location. Again, separate sets of analyses will be conducted on the sample of monitored calls and the sample of followed callers, as these two groups will not overlap and the information available for each will differ. Both sets will consist of a series of logistic regression analyses predicting whether a particular call came in on 1-800-SUICIDE as opposed to 1-800-273-TALK (or whether a particular caller used 1-800-SUICIDE as opposed to the other line). Sets of predictors will be considered one at a time in separate blocks. For example, in the analysis of followed callers the predictors will include: a block of 19 dummy variables to represent the 18 crisis centers (sites), a block representing the demographic variables, a block representing suicide behavior history, a block representing psychiatric history, etc. The final model for both of the two sets of analyses will include all predictors. This final model will provide the independent associations of each of the independent variables to the odds of using 1-800-SUICIDE as opposed to 1-800-273-TALK, net of all other variables in the model.
The descriptive analyses of the monitor forms will primarily use frequency distributions and counts from items in the monitor form, including; presenting problem; whether the crisis worker explored suicide; and whether the crisis worker inquired about current ideation, plans, intent to die, hopelessness, depression, and previous suicide attempts. Frequencies will also be derived as to type of safety plans for callers, whether a rescue protocol was initiated, and whether/what type of referral was provided. Changes in frequencies of these interventions between the two data collection periods will also be assessed in a pre-post no-comparison group design.
To examine the extent to which the application of MI/SP by counselors is directly related to caller outcome (at research follow-up), follow-up assessments will be linked to the monitored calls. Moreover, the assessments of the implementation and fidelity of the MI/SP training and the clinical follow-up site processes will be linked to the silent monitoring and follow-up assessments to allow the examination of the possible mediating influence of the fidelity and process variables on outcome. The statistical analyses will take into account the hierarchical structure of the study design (i.e., callers within five different crisis centers). Clustering of observations within the site will be handled primarily using fixed effects methods. Random effects models are likely to be biased by failing to account for all site level influences. Nevertheless, alternative random effects models will be utilized and goodness-of-fit tests implemented using the Hausman method (Greene, 2000).The effectiveness of the MI/SP training should be reflected in caller outcomes during the call and in the short-term (i.e., six weeks after the call, as assessed by the follow-up interviews) as well as improvements among callers who talked with trained counselors relative to untrained ones.
A17. Display of Expiration Date
The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.
A18. Exceptions to Certification Statement
This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.
Statistical Methods
B1. Respondent Universe and Sampling Methods
There are 106 crisis centers in the National Suicide Prevention Lifeline Network. The evaluation will continue to work with 18 of those centers. The centers were selected based on the geographic region(s) they serve. The first type of center only takes calls from a metropolitan or county area (“local” centers); the second type of center take calls from their entire state (“within state” centers); the third type of center takes calls from outside of their state (“outside state” centers). All crisis counselors at participating centers will be invited to participate. If even one counselor on a shift chooses not to participate, no calls will be monitored during that shift.
Eligible calls will include those involving a suicide or crisis situation, using a broad definition of “crisis.”
Five SAMHSA funded crisis centers will participate in the expanded evaluation effort. These crisis centers were selected, in part, because the crisis center has a focus on one or more of the following high risk, high priority, or underserved populations:
Lifeline Veterans Initiative centers providing backup to Canandaigua, VA center for callers who press 1 when they call 1–800–273–TALK;
Lifeline Native American Initiative pilot centers;
Lifeline centers participating in the Spanish sub–network who provide 24/7 Spanish language coverage; or
Lifeline centers that are currently, through an existing contractual relationship with a State Mental Health Authority.
Calls to the five centers, occurring within the 6 p.m. to midnight shift (local center times), will be monitored. While all calls will be initially monitored, only calls from suicidal callers will be subject to a complete abstraction by the monitors and potentially included (e.g., coding presence and adequacy of risk assessments; whether components of MI/SP were conducted; whether referrals were given, and if so, the type of referral, etc.). Calls in which the caller is seeking information and referral, third party calls, obscene calls, and non-suicidal crisis calls will not be abstracted. It is anticipated that there will be three eligible calls per center per week for a total of 78 monitored calls per center. Eligible calls are those calls from suicidal callers.
B2. Information Collection Procedures
Data for the continued evaluation are collected during calls to a participating suicide crisis hotline. Silent monitors listen to the calls and collect data in hard copy format. For standard collection of these data across sites, the National Suicide Prevention Lifeline—Call Monitoring Form was developed and previously approved by SAMHSA (OMB No. 0930–0274). Additional data are collected through the Crisis Hotline Follow-up Assessment. The assessment includes an evaluation of the caller’s suicide risk status at the time of and since the call, depressive symptoms at follow-up, service utilization since the call, barriers to access, and the clients’ perception of the efficacy of the hotline intervention.
Data for the expanded evaluation are collected during calls to the five participating crisis centers. Silent monitors listen to the calls and collect data in hard copy format. For standard collection of these data across sites, the MI/SP Silent Monitoring Form was developed. Approximately six weeks after the initial call to the hotline, at which time initial consent to contact is obtained, a trained counselor contacts the caller and obtains data through the MI/SP Caller Follow-up Interview. Demographic and historical data are collected along with indicators of the efficacy of the intervention, including the safety plan and provision of resources for help.
In addition, once counselors are training in MI/SP, they will provide structured feedback in hard copy form through the MI/SP Counselor Attitudes Questionnaire. The data collected will include utility of the training, likelihood of implementation of MI/SP with crisis callers, and the extent to which the counselor will be able to execute MI/SP as intended.
Table 2 summarizes the information collection procedures across all components of the evaluation.
Procedures for the Collection of Information
Measure |
Indicators |
Data Source(s) |
Method |
When Collected |
Evaluation Continuation |
||||
National Suicide Prevention Lifeline—Call Monitoring Form |
|
Hotline caller recorded via research staff |
Silent monitoring of suicidal caller to crisis hotline |
At time of the call to the crisis hotline |
Crisis Hotline Telephone Follow-Up Assessment |
|
Hotline caller |
Interview |
Approximately one week after initial consent to contact is obtained |
MI/SP Silent Monitoring Form |
|
Hotline caller recorded via research staff |
Interview |
At time of the call to the crisis hotline. |
MI/SP Caller Follow-up Interview |
|
Hotline caller |
Interview |
Approximately six weeks after initial hotline call |
MI/SP Counselor Attitudes Questionnaire |
|
MI/SP trained crisis counselor |
Hard copy survey |
Immediately following the training |
MI/SP Counselor Follow-Up Questionnaire |
|
MI/SP trained crisis counselor |
Hard copy survey |
Immediately after the follow-up call with the crisis caller |
B3. Methods to Maximize Response Rates
The directors of crisis centers that agree to participate will be asked to talk to their supervisory staff about describing the study to their staff, noting its confidential/anonymous nature, and encouraging counselors to participate. Most counselors will be accustomed to “silent monitoring,” since most crisis centers use this method for supervisory and quality assurance purposes. Since the counselors will never know whether one of “their” calls has been monitored, and since the data collected will not identify the crisis center or consenting counselor, it is anticipated that counselors will feel “safe” and be willing to participate. CMHS anticipates an 80% response rate.
To increase participation of callers in follow-up interviews (i.e., Crisis Hotline Telephone Follow-up Assessment and MI/SP Caller Follow-up Interview), callers are being offered a $50 remuneration for their participation.
B4. Tests of Procedures
The National Suicide Prevention Lifeline—Call Monitor Form was pilot tested with four different graduate students during May 2005. To review the form, the students received a training (similar to the one that the actual monitors receive), then completed it while listening to two other students role playing a crisis telephone call. The students found the form easy to complete and felt it was understandable.
The Crisis Hotline Telephone Initial Script and Crisis Hotline Telephone Consent Script were both pilot tested during a previous evaluation conducted by Columbia University. At that point, the scripts were refined to make them as clear as possible. A number of items in the Crisis Hotline Telephone Follow-up Assessment were also included in the previous evaluation, during which they were piloted and refined.
The Crisis Hotline Telephone Follow-up Assessment was developed by a team of suicide prevention researchers as well as experienced crisis center directors. During the week of April 11, 2005, three crisis center workers at the Mental Health Association of New York City’s LifeNet crisis center pilot tested the scripts and Crisis Hotline Telephone Follow-up Assessment on nine acquaintances. No changes were made to the scripts as a result of the pilot testing. However, several items were deleted from the Crisis Hotline Telephone Follow-up Assessment because they were found to be redundant. Additionally, two sets of questions were each collapsed from four items to two items (now #17 and 18, and #104 and 105).
The Crisis Hotline Telephone Follow-up Assessment includes the Center for Epidemiological Studies Depression Scale (CES-D, Radloff, 1977), a 20-item measure of current (past week) depressive symptomatology. It is one of the most frequently used and well-validated measures of depression. We had considered using the Beck Depression Inventory (BDI-II, Beck, Steer & Brown, 1996), another frequently used measure with similar psychometric properties and highly correlated with the CES-D (e.g., Roberts et al., 1991; Skorikov & VanderVoort, 2003); however, in our experiences with using the BDI in our earlier studies, we found that its response format - one of four graded responses reflecting different degrees of severity of each of its 21 symptoms – was difficult to administer over the telephone, and was taking approximately 30 minutes. The CES-D’s response format - a 4-point scale of the frequency with which the participants experienced the symptoms in the past week – is simpler to administer over the telephone. In addition to being used as a continuous indicator of severity, recommended cutoff points are available to detect clinical depression. Some references include:
Radloff, L.S. (1977). The CES-D Scale: A Self-Report Depression Scale for Research in the General Population. Applied Psychological Measurement, 1(3), 385-401.
Roberts, R.E., Lewinsohn, P.M., & Seeley, J.R. (1991). Screening for adolescent depression: a comparison of depression scales. Journal of the American Academy of Child and Adolescent Psychiatry, 30, 58-66.
Skorikov, V.B., & VanderVoort, D.J. (2003). Relationships between the underlying constructs of the Beck Depression Inventory and the Center for Epidemiological Studies Depression Scale. Educational and Psychological Measurement, 63, 319-335.
Similar procedures to those in the evaluation continuation will be employed with the expanded evaluation effort. All expanded evaluation components have been reviewed by experts in the field of mental health and piloted to determine burden levels
B5. Statistical Consultants
The evaluator has full responsibility for the development of the overall statistical design and assumes oversight responsibility for data collection and analysis for the evaluation. Training and monitoring of data collection will be provided by the evaluator. The following individuals are primarily responsible for overseeing data collection and analysis:
Madelyn S. Gould, Ph.D., M.P.H.
Professor,
Psychiatry and Public Health (Epidemiology)
Columbia University/NYSPI
1051 Riverside Drive, Unit 72
New York, NY 10032
212-543-5329
Jimmie Lou Munfakh, B.A.
Psychiatry and Public Health (Epidemiology)
Columbia University/NYSPI
1051 Riverside Drive, Unit 72
New York, NY 10032
212-543-5482
Alison Lake, M.A.
Psychiatry and Public Health (Epidemiology)
Columbia University/NYSPI
1051 Riverside Drive, Unit 72
New York, NY 10032
212-543-6714
The SAMHSA project officer responsible for receiving and approving deliverables is:
Danyelle Mannix, M.P.H.
Center for Mental Health Services
SAMHSA
1 Choke Cherry Road, Room 6-1015
Rockville, MD 20857
240-276-1879
References
Gould, M.S., Kalafat, J., Harris–Munfakh, J.L., and Kleinman, M. (2007). An evaluation of crisis hotline outcomes part 2: Suicidal callers. Suicide and Life–Threatening Behavior, 37(3), 338–352.
Greene, W. H. (2000). Econometric analysis (4th ed.). Upper Saddle River, NJ: Prentice Hall.
Kalafat, J., Gould, M.S., Harris–Munfakh, J.L., and Kleinman, M. (2007). An evaluation of crisis hotline outcomes part 1: Nonsuicidal crisis callers. Suicide and Life–Threatening Behavior, 37(3), 322–337.
Radloff, L.S. (1977). The CES-D Scale: A Self-Report Depression Scale for Research in the General Population. Applied Psychological Measurement, 1(3), 385-401.
Roberts, R.E., Lewinsohn, P.M., & Seeley, J.R. (1991). Screening for adolescent depression: a comparison of depression scales. Journal of the American Academy of Child and Adolescent Psychiatry, 30, 58-66.
Skorikov, V.B., & VanderVoort, D.J. (2003). Relationships between the underlying constructs of the Beck Depression Inventory and the Center for Epidemiological Studies Depression Scale. Educational and Psychological Measurement, 63, 319-335.
List of Attachments
Attachment A |
National Suicide Prevention Lifeline—Call Monitor Form |
Attachment B |
Crisis Hotline Telephone Initial Script |
Attachment C |
Crisis Hotline Telephone Consent Script |
Attachment D |
Crisis Hotline Telephone Follow-up |
Attachment E |
MI/SP Silent Monitoring Form |
Attachment F |
MI/SP Caller Initial Script |
Attachment G |
MI/SP Caller Follow-up Consent Script |
Attachment H |
MI/SP Caller Follow-up Interview |
Attachment I |
MI/SP Counselor Consent |
Attachment J |
MI/SP Counselor Attitudes Questionnaire |
Attachment K |
MI/SP Counselor Follow-up Questionnaire |
Page
| File Type | application/msword |
| Author | natalie.j.henrich |
| Last Modified By | SKING |
| File Modified | 2009-03-25 |
| File Created | 2009-03-25 |