Final Hotline SS_B_07.18.13

Monitoring of National Suicide Prevention Lifeline—revision (OMB No. 0930-0274) Statistical Methods

B1. Respondent Universe and Sampling Methods

There are 159 crisis centers in the National Suicide Prevention Lifeline Network. The monitoring will continue to work with the funded crisis centers. Crisis centers funded by SAMHSA to conduct follow-up with suicidal will participate in the data collection. Participating crisis centers have been chosen by SAMHSA to receive funding to support the development or expansion of a clinical follow-up program for suicidal callers and other suicidal individuals. Centers were chosen on the basis of a competitive application process in which attention was paid to centers’ proposed plans for implementing follow-up, their experience and capacity to successfully implement their plans, their provision of services to special populations of interest including veterans, American Indians, and Spanish-speaking individuals, and their commitment to sustaining their follow-up programs after the termination of their grants. Only suicidal callers who accept the center’s clinical follow-up will be subject to an evaluation follow-up (i.e., only callers who are successfully contacted for clinical follow-up by a participating crisis center are eligible for the interview). Due to budgeting and time constraints; the evaluation does not include follow-up with all callers; rather, the evaluation team will attempt to interview only a subset of those callers who are reached by the centers for clinical follow-up, and who give initial permission to a follow-up counselor to be contacted by evaluation staff. Interview participants are randomly selected, stratified by crisis center, from among those callers who give initial permission to be contacted. Based on data collection during previously approved years of this project, approximately 60% of callers who give initial permission to a follow-up counselor to be contacted by evaluation staff are interviewed.

B2. Information Collection Procedures

Approximately six weeks after the initial call to the hotline, for those callers who have given permission to be contacted, an evaluation interviewer who is a trained counselor contacts the caller and obtains data through the MI/SP Caller Follow-up Interview. Demographic and historical data are collected along with indicators of the efficacy of the intervention, including the safety plan and provision of resources for help. In addition, once counselors are trained in MI/SP, they will provide structured feedback in hard copy form through the MI/SP Counselor Attitudes Questionnaire. The data collected will include utility of the training, likelihood of implementation of MI/SP with crisis callers, and the extent to which the counselor will be able to execute MI/SP as intended. Finally, crisis counselors will complete the MI/SP Counselor Follow-up Questionnaire, describing their clinical follow-up protocol with their suicidal clients.

All of the data collection and analytic strategies detailed in this package are linked to the main questions of interest, which are to determine the efficacy of follow-up, what factors might modify its efficacy, and whether the MI/SP training has an impact on the process and efficacy of follow-up. Centers will receive MI/SP training mid-way through the data collection period, enabling a comparison of pre-training and post-training follow-up outcomes. The burden for MI/SP and non-MI/SP trained counselors is included in the annualized burden (i.e., there are not separate groups of trained and not trained counselors; rather, training is introduced with the same group mid-way through the data collection period).

The statistical analyses will take into account the hierarchical structure of our sampling design. Mixed effects linear models will be estimated. The data analyses will be based on a two-level model, which has the benefit of accounting for the clustering of observations within center (callers or counselors nested within center). Analyses will be performed using SuperMix version 1.2 (Hedeker et al., 2009). SuperMix can fit models with continuous, count, ordinal, nominal, and survival outcome variables with nested data, allowing up to three levels of nesting. For analyses of covariance that includes covariates with missing data, we will use multiple imputation (Allison, 2001; Little & Rubin, 2002) for missing values to avoid information loss and potential non-response bias that might arise with complete case analysis that excludes cases with missing data by default.

Table 4 summarizes the information collection procedures across all components of the evaluation.

TABLE 4

Procedures for the Collection of Information

Measure	Indicators	Data Source(s)	Method	When Collected
MI/SP Caller Follow-up Interview	Demographic information Historical data Risk status – current and at the time of the call Efficacy of the hotline intervention Perceptions of crisis counselor Safety plan assessment Resources provided Crisis counselor follow-up call(s) assessment	Hotline caller	Interview administered by evaluation staff	Approximately six weeks after initial hotline call
MI/SP Counselor Attitudes Questionnaire	Ease of implementing MI/SP with callers Perceived helpfulness of MI/SP with potential callers Whether counselor will supplement MI/SP with other resources Potential challenges to implementation of MI/SP with callers Reactions and response to MI/SP training and utilization	MI/SP trained crisis counselor	Self-administered hard copy survey	Immediately following the training
MI/SP Counselor Follow-Up Questionnaire	Callers demographic information Follow-up counselor’s experience and training Crisis center follow-up protocols Contact protocol employed Barriers to follow-up implementation Topical areas if follow-up completed Referrals/resources utilized by caller since initial call MI/SP utilization Challenges/benefits to MI/SP utilization	MI/SP trained crisis counselor	Self-administered abstraction form based on already available clinical data	Immediately after the follow-up call with the crisis caller

B3. Methods to Maximize Response Rates

The directors of crisis centers that agree to participate will be asked to talk to their supervisory staff about describing the study to their staff, noting its private/anonymous nature, and encouraging counselors to participate. Since the data collected will not identify the crisis center or consenting counselor, it is anticipated that counselors will feel “safe” and be willing to participate. CMHS anticipates an 80% response rate.

To increase participation of callers in follow-up interviews (i.e., MI/SP Caller Follow-up Interview), callers are being offered a $50 remuneration for their participation.

B4. Tests of Procedures

The MI/SP Caller Follow-up Interview, MI/SP Counselor Attitudes Questionnaire, MI/SP Counselor Follow-up Questionnaire and all associated consents were piloted during a previous evaluation conducted by Columbia University. At that point, the scripts and data collection tools were refined to make them as clear as possible. All monitoring components have been reviewed by experts in the field of mental health and piloted to determine burden levels.

B5. Statistical Consultants

The evaluator has full responsibility for the development of the overall statistical design and assumes oversight responsibility for data collection and analysis for the evaluation. Training and monitoring of data collection will be provided by the evaluator. The following individuals are primarily responsible for overseeing data collection and analysis:

Madelyn S. Gould, Ph.D., M.P.H.

Professor,

Psychiatry and Public Health (Epidemiology)

Columbia University/NYSPI

1051 Riverside Drive, Unit 72

New York, NY 10032

212-543-5329

Jimmie Lou Munfakh, B.A.

Psychiatry and Public Health (Epidemiology)

Columbia University/NYSPI

1051 Riverside Drive, Unit 72

New York, NY 10032

212-543-5482

Alison Lake, M.A.

Psychiatry and Public Health (Epidemiology)

Columbia University/NYSPI

1051 Riverside Drive, Unit 72

New York, NY 10032

212-543-6714

The SAMHSA project officer responsible for receiving and approving deliverables is:

James Wright, MS, LCPC

Suicide Prevention Branch

Center for Mental Health Services

SAMHSA

1 Choke Cherry Road

Rockville, MD 20857

240-276-1854

[email protected]

References

Allison PD. 2001. Missing Data. Thousand Oaks, CA: Sage.

Gould, M.S., Marrocco, F.A., Kleinman, M., Thomas, J.G., Mostkoff, K., Cote, J., & Davies, M. (2005). Evaluating iatrogenic risk of youth suicide screening programs: A randomized controlled trial. Journal of the American Medical Association, 293(13), 1635-1643.

Gould, M.S., Kalafat, J., Harris–Munfakh, J.L., and Kleinman, M. (2007.) An evaluation of crisis hotline outcomes part 2: Suicidal callers. Suicide and Life–Threatening Behavior, 37(3), 338–352.

Gould, M.S., Munfakh, J.L., Kleinman, M., & Lake, A.M. (2012). National Suicide Prevention Lifeline: Enhancing mental health care for suicidal individuals and other people in crisis. Suicide and Life–Threatening Behavior, 42(1), 22–35.

Hedeker D, Gibbons R, du Toit M, Cheng Y. 2009. SuperMix: for Mixed Effects Models. Skokie, IL: Scientific Software International.

Kalafat, J., Gould, M.S., Harris–Munfakh, J.L., and Kleinman, M. (2007.) An evaluation of crisis hotline outcomes part 1: Nonsuicidal crisis callers. Suicide and Life–Threatening Behavior, 37(3), 322–337.

Little R, Rubin D. 2002. Statistical Analysis with Missing Data. New York: Wiley. 2^nd ed.

List of Attachments

Attachment A	MI/SP Caller Initial Script
Attachment B	MI/SP Caller Follow-up Consent Script
Attachment C	MI/SP Caller Follow-up Interview
Attachment D	MI/SP Counselor Consent
Attachment E	MI/SP Counselor Attitudes Questionnaire
Attachment F	MI/SP Counselor Follow-up Questionnaire

Page 7