Crisis Counseling Assistance and Training Program Data Toolkit
SUPPORTING STATEMENT
B. STATISTICAL METHODS
1. Sampling Methods
Ruggiero, K. J., Davidson, T. M., McCauley, J., Gros, K. S., Welsh, K., Price, M., Amstadter, A. B. (2015). Bounce Back Now! Protocol of a population-based randomized controlled trial to examine the efficacy of a web-based intervention with disaster-affected families. Contemporary Clinical Trials, 40,138–149.
Efforts used to inform this section of the OMB Supporting Statement include the following: (1) a retrospective evaluation of the CCP, 2) a cross-site evaluation of CCP grants funded as a result of the 2005 Gulf Coast hurricanes, and (3) analysis of the data from 2009 to 2011. The first two studies were performed by the National Center for PTSD as an interagency agreement with SAMHSA CMHS.
The retrospective evaluation involved an archival analysis of all available reports for crisis counseling grants implemented in the 50 United States that closed out over a 5-year interval. As such, all crisis counseling projects that closed out between October 1, 1996, and September 30, 2001, and were administered by states rather than territories, were eligible for analysis. Reports from 44 grants covering 28 disaster events were included in the quantitative analysis for the retrospective evaluation.
Given the catastrophic nature of the 2005 Gulf Coast hurricanes, a disaster-specific cross-site evaluation of funded CCPs was supported by SAMHSA CMHS. The cross-site evaluation of the 2005 Gulf Coast hurricane programs consisted of 22 grants in 17 states active between November 2005 and February 2007, thus 3 to 18 months after Hurricane Katrina. These grants utilized the 2005 OMB-approved CCP data collection forms.
The means (or averages) from these studies, combined with a review of the data trends from the past 2 years and consideration of the current CCP grants, were used to inform this section of the OMB Supporting Statement. The estimated number of respondents is 53,000 per year for the CCPs, distributed as follows:
Individual/Family Crisis Counseling Services Encounter Log = 39,200
Service providers (i.e., crisis counselors/program staff) will be required to complete this form for all service recipients who access individual or family crisis counseling services of 15 minutes or more.
Group Encounter Log = 3,300
Service providers will be required to complete this form for each group of service recipients who access group crisis counseling services and/or group public education services.
Weekly Tally Sheet = 6,600
Service providers will be required to complete this form for all contacts for whom data are not captured on either the Individual/Family Crisis Counseling Services or Group Encounter Logs.
Assessment and Referral Tools = 2,800
These tools (Adult or Child/Youth) are intended for intense users of services, defined as all individuals receiving a third or fifth individual crisis counseling visit, or those experiencing serious reactions to the disaster. This tool will be used beginning 3 months after the disaster.
Participant Feedback Form = 1,000
These forms will be collected from a sample of service recipients, not every recipient. The use of a time sampling approach is recommended, e.g., soliciting participation from all counseling encounters 1 week per quarter.
Service Provider Feedback Form = 100
These forms will be administered to all service providers, and data will be collected anonymously at least once at 6 months and/or 1 year after the disaster.
Most CCPs will collect their client information using a paper-and-pencil method on forms that can be forwarded to a central location for data entry into the online system and analysis, although the Service Provider Feedback Form may also be administered online. The completion of forms is a part of the daily work requirements for service providers and does not interfere with ongoing program operations. SAMHSA DTAC staff members are also currently developing a mobile app to better aid CCP collection of timely, quality data. As mentioned previously, the goal is to develop the following three forms by September 2015: (1) Individual/Family Crisis Counseling Services Encounter Log, (2) Group Encounter Log, and (3) Weekly Tally Sheet. Data entered via the app will be seamlessly incorporated into the grants dataset, but with the added benefit of not having to be keyed in from paper copies and with strict validation of form entries.
2. Information Collection Procedures
CCP crisis counselors and outreach workers will complete the Individual/Family Crisis Counseling Services Encounter Log and the Group Encounter Log after the service recipient(s) has/have left the encounter location. These forms will not be used as a checklist and will not be completed while the service recipient(s) is/are present. Service providers will select variables based on their own perceptions of the service recipient or on information that recipients spontaneously reveal during the encounter.
The Weekly Tally Sheet will also be completed by the CCP crisis counselor or outreach worker at the end of the designated “week” period (e.g., Sunday through Saturday as determined by the CCP and the SAMHSA CMHS project officer).
The Assessment and Referral Tools will be used as a checklist with the service recipient present. These tools will be read aloud to the recipient by the crisis counselor, and the recipient will respond to questions accordingly.
Both the Service Provider and Participant Feedback Forms will be completed in private and anonymously. The Service Provider Feedback Form will be administered online, and the Participant Feedback Form will be returned by mail. Both methods assure the anonymity of the respondent. The SAMHSA CMHS project officer along with FEMA staff will provide guidance to the state in determining the most appropriate method for the collection, processing, and sharing of findings from these forms.
3. Methods to Maximize Response Rates
Response rates for the encounter logs (i.e., Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, and Weekly Tally Sheet) and the Assessment and Referral Tools are intended to be 100 percent. However, it is acknowledged that a 100 percent completion rate is not always possible due to various reasons (e.g., the counselor may fail to complete a form, a form may not be available, or a form may be lost in the administrative process). Therefore, we estimate there may be a loss rate of up to 5 percent yielding a completion rate of 95 percent for each of these forms.
In August 2015, SAMHSA anticipates launching a pilot program to collect data through use of a mobile application. The mobile application will permit entry of data for the Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, and Weekly Tally Sheets through an application on a mobile device. The mobile version is intuitive, easy to navigate, and streamlines the order of questions, permitting outreach workers to complete the forms faster and more accurately. The mobile devices will upload the data into the online system whenever the mobile device is within cellular or wireless access range. Use of the mobile application is expected to decrease the time necessary to enter information, decrease the number of people who have to touch the data, and decrease the lag in aggregating data.
Using baseline information on the number of forms completed, the completeness of forms, the average lag between encounter and data submission, the mobile application is expected have a positive impact on data quality and cost.
The Individual/Family Crisis Counseling Services Encounter Log will be completed by the crisis counselor for all individuals who access this service. The Group Encounter Log will be completed by the crisis counselor/outreach worker for all groups that meet for crisis counseling or for public education. The Weekly Tally Sheet will be completed by the crisis counselor for all brief educational or supportive encounters not captured by any other form.
The Assessment and Referral Tools are intended to be completed by a trained crisis counselor for 100 percent of all service recipients who access individual crisis counseling for the third or fifth time or express a strong need for intensive services. However, rates may be as low as 95 percent. Analysis of previously collected data indicated that that number of individuals to whom this will apply will be approximately 5 percent of all service users.
The Participant Feedback Form will be completed by service recipients. The Participant Feedback Form is given to a sample of people for whom individual or group crisis counseling services were provided (i.e., not for persons denoted only on a Weekly Tally Sheet or who participated in group public education activities). It is given only to adults. The form is administered at 6 and 12 months after the event. At 6 and 12 months after the event, the program chooses a 1- or 2-week period during which all appropriate persons are asked to complete an anonymous feedback form. The number of respondents is compared to the total number of eligible adults served in individual crisis counseling or group crisis counseling (not including public education groups) during that administration period to estimate the response rate.
Service recipients will be provided with stamped return envelopes to increase their anonymity. The form is also accompanied by letters from the program director encouraging participation. The form is brief enough to also facilitate completion.
A review of previous data collected in the ODCES indicates that the typical response rate for the Participant Feedback Form is approximately 10 percent. We anticipate the same response rate for the current collection request. However, we may have a slight increase in reporting from previous years, as we are requiring CCPs to administer this form if a longer ISP grant is requested and at least once during the RSP grant period.
Given this low response rate, it is likely that people who complete the Participant Feedback Form differ in program-important ways from people who do not. When findings are reported, the response rate and the likelihood of bias in the findings will be communicated clearly, and aggregate level characteristics of responders as they relate to the program will be provided.
The Service Provider Feedback Form will be made available via an online link or paper distribution to all CCP service providers (i.e., crisis counselors/outreach workers) at 6 months and/or 1 year after the disaster. This will not be included in the app, but rather, questions related to the app will be incorporated in the online survey described here. Service providers will be encouraged to complete these forms.
4. Tests of Procedures
Many instruments in the CCP Data Toolkit have been taken from established data collection tools that have already been tested for validity and reliability. SAMHSA CMHS staff members have had an opportunity to review the revised tools and all are in agreement with the data items. The Assessment and Referral Tools, the Participant Feedback Form, and the Service Provider Feedback Form contain elements of the Short PTSD Rating Interview, or SPRINT, and the SPRINT-E, an expanded version of this form, both of which have been determined in research to be reliable and internally consistent. The Child/Youth Assessment and Referral Tool has items from the UCLA Post-traumatic Stress Disorder Reaction Index. Other items on these three forms, as well as on the Individual/Family Crisis Counseling Services Encounter Log and Group Encounter Log forms, evolved directly from previous studies (e.g., retrospective and cross-site evaluation of 2005 Gulf Coast hurricanes), and through site visits, interviews, and focus groups with states, direct service providers, and federal staff. Demographics collected across all forms are considered standard items for collection in the research literature and speak directly to the goals of the CCP.
5. Statistical Consultants
The names and phone numbers of project officers and the consultant are as follows:
Federal Project Officers
Nikki D. Bellamy, Ph.D.
Center for Mental Health Services
Division of Prevention, Traumatic Stress, and Special Programs
Emergency Mental Health and Traumatic Stress Services Branch
1 Choke Cherry Road, Room 6-1003
Rockville, MD 20875
Phone: 240-276-2418
Fax: 240-276-1890
Erik Hierholzer
CAPT, United States Public Health Service
Substance Abuse and Mental Health Services Administration
Lead Public Health Advisor
Emergency Mental Health and Traumatic Stress Services Branch
Center for Mental Health Services
1 Choke Cherry Road, Room 6-1109
Rockville, MD 20857
240-276-0408 (Office)
240-499-5859 (Cell)
240-276-1890 (Fax)
Statistical Consultant
Everly Macario, Sc.D., M.S., Ed.M.
Senior Evaluation Research Director | [email protected]
SAMHSA Disaster Technical Assistance Center
C/O IQ Solutions, Inc. | 224.244.3965 V | 800-308-3515 Toll Free
11300 Rockville Pike, Suite 901, Rockville, MD 20852
http://www.samhsa.gov/dtac | [email protected]ov
ATTACHMENTS
Individual/Family Crisis Counseling Services Encounter Log
Group Encounter Log
Weekly Tally Sheet
Assessment and Referral Tool (Adult)
Assessment and Referral Tool (Child/Youth)
Participant Feedback Form (REVISED FROM 2015 EXPIRATION FORM)
Service Provider Feedback Form (REVISED FROM 2015 EXPIRATION FORM)
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |