Attachment B - Summary of Recommendations

7

SUMMARY OF RECOMMENDATIONS FOR THE

REVISED CCP DATATOOL KIT

NOTE: Issues/Comments were gathered from the Assessment of CCP Evaluation Report (2007) and OMB 60-Day Period (2008) on the Revised Tools

General Issues

Issue 1(Training): Training on CCP Data Collection Tools, in general training (previous telephone training conducted) was perceived as dissatisfactory (i.e. reported from the Assessment of the CCP Evaluation). Staff turnover at State and/or local level was also problematic for continuity of briefing/training on data tools and entry into MS Access database. It was also noted that there was a lack of single point of contact (at federal level) regarding questions/information on data collection tools.

Response: As the CCP Team looks to realign its trainings, especially the Train-the-Trainer Toolkit, critical attention will be paid to the CCP Data Toolkit training module to ensure that guidance on the purpose and content of each tool, procedures for administration at the local level, and data entry into a database is clear and concise. Data and evaluation presentation at the standard Emergency Management Institute (EMI) will also be considered as an effective avenue to improve further understanding in the administration of the CCP data tools. Additionally, how data information can used by local providers to improve service provision will also be described and discussed. It is also suggested that CCP Project Officers as well as the assigned SAMHSA Disaster Technical Assistance Center (DTAC) liaison may be the point of contact as always federal funding availability for ongoing training outside of CCP Project Officer (PO) and DTAC is a consideration.

Issue 2 (Data Entry): Use of Microsoft Access (MS Access) database for data entry, the current MS Access database for data entry has some limitations including data field restriction on the amount of data that can be entered and reports not always seen as useful to improve service provision.

Response: In the future, CCP Federal staff will considered a web-based training protocol, web-based data entry platform that would also allow for real-time data upload/download, graphic reporting of data (frequency charts, bar/pie charts), and geographical reporting of data at the local/state level accessible by multi-persons (e.g., Cross-Site Evaluation Contractor/Consultant, CCP Project Officers, Federal Emergency Management Agency (FEMA), CCP State/Local Providers).

Issue 3 (Burden Estimate to Complete Tools – Time): The burden to complete the tools is underestimated and that time reported in the burden table should be drastically increased for each of the tools.

Response: The burden table will be reviewed to determine where any changes can be made to more appropriately reflect the actual use of the forms.

Issue 4 (Consistency in Demographic Section across all Tools): There was some inconsistency reported across the tools for the demographic category – Ethnicity.

Response: The demographic Ethnicity category on the Individual Encounter Forms, Adult and Assessment & Referral Tools has been modified so that this item is reported the same – “Ethnicity (select one)” as well as the subsequent instructions for completing this item. To be sure that this is completed, on the Provider and Participant Feedback Survey remove phrase in instruction for Are you Hispanic/Latino? “That is, are you or your ancestors from Spain, Mexico, Cuba, Puerto Rico, the Dominican Republic or center or South America” because like on the encounter forms, a person can self-identify themselves as such.

Issue 5 (Look/Format/Visual Presentation of Tools) Format of tools should be more visually appealing to facilitate increased compliance

Response: Current format of CCP data tools allows for documents to be “scanned.” Some States/Counties that have a Data/Evaluation Staff infrastructure also have the ability to scan these tools into an electronic data entry system. In the absence of a national “web-based data entry/data upload/download mechanism for all CCP data that can be accessed and monitored, still like to offer CCP grant sites to have the ability to “scan in” data at the State, county, or local provider level.

Issue 6 (Reporting of Hotline Information): Some confusion on where hotline data is to reporting on CCP Data Tools.

Response: Hotline data will be proposed to be collected on the revised Individual Crisis Counseling Services Encounter Log.

Issue 7 (CCP Data Toolkit Did not Include a Child Assessment and Referral Tool): Assessment of the Evaluation/CCP Cross-Site Evaluation/CCP POs indicated there was a lack of child/youth data collected in relation to CCP service provision

Response: A Child Assessment and Referral Tool was created and now part of the current OMB revised CCP Data Toolkit package in coordination with the National Center for Child Traumatic Stress (Pat, Melissa, Alan) and the National Center for Post Traumatic Stress Disorder (Fran).

Issue 8 (Administration of Adult/Child Assessment and Referral Tools): Some confusion on procedures for crisis counselors on how to best administer these tools.

Response: In guidance document developed for using the CCP Data Toolkit will include suggestions on how CCP grants may wish to set up procedures/protocols with their crisis counselors on administration of these tools. Given, the variance of expertise/training/experience of crisis counselors’ administration procedures may be different. For example, CCPs with less experienced or paraprofessionals may wish that when crisis counselors encounter a person that they feel may need more intensive follow-up would want the person referred to their team leader. At which time if the person so indicates, the crisis counselor would get in touch with the team leader, who then administers/completes the appropriate (adult or child) assessment and referral tool to the individual in need.

Specific Issues for Each of the Forms

Individual Encounter Log (Short/Long)

Issue (Short vs. Long Form): Previous encounter logs did not include items on event reactions.

Response: Based on previous comments on the encounter log, items on event reactions has been added.

Issue (Characteristics of Encounter): Field suggests including an item for group living arrangement. Bilingual services should be added as a type of encounter characteristic

Response: Group homes added in description for “home” on both short and long forms. Bilingual service was not added as a characteristic of encounter on the encounter forms. Rather the form includes an item for the primary language spoken during the service contact encounter.

Issue (Demographics): Ethnicity (instruction for Hispanic/Latino), Race identification section

Response: Instructions have been rewritten to include phrase –“Based on your observations and your conversations with the individual, does this person self-identify as Hispanic/Latino?”

Issue (Referral): Clarity of “other crisis counseling services,” Utility of the “other” option

Response: Referral section has been redesigned to the include the following options: crisis counseling program services, mental health services, substance abuse services, community services, and other.

Issue (Event Reactions on Long Form Version): Suggests including an option for coping or doing well within the event reactions section as reaction items in general seem negative.

Response: An item “coping well; not of the above apply” has been added to the event reactions section.

Group Encounter Form

Issue (Separate Forms for Group Counseling versus Public Education): CCP POs suggests that separate forms be created for a group that has a “counseling” focus versus a group that has a “public education/information sharing” focus.

Response: Given the number of forms one form for group encounters is considered where the person completing the form would indicate whether the group is a meeting/gathering to listen or elicit responses and reactions from the group members (e.g., counseling) or public education/presentation to a group (e.g., persons are gathered to hear about nature of typical disaster reactions, CCP services, locations, referrals, etc.).

Issue (Number of Forms to Completed): Suggests that instruction include that only one form per group is expected to be filled out/completed as previously some crisis counselors have completed the group form for each individual group attendee.

Response: “Complete only one form per group” has been added to the instruction for this tool.

Issue (How Often to Be Completed): – Field raises issue of confusion of whether or not this form needs to be completed every time the same group meets.

Response: Form should be completed every time the group meets, need to include in instruction for group encounter form that this log should be every time a group meeting occurs, even if the same group is meeting more than once.

Issue (Group Identity Classification): Suggested that previous group identity classification was confusing when worded as “role in the community” and left many instances where this section was not completed or many instances of other was reported. Crisis counselors in the field where not necessarily addressing groups that shared the same role, rather addressing groups that were made up of children or adults that might have multiple roles in the community (e.g., parent, youth volunteer, disaster worker, teacher, etc.).

Response: Reworded group identities section to indicate whether the group was composed of: children/adolescents, adult survivors, public safety workers, other recovery workers, mix of the above or none of the above. These response categories appeared to be the most frequent responses for group encounter make-up.

Weekly Tally Sheet/Brief Educational and Supportive Services Not Elsewhere Included

Issue (Type of Contact): Field is unclear as to whether “material left in public spaces” includes individual/single items or bundles.

Response: The materials left in public spaces could include a single item or bundle/multiple copies of an item. The crisis counselor has the ability to record either on the tally sheet and is further clarified in the instructions.

Issue (Separation of Community Networking from Coalition Building): Suggest having community networking and coalition building as a separate form. Additionally, it is proposed to have a new program management/monitoring tool (i.e., way to monitor CCP reach regarding level of networking, type of networking, increase in networking, challenges in networking, strategies in networking, etc.).

Response: This type of information will be considered to be captured through progress reporting by State CCPs that incorporates program management areas as opposed the type of data considered as part of the revised data toolkit.

Adult Assessment and Referral Tool

Issue (Format/Visual Consistency with Other Tools): Suggest that the revised Adult Assessment and Referral Tool be aligned to following similar topic sequence (provider information, characteristic of encounter/location, risk categories, demographic information, and referral groups)

Response: This tool has been restructured so that the topic order is similar to other data forms/tools along with respective topic items. The order is such – provider information, characteristics of encounter, risk categories, demographic information, assessment questions, and referral(s) that were communicated. Risk categories will be the same as those indicated on the Individual Encounter Log form. Since this tool has more than 1 page, the title of the tool is also provided at the top of the second page.

Issue (Visit for Which Form is to be Completed): Suggest that form may be completed (if necessary) at any time, not just at 3 or more visits.

Response: Instructions on this form has been modified in two ways. First, at the top of the this tool written guidance is provided that indicates this tool may be used with adults who have received individual crisis counseling on two (2) or more occasions before this visit or with any adult at any time if you suspect the adult may be experiencing serious reactions to the disaster. Second, similar instruction is provided in the supportive instruction at the end of the form. Additionally, in the characteristic of encounters section, the visit number has been revised so that a crisis counselor may indicate the visit number with a range of 1^st visit to 5^th visit or more.

Issue (Change Wording of Assessment Items and/or Include Additional Items): Suggest word changes to assessment items and/or including of additional items that include substance use, resiliency or effective coping strategies.

Response: The Adult assessment items are taken from a longstanding, field tested standardized instrument – the SPRINT-E that has significant research data support (e.g., reliability, validity, etc.). The SPRINT-E does make reference to alcohol or substance use in Item #7. Thus, any word changes to these items would not allow CCP data to be compared other reported disaster behavioral health data using this measure. The intent of the SRINT-E items is to generate an immediate risk score (no risk to high risk), thus if resiliency and effective coping items were included it this assessment it would take away from determining range of risk, thus level of referral urgency. However, if after completion of the assessment items the individual has a total score from 1 to 3, then further instruction is provided to the crisis counselor/team leader to elicit a response from the individual to indicate that they are indeed managing their reactions.

Issue (Acceptance of Referral Information): Suggests further instruction on form to indicate whether or not referral source provided was accepted by individual.

Response: Given the voluntary nature of whether or not a person follows up on referral information, at the end of the SPRINT-E assessment question items, instruction is provided for the crisis counselor/team leader to discuss with the individual what their (individual) next steps or goals would be.

Child Assessment and Referral Tool

Issue (Similar Formatting/Visual as Other Tools and Assessment Response Structure as the Adult Tool): Ensure as much as possible similar format/visual as other tools as well as similar response structure as adult assessment and referral tool.

Response: Formatting changes on the child tool have been to address similarity with other data tools. Instruction for visit number and assessment response category matches that of Adult Assessment and Referral Tool. Whether referral was accepted or not is also indicated.

Issue (Week versus Month on Assessment Items): Inconsistency between item in their reference to week or month.

Response: Items and instruction has been modified for consistency so that “month” is now indicated.

Issue (Response Card): – Indication that the response card instruction was confusing and should be presented on a separate page then with form question items.

Response: Response card instruction has been moved to a separate page and incorporated in the specific tool instructions to crisis counselors.

Service Provider Feedback Survey

Issue (CCP Required Trainings Not Indicated): Required CCP Trainings were not identified in the tool as well as utility of training in performing crisis counseling work responsibilities.

Response: CCP required trainings are now indicated in the form to determine level of participation and rating of training if received as well as specific training on CCP evaluation tools and other trainings offered by the State or provider agency. Items have also been added to determine the utility of trainings in performing crisis counseling work responsibilities.

Issue (Response Choice Option): Question as to the practicality of having a 10 item response choice versus more traditional 4 or 5 item response choice.

Response: Based on previous CCP collected data, the 10 item response category has allowed for more variance in the data responses, whereas the typical 4 or 5 item response option tended to have almost all of responses rated very good to great.

Participant Feedback Survey

Issue (Literacy Level): CCP POs expressed concern with literacy level, also take into account Spanish translation literacy level.

Response: As with all of the tools, literacy will be reviewed and determined. However based on findings from CCP Cross-Site Evaluation the participant response rate on this survey has been as expected.