Site Visits Leadership and Line/other Frontline staff
Evaluation of the National Training on Trauma-Informed care (TIC)
0990-OWH TIC TTA Evaluation_Site Visit Protocol
Site Visits Leadership and Line/other Frontline staff
OMB: 0990-0440
S
ite
Visit Protocol
Cross-site Evaluation of the National Training Initiative on Trauma-Informed Care (TIC) for Community-Based Providers from Diverse Service Systems
HHSP23320095624WC
Final
May 16, 2014
Prepared
for:
Aleisha Langhorne
OWH/HHS
200
Independence Ave S.W.
Room 728E
Washington, DC 20201
Submitted by:
Abt
Associates
55 Wheeler St.
Cambridge, MA 02138
In Partnership
with:
Rutgers’ School of Criminal Justice
1. Site and Participant Selection 3
1.1 Site Selection: On-site and Virtual Site Visits 3
1.2 Site Selection: Remaining Teams 4
1.3 Participant Selection: On-site and Virtual Site Visits 4
2. Preparation for Site Visits and Calls with Remaining Teams 6
2.2 Scheduling Calls with Selected Teams Drawn from Remaining Sites 6
2.3 Pre-Visit Planning and Preparation (Including Preliminary Contact) 6
2.4 Pre-visit Documentation Request 7
3.1 Agency/Organization Site Visit (3-4 hours) 8
3.1.1 Agency/Organization group meeting (30 minutes) 8
3.1.2 Interviews (2-3 hours) 8
3.1.4 Exit and Follow-up (Optional) 9
3.2 Development of Site Visit Report 9
6. Appendix C: Overview of OWH TIC Evaluation for Email Attachment 12
7. Appendix D: Pre-site Visit Conference Call Agenda Template 13
8. Appendix E: Organizational Assessment Template 14
9. Appendix F: Informed Consent Form 15
10. Appendix G: Interview Protocols 17
10.1 Protocol for Site Leadership, Line Staff, and Other Frontline Staff 17
11. Appendix H: Environment and Accessibility Checklist 20
Overview
Under contract with the Office on Women’s Health (OWH), Abt has designed and is currently conducting a cross-site evaluation of the National Training Initiative on Trauma-Informed Care for Community-Based Providers from Diverse Service Systems. The evaluation includes both quantitative and qualitative data collection and analysis. The evaluation is designed to provide OWH with not only a picture of who was involved in the training and technical assistance (TA), including individual participant characteristics, dynamics of the organizations they represent, and the composition of teams in which they participated for the training, but also of any changes that resulted at the individual participant and organizational level.
The evaluation study is being conducted in two distinct phases: a formative phase, during which we are consulting with curriculum developers, training site staff, and other experts in trauma-informed care (TIC) as key informants to apprise the Abt team of considerations, materials and data to refine the overall study design; and an evaluative phase, during which we will perform data collection and analysis to assess implementation (process evaluation) and outcomes across levels of impact (i.e., knowledge, attitudes, practice, organizational change).
During the evaluative phase, the Abt team will conduct two major data collection activities: survey-based training assessments; and site visits. Site visits will include: on-site interviews and focus groups with agency administrators and staff; agency “walkabouts"1 and completion of the environmental and accessibility checklist; collection and review of administrative records; and review of action plans.
Site visits will enable us to examine the action plans developed during the training and TA, and assess any follow-up steps or changes in the organization that were both planned and successfully executed around the training. Site visits will also provide us the opportunity to gather ground-level information about the training, selection, and recruitment of participants.
Overall, the Abt team will evaluate through the site visits and related data collection:
The fidelity of the training to the curriculum developed;
The level of knowledge gained by participants;
The actions participants have taken as a result of the training and TA;
The actions each community organization’s staff have taken as a result of the training and TA; and
The changes in programs, agency/organizational policy and/or practice as a result of the training and TA.
As stated in our proposal, Abt plans to visit a number of agencies/organizations that participated in the National Training Initiative on Trauma-Informed Care for Community-Based Providers from Diverse Service Systems initiative within the 10 training sites. One site visit to a Phase I pilot site will be conducted during the formative phase of the project. Site visits within the remaining nine training sites will take place during the evaluative phase of the project. Abt will conduct site visits of agencies/organizations in five of the regions that received TIC training and follow-up TA in FY 2013 (Phase II) and site visits of agencies/organizations in four of the regions scheduled for training and follow-up TA in FY 2014 (Phase III). We anticipate visiting four agencies/organizations per region (see Exhibit A).
Selection of agencies/organizations within a region will be based on the consideration of a number of factors. Selection criteria will include:
Type of agency/organization
Abt will work with OWH project staff to select specific sites to participate in the evaluation to ensure that they reflect a diversity of regions, settings, and other site characteristics identified by OWH and key informants as potentially relevant to the training and its objectives.
Abt will select a range of sites, based on criteria such as agency/organization size, mission, and types of services provided.
Abt plans to limit site visit duplication regarding types of agencies/organizations within a particular HHS region.
Capacity and willingness of an agency/organization to participate in site visits
As some of the participating agencies/organizations may have limited staff, they may be unable to accommodate the time necessary for our site visits.
Geographic diversity
Abt will consider geographic diversity when selecting sites with representation of both rural and urban areas.
Travel implications
Many HHS regions cover a large geographic area. Given the limited time available for each site visit, it is more prudent to pick agencies/organizations that are a reasonable distance from each other, rather than travelling long distances between each, assuming we can identify representative organizations within these bounds.
Abt will include agencies/organizations from outside of the planned travel radius. Abt has capabilities to conduct interviews in a virtual or telephonic manner to accommodate participants from HHS regions that are difficult to access physically. For example, in the case of the Region 9, if we include agencies/organizations from Guam or Hawaii, we will conduct a telephonic/virtual visit, as our physical base for the Region 9 site visit will be California. In the case of Region 6, if we include agencies/organizations from Arkansas or Louisiana, we will conduct a telephonic/virtual visit, as the physical base of our Region 6 site visit will be Texas.
While the site visit protocol will vary slightly for such agencies/organizations (i.e. no tour of physical space, but review of Environmental and Accessibility Checklist), interviews and review of documentation can be successfully conducted via telephone or teleconference. Abt has had success conducting interviews in this manner in other evaluation projects.
Informed consent for participants for virtual site visits will be sent ahead of discussions, signed, scanned, and sent back prior to interviews.
EXIBIT A: Site Selection Overview
In addition to the on-site and virtual site visits of the selected agencies/organizations, Abt wishes to retain the option to conduct telephonic meetings with representatives of trainee organizations that are not selected for physical and virtual site visits. The purpose of these selected calls will be to fill gaps in information needed to better understand Phase I pilot testing or Phase II and Phase III training and technical assistance. Using the interview protocols developed for the on-site and virtual site visits, we propose to conduct approximately four conference calls with selected agency/organization teams drawn from those agencies/organization not selected for visits in the formative or evaluation phases (see Appendix H). These calls will differ from, and be considerably less time consuming than, both the on-site and virtual site visits in that there will be no travel time, no tour, no one-on-one interviews, etc. As such, these calls can be completed by project staff in tandem with the scheduling of on-site and virtual site visits.
Once Abt has selected the sites within each Phase II and Phase III training site, we will select four representatives from within each agency/organization to interview during the site visits. Given the small number of organizations (five to six) that participated in each region’s training and technical assistance, and the numbers of participants from each site (also five to six), it is not feasible to randomize the selection of participants. The Abt team will engage the agency/organization points of contact, to be identified prior to initial contact (see Section 2.3 below), to select interview participants from among those who participated in the training (contact list provided by OWH and Westat) with representation of various roles in the agency/organization, and with the addition of interviewees who may offer a unique perspective of changes in the organization.
Specifically, we will ask the points of contact to identify participants from two general categories within the agencies/organizations:
Leadership
Leadership includes staff in roles such as Director of Agency, CEO, and/or Executive Director.
Leadership representatives may include staff from a peer organization that participated in the training.
Leadership representatives may also include individuals who are referenced by OWH as “peers” and have lived experience of trauma.
Program and Line Staff (to include clinicians, program director/manager, direct program/service staff, and other frontline staff); and
Line Staff should include those people who have an administrative role in the agency, as well as those who have front line program or service delivery responsibilities.
Line staff representatives may include individuals who are referenced by OWH as “peers” and have lived experience of trauma.
While we would, ideally, like to speak with agency/organization representatives within the leadership and line staff roles who were also training participants, we are cognizant that a number of circumstances that may limit our ability to do so, including:
Staff turnover
Given the length of time between the trainings and the site visits, there is a strong likelihood that there will be staff turnover at a number of the agencies/organizations.
Staff unavailable/unwilling
Training participants from the selected agencies/organizations may be unavailable or unwilling to be interviewed.
Homogeneity of training participant roles
While agencies/organizations were asked to select training participants from a variety of roles/responsibilities some may have been unable to do so.
Given these circumstances, we expect to interview some agency/organization representatives from the roles defined above who may not have attended the training. As agency/organization representatives, the perspectives of staff in these roles are valuable in gauging changes and impacts that have occurred as a result of the training.
In summary, we will request that the points of contact first attempt to select interviewees from leadership and line staff who were training participants. If such representatives are unavailable for the above reasons, we will request that the POCs identify other representatives from within those roles. We emphasize that while not all staff who participated in the OWH training and TA events may be interviewed during the site visits, all training participants will be provided the online survey portion of the evaluation.
EXIBIT B: Participant Selection Overview
Given the time requirement for sites and travel considerations for Abt project staff, we anticipate contacting each evaluation site approximately two months prior to the site visit, by Region. We anticipate that site visits will occur between October 2014 and March 2015, with each visit lasting approximately three days. Therefore, initial conversations with the first regions to be visited will begin in early August 2014. It is our plan to have the date and time of each site visit confirmed at least one month prior to the proposed site visit. As mentioned earlier, some interviews may be conducted in a virtual or telephonic manner given the diverse geographic locations of participants within some HHS regions. As detailed in the following section, of the three days planned for each site visit, we reserve one travel day and two days for on-site meetings and review. Each site visit is divided into four half-day segments with the selected agencies/organizations being drawn from those that participated in the OWH TIC training and technical assistance.
Similar to the site visits, we anticipate contacting selected teams drawn from the remaining agency/organization teams approximately two months prior to scheduling the conference calls. The conference calls will occur in tandem with the site visits, generally between October 2014 and March 2015, with some flexibility if necessary. We anticipate that these calls will take approximately one hour each.
Abt will collaborate with OWH to identify a point of contact (POC) from each agency/organization to be visited. Once the POCs have been identified, OWH will send a preliminary email to the POCs at all selected agencies/organizations (to be drafted by Abt) introducing the project, the evaluators’ role, and establishing OWH’s goals and objectives for the evaluation (see Appendix A).
Initial contact from Abt, via email, will follow the OWH email and will focus on introductions and scheduling a collaborative preliminary conference call with the POCs and leadership representatives (if different from the POC) from all of the selected agencies/organization within the region (see Appendix A). Agency/organization representatives may elect to have additional staff present on the conference call. The initial email will include a one-page overview of the evaluation project for leadership to share with their staff (see Appendix C), a site visit process map, and site visit structure checklist to prepare for the site visit (see Appendix I). In advance of the conference calls, the team will develop a structured agenda, with a set of standardized questions and topic areas to be touched on during the discussions (see Appendices D, E, and F). Examples of topics to be discussed during the conference call include: general timing of and availability for the site visit; method by which one-on-one interviews will take place (in person vs. telephonically); location of interviews; suggested inclusion of individuals who OWH references as “peers” – persons with lived experience of trauma - in interviews while on-site; and the optional exit meeting.
Based on the results of the preliminary conference call, follow-up conversations (including logistical confirmation) will be held via email and/or telephone and will focus on establishing the date and time for the site visit and confirming the list of interviewees.
Similar to the agencies/organizations selected for on-site and virtual site visits, Abt will identify a point of contact at selected organizations drawn from the remaining teams to send the initial contact email and corresponding information.
During initial conversations with the each of the selected sites, Abt will request a copy of their action plans (per the training curriculum), an organization chart, and documentation of any and all policies, procedures, and practices that have changed as a result of participation in the training. Any documentation that sites are unable to provide in advance of the visit will be requested while on-site.
Site visits will take place over three days (including up to one travel day and a minimum of two on-site days) and will include two project staff (one senior-level staff and one junior-level staff). As a rule, the site visits will be divided into four half-day segments with selected agencies/organizations.
Each agency/organization site visit will begin with an initial group meeting which will include all representatives to be interviewed who are available and any additional staff that leadership deems appropriate (interviewees are encouraged, but not required, to attend this meeting). During this initial meeting we will:
Lead introductions;
Provide a brief overview of the evaluation project goals and objectives;
Review logistics for interviews;
Interviewee availability/timing
Space for conducting one-on-one interviews
Gather any documentation or information that was previously requested but not acquired; and
Discuss any changes made to the policies, procedures, practices and/or physical space of the organization as a result of the training; and
Discuss exit meeting logistics, if one has been requested.
One-on-one interviews with previously identified leadership and line staff will begin following the tour (see Appendix G). Interviews will be conducted in a private space determined prior to the site visit and confirmed during the group meeting. We estimate conducting approximately four interviews per site, allowing 30–45 minutes per interview. In the event that interviews finish early, and there are additional staff who are willing and available to speak with us, we will accommodate additional interviews.
Before initiating any of the interviews, the interviewer will provide interviewees with a copy of the informed consent form to review (see Appendix F). If the interviewee agrees to participate, the interviewers will continue with the interview protocol (see Appendix G). If the interviewee declines to participate, Abt will ask the site visit POC if they wish to make a substitution.
Following the interviews, we will request a tour of the physical space of the agency/organization. The tour is intended to help assess any changes in the physical space of the agency/organization that were made based on the environmental items covered in the training curriculum based upon the knowledge gained during the training and outlined in their Action Plan. The purpose of the tour is to help in evaluating the impact of the training, not to evaluate the success of the agency/organization itself. We will use the environmental and accessibility checklist (see Appendix H) to capture such changes. The Environmental and Accessibility Checklist is an adaptation of Handout 2.2 that the training participants received as part of the training curriculum. For virtual site visits, we will review items on the Environmental and Accessibility Checklist with participants, but will depend on self-reporting as we will not be present for direct observation.
We will offer agencies/organizations the option of an exit meeting during the introductory conference call with agency/organization POCs and leadership. If the agency/organization requests an exit meeting, we will be available for a brief overview of interviewer impressions and general feedback. These meetings should last no longer than 15–20 minutes. If the agency/organization declines the exit meeting, we will provide notice upon leaving to the administrative offices of the agency/organization that we are departing and that we will provide them with a copy of any reports that OWH disseminates as a result of the evaluation. Within 10 business days of return from the site visit, we will send all participating agency/organizations a letter of thanks addressed to the leadership of each organization.
Interviewers will take notes during interviews and will later summarize these notes into succinct site visit summaries employing a unified format for use in qualitative analysis. Abt will also provide OWH a general overview of activities conducted during site visits in the quarterly progress reports. The information collected during and surrounding site visits will be used to identify and assess common barriers as well as facilitators to implementation of training content across the ten Regions. Through the site-visit interviews, we will also examine whether, in each site and across sites, training alone or training with technical assistance increased the capacity of sites to provide trauma informed services.
After completing the formative site visit, the project team will develop a preliminary NVivo "codebook" that defines overarching themes and curriculum components we plan to discuss during site visits and team conference calls.
Implementation data will be summarized by site and by groupings in narrative and, where appropriate, in simple descriptive tables. For example, we can examine across sites recruitment and completion rates for participants, participant satisfaction, etc. Specific comments may be coded in more than one theme and comments will be coded at the most specific theme. As visits are completed and notes are coded, two team members will review the content of all the themes to check for inconsistencies, redundancies and imprecision. All qualitative data will be coded using this theme structure.
For the purpose of the outcome evaluation, information from this phase may also be used to cluster training site characteristics and experiences to create contrasts for the purpose of understanding of what works for whom under what circumstances in regard to attitudes, knowledge and behavior at the individual level and organizational/environmental change at the organizational/community level.
[Insert OWH introduction]
[Insert Agency/Organization Name]
[Insert Date and Site of Training (that program attended)]
[Insert CBO staff team names]
The Office on Women’s Health (OWH) of the US Department of Health and Human Services (HHS) has been engaged in providing training and technical assistance on trauma informed care to a range of service organizations.
Under contract with OWH, Abt Associates has designed and is currently conducting a cross-site evaluation of the National Training Initiative on Trauma-Informed Care for Community-Based Providers from Diverse Service Systems. The evaluation will provide OWH not only a picture of who was involved in the training and TA, including individual participant characteristics, dynamics of the organizations they represent, and the composition of teams in which they participated for the training, but more specifically, any changes that resulted at the individual participant and organizational level Attached to this email please find a 1-page overview of the cross-site evaluation.
As part of the evaluation, the Abt team will be conducting site visits of agencies/organizations that participated in the training. Site visits will include interviews (either on-site or via telephone) and focus groups with agency administrators and staff, a tour of the agency, the completion of observational checklists, and the collection and review of agency/organization documentation that relate specifically to organizational changes resulting from participation in the training, and Action Plans. Site visits will provide Abt the opportunity to gather ground-level information about the training, selection, and recruitment of participants, and any follow-up steps or changes in the organization that were both planned and successfully executed around the training.
As a representative from an agency/organization that participated in the training, your input is critical to our work. Regardless of whether you participated in the TIC training, you and other staff at your organization possess critical knowledge about and perspectives about the program’s trauma informed care practices. If you did attend the training, we are particularly interested in your feedback regarding any changes or impacts that you have noticed or experienced as a result of the TIC training and technical assistance effort.
Participation in the site visits is voluntary, but we hope that you will be open to speaking with the Abt project staff.
[Insert OWH conclusion, thanks, and signature]
Appendix B: Introductory Email from Abt to POC and Leadership Representatives at Participating Agency/Organization
[INSERT DATE]
Dear [INSERT],
I am writing to you in my capacity as the co-principal investigator of an evaluation study being conducted on behalf of the HHS Office on Women’s Health. By now you should have received an email from the HHS Office on Women’s Health introducing Abt Associates and our evaluation of the impact and utility of the National Training Initiative on Trauma-Informed Care for Community- Based Providers from Diverse Service Systems curriculum and training. We have received participant lists from the training sessions, and have determined that you or other staff at your agency/organization participated in the TIC training.
Regardless of whether you participated in the TIC training, we are very interested in speaking with you and/or other staff regarding trauma informed care practices at your agency/organization. In order to gain a better understanding of these practices, and, potentially any impacts the training may have had on your program, we will be scheduling a site visit (to be conducted on-site or via virtual or telephonic method) in the next couple of months. In the meantime, we would like to schedule a time to speak with you or a representative from your program regarding the project, our proposal for the site visits, and recommendations for interviewees.
If you would be kind enough to complete the Doodle scheduling poll below, we can schedule a preliminary call at a time that is convenient for you. If you would like others on your team to join the call, please forward this email to them.
http://www.doodle.com/XXXXXXX
I would also like to introduce my fellow co-principal investigator, Dr. Bonita Veysey, and the evaluation project manager, Ms. Mica Astion. I am copying them on this communication, as you may hear from or be in contact with them as this effort proceeds.
Thank you very much for your consideration in working with us on the evaluation of this important initiative. We look forward to speaking with you soon.
Warm regards,
Danna Mauch, PhD | Principal Associate | Senior Fellow | Abt Associates
55 Wheeler Street | Cambridge MA 02138
O:
617.349.2354 | C: 617.680.8200 | F: 617.386.8528
CC: Bonita Veysey, PhD; Mica Astion, MA
T
he
Office on Women’s Health (OWH) in the U.S. Department of Health
and Human Services (HHS) recognizes that women and girls report a
disproportionate level of exposure to trauma and violence throughout
their lives. Experiences of trauma have an impact on the physical
health, mental health, and general wellbeing of women throughout the
life span. For girls and women impacted by trauma, using health and
human services in provider organizations that are uninformed about
trauma poses at a minimum risks that needs will not be adequately
met. There are also significant risks of re-traumatization, if
uninformed providers employ coercive or insensitive practices as are
used in some settings across diverse service systems. By contrast, an
informed provider engaged in the provision of trauma-informed care
promotes recovery and fosters resiliency in the girls and women who
turn to the organization or program for service. The long unmet needs
of these survivors in our communities and the growing shift towards
home and community-based services make it imperative to implement
effective strategies to promote trauma-informed care (TIC) in these
settings.
Among the public health initiatives designed to meet its goals, OWH has overseen the development of a TIC training curriculum, and tested it across regions of the United States and Territories with a plan to add sites and offer technical assistance support to a range of health and human services organizations in all 10 HHS regions. Abt Associates has been recruited to evaluate the initiative by examining the training and technical assistance planning, development and administration processes, including participant satisfaction, as well as outcomes for individual participants and their organizations.
The Abt team will assess aspects of the knowledge acquisition levels among individual participants (through an on line survey), and behavior change resulting from what was learned, and measurable results not only on participant behavior, but also on organizational plans and operations (through site visits). Data from at least 10 training sites (including pilot sites, of trainings completed, and those that will take place) be analyzed in order to produce a comprehensive evaluation report to OWH. The report will focus on the effectiveness of the training curriculum to convey information on the impact of trauma on the health and wellbeing of women and to build the capacity of community organizations to incorporate a trauma-informed approach into their programs and services. In addition, if needed, recommendations for improvement will be provided.
For Additional Information, Contact: Mica Astion, Project Manager, Abt Associates
OUTLINE FOR PRE-VISIT CONFERENCE CALL WITH AGENCY/ORGANIZAITON POCS AND LEADERSHIP REPRESENTATIVES
AGENDA
Date:
Conference Call #:
Attendees:
Introduction
Overview of Evaluation
Site Visit
Timing/Availability
Method of Interviews (in-person vs. electronically)
Location for on-site interviews
Suggested Site Leadership, Line Staff, and Other Frontline Staff for Interview
Optional Exit Meeting
Request of Documentation
Action Plans (if not already received from Westat)
Organization Chart
Sample Agency/Organization Documentation (Policy/Procedures/Protocols that have changed as a Result of the Training and/or Technical Assistance. Please note that this request refers to pertinent records that relate to organizational changes, not general administrative records.
Request of Additional Information
Organizational Assessment
Next Steps
Other
Below is the template Abt has developed for data to be gathered on each participating organization. Some of the data will be generated through team research of participating organization websites and project materials, while other information may be requested directly from these organizations. We will make every effort to minimize the burden on the participating organizations.
Cross-site Evaluation of the National Training Initiative on Trauma-Informed Care (TIC) for Community-Based Providers from Diverse Service Systems
CONSENT FORM
Introduction and purpose
The Office on Women’s Health (OWH) has contracted with Abt Associates Inc. to conduct an evaluation to assess the impact of its National Training Initiative on Trauma-Informed Care (TIC) for Community-Based Providers from Diverse Service Systems. To this end, Abt is visiting programs, agencies, and organizations and meeting with individuals from those entities who participated in the training and technical assistance sessions. Our site visits are designed to capture both the knowledge gained by participants and the implementation impact achieved in their organizations as a result of the OWH TIC training and technical assistance. Abt’s analyses and findings will be used to further refine the TIC curriculum and training approach, and can help inform OWH in future policymaking efforts. Information collected will also help researchers and practitioners better understand the impact of adopting a trauma-informed approach on the provision of care community-based practitioners.
What we are doing
Abt Associates plans to visit ten of the OWH sites where the TIC training (or training and TA) was offered. Within each of those regions, Abt will visit approximately four agencies/organizations that had representatives who attended the training and technical assistance.
Your participation
We are inviting you to voluntarily participate in an interview that should take no more than 30-45 minutes. In keeping with the purpose of this evaluation, as outlined above, we would like to learn about your knowledge of and perspectives on your program’s trauma-informed care practices. If you attended the training in your region, we are particularly interested in your feedback regarding any changes or impacts that you have instituted, experienced or observed as a result of the TIC training (or training and technical assistance effort). You will not be asked, either directly or indirectly, about any experiences of trauma.
Please understand that your participation is voluntary. Your decision about whether or not to participate will not affect your program, agency, or organization, your position or employment status in any way. If you do agree to participate, you may stop the interview at any time without penalty.
Privacy
If you choose to participate in the interview, you will be assigned an interviewee ID, which will be used in place of your name or other personally identifiable information. Interviewers will be instructed to keep your identity private. This interview will not be tape-recorded. Instead, interviewers will take notes. These notes will be kept private to the extent allowed by law and used to create site visit summaries. Again, there will be no personally identifiable information contained in the notes. The site visit summaries will be used to write a report to the Office on Women’s Health. The report will not identify you by name and will not attribute any statements or information to you or any other participant in the evaluation.
The reports will disclose information such as the agency/organization name, agency/organization characteristics and demographics of persons served, OWH Region, date of training (if you attended the training), and type of staff interviewed. Despite efforts to mask the identity of study participants, there is some risk that individual participants may be identifiable in reports and the data submitted to OWH due to the relatively small number of trainees in some participating agencies/organizations.
Risks/discomfort
We do not anticipate any immediate risk from your participation in this study. However, it should be noted that there is a small risk of breach of privacy should the data be lost or stolen. Given the relatively small number of staff within each facility, it is possible that, as noted above, your data could be re-identified; however, the procedures mentioned in the privacy section are in place to reduce these risks. These data will only be used for research purposes.
Benefits
There are no benefits or monetary incentives to you for participating in this study.
Cost
There are no costs to you for taking part in this study.
Who to contact if you have been harmed or have any concerns
This research has been approved by the Institutional Review Board (IRB) of Abt Associates. If you have any questions about the study please contact the Project Manager, Mica Astion, MSCJ, at [email protected] or 617-520-2568. If you have questions about your rights as a research participant or questions, concerns or complaints this study, you can contact Katie Speanburg with the Abt Associates IRB at 1-877-520-6835. A copy of this consent form will be provided to you at your request.
Statement of Agreement to Participate in the Research Study (answer yes/no)
Do you agree with the conditions described to voluntarily participate in this study? _________
Interviewer’s Name (Printed) |
|
|
|
|
|
Interviewer’s Signature |
|
Date Time |
OWH TIC Site Visit Interview
Protocol
Leadership, Line Staff (to include Clinicians, Program
Director/Manager, Direct Program/Service Staff), and Other Frontline
Staff
Name of Agency/Organization:
Interviewee ID:
Role in Organization:
Leadership
Line Staff
Other Frontline Staff
Lived Experience of Trauma (per participant training list information):
Yes No Information Not Available
Interview Date:
Attended Training: Yes No
[If Yes] Date of training:
Attended Training as Peer Agency/Organization: Yes No
The following questions should be asked of all interviewees.
What are the major goals and objectives of your group/organization?
How long have you worked for X agency/organization?
Can you describe your role in the group/organization (not in terms of the "official job description," but in terms of what you actually do)?
Can you describe any previous exposure to trauma-informed care education/trainings? (Note: Trauma Informed Care education transmits knowledge about how trauma affects persons seeking services, how a history of trauma creates vulnerabilities to practices, actions or circumstances that may trigger memories of traumatic experiences or events. These may include traditional service delivery approaches that may exacerbate trauma or re-traumatize an individual.)
Through this organization or elsewhere?
Does your agency/organization have a mission statement or other written policy regarding the trauma-informed approach to care? Explain.
Do you know if a written policy has been incorporated into service delivery?
If participant attended the training, also ask the following question: Is this a result of the training or did it exist previously?
Do you feel that this agency/organization provides a safe environment? Physically? Emotionally?
Do you feel comfortable sharing life experiences with other people here?
Is staff supervision made available in a private confidential space?
Do administrative and/or program structures provide enough support to you and your work?
If participant attended the training, ask the following question: Did the training have any impact on such support?
Are people with lived experience involved in providing services, resources, and/or support?
Is this new or has this involvement always been part of the agency?
Have you experienced any barriers to implementing trauma-informed practices within your agency/organization?
Have you developed any specific strategies to support implementation of trauma-informed practices?
Have you noticed any changes in the wider community, specifically in regards to trauma-informed care?
If the interviewee participated in the training, also ask the following questions:
What did you learn/what did you gain from the training that was useful?
Has the training impacted how you view trauma and/or trauma-informed care? Explain.
Did the training teach you any new skills that you found useful? Explain
As a result of the training, do you interact with women in the agency/organization differently than you did before? Explain
Your agency’s action plan listed the following planned changes:
Note all action items: ________________________
To the best of your knowledge, were these accomplished (by item)?
Were you involved in the implementation of these?
If not accomplished, why not?
Have any strategies that your organization outlined in their action plan been implemented?
Were you part of this process?
Has anything changed in the physical environment of the agency/organization as a result of the training? Explain.
Do you think the agency’s physical setting promotes safety?
Can you describe any changes you have noticed in how other staff interacts with women as a result of the training?
Have you or the agency/organization made any changes to policies, procedures, or practices as a result of the training?
What information that you learned during the training has been applied to practice?
In implementing any changes in policy, practice, or physical space of the agency/organization as a result of the training, has leadership encouraged collaboration among staff at all levels?
Has leadership engaged peers/service users in collaborating?
As a result of the training and/or any changes made since the training, have you noticed any impact on staff morale, turnover, and vicarious trauma/compassion fatigue?
As a result of the training and/or any changes made since the training, have you noticed any improvements in program outcomes?
Have you noticed any improvements in consumer/service recipient outcomes?
Since the training, have you noticed an increase of organizational support in regards to trauma-informed practices?
The checklist below is an adaptation of Handout 2.2 Mapping Tool which was part of Module 2. The content of the checklist is not exhaustive, but tracks the content of the Mapping Tool provided in the training curriculum. The checklist will be used by the project staff during tours of the selected agency/organizations in face-to-face site visits. For virtual site visits, the items on the checklist will be reviewed with the designated program director or manager as part of the interview process. Since responses will not be observed by the evaluators, this will be recorded in the Notes/Comments section of the checklist form. The checklist is intended to help Abt project staff assess implementation of action plan items indicated by the individual agencies/organizations.
Creating Conditions for Safety: Elements of a Trauma Informed Environment – Physical Environment and Accessibility Checklist |
|
Action Plan Item (Y/N) |
Observable |
Notes/Comments2 |
||
Yes |
No |
N/A |
|||
Confidentiality and Privacy |
|||||
Space for private conversations for program participants and staff |
|
|
|
|
|
Accessibility |
|||||
All doors have automatic openers and all furnishings set up for ease of movement of wheelchairs and walkers |
|
|
|
|
|
All materials available in audio versions as well as big print |
|
|
|
|
|
Interpreters available for the deaf and hard of hearing when requested |
|
|
|
|
|
Building accessible from street level to program area |
|
|
|
|
|
Appearance |
|||||
Space kept clean and neat |
|
|
|
|
|
Well lit space |
|
|
|
|
|
Well lit parking area at all times |
|
|
|
|
|
Furnishings comfortable |
|
|
|
|
|
Hours of operation posted |
|
|
|
|
|
Climate |
|||||
Layout of space promotes interactions between program participants and staff |
|
|
|
|
|
Posted signs have "person first language" |
|
|
|
|
|
Someone always available to welcome anyone walking into space |
|
|
|
|
|
Code of ethics is posted in common areas |
|
|
|
|
|
Cultural Competence |
|||||
Signs posted in different languages to meet needs of community |
|
|
|
|
|
Images and language on posters and artwork represent the demographics of the community |
|
|
|
|
|
Staff in reception area and program services can communicate or access interpreters in key languages |
|
|
|
|
|
Common areas – please describe the common areas, note number of people, perceived comfort level, and anything else of interest. |
|
|
|||
Cross-site Evaluation of the National Training Initiative on Trauma-Informed Care (TIC) for Community-Based Providers from Diverse Service Systems
Site visits will take place over 3 days (including up to 1 travel day and a minimum of two on site days) and will include two project staff (one senior-level staff and one junior-level staff). As a rule, the site visits will be divided into four ½ day segments with selected agencies/organizations, generally lasting approximately 3–4 hours each.
Agency/Organization Group Meeting (30 minutes)
Each agency/organization site visit will begin with an initial group meeting which will include all representatives to be interviewed and any additional staff that leadership deems appropriate. During this initial meeting we will:
Lead introductions;
Provide a brief overview of the evaluation project goals and objectives;
Review logistics for interviews;
Gather any documentation or information that was previously requested but not acquired;
Discuss any changes made to the policies, procedures, practices and/or physical space of the organization as a result of the training; and
Discuss exit meeting logistics, if one has been requested
Interviews (2–3 hours)
One-on-one
interviews with previously identified leadership and line staff, and
will begin following the tour. Interviews will be conducted in a
private space determined pre-site visit and confirmed during the
group meeting. We estimate conducting approximately 4 interviews per
site, allowing
30–45 minutes per interview. In the event
that interviews finish early, and there are additional staff who are
willing and available to speak with us, we will accommodate
additional interviews.
Before initiating any of the interviews, the interviewer will provide interviewees with a copy of the informed consent form to review. If the interviewee agrees to participate, the interviewers will continue with the interview protocol. If the interviewee declines to participate, Abt will ask the site visit POC if they wish to make a substitution.
Interviewee 1 (30–40 minutes)
Interviewee 2 (30–40 minutes)
Interviewee 3 (30–40 minutes)
Interviewee 4 (30–40 minutes)
Tour (30 minutes)
Following the interviews, we will request a tour of the physical space of the agency/organization. The tour is intended to help assess any changes in the physical space of the agency/organization that were made based on knowledge gained during the training and outlined in their Action Plan. The purpose of the tour is to help in evaluating the impact of the training, not to evaluate the success of the agency/organization itself. We will use an observational checklist to capture such changes and areas of notice.
Exit and Follow-up (15–20 minutes)
We will offer agencies/organizations the option of an exit meeting during the introductory conference call with agency/organization leadership. If the agency/organization requests an exit meeting, we will be available for a brief overview of interviewer impressions and general feedback. These meetings should last no longer than 15–20 minutes. If the agency/organization declines the exit meeting, we will provide notice upon leaving to the administrative offices of the agency/organization that we are departing and that we will provide them with a copy of any reports that OWH disseminates as a result of the evaluation. Within 10 business days of return from the site visit, we will send all participating agency/organizations letter of thanks to the leadership, and summarize in brief our visit.
Notes:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 The term “walkabouts” refers to a tour of the organization that is intended to help assess changes in the physical space of the organization that were made based on knowledge gained during the training.
2 Comments may include successful interventions, barriers to implementation, or any other specifics indicated by the agency/organization.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Site Visit Protocol |
| Author | OWH/HHS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |
© 2024 OMB.report | Privacy Policy