HHS/CDC/NCIPC
SUPPORTING STATEMENT FOR
OMB INFORMATION COLLECTION REQUEST
Part A
A Controlled Evaluation of Expect Respect Support Groups (ERSG): Preventing and Interrupting Teen Dating Violence among At-Risk Middle and High School Students
OMB# 0920-0861 (exp. 8/31/13)
Supported by:
Centers for Disease Control and Prevention
National Center for Injury Prevention and Control
Division of Violence Prevention
Research and Evaluation Branch
(CDC/NCIPC/DVP/RE)
Government Project Officers:
Khiya Marshall, Dr.P.H., Science Officer
[email protected]/ 770-488-3911
Kristin Holland, M.P.H., Project officer
[email protected] / 770-488-3954
National Center for Injury Prevention and Control
Division of Violence Prevention
Prevention Development and Evaluation Branch
4770 Buford Hwy, NE, MS F-64
Atlanta, GA 30341
Table of Contents
Abstract
A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
2. Purpose and Use of Information Collection
3. Use of Information Technology and Burden Reduction
4. Efforts to Identify Duplication and Use of Similar Information
5. Impact on Small Business or other Small Entities.
6. Consequences of Collecting Information Less Frequently.
7. Special Circumstances Relating to the Guidelines of 5CFR 13205
8. Comments in Response to Federal Register and Efforts to Consult Outside the Agency.
9. Explanation of Any Payments or Gifts to Respondents.
10. Assurance of Confidentiality Provided to Respondents.
11. Justification for Sensitive Questions.
12. Estimates of Annualized Burden Hours, and Costs.
13. Estimates of Other Total Annual Cost Burden to Respondents or Recordkeepers.
14. Annualized Costs to the Federal Government
15. Explanation for Program Changes
16. Plans for Tabulation, Publication, and Project Time Schedule.
17. Reason(s) Display of OMB Expiration Date Inappropriate.
18. Exemptions to Certification for Paperwork Reduction Act Submissions
LIST OF ATTACHMENTS
Attachment A. Public Health Service Act: Sections 301 (42 U.S.C. 241)
Attachment B. Published 60-Day Federal Register Notice
Attachment C. Intake Assessment
Attachment D. Baseline Survey
Attachment E. Completion and Follow-up Survey (12-months)
Attachment F IRB Approval (CDC)
Attachment G. Student Assent Form, Script and Parent Notification Form
Attachment H. Expect Respect Support Group / Texas Policy for Mandatory Reporting
Attachment I. Evaluation of Expect Respect Support Groups Fidelity Monitoring Survey
Attachment J. Fidelity Monitoring Observation Survey of Expect Respect Support Group Sessions
Attachment K. Mid-year ERSG Facilitator Qualitative Interview Guide
Attachment L. End-of-year ERSG Facilitator Qualitative Interview Guide
Attachment M. Public Comment
The prevalence and consequences of teen dating violence make it a public health concern that requires early and effective prevention. To date, only three prevention strategies – Safe Dates, the Youth Relationships Project, and 4th R – have demonstrated reductions in dating violence behaviors in rigorous, controlled evaluations. In order to protect young people and build an evidence-base of effective prevention strategies, evaluation of additional programs is needed, including those programs currently in the field. Expect Respect Support Groups (ERSG) were identified by CDC through the empowerment evaluation process as a program in the field that is in need of rigorous evaluation. The ERSG program has demonstrated favorable, albeit preliminary, results in a pre-post program evaluation, which strongly suggest that a controlled evaluation is needed to more rigorously examine program effects. The study has two primary goals and two exploratory aims. The primary goals are: 1) To evaluate the effectiveness of Expect Respect Support Groups (ERSG) in preventing and reducing teen dating violence and 2) To compare whether there are increased healthy conflict resolution skills reported by at-risk male and female middle and high school students participating in ERSG, compared to at-risk students in control schools who do not receive ERSG. The exploratory aims are: 1) To evaluate whether the effectiveness of ERSG is enhanced by the presence of a universal, school-wide prevention programs, and 2) To examine whether participants with different characteristics (e.g., girls vs. boys) respond differently to the intervention.
A. Justification
The Centers for Disease Control and Prevention (CDC) is seeking a 3-year extension of OMB package #0920-0861, entitled “A Controlled Evaluation of Expect Respect Support Groups (ERSG): Preventing and Interrupting Teen Dating Violence among At-Risk Middle and High School Students”. The scope of the study has not changed since its initial OMB approval in August 2010. ERSG is a research-based adolescent dating violence prevention program, which consists of a 24-session dating violence curriculum. The proposed participants are middle and high school students representing diversity in race/ethnicity, geography, and family income.
Progress Updates and Changes since Original OMB Approval
This evaluation was originally planned to be conducted over a period of four years. However, due to problems encountered with participant enrollment stemming from a school district requirement for active parental consent, the first year of data collection (2010-2011) was modified by the CDC Procurement and Grants Office in collaboration with the Division of Violence Prevention to be considered a pilot year. After implementing active parental consent from September 2010 through May 2011, it was noted that requiring parental permission to participate in the study dramatically affected participant enrollment. As such, the administrators from the control district with strict regulations requiring active parental consent was approached to discuss their willingness to allow the study to move forward using passive parental consent instead. The district superintendent and principals were informed of similar studies that have taken place elsewhere in the United States (e.g., the Second Step evaluation in Chicago, IL) which have required passive parental consent and thus obtained higher levels of participant enrollment. Due to the important nature of this study and its potential impact on teen dating violence, local regulations were modified in order to help SafePlace staff increase student enrollment.
The evaluation plan for the contract was thus modified so that Year 1 (September 2010-August 2011) was designated a pilot year and was used to improve program implementation procedures (e.g., student and school recruitment, training, and referral protocols were established; control schools were recruited; and standard practices for control and intervention group data collection and implementation were established). Years 2-4 (September 2011-August 2014) will be used to carry out the program evaluation at full-scale, including 1) data collection from 200 students every year for 3 years in control and 200 students every year for 3 years in intervention groups at baseline and completion, and 2) retention of at least 90% of participants at follow-up (i.e., 12-month follow up period). Finally, 12-month follow-up of the third cohort of participants and all data analyses and manuscript preparation will take place in Year 5 (September 2014-September 2015).
To summarize, data are being collected in three waves among three separate cohorts of participants:
Year 2 (Cohort 1): August 2011 – March 2013 (includes intake/assessment, baseline, completion, and 12-month follow-up survey administration / data collection)
Year 3 (Cohort 2): August 2012 – March 2014 (includes intake/assessment, baseline, completion, and 12-month follow-up survey administration / data collection)
Year 4 (Cohort 3): August 2013 – March 2015 (includes intake/assessment, baseline, completion, and 12-month follow-up survey administration / data collection)
To date, the contractor has met all recruitment and enrollment deadlines and requirements for Years 2 and 3 of the evaluation period. Baseline survey data were collected for Years 2 and 3 from August 2011-January 2012 (Cohort 1) and August 2012- January 2013 (Cohort 2), respectively. Completion survey data were collected from April 2012-May 2012 for Cohort 1, and follow-up survey data were collected for Cohort 1 from August 2012-February 2012. Completion survey data collection for Cohort 2 will be collected from April-May 2013.
Over the past three years, the following three non-substantive change requests have been submitted to and approved by OMB:
Four items were removed from the baseline and completion/follow-up questionnaires because it was determined that these items were not necessary to address study questions.
In order to measure how closely the ERSG curriculum is being followed and how the program is being implemented in general, we added a self-report fidelity measure which the eight ERSG facilitators complete at two separate time points during the study period. We also added a second observational fidelity measure to be completed by the lead ERSG facilitator, who will observe two separate sessions for the eight facilitators during the study period. The facilitators’ supervisor completes a fidelity instrument to record the extent to which the curriculum is followed and to record how the program is being implemented in general. These fidelity measures will be used as a tool to inform program improvement. Additionally, because of difficulty following up with a transient student population, the 18-month follow-up survey was removed from this study. This resulted in a reduction in participant burden by 400 hours.
Qualitative interviews with ERSG facilitators were added to the study protocol. The goal of these interviews is to better understand the implementation process for ERSG. For example, interview questions aim to explore factors that contribute to group engagement for participants, how facilitators make decisions about the format and delivery of session content, and the various group dynamics within ERSG. This qualitative information will provide a more nuanced picture of implementation processes of the program.
Background
The
prevalence and consequences of teen dating violence make it a public
health concern (Wolitzky-Taylor et al., 2008; Eaton, Davis, Barrios,
Brener, & Noonan, 2007) that requires early and effective
prevention. Despite recent legislation in Texas and Rhode Island that
requires schools to provide teen dating violence education, to date,
only three prevention strategies—Safe Dates, the Youth
Relationships Project, and 4th R (Foshee et al., 1998; Wolfe et al.,
2003; Wolfe et al., 2009) – have demonstrated reductions in
dating violence behaviors in rigorous, controlled evaluations
(Hickman, Jaycox, & Aronoff, 2004). In order to protect young
people and build an evidence-base of effective prevention strategies,
evaluation of additional programs is needed, including those programs
currently in the field (Teten, Ball, Valle, Noonan, & Rosenbluth,
2009).
Teens are at risk for experiencing dating abuse
beginning with the initiation of dating relationships during early
adolescence. Nearly half of 11- to 14-year olds report that they have
been in a dating relationship (Teen Research Unlimited, 2008). Among
those who experience dating violence victimization, 29% report their
first experience of abuse occurred at age 12-13, 40% at age 14-15,
and 29% at age 16-17 (Burcky, Reuterman, & Kopsky, 1988). Up to
45% of high school students report experiencing some form of
emotional, physical, or sexual violence in their dating relationships
(Foshee, 1996; O’Keefe, 1997; Silverman, Rai, Mucci, &
Hathaway, 2001). Dating violence is associated with subsequent
adverse consequences, including substance abuse, sexual risk
behaviors, unintended pregnancy, sexually transmitted diseases,
unhealthy weight control behaviors, depression, and suicidality
(Teten et al., 2009).
Although traditional
conceptualizations of dating violence suggested males were
perpetrators and females were victims, surveys of teen dating
violence that assess both girls’ and boys’ violence
perpetration consistently report a higher percentage of girls than
boys perpetrate physical violence (Foshee, 1996; O’Keefe, 1997;
Hickman et al., 2004; Sears, Byers, & Price, 2006). Critics of
these findings suggest the effects of male and female-perpetrated
physically violent acts differ substantially in terms of injury
sustained and fear evoked (Dobash, Dobash, Wilson, & Daly, 1992).
Molidor, Tolman, and Kober (2000) found that 48% of girls in their
sample and only 4% of boys reported that a violent incident “hurt
a lot.” Boys more often (54%) responded that they “laughed”
about the violence compared to girls (10%). While the context,
meaning, and effect of violent behaviors may differ between boys and
girls, studies nevertheless point to the importance of supporting
both boys and girls in learning skills for healthy relationships. As
noted by O’Keefe (1997): “every violent action creates a
risk for a violent response or future violent acts” (p. 6).
Given the prevalence and age of initiation reported for teen dating
and teen dating violence, research suggests that prevention programs
should involve boys and girls beginning with the middle school years.
Multiple studies suggest that teens’ experiences
with violence and violence-supportive or accepting attitudes are
linked with the perpetration of dating violence (Wolfe, Wekerle,
Reitzel-Jaffe, & Lefebvre, 1998; Wolfe, Wekerle, Scott, &
Pittman, 2001; Wolfe, Wekerle, Scott, Straatman, & Grasley, 2004;
Malik, Sorenson, & Anehensel, 1997; O’Keefe, 1997, 1998).
While the mechanism of these associations is unclear, a history of
child maltreatment has been associated with boys threatening or
carrying out dating violence, with boys experiencing dating violence,
and with girls being victims of such violence (Wolfe, Wekerle, Scott,
& Pittman, 2001). Witnessing parental violence (O’Keefe,
1997, 1998) has been associated with the perpetration of dating
violence among boys but not girls. Wolitzky-Taylor et al. (2008)
observed an association between experiencing serious physical and
sexual dating violence and having experienced a previous traumatic
event, such as sexual or physical assault by someone other than a
dating partner, having witnessed community violence and parental
violence, and experiencing the loss of a loved one.
Prevention
approaches for dating violence may be applied universally to all
individuals in a population, regardless of the risk factors described
above, or they may have targeted application, such that individuals
at risk for experiencing or perpetrating violence receive the
prevention intervention. Universal approaches are often didactic and
classroom-based, aiming to educate teens about healthy and abusive
relationships (e.g., Avery-Leaf, Cascardi, O’Leary, & Cano,
1997; Foshee et al., 1998; Jaffe, Suderman, Reitzel, & Kilip,
1992; Schewe, 2002), whereas targeted approaches may be tailored to
the particular needs of the at-risk group and offered during the
school day but outside of a classroom curriculum. Experts have
recommended intensive prevention programs for youth who have
experienced violence in the family and in the community and who
perpetrate or experience abuse in their peer and early dating
relationships (Wolfe et al., 2003; Pepler et al., 2006; Williams et
al., 2008). In addition to countering risk factors, such as violence
supportive attitudes (Capaldi et al., 2001), targeted prevention
programs may also enhance potential protective factors, such as
establishing positive relationship norms in the peer group (Williams
et al., 2008). Teaching and practicing positive behaviors is an
important aspect of building relationship competence. Cornelius and
Resseguie (2007) suggest that without a skill-building component to
improve proficiency of communication, negotiation, and problem
solving skills the likelihood of behavior change is improbable. Teens
themselves express the importance of learning skills and ask for
assistance to learn “how to make relationships work”
(Ball, Kerig, & Rosenbluth, 2009; Sears, Byers, Whelan, &
Saint-Pierre, 2006). Because some students may be at higher risk for
dating violence, the intensity of a targeted approach in addition to
(or in place of) a universal strategy may be needed for these
students (e.g., Eaton et al., 2007; Whitaker et al., 2006).
Thus
far, the Youth Relationships Project (YRP; Wolfe et al., 2003) is the
only published and rigorously evaluated dating violence prevention
program that targets at-risk adolescents. The YRP is an 18-session
program that provides coeducational groups in community locations for
youth with a history of maltreatment. The controlled evaluation
demonstrated decreases in abuse perpetration and victimization and
emotional distress symptoms; however, participants did not show
expected growth in healthy relationship skills.
In
contrast to the Youth Relationship Project, Expect Respect Support
Groups (Ball, Rosenbluth, & Aoki, 2008) are offered at middle and
high schools - that is, in the social environment where about 40% of
the worst dating violence incidents occur (Molidor, Tolman, &
Kober, 2000). School-based support groups are designed to reach a
wide range of at-risk students and to maximize accessibility.
Students are eligible for participation in ERSG if they have
witnessed domestic violence and/or experienced child abuse
(emotional/mental, physical, sexual, neglect) and/or are involved in
abusive peer and dating relationships. Boys and girls meet in
separate gender groups facilitated by same-gender group leaders,
which is intended to increase the sense of comfort and safety for
teens, to allow them to bond more quickly, and to explore their
expectations for relationships. The primary goal of the 24-session
program is to prevent at-risk youth from becoming future victims and
perpetrators in their intimate relationships and to promote healthy,
non-violent relationship behaviors.
The present application proposes the extension of an ongoing evaluation of Expect Respect Support Groups that involves a quasi-experimental design with a well-matched comparison group. The Expect Respect program was selected as the sexual and teen dating violence prevention program to be evaluated through the recent CDC Empowerment Evaluation (EE). Although many sexual violence prevention programs are in practice, very few have been rigorously evaluated. The EE “was CDC’s response to this gap in prevention programming and involved two strategic decisions: the focus on a small number of established programs that aim to prevent first-time male perpetration of sexual violence and the use of empowerment evaluation. This twin approach offered the greatest possible benefit in building evaluation capacity among organizations in the field while expediting the development of an evidence base for prevention programs. The first strategy guiding CDC’s approach was the decision to work with existing programs to build a knowledge base that could expedite widespread use of evidence-based prevention efforts in practice settings. This decision was novel because, according to the public health model, the starting point of program development and evaluation is often efficacy research, an approach with considerable merits (e.g., use of theory, knowledge of risk and protective factors, tightly controlled conditions, and ability to make causal inferences with random assignment designs). However, in this case, CDC chose to work with existing programs because of the growing recognition that most prevention efforts developed in research institutions (however effective they may be) do not get adopted in practice settings for myriad reasons (e.g., Institute of Medicine, 2001; Miller & Shinn, 2005; Wandersman et al., 2008). By working with extant programs, this project sampled from a universe of strategies with demonstrated real-world feasibility and appeal to practitioner audiences, two keys to promoting adoption and use of health innovations (Dearing, Larson, Randall, & Pope, 1998; Glasgow, Lichtenstein, & Marcus, 2003; Kim & Cho, 2000). As a first step in preparation for the empowerment evaluation, it was necessary to identify and describe these programs. To this end, the project team reviewed lists of experts, government documents, Web sites, published literature, and unpublished reports in the field of sexual violence prevention. Individuals and agencies identified through these activities were contacted for information about their programs. During these conversations, a snowball sampling approach was used to find additional programs, with 37 eventually identified. Publicly available information was supplemented via structured telephone interviews with program leaders to compile profiles for each program. The resulting summary included descriptions of the populations served, intervention approach, goals and objectives, theoretical frameworks, evaluation activities, and staff capacity (RTI International, 2003).
Each of the programs identified through this process was invited to apply to participate in the empowerment process, with invitations also circulated on relevant list servers. A total of 17 applications were received and reviewed by an expert panel assembled by the project team. Selection criteria included the extent to which programs focused on prevention of first-time male perpetration in a multisession format, prior experience in evaluation, commitment to using evaluation for program improvement, and ability to commit staff resources to participation in the empowerment evaluation process. Following extensive review, four organizations were selected,” (p 6s-8s, Noonan & Gibbs, 2009) of which one was SafePlace of Austin, Texas. Safe Place worked with CDC through the EE process to build its evaluation capacity (The EE process is also described in detail in other CDC documents such as Cox et al., [2009]). SafePlace stood out among the other programs in their willingness and ability to apply program evaluation to their activities. These activities in addition to their evaluation capacity increased the impetus for a controlled evaluation of their program.
For example, as a result of the EE, SafePlace conducted a non-CDC funded pilot evaluation of 26 ERSG (in 16 schools). In this pre-post design, significant increases in positive dating behaviors and significant declines in emotional and physical victimization and perpetration were found for students who reported a history of peer and dating violence (Ball, Teten, Noonan, Valle, Hamburger, & Rosenbluth, 2012). Perpetration of sexual violence increased over time. We attribute the rise in this low base rate behavior (M = 0.03 to M = 0.13) to its measurement (2 items), increases in awareness due to intervention, and timing of the final assessment (e.g., students completed final assessments in session 23 of 24 and reported on their behavior over the previous 3 months, so intervention effects may not yet have been achieved).
Based on these preliminary results, the current study proposes the extension of a controlled evaluation of ERSG. In this evaluation designed by CDC, we address the weaknesses of the aforementioned pilot by expanding the assessment of sexual violence and using a control group and follow-up assessment to clarify change in sexual and teen dating violence as a result of the intervention. A controlled evaluation rather than an RCT was selected as the design for the proposed evaluation, as results of the pre-post evaluation warranted further study, but the potential for iatrogenic effects discouraged the application of a large-scale independently conducted randomized design. Instead, the logical next step for the program was determined to be a quasi-experimental design that capitalizes on the capacity within Austin and SafePlace that was developed through the EE process. This evolution of evaluation is also consistent with CDC’s continuum of evidence (CDC, 2010) and demonstrates a developmental progression of evaluation rigor that is applied to programs moving from the right to the left of the public health model, or, in other words, programs that are in practice that are then subjected to evaluation.
Such an evaluation is also consistent with past evaluations funded by CDC, such that the first trial is conducted in close collaboration with the developer to ensure implementation fidelity (e.g., evaluation of Safe Dates).
In
addition to implementing a more rigorous evaluation design, CDC has
developed mechanisms for training and oversight of the data
collection and evaluation (described in detail below) that will
provide safeguards for the objectivity of the evaluation.
The proposed data collection fits into the National Center for Injury Prevention and Control (NCIPC) Research Agenda Priorities in Preventing Sexual Violence and Intimate Partner Violence (http://www.cdc.gov/injury/ResearchAgenda/index.html) with regard to Tier 1 Part E to “Evaluate the efficacy and effectiveness of programs, strategies, and policies across all levels of the social ecology to prevent and interrupt development of perpetration of sexual violence and intimate partner violence,” and Part F to “Evaluate the efficacy and effectiveness of programs, strategies, and policies to prevent both sexual violence and intimate partner violence, multiple types of sexual violence and intimate partner violence, and other forms of violence.” The proposed data collection also addresses NCIPC Research Agenda Priorities in Preventing Sexual Violence and Intimate Partner Violence with regard to Tier 2 Part I to “Evaluate interventions for persons exposed to sexual violence and intimate partner violence to reduce risk for associated negative health consequences.”
Authority for CDC’s National Center for Injury Prevention and Control to collect this data is granted by Section 301 of the Public Health Service Act (42 U.S.C. 241) (Attachment A). This act gives federal health agencies, such as CDC, broad authority to collect data and do other public health activities, including this type of study.
A.1.1. Privacy Impact Assessment
Overview of the Data Collection System
Data Collectors: Data collection will be conducted by qualified individuals, other than the school’s Expect Respect facilitator, who are employed by the contractor. Data collectors will have training in working with at-risk students. The following steps have been and will continue to be implemented by CDC to safeguard the objectivity of the evaluation: 1) all data collectors (also referred to as survey administrators) will receive human subjects training; 2) documents to support data collection which contain standardized responses to common questions asked during data collection will be provided to data collectors. To develop these documents CDC worked with SafePlace to identify particular words, instructions, or issues in the survey that may be unclear to students with lower verbal ability. Standard responses were drafted so that data collectors across schools will provide similar answers to participants; 3) CDC prepared data collection flow charts that provide detailed instructions to the data collector and ensure fidelity to standardized survey administration; 4) CDC will conduct site visits and hold weekly or biweekly conference calls with the contractors to provide oversight and discuss data collection procedures and progress; 5) CDC staff will provide consultation via email and conference calls to the contractor; and 6) when participants complete the survey, they will place the survey in an envelope and seal the envelope. The survey administrator will then deliver the sealed envelope to the data manager. The data collectors will not have access to the survey responses. As the CDC principal investigator on the CDC IRB protocol, Dr. Khiya Marshall is responsible for overseeing the scientific and human subjects integrity of the study. These aspects of oversight, in addition to other measures determined to be appropriate over the course of data collection, will ensure that CDC will be able to have confidence in the results of the evaluation of Expect Respect. It should be noted that it is necessary to have the data collectors in the room with the students while they complete the survey both to manage emotional upset that may occur when reading survey questions about relationships and to answer clarifying questions about the survey questions or process (e.g., skip patterns).
Recruitment: To determine appropriateness for the evaluation, students referred (by self or school staff) will undergo a brief screening assessment (Attachment C). The intake assessment follows a screening intake session, which is a service provided by SafePlace to intervention and control schools. All intake assessments are conducted by trained data collectors with a background in mental health. Data collected during the intake assessment is not federally funded and will not be used for the evaluation. During the intake interview, ERSG data collectors conduct an open-ended interview with students to determine their safety level of risk, and appropriate service referrals, which may include ERSG. If, based on this screening, which is considered service provision and not part of the evaluation, the data collector determines that the student is eligible for the evaluation, he/she will complete the intake assessment form, will discuss the evaluation with the student. If the student indicates that he/she would like to participate, a parent notification form (Attachment G) is sent to the student’s home. After a 2-week period during which parents can opt their child out of the study, eligible students whose parents did not dissent to the study complete a signed assent form (Attachment G) to participate in the study. Students who have participated in ERSG in past years will not be eligible for completing the baseline survey. However, because students may participate in ERSG for multiple years, if it is determined at follow-up that a participant is continuing to take part in ERSG, he/she will indicate this on the portion of the survey that assesses other prevention programs in which the student is participating.
Baseline Survey. Students who are eligible for the evaluation and who participate in the ERSG and control groups will complete a baseline survey (Attachment D) to assess recent peer and dating violence, healthy relationships behaviors, and demographic and attitudinal variables which may mediate/moderate intervention response. The baseline survey will be administered to the entire support group during the scheduled group meeting time the week after the first session.
Post-test
and Follow-up. After completing the baseline survey, students in
the ERSG schools will then attend 24 weekly support group sessions
that take up to 8 months to complete. Students in the control group
will go about their schedule as normal, not attending support group
sessions, but may also receive short-term services (individual
psycho-education at school [1-3 sessions] by trained counselors
staffed by SafePlace), if needed. In addition to the intake
assessment, surveys will be conducted before group participation (at
baseline), at the completion of group (8 months; Completion Survey,
Attachment E), and at one follow-up time (a minimum of 12
months after baseline; Follow-Up Survey, Attachment E). The
completion survey will be administered to the entire support group no
earlier than two weeks prior to the scheduled last session and no
later than two weeks after the last group session. The follow-up
survey will be administered 12 to 15 months after baseline survey
administration.
Students who miss the group survey
administration will be contacted to complete the survey (baseline or
completion) individually. To ensure privacy, a survey packet will be
assembled for each student. The package will include a manila
envelope, 2 copies of the student assent form, and a copy of the
survey with the student’s SafePlace identification number, the
school identification number, and the facilitator’s
identification number filled in on page 2. A sticky note with the
student’s name on it will be affixed to the package prior to
survey administration but will be removed after handing out the
survey. To further ensure student privacy, an appropriate room in
which to administer the survey will be selected. The room will have
enough space for each student to have privacy while answering the
questions, and students will not have to worry about others seeing
their answers. The group facilitator will leave the room during the
survey administration, but will be nearby in case he/she is needed in
the event that a student experiences emotional distress or an adverse
event occurs. However, the survey administrator will remain in the
room during survey administration to read instructions and answer any
questions students might have about the survey. After students have
completed the survey, they will be instructed to place the completed
survey in the manila envelope and to seal the envelope, which will
remain sealed until they are opened at SafePlace and data are ready
to be entered into the computer. In control schools, surveys will be
administered in a one-on-one setting with only a research assistant
or intern and the student present in an empty office or classroom to
ensure student privacy. Follow-up data will be collected in
one-on-one sessions with the student 12 to 15 months after the
baseline assessment. Follow-up surveys will be administered in an
empty office or classroom, in order to allow the student privacy and
to ask the survey administrator questions about the survey.
Below is a summary of the three cohorts of data that have been and will be collected as part of the evaluation:
Cohort 1: August 2011 – March 2013 (includes intake/assessment, baseline, completion, and follow-up surveys administration / data collection)
Cohort 2: August 2012 – March 2014 (includes intake/assessment, baseline, completion, and follow-up surveys administration / data collection)
Cohort 3: August 2013 – March 2015 (includes intake/assessment, baseline, completion, and follow-up surveys administration / data collection)
Data collected thus far has allowed us to assess whether there are statistically significant differences between the treatment and control groups and will help us to evaluate the program's effectiveness in preventing peer and dating violence, reducing risk factors, and promoting protective factors for violence and victimization.
ERSG Facilitator Fidelity Monitoring Surveys. Twice a year, ERSG facilitators will complete a self-report Program Implementation Fidelity Monitoring Survey (Attachment I) in order to assess how closely the ERSG curriculum is being followed and how the program is being implemented in general. Additionally, a second observational fidelity measure, the Fidelity Monitoring Observation Survey (Attachment J) will be completed by the lead ERSG facilitator, who will observe two separate sessions for the eight facilitators during the study period. The facilitators’ supervisor will complete a fidelity instrument to record the extent to which the curriculum is followed and to record how the program is being implemented in general. These fidelity measures will be used as a tool to inform program improvement (Attachment F).
ERSG Facilitator Qualitative Interviews. ERSG facilitators will take part in a qualitative interview at one mid-year time point and at the end of each school year (Attachments K and L, respectively). This interview will serve to help program evaluators understand the implementation process for ERSG. For example, interview questions will aim to explore factors that contribute to group engagement for participants, how facilitators make decisions about the format and delivery of session content, and the various group dynamics within ERSG. These interviews will be conducted by CDC staff (Attachment F).
Data collected from the Program Implementation Fidelity Measures and the Mid- and End-of-Year Qualitative Interviews will be used for program improvement purposes. Interviews are recorded by CDC staff conducting the interviews, and following completion are transcribed by CDC staff. During transcription, all identifying information about the facilitators is removed prior to sharing the transcripts with SafePlace staff, thereby ensuring the privacy of the facilitators.
Items of Information to be Collected
The various survey instruments to be used in the evaluation (Attachments C-E) collect information about emotional, physical, and sexual peer and dating violence victimization and perpetration, use of healthy relationship skills, relationships characteristics, peer relationships, demographics, social desirability, service utilization, attitudes toward dating violence, and exposure to prevention programs other than ERSG. These measures were developed in collaboration with scientists at the CDC through the Empowerment Evaluation process and underwent subsequent revision by CDC sexual and teen dating violence experts. Measures are adapted from validated measures of teen dating violence, and reflect the behaviors of interest and theory of change of Expect Respect. The Reactive Proactive Questionnaire (Raine et al., 2006; Part 4 of Attachments D and E) has also been included in the instrument packet and will be used to determine if subtype of aggression moderates response to intervention.
Participants will be assigned a random identification number by the survey administrator, and the number, rather than personally identifying information, will be used on the survey instruments. Only the survey administrator and data manager will be able to link the student’s survey responses to his/her name.
Fidelity monitoring surveys (Attachment I) completed by the facilitators collect information about how closely the ERSG curriculum is being followed and how the program is being implemented in general. The observational fidelity monitoring surveys (Attachment J) completed by the lead ERSG facilitator collect information about the extent to which the curriculum is followed and to record how the program is being implemented in general.
ERSG facilitator qualitative interview questions (Attachments K-L) aim to explore factors that contribute to group engagement for participants, how facilitators make decisions about the format and delivery of session content, and the various group dynamics within ERSG.
A.2. Purpose and Use of Information Collection
The information collected under the proposed data collection will be used to:
Evaluate the effectiveness of ERSG for preventing and reducing teen dating violence perpetration and victimization and increasing healthy relationship skills reported by at-risk male and female middle and high school students (Primary Aim)
Evaluate whether the effectiveness of ERSG is enhanced by the presence of a universal, school-wide teen dating violence prevention program (Exploratory Aim 1)
Examine whether participants with different characteristics respond differently to the intervention. For example, we will determine whether outcome for boys or girls are the same. (Exploratory Aim 2)
SafePlace (http://www.safeplace.org/) originated as a domestic violence and sexual violence emergency shelter. Over time, while continuing to support the immediate needs of survivors through the 24-hour hotline, hospital accompaniment, shelter and counseling services, the agency has added programs that educate, build awareness, provide support in accessing resources, and work with the children of survivors to move beyond the trauma resulting from the violence in their lives. SafePlace provides extensive community outreach, education and prevention programs through the Expect Respect Program for teens and youth, Disability Services, Deaf Services, Community Education and Community Dialogue. SafePlace is a non-profit organization that is funded by donations from individuals, community groups, businesses and foundations. Eighty-four percent of SafePlace’s operating expenses are spent directly on programs and services for individuals, families and our community.
SafePlace
currently implements ERSG in many middle and high schools in Austin
Independent School District (AISD) in Texas. The agency has an
ongoing relationship with AISD and offers ERSG as part of their
services to students and families. SafePlace has traditionally
collected information from ERSG participants as part of their
internal program evaluation procedures. Promising results from these
preliminary evaluations suggests a controlled trial of ERSG is
warranted.
Student data collected in the research study
will be used for a controlled evaluation. Three cohorts of data
(described above) will be collected beginning in Fall 2011, Fall
2012, and Fall 2013, respectively. Data will be collected over an
approximately 32-week period during which two surveys (baseline and
completion; Attachments D and E) will be administered.
Additionally, one follow up survey (Attachment E) will be
administered 12 months after the initial session. Preliminary
analyses will be conducted after each cohort of data collection is
completed. However, final analyses will be conducted after the
12-month follow-up survey of Cohort 3 is complete to determine the
effectiveness of the ERSG program.
The assessment, treatment, and prevention of interpersonal violence are perhaps the most important and most difficult tasks facing students, parents, and teachers. The legal, interpersonal, social, physical health, and psychological consequences of violence are far-reaching and are a constant strain on individuals and society. Expanding the understanding of effective teen dating violence prevention programs builds the evidence base, a goal which will benefit youth who are at risk or who are already perpetrating or experiencing dating violence. The data collected from subjects will allow us to evaluate the ERSG program, which could provide strong support for the implementation of similar programs in schools with at risk student populations. The negative consequences of not obtaining the proposed data include not being able to build the evidence base regarding effective programs that lead to a reduction in teen dating violence victimization and perpetration, as well as allowing for continued violence perpetration by and victimization of middle and high school students in the proposed intervention area since those students would not be able to benefit from the information provided to them regarding local violence prevention and mental health treatment resources, the valuable learning experience ERSG provides, and the increased awareness of warning signs and risk factors of teen dating violence.
A.2.1. Privacy Impact Assessment Information
How the Information Will Be Shared and for What Purpose
The purpose of the effectiveness evaluation is to evaluate the effect of the ERSG program on desired outcomes, including main effects of the program on emotional, physical, and sexual dating violence perpetration and victimization and healthy conflict resolution skills; mediator variables targeted by the ERSG program (and found to be significant mediators by Foshee et al., 1998), and variance through potential moderator variables. Changes in the following outcomes will be examined: dating violence perpetration and victimization, dating violence norms, gender stereotyping, conflict management skills, belief in the need for help when violence occurs, and awareness of services. The information from this study will provide empirical evidence of the effectiveness of a universal and selected program to prevent dating violence perpetration and victimization in a targeted community. Results from the study will guide CDC in formulating its recommendations regarding community implementation of dating violence prevention programs, as well as guiding other governmental agencies, professional and health care organizations, and women’s advocate groups in formulating their policies on dating violence prevention programs. The results of the study will help determine whether a large scale, independently conducted randomized trial of the ERSG program is warranted.
Participation in this study is voluntary and intrusions to the participants' sense of privacy will be minimized by only using data collected from students who have agreed for us to do so (through student assent and signed distribution of passive parental consent forms) and having the data coded in such a way to protect subjects’ privacy. Study participants are told in the assent form that their responses to the survey will be kept private as allowed by law. This level of protection should be sufficient to provide participants with the assurance of protection needed for full participation in the study. No information that could link a students' number to his/her data will be released to anyone outside of SafePlace. Instead responses will be assigned random codes. Using the codes, CDC faculty will never be able to link the participants' data to their identity. This protocol has been approved by the CDC Institutional Review Board (IRB #5937; Attachment F).
De-identified data will be shared with CDC staff by SafePlace staff by sending the data to CDC on a compact disc. Data will be shared for analysis purposes, so that CDC staff can assist with analyses upon completion of data collection.
Additionally, the Principal Investigator of this contract at SafePlace has extensive experience in analyzing qualitative data, so upon de-identification of the ERSG mid- and end-of-year qualitative interview transcripts, the transcripts will be shared with SafePlace staff who will assist with qualitative analysis. These de-identified data will be shared by compact disc. The data obtained from these qualitative interviews will be used for program improvement purposes only, not for research purposes.
Impact the Data Collection Will Have on the Respondents’ Privacy
Information will be collected from adolescent respondents to determine whether the effects of the program vary based on variables including previous history of dating violence and demographic characteristics. Of these variables, the only personally identifiable information that will be collected is the student’s name, which will only be used in the screening assessment. For all subsequent assessments, students will use a random identification number provided by SafePlace staff to identify themselves on their survey responses. When participants complete the survey, they will place the survey in an envelope and seal the envelope. The survey administrator will then deliver the sealed envelope to the data manager. Only the data manager will be able to link the student’s response to his/her name. The impact of this data collection on students’ privacy is very low since no information that could link a student's number to his/her data will be released to anyone outside of SafePlace, and the risk of breach of privacy is minimal. SafePlace staff will store personally identifying information in a locked cabinet, will use random subject numbers, will not use any personally identifiably information in databases, and will only distribute de-identified data sets to study collaborators.
ERSG facilitators will participate in the qualitative interviews voluntarily. Interviews will be conducted by CDC staff and recorded with a digital voice recorder. The recording of interviews will be transcribed and de-identified during the transcription process by using different person, school, district, and other identifying names when necessary. Upon de-identification of transcripts, the transcripts will be shared with SafePlace staff who will assist with qualitative analysis. The impact of this data collection on facilitators’ privacy is very low since no information that could link a facilitator’s name to his/her data will be released to anyone outside of CDC.
Effectiveness data will be collected from students using classroom-administered scannable paper and pencil questionnaires or questionnaires that will be coded and entered into a database. Although we considered alternate modes of administration of these questionnaires, we determined that conducting Web-based questionnaires could be difficult because not every student has access to the hard/software needed. Even if each classroom had a computer, there would be no privacy for the students and little availability for all students to use the computer to complete the survey in a timely manner. In sum, we determined machine-scannable or other paper and pencil questionnaires that could be systematically entered into a database would be the best and most private methodology for collecting student effectiveness data, while minimizing potential biases that might jeopardize our ability to address the evaluation research questions.
A literature search conducted in July 2009 and again in December 2012 of the Google Scholar, MEDLINE, PsycINFO, and CINAHL databases found no evidence of a rigorous, controlled evaluation of targeted programs to reduce teen dating violence among high-risk youth and comparison of those programs to the implementation of universal programs in a school system. No date restrictions were used in the search. The following key terms were used in the search: dating violence, prevention, youth, selected intervention, universal program, rigor, control, comparison.
For their clinical and program improvement purposes, SafePlace collects its own data. This data is separate from the proposed evaluation and is not federally-funded. The data collection for this evaluation is new. While we are not planning to access any of the clinical data collected by SafePlace, if, in the future, there seems to be a potential benefit of including this information in the evaluation, we will submit a change request to access the clinical information.
A.5. Impact on Small Businesses or Other Small Entities
No small businesses will be involved in this data collection.
The present study will provide the primary data needed for local, state, and federal policy makers to assess the effectiveness of the ERSG program on dating violence among adolescents at baseline, completion of the study (8 months) and at one follow up time (12 to 15 months after baseline). Less frequent effectiveness evaluation data collection would not allow for measurement of pre-, post-, and follow-up student characteristics and behaviors. One of the limitations of the pilot evaluation was lack of follow-up data, as the goal of ERSG is to have a lasting effect on dating relationship behavior. Furthermore, if this evaluation were not conducted, it would not be possible to determine the effectiveness of the program or its value and impact on the lives of the people it is intended to serve. Failure to collect these data or less frequent data collection could preclude effective use of school resources to benefit students.
There are no legal obstacles to reduce the burden.
This request fully complies with the regulation 5 CFR 1320.5.
A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency
A 60-day notice to solicit public comments regarding this study was published in the Federal Register (volume 78, No. 60, pages 18985 and 18986) on March 28, 2013. Attachment B contains a copy of the notice. There was one public comment received in response to the 60-day Federal Register Notice (Attachment M). The public comment generally referenced the wasteful spending of federal tax dollars, but did not specifically comment on burden hours or the cost of this evaluation. The standard CDC response to public comments was sent to the commenter.
As mentioned above, Expect Respect was selected as a participant of the empowerment evaluation process through a separate process at CDC that involved an environmental scan of existing programs, a competitive application process, and expert consultation. As a result of the empowerment process, due to its progress independently conducting program evaluation and programs improvement and because of the unique gap ERSG filled in the field, as a targeted prevention strategy, ERSG developers and implementers met with internal CDC investigators and subject matter experts, on June 15, 2009. This panel relied on the substantial work and documentation of this work (see January 2009 special issue of Health Promotion Practice) in the empowerment evaluation to select the experts and consult on logical next steps for evaluating promising programs. The panel consisted of two experts in the areas of teen dating violence and at-risk youth who convened to provide guidance in addressing the research question. Following the meeting, an additional member (Dr. Amy Cuellar) was consulted regarding human subjects issues. Members of this panel were as follows:
Table 1. Guidance Panel
Ball, Barbara |
Start Strong Austin Project Director Evaluation Specialist Expect Respect Program SafePlace Austin, Texas |
|
Rosenbluth, Barri |
512-356-1628 |
Expect Respect Program Director SafePlace |
Cuellar, Amy |
713-794-1414 |
Menninger Department of Psychiatry and Behavioral Sciences Baylor College of Medicine |
The panel concluded that a controlled evaluation was warranted to explore program effects, but that a randomized control trial (RCT) was not yet warranted given the mixed effects of the preliminary program evaluation and the current implementation of ERSG in many schools in AISD. Therefore, it was decided that all intervention schools would be selected from AISD and all matched control schools would be selected from neighboring districts carefully matched based on demographic characteristics.
For this study, the following CDC staff have been actively involved in developing the procedures and revising the questionnaires:
Andra Tharp, Behavioral Scientist, ([email protected]), phone 770-488-3936
Greta Massetti, Branch Chief, ([email protected]), phone 770-488-3943
Kristin Holland, Project Officer, ([email protected]), phone 770-488-3954
A.9. Explanation of Any Payment or Gift to Respondents.
School incentives. Experience has taught us that schools serving in a control condition of a study involving sensitive topics such as youth sexual and dating violence can be reluctant to burden their staff in light of the multiple demands on schools and staff over and above the multiple educational demands. Participating as a control school in the proposed evaluation will include meeting with the contractor to discuss the study, allowing the contractor to meet with school personnel who will be making referrals to the study, administrative time to work with the data collectors to locate participants for survey completion, and use of space in a private location for survey completion. A school’s participation will also be affected by the sensitive nature of the evaluation and survey questions. Our experience in other CDC-funded school based evaluations of sexual and teen dating violence prevention (e.g., Green Dot in Kentucky) indicates that although schools are often aware the their students deal with these issues, a decision to participate in a data collection involving these topics may be influenced by political and ideological views of sexuality and youth relationships.
All of the activities named above will be conducted by the school, although the control school, unlike the intervention school, will not receive the services of a targeted prevention program. Moreover, due to logistical limitations and recruiting schools in less than two months within the three neighboring districts that most closely match AISD in student demographics and other variables (described in detail in matching section), only 15 potential control schools exist from which 10 schools’ participation is needed for sufficient power. Therefore, to encourage cooperation and offset staff time and effort and acknowledge the sensitive and potentially politically challenging nature of the survey and evaluation, each control school will receive $300 for each year of their participation as a study site ($900 possible). The control school incentives are intended to recognize and offset the time and resource burden placed on teachers and school administrators and to convey appreciation for their contribution to this important, albeit sensitive study, and increase participation from a small sample of schools, as the evaluation would be impossible without their participation. The amount of the incentive was based on past work and based on SafePlace’s experience working with schools.
Intervention schools will not receive an incentive, since the intervention schools will receive ERSG, which is a social service to the schools, over and above what they offer. These services could be perceived as a benefit to the school, so in return for participating in the study, no monetary incentives will be provided to them. Furthermore, AISD currently has a contract with SafePlace, to provide these services in the schools, so it could be contractually problematic for SafePlace to receive funding to provide the services in AISD schools and then incentivize the AISD schools that will be participating in the evaluation.
We expect that 66% (10 of 15) of the control schools we contact will agree to participate in the study, partly as a result of the incentive offered. Table 2 from one of CDC’s former contractors, Research Triangle, Inc. (RTI) shows several studies where equivalent school incentives were employed. Although these studies differ in other respects that could account for some variability in response rates, overall, school incentives of $1,000 to $1,500 per year were generally associated with higher participation rates, than would be expected without an incentive (10%, CDC OMB Control No. 0920-0783). Our proposed incentive of $300 per year for up to 3 years is lower than that employed in these past studies, but we recognize that the qualities of the proposed evaluation differ from these trials (e.g., non-randomized). That said, the current evaluation may also be more sensitive than these past studies as we include detailed questions about sexual violence and have obtained a waiver of parental consent for study participation.
Table 2. RTI Studies Involving School Incentives and Corresponding Response Rates
RTI Study |
School Incentive Provided |
Participation Rate Achieved |
Impact Evaluation of a School-based Violence Prevention Program (2004 – 2009) |
$1500 each year of participation (3 years) |
10% |
Middle School Coordinator Initiative (1999 – 2004) |
$1500 each year of participation for completion of student surveys (3 years) |
33% |
Program for International Student Assessment (PISA) (2004 – 2009) |
Conference for schools that enrolled in study (value of $1500) or check for $1500 (1 year) |
70% |
A.10. Assurance of Confidentiality Provided to Respondents.
This submission has been reviewed by ICRO, and it was determined that the Privacy Act does apply. All procedures have been developed, in accordance with federal, state, and local guidelines, to ensure that the rights and privacy of school principals, teachers, prevention coordinators, and students are protected and that the relationships between students and school staff will be protected and maintained. in an effort to maintain facilitator privacy, all identifying information will be removed from facilitator interview transcripts prior to sharing transcripts with SafePlace staff. In addition,.
All respondents will be assured that the information they provide will be treated in a secure manner and will be used only for the purpose of this research, unless otherwise compelled by law. Copies of assent forms provided in writing to students and scripts to be read to students are provided in Attachment G. All students will be informed that what students report in the surveys will not have any effect on their academic performance in the school, relationship with the school, or the services they receive at the school. Trained survey administrators will assure students that student names will not be associated with responses provided. Moreover, survey administrators will receive extensive training in administering the surveys in a way that will not influence students responses and will maintain student privacy. For example, standardized responses to questions about the survey will be developed so that the administrators’ responses are consistent across schools, students, and administrators. For example, past experience tells us that some students have reading abilities not commensurate with their grade level, so we anticipate that some students may not know the meanings of certain words, and standardized methods will ensure administrators provided clarification assistance in the same way. Respondents will be told that the information obtained from all of the surveys will be combined into a summary report so that details of individual questionnaires cannot be linked to a specific participant. To ensure privacy, students will not use any identifiable information on survey responses. Instead, all students will receive a random identification number. Once the student completes the survey, only the survey administrator and data manager will be able to link a student’s response to his/her name, and will do so only for data collection and management purposes.
In order to complete the surveys, students will be pulled out of class during the school day. The procedures used to get the students from class will minimize the chance of negatively affecting their academic responsibilities and will not indicate why the student is being drawn away from the classroom. This privacy will keep other students from knowing that the student is participating in an evaluation study and the procedures are consistent with those used in other areas of medical and mental health treatment when the student receives treatment during the school day. These procedures will be the same for the control and implementation schools.
The effectiveness survey does not specifically ask about danger of immediate or potential physical harm. However, mandatory reporting requirements will apply to the interactions between the data collectors and students, such that data collectors must report any abuse of a child of which they are aware. If the student reports to a data collector an event of abuse that would constitute a mandatory report, in accordance with the Texas Family Code, data collectors will make a report if students report experiencing child abuse (mental/emotional injury, physical injury, sexual conduct harmful to a child, or neglect). Harmful sexual contact is defined as any sexual contact between a child and adult or a child 14 or older who has contact with a person 3 or more years older. If the reported abuse happened within the family or involves a caretaker's failure to protect the child, a report will be made orally or online to the statewide DPRS hotline number 1-800-252-5400. If the reported abuse occurred outside the family or did not occur as a result of a caretaker's failure to protect the child, a report will be made orally to the local law enforcement agency (either the Austin Police Department, Child Abuse Division, or the Travis County Sheriff's Department.) If students report during the interview that they are perpetrating emotional, physical or sexual abuse of another child, that information will be reported to DPRS or the Austin Police Department, respectively the Travis County Sheriff’s Department. The Expect Respect policy for mandatory reporting is included in Attachment H. This exception to privacy will be fully described to students through informed assent forms and an assent script read to students (script has been uploaded to ROCIS) and is included in Attachment G).
Because the survey contains sensitive questions about student physical and sexual dating violence perpetration and victimization, students will be reminded during the assent process that their answers will not be shared with anyone outside the research project team, including their parent or guardian, unless required by law. Trained survey administrators will let students know before the survey that any disclosure of potential harm to the student or others will result in a report to the proper authorities, as well as the parents of the student. Furthermore, students will not use any identifiable information on survey responses. Instead, all students will receive a random identification number. Only the survey administrator and data manager will be able to link a student’s response to his/her name, and will do so only for data collection and management purposes.
It is possible that another student could view survey responses while survey administration is in progress, so students will be spaced out around the room to avoid the possibility of another student being able to view survey responses. After completion of the survey, students will place questionnaires in an envelope provided by Expect Respect evaluators. The evaluation field staff will seal the envelope, and it will not be unsealed until it is received by the data manager. School staff will not have access to any survey information provided by individual students.
To ensure data security, all SafePlace project staff are required to adhere to strict standards and to sign privacy agreements as a condition of employment on this project. Effectiveness evaluation survey administrators will be thoroughly educated in methods of maximizing student understanding of the government’s commitment to privacy.
IRB Approval
The CDC IRB (Protocol # 5937) has reviewed all instruments, informed consent materials, and procedures to ensure that the rights of individuals participating in the study are safeguarded. A copy of the IRB approval notice is included as Attachment F
A.10.1 Privacy Impact Assessment Information
This project is subject to the Privacy Act. The applicable System of Records Notice (SORN) is 09-20-0160, “Records of Subjects in Health Promotion and Education Studies”.
Students will be assured that participation in the proposed evaluation is voluntary. It will be explained that the decision to release de-identified survey responses for this research is voluntary, and that the decision to assent to the evaluation in no way affects his/her ability to participate in ERSG or other prevention services. Students will also be informed that they may rescind their assent to release their data for any reason at any time. Students will be informed that the data they provide will be used to help investigators understand teens and plan programs to keep young people safe and healthy. Students will be informed that only certain SafePlace staff will have access to their identifiable data and that it will be securely stored and will not be able to be accessed by unauthorized persons. The OMB approval number will be listed on the final student assent form as evidence of legal authority for collecting data.
Respondent assent will be obtained prior to data collection. The student assent form is attached (Attachment G). Students will be informed about the intended use of the information by an ERSG data collector before the baseline survey is administered. Data collectors will inform students that, should they decide to participate in this study, they will be asked to complete surveys about their relationships and related feelings and attitudes. Students will be informed of the limits of privacy and that they will be asked to complete 4 surveys – one during the first ERSG session, one at completion of the ERSG sessions (8 months), and one at a follow up time (12 months after baseline). Students will be informed that they will receive no direct benefit by participating in the study, but that their participation will help the investigators better understand teens and plan programs to keep young people safe and healthy. Students will be informed that they should not write their name on the surveys and that all of the information they write will be kept private as allowed by law. Students will also be informed that only certain people working on this project will be allowed to look at the surveys and that the participants’ names will not be used in any reports as a result of this data collection.
Data that are collected will be stored at the Resource Center at SafePlace (1515A Grove Blvd., Austin, TX 78741). Data will be double locked in a file cabinet with a locking mechanism and in an office with a locked door. Security will be managed by assigning a random code to students (by Safe Place co-investigators or their staff), by SafePlace personnel not releasing any information that could link a students' number to his/her data, by storing personally identifying information in a locked cabinet, by using random subject numbers and not personally identifiable information in databases, and by emailing only de-identified datasets. CDC faculty will never be able to link the participants' data to their identity. Only select SafePlace personnel will have access to the master file that links the survey codes to participants’ names.
Individual identifying information will be kept separate from survey responses, and ID numbers will be assigned to participants for identification purposes. Expect Respect evaluation staff will never leave completed consent/assent forms or questionnaires unattended. No respondent identifiers will be contained in public use data files made available from the study, and no data will be released in a form that identifies individual respondents.
Some questions included in survey instruments might be considered sensitive by some respondents. The surveys include questions on sensitive issues such as emotional abuse, physical violence, and sexual violence. Table 3 identifies the sensitive questions, explains the justification for their inclusion in the surveys, and describes how the data will be used. The informed assent protocol apprises students that these topics will be covered during the surveys. To minimize the risk that respondents should be upset by these questions, staff will be made aware of the sensitive nature of the questions during training and will be trained to respond empathetically. In fact, all survey administrations will have training in counseling. If a respondent shows any signs of being upset or requests additional help, the administrator will refer him/her to appropriate local community and school-based mental health services, although we anticipate that the administrators will also have adequate training to manage mild student upset. These questions are included in the surveys because of their importance in understanding changes in dating violence behaviors among students. All sensitive questions have been used previously in internal program evaluation processes and are adapted from validated assessment tools (e.g., CADRI, Wolfe et al., 2001). As with all information collected, these data will be presented with all identifiers removed, including school and school district identifiers.
Table 3.
Description of Questions |
Justification for Inclusion |
Use of Data |
Student peer and/or dating violence victimization or perpetration, including psychological abuse, stalking, cyberbullying, physical violence, or sexual violence |
Necessary to determine effects of the Expect Respect support groups in preventing or reducing adolescent dating violence |
Used as dependent variable for multivariate analysis comparing students at intervention and control schools |
Attitudes related to peer and/or dating violence, including psychological abuse, physical violence, or sexual violence |
Necessary to determine whether changes in attitudes explain effects of the Expect Respect support groups on violent behaviors among dating partners |
Used as mediating variable for multivariate analysis to assess attitude changes as the pathway of program effects on adolescent dating violence behaviors |
Relationship characteristics and dating behaviors |
Necessary to determine the context of dating violence incidents and whether students engaging in risky dating behaviors are equally or less likely to benefit from the Expect Respect support groups than those not engaging in risky dating behaviors |
Used as a moderating variable for multivariate analysis to assess interaction between exposure to Expect Respect support groups and risky dating behaviors as a significant predictor of adolescent dating violence |
Table A-12 details the annualized number of respondents, the average response burden per survey, and the total response burden for the intake assessment, baseline questionnaire, completion questionnaire, and follow-up questionnaire. The intake assessment takes approximately 15 minutes to complete, and each data collection session (baseline survey, completion survey, follow-up survey) takes approximately 1 hour. Based on a past, non-CDC funded program evaluation of Expect Respect Support groups, we anticipate that in the Austin Independent School District and neighboring control districts, 800 students will undergo an intake assessment, of whom 600 will be eligible for Expect Respect Support groups and will complete the baseline survey. We expect 400 students to complete the survey and follow-up assessment. Therefore, over three years 2400 students will undergo an intake assessment, of whom we will recruit 1800 students into the study (300 per year from intervention schools and 300 per year from control schools), of whom we anticipate 1200 will have complete data. For facilitator fidelity monitoring data collection, data will be collected twice a year from 8 facilitators. Completion of the fidelity monitoring survey takes approximately 15 minutes. The lead facilitator will complete 16 observational measures over the course of the year (2 per ERSG facilitator). These measures take approximately 15 minutes to complete. Finally, 16 interviews (8 mid-year interviews, and 8 end-of-year interviews) will be conducted by CDC staff each year. The mid-year interview lasts approximately 45 minutes, and the end-of-year interview is approximately 60 minutes.
Table A.12- Estimate of Annual Burden Hours.
Type of Respondent |
Form Name |
No. of Respondents |
No. of Responses per Respondent |
Response Burden (hours) |
Total Burden Hours |
Intervention and Control Schools |
Intake assessment (Attachment C) |
800 |
1 |
15/60 |
200 |
Baseline Survey(Attachment D) |
600 |
1 |
1 |
600 |
|
Completion Survey (Attachment E) |
400 |
1 |
1 |
400 |
|
Follow-up Survey (12 month) (Attachment E) |
400 |
1 |
1 |
400 |
|
ERSG Facilitator |
ERSG Facilitator Program Implementation Fidelity Measure (Attachment I) |
8 |
2 |
15/60 |
4 |
ERSG Facilitator Supervisor |
ERSG Observational Program Implementation Fidelity Measure (Attachment J) |
1 |
16 |
15/60 |
4 |
ERSG Facilitator |
Mid-Year Qualitative Interview with ERSG Facilitators (Attachment K) |
8 |
1 |
45/60 |
6 |
ERSG Facilitator |
End of Year Qualitative Interview with ERSG Facilitators (Attachment L) |
8 |
1 |
1 |
8 |
Total |
|
1622 |
|||
A.12.B. Estimated Annualized Burden Cost
The hourly wage used to calculate the Respondent Cost is $7.25, which is the minimum wage under the Fair Labor Standards Act (FLSA). The average salary of Expect Respect Support Group facilitators is $22.76 per hour. This estimate was used to calculate the cost burden associated with the fidelity surveys and mid- and end-of-year facilitator interviews. Total Respondent Cost for this evaluation is $12,100.72.
Type of Respondent |
Form Name |
No. of Respondents |
No. of Responses per Respondent |
Response Burden (hours) |
Hourly Wage Cost |
Respondent Cost |
Control Schools (School districts surrounding Austin Independent School District) |
Intake assessment (Attachment C) |
400 |
1 |
15/60 |
$7.25 |
$725 |
Baseline Survey (Attachment D) |
300 |
1 |
1 |
$7.25 |
$2,175 |
|
Completion Survey (Attachment E) |
200 |
1 |
1 |
$7.25 |
$1,450 |
|
Follow-up Survey (12 month) (Attachment E) |
200 |
1 |
1 |
$7.25 |
$1,450 |
|
Intervention Schools (Austin Independent School District) |
Intake assessment |
400 |
1 |
15/60 |
$7.25 |
$725 |
Baseline Survey |
300 |
1 |
1 |
$7.25 |
$2,175 |
|
Completion Survey |
200 |
1 |
1 |
$7.25 |
$1,450 |
|
Follow-up Survey (12 month) |
200 |
1 |
1 |
$7.25 |
$1,450 |
|
ERSG Facilitators
|
ERSG Facilitator Program Implementation Fidelity Measure (Attachment I) |
8 |
2 |
15/60 |
$22.76 |
$91.04 |
ERSG Facilitator Supervisor |
ERSG Observational Program Implementation Fidelity Measure (Attachment J) |
1 |
16 |
15/60 |
$22.76 |
$91.04 |
Facilitators
|
Mid-Year Qualitative Interview with ERSG Facilitators (Attachment K) |
8 |
1 |
45/60 |
$22.76 |
$136.56 |
Facilitators |
End of Year Qualitative Interview with ERSG Facilitators (Attachment L) |
8 |
1 |
1 |
$22.76 |
$182.08 |
Total |
|
|
$12,100.72 |
|||
A.13. Estimates of Other Total Annual Cost Burden to Respondents or Recordkeepers.
Respondents will incur no capital or maintenance costs.
A.14. Estimates of Annualized Cost to the Federal Government.
Two types of government costs will be incurred: (1) government personnel, and (2) contracted data collection.
- NCIPC has assigned a Project Officer and Science Officer to assist with and oversee this data collection. Each of these personnel is assigned for 10 percent time for the duration of the contract. Based on an annual salary of $80,000, this equates to $16,000 for each year for cost of government personnel ($80,000 x 10% effort x 2 employees = $16,000).
- An anticipated budget for years 1-3 of the ERSG Program Evaluation contract is below:
Year 1
I. |
Personnel |
||
|
Project Director |
$36.06/hr x 1560 person-hours |
56,253.60 |
|
Support Group Data collectors (Field Staff) |
$21.63/hr x 1040 person-hours x 5 data collectors |
112,476.00 |
|
Data Entry and Administrative Support |
$21.63/hr x 1040 person-hours |
22,495.20 |
|
Clinical Supervisor |
$36.06/hr x 416 person hours |
15,000.96 |
|
Subtotal Personnel |
|
206,225.76 |
|
Fringe |
30% |
61,867.73 |
|
Total Personnel |
|
$268,093.49 |
|
|
|
|
II |
Equipment, supplies, and other |
|
|
|
Scantron equipment |
$4,000 for one scantron machine |
4,000 |
|
Computer purchase |
$4,000 for one computer purchase |
4,000 |
|
Computer/data analysis software |
$3,000 |
3,000 |
|
Survey printing fees |
3,000 |
3,000 |
|
School incentives |
$300 incentive per school for up to 8 schools |
2,400 |
|
Total |
|
$16,400 |
|
|
|
|
IV |
Travel |
|
|
|
Visit to Atlanta CDC |
1 visit – 2 people, 2 days = $3,000 |
3,000 |
|
Conference attendance |
1 conference – 2 people, 2 days = $3,000 |
3,000 |
|
Total |
|
$6,000 |
|
|
|
|
|
|
Total Year 1 Funding |
$290,493.49 |
Year 2
I. |
Personnel |
||
|
Project Director |
$36.06/hr x 1560 person-hours |
56,253.60 |
|
Support Group Data collectors (Field Staff) |
$21.63/hr x 1040 person-hours x 2 data collectors |
44,990.40 |
|
Data Entry and Administrative Support |
$21.63/hr x 1040 person-hours |
22,495.20 |
|
Subtotal Personnel |
|
123,739.20 |
|
Fringe |
30% |
37,121.76 |
|
Total Personnel |
|
$160,860.96 |
|
|
|
|
II |
Equipment, supplies, and other |
|
|
|
Computer/data analysis software updates |
|
2,000 |
|
School incentives |
$300 incentive per school for up to 8 schools |
2,400 |
|
Total |
|
$4,400 |
|
|
|
|
IV |
Travel |
|
|
|
Visit to Atlanta CDC |
1 visit – 2 people, 2 days = $3,000 |
3,000 |
|
Conference attendance |
1 conference – 2 people, 2 days = $3,000 |
3,000 |
|
Total |
|
$6,000 |
|
|
|
|
|
|
Total Year 2 Funding |
$171,260.96 |
Year 3
I. |
Personnel |
||
|
Project Director |
$36.06/hr x 1560 person-hours |
56,253.60 |
|
Support Group Data collectors (Field Staff) |
$21.63/hr x 1040 person-hours x 2 data collectors |
44,990.40 |
|
Data Entry and Administrative Support |
$21.63/hr x 1040 person-hours |
22,495.20 |
|
Subtotal Personnel |
|
123,739.20 |
|
Fringe |
30% |
37,121.76 |
|
Total Personnel |
|
$160,860.96 |
|
|
|
|
II |
Equipment, supplies, and other |
|
|
|
Computer/data analysis software updates |
|
2,000 |
|
School incentives |
$300 incentive per school for up to 8 schools |
2,400 |
|
Total Equipment |
|
$4,400 |
|
|
|
|
IV |
Travel |
|
|
|
Visit to Atlanta CDC |
1 visit – 2 people, 2 days = $3,000 |
3,000 |
|
Conference attendance |
1 conference – 2 people, 2 days = $3,000 |
3,000 |
|
Total |
|
$6,000 |
|
|
|
|
|
|
Total Year 3 Funding |
$171,260.96 |
The average annualized direct costs for this project are $227,005. This averaged amount includes all costs for the contracted data collection, plus the personnel costs of federal employees involved in oversight and analysis.
This is an extension with no new changes. The burden has not changed from the burden shown in the current inventory.
A.16. Plans for Tabulation and Publication and Project Time Schedule.
Data analysis will focus primarily on assessing the overall program effectiveness.
Data
analysis for our study aims (examine group differences between
control and intervention students, evaluate whether or not the
effectiveness of ERSG is enhanced by the presence of a universal,
school-wide prevention programs, and examine moderators and mediators
of targeted and universal teen dating violence interventions) will
involve: 1) descriptive analyses of group characteristics to examine
prevalence and incidence of dating violence and chi-squared tests of
independence with odds ratios to examine differences in prevalence of
dating violence for control and intervention students. Study
hypotheses will be examined in several ways depending on the unit of
analysis (e.g., student, school, school district) and the group of
interest (Aim 1: control vs. intervention; Exploratory Aim: combined
vs. universal only): 1) Parametric tests such as logistic regression
will be used to examine group differences over time, 2) Nonparametric
tests such as Wilcoxon rank sum test will be used to examine group
differences on non-normal outcomes (e.g., sexual violence), and 3)
structural equation modeling (panel and growth curve models) will be
used to test for differences between intervention and control groups
over time, as well as to test the effect of moderator and mediator
variables on study outcomes.
For logistic regressions and
structural equation models we will take into account baseline
differences on demographic characteristics, mediators, and dating
violence outcomes for students who are lost to follow-up between
baseline and follow-up. To determine non-response bias, variables
found to differ between follow-up survey responders and nonresponders
will be included as covariate in multivariate analyses of program
effectiveness. While the unit of analysis for the aims is the school,
some analyses, such as missing data analysis and non-response bias
analyses will be conducted at the student level to determine the
equivalence of school units prior to conducting cluster analyses.
Publications
The results of the analysis will be reported in a Data Summary. In conjunction with the evaluator we will also publish an Evaluation Report, including a 1-page press release, a 2- to 3-page executive summary written in clear language and understandable by a wide range of audiences (parents, practitioners, policy makers, researchers), a 10-page executive summary, a report of less than 100 pages (including an overview of background literature to provide contextual information about the purpose of the ERSG program and evaluation approach, a detailed summary of evaluation methods and activities, the evaluation results, discussion of findings in comparison with those of other relevant program evaluations, strengths and limitations of the evaluation, and recommendations for future evaluations of this scope for practitioners, evaluators, and policy makers), and appendices. The report will also identify challenges encountered during program implementation and evaluation, as well as their solutions. The results of our study also will be used to develop peer-reviewed journal articles (e.g., American Journal of Public Health, Journal of Adolescent Health, and/or Prevention Science), conference presentations, research briefs, and Web-based papers for dissemination to researchers, schools, and the public.
Table 16-1. Time Schedule
Activity |
Time schedule |
|
Began in September 2010 immediately following OMB approval. Continued as follows:
August 2011 – February 2012 August 2012 – February 2013 August 2013 – February 2014 (or immediately upon OMB extension approval) |
|
*To take place within two weeks of participant recruitment
August 2011 – February 2012 August 2012 – February 2013 August 2013 – February 2014 (or immediately upon OMB extension approval) |
|
Sessions 1-24 will begin 2-4 weeks after OMB approval and continue once a week for up to 8 months during each year of data collection
September 2011-May 2012 September 2012-May 2013 September 2013 – May 2013 (or 2-4 weeks after OMB extension approval) |
|
12 months after baseline survey
September 2012-February 2013 September 2013- February 2014 (or immediately upon OMB extension approval) September 2014 – February 2015 |
|
Continuous activity throughout data collection |
|
12-24 months after OMB extension approval |
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate
The display of the OMB expiration date is not inappropriate
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions.
There are no exceptions to the certification
References
Avery-Leaf, S., Cascardi, M., O’Leary, K.D., & Cano, A. (1997). Efficacy of a dating violence prevention program on attitudes justifying aggression. Journal of Adolescent Health, 21, 11-17.
Ball, B., Kerig, K., & Rosenbluth, B. (2009). “Like a family but better because you can actually trust each other”: The Expect Respect dating violence prevention program for at-risk youth. Health Promotion Practice , 10(1), 45S-58S.
Ball, B., Rosenbluth, B., & Aoki, A. (2008). The Expect Respect Program Manual Part I: Support Group Curriculum and Facilitator Guide. Austin, TX: SafePlace.
Burcky, W., Reuterman, N., & Kopsky, S. (1988). Dating violence among high school students. The School Counselor, 35, 353-358.
Capaldi, D. M., Dishion, T. J., Stoolmiller, M., & Yoerger, K. (2001). Aggression toward female partners by at-risk young men: The contribution of male adolescent friendships. Developmental Psychology, 37, 61-73.
Cornelius, T., & Resseguie, N. (2007). Primary and secondary prevention programs for dating violence: A review of the literature. Aggression and Violent Behavior, 12, 364-375.
Dobash, R. P., Dobash, R. E., Wilson, M., & Daly, M. (1992). The myth of sexual symmetry in marital violence. Social Problems, 39, 71-91.
Eaton, D. K., Davis, K. S., Barrios, L., Brener, N. D., & Noonan, R. K. (2007). Associations of dating violence victimization with lifetime participation, co-occurance, and early initiation of risk behaviors among U.S. high school students. Interpersonal Violence, 22, 585-602.
Foshee, V. (1996). Gender differences in adolescent dating abuse prevalence, types and injuries. Health Education Research, 11, 275-286.
Foshee, V., Bauman, K., Arriaga, X., Helms, R., Koch, G., & Linder, G. (1998). An evaluation of SafeDates, an adolescent dating violence prevention program. American Journal of Public Health, 88, 45-50.
Foshee VA, & Matthew RA. (2007). Adolescent dating abuse perpetration: A review of findings, methological limitations, and suggestions for future research. In DJ Flannery, AT Vazjoni, & ID Waldman (Eds.), The Cambridge Handbook of Violence Behavior and Aggression (pp. 431-449). New York: Cambridge.
Hickman, L. J., Jaycox, L. H., & Aronoff, J. (2004). Dating violence among adolescents: Prevalence, gender distribution, and prevention program effectiveness. Trauma, Violence, & Abuse, 5, 123-142.
Jaffe, P., Sudermann, M., Reitzel, D., & Killip, S. (1992). An evaluation of a secondary school primary prevention program on violence in intimate relationships. Violence and Victims, 7, 129-146.
Malik, S., Sorenson, S., & Anehensel, S. (1997). Community and dating violence among adolescents: Perpetration and victimization. Journal of Adolescent Health, 21, 291 – 302.
Molidor, C., Tolman, R., & Kober, J. (2000). Gender and contextual factors in adolescent dating violence. The Prevention Researcher, 7(1), 1-4.
Noonan, R. & Gibbs, D. (2009). Empowerment evaluation with programs designed to prevent first-time male perpetration of sexual violence. Health Promotion Practice, 10, 5s-10s.
O’Keefe, M. (1997). Predictors of dating violence among high school students. Journal of Interpersonal Violence, 12, 546-568.
O’Keefe, M. (1998). Factors mediating the link between witnessing interparental violence and dating violence. Journal of Family Violence, 13(1), 39-57.
Pepler, D. J., Craig, W. M., Connolly, J. A., Yuile, A., McMaster, L., & Jian, D. (2006). Aggressive Behavior, 32, 376-384.
Schewe, P. (2002). Guidelines for developing rape prevention and risk reduction interventions: Lessons from evaluation research. In P. Schewe (Ed.), Preventing violence in relationships: Interventions across the life span (pp. 107-136). Washington, DC: American Psychological Association.
Sears, H. A., Byers, S., Whelan, J. J., & Saint-Pierre, M. (2006). “If it hurts you, then it is not a joke”: Adolescents’ ideas about girls’ and boys’ use and experience of abusive behavior in dating relationships. Journal of Interpersonal Violence, 21, 1191-1207.
Silverman, J., Rai, A., Mucci, L., & Hathaway, J. (2001). Dating violence against adolescent girls and associated substance abuse, unhealthy weight control, sexual risk behavior, pregnancy, and suicidality. Journal of the American Medical Association. 286, 572-579.
Teen Research Unlimited. (July 2008). Study on teen dating abuse. Liz Claiborne Inc. Retrieved April 14, 2009 from: http://www.loveisnotabuse.com/statistics.htm.
Teten, A. L., Ball, B, Valle, L. A., Noonan, R, & Rosenbluth, B. (2009). Report from CDC: Considerations for the definition, measurement, consequences, and prevention of dating violence victimization among adolescent girls. Journal of Women’s Health, 18, 923-927.
Whitaker, D., Morrison, S., Lindquist, C., Hawkins, S., O’Neil, J. A., Nesius, A. M., Mathew, A., & Reese, L. (2006). A critical review of interventions for the primary prevention fo perpetration of partner violence. Aggression and Violent Behavior, 11, 151-166.
Williams, T., S., Conolly, J., Pepler, D., Craig, W., & Laporte, L. (2008). Risk models of dating aggression across different adolescent relationships: A developmental psychopathology approach. Consulting and Clinical Psychology, 76, 622-632.
Wolfe, D.A., Crooks, C.V., Jaffe, P., Chiodo, D., Hughes, R., Ellis, W., Stitt, L., & Donner, A. (2009). A school-based program to prevent adolescent dating violence: A cluster randomized trial. Archives of Pediatric and Adolescent Medicine, 163: 692 - 699.
Wolfe, D. A., Scott, K., Reitzel-Jaffe, D., Wekerle, C., Grasley, C., & Straatman, A. L. (2001). Development and validation of the Conflict in Adolescent Dating Relationships Inventory. Psychological Assessment, 13, 277-293.
Wolfe, D., Wekerle, C., Scott, K., Straatman, A., Grasley, C., & Jaffe, D., (2003). Dating violence prevention with at-risk youth: A controlled outcome evaluation. Journal of Consulting and Clinical Psychology, 71, 279-291.
Wolfe, D. A., Wekerle, C., Reitzel-Jaffe, D., & Lefebvre, L. (1998). Factors associated with abusive relationships among maltreated and nonmaltreated youth. Development and Psychopathology, 10, 61-85.
Wolfe, D. A., Wekerle, C., Scott, K., & Pittman, A. (2001). Child maltreatment: Risk of adjustment problems and dating violence in adolescence. Journal of the American Academy of Child and Adolescent Psychiatry, 40(3), 282-289.
Wolfe, D. A., Wekerle, C., Scott, K., Straatman, A., & Grasley, C. (2004). Predicting abuse in adolescent dating relationships over 1 year: The role of child maltreatment and trauma. Journal of Abnormal Psychology, 113, 406-415.
Wolitzky-Taylor, K. B., Ruggiero, K. J., Danielson, C. K., Resnik, H. S., Hanson, R. F., Smith, D. W., Saunders, B. E., & Kipatrick, D. G. (2008). Prevalence and correlated of dating violence in a national sample of adolescents. American Academy of Child and Adolescent Psychiatry, 47, 755-762.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | SUPPORTING STATEMENT FOR |
| Author | Lois P. Voelker |
| File Modified | 0000-00-00 |
| File Created | 2021-01-29 |