HHS/CDC/NCIPC
SUPPORTING STATEMENT FOR
OMB INFORMATION COLLECTION REQUEST
Part B
A Controlled Evaluation of Expect Respect Support Groups (ERSG): Preventing and Interrupting Teen Dating Violence among At-Risk Middle and High School Students
Supported by:
Department of Health and Human Services
National Center for Injury Prevention and Control
Centers for Disease Control and Prevention
Division of Violence Prevention
Research and Evaluation Branch
(CDC/NCIPC/DVP/RE)
Andra Teten, Ph.D., Science Officer
[email protected]/ 770-488-3936
Kristin Holland, M.P.H., Project officer
[email protected] / 770-488-3954
National Center for Injury Prevention and Control
Division of Violence Prevention
Prevention Development and Evaluation Branch
4770 Buford Hwy, NE, MS F-64
Atlanta, GA 30341
Table of Contents
B. Collections of Information Employing statistical procedures
1. Respondent Universe and Sampling Methods
2. Procedures for the Collection of Information
3. Methods to Maximize Response Rates and Deal with Non-Response
4. Test of Procedures or Methods to be Undertaken
5. Individuals consulted on Statistical Aspects and Individuals collecting and/or analyzing information
B. Collections of Information Employing Statistical Methods
Participants receiving the ERSG will be recruited from the Austin Independent School District (ISD), as a convenience sample, as these schools have ongoing ERSG. There are 17 middle schools and 13 high schools in the Austin ISD, and we expect 24 of those schools to implement the ERSG program. For the control sample, ten schools of the existing 15 middle and high schools in three neighboring districts--(Del Valle Independent School District (ISD, which contains 3 schools), Manor ISD (which contains 4 schools), and Pfluggerville ISD (which contains 8 schools)--that are matched on the following characteristics to participating AISD schools will be selected to participate in the evaluation: ethnic composition, economic disadvantage, limited English proficiency, proportion of students with disciplinary placements, and proportion of at-risk students as defined by the Texas Education Agency). These matching characteristics were selected because sexual and teen dating violence perpetration and victimization, as well as the risk factors that ERSG addresses (e.g., child maltreatment) are overrepresented in groups with these characteristics. For example, students with behavior problems in areas other than dating relationship are also at risk for violence in dating relationships (Foshee & Matthews, 2007).Please note that matching will account for similarities of ERSG participants at the school level, but that the eligibility criteria for the ERSG administered through the screening process will ensure control and intervention participants are at comparable risk for sexual and teen dating violence.
We selected our sample size based on our power analysis, in which we desired > 80% chance of detecting significant differences and based on data collected through a previous uncontrolled, non-CDC funded program evaluation of ERSG, which provided estimates for the number of students we could realistically recruit and maintain in the study. With these two considerations, during each year of data collection (2010-11, 2011-12, and 2012-13 school years), we anticipate that in the Austin Independent School District and 3 neighboring districts, 800 students will provide student assent and undergo an intake assessment. Of those 800 students, 600 will be eligible for Expect Respect Support groups and will complete the baseline survey. However, based on the attrition estimates from the preliminary evaluation, we expect 400 of the 600 eligible students per year to have complete data at all time points. Therefore, over three years 2400 students will undergo an intake assessment, of whom we will recruit approximately 1800 students into the study (300 per year from intervention schools and 300 per year from control schools), of whom we anticipate approximately 1200 will have complete data.
Respondent |
Instrument |
Respondent Universe |
Expected Response Rate |
Expected Completed Instruments |
||
Year 1 |
Year 2 |
Year 3 |
|
|
||
Middle and High School Students |
Student assent |
800 |
800 |
800 |
90% |
2160 |
Intake assessment |
800 |
800 |
800 |
100% |
2400 |
|
Baseline survey |
600 |
600 |
600 |
95% |
1710 |
|
Completion survey |
400 |
400 |
400 |
95% |
1140 |
|
Follow-up survey 1 |
400 |
400 |
400 |
95% |
1140 |
|
Follow-up survey 2 |
400 |
400 |
400 |
95% |
1140 |
|
Sampling frame. Male and female adolescents age 11 to 17 in middle and high schools in the Austin and surrounding school districts. The following table provides our exclusion criteria and rationale:
Exclusion Criteria |
Rationale |
Non-English speaking |
Developing curriculum in multiple languages could be cost prohibitive, and services in English would not be appropriate for students who are not fluent given the group nature of the ERSG program |
Having never experienced (i.e., been a victim, perpetrator, or witness of) any form of violence |
This is an evaluation of a prevention approach targeted to youth at high risk of being involved in violent relationships, so the study sample should reflect individuals who would be eligible for the support groups, which require a history of exposure to violence |
Having participated in ERSG in past years |
Students who have already participated in ERSG will not be eligible to initiate participation in the evaluation. However, if a student chooses to participate for a second year (is still participating during follow-up) he/she would simply indicate this on his/her follow-up survey, but would not be excluded from follow-up data collection. |
Requiring a higher level of care than ERSG can provide (i.e. students in crisis who are experiencing acute emotional upset, suicidal or homicidal ideations) |
Students who require a higher level of care than ERSG can provide will be referred to an agency or professional who is best suited to deal with their crisis. It is generally accepted that an individual in crisis is not appropriate to participate in an evaluation |
Potentially eligible students in the intervention and control schools will be referred to the service provider for screening and participation in ERSG. It is anticipated that approximately 300 students will be enrolled in the ERSG per year for three years, with 200 of these providing complete data on all timepoints. Some students may join the ERSG program after their peers started and others may drop out before the intervention program concluded. There will be five (screening + four surveys) data collection occasions for the majority of each cohort of students during a one and a half year period. The first data collection occasion will be at ERSG eligibility screening, an intake assessment. The subsequent data collections will occur before the first ERSG session (baseline survey), last ERSG session (completion survey/8 months for control group), and two follow up sessions at 12 months and at 18 months. Students who have participated in ERSG in past years will not be eligible for completing the baseline survey. However, because students, in practice, may participate in ERSG for multiple years, if it is determined at follow-up that a participant is continuing to take part in ERSG, he/she will indicate this on the portion of the survey that assesses other prevention programs in which the student is participating.
All data collectors will be thoroughly screened and their interviewing abilities tested prior to their being employed by the research team. For this study, data collectors will receive approximately 8-12 hours of project‑specific training. The content of training will include: eligibility determination, recruitment, informed assent, enrollment, and survey administration. In addition, data collectors will also receive information on dating violence and its consequences and will be given a comprehensive set of questions and answers that will provide standardized responses to questions that respondents may ask. Moreover, data collectors will complete human subjects training and CDC will prepare data collection flow charts that will provide detailed instructions to the data collector and will ensure fidelity to standardized survey administration. CDC will conduct site visits, will hold bi-weekly or monthly conference calls with the contractors to provide oversight and discuss data collection procedures, and CDC in collaboration with the study consultant will provide consultation via email and conference calls to the contractor. As the CDC principal investigator on the CDC IRB protocol Dr. Andra Teten is responsible for overseeing the scientific and human subject’s integrity of the study. Data collectors hired by the contractor will be responsible for collecting the survey data, though only the data manager will have access to the responses. The data collectors will be employees of the contractor but will not be the school’s Expect Respect facilitator.
In
intervention schools, ERSG will be promoted in the staff orientation
and will be advertised in the school through posters, and school
newsletter. Students will be referred by school counselors/teachers
or by self-referral. When a student is referred to data collectors to
participate in the study, an intake session will be completed during
which the data collector determines the student's eligibility. At the
end of the intake session, students eligible for participation in the
ERSG intervention will be given information about ERSG and asked if
they are interested in participating. Participation in ERSG is
voluntary. If the student gives his/her assent to participate in the
program, he/she will then be told about the evaluation. It will be
explained that the decision to release the students’
de-identified survey responses for this study is voluntary, and that
the decision to participate in the study in no way affects his/her
ability to participate in ERSG or other prevention services.
Following this discussion, the data collector will obtain informed
assent from the student for participating in this study. Students
will also be informed that they may rescind their assent to release
their data for any reason at any time.
In control
schools, students will be told that they have an opportunity to talk
to data collectors about questions/problems in their peer and dating
relationships and to participate in a study. In control schools the
data collectors will conduct the intake interviews, but also will
administer the evaluation surveys for the control participants. The
study will be promoted in the staff orientation and will also be
advertised in the school through posters and the school newsletter.
Students will be referred to the study by school counselors or
teachers, or may self-refer. When a student is referred, a data
collector will determine the student’s eligibility for
participation in the control group. Students eligible for the
control condition will be given information about the evaluation and
asked if they are interested in completing surveys as part of the
control condition. If the student is interested, it will be explained
that the decision to complete the surveys and release their data is
voluntary, and that the decision to assent to the evaluation in no
way affects his/her ability to participate in other community-based
prevention programs or other services. In other words, a control
student is not prohibited from seeking services in the community
(though ERSG will not be offered), if he/she is in need of these
services.
Subjects participating in the ERSG intervention or in the control group must have a history of witnessing domestic violence, and/or a history of experiencing child abuse (emotional/mental, physical, sexual, neglect) and/or a history of being involved in physically, emotionally, or verbally abusive peer and/or dating relationships. Exclusion criteria for the evaluation are consistent with the exclusion criteria for ERSG: having never experienced (i.e., been a victim, perpetrator, or witness of) any form of violence or requiring a higher level of care than the ERSG can provide. In other words, students in crisis (e.g., acute emotional upset, suicidal or homicidal ideations) will not be eligible. Students in crisis (at either intervention or control schools) will be referred for mental health, social and/or legal services. Additionally, students would not be eligible to initiate the evaluation if they have already participated in ERSG.
Students
will receive educational materials about dating violence, and local
and online resources. Regardless of their decision to participate in
the evaluation, students in control schools who are in need of a
minimal intervention will be offered short-term individual
psycho-education at school (1-3 sessions). Although a waiver of
parental consent has been granted in the study, participants in both
study groups also will be given a parental information letter that
they may choose to give to their parents.
Students in the ERSG schools who are eligible to participate in the ERSG program will attend 24 support group sessions on a weekly basis. Students in the control group will go about their schedule as normal, not attending support group sessions.
Data collectors will administer the baseline survey, completion survey, and two follow-up surveys (at 12 and 18 months following completion of the study). Data collectors will collect follow-up data using one of three methods: 1) Contacting ERSG participant through a school official; 2) Contacting ERSG participant in person during school hours; or 3) For students who will be graduating, moving, or transitioning to high school from middle school, contacting the students via phone for example (if the student grants permission to do so).
Data collected from the four surveys will allow us to assess statistically significant differences between the treatment and control groups and will help us to evaluate the program’s effectiveness.
Statistical concerns
Even though there may still be a debate among statisticians on the utility of randomization in comparing different treatment effects, it is assumed that a randomized group design would be the design to proceed for assessing program effects of ERSG, especially if the program effect for some outcome is relatively small (e.g., sexual violence). The proposal appears to exhibit a gap between the proposed study design and a typical group-randomized trial for the evaluation of community-based intervention programs. However, the controlled evaluation is an appropriate design to further explore the mixed program effects found during the program evaluation and determine whether or not a randomized control trial (RCT) of ERSG is justified.
The assumption in assessing a mediation effect is that there exists a causal relationship among analysis variables. It is not clear if such causal assumption for modeling is satisfied.
Power estimation
The method of estimation was selected and power was calculated based on the following study characteristics:
Schools
In our design, the school is the unit of analysis.
To approximate the equivalence of the schools, they will be matched based on characteristics that are relevant to teen dating violence (e.g., socioeconomic status).
For the intervention group, based on the current implementation of ERSG, we will expect to recruit 24 of the 30 eligible schools in the Austin Independent School District.
For the control group, we will expect to recruit 10 of the 15 eligible schools in the three neighboring districts.
Participants
Based on the preliminary evaluation and SafePlace’s experience with schools:
We expect approximately 10 participants per year per intervention school to participate in the study for a total of 30 intervention participants per school over 3 years.
Due to the lower number of control schools and the variation in size among potential control schools, we want to allow for the possibility that up to 20 participants per year per control school may participate in the study for a total of 30-60 control participants per school over 3 years.
Change in violence
Based on the preliminary evaluation and published literature on teen dating violence, we expect the frequency of teen dating violence to differ by 10% between the intervention and control group by the end of the evaluation.
Intra-cluster correlation coefficients (ICCs)
We based our ICCs on those published by Siddiqui, Hedeker, Flay, and Huet (1996) from a school-based intervention.
Based on these characteristics, inequality tests for two proportions in a cluster-randomized design were chosen for the calculation of power. Although we are not employing a randomized design, this conservative method accounted for the equivalence of schools that we anticipate and the characteristics of our sample reflected the cluster characteristics of this method (e.g., equivalence of cluster size). Software PASS version 08.0.1 was used for the power estimation.
With an ICC of 0.01, our power is > 80% with complete data from approximately 920-1260 participants.
Estimated power would be higher if differences between proportions were bigger, cluster size increased, ICC were smaller, or missing data were imputed. Power would be less if the ICC is larger than we anticipated. For example, if the ICC is 0.05 power would be > 59%. Although steps will be taken to retain students in the control and intervention groups, we expect some attrition based on the preliminary evaluation (anticipated 1800 enrolled, 1200 completers), however since we will be using a nested design and the school and not the student is the unit of analysis, power estimates are relatively consistent with varying numbers of students participating in each school.
All data collectors will be thoroughly screened and their interviewing abilities tested prior to their being employed by the research team. For this study, data collectors will receive approximately 8-12 hours of project‑specific training. The content of training will include: eligibility determination, recruitment, informed assent, enrollment, and survey administration. In addition, data collectors will also receive information on dating violence and its consequences and will be given a comprehensive set of questions and answers that will provide standardized responses to questions that respondents may ask. Trainees will be observed and given feedback during simulations of the intake interview until they are able to perform each skill to the Principal Investigator’s satisfaction. Additional opportunities for practice and feedback will occur during the pretest of the study’s procedures. Moreover, data collectors will complete human subjects training and CDC will prepare data collection flow charts that will provide detailed instructions to the data collector and will ensure fidelity to standardized survey administration. CDC will conduct site visits, will hold bi-weekly or monthly conference calls with the contractors to provide oversight and discuss data collection procedures, and CDC in collaboration with the study consultant will provide consultation via email and conference calls to the contractor. As the CDC principal investigator on the CDC IRB protocol Dr. Andra Teten is responsible for overseeing the scientific and human subject’s integrity of the study. Data collectors hired by the contractor will be responsible for collecting the survey data, though only the data manager will have access to the responses. The data collectors will be employees of the contractor but will not be the school’s Expect Respect facilitator.
In ERSG, boys and girls meet in separate sex groups facilitated by same-sex group leaders (who are not the data collectors for the group participants), which is intended to increase the sense of comfort and safety for teens, to allow them to bond more quickly, and to explore their expectations for relationships.
The following procedures will be used to maximize cooperation and to achieve the desired high response rates:
Participants will be contacted by the data collectors via a school official for follow-up
Participants will be contacted during school hours by the data collectors
For participants who will be graduating, moving, or transitioning to high school from middle school, contacting the students via phone or other forms of communication (if the student grants permission to do so).
CDC and the data collectors will continue to use our best practices in developing relationships of trust and cooperation with schools and service providing organizations, as these relationships will impact our ability to engage selected students and schools for participation and to maintain their involvement throughout the study period.
All participating schools will sign a letter of agreement with the contractor.
Information about the study will be presented in posters and the school newsletter.
Focused training will be provided to data collectors on the issues surrounding decisions to participate for students. Students will be surveyed during school hours to maximize the possibility and continuity of participation. We will thoroughly address concerns about the confidential manner of the data and about the purposes of the study and uses of the data through careful training of all data collectors.
School-level incentives will be offered to control schools.
The data collectors will work with the schools to address concerns that may arise.
If attrition does occur, missing data and non-response bias will be examined and managed statistically as described SSA 16.A.
B.4. Test of Procedures or Methods to be Undertaken
In a pilot, uncontrolled evaluation of 26 ERSG (16 schools) conducted by Safe Place, significant increases in positive dating behaviors and significant declines in emotional and physical victimization and perpetration were found for students who reported a history of peer and dating violence (Ball, Teten, Noonan, Valle, Hamburger, & Rosenbluth, in preparation). Perpetration of sexual violence increased over time. The rise in this low base rate behavior (M = 0.03 to M = 0.13) was attributed to its measurement (2 items), increases in awareness due to intervention, and timing of the final assessment (e.g., students completed final assessments in session 23 of 24 and reported on their behavior over the previous 3 months, so intervention effects may not yet have been achieved). These limitations have been addressed by CDC’s revision of the data collection instrument which included expanding the assessment of sexual violence. Moreover, the current proposal will use a control group and follow-up assessment to clarify change in sexual and teen dating violence as a result of the intervention. Based on these preliminary results, the current study proposes a controlled evaluation of ERSG.
All instruments and procedures have been reviewed extensively by CDC. The following individuals have worked closely in developing the instrument and procedures that will be used, and will be responsible for data analysis:
Andra Teten, Ph.D.
Science Officer
CDC Division of Violence Prevention
770-488-3936
Jieru Chen, Ph.D.
Mathematical Statistician
CDC Division of Violence Prevention
770-488-1288
References
Siddiqui, O., Hedeker, D., Flay, B.R., & Hu, F.B. (1996). Intraclass correlation estimates in a school-based smoking prevention study. American Journal of Epidemiology, 144, 425-433.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | T. Taylor |
| File Modified | 0000-00-00 |
| File Created | 2021-01-29 |