ombsupportingstatement_5_13_13 final

ombsupportingstatement_5_13_13 final.docx

Teen Dating Relationships Opportunities for Youth to Define What's Healthy and Unhealthy (Pilot study)

OMB: 1121-0333

Document [docx]

Download: docx | pdf

Shape1

TDV OMB Clearance

Supporting Statement

Information Collection Request

Supporting Statement for the

Teen Dating Relationships: Opportunities for Youth to Define what’s Healthy and Unhealthy

also known as the

Understanding Abuse in Teen Dating Relationships through Concept Mapping Project

Project Officer: Carrie F. Mulford, PhD

Crime, Violence and Victimization Research Division

Office of Research and Evaluation National Institute of Justice Office of Justice Programs Department of Justice

810 7th Street NW Washington, DC 20531

Phone: (202) 307-2959

Fax: (202) 616-0275

E-mail: carrie.mulford@usdoj.gov

Section A: Justification

A.1. Circumstances Making the Collection of Information Necessary…………………………………….3

A.2. Purpose and Use of the Information…………………………………………………………………..9

A.3. Use of Information Technology and Burden Reduction……………………………………………..12

A.4. Efforts to Identify Duplication and Use of Similar Information……………………………………..13

A.5. Impact on Small Businesses or Other Small Entities………………………………………………...13

A.6. Consequences of Collecting the Information Less Frequently………………………………………14

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5………………………………...15

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency……15

A.9. Explanation of Any Payment or Gift to Respondents ……………………………………………….16

A.10. Assurance of Confidentiality Provided to Respondents……………………………………………16

A.11. Justification for Sensitive Questions………………………………………………………………..18

A.12. Estimates of Hour Burden Including Annualized Hourly Costs……………………………………20

A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers………………….22

A.14. Annualized Cost to the Federal Government……………………………………………………….22

A.15. Explanation for Program Changes or Adjustments…………………………………………………23

A.16. Plans for Tabulation and Publication and Project Time Schedule………………………………….23

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate……………………………………...27

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions…………………………...27

Section B: Collections of Information Employing Statistical Methods

B.1. Respondent Universe and Sampling Methods……………………………………………………….27

B.2. Procedures for the Collection of Information………………………………………………………...28

B.3. Methods to Maximize Response Rates and Deal with Nonresponse………………………………...29

B.4. Test of Procedures or Methods to be Undertaken……………………………………………………30

B.5. Individuals Consulted on Statistical Aspects and Individuals Collection and/or Analyzing Data…..30

References………………………………………………………………………………………………...31

A.1. Circumstances Making the Collection of Information Necessary

The prevalence of adolescent dating violence has surfaced as a significant issue with implications for the criminal justice, public health and education fields. Particularly, mounting evidence suggests that intimate partner violence has precursors in youth dating relationships that may be identified and about which interventions may be developed. In a recent national survey, 9.8 percent of high school students reported being hit, slapped or physically hurt on purpose by their partner within the year prior to the survey (Centers for Disease Control and Prevention, 2009 Youth Risk Behavior Survey). Additionally, about 1 in

5 women and 1 in 7 men who ever experienced rape, physical violence, and/or stalking by an intimate partner first experienced some form of partner violence between 11 and 17 years of age (Centers for Disease Control and Prevention, 2010 National Intimate Partner and Sexual Violence Survey). Unfortunately, these disturbing estimates describe only victimization, revealing only part of the story. Information on the prevalence of perpetration is lacking, in part because of gaps in understanding the dynamics of teen romantic relationships, the contexts in which these relationships exist, and attitudinal perceptions that frame dating behaviors. Current research makes clear the need to more fully identify and explicate these issues, as acceptance of dating abuse among friends is one of the strongest links to future involvement in dating abuse (Arriaga & Foshee, 2004; Bergman, 1992).

Existing research provides some evidence of conceptual disparities and inconsistencies between adult and teen populations in the way interpersonal relationships are perceived and understood along an age continuum. Previous studies indicate that teens may perceive certain emotions and behaviors differently from adults (including practitioners and researchers). Teens, for example, tend to perceive controlling

and jealous behaviors as signs of love (Levy, 1990), and “…do not perceive of dating aggression as deleterious to the relationship, nor do they view violence as a cause for ending the relationship” (O’Keefe,

2005). In other recent studies, researchers found that adolescent couples tended to work harder than adult couples to limit the intensity of negative exchanges (Galliher, Enno, & Wright, 2008); invested more in

circumventing, minimizing, and disowning the differences between them and their partner (Tuval- Mashiach & Shulman, 2006); and described their romantic relationships in mainly positive terms (Shulman & Kipnis, 2001).

These findings indicate that disparities between how adults and adolescents perceive certain behaviors and emotions in romantic relationships exist; however, little research has been conducted to describe these differences and directly compare how teen and adult populations conceptualize overall healthy and unhealthy relationship behaviors and characteristics.

Furthermore, while existing research has recognized distinctions among adult and adolescent dating behaviors, few studies have examined these dynamics from a standpoint of preventing teen dating violence in particular, and domestic violence in general. It has been argued recently that “more research, particularly qualitative studies, are needed to enhance our understanding of adolescent dating violence, including the nature of relationship conflicts, as well as the meaning, content, intent and consequences of

the violence” (O’Keefe, 2005). Before we are able to address teen dating violence prevention, however, it is “critical to understand how [emotions and behaviors] at the core of adult relationships…may be played out in adolescent relationships” (O’Keefe, 2005). The present study intends to fill this gap by strengthening our understanding of the characteristics and dynamics of teen dating relationships through a process of identification and construct validation using concept mapping.

Few studies to date have considered teens’ perceptions or definitions of dating behavior as described in their own words; rather, the majority of research on adolescent dating has been analyzed through measures and concepts developed by adult researchers and practitioners (Shulman & Kipnis, 2001; O’Keefe, 2005; Tuval-Mashiach & Shulman, 2006; Galliher, Enno, & Wright, 2008; Murphy & Smith,

2010). This field of research, therefore, lacks a definition of dating and dating behaviors that considers the perspective of adolescents, and signals the need to reliably identify and consider the developmental,

social, and environmental context of their interactions. Much of the prior research on teen dating violence

applies adult frameworks and assumptions to characterizing relationships, “further research is needed to test these assumptions” and to assess their validity in being applied to adolescents (Murphy & Smith,

2010).

Methodology

While focus groups and key informant interviews have traditionally been employed in qualitative research, combining these methods with the quantitative methods of concept mapping adds a degree of rigor that enhances our confidence in study outcomes. Concept mapping is a relatively new approach to examining victimization issues, but it is a rigorous approach that has been selected by numerous researchers as an empirical means of examining a variety of social science issues, including topics such as those factors related to adolescent sexuality (Bayer, Cabrera, Gilman, Hindin, & Tsui, 2010), neighborhood influences on mental well-being (Burke, & O’Campo, Salmon & Walker, 2009), barriers to help-seeking for survivors of domestic violence (Simmons, Farrar, Frazer, & Thompson, 2011), factors in the school environment that contribute to school violence (Johnson, Burke, & Gielen, 2011), and conceptualizing elder abuse (Conrad, Ridings, & Iris, 2011; Conrad, Iris, & Ridings, 2011) and elder self- neglect (Iris, Ridings, & Conrad, 2010).

In addition to the studies above that use concept mapping, many federal offices and departments have selected concept mapping as the most appropriate method for clarifying conceptual definitions, program planning, and evaluation. For example, the Women’s Mental Health Initiative (a collaborative between the Office on Women’s Health, the National Institute of Mental Health, the Assistant Secretary for Planning and Evaluation, the Office of the Surgeon General, the Office of Minority Health and the Substance Abuse and Mental Health Services Administration) used concept mapping in the initial stages of examining sex and gender differences in women’s mental health and developing a plan for addressing women’s mental health issues (see http://www.womenshealth.gov/mental-health/government-in-

action/index.html). Concept mapping also has been used by the Centers for Disease Control and

Prevention (Anderson et al., 2006) and the National Cancer Institute (Fox, Bialous, Trochim, Stillman, & Schmitt, 2006), among others, and currently is being used by the Department of Justice to develop a roadmap for addressing elder mistreatment in the United States.

Our research team carefully considered other options, and opted for the concept mapping methodology for four important reasons: 1) concept mapping provides a more structured, efficient, focused and objective approach than just qualitative data analysis; 2) concept mapping is a more participatory approach, including representatives of the target population in the data analysis and interpretation phases and thereby enhancing the construct validity of factors related to teen dating violence, 3) the focused approach enables the use of larger and more diverse samples that increase generalizability and external validity of the findings, and 4) the concept maps generated in the process are ideal for developing a conceptual framework that is valid, reliable, and understood and accessible to different audiences. In addition, the statement set generated through the concept mapping approach can be used to enhance research in several ways. First, specific demographic populations (e.g., LGBTTQ youth, youth with physical disabilities) could be asked to sort the statement set to see if their conceptions of dating relationships are similar to the full sample used in this exploratory study. The validated statement set also can be used as a framework for further measurement development and development of interventions. The need for integrating healthy and unhealthy aspects of teen dating relationships has been underscored by a recent (November 17-18, 2011) meeting hosted by the Robert Wood Johnson Foundation and the Centers for Disease Control and Prevention on defining healthy relationships. The concept map developed from this project could provide a framework for developing healthy and unhealthy aspects of relationships in assessment and interventions with high risk populations. Giordano’s research on adolescent and young adult relationships makes it clear that abusive relationships have many characteristics that would be considered healthy or positive (Giardano, Soto, Manning, & Longmore, 2010). An assessment tool developed from this project could assist in identifying constellations of relationship characteristics associated with relationships with high risk for severe violence or abuse.

The concept mapping method was selected by a well-informed group of agency representatives to the Federal Interagency Workgroup on Teen Dating Violence. Funding has been pooled across seven federal agencies and three additional agencies have provided substantial staff-time support to the project.

Concept mapping culminates in a series of facilitated discussions, one of which will be conducted with federal staff on the Federal Interagency Workgroup on Teen Dating Violence. The facilitated discussion will focus on how the information collected in the concept mapping project will be used for programmatic planning, policy development, and future research.

With respect to the present study, the use of concept mapping offers a unique approach for examining a broad population of teens and adults. It is particularly advantageous in this field of inquiry, as it allows researchers to elicit perspectives from youth on complex topics like teen dating relationships through a relatively simple sequence of participatory steps. Given that a primary objective of this study is to better understand teens’ perspectives of the characteristics of teen dating relationships and their interrelationships and compare them with those of adults, a method is needed that can appropriately and uniformly elicit these data from participants with a range of maturity, life experience and capacity to understand and respond to study instructions. The structured concept mapping approach to the data collection, analysis, and representation will enable the research team to compare conceptualizations both quantitatively and qualitatively, providing a much more robust comparative analysis than simply relying on interview data. The steps of the concept mapping process meet these requirements. The approach

begins with a prompt used during an on-line brainstorming activity that takes the form of a basic sentence completion task, for example, “A thought, action, feeling or behavior that teens in dating relationships might have or do is…” The sentence completion effectively generates a range of content on the topic of interest while maintaining sufficient focus to characterize and define the topic at hand. Other open-ended, participatory methods, such as interviews and focus groups used in prior teen dating relationship studies,

are more conducive to informal dialogue and elicitation of a range of opinions and perspectives. The prompt used in brainstorming is a more structured means for participants to provide quickly and efficiently a focused, direct response to the specific topic of interest. At the same time, it allows for simultaneous consideration of the ideas of others already generated during the brainstorming activity.

Sorting is the essential participatory activity by which the ideas generated during brainstorming acquire relationships with one another, and distinguishes concept mapping from many other methods used in this research area. Prior studies that use interviews, focus groups, rating scales, observation or other methods to understand teen dating relationships better also have focused largely on either a single dimension or the relationship(s) among a select set of two or more dimensions of teen dating relationships, e.g, jealousy, conflict, aggression. And although these methods generate useful insight about adolescents’ conceptual understanding of a specific emotion or experience, they neglect to consider the much broader context of emotions and experiences that exist in teen dating relationships and that subsequently can influence how more specific feelings or behaviors are understood. Concept mapping allows for the explication of a wide range of thoughts, actions, feelings, and behaviors associated with teen dating relationships and a valid

and reliable representation of the nexus of those ideas.

Also noteworthy to the concept mapping approach is the inclusion of what one might consider “healthy” and “unhealthy” ideas in constituting a conceptual domain, which allows researchers to understand better how ideas of different qualities intersect with one another from the perspective of teens and adults. Whereas other studies intend to examine a predetermined idea or concept in isolation from others, concept mapping is advantageous particularly in illuminating how specific ideas, both healthy and unhealthy, may conceptually relate or align with each other, which other methods are not capable of identifying or eliciting. These intersections hold important implications for practitioners and researchers seeking to understand better the underlying associations among ideas that can provide insight into how or why teens conceptualize certain aspects of teen dating relationships differently from adults.

To this end, concept mapping effectively captures and represents the complexity of elements that constitute teen dating relationships and their interrelatedness in a way that other methods do not. This process allows researchers to capture these complex cognitive representations from the adolescent population, a population that historically has not had the opportunity to otherwise articulate these intricacies.

Like the brainstorming activity, the sorting activity is a relatively simple, straightforward task in which participants virtually drag and drop “cards” into piles in a way that makes sense to them. They label those piles in a way that is reflective of what the contents of each pile collectively represent to them. This step is critical to the methodological approach. In the absence of the sorting task, we are left with a list of qualitative descriptors of teen dating relationships with no information about how adolescents and adults perceive these ideas as meaningfully interacting or relating to one another. Additionally, the web-based format allows teens to contribute perspectives on a potentially sensitive topic in a way that is anonymous and convenient and eliminates power dynamics that may occur during interviews or focus groups facilitated by an adult.

Finally, the relatively simple visual output of the concept mapping process allows researchers to communicate effectively the complexity of the concept maps to adolescents in a way they can understand and allow for them to comment on and verify the validity of the results. During the pilot facilitated discussions (see discussion below), the research team explained the results by drawing analogies between the concept maps and geographic maps (with which teens were more familiar), referring to the statements as “cities” and clusters as “states,” thereby conveying the meaning of physical proximity and conceptual similarity. Conceptual dimensions were compared to weather variation and time zones along the north- south and east-west dimensions of a geographic map. Teens were able to comprehend easily the meaning of conceptual relationships on the map and provide the research team with considerable insight in articulating the themes and dimensions on the map in a way that was reflective of their perspective. Other

methods that aim to represent quantitatively and/or summarize data (e.g. rating scales or observations) on this topic may be able to verify or communicate the meaning of results to the teen population less easily, as statistical conclusions may be more difficult for adolescents to understand and provide feedback.

Beyond the value of concept mapping in informing teen dating violence prevention efforts, this information collection can support further the utility of the method in future public health inquiries beyond its more standard use in government-funded projects. In prior federal information collections, concept mapping has been used due to its ability to aggregate and represent the opinions of a large

number of diverse stakeholders, providing a consensus-based conceptual framework for strategic planning and policy recommendations. The effectiveness of concept mapping in this study can support the use of the method in other government-sponsored initiatives that demand a more informed understanding of stakeholders’ theories on focal issues. This information collection can broaden the use of concept

mapping as a tool for other prevention and intervention efforts that to date have based their efforts largely on assumptions of stakeholder understandings and that lack an accurate representation of constituents’ conceptualizations.

The advantage of concept mapping is that the focused, qualitative mixed-method approach allows for the generation of unique ideas and constructs that may not have been incorporated into previous theory. The explication of a rigorously validated and developmentally appropriate theory of teen dating relationship is critical, especially as we anticipate how the results of this study will help to inform future survey and measurement development efforts. While we anticipate that several constructs from existing theory will result from the concept mapping procedure, we also expect that concept mapping will elicit new constructs and also will provide information on which constructs are the most important in informing healthy and unhealthy teen dating relationships. Because the concept map likely will represent a broad range of healthy and unhealthy teen dating relationship characteristics, and will specify the interrelationships between these characteristics, we are able to locate target constructs (both existing and

new) within a semantic net of characteristics hypothesized to influence healthy and unhealthy dating behavior. Use of the concept map as the theoretical measurement pattern helps avoid construct under- representation through the identification of a wide number of items within the content domain (Rosas & Camphausen, 2007). Thus, the map enables the articulation of the relationship between the construct(s) of interest and other related constructs from which they can be differentiated. In that sense, the results of this study will be used to identify and improve the field’s understanding of healthy and unhealthy the characteristics of teen dating relationships, and inform population-based measurement strategies.

After the concept mapping process is complete, NIJ would envision using the individual points on the map as the basis for research questions in an item bank for characteristics of adolescent romantic relationships. NIJ is planning to use the results of the concept mapping study in developing research priority areas for our 2014 and/or 2015 solicitations on teen dating violence, particularly related to developing and testing of measures and articulating theories to explain adolescent relationship abuse. A couple of ideas related to theory testing have already been generated by individuals who have seen presentations of the pilot results. For example, one of the youth in our pilot test facilitated discussion raised the issue of how the relationship characteristics on the map might change over the course of a relationship. Similarly, a prominent researcher at a conference presentation indicated that it would be helpful to see how much overlap there is between the positive/healthy relationship characteristics and the abusive characteristics within a relationship. These are theories that could be tested within the framework developed in the concept mapping study. Additional theoretical explanations are likely to emerge from the facilitated discussions at the end stage of the project. NIJ would also consider funding research to expand on the current study to determine the stability of the concept map with specific subpopulations, including LGBTQ and middle school-aged youth. . The item bank can also be used for testing and development of measures of healthy and abusive aspects of teen dating relationships. A subsequent step would be to test the measures for internal and external validity and to validate the measures with different subpopulations of youth.

Summary of Pilot Study

To date, the Office of Justice Programs within the Department of Justice has conducted a pilot information collection (OMB Control #1121-0333) with a small group of teens ages 14-18 to assess the use of concept mapping as an appropriate tool for creating an adolescent-based conceptual framework that can be used to understand better how the youth population considers relationship behaviors.

Additionally, the purpose of the pilot was to test the data collection protocols and processes (concept mapping and facilitated discussions) in order to refine both the recruitment procedures and the project research questions before conducting further developmental research with a larger group of youth ages

14-22 and adults. The insights gained from the pilot study have proved instrumental in enhancing the participation approach and refining information collection for further validation research using a combination of formal procedures for concept mapping and facilitated discussions, which are further detailed in this supporting statement.

In accordance with the concept mapping process developed by Trochim (1989), approximately 50 teens in the DC and New York area were recruited through youth-serving organizations to participate in the pilot information collection. For the purpose of idea/item generation, these teens were asked to submit ideas through a dedicated project website that completed the sentence: “A specific thought, feeling, action or behavior that teens in dating relationships might have or do is…” A final set of 86 unique ideas were

elicited through this item generation/brainstorming activity. The teens then were asked to sort these ideas into group piles in ways that made sense to them (sorting activity), and to rate these ideas on their perceived relative frequency in teen dating relationships and relative desirability as part of teen dating relationships (rating activity). Cluster analysis and aggregation of the teens’ sort data formed the basis of the conceptual framework (Figure 1). The emergent concept map reveals nine higher-order themes or concepts by which the ideas can be considered in a meaningful structure to describe and understand how

_t_ee_n_s_co_n_ce_p_t_u_a_li_z_e_t_hou_g_h_t_s,
a_c_ti_o_n_s,_f_e_e_li_n_g_s_a_n_d
be_h_a_v_i_o_r_s_r_e_l_a_t_ed_t_o_t_e_en
d_a_t_i_ng_r_e_l_a_t_i_on_s_h_i_p_s.

_{4. Hurtful}_or_Ab_u_s_i_v_e_B_e_h_a_v_i_o_r_s_a_n_d_F_eel_i_n_g_s

_3._W_hen_a_R_elat_i_o_ns_h_i_p

^T^a^k^e^s^O^v^er^P^rⁱ^orⁱ^tⁱ^e^s

_{2. D}_o_u_b_t_a_n_d_T_r_u_s_t_I_s_s_u_es

6. Unpleasant Feelings

_5._N_a_i_v_eté

_{1. E}_x_ci_t_i_n_g_B_e_h_a_vi_ors_w_i_th_P_ote_n_t_i_al
R_i_s_k

_9._P_o_s_i_t_i_v_e_F_e_e_l_i_ng_s

8. Expression of Affection

_7._P_o_s_i_t_i_v_e_A_c_t_i_v_i_t_i_e_s_a_n_d_B_eha_vi_ors

Shape14 Figure 1. Pilot Concept Map

The agency and research team facilitated discussions with the project Planning Group and teens in the DC and New York areas to interpret the meaning of the results. The output of these discussions has contributed basic concepts to use in refining the goals of subsequent information collections and

validation research. For instance, participants in all interpretation sessions recognized the emergence of

two latent dimensions by which the content was structured on the map (Figure 2). Teen and Planning Group participants suggested that the map content appears to describe increasingly healthy, or positive, aspects of dating relationships the farther “south” one looked along the “north-south” dimension. Likewise, ideas seem to describe increasingly unhealthy or negative aspects of teen dating the farther “north” one looked along this same dimension. A review of the content as it appears along the “east- west” dimension suggests that the farther “east” one reads, the more internal to the individual the ideas may be in content. This gradation corresponds with the location of most feelings-related statements on the eastern side of the map. Whereas, on the contrary, the farther “west” one reads on the map, the more

_ex_t_e_r_n_a_l_,_o_r_pu_b_l_i_c,_t_h_e_i_d_e_a_s_a_r_e_i_n_co_n_t_e_n_t_.

_U_n_h_e_a_l_t_h_y

₄_._Hu_r_t_f_u_l_or_A_b_u_s_i_v_e_B_eha_v_i_o_r_s_a_n_d_F_ee_l_i_ng_s

₃_._W_he_n_a_Re_l_a_t_i_o_ns_h_i_p

Takes Over Priorities

Shape17 ₂_._D_o_u_b_t_a_n_d_T_r_u_s_t_I_ss_ue_s

6. Unpleasant Feelings

_Ex_te_r_n_a_l_/Pub_l_i_c

₅_._N_a_i_v_e_té

_In_t_e_rna_l_/P_r_i_va_t_e

₁_._E_x_c_i_t_i_n_g_B_eha_v_i_o_r_s_w_i_t_h_P_o_t_e_n_t_i_a_l_R_i_s_k

Shape18 ₉_._P_o_s_i_t_i_v_e_F_ee_l_i_ng_s

Shape19 8. Expression of Affection

7. Positive Activities and Behaviors

_H_e_a_l_t_h_y

Shape20

Figure 2. Concept Map: A Dimensional Perspective

In further research, subsequent information collection among a broader age range of youth and adults is necessary to determine whether there is convergence or divergence of these dimensions when young

adults (19-22) and adults conceptualize characteristics of teen dating relationships. The broader participant group also will include a more demographically and geographically varied respondent universe, thereby allowing for an overall more diverse sample with which the pilot map results can be compared. Although the full study participant group will be more diverse than the group used for the pilot

test, the purposes of this data collection are exploratory. The goals of the proposed study are to determine how, if at all, teens of various age ranges and adults understand the relationships among thoughts, actions, feelings and behaviors in fundamentally different ways. The Healthy/Unhealthy and Internal/External or Private/Public gradients from the pilot study provide benchmarks for further examining convergence and divergence in youth and adult conceptualizations.

In the pilot study, facilitated discussions with teen participants also illuminated specific statements and clusters, particularly those located toward the center of the map, that could be perceived as healthy or unhealthy depending on the specific context or circumstances in which they occur. Discussion with project Advisory Group members suggested that this ambiguity or continuum along which teens might consider a certain idea as healthy/unhealthy or desirable/undesirable may be reflective of their stage of development or level of experience with dating relationships. Subsequent information collection that includes a wider age range of respondents is necessary to understand better whether such ambiguity or context dependency exists among teens and young adults, or whether it is more likely a factor of teens’ developmental stage. If developmental, one might predict that ambiguity would dissipate with age and experience. Further research is necessary to validate these factors or dimensions with different ages and demographics.

Pattern matching analysis and aggregation of all pilot concept mapping participants’ ratings data also provided a basis for further inquiry. As the Pattern Match (Figure 3) demonstrates, the correlation between the average Frequency and Desirability ratings by cluster is 0.89, indicating that perceptions of frequency are aligned highly predictably with their perceptions of desirability.

Frequency

N=52

₃_.₇₃

Desirability

N=47

₃_.₇₃

_P_o_s_i_t_i_ve_F_eel_i_n_gs

_P_o_s_i_t_i_ve_F_ee_l_i_n_g_s

_P_o_s_i_t_i_ve_Ac_t_i_v_i_t_i_es_a_n_d_B_e_h_av_i_or_s

^Naⁱ^ve^t^é

Expression of Affection

Positive Activities and Behaviors

_W_h_en_a_R_ela_ti_o_ns_hi_p_Ta_k_es_O_ver_P_ri_o_ri_t_i_es

Doubt and Trust Issues

Expression of Affection

Exciting Behaviors with Potential Risk Hurtful or Abusive Behaviors and Feelings Unpleasant Feelings

Naiveté

Exciting Behaviors with Potential Risk

_D_ou_b_t_a_n_d_T_ru_s_t
I_ss_u_es

When a Relationship Takes Over Priorities

1.46

_r₌_.8₉

Unpleasant Feelings

Hurtful or Abusive Behaviors and Feelings

1.46

Shape23 Figure 3. Frequency and Desirability Pattern Match: All Participants

Additional analyses that examined the correlations between the Frequency and Desirability ratings by participant subgroups (age, sex, dating experience) also revealed near perfect correlations (.96, .99 and

.95, respectively) demonstrating convergence. Despite these considerably high correlations, consultation with project Planning and Advisory Group members supported the inclusion of these rating scales in the larger study, as the broader participant group may yield further insight into variation in participant ratings. In particular, from the perspective of practitioners who work with youth, it is important to validate

whether or not their perceptions of the frequency that negative and positive relationship experiences occur are convergent with youth perceptions. One of the critical steps toward construct validation, and

discerning and better understanding commonalities and variation in the opinions of participant groups is to broaden the respondent universe.

Although this next study does not intend to be nationally representative or generalizable, the information collection outlined in this supporting statement is to include an adequately sized and more geographically diverse sample of respondents than the pilot, to allow further examination and appropriate analysis of differences among teens, young adults and adults. A broader sample will include a more racially, ethnically and socioeconomically diverse respondent universe to examine similarities and differences involving demographic variables associated with different rating responses that will inform future research on the subject. Also, because the adult respondent universe will include a subset of teachers,

practitioners, researchers and advocates, the information collection will provide an analysis of variation in perceived frequency and desirability across professional affiliation, which may have important implications for how the field can address and intervene with the problem of teen dating violence more effectively. As such, the proposed information collection is necessary for further validation and to determine whether further variation in these rating scales emerges with a respondent universe greater than the more limited pilot participant group, and what implications this variation may have for future research and the field at large. Appendix G includes a comprehensive summary of the pilot collection, including planning, recruitment, methodology, results and interpretation.

Through the use of concept mapping, results of the pilot information collection provide a rich foundation for understanding how youth conceptualize teen dating relationships. The present information collection intends to enhance the information gathered in the pilot to understand better how both adolescents and adults think about dating relationships, and to provide the field with guidance for future research inquiries and methodological tools for undertaking such research.

The pilot information collection has confirmed the feasibility of concept mapping as a means for arriving at a conceptual framework that will allow for an initial understanding of how youth conceptualize

relationships. The pilot also confirmed that the content derived from the concept mapping process comports with broader research questions regarding how youth understand the range of healthy and unhealthy characteristics associated with teen dating. Given the appropriateness of the method to illustrate the perspective of teens on this topic, a primary purpose of the present information collection is to include adults and a broader sample of youth in the concept mapping process as a means to understand how the conceptualizations among these respondent groups comport with one another.

A.2. Purpose and Use of the Information

The pilot study has set the stage for developing a process that will allow for comparisons between teen

and adult characterizations of teen dating violence. Through the brainstorming activity, the research team will be able to compare the statements generated by youth and adults, and develop a single set of ideas for sorting and rating that represent a synthesis of both respondent groups’ ideas. As a result of the sorting activity, the research team will be able to compare the emergent frameworks and determine a) to what extent, if at all, the dimensions that emerged in the pilot framework are reproduced and/or b) specific

areas of convergence and divergence in how youth and adults conceptualize teen dating relationships. The process will allow the research team to compare cluster solutions among both groups to reveal any tendencies in how youth and adults categorize features of dating relationships. For example, do adults and teens differ in the number of concept piles into which they sort ideas, thereby suggesting that one group may conceptualize the topic more specifically or distinctively than the other? As a result of the ratings activity, the research team will be able to compare the extent to which youth and adults (and subgroups of both populations) perceive the relative frequency and desirability of the ideas. This comparison will be instrumental in illuminating any areas of disagreement between how teens and adults working in the field perceive what occurs in dating relationships. Finally, the facilitated discussions will enrich the conceptual framework even further by allowing groups of youth and adults in various geographic locations to comment on their interpretation of the map and its potential utility.

In addition to providing a set of constructs that capture the conceptualizations of youth and adults, the results of the concept mapping and discussions will be used to generate further research questions on the topic of teen dating relationships. The respondent universe for the present information collection will be more geographically diverse than the pilot, but will still include a convenience sample based on youth- serving organizations and a select sample of adult researchers, practitioners, teachers and advocates that are nominated by project Planning, Advisory Group and Federal Interagency Workgroup on Teen Dating Violence members. As such, the insight derived from this information collection will provide the basis for future research with samples that are intentionally representative of demographics such as race, ethnicity, sexual orientation and socioeconomic status, and in further determining distinctions in how demographic subgroups conceptualize teen dating relationships. Such distinctions may provide important insight in the development of effective teen dating violence prevention efforts that are designed to be implemented among certain populations.

Additionally, the concept mapping process will produce empirically validated parameters that can be used as a basis in future studies. The ideas generated through the brainstorming activity may be used as an “idea bank” that researchers can ask specific respondent groups (e.g., LGBTQ teens, youth in rural communities) to sort and/or rate in order to generate a population-specific conceptual framework on the topic that can be compared with other frameworks using the same ideas. In an interpretation session with the project Advisory Group, several members noted aspects of the pilot results, particularly around the context dependency of whether certain ideas are perceived as healthy or unhealthy and comport with some of their ongoing research agendas. To this end, the results of this information collection may be used to confirm, supplement and/or enhance other studies on the topic.

In this regard, the information collection will create an initial conceptual framework and definitional model that can serve as inroads for more specific research efforts in the future. The broader purpose of this information collection and subsequent research is to ultimately advance our understanding of ways in

which conceptualizations of dating relationship characteristics vary across and within groups. The results of this and later research can inform the language and content of interventions and responses to teen dating violence. Further, they may help to resolve existing definitional ambiguity and methodological variations across studies that have led to a lack of consensus and compounded challenges in It’s in the ROCIS abstract:

“Under Title 42 USC 3722, National Institute of Justice, would require teen dating violence information from youth in order to show that it is critical that the conceptualization that researchers and practitioners use to define relationship abuse resonates with the population to which the definitions are being applied. To understand how youth views of teen dating relationship characteristics (both healthy and unhealthy aspects) is different from or similar to adult views the National Institute of Justice and the Federal Interagency Workgroup on Teen Dating VIolence are undertaking a study using concept mapping.”

understanding accurately the prevalence of and responses to dating violence (Barnett, Miller-Perrin, & Perrin, 2005). The emergence of the conceptualization patterns among the adolescent population, which concept mapping is able to achieve, is critical for the development of measurement approaches.

Finally, the work proposed here is critical as an emergent framework that emphasizes the complex, interrelated aspects of teen dating violence from multiple perspectives (particularly those from youth). In turn, the research undertaken here will help to advance a unified practice, research, and policy agenda. The Federal Interagency Workgroup on Teen Dating Violence comprised of representatives from 18 agencies, representing the Departments of Health and Human Services, Justice, Education, and Defense will review and use the collected information to shape future efforts to understand better the issues and organize effective responses to violence prevention and healthy relationship development. Without such information, planning and development may be limited in terms of matching language and communications, intervention and responses, and measurement and data collection with targeted beneficiaries.

A.3. Use of Information Technology and Burden Reduction

The majority of the information collected for the concept mapping portion of this study will be conducted via a dedicated project website. The website will be administered by Concept Systems, Inc. (CSI), the contractor for this project. This web-based collection technique will reduce the participation time burden for respondents, as it will allow them to respond virtually and remotely, during time that is convenient for them. The project website also will allow for respondents to complete the concept mapping activities

over multiple visits to the website (i.e., respondents will not need to complete the activities in one sitting). The use of information technology for the concept mapping also will reduce the financial burden for respondents, as they will not incur any travel expenses in completing the tasks.

The facilitated discussions will not use information technology, as the in-person, face-to-face context of these discussions will be critical to engaging participants in a productive conversation that will elicit the necessary feedback on the conceptual framework and will confirm its validity for use in teen dating violence prevention efforts.

A.4. Efforts to Identify Duplication and Use of Similar Information

The method for collection of information has never been used in a teen dating violence context and therefore involves no duplication of effort. Although similar information was derived through the pilot information collection, this information was collected for testing purposes only and is insufficient for fulfilling the aforementioned purposes of the present information collection. The present collection seeks to obtain information from a much broader range of participants (both youth and adults) to address those research questions that have been further refined since the pilot collection. However, many items from the brainstorming activity will be prepopulated into the brainstorming website to avoid duplication of effort. The sorts from the pilot study cannot be used because the bank of items generated by brainstorming for the full study will be different from the original bank generated in the pilot study.

The Federal Interagency Workgroup on Teen Dating Violence has been meeting every 6-8 weeks since September 2006. During this time, participating agencies have collaborated on several joint efforts, including co-hosting an HHS-DOJ scientific 2-day workshop in December 2007 examining the research, both basic and applied, on teen dating violence, and determining research, evaluation and practice gaps for further study. Through consultation with organizations and research partners and a review of existing

literature on the subject, it has been determined that a project focusing on the research questions outlined in this data collection does not currently exist.

A.5. Impact on Small Businesses or Other Small Entities

The collection of information may impact not-for-profit youth-serving organizations or agencies with which researchers request assistance in youth respondent recruitment. These organizations will receive recruitment flyers and parental permission slips to distribute to potential youth respondents. To minimize the burden of these requests, the research team will mail a set of instructions and recruitment materials to organizational representatives who will be prepared in advance with distribution instructions. As part of the pilot study, the research team was able to confirm that the recruitment instructions and process materials are clear and effective for organizational representatives to use appropriately. The research team will host a conference call with organizational representatives to answer any questions about the recruitment tasks, and will provide ongoing support on an individual basis as needed. The research team anticipates that the recruitment requests will take organizational representatives no more than two hours total over the course of the information collection period.

The pilot information collection allowed the agency and research team to test the youth recruitment process to be as efficient as possible with organization time and resources. The recruitment instruments themselves (recruitment instructions, consent forms, assent forms, activity instructions) all proved to be clear, concise and easy to follow, such that no changes are needed. The pilot recruitment process revealed several important factors for the research team to consider in subsequent collections, including: 1)

ensuring to the extent possible that brainstorming, sorting and rating recruitment does not coincide with school vacations and holidays; 2) seeking organizations with regular in-person contact with youth to facilitate reminders to turn in signed permission forms and/or complete the activities; 3) seeking organizations that regularly contact youth virtually, e.g., email, Facebook, to facilitate website visits to

complete the online activities, and 4) preparing organizational representatives as far in advance as possible of the necessary recruitment tasks.

A.6. Consequences of Collecting the Information Less Frequently

This information will be collected only once, although certain respondents may be asked to participate up to three to four times during the course of the study, depending on the tasks for which they are recruited. Responding to multiple participation requests is voluntary. Once participants provide their ideas on teen dating relationships (“brainstorming”), a selected subsample of participants will be asked four to eight weeks later to sort the ideas into groups based on how they understand the ideas to be related (“sorting”). All individuals who provided ideas during the brainstorming activity will be asked approximately four to eight weeks after the brainstorming to rate the ideas on perceived frequency and desirability (“ratings”). The respondent re-contact interval is necessary to allow the research team to review the originally brainstormed statement set for clarity, relevance and redundancy, and to synthesize and finalize a set of ideas (100 or fewer) that is manageable in number for respondents to engage in the next stage of the

_r_es_e_a_r_ch₍_so_r_t_and_r_a_t_e₎_,_t_he_r_eby_r_ed_u_c_i_ng_bu_r_den.

_A_.7._Spec_i_al_Ci_rc_u_m_s_t_an_c_es_R_e_l_a_t_i_ng_t_o_t_he_G_u_i_de_li_nes_o_f₅_C_F_R_1320.5

Concept mapping participants will be asked to respond to the brainstorming activity within approximately six weeks. Participants that are asked to respond to the sorting and rating activities will be asked to do so within approximately six weeks after the conclusion of the brainstorming activity. Responses to all portions of the concept mapping activities are voluntary; thus, any individual who is not able to provide a response within the requested time period is not obligated to do so. The information collection outlined fully complies with all guidelines of 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

No comments were received from the public during the 60-day and 30-day Federal Register Notice.

The agency contracted with Concept Systems, Inc. to provide project management and to facilitate the concept mapping and facilitated discussion processes. To ensure the clarity of instructions for all collection materials, the agency also consulted with project Planning and Advisory Groups. These Groups are comprised of researchers, practitioners and advocates in fields related to teen dating violence, as well as representatives from youth-serving organizations. These Groups advised the agency on project design and reviewed respondent materials for their readability to make sure that all youth participants would be able to understand and take part in the different project activities. The agency and research

team has met with the Planning Group on a bi-weekly basis and the Advisory Group on an as needed basis since October 2010.

A.9. Explanation of Any Payment or Gift to Respondents

Teen and young adult respondents who complete the rating activities will receive a $10 Visa gift card. Teen and young adult respondents who complete sorting and rating activities will receive a $15 Visa gift card for completion of both activities. The difference in compensation amount reflects the additional 30-

60 minutes of time it will take respondents to complete the online sorting task in addition to the online rating task. The agency decided to provide youth sorters and raters with a gift because the sorting and rating are the most time intensive tasks of the concept mapping activities, for which youth participation is critical in developing a useful, accurate conceptual framework. Project staff believes that by offering a token of appreciation, teens and young adults will be more likely to complete the sorting and rating tasks than they would if there was no incentive offered. An expression of appreciation for data collection and research activities for youth is common, and the amount offered in this effort is consistent with similar studies. Parental permission forms indicate that youth who complete each activity will receive a gift card, so that parents are fully aware that their child may be receiving such expression of appreciation. Adult respondents will not be compensated for their participation in the concept mapping activities. The adult

respondents will be comprised of teachers, advocates, practitioners and researchers who likely will complete the tasks out of interest in the project and in contributing to the field.

A.10. Assurance of Confidentiality Provided to Respondents

_P_h_a_se_I_:_C_once_p_t_M_app_i_ng

_Y_ou_t_h
p_a_r_t_i_c_i_p_a_ti_o_n_i_n_a_l_l_c_o_m_ponen_t_s_of_t_he_co_n_ce_p_t_m_app_i_ng_p_r_ocess_w_i_l_l_be_ke_pt_co_n_f_i_de_n_ti_a_l_._E_ach

recruitment flyer that partner organizations distribute to participants will include a unique, randomly generated username and password, which will become the username and password for the individual participant who receives the flyer. (Organizational representatives assisting in recruitment will be instructed to distribute flyers only to individuals 17 years old or younger if they return a signed parental permission slip.) The username and password will allow each participant to log onto the project website to complete the concept mapping activities confidentially. Participants will be identifiable only by their username and only to project management.

In the mailings of recruitment flyers and parental permission slips, the research team will include a Master Tracking List for each organizational representative to keep a record of the individual name associated with each username. No one affiliated with this project or project management will ever receive the names of the individual participants. The database serves only as a tool for the organization representatives to keep track of which participants are affiliated with which usernames, so that the researchers can communicate to the representatives which participants have completed the concept mapping activities and eligible to receive a token of appreciation. Gift cards will be distributed to the participants by organizational representatives. The pilot information collection confirmed the effectiveness and efficiency of this process for ensuring confidentiality to respondents.

Adult participation in the brainstorming component of concept mapping will be completely anonymous. However, adult participation in the sorting and rating activities of the concept mapping process will

require that each adult participant access and complete the activities with a unique username and password. The research team, rather than a third-party organization or agency, will be managing the distribution of these username and passwords to adult participants. The agency will maintain complete confidentiality of all input from adult participants, as the data elicited from participants will not be associated with any individual at any point in the project analysis or reporting. Adult participants will be made aware of these parameters and confidentiality assurance during the online consent process (see Appendices A8-A10).

Phase II: Facilitated Discussions

Given the in-person, face-to-face context of the facilitated discussions, participant identity will not be kept anonymous. During the discussions, participants will be identifiable to one another by name tags that will display first names only. The research team will have a database of all participants’ contact

information, as this information is necessary for communication, recordkeeping and travel reimbursement purposes. Youth participants ages 14-17 will be allowed to participate only if both assent and parental consent forms are received by the researchers in advance of the discussions (specific date to be determined). (Please refer to Appendix A for the assent and consent documents that participants and/or parents will receive.)

Any content from these discussions incorporated into the final project report will not be specific to any participant(s). Although the research team and the Planning Group will use the content to enhance and support the final report recommendations, readers will be unable to trace statements or ideas to individual participants. The researchers will maintain a confidential database of participant names and contact information in the security of their Ithaca, New York office on a password protected server for three years after the dates of the facilitated discussions at which time the information will be destroyed. These data will be kept in a password protected folder on the researcher organization’s server to ensure that only members of the research team will have access to the information. Participants will not be contacted after

the conclusion of the project unless they express an interest in being contacted in the future. All participants will be provided with contact information for the research team members.

A.11. Justification for Sensitive Questions

All concept mapping participants will be asked to respond to the focus prompt, “A thought, feeling, action or behavior that teens in dating relationships might have or do is…” After extensive consultation with project Planning and Advisory Groups, the research team determined that this focus prompt was the best question to ask respondents to elicit the most useful content for the purposes of the study. The pilot information collection further confirmed that this focus prompt was appropriate for obtaining the breadth and depth of ideas necessary for the agency to understand sufficiently the range of characteristics that teens associate with dating relationships.

The statements derived from this focus prompt will be reviewed and synthesized by the research team and project Planning Group to yield a set of 100 or fewer ideas that respondents will sort and rate in the subsequent concept mapping activities. These statements will represent the range of ideas that all respondents provided on the topic of teen dating relationships, and will become the basis of the resulting conceptual framework. The pilot-facilitated discussions yielded two important insights into the idea synthesis process for consideration in the subsequent information collection. First, the project Planning Group should ensure that ideas included in the final statement set are clear in meaning. Statements that

the Planning Group members consider to be ambiguous or unclear also will likely also be difficult for the youth and adult respondents to understand and in turn difficult for respondents to sort and rate. The items that youth identified as unclear in the facilitated discussions will be eliminated from the set of items to be used to prepopulate the brainstorming website for the full study. Second, the final statement set should include more ideas related to the role of technology and social media in teen dating relationships (specifically text messaging, Facebook, etc.), as teens felt these ideas were underrepresented in the pilot

framework. Third, the final statement set should include fewer items in group 7 (positive activities and behaviors). The number of different ways that youth in relationships spend time together engaging in prosocial or neutral activities did not yield sufficient new information to justify having them sort the extra items.

Teens may feel some sense of unease around the topic of dating. However, we do not anticipate this discomfort to be more than what teens likely would encounter on a day-to-day basis. All participants will have the option to abstain from any activity of the study at any time without penalty. To limit the discomfort that reading or listening to others’ answers and ideas may cause, the research team will feature

_a_w_eb_s_it_e₍_www_.b_r_e_a_k_t_hec_y_c_l_e._o_r_g₎_at_t_h_e_be_g_i_n_n_i_ng_and
end_o_f_e_a_ch
co_n_ce_p_t_m_app_i_ng_ac_t_i_v_it_y_._U_pon

visiting the website, participants will have access to a list of resources, including websites and phone numbers that participants can use to learn more about healthy teen dating behaviors and what to do if they feel they are in an unhealthy relationship. A hard copy of these resources will be distributed to facilitated discussion participants at the meetings. No issues regarding sensitivity arose among participants of the pilot information collection.

The information and instructions to be provided to concept mapping and facilitated discussion participants, as well as consent forms for each group of respondents, are included as appendices to this document. This informed consent process has been approved by the Institutional Review Board at SUNY-Cortland under Protocol # 101139.

A.12. Estimates of Hour Burden Including Annualized Hourly Costs

Three hundred participants will be invited to participate in the brainstorming task, which will take respondents no more than five to ten minutes to complete. The target was selected on the basis of ensuring opportunity for multiple stakeholder groups to provide input. In developing the frame for

inviting brainstorming participants, we will seek representation from different subgroups, non-randomly and purposefully selected, in an effort to capture a broad set of ideas. An invitation matrix will be developed taking into account the age group (3 groups: youth, young adult; adult), gender (2 groups: male female), and region (4 groups). Thus, the 300 invitees are distributed over a 24-cell matrix, based on the categories for balancing representation in the nationally focused study. In terms of the larger number of targeted youth, we intend to focus our youth recruitment for the full study more heavily on youth-serving organizations that are not focused on healthy relationships or teen dating violence. Based on our recruitment experience in the pilot study, we anticipate teens that are less well-versed on the topic will generate more specific ideas that are considered less healthy or positive, which would theoretically further specify other areas of the map. Thus, by purposefully inviting a broad number of participants across a range of education and knowledge surrounding the topic likely will yield a more diverse set of ideas and new insight in facilitated discussions.

As the number of ideas on the brainstorming website increases, some participants who read those ideas likely will make more specific contributions or “tweaks” to ideas already generated by other participants. The larger participant pool enables the brainstorming process to yield nuances in how teens and others conceptualize dating relationships, and as we learned from the pilot, much of teens’ conceptualizations surrounding this issue are based on specific contextual distinctions. While response rate estimates do not presently exist, the brainstorming activity will be terminated when a point of idea generation has been exhausted or saturation has occurred. Saturation of the topic is based on two factors, considered simultaneously: the judgment of the researchers that the content provided in response to the focus prompt adequately captures the breadth (range of ideas) and depth (range of specificity) of the topic and the extent to which no new information is being provided. During the open brainstorming period, the brainstorming website will be monitored daily for submission of ideas. After a period of three days in

which no new information has been submitted to the site, either by no activity or redundancy of ideas, the

research team in consultation with the Planning group will review the current content for completeness

and adequacy. The activity will be terminated when the project planning team concurs that ideas to complete the prompt have reached a point of being exhausted or saturated. Given that we have had the experience with the pilot study and will be using the list generated from the pilot study to prepopulate the brainstorming web site, it is anticipated that after 100 new visits to the website, the brainstorming activity likely will be complete. Therefore, the burden estimate is based on the first 100 participants to visit the brainstorming site.

One hundred and fifty participants will be invited to participate in the sorting task, which will take respondents approximately 60 to 90 minutes to complete. A recent pooled study analysis of concept mapping studies indicates participant completion rates of approximately 52% for web-based sorting (Rosas & Kane, 2012). Because the analysis plan calls for a minimum of 25 in each subgroup (youth, young adult, and adult) for subgroup comparisons of the conceptualization output, the target of 150 participants was established, assuming a response rate of approximately 52%. Facilitation of the sorting activity during the pilot study revealed that some teen participants did not read or follow the instructions: thereby, rendering their sort data unacceptable for inclusion in analysis. As such, the participation target outlined in this collection is necessary to ensure that we collect sufficient completed sorts from teens that we can use in analysis and subsequent representation.

Three hundred seventy-five participants will be invited to participate in the rating task, which will take respondents approximately 40 to 60 minutes to complete. Again, a 59% response rate for web-based rating has been calculated across multiple concept mapping studies (Rosas & Kane, 2012). Thus, for the purposes of maintaining adequate numbers on the ratings to meet the assumptions for multivariate subgroup comparison with multiple dependent variables, the target of 375 participants was established, assuming a response rate of 59%. The total annual hour burden for a concept mapping participant will vary based on the specific combination of concept mapping activities in which he or she will be asked to participate, but this hour burden will be no more than two hours and forty minutes at most.

Each facilitated discussion will be two to three hours in duration (two hours for youth and young adult groups and three hours for adult group) and will require no more than approximately one hour of total travel time for each participant. These discussions require two to three hours of participants’ time to serve their intended purpose of confirming and enhancing the results of the concept map. The agenda includes time for participants to introduce themselves, for facilitators to provide an overview of the discussion, review the concept mapping process and results map, and allow for ample discussion time.

The extra hour for the adult groups is need to allow for time to discuss and explore how the results can be used to further research and/or practice in the area of teen dating violence. The two to three hours

includes multiple breaks for participants to shift focus, break into smaller group discussions, ask questions, and record on paper any additional thoughts or comments. We anticipate that the topic – teen dating – will be of particular interest to youth participants, and will seem less burdensome than a discussion of a topic unrelated to their personal lives. There will be twelve facilitated discussions (four with adults comprised of fourteen participants each, four with youth ages 14-18 comprised of ten participants each, and four with youth ages 19-22 comprised of ten participants each).

There are an estimated 540 annual total public burden hours associated with this collection. The agency has estimated the annual hour burden for respondents of the concept mapping and facilitated discussion phases based on the contractor’s extensive past experience administering concept mapping activities and facilitated discussions. Recent benchmarking data indicate participant completion rates of approximately

52% for web-based sorting and 59% for web-based rating (Rosas & Kane, 2012) ¹. The hour burden

Shape39 estimation for this collection was calculated using the higher end of the expected ranges of time each activity takes to complete, and assumes the average completion rates suggested by the concept mapping benchmarking data. It is highly likely that most participants will take less time than is reflected in the

¹Online brainstorming allows for participants to anonymously contribute ideas via a dedicated project website. Participants may visit the site as often as they choose and may contribute as many ideas as they like within the allotted time frame. As such, it is not possible to accurately measure average participation in the brainstorming activity.

higher end time estimate. The charts below reflect burden hours for both the higher and lower estimated time frames to complete each activity.

_H_i_gher_end_o_f_t_i_m_e_e_s_t_i_m_a_t_e_rang_e_:

Task	Estimated time (minutes)	Total Participants	Total minutes per task
Brainstorming	10	100	1,000
Sorting	90	78	7,020
Rating	60	221	13,260
Facilitated Discussions with travel	180 240	40 (youth & YA) 56 (adults)	7,200 13,400
Total			41,880 minutes (~698 hours)

Lower end of time estimate range:

Task	Estimated time (minutes)	Total Participants	Total minutes per task
Brainstorming	5	100	500
Sorting	60	78	4,680
Rating	40	221	8,840
Facilitated Discussions with travel	180 240	40 (youth & YA) 56 (adults)	7,200 13,400
Total			34,120 minutes (~569 hours)

A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers

The DOJ anticipates no additional cost burden to respondents or recordkeepers beyond their customary or usually business or private practices.

A.14. Annualized Cost to the Federal Government

The contract to conduct the concept mapping project was awarded competitively to Concept Systems, Inc. The total time and materials contract estimate is based on a 30-month contract amount of $272,511.40. The following are cost estimates to the federal government based upon activities anticipated during the next two years:

a. 2013: $187,616.79 b. 2014: $84,894.61

_T_o_t_al_Feder_a_l_G_ov_e_rn_m_e_n_t_C_os_t_:_$2₇_2,511.40

_A_.15._E_x_p_l_ana_ti_on_f_or_P_r_o_gram_C_ha_n_ges_o_r_A_d_j_u_st_m_en_t_s

The annual hourly burden has been adjusted in A12 from the burden indicated in the pilot to account for the additional number of respondents who will be asked to participate in this information collection. Some adjustments to the hourly burden have been revised based on results from the pilot study. The facilitated discussions have been reduced to two hours (plus an hour of travel time) for the youth and young adult participants. In addition, we anticipate there will be a lower burden required for brainstorming, as many ideas from the pilot will be prepopulated into the brainstorming website. The number of adults who will be invited to brainstorming has been reduced from 150 to 100 to ensure that adult generated ideas do not overwhelm the youth and young-adult-generated ideas. This will be accomplished by including the all-youth-generated pilot brainstorming results. The last change is the elimination of brainstorming for the 11-13 year olds. The project planning group felt that including this group was important because prevention work in the field of teen dating violence often begins in middle school. However, given the burden and complexity in recruiting this age group, we decided to remove

this age-group in part because they were being asked to contribute only to the brainstorming activity.

There is no change in the estimated time that each task will require of respondents. There is no change in the total annual cost burden to respondents or recordkeepers (A13). The annual cost to the federal government has been adjusted in A14 to account for the remaining funds allocated to the contractor since the completion of the pilot information collection.

A.16. Plans for Tabulation and Publication and Project Time Schedule

Concept mapping will be the primary method for analyzing the data. The Concept System® software will be used to combine the individual participant’s sort data and, using several multivariate statistical algorithms, will organize the information and display it in a series of easily readable concept maps. This process will begin with construction from the sort information of an NxN binary, symmetric matrix of similarities, Xij. For any two items i and j, a 1 is placed in Xij if the two items were placed in the same pile by the participant, otherwise a 0 is entered (Weller & Romney, 1988, p. 22). The total NxN

similarity matrix, Tij will be obtained by summing across the individual Xij matrices. Thus, any cell in this matrix could take integer values between 0 and the number of people who sorted the statements. The value indicates the number of people who placed the i,j pair in the same pile.

The total similarity matrix Tij will be analyzed using non-metric multidimensional scaling (MDS) analysis with a two-dimensional solution. The solution will be limited to two dimensions because it is generally easier to work with two-dimensional configurations than with those involving more dimensions, (Kruskal & Wish, 1978). Ease of use considerations are important for decisions about dimensionality.

For example, when an MDS configuration is desired primarily as the foundation on which to display clustering results, a two-dimensional configuration is far more useful than one involving three or more dimensions (p. 58).

The analysis will yield a two-dimensional (x,y) configuration of the set of statements based on the criterion that statements piled together most often are located more proximately in two-dimensional space while those piled together less frequently are further apart. The x,y configuration will serve as the input for the hierarchical cluster analysis using Ward's algorithm (Everitt, 1980) as the basis for defining a cluster. Using the MDS configuration as input to the cluster analysis in effect will force the cluster analysis to partition the MDS configuration into non-overlapping clusters in two-dimensional space.

There is no simple mathematical criterion by which a final number of clusters can be selected. The procedure typically followed is to examine the initial cluster solution that was the maximum desirable for interpretation in this context. Then, successively lower cluster solutions will be examined, with a judgment made at each level about whether the merger seems substantively reasonable. The pattern of judgments of the suitability of different cluster solutions will be examined and the final number of clusters selected to preserve the most detail and still yield substantively interpretable clusters of statements.

The MDS configuration of the statement points will be graphed in two dimensions automatically by the Concept System program. This "point map" will display the location of all the brainstormed statements with statements closer to each other generally expected to be more similar in meaning. In addition, a "cluster map" will be generated that displays the original statement points enclosed by polygon-shaped boundaries for the clusters. In terms of the analysis of ratings data, both descriptive and inferential statistics will be employed. First, computation of average ratings for each statement and cluster of statements for all participants will be conducted. Next, computation of average ratings for each statement and cluster of statements for different subgroups of participants will be conducted. Finally, based on categorical data collected on participant background and demographics, multivariate analyses will be conducted to examine the patterns of similarities and differences in the ratings of frequency and desirability across different subgroups. Additional graphics, including pattern matches and bivariate

scatter plots, will display stakeholders’ ratings of the ideas. The 1-to-5 frequency and desirability rating data will be averaged across persons for each item and each cluster. This rating information will be depicted graphically in a "point rating map" that will show the original point map with the average rating per item displayed as vertical columns in the third dimension, and in a "cluster rating map" that will show the cluster average rating using the third dimension. Two additional graphic and statistical analyses will be computed based upon the map results. A “pattern match” is defined as the bivariate relationship between the cluster average ratings for two groups, variables or occasions. This will be displayed visually as a ladder graph or pair-link diagram with two vertical axes that represent the two variables and

horizontal lines connecting them to represent the ratings for each cluster. A standard Pearson Product- Moment Correlation (r) will be computed to indicate the overall pattern match. In addition, standard descriptive statistics will be produced (mean, SD, N) that will enable significance tests of differences between ratings on clusters. Pattern match graphs will be used to assess consensus or differences between participant groups on the relative frequency and desirability. A “go zone” is defined as a within-cluster bivariate plot of average statement ratings for two groups, variables or occasions. Like a pattern match, it also displays the Pearson Product-Moment Correlation (r) between the two variables. The plot is a restricted form of a standard bivariate plot in that it: (a) sets the minimum and maximum values for all plots to the same range (based on minimum and maximum statement average for that variable); and (b)

the bivariate space is divided into quadrants based on the cluster average of the x and y variables. This effectively means that every go zone plot will have a quadrant that shows which statements in the cluster were rated above average on both variables, one that shows which statements were below average on both and two that show the statements that were above average on one and below on the other. This plot, like

a pattern match, will be used to explore consensus, in this case, within-cluster.

The content derived from the facilitated discussions will be integrated into the final conceptual framework through an iterative process of data reduction and synthesis. We anticipate that the facilitated discussions

will yield a substantial volume of information, from which information can be extracted to further elucidate emergent concepts. Using the concept mapping framework as the analytic structure for organizing, reducing and analyzing the content captured during the facilitated discussions, we expect to identify insights, themes, patterns and constructs that will aid in the interpretation of the domain. Traditional approaches to qualitative data reduction, coding, and analysis will be used within the concept mapping structure to illuminate the connections individuals make between and among the concepts.

The following table indicated the project time schedule:

Task	Month after receiving OMB Approval
Obtain IRB Approval and OMB Clearance	0
Concept Mapping	1-8
Brainstorming	1-2
Idea Synthesis	3
Sorting and Rating	4-5
Analysis	6
Preliminary Interpretation	7
Draft Concept Map Report	7-8
Facilitated Discussions	9-17
Planning and Participant Recruitment	9-10
Conduct Facilitated Discussions	11-13
Conduct Content Analysis of Facilitated Discussions to Link to Framework	14-15
Create Report of Facilitated Discussion Activity	16-17
Final Reporting and Data Aggregation	18-20

A summary report will be developed that will include the aggregated concept mapping and facilitated discussion results. The summary will include a statistical report of the participant response rate for the sorting and rating activities. At the conclusion of the study, the data will be archived at the National Archive of Criminal Justice Data (NACJD).

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

Not applicable. The agency is not seeking approval to not display the expiration date for OMB approval of the information collection.

_A_.18._E_x_cep_ti_o_n_s_t_o_C_e_r_t_i_fi_ca_t_i_on_f_or_P_a_p_e_r_w_ork_R_e_duc_t_i_on_A_ct
Su_b_m_i_s_s_i_ons

Not applicable. There are no exceptions to the Certification for Paperwork Reduction Act Submissions for this information collection.

B. Collections of Information Employing Statistical Methods

B.1. Respondent Universe and Sampling Methods

The following table displays the number of individuals that will be invited to participate in the different information collection tasks:

Concept Mapping Participation Targets
Task			Teens (14-18)	Young Adults (19-22)	Adults	Total task target
Brainstorming	100			100	100	300
Sorting	50			50	50	150
Rating	125			125	125	375
Total group target						375
Facilitated Discussion Participation Targets
Suggested location		Teens (14-18)		Young Adults (19-22)	Adults	Total regional target
Washington, DC		10		10	20	40
Atlanta		10		10	20	40
Chicago or Kansas City		10		10	20	40
San Francisco		10		10	20	40

Total group target

0 40

160

This information collection will not require purposefully sampling on the basis of any specific social demographics other than age, although every effort will be made to include youth representing various demographic groups (e.g., racial, ethnic, SES, religious, sexual minority). As such, the target numbers for each age-group were selected to allow for a range of views from youth and young adults of varying backgrounds and adults with a range of professional experiences.

_B_.2._Proced_u_res_f_or_t_he_C_o_l_l_ec_t_i_on_o_f_I_n_f_o_r_m_a_t_i_on

The information collection approach for this project will be to purposefully sample on the basis of heterogeneity, which is to non-randomly select a broad range of persons who are likely to reflect the full range of ideas as possible that are relevant to the topic under investigation. However, the adequacy of ideas captured is dependent upon whether there is a match between the focus and the participants selected to participate. We are not expecting that the results will be generalizable to the youth and adult populations at large; however, we will be looking for saturation of the topic, as bounded by the participant groups. Thus, we anticipate that in the brainstorming task, we will reach a point at which there will be a fair amount of redundancy and some homeostasis in the topic. The goal is to achieve a broad sampling of ideas rather than a representative sample of individuals. Youth respondents will be recruited through youth-serving organizations with collective access to a diverse group of youth ages 14-22 recommended by members of the project Planning and Advisory Groups. These groups will be selected in part through

a nomination process that solicited recommendations for organizations from the project’s Planning and Advisory Groups. Within this group of youth serving organizations, we are looking to purposefully stratify our sample by age across two age groups: 14-18 (teens) and 19-22 (young adult), to capture the variation of perspectives as they may be influenced by age. The organizations that will be used for recruitment purposes will be instructed by the research team to recruit a specific number of youth within

the specific age ranges for participation in each activity (see Appendices B1-B3). Inclusion of youth participants will be based on a first-come-first-serve criterion; those youth who are first to respond to the organizational representatives’ advertisements will have the opportunity to participate. In addition, adult participants will be selected from Planning and Advisory Group nominations, based on the relevance of their professional roles to the youth population and their relationships (e.g., practitioners, teachers, advocates, researchers).

In terms of quality control, the sorting, rating and demographic/background data will be gathered directly over the web, thus eliminating any concerns about mis-entering or mis-reading handwritten data. The software has some constraints built in that prevent errors in the data entry. For example, because this is an unstructured forced-choice sort method (Weller & Romney, 1988), the software does not allow a statement to be placed in more than one group simultaneously. For the ratings, the software only allows legitimate entries (e.g., the integers 1-to-5). Before any participant data can be used in data analysis, the

software requires that the concept map analyst visually inspect the data and indicate by checking a setting that the data are complete and useable. Without this check, the participant’s data will not appear in any subsequent screen that calls for data analysis. Because the Concept System® software was designed expressly to accomplish the concept mapping process and analysis, there is no danger that the statistical analysis procedure might be mis-specified by the analyst.

B.3. Methods to Maximize Response Rates and Deal with Non-Response

The brainstorming task will be conducted completely anonymously for all respondents. The number of statements elicited in response to the focus prompt will be measureable. However, there will be no way to determine the actual response rate, as respondents will be able to provide as many statements as they choose without any way to identify which respondents provided which statements. One limitation of the

concept mapping method is that there is an inherent bias, such that the most motivated participants will provide the majority of the statements that the rest of the group will be asked to sort and rate.

The response rates for sorting and rating will be calculated over the course of the information collection, as the project website administrators will be able to monitor the progress of each participant according to their username. To maximize response rates for brainstorming, sorting and rating, a reminder notice will be e-mailed to all invited adult respondents at multiple points during the period for each task. Response rates for youth invited to participate in the sorting and rating also will be maximized by offering the incentives described in section A9 of this supporting statement, and by asking youth-serving organization representatives to remind youth participants to complete the activities. In addition, because the use of incentives to maximize completion of the sorting and rating tasks, we anticipate that our response rates will be higher than the estimates outlined in section B1 above.

The facilitated discussion response rate will be calculated based on how many invitees (10-14 per discussion) actually attend. Attendance rates will be maximized by sending invitees reminders of the discussions in advance of the sessions.

B.4. Test of Procedures or Methods to be Undertaken

_N_ot_ap_p_l_i_ca_b_l_e._N_o_t_e_s_t_s_o_f_p_r_o_c_edu_r_es_o_r_m_e_t_hods_wil_l_be_und_e_r_t_a_k_en.

_B_.5._Ind_i_v_i_du_a_l_s_C_ons_u_l_te_{d
on S}_t_a_t_i_s_t_i_cal_A_spe_c_t_s_a_nd
Ind_i_v_i_dua_l_s_C_o_l_l_e_c_t_i_on_and_/_or_A_n_a_l_y_z_i_ng_D_a_t_a

Concept Systems, Inc. (CSI) is the contractor that will be consulted on statistical aspects of the design, and will collect and analyze the information for the agency. Individuals from CSI who will be contributing to the statistical design and analysis are as follows:

i. Mary Kane, Principal Consultant ii. Scott Rosas, Senior Consultant

iii. Alyssa Goldman, Project Manager iv. Jennifer Royer, Junior Consultant

These individuals can be reached by telephone at (607) 272-1206.

The NIJ staff listed below will be involved with statistical analysis or interpretation. All of them can be reached at (202) 307-2942.

i. Carrie Mulford, Social Science Analyst

ii. Dara Blachman-Demner, Social Science Analyst iii. Jaclyn Smith, Research Assistant

References

Anderberg, M. R. (1973). Cluster analysis for applications. New York, NY: Academic Press, Inc. Anderson L.A., Gwaltney M.K., Sundra D.L., Brownson R.C., Kane M., Cross A.W., et al. (2006). Using

concept mapping to develop a logic model for the Prevention Research Centers Program. Preventing Chronic Disease [3:1 serial online] 2006 Jan. Available from: URL: http://www.cdc.gov/pcd/issues/2006/jan/05_0153.htm.

Arriaga X. B., & Foshee, V. A. (2004). Adolescent dating violence. Do adolescents follow in their friends’ or their parents’ footsteps? Journal of Interpersonal Violence,19(2):162–84.

Barnett, O., Miller-Perrin, C. L., & Perrin, R. D. (2005). Family violence across the lifespan 2ⁿ^dEd. Thousand Oaks, CS: Sage Publications.

Bayer, A. M., Cabrera, L. Z., Gilman, R. H., Hindin, M. J., & Tsui, A. O. (2010). Adolescents can know best: Using concept mapping to identify factors and pathways driving adolescent sexuality in Lima, Peru. Social Science & Medicine 70(12): 2085-2095.

_h_tt_p_:_/_/_www_.nc_b_i_.n_l_m_.n_i_h._g_o_v_/_p_m_c_/_a_rt_i_c_l_e_s_/_P_M_C₃₂₄₈₃₄₉_/_pd_f_/_n_i_h_m_s319533.p_d_f

Bergman, L. (1992). Dating violence among high school students. Social Work; 37:21–7.

Brown, J. (2006). Rethinking concept mapping for youth participatory evaluation in the context of youth development programs. Journal of Youth Development, 1(2).

Burke, J., O'Campo, P., Salmon, C., & Walker, R. (2009). Pathways connecting neighborhood influences and mental well-being: Socioeconomic position and gender differences. Social Science & Medicine 68(7):

1294-1304.

Conrad, K. J., Iris, M., & Ridings, J. W. (2011). Conceptual model and map of psychological abuse of older adults. Journal of Elder Abuse and Neglect 23(2): 147-168.

Conrad, K. J., Ridings, J.W., & Iris, M. (2011). Conceptual Model and map of financial exploitation of older adults. Journal of Elder Abuse and Neglect 23(4): 304 - 325.

Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in web- or internet- based surveys. Educational and Psychological Measurement, 60(6), 821-836.

Coxon, A. P. M. (1999). Sorting data: Collection and analysis. Thousand Oaks, CA: Sage.

Everitt, B. (1980). Cluster analysis. 2nd edition. New York, NY: Halsted Press, A Division of John Wiley and Sons.

Fox, B. J., Bialous, S. A., Trochim, W., Stillman, F. A., & Schmitt, C. L. (2006). Evaluating tobacco industry tactics as a counterforce to ASSIST. In National Cancer Institute (Ed.), Evaluating assist: A

blueprint for understanding state-level tobacco control (Vol. 17, pp. 213-264). Bethesda, MD: Department of Health and Human Services, National Institutes of Health, National Cancer Institute. Galliher, R. V., Enno, A. M., & Wright, R. (2008). Convergence and divergence across multiple methods for assessing adolescent romantic relationships. Journal of Adolescence, 31, 747-769.

Giordano, P. C., Soto, D. A., Manning, W. D., & Longmore, M. A. (2010). The characteristics of romantic relationships associated with teen dating violence. Social Science Research, 39(6), 863-874.

Hays, S., & Scholla, K. (2003). Engaging Youth In Tobacco Policy Change: Results from a Concept Mapping Project in Madison County, Illinois. Paper presented at the 2003 Annual Meeting of Midwest Political Science Association, Chicago, Illinois, April 4, 2003.

Iris, M., Ridings, J.W., & Conrad, K. J. (2010). The development of a conceptual model for understanding elder self-neglect. The Gerontologist 50(3): 303-315.

Johnson, S. L., Burke, J. G., & Gielen, A. C. (2011). Prioritizing the school environment in school violence prevention efforts. Journal of School Health 81(6): 331-340.

Kane, M., & Trochim, W. (2007). Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage Publications.

Kaplowitz, M. D., Hadlock, T. D., & Levine, R. (2004). A comparison of web and mail survey response rates. Public Opinion Quarterly, 68(1), 94-101.

Krippendorf, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Newbury Park, CA: Sage Publications.

Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage Publications. Levy, B. (1990). Abusive teen dating relationships: An emerging issue for the 90s. Response to the

Victimization of Women and Children. 13(1).

Levy B. (1991). Dating violence: Women in danger. Seal, Seattle, WA.

Mulford, C., & Giordano, P. C. (2008) Teen dating violence: A closer look at adolescent romantic relationships. NIJ Journal, 261, 34-40.

Murphy, K. & Smith, D. (2010). Adolescent girls’ responses to warning signs of abuse in romantic relationships. Journal of Interpersonal Violence, 25(4),626-647.

O’Keefe, M. (2005). Teen Dating Violence: A Review of Risk Factors and Prevention Efforts. VAWnet: The National Online Resource Center on Violence Against Women (www.vawnet.org: Applied Research Forum). .

Osborn, A. F. (1957). Applied imagination. New York: Scribner.

Patton, M. Q. (1990). Qualitative evaluation and research methods 2ⁿ^dEd. Thousand Oaks, CA: Sage

Publications.

Ries, A., Voorhees, C., Gittelsohn, J., Roche, K. & Astone, N. (2008). Adolescents’ perceptions of environmental influences on physical activity. American Journal of Health Behavior, 32(1), 26-39.

Rosas, S. R., & Camphausen, L. C. (2007). The use of concept mapping for scale development and validation in evaluation. Evaluation and Program Planning, 30(2), 125-135.

Rosas, S. R., & Kane, M. (2012). Quality and rigor of the concept mapping methodology: A pooled study analysis. Evaluation and Program Planning, 35(2), 236-245.

Rosenberg, S., & Kim, M. P. (1975). The method of sorting as a data gathering procedure in multivariate research. Multivariate Behavioral Research, 10, 489-502.

Schwartz, R. M. (1994). The skilled facilitator: Practical wisdom for developing effective groups. San

Francisco: Jossey-Bass.

Shulman, S. & Kipnis, O. (2001). Adolescent romantic relationships: A look from the future. Journal of

Adolescence, 24,337-351.

Silverman J, Raj A, Mucci L, Hathaway J. (2001). Dating violence against adolescent girls and associated substance use, unhealthy weight control, sexual risk behavior, pregnancy, and suicidality. Journal of the American Medical Association, 286(5):572-9.

Simmons, C. A., Farrar, M., Frazer, K., & Thompson, M. J. (2011). From the voices of women: Facilitating survivor access to IPV services. Violence Against Women 17(10): 1226-1243.

Snider, C., Kirst, M. Abubakar, S. Ahmad, F. & Nathens, A. (2010). Community-based articipatory research: Development of an emergency department-based youth violence intervention using concept mapping. Academy of Emergency Medicine, 17,877-885.

Trochim, W. (1989). An introduction to concept mapping for planning and evaluation. Evaluation and

Program Planning, 12(1), 1-16.

Tuval-Mashiach, R. & Shulman, S. (2006). Resolution of disagreements between romantic partners, among adolescents, and young adults: Qualitative analysis of interaction discourse. Journal of Research on Adolescence, 16(4), 561-588.

Weller, S. C., & Romney, A. K. (1988). Systematic data collection. Newbury Park, CA: Sage

Publications.

Shape2

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Information Collection Request
Author	Alyssa Goldman
File Modified	0000-00-00
File Created	2021-01-29