CCT Data Collection Plan

Attachment 7_CCT Data Collection Plan.doc

Drug Free Communities Support Program National Evaluation

CCT Data Collection Plan

OMB: 3201-0012

Document [doc]
Download: doc | pdf

DFC Evaluation Systems, Measures, and Tools

















Attachment 7:

CCT Data Collection Plan



CCT Data Collection Plan









CCT Table of Measures1

Field Name

Data Type

Description

Keep (Keep, Delete, or New)

Std.

Corr.

Rationale

CCT – Coalition Development & Management Global Rating

Likert

Section I: Coalition development and management involves designing and implementing your coalition’s organizational structure and operating procedures. These elements are fundamental and create the delivery mechanism for the array of prevention strategies that will be undertaken.

Keep


  • This is one of four function areas that comprise the coalition typology. Past evaluation reports have extensively used the typology to classify/categorize coalitions (i.e., Establishing, Functioning, Maturing, & Sustaining) and assess if they achieve different results on core outcomes. The typology should be retained in order to compare future findings to historical data.

  • Additional typologies or other methods to categorize coalitions in developmental stages will also be investigated in future analyses.

  • Overall reliability = .88.

  • Assessing needs*

Keep

.72

  • Mobilizing and building capacity in the coalition*

Keep

.73

  • Developing a comprehensive plan*

Keep

.74

  • Implementing*

Keep

.70

  • Evaluating*

Keep

.65

  • Planning for sustainability*

Keep

.63

Coalition Type


Check

one


Item stem - Which of the following best describes your coalition?

Keep


  • This is basic and necessary descriptive information which should be retained.

  • A loosely organized group whose main goal is information sharing.

Keep


  • A semi-formal group of organizations who have begun to work together on prevention programs and strategies.

Keep


  • A formal group or organization who plan and act together to implement prevention programs and strategies.

Keep


  • A highly formal arrangement with most organizations having a clear role in the planning and implementation of community wide prevention strategies.

Keep


Organizational Structure






Check

One


How would you describe your coalition’s organizational status?

Keep


  • This is another basic and necessary item to obtain some descriptive information about coalition structure and functioning.


  • Our coalition reports to a larger organization on programmatic and fiscal matters.

Keep


  • Our coalition reports to another organization on fiscal matters only, but we are able to operate otherwise independently, including choosing our own staff.

Keep


  • Our coalition is a totally independent organization. We make decisions regarding finances, personnel, and programming.

Keep


Text

  • Other, please specify:

Keep


Confidence in Task Completion

Likert

Item stem - How confident are you that your coalition can…?

Keep


  • This is a reliable scale that asks members if they are confident that their coalition can complete a number of tasks, from agenda setting to developing new leaders. This scale can be used to create a measure of collaborative formalization and could possibly be re-worded along strongly disagree to strongly agree anchors, reducing the need for a separate section and decreasing response burden.

  • Factor Analysis – Good factor loadings (all >.50, except “develop agendas” which cross-loads with another factor).

  • Overall reliability = .94.

  • Develop agendas and stick to them in meetings.

Keep

.48

  • Recruit new members who have the ability to take action in the community

Keep

.67

  • Recruit new members who are accountable for reporting to their organization or constituency.

Keep

.69

  • Follow up on decisions made at meetings.

Keep

.66

  • Recruit members from the different sectors needed to address your Coalition’s goals.

Keep

.68

  • Provide direction and vision for the Coalition through its leadership.

Keep

.77

  • Maintain support of Coalition members through its leadership.

Keep

.78

  • Share leadership among Coalition members.

Keep

.73

  • Maintain stable leadership.

Keep

.75

  • Develop new leaders.

Keep

.75

  • Delegate responsibilities to committees.

Keep

.64

  • Engage members of target populations (e.g. youth) and diverse cultural groups as active members and leaders.

Keep

.61

  • Set and achieve annual goals.

Keep

.70

  • Recruit “champions” to act on Coalition’s behalf.

Keep

.68

  • Hold each other accountable.

Keep

.73

Coalition Governing Body

Yes/No

Item stem - We have a board or governing body that sets the direction of the coalition (A board or governing body is defined as a formal group or body that makes decisions for the coalition)?

Keep


  • This item precedes all the questions regarding written procedures and provides some basic descriptive information which is needed. This item should be retained.

Written Procedures

Yes/No





Likert



** 3-point

Item stem - Please Indicate your agreement by selecting “Yes” or disagreement by selecting “No” with each of the following statements as they apply to your coalition.

Keep


  • Yes/No items on collaborative procedures are pretty common and standard items. There are currently eleven items regarding if there are written procedures around coalition activities and other items meant to tap into the “formal” nature of the collaboration.

  • These items can be used to create scales (using a cut point to identify formalized coalitions versus not formalized) or looked at individually. There are some disadvantages to this method though, such as giving equal weighting to each activity and possible lack of variance across coalitions.

  • We have a governing body that sets the direction of the coalition.

Keep


  • We have a written policy for leadership rotation.

Keep


  • We have written expectations for member participation (e.g., policy on missed meetings).

Keep


  • We have a written description of procedures for leader selection.

Keep


  • We have a written description of the procedures for decision making (e.g., majority rule, etc.).

Keep


  • We hold regularly scheduled meetings (i.e., on a specific date/time).

Keep


  • We prepare a written agenda for each Coalition meeting.

Keep


  • We prepare and distribute written minutes of Coalition meetings.

Keep


  • We have a current organizational chart showing Coalition structure and relationships.

Keep


  • We have established subcommittees.

Keep


  • We have paid staff members or in-kind staff (in-kind staff are staff who are paid by someone else to work at your coalition).

Keep


Identification of Community Leaders

Likert

Item stem - Please indicate how much you agree or disagree with each of the following statements.

Keep


  • Adequate factor loadings.

  • This scale has adequate reliability and seems to tap into the grassroots and community-based nature of DFC. Thus, we recommend retaining this scale.

  • Overall reliability = .70.

  • Our coalition has identified all of the community leaders that can help in the coalition’s activities.

Keep

.45

  • Our coalition regularly works with the necessary community leaders and they work on coalition activities or actively support strategies.

Keep

.56

  • The most important community leaders are members of the coalition and work on coalition committees and activities.

Keep

.53

Community Leadership in Coalition Efforts

Likert



** 3-point

Please indicate the degree to which each statement below describes the role of the community leadership (not coalition staff) of the coalition. The coalition community leadership…

Keep


  • This scale follows-up on the previous scale and asks about the extent of community leaders’ involvement in collaborative activities.

  • Takes responsibility for coordinating the coalition.

Keep


  • Sets the agenda for coalition meetings.

Keep


  • Keeps coalition members accountable to tasks.

Keep


  • Promotes cohesiveness and team spirit.

Keep


  • Creates an environment where opinions can be voiced.

Keep


  • Are recognized leaders in the larger community on issues related to substance abuse prevention and related issues.

Keep


  • Are very involved in regional, state or national prevention organizations or coalitions.

Keep


Coalition Leadership

Likert

Item stem - Coalition leadership refers to the official chair or co-chairs of your coalition. Below are several statements about leadership within coalitions.

New


  • In the current CCT, there are two good scales around community leadership – but there is no specific scale for coalition leadership. This may be because one of the co-chairs fills out the survey. If this is often the case, we propose at least including short scales on committee leadership and coalition coordinator leadership. Practitioners stress the importance of leadership and empirical research has demonstrated the strong links between leadership and collaborative effectiveness. These scales were originally adapted from Butterfoss (1998). Overall reliability is .97.

  • Coalition leadership is committed to the coalition’s mission.

New


  • Coalition leader(s) provide leadership and guidance in maintaining the coalition.

New


  • Coalition leader(s) have appropriate time to devote to the coalition.

New


  • Coalition(s) leaders promote equality and collaboration among members.

New


Committee Leadership



Likert

Item stem - Committee leadership refers to the chair or co-chairs of subcommittees.

New


  • Committee leader(s) plan meetings effectively and efficiently.

New


  • Committee leader(s) are flexible in accepting different viewpoints.

New


  • Committee leader(s) is/are adept in organizational and communication skills.

New


  • Committee leader(s) are adept in obtaining resources.

New


Coalition Coordinator Leadership

Likert

Item stem - Coalition coordinator refers to the paid staff on your coalition.


New


Does funding from DFC support a part-time or full-time coalition coordinator?

Yes/No


Does funding from local or another source support a part-time or full-time coalition coordinator?



  • Our coalition coordinator plays a vital role in organizing coalition activities and efforts.

New


  • The coalition coordinator facilitates communication across coalition participants.

New


  • The coalition coordinator supports the coalition’s goals.

New


Collaborative Decision Making

Likert



** 3-item

Item stem - We would like to understand the effectiveness of your coalition in making decision. Please indicate the degree to which you either agree or disagree with each of the following statements describing how your coalition makes decisions. If you have trouble deciding, please select the answer that describes your feelings most of the time.

Keep


  • Decision making processes are key to coalition functioning and member satisfaction. This scale has adequate reliability if the reverse coded item is deleted.

  • Reliability is somewhat low because a 3-point scale is used and it seems there are two issues being asked about: (1) if decisions are made and (2) if the decision making process is democratic and participatory. It might be beneficial to add a few items from pre-existing scales to increase reliability or use a 5-point agreement response set.

  • The coalition makes decisions when they are needed.

Keep


  • The general membership has real decision-making control over the policies and actions of the coalition.

Keep


  • Decisions are made by a small group.

Keep


  • There is formal process for making decisions (e.g., a voting system).

Keep


  • Decisions on the allocation of coalition resources are made in an open and participatory manner.

Keep


  • We effectively use the coalition process to plan and make decisions

Keep


Shared Vision & Cohesion

Likert

Item Stem - Please indicate the extent to which you agree with each statement from (1) Strongly Disagree to (5) Strongly Agree.

New


  • According to reviews of collaborative research, shared vision is one of the key dynamics that explains variability of coalition functioning and effectiveness.

  • 5 items with .87 reliability.

  • The coalition has a feeling of cohesiveness and team spirit.

New


  • There is a shared vision for desired outcomes of DFC coalition work (e.g., reduce use, increase protective factors).

New


  • Coalition members feel valued and important.

New


  • There are clearly defined, attainable goals for the initiative.

New


  • There is a shared vision of what the coalition should accomplish.

New


Frequency of Conflicts

Likert

How frequently do conflicts such as major disagreements and tensions among individual and groups occur in your coalition?

Delete


  • The items around frequency, cause, and impact of conflicts seem to provide less information than other items and they are recommended for deletion due to the need to include additional variables and items. For instance, check all that apply items are difficult to analyze in a meaningful fashion.

  • If retained, CCT should be altered so communities could at least rank the top 2 or 3 causes for conflict.

Cause of Conflicts

Check all that apply

What are the causes of these conflicts? (Please check all that apply?)

Delete


  • Different perspectives or philosophies

Delete


  • Personality differences

Delete


  • Turf battles

Delete


  • Issues in the larger community

Delete


  • Arguments over allocation of coalition resources

Delete


  • Arguments over among or burden of work

Delete


Text

  • Other, please specify

Delete


Impact of Conflicts

Check one

Overall, what has been the impact of these conflicts on the coalition? (Please check only one).

Delete


  • The coalition is a lot worse off/Conflicts have hurt the coalition’s progress.

Delete


  • The coalition is somewhat worse off.

Delete


  • These conflicts have not made a difference.

Delete


  • The coalition is somewhat better off because of the conflict.

Delete


  • The coalition is a lot better off/very positive results.

Delete


Communication and Conflict Management

Likert

Item Stem - Please indicate the extent to which you agree with each statement from (1) Strongly Disagree to (5) Strongly Agree.

New


  • This was added because conflict management is an important coalition dynamic, but prior questions were not obtaining the necessary data.

  • This is an abbreviated scale that we have used in the past which has established reliability. It can be used as one scale or broken out into communication management and conflict management (depending on results of factor analysis).

  • There is a formal process for resolving conflicts among participating organizations.

New


  • The communication procedures are clearly understood among collaborative members.

New


  • Conflicts arise frequently among participating organizations in the collaborative.

New


  • Communication between member organizations is closed and guarded.

New


  • Differences among collaborative members are recognized and worked through.

New


Cultural Diversity

Likert

The following statements cover issues regarding how your coalition addresses cultural diversity and cultural competence. Please read each statement carefully and check the one response that comes closest to your belief. With regard to cultural diversity, your Coalition has....?

Keep


  • Good factor loadings (>.5).

  • This scale has seven items and good reliability. Also, given the defining role of cultural competence or diversity in DFC, we recommend retaining this scale.

  • Overall reliability = .85.

  • Staff members that are representative of the demographic and cultural diversity in your community.

Keep

.54

  • Culturally and language appropriate materials relevant to the target population.

Keep

.62

  • Materials examined by diversity experts or target population members.

Keep

.56

  • Developed a culturally appropriate outreach action plan.

Keep

.61

  • Activities and decision-making processes that are designed to be inclusive.

Keep

.61

  • Meetings and activities scheduled at times that are convenient and at locations that are accessible to the target population

Keep

.57

  • Coalition members that are representative of the demographic and cultural diversity in your community.

Keep

.69



  • Targeted youth are involved in coalition meetings and activities.

New


  • This was added because of the importance of engaging and involving the target community of DFC efforts.

Performance Evaluation

Yes/No

Item stem - Please respond Yes or No for the following statements. Does your coalition …?

Keep


  • This Yes/No list of performance evaluation and regular evaluation items should be retained given the need for GPRA data and substance use data.

  • Items in the CCT can only serve to reinforce the importance of data collection for coalition leaders.

  • Possibly create a Likert scale to provide a reliability estimate for the scale.

  • Record coalition decisions in minutes or otherwise keep track of decisions.

Keep


  • Have a system for monitoring and tracking coalition activities and other efforts.

Keep


  • Monitor on a regular (annual or more frequently) basis community indicators of substance abuse or related information.

Keep


  • Have someone on staff at the coalition or a member organization that assists the coalition with collecting and analyzing data on coalition activities and community indicators.

Keep


  • Use relations with local universities or others to get assistance on evaluating coalition efforts.

Keep


  • Typically hold meetings to reflect on the result of monitoring or evaluation activities in order to make adjustments to implementation.

Keep


  • Collect school survey data that matches the coalition’s geographic boundaries at least once every two years.

Keep


Coalition Sustainability



Likert

There are many factors that may contribute to the developmental of a sustainable coalition. A sustainable coalition is one that can continue to work in the community without DFC funding and share ownership and capacity to collaboratively prevent substance abuse. We would like your assessment of the extent to which your coalition has accomplished each of the following. Your coalition has ….

Keep


  • Sustainability is a major focus for DFC and is even factored in the typology ratings assigned to coalitions, yet we believe it is not being fully assessed in the current version of the CCT. Of the thirteen sustainability items, we propose retaining six and adding an additional five items for an overall reduction in the number of items but expansion in assessing the quality of coalitions’ sustainability efforts.

  • In addition to simply asking if the coalition itself will keep meeting, the items tap into other mechanisms through which DFC has been embedded/ institutionalized in the community (via MOUs, procedures for sharing relevant information, strategies that combine agency resources, such as blended funding, etc.).

  • Factor analysis is OK with a couple items below .5.

  • Good overall reliability (.77).

  • The last six items utilize CADCA’s

  • Established a reputation for ‘being able to get things done’ related to at least one initiative or practice.

Keep

.57

  • Adopted an entrepreneurial spirit in seeking additional support.

Keep

.54

  • Established on-going jobs in your organization.

Keep

.47

  • A mature and stable lead organization or has all functions (501c3 status, etc.) to operate independently.

Keep

.45

  • Aligned the coalition’s goals and priorities well with past work done by the coalition.

Keep

.57

  • A plan for continued leadership.

Keep

.52

  • The DFC community coalition plans on continuing meeting after federal grant funding ends.

New


  • Youth-serving organizations have begun to or are already taking on coalition efforts in order to institutionalize them and provide long –term sustainability

New


  • The coalition has developed strategies to continue to combine agency resources to better serve youth and families (e.g., blended funding, identification of alternative funding, etc.).

New


  • The coalition has secured funding to continue its substance abuse reduction efforts when federal funding ends.

New


  • Procedures for continuing to share relevant information across agencies have been established.

New


These items correspond to CADCA’s 6 steps of sustainability planning


  • Identified what must be sustained to continue our community-based efforts toward drug abuse prevention.

New



  • Identified the resources that are required to continue our community-based efforts toward drug abuse prevention.

New



  • Created case statements that explained our need to continue community-based efforts toward drug abuse prevention efforts.

New



  • Determined funding strategies to continue our community-based efforts toward drug abuse prevention.

New



  • Identified potential partners to continue our community-based efforts toward drug abuse prevention.

New



  • Created an action plan to contact and present to potential partners in our efforts to continue community-based drug abuse prevention.

New


Sustainability

Choose one (Yes/No/Maybe or Don’t Know)

Item stem - Do you think your coalition will be able to sustain itself in your community for the next ten years?

Delete


Check all that apply

Why don’t you think your coalition will be able to sustain itself in your community for the next ten years?

Delete


  • Changes in circumstances, staff, and resources.

Delete


  • DFC is no longer a good “fit” in our community.

Delete


  • Desired outcomes were not achieved.

Delete


  • Changes in population, place, or policy of interest reduced the need for DFC.

Delete


  • Funding and resources are not available to continue

Delete


Text

  • Other, please specify.

Delete


Coalition Focus 1

Check one

Some coalitions focus on coordinating programs or services delivered by their partners (e.g., curriculum for youth or parents, provision of after school programs); others focus on changing aspects of their community environment (e.g., availability, policy, enforcement). Which of the following do you think best describes the current status of your coalition?

Keep


  • Keep for identification of degree of coalition focus on direct services and prevention programming compared to environmental focus.

  • Our coalition focuses exclusively on coordinating the provision of prevention programs or services.

Keep


  • Out coalition focuses primarily on coordinating the provision of prevention programs/services, though occasionally attention is paid to environmental or policy change.

Keep


  • Our coalition’s focus is about evenly split between coordinating prevention programs/services and environmental or policy change.

Keep


  • Our coalition focuses primarily on environmental or policy change, though occasionally attention is paid to coordinating prevention programs and services.

Keep


  • Our coalition focuses exclusively on environmental or policy change.

Keep


CCT – Coordination of Prevention Programs/Services

Likert

Section II: Coordination of Prevention Program/Services. A community uses community data and lists of evidence-based programs and services to identify and coordination an array of “best fit” prevention programs and services to be delivered by its partners. A coalition then aligns and coordinates the integration of these programs and services across the community and evaluates their impact.

Keep


  • This is one of four functional areas that comprise the Coalition Classification Tool. Classifications have been extensively used in data analyses and should be kept for historical comparative purposes.

  • Overall reliability = .91.

  • Assessing needs*

Keep

.73

  • Mobilizing and building capacity of partners*

Keep

.80

  • Developing comprehensive Strategic and Action Plans*

Keep

.78

  • Implementation of Strategic and Action Plan*

Keep

.74

  • Conducting Process & Outcome Evaluations*

Keep

.72

  • Planning for sustainability*

Keep

.69

Action Plan Activities

Likert

Please choose one response for each of the following statements to characterize your coalition. Your coalition has ……

Keep


  • Factor Analysis - All greater than .50 except 1 (evidence-based, which approaches .50 and loads on another factor).

  • This is a reliable scale which obtains good information on the degree of follow-up action planning.

  • Overall reliability = .87.

  • Identified specific strategies and activities to reach its goals.

Keep

.61

  • Created a realistic timeline for completing activities.

Keep

.71

  • Identified responsible person(s)/agencies for each activity.

Keep

.71

  • Developed a strategy to recruit participants for activities or events.

Keep

.61

  • Developed a budget that outlines the funding required for each coalition related activity, training, or event.

Keep

.65

  • Identified other resources needed for each activity.

Keep

.69

  • Planned to use primarily evidence-based strategies.

Keep

.48

  • Plans to evaluate strategies and activities.

Keep

.56

  • Revisited and updated your action plan.

New


  • Added to track a DFC requirement to update action plans annually (after creating the strategic plan in the first year).

CCT - Environmental Strategies

Likert

Section III: Environmental strategies (e.g., reducing access, policy change, increasing enforcement) involve the coalition assessing the community environment to select and implement an array of “best fit” environmental initiatives. The coalition then aligns and integrates them across the community and evaluates their impact.

Keep


  • This is one of four functional areas that comprise the coalition typology. Classifications have been extensively used in data analyses and should be kept for historical comparative purposes.

  • Overall reliability = .93.

  • Assessing needs*

Keep

.79

  • Mobilizing and building capacity*

Keep

.83

  • Developing a comprehensive plan*

Keep

.85

  • Implementing strategies*

Keep

.83

  • Evaluating environmental strategies*

Keep

.77

  • Planning for sustainability*

Keep

.64

Environmental Strategies

Likert

Please indicate how much you agree or disagree with each of the following statements about your coalition.

Keep


  • Factor analysis – there is some cross-loading for this construct, mainly on two constructs but a few on other constructs as well. Given the importance of this variable for DFC, it may need to be modified.

  • Overall reliability = .84.

  • Our members are just learning the importance of environmental strategies.

Keep

.46

  • We have had past success implementing policy and other environmental strategies.

Keep

.67

  • The majority of our members are supportive of the coalition pursuing environmental strategies.

Keep

.59

  • The majority of our members participate in implementing environmental strategies.

Keep

.61

  • Our coalition does not have sufficient ability to advocate for policy changes.

Keep

.54

  • Our coalition does not currently have the ability to implement an effective media campaign.

Keep

.45

  • We have identified the policy and environmental changes relevant to our community.

Keep

.59

  • We have the appropriate members to launch the needed policy change and environmental strategies.

Keep

.58

  • We belong to other coalitions and networks in order to have a larger impact on policy and environmental strategies.

Keep

.43



  • Our coalition uses an appropriate mix of environmental change strategies from providing information to changing consequences and physical design.

New


  • Added in to obtain information regarding using a mix of strategies targeting multiple levels.

Coalition Focus 2

Check one

Whatever your coalition’s focus, some coalitions act directly in the community (e.g., coordinating prevention programs and services or advocating for an environmental/policy change). Other coalitions act indirectly by building capacity of other organizations (through convening, incubating, training, or technical assistance). Which of the following do you think best describes the current status of your coalition?

Keep


  • This is another item that is attempting to assess the coalition’s specific focus – direct or indirect. Recommend keeping item.

  • Our coalition exclusively acts directly (coordinates programs/services and pursues policy change ourselves).

Keep


  • Our coalition primarily acts directly (coordinates programs/services and pursues policy change), though occasional attention is paid to building the capacity of other organizations.

Keep


  • Our coalition is evenly split between acting directly and building the capacity of others to act (acting indirectly).

Keep


  • Our coalition primarily acts indirectly (building the capacity of other organizations) though occasional attention is paid to acting directly to coordinate programs/services or pursue policy change.

Keep


  • Our coalition exclusively acts indirectly (we build the capacity of other organizations to act).

Keep


CCT – Intermediary or Community support Org

Likert

Section IV: Some coalitions evolve into “intermediary or community support organizations.” The coalitions act mainly indirectly in the community by building the capacity of other organizations. They assess training and technical assistance needs and resources, build capacity where needed, develop a capacity building plan, provide an array of support services and evaluate impact on recipient organizations and the community.

Keep


  • This is one of four functional areas that comprise the coalition typology. Classifications have been extensively used in data analyses and should be kept for historical comparative purposes.

  • Overall reliability = .92.

  • Assessing needs*

Keep

.77

  • Mobilizing and building capacity of other organizations*

Keep

.80

  • Developing a comprehensive plan for capacity building*

Keep

.80

  • Implementing or integrating capacity building strategies*

Keep

.78

  • Evaluating capacity building efforts*

Keep

.75

  • Planning for sustainability*

Keep

.71

Capacity Building Capacity

Check All That Apply

Please check all of the following resources your coalition has available for capacity building?

Modify


  • This is a decent checklist on some of the technical assistance and capacity building that coalitions could be providing to community-based organizations; however, need to put on a Likert Scale.

  • Information on research-based prevention programs and strategies.

Keep


  • Staff and consultants who are knowledgeable and skilled in capacity building methods (e.g., adult education, consulting).

Keep


  • Staff and consultants are knowledgeable of evidence-based prevention programs, services and environmental strategies undertaken by the coalition.

Keep


  • Ability to produce educational materials.

Keep


  • Resources to send coalition members to workshops, conferences, or to visit other communities.

Keep


  • Ability to distribute materials and other educational resources.

Keep


Synergy

Likert

What makes coalitions unique is their potential to combine the different perspectives, knowledge, and skills of a group of people and organizations. This unique combining power has been called synergy. We would like to know the extent to which you feel synergy is being created in your coalition. How much do you agree or disagree with each of the following statements.

Keep


  • Inspection of individual item content suggests that some of the items may be capturing related constructs (common language and common goals seems to be getting at shared vision). Depending on stakeholder feedback, this variable may be able to be deleted or modified.

  • Factor analysis – good results (all items >.50).

  • Overall alpha = .84.

  • Our coalition has developed a common language for communication among diverse partners.

Keep

.62

  • Our coalition has developed common goals that are understood and supported by all partners.

Keep

.63

  • Our coalition is better able to carry out its work because of the contributions of diverse partners.

Keep

.63

  • Our coalition has clearly communicated how its action will address problems that are important to people in the community.

Keep

.65

  • Our coalition has committed the perspectives, resources and skills of partners.

Keep

.68

Perceived Effectiveness

Likert

Collective self-efficacy refers to a group’s shared belief in its joint capabilities to attain goals and accomplish desired tasks. We are interested in your assessment of the collective self-efficacy of the coalition. How much do you agree or disagree with each of the following statements.

Keep


  • In the CCT, this scale is called collective self-efficacy. Two items, which reflect connections to other organizations and larger entities, are proposed to be kept while three other more general items (e.g., “can work together as a group”) are proposed to be cut.

  • Seven additional items, specific to the goals of the DFC community change initiative, are proposed to be included.

  • Factor Analysis – Poor results, majority of items want to load on synergy, but display poor loadings.

  • Our coalition has the support of other organizations and influential community leaders.

Keep


  • Our coalition participates in larger coalitions and groups that can affect change at a higher level (e.g., State).

Keep


  • Our coalition has a sufficient combination of skills within its membership to achieve the coalition’s goals.

Delete


  • Our coalition can work together as a group and take collective action to achieve the coalition’s goals.

Delete


  • Whenever our coalition has to move forward towards a goal, almost everyone is willing to join in and do their share of the work.

Delete


  • Coalition activities and efforts have been successful in reducing substance abuse rates among youth in the community.

New


  • Coalition activities and efforts have been successful in increasing community awareness of substance abuse issues in the community.

New


  • The coalition has established ‘credibility’ in the community by their on-going efforts to reduce substance use rates among youth in the community.

New


  • Coalition activities and efforts have been successful in increasing community-based approaches to reducing substance use.

New


  • Coalition activities and efforts have been successful in increasing interagency collaborative efforts to address substance abuse issues.

New


  • Coalition activities and efforts have been successful in increasing protective factors among youth in the community.

New


  • Coalition activities and efforts have been successful in decreasing risk factors among youth in the community.

New


Coalition Self-Assessment

Check one

Coalitions develop as they increase their mastery of key functions needed to prevent substance abuse… For the work you need to do in order to be a successful coalition, how would you rate your coalition’s capacity to perform the key functions presented above?

Keep


Overall assessment of coalition functioning – keep.

  • My coalition is learning how to perform most of these key functions.

Keep


  • My coalition is achieving proficiency (competency) in performing most of the key functions to prevent substance abuse, though some we may be learning or have mastery (expertise).

Keep


  • My coalition has achieved mastery (expertise) in performing most of the key functions needed to prevent substance abuse, however we could not sustain our coalition sufficiently without DFC funding

Keep


  • My coalition has achieved mastery in most of the key functions we perform in the community. The capacities are sustainable and institutionalized (i.e., they can be supported without DFC funding and the capacities and mission are shared among many organizations).

Keep



Text

Additional Comments

Keep



Member Satisfaction

Likert

Please indicate how much you agree or disagree with each of the following statements about your coalition.

New


  • This new variable is proposed to obtain an idea of how member satisfaction contributes to coalition functioning and could even be modeled as a basic output or outcome variable as well.

  • Could be strengthened even further if we move to a multiple-informant data collection format.

  • Members are satisfied with the coalition’s progress in addressing youth substance abuse issues.

New


  • Members are satisfied with the degree of collaboration that has occurred as a result of the coalition’s activities and efforts.

New


  • Members are satisfied with the progress made toward coalition goals and objectives.

New


Interorganizational

Coordination

Likert

Please indicate how much you agree or disagree with each of the following questions about your coalition.

New


  • Given that one of the major goals of DFC is to increase collaboration, this variable is proposed to be included so we can capture whether or not organizations and agencies are indeed coordinating and collaborating to a greater extent.

  • To what extent has the efforts of your coalition resulted in organizations and agencies working together more efficiently?

New


  • To what extent has the efforts of your coalition resulted in members seeing their organization/agency as part of a broader system responding to youth drug abuse?

New


  • To what extent has the efforts of your coalition increased the ability of organizations/agencies to coordinate their efforts?

New


  • To what extent have the efforts of your coalition increased members’ knowledge of the strengths, as well as limitations, of each other’s organizations and agencies?

New


Organizational Impact



Likert

For my organization/agency, participation in the coalition has led to:

New


  • This variable assesses the degree of impact of coalition efforts and activities on member organizations. This could be a good “systems and community change” outcome variable – how has DFC efforts impacted the system -- that is, changed organizational/agency/community work for the benefit of youth and families?

  • The generation of new ideas for improving our practices and/or services.

New


  • The acquisition of useful knowledge about services, programs, or people in the community.

New


  • A decrease in the number or severity of barriers we face in accomplishing our mission.

New


  • A greater ability to identify the source of problems we encounter in order to come up with more effective solutions.

New


  • A heightened public profile for my organization/agency.

New


  • Increased our ability to affect public policy.

New


  • Increased access to tools, best practices, and/or other information that has informed the work of my organization.

New


  • Greater knowledge about how the system works and how organizations and agencies affect one another.

New


  • An increase in our ability to find the answers to questions or problems that arise.

New


  • An improvement in our ability to compete for grants and/or other funding opportunities.

New


  • Increased utilizations of my organizations/agency’s expertise or services.

New


Community Social Organization

Likert

Please indicate how much you agree or disagree with each of the following statements about your coalition.

New


  • This variable is proposed in order to capture more about the community context and is an indicator of a community asset (social organization). Past research has demonstrated how this variable transmits neighborhood disadvantage to outcomes like substance use and delinquency.

  • People in this community know each other.

New


  • People in this community participate in social activities.

New


  • People in this community feel connected to each other.

New


  • People who live here feel they are part of a community.

New


  • People who live here never do things to improve the community.

New


  • People talk to each other about community problems.

New


  • People in this community have a voice regarding important issues.

New


  • Together, people in this community can persuade the city to respond to their needs and concerns.

New


  • It is fairly safe to walk in this community at night.

New


  • People in this community don’t trust each other.

New


  • Residents don’t care about the community’s future.

New


  • People in this community make it a safer place to live.

New


Collective Efficacy

Likert

Please indicate how much you agree or disagree with each of the following statements about your coalition.

New


  • Similar to community social organization, this variable, made famous by the project of human development in Chicago neighborhoods, obtains data on whether or not people feel comfortable in the neighborhood to intervene when kids are misbehaving and has been linked to lower levels of neighborhood violence.

  • Children are skipping school and hanging out on a street corner.

New


  • Children are spray-painting graffiti on a local building.

New


  • Children are showing disrespect to an adult.

New


  • Children are fighting in front of your house.

New


  • Fire station closest to your home was threatened with budget cuts.

New


  • People around here are willing to help their neighbors.

New


  • This is a close knit neighborhood.

New


  • People in this neighborhood can be trusted.

New


  • People in this neighborhood generally don’t get along with each other.

New


  • People in this neighborhood don’t share the same values.

New


Coalition Readiness

Likert

Please indicate how much you agree or disagree with each of the following statements about your coalition.

New


  • As discussed in the analysis plan, this variable obtains data on the degree to which coalition participants are ready and able for a comprehensive community change initiative like DFC.

  • There is widespread knowledge about preventing drug abuse among participating agencies, organizations, and individuals.

New


  • There is widespread support for the prevention of drug abuse among participating agencies, organizations, and individuals.

New


  • There is a history of productive interaction among the stakeholders involved in designing and implementing the coalition’s drug abuse prevention efforts.

New


  • Leaders of participating organizations are willing to commit resources, including staff time, for the coalition’s drug abuse prevention efforts.

New


  • Financial resources (in addition to grant funds) are readily available to support the coalition’s drug abuse prevention efforts.

New


  • Services and supports – formal and informal – are readily available in the community to support the coalition’s drug abuse prevention efforts.

New


Community Readiness

Likert

Please indicate how much you agree or disagree with each of the following statements about your coalition.

New


  • As discussed in the analysis plan, this variable obtains data on the degree to which community residents are ready and able for a comprehensive community change initiative that seeks to reduce substance use rates.

  • There is widespread knowledge about the coalition’s drug abuse prevention efforts in the community.

New


  • There is widespread support for the coalition’s drug abuse prevention efforts in the community.

New


  • Community leaders are concerned about reducing substance abuse rates in the community.

New


  • Community members and stakeholders are concerned about reducing substance abuse rates in the community.

New


  • Existing programs within the community are conducive to developing interagency collaborative relationships.

New














1 Items marked with a “*” were used in the development of the typology of coalition maturity (i.e., Establishing, Functioning, Maturing, Sustaining). These items are being preserved in order to facilitate historical comparisons.

Office of National Drug Control Policy Page 0

File Typeapplication/msword
AuthorICF
Last Modified By15150
File Modified2011-08-05
File Created2011-04-04

© 2024 OMB.report | Privacy Policy