CCI Research Questions by Method Table B

CCI Evaluation Questions, Outcomes and Indicators

Definitions

• Community organizations: organizations and institutions within the local communities that are involved in the design and/or implementation of local CCI-funded project

• Community members: residents, citizens, actors, members of associations that are involved in the design and/or implementation of local CCI-funded project

• Community partners: Community organizations and community members (per above definitions)

• Museums and libraries: All museums and libraries involved in CCI-funded projects—not necessarily grantees

• Grantee partners: Grantees that partner with museums and libraries within their CCI-funded projects but are not museums or libraries

CCI Outcomes Questions

1. To what degree and in what ways did CCI-participating museums, libraries and their grantee partners develop capacity to be community catalysts?

1. In what ways did CCI-participating museums, libraries and their grantee partners change practices to better engage their communities in co-creating and implementing community change?

2. To what degree and in what ways were CCI-participating museums, libraries and their grantee partners able to create a local ecosystem that supports community social change?

3. To what degree and in what ways do CCI communities experience positive social change?

4. How did outcomes vary across types of CCI supports/inputs used, cohorts, or characteristics of involved library/museums, partners, or communities?

Table B.1 | Changes in CCI Grantee Capacity, Practices, and Local Networks/Communities (EQ #1, #2, #3, #4, and #5)

Evaluation Question/Associated Outcome	Example Indicators	Data sources
To what degree and in what ways did CCI-participating museums, libraries and their grantee partners develop capacity to be community catalysts?
Changes in beliefs, attitudes, knowledge and skills among those involved in the CCI-funded project
Increased awareness of power dynamics in communities and openness to shifting power dynamics in community change efforts Note: early change after initial training/TA	Orientation and commitment toward shifting power from institutions to community members Awareness of power differentials in the community Understanding root/systemic causes of power differentials Understanding of potential roles for the library/museum in efforts to shift power differentials/address root causes thereof	Administrative data including ABCD Self-Assessment Toolkit TA provider notes and interviews Project team interviews, museum/library surveys, partner surveys, case studies
Increased understanding of assets and networks, and belief in benefit of using an asset-based approach in local community change efforts Note: early change after discovery processes and additional TA	Understanding of assets in local community Understanding of key actors and influencers within networks in local community Understanding of the historical and cultural context of local community Understanding of what community members care about in local community (i.e., community member priorities) Understanding of areas of local “common ground” between community member priorities and outcomes that institutional partners are working toward Understanding of roles that identified assets can play within community change efforts and how to apply those in own communities Belief in benefit of using an asset-based approach in community change efforts Belief in benefit of adapting strategy to address existing community member priorities	Administrative data including asset maps TA provider notes and interviews Project team interviews, museum/library surveys, partner surveys, case studies
Increased capacity to convene diverse stakeholders and facilitate co-creation and joint implementation of a common agenda (intermediate-term/during project implementation)	Positive orientation toward collaboration and relationship-building as primary mechanism for community change Strategies and skills for guiding diverse partners in dialogue and working together building and sustaining trust and empathy among community members and institutional partners (e.g., active listening, appreciative inquiry) authentically engaging community members at all stages of community change efforts	TA provider notes and interviews Project team interviews, museum/library surveys, partner surveys, case studies
Increased flexibility/ agility/ adaptability to respond to changing nature/ contexts of grantee projects (intermediate-term/during project implementation)	Openness to altering course in response to changing community priorities and contexts Sense of accountability to community members versus institutional partners Strategies and tools for capturing, responding to, and adapting to changing community priorities and contexts	TA provider notes and interviews Project team interviews, museum/library surveys, partner surveys, case studies
Organizational systems changes among museums, libraries, grantee partners
Increased alignment of organizational priorities and expertise with co-designed, jointly implemented, asset-focused, community-driven collaboration	Leadership support and vision for asset-focused, community-driven collaboration (perceived and explicit/public among which levels of leadership/staffing hierarchy) Resources dedicated to asset-focused, community-driven collaboration (e.g., investment in professional development, staff time allocation, part of budget vs dependent on grants) Documented institutional vision, mission statement, and/or strategic plan aligned with/prioritizing asset-focused, community-driven collaboration	TA provider notes and interviews Project team interviews, museum/library surveys, case studies
Increased structures and processes supporting authentic engagement of community members	Empowered decision-making groups comprised of diverse partners Standing meetings and events at times and locations convenient for community members and inclusive agenda structures Internal communication structures/processes to support effective, bi-directional, and culturally responsive information flow and engagement with the community Flexible policies and practices to address community-driven programs	TA provider notes and interviews Project team interviews, museum/library surveys, case studies
In what ways did CCI-participating museums, libraries and their grantee partners change practices to better engage their communities in co-creating and implementing community change?
Increased interactions with community members and non-traditional partners outside of the museum or library (informal discovery practices)	Visits to community-driven events and gathering places (in contrast to only institutional events for the community) Interactions with community associations, leaders, and individuals	Administrative data including power ladders Grantee interviews, surveys and case studies
Increased engagement in discovery processes with community members (asset mapping, learning conversations) (formal discovery practices)	Use of asset mapping to identify a wide range of community assets from individual, associations, institutions, places, and culture Facilitation of community listening sessions to identify community priorities Engagement of additional relevant cross-sector Institutional partners in dialogue with community members	Administrative data including asset maps TA provider notes and interviews Grantee interviews, surveys and case studies, including social network analysis of partner relationships
Increased co-creation and joint implementation of a common agenda with community members and cross-sector partners, within CCI-funded project	Community members have lead/impacted decision-making in implementation of CCI-funded projects There is a common agenda that was co-created by diverse community partners around community-driven priorities (agenda = common vision/outcome goal and understanding of what assets should be brought to bear to help realize vision/plan) Solutions are assets-focused and emerge from community member strengths and priorities Institutional practices support community action (versus drive the efforts) Community partners implement and support strategies and solutions aligned with common agenda Community partners use communication processes and structures to provide input, and support collaboration/ engagement Community partners use common indicators to track progress Community partners collectively engage in ongoing evaluation, data use, and learning to assess progress and impact	Administrative data including asset maps TA provider notes and interviews Grantee interviews, surveys and case studies Partner surveys
Increased planning for engagement in efforts that include co-creation and joint implementation of a common agenda with community members and cross-sector partners, beyond CCI-funded project	Plans for integrating asset-focused, community-driven collaboration into other/future community change efforts Rest same as C but beyond CCI-funded project	NA
Changes in local ecosystems
To what degree and in what ways were CCI-participating museums, libraries and their grantee partners able to create a local ecosystem that supports community social change?
Increased number of connections/ partnerships among museums, libraries, other organizations, and community members, particularly including citizen associations, historically under-represented groups, civic leaders, and local funders	Number and type of partners (e.g., non-institutional and institutional partners, different sectors) Number of partners that are citizen associations, historically under-represented groups, civic leaders, and local funders Network characteristics including density of the network (degree of cohesion/inter-connectivity in the network), degree of centralization (degree to which activity is centered in a few organizations (high) or spread across organizations (low)), and size of the network	Administrative data: Asset maps, TA provider notes Museum/library and partner surveys—social network questions Project team interviews
Deeper connections/ partnerships among museums, libraries, other organizations, and community members, particularly including citizen associations, historically under-represented groups, civic leaders, and local funders	Increased value of the network and partners (including shifts in power and influence of partners, level of involvement and resource/asset contribution or partners) Increased trust (including reliability, openness to discussion/compromise, and support of the collaborative mission) Increased collaboration (sharing information and resources, short-term collaboration on discrete projects, long-term partnership)	Museum/library and partner surveys—social network questions Project team interviews
Changes in local communities
To what degree and in what ways do CCI communities experience positive social change?
Increased agency/empowerment among community members	Sense of agency among community members (e.g., sense of decision-making authority, sense of power to affect community change)	Case studies
Increased social well-being among community members	[Community outcomes are identified by local projects and vary according to project goals]	Administrative data: local evaluation reports Case studies
Increased perceptions that museums and libraries are trusted and important allies in strengthening communities	Perceptions that local museum/library are trusted allies in community change efforts Perceptions that local museum/library are important/effective allies in community change efforts	Case studies
Increased local investment in community member-led community transformation	Perceptions of local investor interest (monetary and non-monetary/in-kind	Case studies
How did outcomes vary across types of CCI supports/inputs used, cohorts, or characteristics of involved library/museums, partners, or communities?
Variation factors	Cohort: 1 or 2 Grantee type: Museum, library, or other Uptake of CCI TA, tools, peer learning opportunity (CoP) Number of partners: small, medium, large (based on analytic rubric) Focus area of work (e.g., early learning, social service) Leadership/institutional support for library/museum engagement in community-driven change: high, med, low (based on analytic rubric) Previous history with and existing practices of collaborative community engagement	Administrative data: Grant reports/ applications

CCI Implementation

6. How did grantee project teams vary in their use of CCI supports/inputs?

   1. Which activities mattered the most and to whom?

   2. What attributes of the model require fidelity and what could be adapted to local contexts?

7. What are the benefits of using a cohort approach, including peer learning networks, in this work?

Table B.1 | CCI Implementation (EQ#6 and #7)

CCI Supports	Dimensions to Assess	Data sources
How did project teams vary in their use of CCI supports? What activities mattered most to whom? What attributes of the model require fidelity and what could be adapted to local contexts?
Training
In-person grantee convenings/workshops Presentations by TA providers Presentations by other grantees Large group discussions Small group discussions Networking opportunities Hands on practice/demonstration activities (e.g., how to implement specific tools and practices)	Satisfaction with quality/utility component Learning from component—in what specific ways did it build capacity? Application of component—how did grantees apply in own project? Relative instrumental value/benefit of component—how much did it contribute to their ability to implement their project effectively?	Administrative data: Grantee feedback on workshop sessions Museum/library and partner surveys additional items for those receiving direct TA support Project team interviews
Webinars
Coaching/ TA Support
Site consultants/coaches	Direct benefits of working with coach (learning from direction, learning from reflection, resources, connections, support) Map contribution of coach efforts to capacity outcomes (to what degree did coaching contribute to. . .?) Satisfaction with overall coaching, frequency/amount of time, quality	Museum/library and partner surveys additional items for those receiving direct TA support Project team interviews TA provider interviews
EPA TA providers
Tools and resources Articles and white papers Asset mapping Power ladders Journey maps Theories of change	Which tools were used? If used, for what purpose? If used, how did the tool support their project work? (qualitative and direct connection to desired practices) If not used? Why? Relative utility and ease of use for each tool? Relevance of each tool for local project? Cultural appropriateness of each tool? What other tools would be useful?	Administrative data: TA provider notes and work product artifacts Museum/library and partner surveys additional items for those receiving direct TA support Project team interviews TA provider interviews
What are the benefits of using a cohort approach, including peer learning networks, in this work?
Community of practice (Also some components of in-person convenings and workshops)	Description of network Changes in number of connections Quality of network Social value—connect to/benefit from other grantees? Increased trust Increased sharing resources and information Increased collaboration Feel part of bigger thing Impact on Project Implementation Learning from component—in what specific ways did it build capacity? Application of component—how did grantees apply in own project? Relative instrumental value/benefit of component—how much did it contribute to their ability to implement their project effectively? Satisfaction Satisfaction with quality/utility of component	Administrative data: Grantee feedback on workshop sessions Museum/library and partner surveys additional items for those receiving direct TA support Project team interviews Administrative data: Cohort network mapping artifacts, TA notes Project team interviews

CCI Contribution

8. What conditions support or inhibit development of grantee capacity and changes in practices related to asset-focused, community-driven collaboration?

9. To what degree does CCI contribute to changes in museums, libraries, and their local community partners?

Table B.1 | CCI Contribution to Outcomes (EQ #8 and EQ #9)

Evaluation Questions and Constructs	Indicators	Data sources
What conditions support or inhibit development of museum, library, and grantee capacity and changes in practices related to asset-focused, community-driven collaboration? [Note: the constructs and indicators will be emergent, but we provide examples below]
Institutional characteristics	Leadership/institutional support for library/museum engagement in community-driven change: high, med, low (based on analytic rubric) Resource allocation Institutional readiness for asset-focused community-driven collaboration	Administrative data: TA provider notes, work product summarizing reflective action, grantee and TA provider reports, local theories of change Project team interviews Process tracing focus groups and interviews (during case studies)
Context factors	Community demographics Community institutions Funding environment (including business model) Policy environment	Administrative data: TA provider notes, work product summarizing reflective action, grantee and TA provider reports, local theories of change Project team interviews Process tracing focus groups and interviews (during case studies)
Support and technical assistance	Exposure to TA Utility of TA Uptake of TA Use of tools Peer support
Collective action	Quality of local CCI implementation Commitment of partners Buy-in from community/ partners
To what degree does CCI contribute to changes in museums, libraries, and their local community partners?
Strategies, outcomes, and alternative drivers from initiative and local theories of change (varies across grantee)	Emergent indicators through Process tracing analysis Thematic analysis Qualitative comparative analysis	Local theories of change Rubric scores (including data from surveys, interviews, focus groups, and administrative data) Process tracing results

Internal and Field Learning Questions (not to be included in PRA)

Answers to the learning questions will emerge through analysis of the questions above, along with strategic learning debriefs with the IMLS program team and the Evaluation Steering Team.

Table B.1 | Museum and Library Sector (EQ #10-13)

10. What cross-cutting lessons can be drawn about the role of capacity-building in helping build effective community engagement models that libraries and museums can use to work with diverse communities?

11. How can CCI efforts inform grantmaking and field building strategies?

12. Did the CCI Cohort Evaluation support changes in IMLS learning culture?

13. What are the implications of CCI for scaling the model across agency investment areas?