CCC Staff Key Informant Interview Guide

Form Approved

OMB No. 0920-xxxx

Exp. Date xx/xx/xxxx

Key Informant Interview Guide

Case Study Guide

Evaluation of CDC’s CCC 1017 Program

A. Interview Introduction

First of all, thank you for making time to meet with us today. We know you are a busy person and we appreciate your cooperation.

Before we get started with the interview I would like to tell you about the study we are doing and how we would like for you to be involved, and I’ll give you a chance to ask questions if you have any. Once I am done with that, I would like to confirm that you are willing to participate and then we will get started!

Administer the informed consent form and obtain signed consent and permission to audio record interview if possible.

If consent provided:

Thank you for agreeing to participate!

We are excited to be part of this national effort to advance cancer prevention and control, and we are interested in learning more about your perspective and experiences.

B. Key Informant Information

Role in PSE Change Strategies

1. Could you tell us about how you’ve been involved with policy, systems, and environmental change strategies related to cancer prevention and control in <jurisdiction>?

2. How did you get involved?

3. How long have you been involved?

C. Planning and Assessment (Wave 1)

Stage of the 1017 Program: Preparation Stage, Prioritization Stage

Stage of PSE Change: PSE Development

EQ2. How did 1017 programs build infrastructure required to develop a PSE agenda, evaluation plans, and media plans?

a. Public health staff and resources

b. Partners

c. TA and training

d. Data, evidence, and evaluation

e. Leadership

Topic Elaboration Questions	Interview Topics/Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
What role did the CCC PD play during planning? How did HD leadership support the planning process?	Leadership: What role has the CCC Program Director played in the planning process? How have higher-level leaders in the health department/agency supported the planning process?	Yes	Yes	Yes	?
Were CCC programs able to hire and retain a PSE Coordinator? If not, why? What competencies and skills did the PSE Coordinator bring to the CCC program?	PSE Coordinator: Please tell us about the process of identifying and hiring a PSE Coordinator to assist with 1017 activities. What competencies and skills were most important in choosing the PSE Coordinator? What were the challenges? What facilitated the process?	Yes	?	?	?
What other CCC program staff contributed to the 1017-related activities and in what roles? What competencies and skills did they bring?	Program Staff: How have other CCC program staff contributed to the 1017-related planning activities? Probe for their roles?	Yes	Yes	?
How did the CCC programs identify and engage partners?	Partners: Please tell us about the overall process you used to identify and engage partners for the PSE Workgroup? How have you used contractors to assist with or carry out activities?	Yes	Yes	?	Yes	?
What was the composition of the PSE Workgroup and how did it change over time (size; sectors represented; knowledge, skills, and connections represented)? How was the PSE Workgroup organized? (e.g., steering committee, workgroups, leadership)?	PSE Workgroup: Why were the specific PSE Workgroup members chosen (e.g., CCC staff, CTG, other chronic disease, etc)? How is the PSE Workgroup organized (e.g., steering committee, workgroups, leadership)? How are decisions made within the PSE Workgroup? How has the PSE Workgroup changed over time?	Yes	Yes, if applicable		Yes
What was the process for obtaining individual TA or ad hoc support? Which training methods were considered effective? Which were not considered effective? How did training of CCC staff “trickle down” to partners?	Technical Assistance & Training: What types of technical assistance and training have you used to support the planning process? What TA or training has been the most helpful or valuable? What has not been helpful or valuable? Have partners been able to participate in or benefit from the TA and training that has been provided? If yes, how?	Yes	Yes	Yes	?	?
How were cancer burden data, community input, and environmental scan data used to select priorities and prepare the PSE agenda and media plan? Did partners provide any data or evidence that would not have been available to the program without their involvement in the PSE Workgroup?	Priority Setting and Planning: How did you use cancer burden and other data, community input, and environmental scan results to set priorities for the PSE Agenda? What role have PSE Workgroup members or other partners played in providing data or evidence?		Yes	Yes	Yes

EQ3. How did 1017 programs plan to advance their PSE goals?

a. What PSE priorities were chosen for their PSE agendas (by topic, sector, strategy and stakeholder and decision maker type)?

b. How well did the PSE agendas align with data, evidence, community input?

c. How well did the PSE agendas align with CCC and other chronic disease plans?

d. What level of engagement and support did partners have with plan development and priorities?

e. How responsive was the plan to local context?

Topic Elaboration Questions	Interview Topics/Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
How were PSE change strategies identified and prioritized?	How were the PSE change objectives and strategies selected?	Yes	Yes		Yes
How did priorities differ from pre-1017 priorities? Specifically, how did the mix of primary, secondary, and tertiary prevention strategies differ from pre-1017 priorities?	Which of the strategies in your 1017 PSE Agenda are new for your CCC program?	Yes	?	Yes	?
What characteristics of the local context were taken into account during the plan development process?	What state/local political or economic considerations were taken into account during the planning process?	Yes	Yes	?	Yes

EQ 11. What type of engagement and support did PSE Workgroups have with plan development and priority setting?

a. What were the roles and responsibilities of the PSE Workgroup for plan development and priority setting?

b. How did member composition influence selection of strategies?

Topic Elaboration Questions	Interview Topics/ Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
How and at what level did the PSE Workgroup contribute to the planning process, especially in relation to assessment, priority setting, and selection of strategies?	How did the PSE Workgroup contribute to the planning process? Probe for contributions to: - assessment - priority setting - strategy selection	Yes	Yes	?	Yes
For which steps of the planning process was the PSE Workgroup involvement most or least important?	For what parts of the planning process was the PSE Workgroup most valuable? Why? Were there any parts of the process that the PSE Workgroup did not contribute to? Why?	Yes	Yes	?	Yes
Were there key champions for the priorities within the PSE Workgroup? If so, who were they and how did they mobilize support?	Which members of Workgroup were the key champions for the priorities that were chosen? How were they able to mobilize support for including those priorities in the PSE Agenda?	Yes	Yes	?	Yes
Was there consensus around the priorities? Why or why not?	Was the PSE Workgroup able to achieve consensus around the chosen priorities? Why or why not?	Yes	Yes	?	Yes
Who were the additional partners (non PSE Workgroup) that played a key role in planning? What did they do?	Did other partners who were not part of the PSE Workgroup get involved in the planning process? What role did they play and how important was their contribution?	Yes	Yes	?	Yes
How did 1017 staff engage the PSE Workgroup members at different steps of the planning process?	How were CCC program staff involved in working with or supporting the PSE Workgroup during the planning process?	Yes	Yes	?	Yes

D. Implementation (Wave 1 and 2)

Stage of the 1017 Program: Implementation Stage

Stage of Policy Development: Increased Awareness, Strengthened Support for PSE, PSE Change Implementation

EQ4. What methods were used by 1017 programs to implement the PSE agenda and media plan?

a. By topic

b. By audience

c. Roles of public health staff and partners

NOTE: Refer to the Case Profile for the list of implementation methods outlined in the PSE agenda and media plan. If PSE agenda and media plan are NOT available before site visit, or do not specify methods, then try to obtain copies (electronic preferred) and clarify methods being used during the interview.

Topic Elaboration Questions	Interview Topics/ Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
What methods were actually used to implement the PSE agenda? What role did staff play during implementation? What were the key roles played by partners?	Thinking of [Strategy from site’s PSE agenda, chosen by interviewer]: What steps did you take to implement the strategy? What methods did you use to move things along at each step? Who played a key role(s) at each step? How were these methods received? What was the role of earned media in moving the PSE strategy forward? Did you have to shift your approach at any point? If so, why? What did you do?	Yes	Yes	?	Yes
What strengths did staff/partners bring to the implementation of selected methods?	What were the strengths (skills, resources) the staff (or partners) were able to bring to the implementation of those methods?	Yes	Yes	Yes	Yes	?

EQ12. How did PSE Workgroups facilitate implementation and achievement of PSE change?

a. What were the roles and responsibilities of the PSE Workgroups for implementation of PSE change strategies?

b. How did member composition influence the adoption of PSE changes strategies at the state and local level?

Topic Elaboration Questions	Interview Topics/ Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
What role did the PSE Workgroup play for the implementation of the plan [or PSE agenda and media plan?]?	What has been the role of the PSE Workgroup as a whole in the implementation of the PSE agenda and media plan?	Yes	Yes	?	Yes
What specific knowledge, skills, and relationships did the PSE Workgroup members bring to their roles related to implementation?	How have individual PSE Workgroup members contributed to the implementation of the PSE agenda and media plan? Probe for: - specific knowledge, skills - resources provided - relationships to other stakeholders	Yes	Yes	?	Yes
Who were the additional partners (non PSE Workgroup) that played a key role in implementation? What did they do?	Did other partners who were not part of the PSE Workgroup get involved in implementation (including contractors/subcontractors)? What role did they play and how important was their contribution?	Yes	Yes	?	Yes

EQ5. What factors facilitated or hindered implementation of PSE agendas and media plans? How were barriers addressed by 1017 programs?

a. Role of context: socioeconomic, political, programmatic

b. Coalition functioning (trust, communication, structure, etc)

Topic Elaboration Questions	Interview Topics/ Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
In what ways did external community or political events play a role in the implementation of 1017 priorities? In what ways did internal HD (leadership, staffing, budgets) or programmatic events (CCC, tobacco, other) affect the implementation of 1017 priorities? How did staff and partners respond to those events?	People working on PSE change acknowledge that progress isn’t always steady. Were there any factors that caused the pace of PSE change work to accelerate or stall? Probe for: -Key political or economic events or conditions that made it possible to get traction on an issue? - Opposing or competing issues that made it impossible to get your issue on the decision maker’s agenda? How have the barriers or challenges to implementation been addressed?	Yes	Yes	Yes	Yes	?
How did the CCC coalition support or hinder implementation of the PSE agenda and media plan?	How would you characterize the support of the CCC coalition as a whole for the implementation of the PSE agenda and media plan? Has there been a lack of support or resistance to any of the parts of the PSE agenda or media plan from any of the coalition members? -Which parts? -How?	Yes	Yes	?	Yes	?

EQ6. How were evaluation results used by 1017 programs to shape implementation?

EQ8. What strategies and methods were evaluated by 1017 programs? Which did they find to be most effective?

NOTE: If an evaluation plan is available prior to the site visit, review the plan beforehand and only ask questions where more information is needed. If the evaluation is NOT available prior to the site visit, ask program staff (or relevant partners) if one is available and obtain a copy for project records (electronic format preferable).

Topic Elaboration Questions	Interview Topics/ Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
How were strategies evaluated? What evidence links specific methods to outcomes?	If evaluation plan not available prior to site visit: How are the different strategies of the PSE agenda and media plan being evaluated? Who is conducting the evaluations? Are contractors/subcontractors involved?	Yes	Yes	Yes	?
Did the evaluation plan include an implementation evaluation? What aspects of PSE agenda and media plan implementation were evaluated?	If evaluation plan not available prior to site visit: Does the evaluation plan include an evaluation of the implementation of the different components or activities of the PSE agenda and media plan? This is also called a process evaluation. If so, how have the activities or components been evaluated for implementation?	Yes	Yes	Yes	?
How were evaluation results shared?	How are the evaluation results being used? How are the evaluation results being shared?	Yes	Yes	Yes	Yes	?
What actions were taken as the result of the implementation evaluation?	How have the results of the implementation evaluation been used? Have the results been used to make adjustments or improvements to the PSE agenda or media plan?	Yes	Yes	Yes	?
How do participants perceive the effectiveness of various strategies? Why were specific methods effective?	What methods have been the most effective in achieving the desired results? Why? Which methods have not worked well? Why?	Yes	Yes	?	Yes	?
How well will successful strategies translate to other contexts?	Which ones of the methods do you think could be used successfully by other CCC programs and their partners? Why?	Yes	Yes	?	Yes	?

EQ13. How do states/programs evaluate their partnerships?

Topic Elaboration Questions	Interview Topics/ Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
What methods did they use?	If evaluation plan not available prior to site visit: How have you evaluated the partnerships used or created to undertake the PSE change strategies in <your jurisdiction>? Who is conducting the evaluations? Are contractors/subcontractors involved?	Yes	Yes	Yes	?
What was their capacity for evaluation?	How well prepared was the CCC program <in your jurisdiction> to evaluate the partnerships? How has the program’s capacity to evaluate partnerships changed since the 1017 funding?	Yes	Yes	Yes	?

D. Outcomes (Wave 2)

EQ7. What key PSE outcomes were achieved by 1017 programs?

a. Internal: Increased strategic capacity;

Sustainability of coalitions and strategic partnerships.

b. External: Shifts in awareness, political will, and active support on the part of stakeholders and decision makers;

PSE changes by topic, sector, level of prevention, and chronic disease prevention domain.

Topic Elaboration Questions	Interview Topics/ Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
Were new strategic alliances formed? If so, what were they? What types of collaborative actions among organizations occurred (joint meetings, aligned messages)?	Since 1017 program started, have any new strategic alliances been formed with other organizations or stakeholders in order to achieve the desired PSE changes? [If yes] Could you tell more about those? What kinds of collaborations or activities have resulted from those alliances?	Yes	Yes		Yes	?
Were new partners brought into CCC? If so, who were they? What types of collaborative actions among organizations occurred (joint meetings, aligned messages)?	Since 1017 program started, have any new partners been brought into the CCC program or the CCC coalition in order to achieve the desired PSE changes? [If yes] Could you tell more about those? How have you used contractors/subcontractors? What kinds of collaborations or activities have resulted from those alliances?	Yes	Yes		Yes	?
How were additional resources leveraged (what kind? from where?)	What kinds of new or additional resources been secured to help with the PSE change strategies? Probe for: - What kinds of resources? -From where?	Yes	Yes	?	Yes	?
How were relationships with elected officials affected?	How have relationships between the CCC program or coalition and elected officials changed since undertaking the 1017-related activities? Probe for examples, if any.	Yes	Yes		Yes	Yes
How were strategic partnerships being sustained?	How are the strategic alliances and partnerships being sustained over the long term, if at all?	Yes	Yes		Yes	?
How was capacity built in the 4 chronic disease domains?	How has the capacity to undertake PSE change strategies improved since the 1017 funding? Probe for improvements, if any, for each of the 4 chronic disease domains.	Yes	Yes	Yes	?
Did stakeholder support increase for specific PSE changes?	Has stakeholder support for cancer-related PSE change increased since the 1017 funding?		Yes		Yes	Yes
Did the support of key decision makers around specific PSE change increase?	Has support of key decision makers for cancer-related PSE change increased since the 1017 funding?		Yes		Yes	Yes

EQ9. What PSE changes were achieved at different points in time during the 5-year funding period?

Topic Elaboration Questions	Interview Topics/ Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
What were the early “wins”?	So far, what have been the successes or “wins” for achieving PSE change in <your jurisdiction> since the 1017 funding?	Yes	Yes		Yes
Why were these successes achieved when they were? What were the key factors that led to each success? Prevented earlier success?	What factors have facilitated achieving the success? What factors prevented achieving the success earlier? What factors have prevented achieving success in other areas?	Yes	Yes		Yes
What timelines are reasonable for selected PSE strategies to be achieved?	What is a reasonable timeline for achieving cancer-related PSE change? Are the timelines different for different PSE objectives?	Yes	Yes		Yes

E. Coalition Functioning (Wave 1 and 2)

Stage of the 1017 Program: Preparation Stage, Prioritization Stage, Implementation Stage

Stage of Policy Development: Policy Development, Increased Awareness, Strengthened Support for PSE, PSE Change Implementation

EQ10. What was the relationship between PSE Workgroup, CCC coalitions, and other chronic disease coalitions?

NOTE: For these questions you will need to review the Case Profile for information on the PSE Workgroup.

Topic Elaboration Questions	Interview Topics/ Questions	CCC Program Leadership	PSE Coordinator	CCC Program Staff	Workgroup Members	Community Members
How much and in what way did the PSE Workgroup interact with the CCC coalition during planning? During implementation?	Are members of the PSE Workgroup also members of the CCC coalition? How has the PSE Workgroup interacted and communicated with the CCC coalition? Probe for: - frequency of interactions - interactions during planning stage - interactions during implementation stages	Yes	Yes		Yes
How much and in what way did the PSE Workgroup interact with other coalitions during planning? During implementation?	How has the PSE Workgroup interacted and communicated with the other coalitions in <your jurisdiction>? Probe for: - frequency of interactions - interactions during planning stage - interactions during implementation stages	Yes	Yes		Yes
How did the interaction facilitate or hinder advancement of PSE change?	How has the PSE Workgroup worked with the broader CCC coalition (or other coalitions) to achieve PSC change in <your jurisdiction>? Have there been any challenges with the PSE Workgroup working with the CCC coalition?	Yes	Yes		Yes

Public reporting burden of this collection of information is estimated to vary between 60 and 90 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer, 1600 Clifton Road NE, MS D-74, Atlanta, GA 30333, ATTN: PRA (0920-xxxx).