IDI Guide for Program Managers (Private Sector)

DRAFT

Form Approved

OMB No.: 0920-13xx

Exp. Date: mm/dd/yyyy

Organizations that Receive CDC DP11-1111 Funding

In-depth Interview Guide for Program Managers

Public reporting burden of this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: OMB 0920-13xx.

Evaluation Questions:

• What are the core components of the DP11-1111 cooperative agreement?

• What are the factors that affect the implementation of DP11-1111 programmatic activities?

• What support services and educational resources have organizations developed and/or implemented as a part of their intervention targeting young breast cancer survivors (YBCS)?

• What are the factors that affect the implementation of support services and educational resources?

• How have CDC’s technical assistance and support activities contributed to grantee capacity and sustainability efforts?

• How have organization’s interventions affected awareness for and access to support services and educational resources among YBCS, providers, and caregivers?

• How have DP11-1111 grantees’ interventions affected knowledge, attitudes and behaviors among YBCS, providers, and caregivers related to the risks of breast cancer in young women?

Domain	Subdomain
Organizational Context	Mission
	History
	Setting/Infrastructure
	Role in organization
Implementation of the Cooperative Agreement (CA)	Core components of the CA
	Training/Technical Assistance (TA)
	Resources
	Facilitators
	Barriers
Implementation of the YBCS Intervention	Intervention Overview, History and Rationale
	Intervention execution
	Implementation Fidelity
	Resources
	Facilitators
	Barriers
Evaluation	Knowledge
	Measurement
	Facilitators
	Barriers
Partnerships	Contribution to the intervention
	Development and Engagement
Capacity and Sustainability	Capacity to meet the requirement of the CA
	Capacity to implement intervention
	Capacity to evaluate
	Sustainability of intervention
Perceptions	Awareness
	Access
	Utilization
	YBCS knowledge, attitudes, and beliefs (KAB)
	Value of the CA
	Value of the YBCS intervention
	Lessons learned
	Recommendations

Organizations that Receive CDC DP11-1111 Funding

In-depth Interview Guide for Program Managers

Domain	Sub-Domain	Questions	Probes
Organizational Context	Role in organization	How would you describe your role within your organization?	How long have you worked at [insert name of organization]? How did you originally get involved?
Implementation of the Cooperative Agreement (CA)	Core Components of the DP11-1111 Cooperative Agreement	Describe your participation in the DP11-1111 cooperative agreement (CA).	How does your program implement the requirements of the CA? How would describe your organization’s experience with participating in the CA?
		Describe the process your organization has used to manage and oversee the CA.	Describe the process your organization used to develop a communication plan. How does your organization use the communication plan? Describe the process your organization used to develop a management plan. How does your organization use the management plan? Describe your experience participating in CDC meetings?
		Describe the process your organization used to conduct a needs assessment.	What methods did your organization use to conduct the needs assessment?
		Describe if and how the CA affected your organization’s ability to create new or strengthen existing partnerships.	What new partnerships were formed? What existing partnerships were enhanced?
		Describe the process you organization used to create a program plan.
		Describe how your organization has worked to implement the program plan.
		Does your program have an evaluation plan? If so, describe the process your organization used to create the evaluation plan.
		Describe how your organization has worked to implement the evaluation plan.
		Does your program have a sustainability plan? If so, describe the process your organization used to create a sustainability plan.
	Training/Technical Assistance	What types of trainings has your organization participated in to help with the management of the CA?	How were these trainings delivered? How often did your organization participate in these trainings? Who delivered the trainings? How were you able to apply the training you received to help with program implementation and/or evaluation?
		What kinds of technical assistance (TA) has your organization received as a part of the CA.	How was the TA delivered? How were you able to apply the TA received to help with program implementation and/or evaluation?
		Describe your organization’s engagement with your CDC program consultant(s).	How often do you communicate? What are the primary methods of communication? What kinds of information does your program consultant share? How often did your organization receive TA from CDC assigned program consultant?
	Resources	Describe the resources that your organization has used to implement the CA.	What are each staff member’s key tasks for the CA? Describe technology that supports your organization’s participation in the CA? Describe other materials that support your organization’s participation in the CA.
	Facilitators	What factors, if any, have facilitated your organization’s participation in the CA?	What internal organizational factors (e.g., staff, resources, expertise) have helped your organization participate in the CA? What external factors (e.g., training, TA, partnerships) have helped your organization participate in the CA?
	Barriers	What factors, if any, have challenged your organization’s participation in the CA?	What internal organizational factors (e.g., staff, resources, expertise) have challenged your organization’s participation in the CA? What external factors (e.g., training, TA, partnerships) have challenged your organization’s participation in the CA?
		What strategies has your organization used to overcome barriers/challenges related to the CA?
Implementation of the YBCS Intervention	Intervention Overview, History and Rationale	Describe your organization’s YBCS intervention that is funded through the CA.	What are the key components of the YBCS intervention? Who is the target audience for the YBCS intervention? How have the key components of the YBCS intervention been tailored to meet the unique needs of the target audience?
		Describe your organization’s goals and objectives for the YBCS intervention.	How would you describe the vision of the YBCS intervention and how does this align with what your organization wants to achieve?
		How did activities conducted as a part of the CA guide the development of the YBCS intervention?	How did the needs assessment inform the development of the YBCS intervention? How did TA and/or training received from CDC inform the development of the YBCS intervention?
		Describe the evidence base and/or factors that informed (e.g., behavioral/communication theory, scientific evidence, guidelines, formative research) the development of the YBCS intervention.	What is the theoretical foundation for the YBCS intervention?
		What intervention components are new as a result of the CA?
		What intervention components were enhanced/expanded as a result of the CA?
		Describe your role in the YBCS intervention.	How long have you been involved with the YBCS intervention? Approximately how much time per week do you spend working on the organization’s YBCS intervention?
	Intervention execution	Describe how the YBCS intervention is implemented.	How often is each component of the YBCS intervention implemented? What is the duration of the YBCS intervention? Who is responsible for delivering key components of the YBCS intervention? How are YBCS intervention participants recruited? How has the delivery of the YBCS intervention been tailored to meet the unique needs of the target audience? How do you engage members of the target audience (e.g., YBCS, families/caregivers, health care providers) in the implementation of the YBCS intervention?
		What communication strategies is your organization using to promote or implement the YBCS intervention?	How is mass media (e.g., radio, newspaper, Internet) being utilized to deliver or promote YBCS intervention components? How is small media (e.g., newsletter, email, brochures) being utilized to deliver or promote YBCS intervention components? How is social media (e.g., Facebook, Twitter) being utilized to deliver or promote YBCS intervention components? How is earned media (e.g., free or donated media) being utilized to deliver or promote YBCS intervention components?
		What tools, guidelines and/or protocols are used to direct the implementation of the YBCS intervention?	What, if any, manuals have been created to support implementation of the YBCS intervention? Who created these guidelines? How are these used to implement the YBCS intervention?
		What trainings are offered to staff to implement the YBCS intervention?	How are trainings identified? Who delivers these trainings? What is the duration of each of these trainings? How often are trainings offered to staff? Are there external trainings that your organization has participated in to support implementation of the YBCS intervention?
	Implementation Fidelity	To what extent has the YBCS intervention been implemented as planned?	How has the YBCS intervention changed over time? What caused changes in the implementation the YBCS intervention?
		How does implementation of the YBCS intervention vary across sites?	Are there differences in the way that the YBCS intervention is delivered across sites (e.g., setting, geographic location, target population)? If so, how is implementation different?
		Does your organization use any process improvement strategies to improve the delivery of the YBCS interventions?
	Resources	How does your organization use funding received through the CA to implement the YBCS intervention?
		What other, if any, monetary(leveraged) or non-monetary(in-kind) resources have been provided to your organization to support the YBCS intervention? If so, please explain.	Have you received additional funding from other organizations to help support implementation of the YBCS intervention? Have you received donated resources, such as materials and/or supplies, to help support implementation of the YBCS intervention? Have people offered their time and/or expertise to help support implementation of the YBCS intervention? Have you received any in-kind, earned media donations to promote and/or implement the YBCS intervention?
	Facilitators	What factors, if any, have facilitated your organization’s implementation of the YBCS intervention?	What internal organizational factors (e.g., leadership, staff, resources, expertise) have helped your organization implement the YBCS intervention? What external factors (e.g., training, TA, partnerships) have helped your organization implement the YBCS intervention?
	Barriers	What factors, if any, have challenged your organization’s implementation of the YBCS intervention?	What internal organizational factors (e.g., leadership staff, resources, expertise) have challenged your organization’s implementation of the YBCS intervention? What external factors (e.g., training, TA, partnerships) have challenged your organization’s implementation of the YBCS intervention?
		What strategies have your organization used to overcome barriers related to implementation of the YBCS intervention?
Evaluation	Knowledge	Describe your understanding of program monitoring and/or evaluation.	Have you participated in any past evaluation efforts? If so, please describe. Do you have experience creating program logic models?
	Measurement	For what the YBCS intervention is trying to accomplish, how would you define success?	How has the definition of success for this YBCS intervention changed over the time? How? Why?
		How is success measured or documented?	Do you have baseline data? Do you intend to collect follow-up data? At what time? Do you collect data at specified points over time (time series)? What length of time? Do you follow a sample of participants over time (cohort study)? What length of time? Do you have a comparison group? What does the evaluation activity tell you about the success of the YBCS intervention?
		Describe how your organization defines and assesses reach for the YBCS intervention.
		Describe how your organization measures the implementation of the YBCS intervention.	What data do you collect? Who is responsible for measuring/monitoring implementation? Are other staff members involved in data collection? How so? Are there plans to expand or change how your organization monitors implementation of the YBCS intervention? If so, please explain?
		What are the available data sources used to support program monitoring and evaluation for the YBCS intervention?	What data collection tools or instruments does your organization use in relation to the YBCS intervention? (Request to see or obtain copies of data, data collection tools, or evaluation reports.) Describe the data collection methods used to collect evaluation data for the YBCS intervention.
		How does your organization analyze data from the YBCS evaluation in order to collect program monitoring and evaluation?	What analytic techniques does your organization use to conduct program monitoring and evaluation for the YBCS intervention? Who within your organization, who is responsible for conducting analysis for the YBCS intervention?
		How does your organization use evaluation findings to inform implementation of the YBCS intervention?
		Describe how your organization shares evaluation findings from the YBCS intervention.	Do you provide feedback to the community or the people you serve about your evaluation findings? Do you share evaluation findings with partners or other relevant stakeholders?
	Facilitators	What, if any, facilitators have helped your organization conduct program monitoring and evaluating for the YBCS intervention?
	Barriers	What, if any, barriers has your organization encountered in evaluating the YBCS intervention?
		What strategies has your organization used to overcome barriers related to program monitoring and evaluation of the YBCS intervention?
Partnerships	Development and Engagement	Describe the process your organization has used to identify and develop partnerships.	Did you develop formal agreements (e.g., MOUs, contracts) with partners?
		How has your organization’s involvement in the CA helped you to develop and maintain partnerships?	Have you collaborated with other grantees as a part of the CA? Has access to other programs serving YBCS helped your organization develop new partnerships? If so, please explain.
		How would you describe partners’ engagement in the YBCS intervention?	What are the expectations for partners’ involvement in the YBCS intervention?
	Contribution to the intervention	Describe how partners participate in the planning or implementation of the YBCS intervention.	How does your organization interact with partners? How does your organization communicate with partners? How often does your organization communicate with partners? How long has your organization worked partners? What YBCS intervention components do partners support?
		What partnerships have been the most beneficial for your organization in the implementation of the YBCS intervention?	Please briefly describe the benefits of this relationship.
		What additional partnerships would [insert Grantee Organization name] benefit from to support the implementation of the YBCS intervention?
Capacity and Sustainability	Capacity to meet the requirement of the CA.	Describe your organization’s capacity to meet the requirements of the CA and how capacity has changed over time.
		Describe how support received from CDC through the CA has affected your organization’s ability to meet the requirements of the CA.	To what extent was your organization able to utilize CDC’s TA to enhance reporting? To what extent was your organization able to utilize CDC training opportunities to enhance reporting?
	Capacity to implement intervention	Describe your organization’s capacity to implement the YBCS intervention and how capacity has changed over time.
		Describe how the CA has enabled your organization to implement the YBCS intervention.	To what extent was your organization able to utilize CDC’s TA to improve the implementation of the YBCS intervention? To what extent was your organization able to utilize CDC training opportunities to improve the implementation of the YBCS intervention?
	Capacity to evaluate	Describe your organization's capacity to plan and implement program monitoring and evaluation activities, such as creating data collection tools, collecting data, conducing analysis, and reporting.	Describe program staff members who have the expertise/experience to collect and/or analyze data. What additional outside resources does your organization rely on to help with evaluation (e.g., does your organization use an external evaluator)? Describe the resources your organization reserves to support data collection (e.g., time, materials, technology) to support program monitoring and/or evaluation? What other resources does your organization designate for program evaluation/monitoring (e.g., staff, time, technology, materials)?
		Describe how the CA has enabled your organization to plan and implement program monitoring and evaluation activities for the YBCS intervention.	To what extent was your organization able to utilize CDC’s TA to improve the program monitoring and evaluation for the YBCS intervention? To what extent was your organization able to utilize CDC training opportunities to improve the program monitoring and evaluation for the YBCS intervention?
	Sustainability of intervention	Describe the sustainability of the YBCS intervention and how sustainability has changed over time.	What factors would enhance the sustainability of the YBCS intervention?
		Describe how the CA has enabled your organization to plan for and ensure the sustainability of the YBCS intervention?	To what extent has your organization able to utilize CDC’s TA to improve the sustainability of your YBCS intervention? To what extent has your organization been able to utilize CDC’s training opportunities to improve the sustainability of the YBCS intervention?
Perceptions	Awareness	How has the YBCS intervention affected awareness for the existence of support services and/or educational resources among the target audience?	In your opinion, what aspect of the YBCS intervention has been the most effective in increasing awareness for support services and/or educational resources? In your opinion, what communication strategies were the most effective in promoting and increasing awareness for the YBCS intervention? Does your organization have data to support the effectiveness of this? If so, explain how this data is used. (Request to see or obtain copies of data, data collection tools, or reports pertaining to how a program has assessed awareness.)
	Access	How has the YBCS intervention affected access to support services and/or educational resources for the target audience?	In your opinion, what aspect of the YBCS intervention has been the most effective in increasing access to support services and/or educational resources? Does your organization have data to support the effectiveness of this? If so, explain how this data is used. (Request to see or obtain copies of data, data collection tools, or reports pertaining to how a program has assessed access.)
	Utilization	How has the YBCS intervention affected utilization of support services and/or educational resources by the target audience?	In your opinion, what aspect of the YBCS intervention has been the most effective in increasing utilization of support services and/or educational resources? Does your organization have data to support the effectiveness of this? If so, explain how this data is used. (Request to see or obtain copies of data, data collection tools, or reports pertaining to how a program has assessed utilization.)
	YBCS knowledge, attitudes, and beliefs (KAB)	What knowledge, attitudes, and beliefs does the YBCS intervention aim to affect? How has the YBCS intervention impacted those specific knowledge, attitudes, and/or beliefs among the target population? (Examples of KAB targets include: strategies for reducing the risk of recurrence and the development of new malignancies, chronic disease onset, and improving overall health and quality of life).	In your opinion, what aspect of the YBCS intervention has been the most effective in KAB among the target population? Does your organization have data to support the effectiveness of this? If so, explain how this data is used. (Request to see or obtain copies of data, data collection tools, or reports pertaining to how a program has assessed KAB among target audience(s).)
	Value of the CA	What has been the value for your organization to participate in the CA?
	Value of the YBCS intervention	What has been the value of the YBCS intervention?	In your opinion, how did the YBCS intervention impact the target audience? In your opinion, how did the YBCS intervention contribute to the evidence base for promising strategies to reach YCBS, YBCS caregivers, and/or YBCS health care providers?
	Lessons learned	What have been some key lessons learned from your organization’s experience with the CA?	In your opinion, what aspects of your CA worked well? In your opinion, what aspects of your CA did not work well?
		What have been some key lessons learned from your organization’s experience with the YBCS intervention?	In your opinion, what aspects of your organization’s YBCS intervention worked well? In your opinion, what aspects of your organization’s YBCS intervention did not work well?
	Recommendations	What ideas do you have for improving your organization’s YBCS intervention?