IDI Guide for Program Managers (Private Sector)

DRAFT

Form Approved

OMB No.: 0920-13xx

Exp. Date: mm/dd/yyyy

Organizations that do not Receive CDC DP11-1111 Funding

In-depth Interview Guide for Program Managers

Public reporting burden of this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: OMB 0920-13xx

Evaluation Questions:

• What are the core components of the DP11-1111 cooperative agreement?

• What are the factors that affect the implementation of DP11-1111 programmatic activities?

• What support services and educational resources have organizations developed and/or implemented as a part of their intervention targeting young breast cancer survivors (YBCS)?

• What are the factors that affect the implementation of support services and educational resources?

• How have CDC’s technical assistance and support activities contributed to grantee capacity and sustainability efforts?

• How have organization’s interventions affected awareness for and access to support services and educational resources among YBCS, providers, and caregivers?

• How have DP11-1111 grantees’ interventions affected knowledge, attitudes and behaviors among YBCS, providers, and caregivers related to the risks of breast cancer in young women?

Domain	Subdomain
Organizational Context	Mission
	History
	Setting/Infrastructure
	Role in organization
Implementation of the Cooperative Agreement (CA)	Core components of the CA
	Training/Technical Assistance (TA)
	Resources
	Facilitators
	Barriers
Implementation of the YBCS Intervention	Intervention Overview, History and Rationale
	Intervention execution
	Implementation Fidelity
	Resources
	Facilitators
	Barriers
Evaluation	Knowledge
	Measurement
	Facilitators
	Barriers
Partnerships	Contribution to the intervention
	Development and Engagement
Capacity and Sustainability	Capacity to meet the requirement of the CA
	Capacity to implement intervention
	Capacity to evaluate
	Sustainability of intervention
Perceptions	Awareness
	Access
	Utilization
	YBCS knowledge, attitudes, and beliefs (KAB)
	Value of the CA
	Value of the YBCS intervention
	Lessons learned
	Recommendations

Organizations that do not Receive CDC DP11-1111 Funding

In-depth Interview Guide for Program Managers

Domain	Subdomain	Questions	Probes
Organizational Context	Role in organization	How would you describe your role within your organization?	How long have you worked at [insert name of organization]? How did you originally get involved?
Implementation of the YBCS Intervention	Intervention Overview, History and Rationale	Describe your organization’s YBCS intervention.	What are the key components of the YBCS intervention? Who is the target audience for the YBCS intervention? How have the key components of the YBCS intervention been tailored to meet the unique needs of the target audience? Was this YBCS intervention adapted from an already existing program or intervention?
		Describe your organization’s goals and objectives for the YBCS intervention.	How would you describe the vision of the YBCS intervention and how does this align with what your organization wants to achieve?
		Describe the evidence base and/or factors that informed (e.g., behavioral/communication theory, scientific evidence, guidelines, formative research) the development of the YBCS intervention.	What is the theoretical foundation for the YBCS intervention?
		Describe your role in the YBCS intervention.	How long have you been involved with the YBCS intervention? Approximately how much time per week do you spend working on the organization’s YBCS intervention?
	Intervention execution	Describe how the YBCS intervention is implemented.	How often is each component of the YBCS intervention implemented? What is the duration of the YBCS intervention? Who is responsible for delivering key components of the YBCS intervention? How are YBCS intervention participants recruited? How has the delivery of the YBCS intervention been tailored to meet the unique needs of the target audience? How do you engage members of the target audience (e.g., YBCS, families/caregivers, health care providers) in the implementation of the YBCS intervention?
		What communication strategies is your organization using to promote or implement the YBCS intervention?	How is mass media (e.g., radio, newspaper, Internet) being utilized to deliver or promote YBCS intervention components? How is small media (e.g., newsletter, email, brochures) being utilized to deliver or promote YBCS intervention components? How is social media (e.g., Facebook, Twitter) being utilized to deliver or promote YBCS intervention components? How is earned media (e.g., free or donated media) being utilized to deliver or promote YBCS intervention components?
		What tools, guidelines and/or protocols are used to direct the implementation of the YBCS intervention?	What, if any, manuals have been created to support implementation of the YBCS intervention? Who created these guidelines? How are these used to implement the YBCS intervention?
		What trainings are offered to staff to implement the YBCS intervention?	How are trainings identified? Who delivers these trainings? What is the duration of each of these trainings? How often are trainings offered to staff? Are there external trainings that your organization has participated in to support implementation of the YBCS intervention?
	Implementation Fidelity	To what extent has the YBCS intervention been implemented as planned?	How has the YBCS intervention changed over time? What caused changes in the implementation the YBCS intervention?
		How does implementation of the YBCS intervention vary across sites?	Are there differences in the way that the YBCS intervention is delivered across sites (e.g., setting, geographic location, target population)? If so, how is implementation different?
		Does your organization use any process improvement strategies to improve the delivery of the YBCS interventions?
	Resources	How does your organization use funding to implement the YBCS intervention?
		What other, if any, non-monetary resources have been provided to your organization to support the YBCS intervention? If any, please explain.	Have you received donated resources, such as materials and/or supplies, to help support implementation of the YBCS intervention? Have people offered their time and/or expertise to help support implementation of the YBCS intervention? Have you received any in-kind, paid media donations to promote and/or implement the YBCS intervention?
	Facilitators	What factors, if any, have facilitated your organization’s implementation of the YBCS intervention?	What internal organizational factors (e.g., leadership, staff, resources, expertise) have helped your organization implement the YBCS intervention? What external factors (e.g., training, TA, partnerships) have helped your organization implement the YBCS intervention?
	Barriers	What factors, if any, have challenged your organization’s implementation of the YBCS intervention?	What internal organizational factors (e.g., leadership staff, resources, expertise) have challenged your organization’s implementation of the YBCS intervention? What external factors (e.g., training, TA, partnerships) have challenged your organization’s implementation of the YBCS intervention?
		What strategies has your organization used to overcome barriers related to implementation of the YBCS intervention?
Evaluation	Knowledge	Describe your understanding of program monitoring and/or evaluation.	Have you participated in any past evaluation efforts? If so, please describe. Do you have experience creating program logic models?
	Measurement	For what the YBCS intervention is trying to accomplish, how would you define success?	How has the definition of success for this YBCS intervention changed over the time? How? Why?
		How is success measured or documented?	Do you have baseline data? Do you intend to collect follow-up data? At what time? Do you collect data at specified points over time (time series)? What length of time? Do you follow a sample of participants over time (cohort study)? What length of time? Do you have a comparison group? What does the evaluation activity tell you about the success of the YBCS intervention?
		Describe how your organization defines and assesses reach for the YBCS intervention.
		Describe how your organization measures the implementation of the YBCS intervention.	What data do you collect? Who is responsible for measuring/monitoring implementation? Are other staff members involved in data collection? How so? Are there plans to expand or change how your organization monitors implementation of the YBCS intervention? If so, please explain?
		What are the available data sources used to support program monitoring and evaluation for the YBCS intervention?	What data collection tools or instruments does your organization use in relation to the YBCS intervention? (Request to see or obtain copies of data, data collection tools, or evaluation reports.) Describe the data collection methods used to collect evaluation data for the YBCS intervention.
		How does your organization analyze data from the YBCS evaluation in order to collect program monitoring and evaluation?	What analytic techniques does your organization use to conduct program monitoring and evaluation for the YBCS intervention? Who within your organization, who is responsible for conducting analysis for the YBCS intervention?
		How does your organization use evaluation findings to inform implementation of the YBCS intervention?
		Describe how your organization shares evaluation findings from the YBCS intervention.	Do you provide feedback to the community or the people you serve about your evaluation findings? Do you share evaluation findings with partners or other relevant stakeholders?
	Facilitators	What, if any, facilitators have helped your organization conduct program monitoring and evaluating for the YBCS intervention?
	Barriers	What, if any, barriers has your organization encountered in evaluating the YBCS intervention?
		What strategies has your organization used to overcome barriers related to program monitoring and evaluation of the YBCS intervention?
Partnerships	Development and Engagement	Describe the process your organization has used to identify and develop partnerships.	Did you develop formal agreements (e.g., MOUs, contracts) with partners?
	Contribution to the intervention	Describe how partners participate in the planning or implementation of the YBCS intervention.	How does your organization interact with partners? How does your organization communicate with partners? How often does your organization communicate with partners? How long has your organization worked partners? What YBCS intervention components do partners support?
		What partnerships have been the most beneficial for your organization in the implementation of the YBCS intervention?	Please briefly describe the benefits of this relationship.
		What additional partnerships would [insert Nonfunded Organization name] benefit from to support the implementation of the YBCS intervention?
Capacity and Sustainability	Capacity to implement intervention	Describe your organization’s capacity to implement the YBCS intervention and how capacity has changed over time.
	Capacity to evaluate	Describe your organization's capacity to plan and implement program monitoring and evaluation activities, such as creating data collection tools, collecting data, conducing analysis, and reporting.	Describe program staff members who have the expertise/experience to collect and/or analyze data. What additional outside resources does your organization rely on to help with evaluation (e.g., does your organization use an external evaluator)? Describe the resources your organization reserves to support data collection (e.g., time, materials, technology) to support program monitoring and/or evaluation? What other resources does your organization designate for program evaluation/monitoring (e.g., staff, time, technology, materials)?
	Sustainability of intervention	Describe the sustainability of the YBCS intervention and how sustainability has changed over time.	What factors would enhance the sustainability of the YBCS intervention?
Perceptions	Awareness	How has the YBCS intervention affected awareness for the existence of support services and/or educational resources among the target audience?	In your opinion, what aspect of the YBCS intervention has been the most effective in increasing awareness for support services and/or educational resources? In your opinion, what communication strategies were the most effective in promoting and increasing awareness for the YBCS intervention? Does your organization have data to support the effectiveness of this? If so, explain how this data is used. (Request to see or obtain copies of data, data collection tools, or reports pertaining to how a program has assessed awareness.)
	Access	How has the YBCS intervention affected access to support services and/or educational resources for the target audience?	In your opinion, what aspect of the YBCS intervention has been the most effective in increasing access to support services and/or educational resources? Does your organization have data to support the effectiveness of this? If so, explain how this data is used. (Request to see or obtain copies of data, data collection tools, or reports pertaining to how a program has assessed access.)
	Utilization	How has the YBCS intervention affected utilization of support services and/or educational resources by the target audience?	In your opinion, what aspect of the YBCS intervention has been the most effective in increasing utilization of support services and/or educational resources? Does your organization have data to support the effectiveness of this? If so, explain how this data is used. (Request to see or obtain copies of data, data collection tools, or reports pertaining to how a program has assessed utilization.)
	YBCS knowledge, attitudes, and beliefs (KAB)	What knowledge, attitudes, and beliefs does the YBCS intervention aim to affect? How has the YBCS intervention impacted those specific knowledge, attitudes, and/or beliefs among the target population? (Examples of KAB targets include: strategies for reducing the risk of recurrence and the development of new malignancies, chronic disease onset, and improving overall health and quality of life).	In your opinion, what aspect of the YBCS intervention has been the most effective in KAB among the target population? Does your organization have data to support the effectiveness of this? If so, explain how this data is used. (Request to see or obtain copies of data, data collection tools, or reports pertaining to how a program has assessed KAB among target audience(s).)
	Value of the YBCS intervention	What has been the value of the YBCS intervention?	In your opinion, how did the YBCS intervention impact the target audience? In your opinion, how did the YBCS intervention contribute to the evidence base for promising strategies to reach YCBS, YBCS caregivers, and/or YBCS health care providers?
	Lessons learned	What have been some key lessons learned from your organization’s experience with the YBCS intervention?	In your opinion, what aspects of your organization’s YBCS intervention worked well? In your opinion, what aspects of your organization’s YBCS intervention did not work well?
	Recommendations	What ideas do you have for improving your organization’s YBCS intervention?

5