Form Approved
OMB No. 0920-XXXX
Expiration Date: XX/XX/XXXX
Question Bank
Improving the Quality and Delivery of CDC’s Heart Disease and Stroke Prevention Programs
Public reporting burden of this collection of information is estimated to average 30 – 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-XXXX)
INSTRUCTIONS TO USERS
Introduction
The Division for Heart Disease and Stroke Prevention (DHDSP) has developed a coordinated, integrated, and focused evaluation plan to assess its current efforts in reducing the burden of heart disease and stroke. The evaluation plan is guided by three evaluation questions:
Is the Division engaged in the most appropriate activities (Relevance)?
Are these activities well implemented (Quality)?
Is the work of the Division having the intended effect (Impact)?
In order to aid in the implementation of its Evaluation Plan, the DHDSP has received a generic OMB clearance, “Improving the Quality and Delivery of CDC’s Heart Disease and Stroke Prevention Programs” to conduct a series of information collections based on the following reference set of questions that address relevance, quality and impact of DHDSP trainings, technical assistance, and guidance. Respondents will be the DHDSP’s partners in state and local government, partner organizations in the private sector including public health organizations, nonprofit organizations, and professional organizations, as well as academic institutions.
Requirements for Use of the Generic Clearance
Each prospective user of the generic OMB clearance “Improving the Quality and Delivery of CDC’s Heart Disease and Stroke Prevention Programs” must agree to follow these guidelines and instructions.
1. For each proposed data collection, the user must complete a “Data Collection Request Form”. All proposed data collections will be reviewed by the DHDSP project contact as well as the NCCDPHP OMB contact. The DHDSP contact will ensure that the proposed data collection is consistent with the DHDSP evaluation plans and goals, as outlined in 0920-XXXX. In addition, the NCCDPHP OMB contact will review each proposed data collection to ensure that it conforms to OMB regulatory requirements. You may NOT begin data collection until you have written approval from these individuals.
2. Questions for data collection must be exclusively drawn verbatim from the Question Bank pre-approved by OMB.
3. Data collection instruments must be pilot tested and the time burden on pilot test respondents must be recorded. Surveys must be limited to 30 minutes and interviews and focus groups must be limited to one hour.
Further information on how to use the Generic Clearance can be found in the “Generic Clearance Users Guide” (See Attachment 9).
How to Select Questions from the Question Bank
Questions for data collection must be exclusively drawn from the Question Bank pre-approved by OMB. The Question Bank is arranged into six sections:
Screening Questions: This section includes questions that are used to determine if the respondent is eligible to participate in a survey, focus group, or interview. These questions are asked before beginning a survey, focus group, or interview, for example, to make sure a respondent has used the CDC product or service that is the topic of the data collection.
Relevance Questions: This section contains questions designed to assess the extent to which DHDSP’s training, technical assistance, and other guidance products are perceived as relevant and to assess needs for additional products and services.
Quality Questions: This section contains questions designed to assess the perceived quality of DHDSP’s training, technical assistance, and other guidance products.
Impact Questions: This section contains questions designed to assess the effectiveness and impact of DHDSP activities
Follow-Up Questions: This section contains questions which invite the respondent to further explain an answer. In a focus group setting, follow-up questions are used as probes to gain a fuller understanding of a respondent’s initial answer.
Demographics Questions: This section includes descriptive questions such as work experience, job role, education level, and funding status. When included in an instrument, these questions should be placed near the end, as they can be perceived as sensitive by some respondents.
Further information about the types of questions found in each section is contained in the introductory text to each of the six major sections of the question bank. In addition, the relevance, quality, and impact sections all contain additional explanations about the types of questions contained within each of their sub-domains.
It is important that ALL questions you choose to include in your instrument are drawn verbatim from the question bank.
How to Fill in the Brackets Contained in Questions
Many questions contain brackets which need to be completed with name of the product, the type of technical assistance, or the name of the training of interest. For example,
Was the [name of product, type of technical assistance, name of training/ workshop] appropriate for the skills you have?
If the survey was asking about the DHDSP Cost Calculator, the question blanks would need to be filled in to word the question as follows:
Was the DHDSP Cost Calculator appropriate for the skills you have?
How to Format Questions
Within each sub-section of the Question Bank (indicated by capital letters), questions are divided into blocks. The first block are questions that are worded in a closed ended, or “yes/no,” format. For example,
Does the DHDSP Cost Calculator address a specific problem or need?
Yes
No
Don’t know/Not sure
As it is worded, the response options to this question are “yes” and “no” and “don’t know/not sure.” However, this type of question can be transformed into two alternate formats, depending on your needs. First, they can be transformed into a scale format. This can be done by adding the phrase “To what extent” in front of the question. For example,
To what extent does the DHDSP Cost Calculator address a specific problem or need?
Does not address a need at all
Barely addresses a need
Somewhat addresses a need
Fully addresses a need
Don’t know/Not sure
On the left side of the page there is a column that indicates whether you may add the phrase “to what extent”. You may only add “to what extent” to questions with an “X” in the “OK to add ‘To what extent’” column. The question must be kept verbatim and only the phrase “to what extent” may be added at the beginning.
You do have flexibility in selecting the response options. The response options in the above example include any number between 1 and 5; however, you can use a different range of numbers. The most common types of scales include a 4 point scale (as shown above with four “scaled” responses and an additional response for “don’t know/not sure”), a 5 point scale, and a 10 point scale.
If you are looking to get more in-depth information, you can also choose to add on a follow-up question after some of the questions. For example,
Does the DHDSP Cost Calculator address a specific problem or need? Please explain.
The “please explain” is a follow-up question, taken from the “follow up question” section of the Question Bank (Section 5). On the left side of the page there is a column that indicates whether you may add a follow-up question. You may only add a follow-up question to questions with an “X” in the “OK to add Follow-Up” column. The question must be kept verbatim and you may only use follow-up questions that are included in section 5 of the question bank.
Follow up questions are often more time consuming for respondents to answer. Therefore, the number of open-ended questions you include in any instrument should be limited.
Pre-Testing the Instrument
Example instruments can be found in Attachments 6, 7, and 8 of this OMB Generic Clearance Packet. When putting together instruments, it is important to keep in mind the burden hours placed on respondents. Surveys must be limited to 30 minutes and interviews and focus groups must be limited to one hour, including time dedicated to reading instructions. To give you a sense of the proper length of instruments,
Attachment 6, an example web-based survey, asks respondents who have accessed evaluation technical assistance in the last six months to answer 20 closed-ended and 9 open-ended questions. This survey takes about 30 minutes to complete.
Attachment 7, an example interview script, asks respondents 5 closed-ended and 21 open-ended questions. This interview takes about 1 hour to complete
Attachment 8, an example focus group guide, asks focus group participants to discuss 9 open-ended questions. This focus group lasts about 1 hour.
Once an instrument is developed, it is important to pilot test it with either a few members of the target population (i.e. State epidemiologists) or, if such a group cannot be accessed, individuals in the DHDSP who work closely with the target population (i.e. epidemiologists in the DHDSP). Pilot testing is important for two reasons 1) to ensure compliance with time limitations, as described above, and 2) to make sure the instrument is understandable and clear. Modifications should be made based on feedback received from the pilot test before distributing to the target population.
I. SCREENING QUESTIONS
This section includes questions that are used to determine if the respondent is eligible to participate in a survey, focus group, or interview.
Which of the following trainings have you participated in?
Did you attend the [name of training]?
Did you participate in the [name of training/workshop]?
Did you have a leadership role in the [name of training/workshop]?
Which plenary sessions did you attend during the [name of training]?
Which workshop(s) did you attend during the [name of training/session]?
Have you received any products (e.g., printed, web, or other) from the DHDSP?
Have you viewed [name of product] over the last [time frame]?
Have you seen [name of product]?
Have you used [name of product] over the last [time frame]?
Have you used [name of product]?
When did you last [use, access, attend] the [name of product, type of technical assistance, name of training/workshop]?
Did you participate in the development of [name of product]?
Did you participate in planning or implementing the [name of product, name of training/workshop]?
Did you author or co-author the [name of product]?
Which format of the [name of product] have you used most?
Which sections of the [name of product] have you used?
What DHDSP products have you used?
Are you aware of the [name of product]?
Have you read the [name of product]?
To your knowledge, who [sponsored/funded] the [name of product, name of training/ workshop]?
Have you [accessed/received] any [type of technical assistance] over the last [time frame]?
Have you [accessed/received] any [type of technical assistance] over the last [time frame] on how to use information from the [name of training/workshop, name of product]?
II. RELEVANCE
The DHDSP needs to know if it is engaged in the most appropriate activities, whether the training, technical assistance, and other guidance products and services that it currently offers are the proper ways to achieve its goals. Furthermore, the DHDSP needs to assess whether its services are perceived as relevant by its State and National partners. For the purpose of this evaluation, the following dimension of relevance will be explored: clear purpose, clear audience, solve a problem, novel, easy to access, timely, appropriate level, pertinent, transparency, and improvements.
C
OK to add Follow-Up
OK to add “To what
extent”
[
X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X
[Is/Was] the purpose of the [name of product, name of training, type of technical assistance] communicated accurately?
[Is/Was] the purpose of the [name of product, name of training/workshop] clear?
Did [name of product, name of training, type of technical assistance] meet its [objectives/purpose]?
[Does/Did] your use of the [name of product, name of training/workshop] align with its intended purpose?
Did you use the [name of product] for another purpose?
[Is/Was] the [name of product, name of training/workshop] beneficial for your purposes?
[Are/Were] the benefits of the [name of product, name of training/workshop] clearly defined?
[Were/Are] you confused about the purpose of the [name of product, name of training/workshop, type of technical assistance]?
Is it clear why the DHDSP is involved in providing the [name of product, name of training/workshop, type of technical assistance]?
Has the purpose of the [name of product, name of training/workshop, type of technical assistance] change over time? Is this change positive or negative?
Do the benefits of the [name of product, name of training/workshop] align with DHDSP’s priority areas, as outlined in the FOA?
Do the benefits of the [name of product, name of training/workshop] align with the [name of DHDSP’s priority area]?
Do the benefits of the [name of product, name of training/workshop] align with the DHDSP strategic plan and its goals?
Do the benefits of the [name of product, name of training/workshop] align with the DHDSP’s goal to [name of DHDSP goal]?
Do the benefits of the [name of product, name of training/workshop] align with CDC goals?
Do the benefits of the [name of product, name of training/workshop] align with Healthy People goals?
W
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X X X
How have you used the [name of product, information from training/workshop]?
How did the [name of product, name of training/workshop] benefit your work?
B. Clear audience: DHDSP guidance should have a clearly defined audience
[Is/Was] the audience for the [name of product, name of training/workshop, type of technical assistance] clearly defined?
[Is/Was] the audience for the [name of product, name of training/workshop, type of technical assistance] the right audience?
[Is/Was] the audience for the [name of product, name of training/workshop, type of technical assistance] relevant for your work?
[Is/Was] it clear that the [name of product, type of technical assistance] [is/was] designed for [intended audience]?
Was it clear that the [name of product, type of technical assistance] was designed for [intended audience] to use?
Was it clear that the [name of training] was hosted for [intended audience] to attend?
Was it clear that the [name of training] was designed for [intended audience]?
Was it clear that the [name of training] was hosted for [intended audience] to attend?
Would you be likely to [use/attend] [name of product, name of training/ workshop, type of technical assistance]?
Whom did you perceive to be the [major/primary] audiences for the [name of product, name of training/workshop, type of technical assistance]?
Who would be most likely to [use/attend] [name of product, name of training/ workshop, type of technical assistance]?
C. Solve a problem: DHDSP guidance should address a specific and existing problem, interest, or need of the target audience. Guidance should be tailored to meet the needs of the target audience
[Does/Did] the [name of product, name of training/workshop, type of technical assistance] address a specific problem, interest or need that [you/your organization] [had/have]?
[Does/Did] the activities included in the [name of training/workshop] address a specific problem, interest or need that [you/your organization] [had/have]?
[Were/Are] the activities included in the [name of training/workshop] useful?
[Were/Are] the examples provided in the [name of training/workshop, name of product] useful?
[Is/Was] the [name of product, name of training, type of technical assistance] relevant to the work you do?
[Is/Was] this [name of product] used to [name of problem, interest, need: listed below]?
Facilitate policy changes within your organization
Share information with policy makers
Share information within organization
Share information with other partners
Inform budget development
Influence intervention or programmatic decisions
F
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X
X X X X X X X X X X X X X X X X X X X
Share information with your target audiences or stakeholders.
[Other problem, interest, need]
What problem or need [does/did] the [name of product, name of training/workshop, type of technical assistance] address?
How effective was the [name of product, name of training/workshop, type of technical assistance] in addressing this need?
How does the [name of product, name of training/workshop, type of technical assistance] address [you/your agency’s] problems, interests, or needs?
What problems, interests, or needs do you still have that the [name of product, name of training/workshop, type of technical assistance] [did/does] not address?
What other guidance or resources do you need to enhance your organization’s heart disease and stroke prevention efforts?
What was the main reason you decided to [use/access] the [name of product, type of technical assistance]?
What [was/is] the main reason you decided to attend the [name of training/workshop]?
What [was/is] the main reason you decided to [join/participate] in the [name of training/workshop]?
D. Novel: DHDSP guidance should not be duplicative of other heart disease and stroke prevention efforts; rather it should complement guidance supplied by partner agencies.
[Is/Was] the [name of product, name of training/workshop] novel?
[Is/Was] the [name of product, name of training/workshop] unique?
[Is/Was] the [name of product, name of training/workshop] innovative?
[Did/Does] the [name of product, name of training/workshop, type of technical assistance] represent a unique approach to heart disease and stroke prevention?
[Did/Does] the [name of product, name of training/workshop, type of technical assistance] duplicate other [efforts/materials/trainings]?
[Did/Does] the [name of product, name of training/workshop, type of technical assistance] complement other [efforts/materials/trainings]?
Do you know of a [product, training/workshop, technical assistance service] similar to the [name of product, name of training/workshop, type of technical assistance]?
[Did/Does] the [name of product, name of training/workshop, type of technical assistance] fill an information need?
How much of the information presented in the [name of product, name of training/workshop, type of technical assistance] [is/was] new to you?
In what ways is the [name of product, name of training/workshop, type of technical assistance] similar to other heart disease and stroke [efforts/materials/ trainings]?
In what ways does the [name of product, name of training/workshop, type of technical assistance] differ from other heart disease and stroke [efforts/materials/ trainings]?
W
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X
How [are/have] you [using/used] the results of the [name of product, name of training/workshop, type of technical assistance]?
E. Easy to access: DHDSP guidance should be designed, targeted, and distributed so that it will reach the target audience.
[Did/Do] you have difficulties getting or using the [name of product, type of technical assistance]?
[Was/Is] the information contained in the [name of product] presented in the most effective way for [its intended audience(s)/you]?
[Was/Is] the information contained in the [name of product, name of training/ workshop] presented effectively?
Did the information contained in the [name of product, name of training/ workshop] reach you effectively?
[Was/Is] the information contained in the [name of product, name of training/ workshop] disseminated in the most effective way for [its intended audience(s)/ you]?
[Was/Is] the information contained in the [name of product] distributed in the most effective way for [its intended audience(s)/you]?
[Was/Is] communicating exclusively in the virtual world (ex. phone conferences and email) a hindrance to the [name of collaborative effort, type of technical assistance]?
[Was/Is] it easy for you to access the [name of product, type of technical assistance]?
[Was/Is] it easy for you to attend the [name of workshop/training]?
What would make it easier for you to [access/attend] the [name of product, type of technical assistance, training/workshop]?
What barriers [did/do] you face in [attending/accessing] the [name of product, type of technical assistance, training/workshop]?
Through which mechanism are you most likely to access the [name of product, type of technical assistance]?
How do you prefer to access the [name of product, type of technical assistance]?
How did you hear about the [name of product, name of training/workshop, type of technical assistance]?
How did you find out about the [name of product, name of training/workshop, type of technical assistance]?
Is this the way you prefer to find out about CDC [products/trainings/type of technical assistance] such as the [name of product, name of training/workshop, type of technical assistance]?
How do you prefer to find out about CDC [products/trainings/type of technical assistance] such as the [name of product, name of training/workshop, type of technical assistance]?
What format would you prefer for CDC to produce materials (i.e. one pagers, extensive lit reviews, web based, etc)?
W
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X X X X
What would be a more effective way to disseminate the information contained in the [name of product, name of training/workshop]?
If you wanted support on [name of public health competency] how likely would you be to contact the DHDSP?
If you wanted support on/with [name of public health competency] who would you contact?
If you wanted support on/with [name of public health competency] would you know who to contact at the DHDSP?
If you wanted support on/with [name of public health competency] would you know how to contact the DHDSP?
If you wanted support on/with [name of public health competency] how would you contact the DHDSP?
How [did you/have you] [access/accessed] the [name of product, type of technical assistance]?
F. Timely: Turn around time for the delivery of DHDSP products and services should be minimal.
Was the [name of product] provided to you in a timely fashion?
Should the [name of product] have been provided to you sooner?
Did you receive a prompt reply after you contacted the DHDSP for the [name of product, type of technical assistance]?
Did you receive the [name of product, information on the training/workshop] you requested in a timely manner?
[Is/Was] the [name of product, name of training/workshop, type of technical assistance] made available to you in a timeframe that met your needs?
[Is/Was] the [name of product, name of training/workshop, type of technical assistance] already outdated by the time it was first made available to you?
Did the timeframe of DHDSP’s reply meet your expectations?
Is the amount of communication you receive from the [DHDSP, name of collaborative effort] at the right frequency?
Does the [name of collaborative effort] meet often enough?
Is the [name of product, name of training/workshop, type of technical assistance] [distributed/offered] at an adequate frequency?
Are DHDSP products updated regularly?
G. Appropriate level: DHDSP guidance should exhibit an understanding of the target audiences’ current level of competence, abilities, and skills. Guidance should be tailored to meet the target audience at its current skill level.
Does the [name of product] reflect an understanding of [the target audience’s/ your] skills and abilities with the subject material?
Does the [name of product] reflect an understanding of [the target audience’s/ your] level of competence with the subject material?
[Did/Do] you understand the information contained in [name of product, name of training/workshop]?
W
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X X X X X X
Do you think the [the target audience/you] will understand the material in the [name of product, name of training/workshop]?
Do you think the [name of product, name of training/workshop] will help [the audience /you] to develop skills or competencies?
Is the [name of product] designed to address [audience/your] current competency?
Is the [name of product, name of training/workshop] appropriately designed for your skill level?
Is the [name of product, name of training/workshop] appropriately designed for [intended audience] skill level?
[Is/Was] the [name of product, type of technical assistance, name of training/ workshop] appropriate for the skills you have?
[Is/Was] the [name of product, type of technical assistance, name of training/ workshop] appropriate for the knowledge you have?
[Does/Did] the [name of product, type of technical assistance, name of training/ workshop] fill a gap in your [skills/knowledge]?
[Does/Did] the [name of product, type of technical assistance, name of training/ workshop] assume the appropriate level of familiarity with the relevant issues?
Which [workshops/sessions] matched your skill level?
Which [workshops/sessions] did not match your skill level?
How could the [workshops/sessions] have been tailored to better match your skill level?
What made the information contained in [name of product, name of training/ workshop] [easy/difficult] to understand?
H. Pertinent: Guidance should exhibit an understanding of the target audiences’ context and the work in which they are engaged. DHDSP will assess what heart disease and stroke prevention activities, including evaluation, States and partners are engaged in and ask them to identify the guidance they perceive as relevant to their work.
[Does/Did] the [name of product] fit into [your/audiences’] current activities and undertakings?
[Is/Was] the [name of training, type of technical assistance, name of product] applicable to the work that you do?
[Is/Was] the [name of training, type of technical assistance, name of product] useful?
[Is/Was] the [name of training, type of technical assistance, name of product] relevant to work that you do?
Which [sessions/workshops, name of product, or parts] were the most relevant?
Which [sessions/workshops, name of product, or parts] were the least relevant?
Given your current context, [are/were] you able to [implement/apply/use] the [name of training, type of technical assistance, name of product]?
A
OK to add Follow-Up
OK to add “To what
extent”
X X X X X
X X X X X X X X X X X X X X X X X X X X X X X X X X
Are there factors that you expect will make it easer to [implement/apply/use] the [name of training, type of technical assistance, name of product]?
[Is/Was] the [name of training/workshop, type of technical assistance, name of product] useful to your advocacy group, community partners, or other stakeholders?
What heart disease and stroke prevention activities [is/are] [you/your organization] currently engaged in?
In what ways has the [name of training/workshop, type of technical assistance, name of product] supported that work?
In what ways has the [name of training/workshop, type of technical assistance, name of product] been less helpful in your work?
Which [activities/parts/lessons/information] from the [name of training/workshop, name of product, type of technical assistance] will you most likely use in your work?
Which [activities/parts/lessons/information] from the [name of training/workshop, name of product, type of technical assistance] will you most likely use or incorporate into your work on [task]?
Which [activities/parts/lessons/information] from the [name of training/workshop, name of product, type of technical assistance] are you least likely use?
For which topics are you most likely to seek [products/guidance/trainings/ technical assistance] from the DHDSP?
For which topics are you most likely to disregard [products/guidance/trainings/ technical assistance] from the DHDSP?
For which topics are you most in need of training, technical assistance or products?
What would make [name of training, type of technical assistance, name of product] more applicable to the work that you do?
What would make [name of training, type of technical assistance, name of product] more useful?
How could the [name of training, type of technical assistance, name of product] be more relevant to the work you do?
What type of tools, products or resources [will/could] best assist you in implementing the [DHDSP strategy]?
I. Transparency: DHDSP guidance should allow for target audience input and participation while it is being developed.
[Is/Was] it clear who [was involved/developed] the [name of product, name of training/workshop]?
[Is/Was] it clear how the [name of product, name of training/workshop] was developed?
D
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X
Do you believe there is a “hidden agenda” behind the [name of product, name of training/workshop, type of technical assistance]?
Do you think [target audience] was involved in developing the [name of product, name of training/workshop]?
Does the [name of product, name of training/workshop] reflect the participation and input of [audiences]?
Does the [name of product, name of training/workshop, type of technical assistance] reflect the input of necessary [experts, policy makers, key stakeholders]?
Does [name of product, name of training/workshop] reflect the participation and input of target audiences?
Did the [name of product, name of training/workshop] meet the needs of the target audience?
Did you participate in formative research for [name of product]?
Did you participate in the planning of the [name of training/workshop]?
Did the [name of product, name of training/workshop] increase transparency between partners?
Did the [name of training/workshop, name of collaborative effort] increase your input or participation in the development of [name of product]?
Would you like to be involved in developing future [versions/iterations] of the [name of product]?
Would you like to be involved in planning future [name of training/workshop]?
How could involvement from key audiences be improved?
In what ways could the [name of product, name of training/workshop] [have been/be] improved to meet [your needs/the needs of the target audience]?
In what ways was the [name of product, name of training/workshop] particularly responsive in meeting [your needs/the needs of the target audience]?
What suggestions would you have for improving similar [products/trainings]?
J. Improvements: The DHDSP will examine ways to improve the relevance of the training, technical assistance, and other guidance products and services it currently offers
What [number] specific content areas and/or skills would you like to have included in the next training?
How could the [type of technical assistance, name of training/workshop or product] be improved?
What [activities/parts/components] of the [name of product, name of training/workshop] should be deleted?
What additional [activities/parts/components] of the [name of product, name of training/ workshop] should be [included/added]?
Please provide any additional comments or suggestions for improvement.
What could DHDSP do to better meet the needs of [you/your organization]?
What are the limitations of the [type of technical assistance, name of training/workshop or product]?
What could the DHDSP do to better address these limitations?
W
OK to add Follow-Up
OK to add “To what
extent”
X X X X X
What would make the [name of training/workshop, name of product, type of technical assistance] more useful to [you/your organization]?
What additional information do you need as a result of [using/accessing/ attending] the [name of training/workshop, name of product, type of technical assistance]?
What additional training or technical assistance do you need to address [public health competency]?
What content was missing from the [name of training/workshop, name of product, type of technical assistance]?
III. QUALITY
The DHDSP needs to know if the activities that it is engaged in are being well implemented, whether its training, technical assistance, and other guidance products and services are perceived by the States and partners as being of high quality. For the purpose of this evaluation, the following dimensions of quality will be explored: understandable, credible, accurate, consistent, feasible, pleasant, meet expectations.
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X
A. Understandable: DHDSP guidance should be easy to understand. Members of the target audience should be able to use and apply the information they receive.
Did you understand the information presented [in/during] the [type of technical assistance, name of training/workshop, name of product]?
[Was/Is] there information [contained in the/presented at the] [name of product, name of training/workshop, type of technical assistance] that [was/is] difficult to understand?
[Was/Is] there information [contained in the/presented at the] [name of product, name of training/workshop, type of technical assistance] that [was/is] confusing?
[Was/Is] the information [contained in the/presented at the] [name of product, name of training/workshop] easy to understand?
[Was/were/are] the [name of product/training materials] easy to read?
[Was/Is] the information [contained in the/presented at the] [name of product, name of training/workshop] useful?
[Was/Is] the information [contained in the/presented at the] [name of product, name of training/workshop] presented in an organized way?
[Will/Have] you [be/been] able to apply the information [contained in the/ presented at the] [name of product, name of training/workshop]?
[Will/Have] you [be/been] able to [apply/use] the guidance you received?
[Were/Are] you able to understand the recommendations in [name or product, name of training/workshop, type of technical assistance]?
[Were/Are] the guidelines for the [name of training/workshop, name of collaborative effort] clear?
[Was/Is] the [name of collaborative effort, name of training/workshop, type of technical assistance, name of product] overly complex?
[Was/Is] the [name a product] in a format that [is/was] easy to understand?
What problems did you have with the format?
How could the format be improved?
Which [parts/sections/workshop/sessions] of the [name of product, name of training/workshop] confused you?
B. Credible: DHDSP guidance should be perceived as credible by the target audience. Individuals accessing DHDSP guidance should perceive the source as trustworthy.
Do you believe that the [type of technical assistance, name of product] came from a credible source?
[
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X
[Is/Was] the information presented in the [name of training/workshop, name of product] from trustworthy sources?
Is the credibility of [name of product] enhanced because it was [developed/ distributed] by the [CDC/DHDSP]?
Do you have concerns or reservations about the credibility of the information you receive from the [CDC/DHDSP]?
Does the credibility of the organization that [produced/presented] [name of training/workshop, name of product] impact your opinion of the [name of training/workshop, name of product]?
Is the source of the [name of training/workshop, name of product] perceived as a [credible/reliable] source by your [organization/partners/stakeholders/audiences]?
[Who/What] do you consider to be credible sources of heart disease and stroke prevention information?
Where do you access heart disease and stroke prevention information?
C. Accurate: DHDSP guidance should be technically accurate, based on the most current knowledge and state of the science. DHDSP guidance should contain complete information, be free of major flaws or errors, and align with guidance from partner agencies.
Was the [type of technical assistance] you received accurate?
[Is/Are] the [type of information] [contained in/presented at] the [name of product, name of training/workshop] is/are accurate?
[Does/Did] the [name of training/workshop, name of product] provide up to date information?
[Does/Did] the [name of training/workshop, name of product, type of technical assistance] contain any flaws or errors?
[Does/Did] the [name of training/workshop, name of product, type of technical assistance] make use of the most current and best available science, data or evidence?
[Is/Was] the [name of product, name of training/workshop, type of technical assistance]’s presentation of the relevant information complete?
[Is/Was] the information presented in the [name of training/workshop, name of product, type of technical assistance] free from major flaws or errors?
[Did/Does] the information presented in the [name of training/workshop, name of product, type of technical assistance] contain flaws or errors?
[Is/Was] the information [presented in/you received from] the [name of training/workshop, name of product, type of technical assistance] consistent with information you have received from [name of partner agency]?
Does the information [presented in/you received from] the [name of training/workshop, name of product, type of technical assistance] differ from information you have received from [name of partner agency]?
D
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X
Has information about [subject] been presented consistently by [DHDSP, CDC]?
Has information about [subject] been presented inconsistently by [DHDSP, CDC, partner agencies]?
[Is/Was] the information you received from [name of product, type of technical assistance, name of training/workshop] consistent with other guidance you have received from the [DHDSP, CDC]?
[Is/Was] the presentation of information in the [name of training/workshop, name of product, type of technical assistance] consistent with guidance from partner agencies?
[Does/Did] the presentation of information in the [name of training/workshop, name of product, type of technical assistance] conflict with guidance from partner agencies?
Is the DHDSP guidance you received about [subject] consistent in both context and delivery with other guidance you have received from DHDSP?
Is DHDSP guidance consistent with current practice in your [organization/State]?
E. Feasible: DHDSP guidance should have practical utility. DHDSP will assess the labor, time, and other costs associated with implementing its guidance recommendations.
[Is/Was] it feasible for [you/your organization] to implement [guidance recommendation]?
[Is/Was] it feasible for [you/your organization] to implement [guidance recommendation] based on your current resources?
[Does/Did] the [name of product, type of technical assistance, name of training/ workshop] call for activities that can be realistically undertaken by [your organization/partners/stakholders/audiences]?
[Was/Is] participating in the [name of collaborative effort] overly time-consuming?
Would participating in the [name of collaborative effort] be feasible in future projects?
Are the [name of product, type of technical assistance, name of training/ workshop]’s recommendations or guidance realistic in terms of the costs (in labor, time, and other resources) required for implementation?
[Does/Did] the [name of product, type of technical assistance, name of training/ workshop] have practical utility in your organization?
Have [you/your organization] been able to implement the ideas, activities, or recommendations made [by/during] the [name of product, type of technical assistance, name of training/workshop]?
[Were/Are] the timelines or benchmarks set during the [name of collaborative effort, name of training/workshop] attainable?
[Was/Is] adequate training provided for you to [use/adopt] [DHDSP strategy]?
How much effort did it take [you/your organization] to implement [guidance recommendation]?
How much time did it take [you/your organization] to implement [guidance recommendation]?
H
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X X X X
What has prevented [you/your organization] from implementing the ideas, activities, or recommendations made [by/during] the [name of product, type of technical assistance, name of training/workshop]?
What are [your/your organization’s] suggestions to help implement the ideas, activities, or recommendations made [by/during] the [name of product, type of technical assistance, name of training/workshop]?
G. Pleasant: Accessing DHDSP guidance services should be a positive experience. State and partner interactions with the DHDSP should be the foundation for advancing partnerships and collaborative efforts.
[Was/Is] [using/attending/accessing] the [name of product, name of training/ workshop, type of technical assistance] pleasant?
Are you motivated to [use/attend/access] the [name of product, name of training/ workshop, type of technical assistance]?
[Did/Do] you want to [use/attend/access] the [name of product, name of training/ workshop, type of technical assistance]?
[Did/Do] you enjoy [using/attending/participating in] the [name of product, name of training/workshop, name of collaborative effort]?
[Did/Do] you enjoy speaking with the individual from whom you received [type of technical assistance]?
[Did/Do] the individual[s] providing [type of technical assistance] treat you with respect?
[Did/Do] the individual[s] providing [type of technical assistance] treat you professionally?
[Does/Did] the [name of product, name of training/workshop, type of technical assistance] enhance your view of DHDSP as a partner?
[Does/Did] the [name of product, name of training/workshop, type of technical assistance] enhance the prospect of your future [consultation, collaboration] with DHDSP?
[Would/Will] you [use/attend/access] [name of product, name of training/ workshop, type of technical assistance] again?
Would you recommend the [name of training/workshop, name of product, type of technical assistance] to others?
[Is/Was] the [name of training/workshop, name of product] acceptable to [you/target audiences]?
[Is/Was] the [name of product/component of product] attractive?
[Is/Was] use of the [name of product] appealing?
[Is/Was] the [name of training/workshop, name of product] easy to understand?
Does the [name of training/workshop, name of product, type of technical assistance] acknowledge the expertise of others outside of the DHDSP?
[Are/Were] there aspects of the [name of training/workshop, name of product, type of technical assistance] that felt condescending?
[Are/Were] there aspects of the [name of training/workshop, name of product, type of technical assistance] that felt disrespectful?
D
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X X X X X X
[Was/Is] the [name of collaborative effort, type of technical assistance, name of training/workshop] a comfortable learning environment?
Was there sufficient positive reinforcement throughout the [name of collaborative effort, type of technical assistance, name of training/workshop]?
Was the feedback provided sufficient throughout the [name of collaborative effort, type of technical assistance, name of training/workshop]?
Did you have adequate opportunities to provide feedback on [name of collaborative effort, type of technical assistance, name of training/workshop, name of product]?
How would you rate the support provided by DHDSP?
What are the strengths of the [name of training/workshop, name of product, type of technical assistance]?
What are the limitations of the [name of training/workshop, name of product, type of technical assistance]?
What would make the [name of product, name of training/workshop, type of technical assistance] more attractive or more of a positive experience?
H. Meet expectations: DHDSP guidance should meet or exceed the expectations or needs of the target audience. Users of DHDSP guidance services should express a high level of satisfaction with the trainings, technical assistance, and products that DHDSP offers.
[Did/Does] the [name of training/workshop, name of product, type of technical assistance] meet your expectations?
[Did/Does] the [name of training/workshop, name of product, type of technical assistance] exceed your expectations?
Were there enough peer-to-peer learning opportunities?
[Did/Does] the [name of training/workshop, name of product, type of technical assistance, name of collaborative effort] provide you additional opportunities to learn?
[Are/Were] you satisfied with the [name of training/workshop, name of product, type of technical assistance]?
[Did/Do] the pros outweigh the cons in the [name of training/workshop, name of collaborative effort]?
[Was/Is] the DHDSP committed to the [name of training/workshop, name of collaborative effort]?
In what ways did the [name of training/workshop, name of product, type of technical assistance] meet your expectations?
In what ways did the [name of training/workshop, name of product, type of technical assistance] exceed your expectations?
In what ways did the [name of training/workshop, name of product, type of technical assistance] fail to meet your expectations?
What could have been done differently in order for the [name of training/ workshop, name of product, type of technical assistance] to more fully meet your expectations?
H
OK to add Follow-Up
OK to add “To what
extent”
X X X X X X X X X X X X X X X X X X X X X X X
What could [be/have been] done differently in order for you to be more satisfied with the [name of training/workshop, name of product, type of technical assistance]?
IV. IMPACT
The DHDSP needs to know if the activities that it is engaged in are having their intended effect. PART emphasizes that “the most significant aspect of program effectiveness is impact - the outcome of the program, which otherwise would not have occurred without the program intervention.” PART emphasizes the importance of examining what difference a program makes as well as how or why that program is effective (www.whitehouse.gov). For the purpose of this evaluation, the following dimension of impact will be explored: potential impact, use, reach, benefits, and gaps.
OK to add Follow-Up
OK to add “To what
extent”
A
X X X X X
X X X X X X X X X X X X X X X
[Were/Are] the intended benefits of the [name of product] and its [recommendations/ guidance] clear?
[Will/Did] the [name of product, type of technical assistance, name of training/ workshop] help to increase your organization’s capacity to prevent heart disease and stroke?
[Will/Has] the use of [name of product, type of technical assistance, name of training/ workshop] [have/had] an impact on [public health outcome]?
Do you foresee the [name of product, type of technical assistance, name of training/ workshop] having an impact on [public health outcome] in the future?
[Will/Has] the [name of product, type of technical assistance, name of training/ workshop, name of collaborative effort] [have/had] long term impacts?
Do the intended benefits of [name of product] align with your organization’s [strategic goals/priorities]?
What impact do you see the [name of product, type of technical assistance, name of training/ workshop] having on [you/your organization]?
What are the potential negative impacts of the [name of product, type of technical assistance, name of training/workshop]?
How will you use information from [name of product, type of technical assistance, name of training/workshop] to impact policy changes?
How will you use this information from [name of product, type of technical assistance, name of training/workshop] impact programmatic changes?
How will you use this information from [name of product, type of technical assistance, name of training/workshop] impact budget changes?
How will you use this information from [name of product, type of technical assistance, name of training/workshop] impact advocacy for prevention?
How will you use this information from [name of product, type of technical assistance, name of training/workshop] impact program evaluation activities?
How will you use this information from [name of product, type of technical assistance, name of training/workshop] to impact partnership and stakeholder activities?
B
X X X X X X X X X X X X X X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
What type of [name of HDSP topic] [tools/products/resources] do you currently use?
How many times have you [used/accessed] [type of technical assistance, name of product, name of training/workshop] in the past [time frame]?
Why haven’t you [used/accessed] [type of technical assistance, name of product, name of training/workshop]?
If [you/your organization] [have/has] a [type of need] in the future, how likely are you to [use/access] [type of technical assistance, name of product]?
Within the past [time frame], have you [used/accessed] [type of technical assistance, type of product] from another source other than the DHDSP?
What [makes/made] you use this source for [type of technical assistance, type of product]?
In what ways is [type of technical assistance, type of product] from this source different than that provided by the DHDSP?
Compared to the [type of technical assistance, type of product] you just identified, would you say that DHDSP [type of technical assistance, type of product] is [much better/much worse/about the same]?
What would CDC need to do to make you inclined to ask CDC for [type of technical assistance, type of product]?
For which topics have you accessed [type of technical assistance]?
Thinking about the last time you accessed [type of technical assistance], describe the topic(s) that were discussed during this session.
Have you contacted DHDSP for support related to [insert competency]?
What support did you receive when you contacted DHDSP?
What type(s) of DHDSP [products, technical assistance, trainings/workshops] have you [accessed, used, attended]?
How many times have you used the [name of product] in the past [time frame]?
How [did/have] you [use/used] the [name of product]?
What is the most common reason you use [name of product]?
Did the [name of product] change the way you do your job?
Has the use of [name of product] had any impact on [name of job function]?
Which products have you found to be most useful in [name of job function]?
Which products have been less useful for [name of job function]?
Will the information you received from [name of product, name of training/ workshop, type of technical assistance] be used to [influence policy decision making/enhance collaboration]?
Please think about the last time you used the [name of product].
Why did you decide to use the [name of product]? Describe the situation.
What information did you need to know?
What made you think to use the [name of product]?
D
X X X
X X X X X X X X X X X X X X X X X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
X
Did the [name of product] solve the problem?
How did you use the information provided by the [name of product]?
What benefits did you experience after using the [name of product]?
What problems or issues did you experience after using the [name of product]?
Would you choose to use the [name of product] again in this situation?
What effect has the [name of product] had on you or your organization’s work?
Do you believe the [name of product] has contributed to an increased capacity to [type of capacity] in your organization?
Did you experience any [unintended/unexpected] consequences after using the [name of product]?
C. Reach: DHDSP guidance should be developed, translated and disseminated to promote the greatest possible reach to States and partners and to facilitate uptake and utilization by secondary users to the extent possible. The DHDSP will examine the scope and range of States and partners utilizing its guidance and identify with whom the target audience has shared DHDSP products.
How did you learn about [name of product, name of training/workshop, type of technical assistance]?
With whom have you shared the [name of product, information from training]?
[Have/Will] you [trained/train] other staff, partners, or stakeholders on the information you gained from the [name of product, type of technical assistance, name of training/workshop]?
Have you trained other staff on the information you have gained from the DHDSP?
What types of groups, organizations, and partners in your State have received this [name of product]?
Have your partners shared the [name of product] with others?
What methods [have you used/will you use] to disseminate the [name of product]?
In what ways has the [name of product] been used by others in your organization?
Who else could benefit from [use of the product, attending the name of training/workshop]?
To whom [have/will] you [recommended/recommend] the [name of product, name of training/ workshop]?
To whom have others in your organization recommended the [name of product, name of training/ workshop]?
Do you know of anyone who should be [attending/using/accessing] the [name of product, name of training/workshop], but is not?
Who else has used [name of product] outside of your organization?
What dissemination methods could be used to reach [target audience]?
How can DHDSP further support information sharing (i.e. interactively sharing you insights, asking others questions, finding out how others are using the results) regarding the [name of product, name of training/ workshop]?
Who are some of the partners to which you would like to distribute [type of information]?
W
X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
D. Benefits: DHDSP guidance should result in direct and indirect benefits to States and partners. The DHDSP will examine benefits experienced by the target audience as a result of its guidance. This will be examined in the context of benefits of heart disease and stroke prevention services and products supplied by other partners.
Did you get the benefits you expected from [name of product, type of technical assistance, name of training/workshop]? If not, what benefits did you not get?
Did the [name of product, type of technical assistance, name of training/ workshop] increase your knowledge?
Did the [name of product, type of technical assistance, name of training/ workshop] increase your knowledge of [topic]?
Did you gain new skills from [using/accessing/participating in] the [name of product, type of technical assistance, name of training/workshop, name of collaborative effort]?
Did participating in the [name of collaborative effort] help increase your confidence in working with a team?
Did [name of product, type of technical assistance, name of training/workshop] provide you with skills you can apply at your job?
Has the [name of product, type of technical assistance, name of training/ workshop] assisted you at your job?
[Did/have] you [use/used] anything you learned from the [name of product, type of technical assistance, name of training/workshop]?
Did the [name of product, type of technical assistance, name of training/ workshop] help you address a specific problem, interest or need that [you/your organization] had?
Did the [name of product, type of technical assistance, name of training/ workshop] help to increase your organization’s capacity to prevent heart disease and stroke?
Did the [name of product, type of technical assistance, name of training/ workshop] help to increase your organization’s capacity to [name of public health competency]?
Did the [name of product, type of technical assistance, name of training/workshop] increase your knowledge about the HDSP program priorities?
Did the [name of product, type of technical assistance, name of training/workshop] increase your knowledge about how to address HDSP program priorities?
Did the [name of product, type of technical assistance, name of training/ workshop] help you understand the intent of the components in the current HDSP Program Announcement?
Did the [name of product, type of technical assistance, name of training/workshop] increase your knowledge about policy and system change for heart disease and stroke prevention?
D
X X X X X X X X
X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
Did the [name of product, type of technical assistance, name of training/ workshop] help increase or improve your partnerships to prevent heart disease and stroke?
Did the [name of product, type of technical assistance, name of training/ workshop] affect a policy to prevent heart disease and stroke?
Did the [name of product, type of technical assistance, name of training/ workshop] impact your organizational structure? In a positive way or a negative way?
Did the [name of product, type of technical assistance, name of training/ workshop] affect a [name of health care system] to prevent heart disease and stroke?
Did the [name of product, type of technical assistance, name of training/workshop] increase your knowledge and skills related to program evaluation?
Did the [name of product, type of technical assistance, name of training/workshop] increase your knowledge and skills related to surveillance?
What did you expect to be the benefits of the [name of product, type of technical assistance, name of training/workshop]?
What did you learn after [using, accessing, attending] the [name of product, type of technical assistance, name of training/workshop]?
Describe some of the ways either your or your organization has benefited from the [name of product, type of technical assistance, name of training/workshop].
How has the [name of product, type of technical assistance, name of training/ workshop] assisted you in bringing about policy and/or systems change in [name of setting]?
What knowledge or skill item(s) that you learned [from, during, at] the [name of product, type of technical assistance, name of training/workshop] will you be most able to use in your current job? (Knowledge/Skills options are listed below)
Program
Strategic planning
Working with EMS for system change
Developing a work plan
Influencing policy change
Collaborating with managed care
Intervening at worksites
Working with employers or business coalitions
Improving Quality of Care in Health Care Systems
Hypertension Interventions or Initiatives
Cholesterol Interventions or Initiatives
Working on signs and symptoms of heart attack or stroke initiatives
Working on emergency response initiatives
Working on disparities initiatives
Building/Maintaining Partnerships and Coalitions
Integrating with WISEWOMAN or other chronic disease programs
Working with priority populations
Education and training around heart disease and stroke
Translating HDSP science into practice
Using data in developing programs
[Other program components]
Evaluation
Setting evaluation priorities
Developing key evaluation questions
Using your evaluation results for program improvement
Developing an evaluation plan
Evaluating media campaigns
Evaluating a partnership
Evaluating policy/systems change
Using qualitative evaluation methods
Building evaluation capacity
Evaluating research or demonstration projects
Using indicators for program planning and evaluation
[Other evaluation components]
Epidemiology
Conducting surveillance activities
Analyzing complex data
C
OK to add Follow-Up
OK to add “To what
extent”
[Other epidemiology component]
X X
X X X X X X X X
E. Gaps: DHDSP guidance should seek to address gaps in knowledge, services and identified needs of States and partners. These needs may include elements relevant to enhancing capacity, job performance and their organization’s heart disease and stroke prevention efforts. The DHDSP will examine gaps in the services it currently offers as well as potential enhancements to existing guidance and resources.
Has [name of product, name of training/workshop, type of technical assistance] helped [you/your organization] addressed a gap in knowledge, services, and/or needs related to heart disease and stroke prevention?
How has [name of product, name of training/workshop, type of technical assistance] helped [you/your organization] address a gap in knowledge, services, and needs related to heart disease and stroke prevention?
What needs do [you/your organization] have that the DHDP has not addressed?
What services or guidance can the DHDSP provide to enhance your organization’s heart disease and stroke prevention efforts?
What services or guidance can the DHDSP provide to enhance your capacity to [insert public health competency]?
How can the DHDSP be of further support to [you/your organization]?
What do you still need to know in order to enhance your organization’s heart disease and stroke prevention efforts?
What additional data do you need to enhance your organization’s heart disease and stroke prevention efforts?
W
X X
OK to add Follow-Up
OK to add “To what
extent”
What problems, interests, or needs do you still have that the [name of training/workshop, type of technical assistance, name of product] [did/does] not address?
In addition to these dimensions, the DHDSP will examine progress toward its short and longer term goals. These are identified as short and long term outcomes on the DHDSP Logic Model (See Attachment 4).
OK to add Follow-Up
OK to add “To what
extent”
X
X X X X X X X X X X X X X X X X X X X X X X X X X
A. Engaged network of states and partners
Are you currently engaged in collaborative work with DHDSP?
How long have [you/your organization] participated in [name of collaborative effort]?
Why did [you/your organization] become involved in [name of collaborative effort]?
Describe the roles and responsibilities of the partners in [name of collaborative effort].
Describe your role and responsibilities in [name of collaborative effort].
What did you contribute to the [name of collaborative effort]?
How is the [name of collaborative effort] working on [internal/external] issues?
What have been the outcomes of the [name of collaborative effort]?
What about the [name of collaborative effort] is working well?
What about the [name of collaborative effort] is working less well or not working well?
What facilitators have promoted [you/your organization]’s ongoing involvement in [name of collaborative effort]?
Was participation in [name of collaborative effort] satisfying professionally?
What challenges has the [name of collaborative effort] faced?
How have you overcome these challenges?
What are the costs associated with the [name of collaborative effort]?
What resources are needed to support [name of collaborative effort]?
What resources have been most helpful?
What resources are still needed? Are they available?
Who are the primary and most active players in the [name of collaborative effort]?
Have you ever been engaged in collaborative work with DHDSP?
Why has this collaboration discontinued?
Why did [you/your organization] leave the [name of collaborative effort]?
Would you consider collaborating with DHDSP on other efforts?
What happened as a result of the [name of collaborative effort]?
How would you rate the [insert collaboration factor – listed below] [within/ affecting] the [name of collaborative effort]?
Collaborative factors include;
History of the collaboration
Perception of the collaborative group as a legitimate leader
Political climate
Social climate
Existence of partnership champions
Mutual trust
Partners’ tolerance for sharing power
Partners’ willingness to adapt to meet collaboration needs
Partners’ ability to compromise
Mutual understanding
Mutual respect
Partner representation
Partners’ belief that the collaboration is in their self-interest
Accountability of partners in both process and outcome
Ability to meet performance expectations
Senior management support
Flexibility in organizing itself
Flexibility in accomplishing its work
Development of clear roles
Development of policy guidelines for partners
Adaptability
Appropriateness of pace of development
Openness of communication
Establishment of informal relationships
Informal communication links
Partner compatibility
Degree of conflict
Frequency of conflict
Shared vision
Unique purpose
Resource exchange
Transparency
Value-added
Partners’ satisfaction
Sufficiency of funds
Sufficiency of staff
Sufficiency of materials
Sufficiency of time
Competency of leadership
Lead agency
Staff
Leaders (i.e. steering committee)
Members
Structures (i.e. bylaws, mission statement)
P
OK to add Follow-Up
OK to add “To what
extent”
Communication
Statewide involvement
I
X
X X X X X X X X X X X X X X X X X X X X X X X X X X
[Other collaboration factors]
If [insert collaboration factor] was rated poorly, why?
What do you believe could have improved [insert collaboration factor]?
What made the [insert collaboration factor] strong?
D
X X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
[Does/Did] the [name of collaborative effort] benefit you/your organization?
[Does/Did] the [name of collaborative effort] benefit the other partners?
Did the [name of collaborative effort] achieve its intended goals?
[Is/Was] there a shared purpose for the [name of collaborative effort]?
[Does/Did] leadership clearly define the roles in the [name of collaborative effort]?
[Is/Was] the [name of collaborative effort] an effective process?
[Is/Was] the [name of collaborative effort] an efficient process?
[Does/Did] the [name of collaborative effort] increase your existing professional network?
[Does/Did] you make any lasting connections or professional development in the [name of collaborative effort]?
[Is/Was] the [name of collaborative effort] inclusive of all members?
[Does/Did] the [name of collaborative effort] have adequate diversity of skills and knowledge?
What suggestions do you have for sustaining the [name of collaborative effort]?
Could you describe the goals of the [name of collaborative effort]?
Have similar collaborative efforts been considered for other health issues?
How could future collaborations be improved?
What advice would you give other organizations considering similar collaborative efforts?
In what stage of development is the [name of collaborative effort]?
B. More activated constituency
Did the [name of collaborative effort, name of product, name of training/workshop, type of technical asssitance] improve heart disease and stroke prevention advocacy efforts?
[Has/Does] the guidance provided by the DHDSP influenced heart disease and stroke prevention advocacy efforts?
Has the [guidance/funding] provided by DHDSP influenced your program to engage in collaborations to increase heart disease and stroke prevention advocacy?
Has the [guidance/funding] provided by DHDSP helped to increase attention to heart disease and stroke prevention among policy and decision-makers?
Has the [name of collaborative effort] successfully reached out to the community?
Does your program provide information to state policy-makers in order to influence heart disease and stroke prevention?
What type of information are your State’s policy-makers interested in receiving?
What has been challenging about assessing community strengths?
Are some [name of collaborative effort] goals or objectives getting more or less attention than others?
What has the [name of collaborative effort] accomplished?
Has the [name of collaborative effort] enhanced your professional
network?
C
X X X X X X X X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
Has the [work of collaborative effort, name of product, name of training/ workshop, type of technical assistance] led to systems changes?
Has the [work of collaborative effort, name of product, name of training/ workshop, type of technical assistance] led to policy changes?
Has the [work of collaborative effort, name of product, name of training/ workshop, type of technical assistance] led to environmental changes?
Did the [name of collaborative effort, name of product, name of training/ workshop, type of technical assistance] bring about changes in [public health/heart disease and stroke knowledge]?
Did the [name of collaborative effort, name of product, name of training/workshop, type of technical assistance] bring about changes in [public health/heart disease and stroke behaviors]?
Did the [name of collaborative effort, name of product, name of training/ workshop, type of technical assistance] advance [priority area, strategic plan goal]?
Did the [name of collaborative effort, name of product, name of training/ workshop, type of technical assistance] result in public health and/or heart disease and stroke impacts?
Have you adopted strategies to achieve [priority areas, strategic plan goals] set forth by the [DHDSP, National Forum]?
Have you expanded the reach of public health strategies designed to achieve [strategic plan goal, priority area]?
Have you implemented strategies to achieve [strategic plan goal, priority area]?
Have you sustained your efforts to achieve [strategic plan goal, priority area]?
Has DHDSP emphasized and encouraged you to achieve [strategic plan goal, priority area]?
How has your state heart disease and stroke program changed over the past [time frame]?
Has the DHDSP has influenced your work on disparities?
Has your program changed its focus (e.g., blood pressure, cholesterol, disparities) since receiving funding from DHDSP at the [funding level]?
Has the funding by DHDSP influenced the focus areas (e.g., blood pressure, cholesterol, disparities) of your program?
Do you have sufficient [resources, funding, capacity] to address [strategic plan goal, priority area] in your [State/agency]?
Do you feel your state is making progress toward achieving [any/all five] of the [priority areas, strategic plan goals] set forth by the [DHDSP, National Forum]?
Are there particular [priority areas, goals] you feel you have excelled in achieving?
Are there particular [priority areas, goals] in which you lack achievements?
Are you aware of the [priority areas, strategic plan goals] set forth by the [DHDSP, National Forum]?
I
X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
What activities have moved your state toward achieving the [priority areas, strategic plan goals] set forth by the [DHDSP, National Forum]?
What type of training activities could assist you in achieving [strategic plan goal, priority area]?
What advice would you give to other [states/groups] working toward achieving the [strategic plan goal, priority area]?
Have you employed specific strategies to achieve [strategic plan goal, priority area]? Please describe these strategies (i.e. activities, interventions, evaluations, etc).
Other than the [strategic plan goals, priority areas] what areas do you think DHDSP should address?
How do State partners perceive the State HDSP Program?
What else do you need from DHDSP to address [strategic plan goal, priority area]?
D. Enhanced ability of programs to apply findings to improve public health: The DHDSP will examine changes in the target audience’s ability to apply findings to improve public health.
Has the “visibility” of the state HDSP program changed over the past [time frame]? If so, how and to what do you attribute this change?
Has the “visibility” of the WISEWOMAN program changed over the past [time frame]? If so, how and to what do you attribute this change?
Has the “visibility” of the Coverdell program changed over the past [time frame]? If so, how and to what do you attribute this change?
What changes in knowledge and practice among [target audience] have occurred as a result of DHDSP guidance?
How will you use the information from [name of training/workshop, name of product, type of technical assistance] to affect your work related to [strategic plan goal, priority area]?
How is your [state /agency] working toward reducing or eliminating disparities?
E. Enhanced competency of the public health workforce: The DHDSP will examine the ways in which its products have enhanced the competency of the public health workforce and increased the capacity of the States and partners to engage in heart disease and stroke prevention efforts.
[Did/Does] the [name of product, type of technical assistance, name of training/workshop] contribute to your ability to enhance program planning?
[Did/Does] the [name of product, type of technical assistance, name of training/ workshop] contribute to your ability to enhance program implementation?
[Did/Does] the [name of product, type of technical assistance, name of training/ workshop] contribute to your ability to collaborate with partners?
[Did/Does] the [name of product, type of technical assistance, name of training/ workshop] contribute to your ability to conduct surveillance?
[
X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
[Did/Does] the [name of product, type of technical assistance, name of training/ workshop] contribute to your ability to [insert job function]?
What are the implications of the [name of product, name of training/workshop, type of technical assistance, name of collaborative effort] for [name of organization’s] activities in the area of strategic plan goal, priority area]?
How has the [name of product, name of training/workshop, type of technical assistance, name of collaborative effort] enhanced the [type of capacity] of your organization?
How has the [name of product, name of training/workshop, type of technical assistance, name of collaborative effort] enhanced your [competency/ability] to [name of skill/activity]?
What changes have occurred in your organization after receiving [name of product, level of funding] from the DHDSP?
What skills did the [name of product, type of technical assistance, name of training/workshop] enhance?
Has the use of any products developed by DHDSP influenced your capacity in heart disease and stroke prevention efforts?
What guidance from DHDSP has been most useful in improving the efforts of your and your partners’ staff?
What guidance from DHDSP has been least useful in improving the efforts of your and your partners’ staff?
F. Enhanced integration among chronic disease programs: The DHDSP will examine the level of engagement and commitment of its partners. The DHDSP expects that its guidance and products will foster collaboration and a more engaged network of states and partners and will enhance integration among chronic disease programs.
Has the DHDSP supported integration among chronic disease programs in your [State/agency]?
Did the [name of collaborative effort] enhance integration of heart disease and stroke among chronic disease programs?
How could the DHDSP further support integration among chronic disease programs in your [State/agency]?
In what ways have the DHDSP guidance and products fostered a more engaged network of states and partners?
Regarding chronic disease integration, where does more work need to be done?
What partners were involved in developing the state’s heart disease and stroke prevention State plan?
How were these partners engaged in the process of developing the State plan?
Will these partners be involved with future revisions of the State plan?
Are there other partners that you would like to involve in future revisions of the State plan?
Was having the [name of partner group] engaged in developing the State plan useful?
W
X X X X X X
X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
Tobacco state program
Diabetes state program
Nutrition and physical activity program
Minority health programs
Coordinated school health programs
State WISEWOMAN
Prevention Research Center(s)
Colleges/universities
Community colleges
Cooperative extensions
Health clinics
[Other partner]
What partners have you collaborated with to develop, implement, and evaluate interventions to address the [strategic plan goals, priority areas]?
What were the roles of those involved in the collaboration?
What collaborations exist in which multiple types of partners work together to address hearth disease and stroke?
Identify which of the following types of partners you have recently collaborated with to develop, implement, or evaluate an intervention aimed at addressing one or more of the [strategic plan goals, priority areas]. (Partners listed below)
State health departments that address related risk factors (e.g., high blood pressure, tobacco use, diabetes), populations, or settings
State-level data partners within the health department (e.g., vital statistics and BRFSS)
State government programs that address heart disease and stroke, or related risk factors or conditions, populations or settings
State government data partners (e.g., Medicaid)
State organizations that
address prevention and control of high blood pressure and high blood cholesterol (e.g., voluntary health organizations)
provide knowledge of and access to Priority Populations (e.g., State black nurses’ association)
provide knowledge and access to settings (e.g., business coalition on health)
improve health (e.g., EMS)
Universities, medical schools, or schools or public health
Community colleges
Cooperative extensions
Rural health clinics
Media organizations
[Other partner]
What partnerships do you think have been most useful in moving your state program to meet HDSP State plan objectives?
W
X X X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
[Do/Have] you provide training to partners in order to increase their knowledge and capacity toward addressing heart disease and stroke? On what topics?
How have collaborations leveraged resources?
What benefits do you expect from an integrated chronic disease approach?
What challenges do you see with an integrated chronic disease approach?
How can these challenges be addressed?
G. Increased focus on heart disease and stroke prevention efforts by states and partners
Has your [state, partners, program] increased its focus on heart disease and stroke prevention efforts?
Has your program changed its focus (e.g., blood pressure, cholesterol, disparities) since receiving funding at the [funding level]?
Has your program changed its focus (e.g., blood pressure, cholesterol, disparities) since [using/receiving/hearing about] [name of product]?
Has your program changed its focus (e.g., blood pressure, cholesterol, disparities) since participating in [name of collaborative effort]?
Has your program changed its focus (e.g., blood pressure, cholesterol, disparities) since attending [name of training]?
Has the funding by DHDSP influenced the focus areas (e.g., blood pressure, cholesterol, disparities) of your program?
H. Increased adoption, reach, and sustainability of recommended public health strategies
Has your work on the [strategic plan goals, priority areas] influenced the work of your partners?
Has the guidance provided by the DHDSP been useful in helping you reach a greater number of [members/partners] in your [state/agency]?
What guidance provided by the DHDSP has been less useful in helping your program reach a greater number of [members/partners] in your [state/agency]?
I. Improved health of the public: The DHDSP will examine long term outcomes of use of its guidance, including changes in the public’s heart disease and stroke knowledge, risk factors, quality of care, emergency response, and health outcomes (e.g. disparities, morbidity and mortality).
Has the guidance and funding provided by DHDSP led to policy or system improvements?
Has the guidance and funding provided by DHDSP led to systems change?
Has the guidance and funding provided by DHDSP led to changes in emergency response times for heart attack and stroke?
Has the DHDSP guidance and funding led to changes in hypertension control?
Has the DHDSP guidance and funding led to changes in cholesterol control?
H
X X X X X X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X
OK to add Follow-Up
OK to add “To what
extent”
Has the DHDSP guidance and funding led to a reduction in disparities for heart disease and stroke?
Have the rates of heart disease and stroke among disparate groups changed based on any guidance by DHDSP?
V. FOLLOW-UP QUESTIONS
This section contains questions which invite the respondent to further explain an answer. In a focus group setting, follow-up questions are used as probes to gain a fuller understanding of a respondent’s initial answer.
How?
How not?
In what way(s)?
What do you mean?
Please explain.
Please give an example
Please describe.
Can you please give an example?
Can you say more about that?
Why?
Why not?
If yes, please describe.
To whom?
Anything else?
What else?
Anyone else?
How could it be improved?
How could it be strengthened?
Why do you say that?
What makes you say that?
Was this positive or negative?
What went well?
What went less well?
Did it meet that need?
Can you say more?
VI. DEMOGRAPHIC
This section includes descriptive questions such as work experience, job role, education level, and funding status.
In what type of agency do you work?
Are you an employee, contractor?
What is your job?
HDSP Program Coordinator/Manager
Epidemiologist
Evaluator
Health educator
Other, please specify
[Other job descriptions]
Please indicate the categories that most accurately describes what you do on your job
Manage program
Conduct evaluation
Analyze health information or data
Provide educational information to the general public
[Other job functions]
What percent of time do you work with the [HDSP program/agency]?
What is the primary organization with which you are affiliated?
How many years of experience do you have in your current [job/position/ agency]?
How long have you been working in public health?
What is your educational level?
What is the concentration area of your degree?
Are [you/your state program] currently funded by the DHDSP?
What [type/level] of funding do you currently receive from the DHDSP?
Before this year, have you received funding from the DHDSP?
How many years have you been funded by the DHDSP?
Please indicate how long [you/your organization] have received [type of funding]?
Besides financial resources, what other kinds of resources has CDC provided to [you/your organization]?
What [type/level] of funding do you currently receive from sources other than DHDSP?
In the past [type frame], what kinds of funding [or other resources] have you received for heart disease and stroke prevention from sources other than DHDSP? Please list the organization name, type of resources received, amount of funding, and period of the grant or award.
Please describe the working relationship you have had with DHDSP over the last 12 months.
Is the relationship during the past year different than previous years?
If yes, how?
What is the [geographic/demographic] make-up of the population you serve?
What is your priority population served?
Where are you located (rural/urban/suburban, name of state, county, city)?
How many clients do you propose to reach?
What is [you/your organization’s] role in the [name of training/workshop, name of collaborative effort]?
| File Type | application/msword |
| File Title | Relevance |
| Author | hrv9 |
| Last Modified By | arp5 |
| File Modified | 2009-04-14 |
| File Created | 2009-04-08 |