Request for feedback from Teen Pregnancy Prevention Program (TPP) Grantees on the technical assistance (TA) and training provided to them by the Office of Adolescent Health (OAH)

Form Approved

0990-0379

Exp. Date: XX/XX/XX2X

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0990-0379. The time required to complete this information collection is estimated to average 11 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Health & Human Services, OS/OCIO/PRA, 200 Independence Ave., S.W., Suite 336-E, Washington D.C. 20201, Attention: PRA Reports Clearance Officer.

Office of Adolescent Health Technical Assistance (TA) Survey Modules

[Note: We will tailor these items to the specific TA activity, or drop items that are not relevant. For example, we will ask about either recent webinars or conference presentations but not both]

Annual survey

Module 1: Annual survey for feedback on overall TA activities

Please take a few minutes to provide feedback about your overall experience with the Evaluation Technical Assistance (Eval TA) team for the Office of Adolescent Health’s Teen Pregnancy Prevention Program. Your responses are anonymous and will be used for program improvement purposes only. Thank you for your time!

For the questions below, please indicate the extent to which you agree or disagree with the following statements.

SD – I strongly disagree with this statement.

D – I disagree with this statement.

N – I neither agree nor disagree with this statement.

A – I agree with this statement.

SA – I strongly agree with this statement.

NA – Not applicable.

Please indicate your response to the following statements, which relate to the Evaluation Technical Assistance provided, including resources and tools.
The reporting templates such as the formative and process evaluation report template and guidance were concise and clear	SD	D	N	A	SA	NA
The individual conference telephone calls with the Evaluation TA Team were beneficial to the progress of my evaluation	SD	D	N	A	SA	NA
The TPPEval18 page on Max.gov was user friendly, and I was able to access resources and updates on the website	SD	D	N	A	SA	NA
I was provided with evaluation tools that were insightful and useful	SD	D	N	A	SA	NA
The Evaluation TA provided a better understanding of rigorous evaluation	SD	D	N	A	SA	NA
In the future, I will utilize the knowledge gained from Evaluation TA	SD	D	N	A	SA	NA
Overall, I believe my evaluation is higher quality for having received Evaluation TA	SD	D	N	A	SA	NA

Please indicate your responses to the following statements, which relate to your individual experiences with members of the Evaluation TA team during consultations about your projects.
The Evaluation TA Team members were competent in the subject matter	SD	D	N	A	SA	NA
The Evaluation TA Team members were prepared during the telephone calls	SD	D	N	A	SA	NA
The Evaluation TA Team members provided helpful feedback on our research design.	SD	D	N	A	SA	NA
The Evaluation TA Team members explained the subject matter in a style that was clear and concise	SD	D	N	A	SA	NA
I was able to reach the Evaluation TA Team members if I wanted assistance	SD	D	N	A	SA	NA
The Evaluation TA Team members provided a timely response to my inquiries	SD	D	N	A	SA	NA
I felt comfortable asking questions to the Evaluation TA Team members	SD	D	N	A	SA	NA
The Evaluation TA Team members were pleasant to work with	SD	D	N	A	SA	NA

The following, and final section provides an opportunity to openly discuss aspects of the Evaluation Technical Assistance. Your candid feedback will help us improve areas of opportunity within Evaluation TA, and further strengthen areas that are currently working well.

16. What did you enjoy MOST from the Evaluation TA?

______________________________________________________________________________________________________________________________________________________

17. What did you enjoy LEAST from the Evaluation TA?

______________________________________________________________________________________________________________________________________________________

18. Do you have suggestions for improving the Evaluation TA?

______________________________________________________________________________________________________________________________________________________

19. Were there additional tools and/or resources that you believe would have been beneficial?

______________________________________________________________________________________________________________________________________________________

Were the topics that we discussed new or a refresher? Were there any topics that you wish we had focused more on? Less focus on?

Topic (to be updated with content provided in 2019)	New	Refresher	Wish We Focused on More	Wish We Focused on Less
20. Conducting a formative evaluation
21. Monitoring program implementation for continuous improvement
22. Continuous improvements to help identify successful practices
23. Assessment of whether a project is meeting a significant community need
24. Identifying core implementation components
25. Designing a summative evaluation

Other TA surveys

Module 2: Feedback on group technical assistance activities (delivered via webinar or conference presentation)

Please take a few minutes to provide feedback about your experience attending a recent [webinar or conference presentation] by the Evaluation Technical Assistance (Eval TA) team for the Office of Adolescent Health’s Teen Pregnancy Prevention Program. Your responses are anonymous and will be used for program improvement purposes only. Thank you for your time!

For the questions below, please indicate the extent to which you agree or disagree with the following statements.

SD – I strongly disagree with this statement.

D – I disagree with this statement.

N – I neither agree nor disagree with this statement.

A – I agree with this statement.

SA – I strongly agree with this statement.

NA – Not applicable.

1. The goals for the webinar/conference presentation were clearly stated at or before the beginning of the event.	SD	D	N	A	SA	NA
2. The structure of webinar/conference presentation was appropriate for meeting the stated goals.	SD	D	N	A	SA	NA
3. The information was presented clearly and provided examples or practical suggestions for implementation.	SD	D	N	A	SA	NA
4. The webinar/conference presentation actively engaged me in learning the content.	SD	D	N	A	SA	NA
5. The webinar/conference presentation was relevant to an issue currently facing my organization for the TPP grant.	SD	D	N	A	SA	NA
6. The webinar/conference presentation has increased my capacity to use research or effectively incorporate data into decision making for this TPP grant.	SD	D	N	A	SA	NA
7. I expect to apply and share information from the webinar/conference presentation.	SD	D	N	A	SA	NA
8. The content of the webinar/conference presentation will help strengthen my formative/summative evaluation plan.	SD	D	N	A	SA	NA
9. The benefits of attending this webinar/conference presentation were worth the time I invested.	SD	D	N	A	SA	NA

10. What aspects of the webinar/conference presentation were most helpful and why?

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

10. What aspects of the webinar/conference presentation were least helpful and why?

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

10. What additional information, support, tools, or activities on this topic would help you apply the information to your own work?

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

10. What part of this event would you suggest changing to make it better for future participants?

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

10. As a result of the webinar/conference presentation, I plan to take the following action steps:

a)________________________________________________________________________________ b)________________________________________________________________________________

c) _______________________________________________________________________________

15. What is the most pressing need for your evaluation that you would like us to address in a future webinar/conference presentation or an individual TA consultation (Note: if you’d like to make an individual TA request, please do that through the Max.gov portal).

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

16. In what areas do you anticipate needing evaluation technical assistance? (check all that apply) (Note: if you’d like to make an individual TA request, please do that through the Max.gov portal).

• Refining my summative evaluation design

• Developing and piloting survey instruments

• Recruitment

• Tracking and retention

• Data collection methods

• Implementation research methods

• Quantitative data analysis

• Qualitative data analysis

• Continuous quality improvement

Module 3: Individual consultation TA Feedback

Please take a few minutes to provide feedback about your TA interactions with the Evaluation Technical Assistance (Eval TA) team for the Office of Adolescent Health’s Teen Pregnancy Prevention Program. Your responses are anonymous and will be used for program improvement purposes only. Thank you for your time!

For the questions below, please indicate the extent to which you agree or disagree with the following statements.

SD – I strongly disagree with this statement.

D – I disagree with this statement.

N – I neither agree nor disagree with this statement.

A – I agree with this statement.

SA – I strongly agree with this statement.

NA – Not applicable.

1. The consultation offered by the TA Team was relevant to a particular issue facing my organization’s TPP project.	SD	D	N	A	SA	NA
2. The consultation offered by the TA Team increased my understanding of the topic.	SD	D	N	A	SA	NA
3. The consultation offered by the TA Team increased my awareness of available research and data on this topic or issue.	SD	D	N	A	SA	NA
4. The consultation offered by the TA Team was clear, concise, and provided me with next steps to refine my evaluation.	SD	D	N	A	SA	NA
5. The TA team took the time to understand my program and target population before and during our consultation.	SD	D	N	A	SA	NA
6. The consultation offered by the TA Team increased my capacity to use research and data to solve problems in my agency or organization.	SD	D	N	A	SA	NA
7. I would likely participate in additional support activities offered by the TA Team.	SD	D	N	A	SA	NA
8. The consultation provided by the TA Team will enable me to strengthen my formative/summative evaluation plan.	SD	D	N	A	SA	NA
9. My organization received the appropriate level of support from the TA Team for his consultation.	SD	D	N	A	SA	NA
10. The benefits from this project were worth the time and resources my organization invested to participate.	SD	D	N	A	SA	NA

11. What aspects of the technical assistance/consultation were most helpful and why?

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

11. What aspects of the technical assistance /consultation were least helpful and why?

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

11. What additional information, support, tools, or activities on this topic would help you apply the information gained during the consultation?

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

11. What part of this technical assistance /consultation would you suggest changing to make it better for future participants?

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

11. As a result of technical assistance/consultation, I plan to take the following action steps to refine my formative/summative evaluation:

a)________________________________________________________________________________ b)________________________________________________________________________________

c)________________________________________________________________________________

11. Please list more specific areas of need or interest to your agency or organization for future TA workshops or products.

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

Module 4: Product satisfaction

Please take a few minutes to provide feedback about your experience with products recently created by the Evaluation Technical Assistance (Eval TA) team for the Office of Adolescent Health’s Teen Pregnancy Prevention Program. Your responses are anonymous and will be used for program improvement purposes only. Thank you for your time!

For the questions below, please indicate the extent to which you agree or disagree with the following statements.

SD – I strongly disagree with this statement.

D – I disagree with this statement.

N – I neither agree nor disagree with this statement.

A – I agree with this statement.

SA – I strongly agree with this statement.

NA – Not applicable.

The [insert name of recent product] that was provided was useful and timely.	SD	D	N	A	SA	NA
[Insert name of recent product] provided clear guidance on carrying out a key aspect of my evaluation.	SD	D	N	A	SA	NA
The [insert name of recent product] provided a better understanding of rigorous evaluation.	SD	D	N	A	SA	NA
I plan to use techniques that were described in the [insert name of recent product].	SD	D	N	A	SA	NA
The [insert name of recent product] was tailored to an immediate need on my project.	SD	D	N	A	SA	NA
The [insert name of recent product] helped me understand how to carry out my plans for a [formative evaluation/continuous quality improvement strategy/summative evaluation].	SD	D	N	A	SA	NA
I expect to apply information from [insert name of recent product] in my evaluation.	SD	D	N	A	SA	NA
The [insert name of recent product] was written clearly and explained the content concisely.	SD	D	N	A	SA	NA
The [insert name of recent product] was visually appealing and carefully used graphics to enhance the content.	SD	D	N	A	SA	NA

10. What types of topics for TA products would be most helpful and why?

__________________________________________________________________________________________________________________________________________________________________________

_____________________________________________________________________________________

11. What format for TA products would be most helpful and why?

____________________________________________________________________________________________________________________________________________________________