Evidence Building Academy Midpoint Evaluation Form

Evidence Building Academy

Midpoint Evaluation Form

Thank you for participating in the Evidence Building Academy! We are gathering information about your experience to inform the remainder of the Academy and potential future activities. Your participation in this survey is completely voluntary, and all responses will be reported at an aggregate level.

The form should take about 10-15 minutes to complete.

Thank you in advance for your time and input. If you have any questions, please contact us at [email protected].

-----

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN:

This information is being collected to provide the Evidence Building Academy Planning Team feedback regarding participants’ experience with the Academy thus far. The information will be used by the Planning Team to tailor future logistics and content of the Academy. Public reporting burden for this collection of information is estimated to average 10 minutes per respondent, including the time for reviewing instructions, gathering and reviewing information, preparing slides, and presenting them during the Academy. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0401 and the expiration date is 05/31/2021. If you have any comments on the collection information, please contact: Mike Pergamit ([email protected]) or Mark Courtney ([email protected]).

1. Which of the following most closely reflects your current position? (select all that apply)

• Implementer (please indicate the setting)

  1. Community-based Service Provider

  2. State or Local Government

• Evaluator (please indicate the setting)

• University/Academic

• Research Firm/Institution

• State or Local Government

• Other, please specify: _________________

Academy Logistics

2. Please rate the following statements:

	Very Poor	Poor	Fair	Good	Excellent
Ease of the application and registration process:…….......	1	2	3	4	5
Use of the meeting resource page (i.e., Box):………………..	1	2	3	4	5
Use of the meeting platform (i.e., Zoom Meetings):………	1	2	3	4	5
Logistics staff assistance:..……………………………………….......	1	2	3	4	5

Overall Academy

3. Please rate the following statements:

	Very Poor	Poor	Fair	Good	Excellent
Overall, the July Academy was:……………………………….	1	2	3	4	5
The meeting materials (e.g., Participant Guide, Syllabus) provided were:…………………………………........	1	2	3	4	5
The cohort breakouts (e.g., lightning rounds, theory of change exercise) were:………………………………………	1	2	3	4	5
The team breakouts on the last day were:…………….	1	2	3	4	5

4. Please indicate how much you agree with the following statements:

	Strongly Disagree	Disagree	Neither Agree nor Disagree	Agree	Strongly Agree
The material was relevant for my evaluation activities and interests.	1	2	3	4	5
The complexity of the material was appropriate for me.	1	2	3	4	5
The length of the presentations was appropriate for the content covered.	1	2	3	4	5
There was an appropriate balance between the time spent presenting and opportunities to ask questions.	1	2	3	4	5
The presenters were responsive to questions.	1	2	3	4	5
There was an appropriate balance between the time spent in presentations and the time spent in breakout rooms.	1	2	3	4	5

5. Please rate the following statements:

The module increased my understanding of evaluation:	Did Not Attend	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly Agree
1. Introduction to Evaluation (June25th) [Bridgette Lery & Marla McDaniel]………	N/A	1	2	3	4	5
2. Key Concepts [Katrina Brewsaugh]……………………………	N/A	1	2	3	4	5
3. Types of Evaluation [Carrie Furrer & Marla McDaniel]………..	N/A	1	2	3	4	5
4. Implementation and Fidelity [Patti Chamberlain]……………………..………	N/A	1	2	3	4	5
5. Defining Study Population [Mike Pergamit]…………………………………..	N/A	1	2	3	4	5
6. RCT Design in the Real World (Evaluators) [Mike Pergamit]………………	N/A	1	2	3	4	5
7. Implementing RCTs (Implementers) [Sarah Hurley & Trevor Williams]………..	N/A	1	2	3	4	5
8. Evaluation Design: QED [Mark Courtney & Fred Wulcyzn]…………	N/A	1	2	3	4	5
9. What to Include in an Evaluation Plan [Karin Malm & Amy McKlindon]…………..	N/A	1	2	3	4	5

6. Which module(s) did you find most useful to you and why?

7. Which module(s) did you find the least useful to you and why?

8. As you have thought about the Academy content and your own evaluation projects and interests, you may have found that you have questions for the faculty. We would like to address as many of your questions as possible, although we do not have the capacity to provide intensive, individualized technical assistance. Faculty may, for example, offer “virtual office hours” to address common questions. To help us connect you with the appropriate Academy faculty, please ask your question below, and indicate the relevant Academy session. If you do not have any questions for the faculty, please write “NA.”

Select one:

• Introduction to Evaluation (June 25th webinar) [Bridgette Lery & Marla McDaniel]

• Key Concepts [Katrina Brewsaugh]

• Types of Evaluation [Carrie Furrer & Marla McDaniel]

• Implementation and Fidelity [Patti Chamberlain]

• Defining Study Population [Mike Pergamit]

• RCT Design in the Real World (Evaluators) [Mike Pergamit]

• Implementing RCTs (Implementers) [Sarah Hurley & Trevor Williams]

• Evaluation Design: QED [Mark Courtney & Fred Wulcyzn]

• What to Include in an Evaluation Plan [Karin Malm & Amy McKlindon]

• Other (I have a question but it does not apply to a particular module)

• Not Applicable (no questions)

8. If you have another question, please ask it below, and indicate the relevant Academy session. If you do not have any additional questions for the faculty, please write “NA.”

Select one:

• Introduction to Evaluation (June 25th webinar) [Bridgette Lery & Marla McDaniel]

• Key Concepts [Katrina Brewsaugh]

• Types of Evaluation [Carrie Furrer & Marla McDaniel]

• Implementation and Fidelity [Patti Chamberlain]

• Defining Study Population [Mike Pergamit]

• RCT Design in the Real World (Evaluators) [Mike Pergamit]

• Implementing RCTs (Implementers) [Sarah Hurley & Trevor Williams]

• Evaluation Design: QED [Mark Courtney & Fred Wulcyzn]

• What to Include in an Evaluation Plan [Karin Malm & Amy McKlindon]

• Other (I have a question but it does not apply to a particular module)

• Not Applicable (no questions)

8. Faculty are planning for the remainder of the Academy sessions and would like to make these sessions as relevant to you as possible. If you have a question about an upcoming topic, please ask it below, and indicate the relevant Academy session. If you do not have any questions, please write “NA.”

Select one:

• Power Analysis

• Measuring Outcomes

• Types of Data for Impact Evaluation

• Writing a Strong Research Report or Journal Article

• How to Critically Appraise Evidence in Child Welfare

• Presenting Evidence Visually

  • Other (I have a question but it does not apply to a particular module)

  • Not Applicable (no questions)

8. If you have another question, please ask it below, and indicate the relevant Academy session. If you do not have any additional questions for the faculty, please write “NA.”

Select one:

• Power Analysis

• Measuring Outcomes

• Types of Data for Impact Evaluation

• Writing a Strong Research Report or Journal Article

• How to Critically Appraise Evidence in Child Welfare

• Presenting Evidence Visually

  • Other (I have a question but it does not apply to a particular module)

  • Not Applicable (no questions)

12. How can we improve the Academy format (e.g., length of Academy, module format, breakout room system)?

12. Is there anything else we can do to improve the Academy, or are there any other thoughts you would like to share?

Thank you for completing this evaluation form.

8