Building Evidence Training (BET) Project Usability Testing

OMB # 0970-0401

Expiration Date: 05/31/2021

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN: The purpose of this information collection is to gain feedback on the course content and delivery. Public reporting burden for this collection of information is estimated to average .12 hours per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. If you have any comments on this collection of information, please contact the Building Evidence Training Project at: [email protected]

Designing and Implementing Evidence-Supported Interventions in Child Welfare

Unit 6 pre/post test: Full Implementation

Questions

1. An agency can make a strong case for sustaining an intervention when:

1. The intervention is well utilized by the target population.

2. Recipients of the intervention are experiencing the short-term and ultimate desired outcomes predicted by the theory of change.

3. The outcomes experienced have value to the target population, agency, and stakeholders.

4. All of the above. 

2. Select the statement that best describes the difference between data on intervention impact and data on implementation fidelity.

1. Data on intervention impact is focused on monitoring and assessing processes; data on implementation fidelity is focused on the results of rigorous evaluations.

2. Data on intervention impact addresses short-term outcomes predicted in the theory of change; data on implementation fidelity addresses long-term outcomes predicted in the theory of change.

3. Data on intervention impact is based on observation of staff and record reviews; data on implementation is based on reports from intervention recipients.

4. Data on intervention impact addresses improvement in outcomes; data on implementation fidelity addresses whether the intervention is being delivered as intended. 

3. Select the answer that is not a key function of a sustainability plan:

1. Continuing to expand the implementation of the intervention to additional jurisdictions. 

2. Continuing to gather and monitor data that can be used for continuous quality improvement and decision making.

3. Maintaining the implementation supports such as recruitment and hiring, training, coaching, and fidelity assessment.

4. Maintaining staff, leadership, and community stakeholder support for the intervention.

5. Continuing to secure financial resources necessary to provide or contract for the intervention.

4. Scaling of an intervention refers to:

1. How an agency uses data to monitor an intervention and evaluate its impact.

2. Sustaining and improving outcomes and the conditions necessary to achieve those outcomes.

3. Increasing the capacity to offer the intervention to members of the target population throughout the agency’s jurisdiction. 

4. Developing measurement instruments to assess fidelity.

5. Select the answer that correctly orders the major stages of implementation. (modified cumulative question)

1. Initial Implementation, Exploration, Installation, Full Implementation, and Installation

2. Initial Implementation, Installation, Exploration, and Full Implementation

3. Exploration, Installation, Initial Implementation, and Full Implementation 

4. Exploration, Initial Implementation, Full Implementation, and Installation

5. Exploration, Initial Implementation, Installation, Full Implementation

6. How do scientifically rigorous evaluations to build evidence for interventions differ from the monitoring and assessing approach to evaluation described in Unit 6?

1. Rigorous evaluations include assessments of utilization and fidelity but the monitoring and assessing approach does not.

2. Rigorous evaluations rely primarily on the analysis of quantitative data while the monitoring and assessing approach relies primarily on anecdotal reports.

3. Rigorous evaluations measure both short and long-term outcomes more carefully than the monitoring and assessing approach.

4. Rigorous evaluations use random assignment or other procedures to assess whether outcomes are caused by the intervention or other factors. 

5. All of the above.

7. Implementation supports are an important theme in this course. Why are they so critical to conducting a successful implementation process?

1. Implementation supports are necessary to assure that an intervention is provided with high quality and fidelity. 

2. Implementation supports represent a major investment for an agency, requiring that they be applied carefully to avoid wasting resources.

3. Implementation supports provide the connective tissue among the various teams involved with implementation that facilitates the communications necessary for effective implementation.

4. All of the above.

8. If data on utilization shows the numbers of persons referred to, beginning, and completing the intervention are lower than was planned for, the implementation team should investigate:

1. If staff are referring to the intervention and supporting the intervention

2. If leadership support of the intervention through resources and communication

3. How the target population experiences and the community views the intervention

4. All of the above 

9. True or False. For a variety of reasons, agencies may rightly choose that it is not appropriate to sustain an effectively implemented intervention.

1. True 

2. False

10. Why is it critical to include frontline workers and supervisors on teams that implement interventions? (modified cumulative question)

1. They perform the role of securing human and financial resources.

2. They have important knowledge about children, families and services in the jurisdiction and likely will be asked to perform activities critical to the intervention’s success. 

3. Other persons who could be chosen for implementation team members are busier with responsibilities to perform for their agencies.

4. It only is critical to involve frontline workers and supervisors who will actually be delivering the intervention. Otherwise, their participation is not important.

5. All of the above.

Unit 6 Pre/Post Test Questions 3