Building Evidence Training (BET) Project Usability Testing

OMB # 0970-0401

Expiration Date: 05/31/2021

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN: The purpose of this information collection is to gain feedback on the course content and delivery. Public reporting burden for this collection of information is estimated to average .59 hours per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. If you have any comments on this collection of information, please contact the Building Evidence Training Project at: [email protected]

Designing and Implementing Evidence-Supported Interventions in Child Welfare

knowledge check questions

Each unit module includes 1-2 knowledge check questions. Correct answers are underlined.

Unit 1

1. Select the answer that defines evidence-supported intervention:

1. Any specific practice, service, policy, strategy, program, practice model, or combination of these that is clearly defined, operationalized, and distinguishable from alternatives.

2. The integration of the best available research evidence with clinical and child welfare expertise in a manner consistent with the child’s, family’s, and community’s values.

3. The results of rigorous evaluation studies conducted to test the impact of interventions on children and families.

4. Any specific practice, service, policy, strategy, program, practice model, or combination of thereof, which the best available evidence shows, based on rigorous evaluation, has the potential to improve the outcomes for children and families.

Unit 2

1. Select the statement that is not a reason why teams are critical to implementation.

1. Teaming reinforces the need for a top-down approach to implementation processes.

2. Implementation requires a variety of knowledge, skills, and agency roles.

3. Participation increases team members’ understanding and buy-in for implementation and evaluation activities.

4. Implementation through teams promotes communication about the intervention to agency staff, clients, and stakeholders.

Unit 3

1. Systemic barriers refer to:

1. How root causes of a problem may be impacted to achieve specified outcomes.

2. Individuals or families who are at risk for a problem and are intended to be positively impacted by an intervention.

3. Underlying reasons.

4. Challenges related to lack of staffing or other resources and issues under the control of the legislature, courts, or other agencies.

2. Developing a theory of change can help an agency plan for the evaluation of the intervention by:

1. Identifying the short- and long-term impacts of the intervention to be measured in the evaluation

2. Providing external oversight of the evaluation as a substitute for court monitors or Community Oversight Boards

3. Reducing the need for teaming with evaluators because the outcomes are already known

4. Serving as a standing document to use in communication with stakeholders that doesn’t need to be revisited by the team

Unit 4

1. An agency is considering how to adapt an evidence-supported intervention (ESI) to better serve the needs of the agency’s target population. Which of the following statements is the best guidance for the implementation team?

1. Whenever the team believes an adaptation will make an ESI more responsive to the needs of the target population, the team should make the adaptation.

2. If any adaptation is made to an ESI, the evidence supporting the use of the ESI is no longer applicable.

3. The team should carefully weigh whether an adaptation alters an ESI’s core components, perhaps by consulting with the intervention developer or another expert.

4. All of the above.

2. Which of the following statements is inaccurate?

1. The recruitment and selection of the right staff is important to an intervention’s success.

2. Staff training and coaching is important to ensure that interventions are implemented as intended.

3. An advantage of implementing an evidence-supported intervention in a new jurisdiction with fidelity is that no evaluation is necessary.

4. Leadership and stakeholder support is important to successful implementation.

Unit 5

1. A clearly operationalized intervention:

1. Contains loose guiding principles about how the intervention operates

2. Requires the implementing organization to further define the essential functions

3. Includes a specific description of how a practitioner interacts with a client during a first meeting

4. Promises to address all of the problems identified by the organization

2. Which results of usability testing suggest an agency is ready to move forward with implementation?

1. Everything is working perfectly and there is nothing left to test.

2. The innovation needs more clearly defined essential functions.

3. There are multiple systemic barriers.

4. Critical components are working well enough to proceed.

Unit 6

1. Which of the following statements is not consistent with what the course has taught about using different types of data:

1. Implementation support data can help a team understand problems in utilization.

2. If utilization is low, a large impact would not be expected.

3. Impact data tells the implementation team whether the intervention was implemented with fidelity.

4. Data on short-term impacts tells the implementation team whether changes occurred that were expected to lead to ultimate desired results.

2. True or False: After an intervention is implemented and found to be effective, implementation supports to assure the intervention is delivered as intended are no longer necessary.

A Project Funded by the Children’s Bureau 1