Data Collection Plan for the Center for States Evaluation Ancillary Data Collection
OMB Information Collection Request
0970 – 0501
Supporting Statement Part B – Statistical Methods
Submitted By:
Children’s Bureau
Administration for Children and Families
U.S. Department of Health and Human Services
SUPPORTING STATEMENT B – STATISTICAL METHODS
Child Welfare Virtual Conference Session Surveys: This survey will be administered to all participants of the Child Welfare Virtual Conference at the conclusion of each session (6 sessions in total). The expected response rate is 60 percent per session, similar to response rate in previous administrations in 2017, 2018, and 2019. Potential participants include professionals who work in state, territory, and tribal child welfare agencies or in related professions, such as the courts and community-based service providers.
Child Welfare Virtual Conference Interview Guide: These interviews will be conducted with a voluntary sample of up to 20 Child Welfare Virtual Conference participants after the Child Welfare Virtual Conference has concluded.
Child Welfare Virtual Conference Focus Group: These focus groups will be conducted with a voluntary sample of up to 30 Child Welfare Virtual Conference participants after the Child Welfare Virtual Conference has concluded.
Child Welfare Virtual Conference Registration Form: This collection will be administered to all registrants Child Welfare Virtual Conference.
Child Welfare Virtual Conference Exit Survey: This survey will be administered to all participants of the Child Welfare Virtual Conference at the conclusion the conference. The expected response rate from all attendees is 30 percent given attrition in expected conference attendance over the course of the day, and similar to response rate in previous administrations in 2017, 2018, and 2019.
Tailored Services Practice Model Survey: This survey will be appended to the existing Tailored Services intensive or brief project satisfaction surveys (OMB #0970-0484) for up to 26 projects (1 per state) per year. Sample of projects surveyed will be determined in consultation with tailored services staff, using criteria such as adequate representation of states, focus area of projects, etc.
Assessment Observation– Group Debrief: A group debrief will be held at the conclusion of an onsite meeting as part of the assessment process for up to 25 states per year with 2 people expected to participate in each meeting. This onsite meeting is part of the Center for States’ tailored services process.
Service Delivery and Tracking and Adjustment Focus Observation – Group Debrief: A group debrief will be held at the conclusion of an onsite meeting as part of the service delivery process for up two projects pe state, ten states in total. The plan is to debrief these 20 projects twice with 2 participants per debrief. This onsite meeting is part of the Center for States’ tailored services process.
Assessment and Service Delivery State Lead Interviews – Supplemental Questions: An individual phone interview will be conducted with a key stakeholder per state for up to 30 states per year. These questions will be included in Tailored Services Interviews (OMB #0970-0494) as needed, without increasing burden
Annual Assessment Update: Participants in the assessment will include a key child welfare agency staff representative from up to 54 states/territories who can respond to the domains addressed in the assessment protocol through verbal responses and pulling information from existing state documentation.
No statistical methodology for stratification and sample selection will be used for any of the data collection instruments.
Methods to Maximize Response Rates and Deal with Nonresponse
Maximizing response rates is critical to the administration of the aforementioned surveys. The content and format of the instruments were developed in close consultation with key stakeholders and were informed by previously developed measures.
Strategies that emphasize flexibility, confidentiality, and a respect for the respondent’s time facilitate timely participation. The following strategies will be implemented to maximize participation in the data collection and achieve the desired response rates: 1
Introduction and notification:
Strategies to introduce and notify respondents about data collection are used for several instruments. An introductory email will be sent to inform all self-identified respondents about the administration of the Child Welfare Virtual Conference Interviews and Child Welfare Virtual Conference Interviews Focus Groups. Follow-up introductory emails will be sent to prospective respondents to introduce the evaluation team, share logistical information, and to address any questions about the data collection. For the Child Welfare Virtual Conference Session Surveys and Child Welfare Virtual Conference Exit Survey participants are told at the beginning of the conference that their feedback is critical to helping us improve programs and are provided a survey at the end of each session.
For Tailored Services Practice Model Survey: Center for States or cross-center evaluators will inform potential participants about the survey prior to its release. All participants will receive an email invitation asking them to please participate in the survey. They will be provided an email link to take the survey electronically. A follow-up email reminder will be sent out after one week to increase response rates.
For the Assessment and Service Delivery State Lead Interviews – Supplemental Questions: Center for States will notify State participants in person or by phone of the option to participate in these interviews to help improve service delivery and understanding of capacity building approach effectiveness. Potential participants will opt-in if they are interested. Follow-up introductory emails will be sent to prospective respondents to introduce the evaluation team, share logistical information, and to address any questions about the data collection. These questions will be administered alongside the Tailored Services Interview (OMB # 0970-0494) for up to 12 purposively selected service delivery projects and will be administered in isolation for up to 10 states related to assessment services and an additional 8 states related to service delivery.
For the Assessment Observation– Group Debrief and Service Delivery and Tracking and Adjustment Focus Observation – Group Debrief: These questions will be administered following observation at an onsite meeting. Participants will be notified ahead of time regarding the Center for States staff observation at the onsite meeting and will opt in to participate. The debrief will be included in a meeting schedule to be determined by all participants.
For the Annual Assessment Update (8 systematic questions) – These questions will be asked as part of the annual assessment process with each State. Center for States staff will notify State staff ahead of time regarding the purpose and content of the assessment process and these questions will be included in that notification. States will voluntarily decide to participate in the assessment process and work with Center for States staff on scheduling.
Timing of data collection:
The Child Welfare Virtual Conference Session Surveys and Child Welfare Virtual Conference Exit Survey will be administered throughout the virtual conference period following the administration guidelines described in B1. In order to maximize participant recall, the Child Welfare Virtual Conference Interviews and Child Welfare Virtual Conference Focus Groups will be administered following the event. The Tailored Services Practice Model Survey will be administered as individual service projects end/close out throughout the year. The Assessment Observation – Group Debrief and Service Delivery and Tracking and Adjustment Observation – Group Debrief will occur at the conclusion of the identified meetings. The Assessment and Service Delivery State Lead Interviews – Supplemental Questions will occur when a state is engaged in the assessment process or when a service project ends.
Pre-interview preparation:
A copy of the debrief, interview or focus group guide will be sent to respondents in advance of any interview or focus group.
Administration:
For the Child Welfare Virtual Conference Session Surveys, at the conclusion of each session participants will receive a pop-up form of the survey with a request to complete the survey. For the Child Welfare Virtual Conference Exit Survey, at the conclusion of the conference, participants will receive a pop-up form of the survey with a request to complete the survey. They will also receive a link to the survey in a thank you e-mail after the conference. A follow up email reminder will be sent out after one week to increase response rates. For the Tailored Services Practice Model Survey, all participants will receive an email asking them to please participate in the survey. They will be provided an email link to take the survey electronically. A follow-up email reminder will be sent out after one week to increase response rates. Electronic participation will allow respondents the flexibility to complete the survey at the most convenient time with minimal burden. For approximately four weeks after sending this initial email, weekly reminder emails will be sent to those respondents who have not yet completed the survey. For all interviews, focus groups, and debriefs – administration will be either in person if possible or, if not, by telephone/adobe connect virtual meeting. Participants will have the opportunity to volunteer and will not be required to participate. For the Annual Assessment Update, questions will be asked verbally during the course of the onsite meeting focused on state assessment.
Alternate response methods:
Respondents will be given the option to use an alternate method for all interviews and focus groups. If a respondent prefers to submit written responses to a survey in lieu of participating in a telephone interview, we will provide him/her with an electronic version to complete via web, fax, email, or mail. Similarly, paper versions of any of the electronic surveys will be sent to respondents upon request or will be administered through a telephone interview if requested to accommodate any special needs.
There are no incentives provided for participation in any of the surveys.
Test of Procedures or Methods to be Undertaken
The Center for States’ evaluation instruments contained herein were subject to review and feedback by key stakeholders, including CB.
User access and responsiveness to the web-based methodology for completing the Child Welfare Virtual Conference Registration, Child Welfare Virtual Conference Session Surveys, Child Welfare Virtual Conference Exit Survey and Tailored Services Practice Model Survey was informally tested by a small group of Center for States staff. No modifications were needed for implementation of the full data collection.
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
National Cross-Center Evaluation Contractor |
|
James Bell Associates 1001 19th Street North, Suite 1500 Arlington, VA (703) 528-3230 |
ICF International 9300 Lee Highway Fairfax, VA 22031 |
National Capacity Building Center for States |
|
ICF International 9300 Lee Highway Fairfax, VA 22031 |
|
Should you have any questions about the contents of this OMB submission package, please contact one of the following individuals:
Roshanda Shoulders, Children’s Bureau, 1250 Maryland Avenue, SW, Washington, DC 20024, telephone: 202.401.5323, email: [email protected]
Christine Leicht, ICF, 9300 Lee Highway Fairfax, VA 22031, telephone: 703.934.3000, email: [email protected]
1 Strategies that pertain to two or more data collections are discussed together.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement for the |
| Author | Shamegan |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |