SUPPORTING STATEMENT FOR
Center for States Evaluation Ancillary Data Collection
Department of Health & Human Services
Children’s Bureau
Washington, DC
Contact person:
Brian Deakins
Children’s Bureau
Administration on Children, Youth and Families
Mary E. Switzer Building
3rd Floor, Mailstop 3602
330 C Street, SW
Washington, DC 20201
202-205-8769
Table of Contents
Section B: Collections of Information Employing Statistical Methods 3
1. Respondent universe and sampling methods 3
2. Procedures for the collection of information. 3
3. Methods to maximize response rates and deal with nonresponse 3
4. Tests of procedures or methods to be undertaken 5
5. Individuals consulted on statistical aspects and individuals collecting and/or analyzing data 6
Section B: Collections of Information Employing Statistical Methods
Child Welfare Virtual Conference Session Surveys: This survey will be administered to all participants of the Child Welfare Virtual Conference at the conclusion of each session (6). The expected response rate is 60 percent per session.
Child Welfare Virtual Conference Interview Guide: These interviews will be conducted with a voluntary sample of up to 20 Child Welfare Virtual Conference participants after the Child Welfare Virtual Conference has concluded.
Child Welfare Virtual Conference Focus Group: These focus groups will be conducted with a voluntary sample of up to 30 Child Welfare Virtual Conference participants after the Child Welfare Virtual Conference has concluded.
Child Welfare Virtual Conference Registration Form: This collection will be administered for all State and territory public child welfare agency staff, federal staff, and other stakeholders who register to participate in the Center for States Child Welfare Virtual Conference.
Child Welfare Virtual Conference Exit Survey: This survey will be administered to all participants of the Child Welfare Virtual Conference at the conclusion the conference. The expected response rate from all attendees is 30 percent given attrition in expected conference attendance over the course of the day.
Tailored Services Practice Model Survey: This survey will be appended to the Cross-Center Evaluation Capacity Building Survey for up to 26 projects (1 per state) per year.
Assessment Observation– Group Debrief: A group debrief will be held at the conclusion of an onsite meeting as part of the assessment process for up to 25 states per year with 2 people expected to participate.
Service Delivery and Tracking and Adjustment Focus Observation – Group Debrief: A group debrief will be held at the conclusion of an onsite meeting as part of the service delivery process for up to 20 projects (two per state) per year. The plan is to debrief 20 projects twice with 2 participants per project.
Assessment and Service Delivery State Lead Interviews – Supplemental Questions: An individual phone interview will be conducted with a key stakeholder per state for up to 30 states per year.
Annual Assessment Update: Assessment participants will include a key child welfare agency staff representative from up to 54 States/territories who can respond to the domains addressed in the assessment protocol through verbal responses and pulling information from existing state documentation.
No statistical methodology for stratification and sample selection will be used for any of the data collection instruments.
Maximizing response rates is critical to the administration of the aforementioned surveys.1 The content and format of the instruments were developed in close consultation with key stakeholders were informed by previously developed measures.
Strategies that emphasize flexibility, confidentiality, and a respect for the respondent’s time facilitate timely participation. The following strategies will be implemented to maximize participation in the data collection and achieve the desired response rates: 2
Introduction and notification: Strategies to introduce and notify respondents about data collection are used for several instruments. An introductory email will be sent to inform all self-identified respondents about the administration of the Child Welfare Virtual Conference Interviews and Child Welfare Virtual Conference Interviews Focus Groups. Follow-up introductory emails will be sent to prospective respondents to introduce the evaluation team, share logistical information, and to address any questions about the data collection. For the Child Welfare Virtual Conference Session Surveys and Child Welfare Virtual Conference Exit Survey participants are told at the beginning of the conference that their feedback is critical to helping us improve programs and are provided a survey at the end of each session.
For Tailored Services Practice Model Survey: Center for States or Cross-center evaluators will inform potential participants about the survey prior to its release. All participants will receive an email invitation asking them to please participate in the survey. They will be provided an email link to take the survey electronically. A follow-up email reminder will be sent out after one week to increase response rates.
For the Assessment and Service Delivery State Lead Interviews – Supplemental Questions: Center for States will notify State participants in person or by phone of the option to participate in these interviews to help improve service delivery and understanding of capacity building approach effectiveness. Potential participants will opt-in if they are interested. Follow-up introductory emails will be sent to prospective respondents to introduce the evaluation team, share logistical information, and to address any questions about the data collection. These questions will be added on to the existing Tailored Services Interview protocol currently going through OMB approval for up to 12 purposively selected service delivery projects or conducted in isolation with up to 10 states related to assessment services and an additional 8 states related to service delivery. This will equal the total of 30 administrations listed in the burden estimates.
For the Assessment Observation– Group Debrief and Service Delivery and Tracking and Adjustment Focus Observation – Group Debrief: These questions will be administered following observation at an onsite meeting. Participants will be notified ahead of time regarding the observation of Center staff at the onsite meeting and will opt in to participate. The debrief will be included in the meeting schedule and agreed upon by all participants.
For the Annual Assessment Update (8 systematic questions) – these questions will be asked as part of the annual assessment process with each State. Center for States staff will notify State staff ahead of time regarding the purpose and content of the assessment process and these questions will be included in that notification. States will voluntarily decide to participate in the assessment process and work with Center for States staff on scheduling.
Timing of data collection: The Child Welfare Virtual Conference Session Surveys and Child Welfare Virtual Conference Exit Survey will be administered throughout the virtual conference period following the administration guidelines described in section B-1. In order to maximize recall the Child Welfare Virtual Conference Interviews and Child Welfare Virtual Conference Focus Groups will be administered following the event. The Tailored Services Practice Model Survey will be administered as individual service projects end/close out throughout the year. The Assessment Observation – Group Debrief and Service Delivery and Tracking and Adjustment Observation – Group Debrief will occur at the conclusion of the identified meetings. The Assessment and Service Delivery State Lead Interviews – Supplemental Questions will occur when a state is engaged in the assessment process or when a service project ends.
Pre-interview preparation: A copy of the debrief, interview or focus group guide will be sent to respondents in advance of any interview or focus group.
Administration: For the Child Welfare Virtual Conference Session Surveys, at the conclusion of each session participants will receive a pop-up form of the survey with a request to complete the survey. For the Child Welfare Virtual Conference Exit Survey, at the conclusion of the conference, participants will receive a pop-up form of the survey with a request to complete the survey. They will also receive a link to the survey in a thank you e-mail after the conference. A follow up email reminder will be sent out after one week to increase response rates. For the Tailored Services Practice Model Survey, all participants will receive an email asking them to please participate in the survey. They will be provided an email link to take the survey electronically. A follow-up email reminder will be sent out after one week to increase response rates. Electronic participation will allow respondents the flexibility to complete the survey at the most convenient time with minimal burden. For approximately four weeks after sending this initial email, weekly reminder emails will be sent to those respondents who have not yet completed the survey. For all interviews, focus groups, and debriefs – administration will be either in person if possible or, if not, by telephone/adobe connect virtual meeting. Participants will have the opportunity to volunteer and will not be required to participate. For the Annual Assessment Update, questions will be asked verbally during the course of the onsite meeting focused on state assessment.
Alternate response methods: Respondents will be given the option to use an alternate method for all interviews and focus groups. If a respondent prefers to submit written responses to a survey in lieu of participating in a telephone interview, we will provide him/her with an electronic version to complete via web, fax, email, or mail. Similarly, paper versions of any of the electronic surveys will be sent to respondents upon request or will be administered through a telephone interview if requested to accommodate any special needs.
Assurances of data confidentiality: Respondents to all surveys will be assured that reported data are aggregated and not attributable to individuals or organizational entities.
There are no incentives provided for participation in any of the surveys.
The Center evaluation instruments contained herein were subject to review and feedback by key stakeholders, including the CB.
User access and responsiveness to the web-based methodology for completing the Child Welfare Virtual Conference Registration, Child Welfare Virtual Conference Session Surveys, Child Welfare Virtual Conference Exit Survey and Tailored Services Practice Model Survey was pilot-tested. No modifications will be needed for implementation of the full data collection.
National Cross-Center Evaluation Contractor |
|
James Bell Associates 1001 19th Street North, Suite 1500 Arlington, VA (703) 528-3230 |
ICF International 9300 Lee Highway Fairfax, VA 22031 |
National Capacity Building Center for States |
|
ICF International 9300 Lee Highway Fairfax, VA 22031 |
|
2 Strategies that pertain to two or more data collections are discussed together.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | SUPPORTING STATEMENT SECTION B |
| Author | hafford |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |