CenterforStatesEvaluationAncillaryDataCollection_SSA_Final

Data Collection Plan for the Center for States Evaluation Ancillary Data Collection

OMB Information Collection Request

0970 – 0501

Supporting Statement Part A - Justification

August 2020

Submitted By:

Children’s Bureau

Administration for Children and Families

U.S. Department of Health and Human Services

SUPPORTING STATEMENT A – JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

This is a request for an extension of an existing data collection with minor changes to the previous estimates of annualized burden hours and costs to reflect 1) updated respondent and staff labor rates for 2019 and 2) lower than expected response rates for most instruments during previous administration of instruments. For most instruments, response rates and the number of respondents are lower in this submission and for others, they remained the same. No instruments in this submission increased total number of respondents estimated.

The Child Welfare Capacity Building Collaborative, Center for States is sponsored by the Children’s Bureau (CB) in the Administration on Children, Youth and Families (ACYF), Administration for Youth and Families (ACF), U.S. Department of Health and Human Services (DHHS) and seeks approval for the data collection instruments and procedures described herein. The proposed information collection is necessary to facilitate, track, monitor, and evaluate the activities of the Center for States, part of the Capacity Building Collaborative (“The Collaborative.”) The Collaborative includes three federally funded centers (Center for States, Center for Tribes, and Center for Courts) that deliver national child welfare expertise and evidence-informed training and technical assistance services to state, tribal, and territorial public child welfare agencies and Court Improvement Programs (CIPs).

The Collaborative’s collective goal is to build the capacities of state, local, and tribal child welfare systems to successfully undertake practice, organizational, and systemic reforms necessary to implement federal policies, meet federal standards, and achieve better outcomes for the children, youth, and families they serve. This collection consists of instruments only used for the Center for States evaluation.

Legislative Background and Purpose

Agencies that receive formula funding through the Child Abuse Prevention and Treatment Act (CAPTA), and titles IV-B and IV-E of the Social Security Act (the Act) are eligible for technical assistance from CB to support implementation of these programs, compliance with federal requirements, and improvement of outcomes for children, youth, and families.

This information collection is necessary to perform routine evaluation of quality and effectiveness of the Center for State’s services and to inform future planning and decision making about the provision and improvement of technical assistance services authorized under multiple sections of CAPTA and title IV of the Act. This information collection also complies with the statutory requirement for projects authorized by Section 5106 of CAPTA to be evaluated for their effectiveness.

2. Purpose and Use of the Information Collection

The purpose of this evaluation is to respond to a set of cross-cutting evaluation questions posed by CB. This information collection is part of a larger data collection effort being conducted for the evaluation of the Child Welfare Capacity Building Collaborative. This notice details a group of instruments that are specific only to the Center for States. The instruments focus on (1) evaluating an innovative approach to engaging professionals in networking and professional development through virtual conferences, (2) understanding fidelity to and effectiveness of the Center for States' Capacity Building Model, and (3) capturing consistent information for the Center for States during the updated annual assessment process focused on contextual issues impacting potential service delivery, such as implementation of new legislation.

Each of the instruments described below will be administered annually.

The Child Welfare Virtual Conference Session Surveys and Child Welfare Virtual Conference Exit Survey will be administered to all participants of the Child Welfare Virtual Conference at the conclusion of each session and the conference. The Child Welfare Virtual Conference Interviews and Child Welfare Virtual Conference Focus Groups will be conducted with a sample of up to 20 (for interviews) and 30 (for focus groups) virtual conference participants after the conference has concluded. The interviews will focus on individual user navigation/experience. The Child Welfare Virtual Conference Registration Form will be completed by all conference registrants.

The Tailored Services Practice Model Survey will be appended to the existing Tailored Services intensive or brief project satisfaction surveys (OMB #0970-XXXX) for up to 40 service delivery projects per year. It is anticipated that up to five respondents per project will participate.

The Assessment Observation– Group Debrief will be held at the conclusion of an onsite meeting as part of the assessment process for up to 57 states per year.

A Service Delivery and Tracking and Adjustment Observation – Group Debrief will be held at the conclusion of an onsite meeting as part of the service delivery process up to two times for 40 projects per year.

The Assessment and Service Delivery State Lead Interviews - Supplemental Questions will be conducted with a key stakeholder per state for up to 50 states per year

The Annual Assessment Update will include 8 systematic questions verbally asked of a representative from child welfare agency staff for up to 57 States/territories during the course of the Center for States Annual Assessment process.

3. Use of Improved Information Technology and Burden Reduction

Wherever possible and appropriate, information technology will be used to capture information and reduce burden relative to alternative methods of data collection. Administration of any evaluation surveys will be web-based, utilizing email notification and Internet-based survey technologies creating efficiencies for survey administrators, allowing flexibility and convenience for participants, and ideally resulting in a user-friendly experience for respondents. Based on the services provided, survey respondents will receive an email notification inviting them to complete the appropriate survey instrument by accessing a web-link to an online version of the survey.

Nearly all of the targeted respondents are expected to be able to access the web-link to the surveys.¹ The majority of questions in most surveys are closed-ended response items that can be completed quickly (within 5 – 10 minutes), allowing descriptive and comparative analyses.

Child Welfare Virtual Conference Session and Exit surveys are fully web accessible and can be completed in 10 minutes or less. To minimize the burden on Virtual Conference participants, data collection for the Virtual Conference was embedded into the flow of the conference using the Conference technology platform. Participants will have an opportunity after each session to complete surveys that are programmed into the virtual technology, so participants do not have to navigate away from the virtual conference page. Reported metrics can be accessed securely through the platform’s data portal.

4. Efforts to Identify Duplication and Use of Similar Information

The proposed instruments are intended to uniformly collect data that will allow for the evaluation of Center for States-specific processes and outcomes. CB has required its cross-center evaluator and center-specific evaluators to ensure data collection is necessary and complementary. The proposed information collection and evaluation activities have been coordinated to avoid potential duplication of data collection and reduce burden to respondents. To reduce burden on respondents, the Tailored Services Practice Model Survey will be administered with the Tailored Services intensive or brief project satisfaction surveys (OMB #0970-0484). Additionally, the Assessment and Service Delivery State Lead Interviews – Supplemental Questions will be included in Tailored Services Interviews (OMB #0970-0494) as needed, without increasing the burden. The Assessment and Service Delivery State Lead Interviews – Supplemental Questions will also be administered as standalone questions to reach the expected sampling burden included in this approval request, as needed. All questions have been cross-referenced to ensure no duplication. By administering the proposed questions alongside the related data collection (OMB #0970-0494), no respondent receives more than one survey or interview. Additionally, all instruments in this information collection have been reviewed for duplication across all of the instruments for The Collaborative. These instruments have been revised to address potential overlap and the timing of data collection activities is being closely coordinated to minimize burden.

5. Impact on Small Businesses and Other Entities

A number of efforts are in place to minimize respondent burden, regardless of organizational size, for each of the data collection strategies described herein. Skip patterns have been included in the survey instruments based on the types of services received, and the timing of data collection activities is being coordinated to minimize respondent burden. In addition, all instruments were subject to internal review processes to ensure that surveys only included questions and items required to answer research questions.

6. Consequences of Collecting the Information Less Frequently

In order to improve the Center for States’ services and collaborate effectively to provide coordinated support to state and territorial public child welfare agencies, CB and its providers need timely data on the provision of services delivered by the Center for States, the accessibility of services, the perceived effect and quality of the services received, and the interactions of service providers with one another. Less frequent data collection would inhibit the timely use of the information by CB and providers to improve service coordination and service quality and to potentially make decisions about resource allocation.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances associated with this data collection.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and the Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on May 20, 2020, 85 FR 30710, and provided a sixty-day period for public comment. During the notice and comment period, no comments were received.

9. Explanation of Any Payment or Gift to Respondents

This data collection plan does not call for payment to respondents for participation.

10. Assurance of Confidentiality Provided to the Respondents

No assurances of complete confidentiality are provided to respondents. However, all respondents are informed of the importance that CB places on maintaining respondent privacy and that reported data are not attributed to individuals but rather are aggregated and shared with the Center for States service planning teams and CB.

11. Justification for Sensitive Questions

Participants at virtual conferences are informed that a name and email address are required to facilitate access to the conference and/or participation in follow-up focus groups, and that any personally identifiable information (PII) included on the Child Welfare Virtual Conference Registration Form will not be shared externally. All registrants are informed that general demographic data captured on the registration form will be used in aggregate by the Collaborative, Center for States, and CB; therefore there are no opportunities to consent to the sharing of general demographic information. To ensure that the PII being collected meets required ACF security standards, all PII contact data will be stored in a password-protected secure data system managed by ICF, under CB’s Center for States contract. Access to PII controlled by and restricted to system administrators. Contact information for focus group participants will be used to secure focus group participation and then destroyed. All other PII will be removed from the database following the virtual conference and destroyed and unique identifiers assigned.

There are no questions or requirements of a sensitive nature contained in the survey instruments and data collection forms described herein.

12. Estimates of Annualized Burden Hours and Costs

Estimates of response burden are outlined in the following table. We have revised the burden estimates from initial approval in order to account for updated 2019 labor rates for respondents and lower than expected response rates for most instruments during previous administration of instruments. For most instruments, response rates and the number of respondents are lower in this submission and for others, they remained the same. No instruments in this submission increased total number of respondents estimated.

Information Collection Title	Total Number of Respondents	Total Number of Responses per Respondent	Average Burden Hours per Response	Total Burden Hours	Annual Burden Hours	Average Hourly Wage	Annual Cost to Respondents
Child Welfare Virtual Conference Session Surveys	450	6	.08	648	216	$41.28	$8,916.48
Child Welfare Virtual Conference Focus Group Guide	30	1	1	90	30	$41.28	$1238.40
Child Welfare Virtual Conference Interview Guide	20	1	.5	30	10	$41.28	$412.80
Child Welfare Virtual Conference Registration Form	1000	1	.03	90	30	$41.28	$1238.40
Child Welfare Virtual Conference Exit Survey	225	1	.16	108	36	$41.28	$1,486.08
Tailored Services Practice Model Survey	130	1	.12	46.80	15.60	$41.28	$643.97
Assessment Observation– Group Debrief	50	1	.25	37.50	12.50	$41.28	$516.00
Service Delivery and Tracking and Adjustment Observation – Group Debrief	80	1	.25	60	20	$41.28	$825.60
Assessment and Service Delivery State Lead Interviews – Supplemental Questions	30	1	.5	45	15	$41.28	$619.20
Annual Assessment Update (8 systematic questions)	54	1	.08	12.96	4.32	$41.28	$178.33
Total Annual Burden Hours:					389.42	Total Annual Cost to Respondents:	$16,075.26

The cost to respondents was calculated using the Bureau of Labor Statistics (BLS) job code for Individual and Family Services [21-0000] and wage data from May 2019 (https://www.bls.gov/oes/current/oes210000.htm), which is $20.64 per hour. To account for fringe benefits and overhead the rate was multiplied by 2.0 which is $41.28. The total annual burden and cost to the respondents if all data collection instruments were employed in the same given year is 389.42 hours and $16,075.26, respectively.

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There are no additional costs to respondents.

14. Annualized Cost to the Federal Government

The cost to the Federal government was calculated using the Bureau of Labor Statistics (BLS) job code for Social Scientists and Related Workers, All Other [19-3099] wage data from May 2019 (https://www.bls.gov/oes/current/oes193099.htm), which is $42.16 per hour. This is to account for the labor of government contractors. To account for fringe benefits and overhead the rate was multiplied by 2.0, which is $84.32 per hour. We estimated 551.30 total staff hours for all instrument development, administration, and analysis. The estimate of annualized costs to the Federal government is $84.32 multiplied by 551.30 hours which is $46,485.62. The estimated annual Federal costs associated for each instrument are outlined in the table below.

Instrument	Administration Activities	Staff Time (Hours)	Total Cost
Child Welfare Virtual Conference Session Surveys	Distribute Electronic Survey	20.25	$1,707.48
Child Welfare Virtual Conference Focus Group Guide	Conduct Focus Groups	12	$1,011.84
Child Welfare Virtual Conference Interview Guide	Conduct Interviews	70	$5,902.40
Child Welfare Virtual Conference Registration Form	Online registration	3.33	$280.79
Child Welfare Virtual Conference Exit Survey	Distribute Electronic Survey	4.5	$379.44
Tailored Services Practice Model Survey	Distribute Electronic Survey & Follow up	104	$8,769.28
Assessment Observation – Group Debrief	Conduct group debrief	62.5	$5,270.00
Service Delivery and Tracking and Adjustment Observation – Group Debrief	Conduct group debrief	100	$8,432.00
Assessment and Service Delivery State Lead Interviews – Supplemental Questions	Conduct Interviews	165	$13,912.80
Annual Assessment Update	Distribute Electronic Survey & Follow up	9.72	$819.59
Total		551.30	$46,485.62

15. Explanation of Program Changes or Adjustments

This is a request for an extension of an existing data collection, with no changes or adjustments to the instruments. Burden estimates have changed due to updated 2019 labor rates

for respondents and lower than expected response rates for most instruments during previous administration of instruments. For most instruments, response rates and the number of respondents are lower in this submission and for others, they remained the same. No instruments in this submission increased total number of respondents estimated.

16. Plans for Tabulation and Publication and Project Time Schedule

Tabulation: Frequency distributions will be calculated to generate summaries of survey items, as well as to examine variability in the data. Parameter estimates, such as variances and means, will be established for each quantitative item. Cross-tabulations and significance tests will be conducted as appropriate. Content analysis will be conducted on open-ended survey items and interview transcripts, and will entail systematic coding, creation of a hierarchy of codes, and cross-case and cross-source thematic analysis. Analyses will be conducted to determine subgroup variation.

Publication: Findings specific to Center for States service delivery will be reported on an ongoing basis to Center for State staff and CB to inform planning and service delivery. Reports of research findings will include descriptive analyses and the implications of the findings. The Center for States will explore other opportunities for publication, such as conference presentations.

Project Timetable: The Child Welfare Virtual Conference Registration form will be used for participants to register for the conference. The Child Welfare Virtual Conference Session Surveys will be administered over the course of the virtual conference as sessions conclude. The Child Welfare Virtual Conference Exit Survey will be administered at the end of the conference. The Child Welfare Virtual Conference Focus Group and Child Welfare Virtual Conference Interviews will be administered following the Child Welfare Virtual Conference to a sample of voluntary participants.

The Tailored Services Practice Model Survey will be administered as service projects end/close out throughout the year. The Assessment Observation – Group Debrief and the Service Delivery and Tracking and Adjustment Observation – Group Debrief will occur at the conclusion of the identified service delivery meetings. The Assessment and Service Delivery State Lead Interviews – Supplemental Questions will occur after a state has engaged in the assessment process or when a service project ends.

The Annual Assessment Update Survey will be administered once per year as part of each State’s Assessment Update.

17. Reasons Display of OMB Expiration Date is Inappropriate

The OMB expiration date for the information collection will appear on the instruments.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are requested.

1 A hard copy of the surveys will be provided to those who cannot access the surveys online.

ii