SSA Prog Supp GenIC - ACF Evidence Capacity Support

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

ACF Evidence Capacity Support

Formative Data Collections for Program Support

0970 – 0531

Supporting Statement

Part A

January 2024

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Nicole Deterding

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a generic information collection under the generic title of Formative Data Collections for Program Support (0970-0531).

• Description of Request: The Office of Planning, Research, and Evaluation in the Administration for Children and Families proposes to conduct up to 35 one-hour interviews with Office of Refugee Resettlement (ORR) grant recipient programs and up to 12 group interviews with refugees who receive services or funds from ORR programs (up to 40 individuals across the groups). The purpose of the interviews is to gather input about (1) a theory of change for a national refugee program, (2) current performance reporting practices, and (3) potential revisions to performance reporting. OPRE will oversee the data collection, and Mathematica will collect the data. OPRE will share findings with ORR to inform potential changes to performance reporting.

We do not intend for this information to be used as the principal basis for public policy decisions.

• Time Sensitivity: This request is time sensitive. Funding for this work expires on September 30, 2024. As such, we hope to have OMB approval before February 2024 to allow time for interviews, analysis, and reporting before the end of September 2024.

A1. Necessity for Collection

The Office of Refugee Resettlement (ORR) is working to improve performance reporting across programs and grant recipients. As part of the Administration for Children and Families (ACF) Evidence Capacity Support project, the Office of Planning, Research, and Evaluation (OPRE) and its contractors at Mathematica seek to support ORR’s efforts to improve performance reporting through:

• Gathering input from ORR grant recipients on current performance reporting practices, ORR’s theory of change, and potential revisions to reporting.

• Finding ways to address data challenges including uniformity in performance measures across programs, duplicated administrative data across programs, a lack of timely administrative data, and a lack of client-level data.

• Developing a set of performance measures that focus on outcomes related to well-being for refugees served by the grantees. The measures support performance management, performance improvement, and meaningful accountability.

This proposed information collection is necessary to inform these efforts. The information we will collect is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision maker, and is not expected to meet the threshold of influential or highly influential scientific information.

A2. Purpose

Purpose and Use

The purpose of this proposed information collection is to obtain input from ORR grant recipients and individuals from refugee populations to inform ORR’s theory of change and revisions to program performance reporting, including measures and processes. ACF will use this information to improve performance reporting requirements for grant recipients. Specifically, ORR aims to use the feedback to inform the following efforts:

• Updates to ORR’s theory of change

• Improvements to current performance reporting practices, including finding ways to address data challenges such as uniformity in performance measures across programs, duplicated administrative data across programs, a lack of timely administrative data, and a lack of client-level data

• Development of a set of performance measures that focus on outcomes related to well-being for refugees served by the grantees

• Ensuring that measures support performance management, performance improvement, and meaningful accountability

This proposed information collection meets the following goals of ACF’s generic clearance for formative data collections for program support (0970-0531):

• Obtaining input on the development of program performance measures (PM) from grant recipients or experts in a relevant field

Guiding Questions

• Current performance measurement

• How complete and high in quality are current performance measure data? Are the current measures reliable?

• How great is the burden on grant recipients to supply data? Does the burden extend to individuals who receive grant-funded services?

• Who uses the data, for what purpose, and with what level of satisfaction? Do the data inform meaningful decisions about program funding, design, operations, or effectiveness?

• Potential changes to performance measurement

• How well do ORR grant recipients think the theory of change captures the goals of refugee resettlement programs? Are there key concepts that should be added or subtracted?

• How does the theory of change align with (1) the data that resettlement organizations currently report to the federal government and (2) information organizations currently collect and do not currently report?

• What would have to be done to collect uniform information that aligns with the theory of change, streamlines federal reporting, and reduces administrative burden on grant recipients and program participants?

• What kinds of information may be better collected through other means, such as program monitoring site visits, topical case studies, and formal program evaluation?

Study Design and Universe of Data Collection Efforts

Data collection activity	Instruments	Respondent, content, purpose of collection	Mode and duration
Interviews with grant recipients	Instrument 1: Interview discussion guide	Respondents: Up to 35 grant recipients, including people involved in program administration and delivery from state grant recipients, national resettlement agencies, and local affiliates. Content: Questions related to grant recipients’ current performance reporting; feedback on a theory of change for refugee resettlement; alignment of current reporting with that theory of change; and suggested revisions to performance reporting. Purpose: To gather information that ORR can use to collect meaningful and better-quality data.	Mode: Videoconference call Duration: 1 hour/ respondent
Interviews with refugees	Instrument 2: Participant discussion guide	Respondents: Representatives of each of six refugee populations. Populations and representatives will be identified in consultation with ORR. This equates to 12 group interviews with up to 40 individuals. Content: Questions related to components of the theory of change on refugee resettlement, including challenges and outcomes identified in the theory of change. Purpose: To gather information that ORR can use to collect meaningful and better quality data.	Mode: Videoconference call Duration: 1 hour/ group

Other Data Sources and Uses of Information

Interviews will be combined with other sources of information. Before conducting interviews, we will review: (1) written materials related to current or planned ORR performance reporting and (2) portals and online systems that capture ORR grant recipients’ performance data.

A3. Use of Information Technology to Reduce Burden

Participating in this information collection will not require any in-person activities or printing of any materials. The study team will use information technology as appropriate to reduce the burden on respondents. Mathematica will conduct the interviews via a videoconferencing platform that can be accessed on a computer, phone, or tablet (e.g., Webex). When we schedule the interview, we will email respondents an overview of the interview topics.

Interviews and focus groups will be transcribed. With participant consent, information in the transcripts may be revisited for secondary analysis at a future date, with a goal of reducing burden on the group of respondents.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

The proposed interviews do not duplicate other data collection efforts. The information the contractor will collect through the interviews is not available through other existing data sources.

A5. Impact on Small Businesses

Resettlement agencies and local affiliates involved in information collection may be considered small businesses. The study team will make every effort to reduce burden on all participating organizations. The study team will ensure interviews are scheduled at a time convenient for the respondent.

A6. Consequences of Less Frequent Collection

This is a one-time data collection.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published two notices in the Federal Register announcing the agency’s intention to request an OMB review of this information collection request to extend approval of the umbrella generic with minor changes. The notice was published on January 28, 2022, (87 FR 4603), and provided a sixty-day period for public comment. ACF did not receive any comments on the first notice. A second notice was published, allowing a thirty-day period for public comment, in conjunction with submission of the request to OMB. ACF did not receive any comments on the second notice.

Consultation with Experts

No consultations have taken place with experts outside of the project team. 

A9. Tokens of Appreciation

No tokens of appreciation will be provided for respondents to this information collection.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

The only personally identifiable information we will collect is information needed to schedule interviews, which includes names and email addresses.

The study team will not maintain information in a paper or electronic system that retrieves data by using an individual’s personally identifiable information in the way that could trigger the Privacy Act of 1974, as amended (5 U.S.C. 552a).

Assurances of Privacy

Any information collected will be kept private. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private. We may share collected transcripts from interviews/discussions with ACF staff or other researchers to support them in reports about refugee experiences or performance measurement. However, in all circumstances, we will de-identify or remove all PII such as names and email addresses before any usage or sharing of any data including interview notes, records, and transcripts. As specified in the contract, the contractor will comply with all federal and departmental regulations for private information.

Data Security and Monitoring

People who will have access to the artifacts of this data collection, including interview notes, recordings, and transcripts, have received training about data privacy issues. Mathematica will store data on its secure drive, which only authorized users have access to. The secure drive uses Windows Active Directory security groups for access control, and uses Encrypting File System (EFS) on demand. Mathematica will use a secure file transfer protocol to transmit data to ACF.

A11. Sensitive Information ¹

The proposed information collection does not request any sensitive information.

A12. Burden

Explanation of Burden Estimates

For the interviews with up to 35 grant recipients, the number of respondents was chosen to allow for geographic and variation within different categories of parties involved in ORR program administration and delivery. This includes state-level grant recipients (state government agencies and replacement designees), national resettlement agencies, and local affiliates. To ensure a diversity of perspectives, respondents will vary in terms of geography (participants from the four major US Census regions (Northeast, South, Central/Midwest, and West), service area (urban/rural), and the types of ORR-funded programs and services delivered. Respondents will be asked to participate in one interview that is estimated to take approximately 60 minutes to complete.

For the small group interviews with up to 40 refugees, the number of respondents was chosen to allow for variation within the refugee population. Representatives of each of six refugee populations will be selected to ensure a diversity of perspectives. Respondents will be asked to participate in one small group interview that is estimated to take approximately 60 minutes to complete.

Estimated Annualized Cost to Respondents

The estimated annual cost for respondents is shown in Exhibit 2. The source for the mean hourly wage information for respondents is Bureau of Labor Statistics, Occupational Employment and Wages, May 2022. Burden estimates were calculated assuming one program director from each program would contribute to their program’s interview response. Hourly wage for program directors is estimated at $38.13 based on the BLS average earnings for Social and Community Service Managers. Burden estimates for refugees is estimated at $29.76 and reflects the average wage for all occupations.

Instrument	No. of respondents (total over request period)	No. of responses per respondent (total over request period)	Avg. burden per response (in hours)	Total/ annual burden (in hours)	Average hourly wage rate	Total annual respondent cost
Instrument 1: Discussion guide for the grant recipient interview	35	1	1	35	$38.13	$1,334.55
Instrument 2: Discussion guide for the refugee interview	40	1	1	40	$29.76	$1,190.40
Totals:	75	1	1	75		$2,524.95

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Cost to the Federal Government

Activity	Detail	Estimated Cost
Interview administration	Full-time equivalent time Operational expenses (such as equipment, overhead, printing, and staff support) Other expenses that would not have been incurred without this collection of information	$70,534
Analysis and initial dissemination	Full-time equivalent time Operational expenses (such as equipment, overhead, printing, and staff support) Other expenses that would not have been incurred without this collection of information	$61,680
Total costs over the request period		$132,214

A15. Reasons for changes in burden

This is for an individual information collection under the umbrella formative generic clearance for program support (0970-0531).

A16. Timeline

	Jan 2024	Feb 2024	March 2024	April 2024	May 2024	June 2024	July 2024	Aug 2024	Sep 2024
Protocol development
OMB submission and clearance
Conduct discussions and analysis
Draft and finalize results memo

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

• Instrument 1: Program administrators: interview discussion guide

• Instrument 2: Program participants: interview discussion guide

1 Examples of sensitive topics include (but are not limited to): Social Security number; sexual behavior and attitudes; illegal, antisocial, self-incriminating, and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, such as family, pupil-teacher, or employee-supervisor; mental and psychological problems potentially sensitive for respondents; religion and indicators of religion; community activities that indicate political affiliation and attitudes; legally recognized privileged and analogous relationships such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (for example, unemployment or WIC or SNAP); and immigration/citizenship status.

10