Supporting Statement A - ASR Redesign_202006_CLEAN2

Annual Survey of Refugees

OMB Information Collection Request

0970 - 0033

Supporting Statement Part A –

Justification

June 2020

Submitted By:

Office of Planning, Research, and Evaluation

Office of Refugee Resettlement

Administration for Children and Families

U.S. Department of Health and Human Services

Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Executive Summary

• Type of Request: This Information Collection Request is for a revision to the Annual Survey of Refugees (ASR), last approved April 8, 2019. We are requesting 3 years of approval.

• Progress to Date: The ASR is an annual cross-sectional survey that completed its 2018 data collection (1514 respondents) on schedule in March 2019, with a 21% overall response rate. Conditional on successful telephone contact, 76% of sampled individuals completed a survey during ASR 2018. The currently-approved survey instrument was used for the 2019 annual administration, fielded from January to May 2020.

• Summary of changes requested: The proposed revisions are intended to collect information about factors that social science research suggests affect refugee resettlement outcomes. These items will help the Office of Refugee Resettlement (ORR) better understand the experiences of the population it serves, increasing the practical utility of the data collection for the Administration for Children and Families (ACF) and the field.

Based upon paradata from several rounds of survey administration, we are proposing to add a $2 token of appreciation to increase the salience of the introductory letter and encourage participants to complete voluntary updates to contact information. We expect this to produce a 20% increase in the overall response rate (i.e., a four-point increase from roughly 20 to about 24 percent)¹. We do not propose any other changes to the survey administration protocol.

ORR does not intend for information from this collection to be used as the principal basis for public policy decisions, and information is not expected to rise to the level of “highly-influential information.”

• Time Sensitivity: The ASR is fielded annually in January to provide information for ORR’s Annual Report to Congress. The proposed changes to the instrument would be used in the ASR 2020, administered beginning in January 2021.
  
  ACF is supporting a one-year lead up period, beginning September 2019, to allow for advance infrastructure to support the launch of the new revised instrument. Lead year activities include development of interviewer training materials; translation of approved instruments into 17 languages and CATI programming; and other key planning and outreach tasks. Many of these tasks are on hold until approval of this information collection request.

A1. Circumstances Making the Collection of Information Necessary

The Annual Survey of Refugees (ASR) helps the ACF ORR meet its congressional reporting requirements, as specified in The Refugee Act of 1980 (Section 413 of the Immigration and Naturalization Act, [8 U.S.C. 1523]).

In particular, descriptive information from the survey is used to meet the requirement of 413(b)(1): “an updated profile of the employment and labor force statistics for refugees who have entered the United States within the five-fiscal-year period immediately preceding the fiscal year within which the report is to be made.” Descriptive information from the survey is also used alongside quantitative and qualitative data collected during program administration to meet the requirements of 413(b)(5): “the extent to which (A) the services provided under this chapter are assisting refugees in achieving economic self-sufficiency, achieving ability in English, and achieving employment commensurate with their skills and abilities.”

This information collection request (ICR) proposes additions to the survey instrument intended to maximize the utility of the data collection, including for secondary analysis. There are no legal or administrative requirements necessitating the collection of these additional items. ACF is undertaking these revisions at the discretion of the agency. Proposed new items for ASR 2020 are also flagged in Appendix A.

A2. Purpose and Use of the Information Collection

Purpose and Use

From the time of their arrival in the United States, refugees are eligible for social services, including means-tested public assistance, according to eligibility criteria established by the locality in which they are resettled. Following an initial 90-day reception period funded by the U.S. Department of State, ACF, through ORR, provides transitional resettlement assistance through a network of national non-profit grantees and their local affiliates. In many cases, these services are expected to end after an 8-month resettlement period. By that time, refugees are expected to be transitioned to mainstream social services or to self-sufficiency. The design of the resettlement program means that ORR has limited administrative information on refugees’ experiences after the 8-month resettlement period.

The information collected through the ASR generates national population estimates of refugee labor force participation, public benefits receipt, English language acquisition, and conversion to legal permanent residency to meet Congressional reporting requirements. Given that ORR does not have access to cross-program administrative data on refugees’ participation in state-provided services, a representative survey is the most resource-efficient way to meet ORR’s annual reporting requirements, laid out in the Refugee Act of 1980.

As the only scientifically-collected source of national information on the highly mobile and linguistically-diverse U.S. refugee population, the ASR has the potential to be a valuable source of information about the experiences of refugees during their first five years in the United States. To increase the utility of this routine data collection and improve the value of the ASR for the body of knowledge on ACF programs, the redesigned instrument described in this ICR is intended to collect richer information about a variety of factors that social science research indicates are related to refugee outcomes in self-sufficiency and well-being.

Information from the ASR is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.

Research Questions or Tests

The ASR instrument is designed to support national estimates of household demographics; human capital of family members 16 years and older; and economic self-sufficiency, including public benefits receipt. The study will also support multivariate analyses of the relationship between contextual factors and these outcomes. Table A1 summarizes what is measured in the revised ASR instrument, in two categories: domains estimated for the ORR Annual Report to Congress and contextual factors that research suggests are associated with resettlement outcomes.

Table A1. Estimates and Contextual Factors Measured in the Annual Survey of Refugees

Estimates for ORR Annual Report to Congress (8 U.S.C. 1523)

Estimates of employment and labor force statistics for refugees who have entered the United States within the five-fiscal-year period immediately preceding the fiscal year within which the report is to be made

Estimates of refugee self-sufficiency, achievement of English language proficiency, and employment commensurate with their skills and abilities

Applications for Legal Permanent Residence Status by Year of Arrival

Contextual Factors Potentially Associated with Resettlement Outcomes

Demographic information about refugee households

Experiences before and after arrival to the United States

Wellbeing and Social Connections to Receiving Community

Health Care Services

Engagement with Children’s Schools

Study Design

The ASR’s cross-sectional design was chosen to prioritize efficiency for use in statistical estimates of the characteristics and experiences of refugees entering the United States in the previous five fiscal years. The survey is fielded annually over a 13-week period from January-April.

One limitation of the design for its purpose is that each entering refugee cohort has its own demographic, cultural, and social characteristics. This means that outcomes for the oldest cohort of refugees (those entering the U.S. four or five fiscal years prior) should not be understood as predictions of future outcomes for the most recently-arrived cohort. A second limitation of the data for its purpose is that it does not enable a direct assessment of the role or impact of ORR resettlement services in the observed outcomes. These limitations are clearly stated in the text and technical appendix of the Annual Report to Congress, and in the technical documentation for the public-use data file.

Table A2 presents information describing the data collection activities involved in this ICR. As noted above, the ASR has been redesigned. See Supporting Statement B, section B2 for additional information about the redesign. The introduction letter and postcard have been revised to reflect the time estimate for the ASR and the $2 token of appreciation to update participant’s contact information.

Table A2. Data Collection Activities for the Annual Survey of Refugees

Data Collection Activity	Instrument(s)	Respondent, Content, Purpose of Collection	Mode and Duration
Introduction Letter and Postcard (2020 Proposed)	1	Respondents: All sampled Principal Applicants Purpose: Notice of selection into study sample; option to update contact information	Mode: Introductory paper letter, optional update of contact information via pre-stamped postcard or phone Duration: 3 minutes
ORR-9 Annual Survey of Refugees (2020 Proposed)	2	Respondents: Refugee Principal Applicants, on behalf of the household and family members aged 16+. Content: Household roster, detailed demographic information for eligible individuals aged 16+, household-level measures Purpose: Revised survey instrument, beginning in ASR 2020	Mode: Computer Assisted Telephone Interviewing Duration: Varies by household roster size; average time to complete for a household is 48 minutes.

Other Data Sources and Uses of Information

To meet ORR’s congressional reporting requirements, descriptive information from the ASR is presented alongside other quantitative and qualitative information generated by ORR’s grantee performance reporting and compliance monitoring.

A3. Use of Improved Information Technology and Burden Reduction

The Annual Survey of Refugees is administered using Computer Assisted Telephone Interviewing (CATI), to minimize errors, ensure administrative efficiency, and reduce respondent burden.

A4. Efforts to Identify Duplication and Use of Similar Information

Information collected in this ICR is not otherwise available in a consistent manner from ORR grantees or the States. To reduce duplication of information, the ASR redesign process involved outreach to ORR’s nine National Resettlement grantees to discuss their existing individual-level data collections, research interests, and information needs (outreach did not involve requesting uniform information from more than 9 individuals, so was not subject to the Paperwork Reduction Act). To increase the utility of the ASR data collection, this input was taken into consideration in the design of the survey’s conceptual framework, along with other input described in Supplemental Statement B.

The multiyear ASR redesign process also included an assessment of the feasibility of compiling administrative data to meet ORR’s Congressional reporting requirements, particularly for estimates of public benefits receipt. The conclusion was that limits to legal authority and the logistical challenges of collecting information about multiple benefits programs from each of the fifty states would be prohibitive.

A5. Impact on Small Businesses or Other Small Entities

This data collection is a survey of households, and does not have an impact on small businesses or other small entities.

A6. Consequences of Collecting the Information Less Frequently

The survey is conducted annually to meet ORR’s statutory reporting requirements, as laid out in 8 U.S.C. 1523.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances.

A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on July 18, 2019, Volume 84 , Number 138, pages 34395-34396, and provided a sixty-day period for public comment. During the notice and comment period, no substantive comments were received.

Consultation with Experts Outside of the Study

The ASR redesign process (described in more detail in Supplemental Statement B, B3) involved stakeholder outreach and an expert roundtable designed to collect input about improvements to the survey that would increase the utility of this information collection for research and evidence building. A list of participants in the roundtable, including research experts, representatives of refugee resettlement agencies, and Federal Staff, is included as Appendix B- Roundtable Participant List.

The instrument revisions proposed in this ICR were field tested in fall 2017 and winter 2018, approved under OPRE’s Generic Clearance for Pre-Testing (OMB Control Number 0970-0355). This involved cognitive interviews with 47 refugees to enhance clarity of the instructions and minimize respondent burden. Further cognitive testing and revision of a limited number of items was conducted with 8 respondents (not subject to PRA) in early 2020. The revised questionnaire incorporates insights from this effort, including streamlining of information collected via the household roster to reduce respondent burden.

A9. Explanation of Any Payment or Gift to Respondents

Please see Supporting Statement B, Section B3 for an overview of the ASR study design features intended to maximize response rate and minimize non-response bias. As one part of an overall design to secure quality data, and to provide continuity with this survey program, the research team proposes that respondents are offered a $25 gift card upon completion of the survey. In previous survey administrations, the ASR has offered this $25 token of appreciation to refugees completing the 30-minute ASR telephone survey on behalf of their household.

New for 2020

Using information from ASR administration between 2016-2018, ACF has identified that the majority of survey non-response is due to our inability to successfully reach sampled respondents, including those for whom we have updated telephone numbers from batch look-up. As reported in Supplemental Statement B, Table B2, this issue has increased over time, from 32 percent of sampled refugees in 2016 to 40% of sampled refugees in 2018. With a goal of increasing the salience of the introductory letter and inducing more voluntary telephone number updates, we propose to offer a $2 token of appreciation, included in the introductory letter. This is expected to increase the overall response rate through higher participation of refugees who are located and contacted by telephone.

A10. Assurance of Confidentiality Provided to Respondents

Personally Identifiable Information

The Refugee Arrivals Data System (RADS), from which data are extracted for the purposes of developing a survey sample and producing statistical weights, is a Privacy Act System of Records (System of Records Number # 09-80-0325, published 07/18/2016). This information collection requires the transfer of personally identifiable information (PII) from RADS to the ASR contractor.

This information collection gathers voluntary updated contact information by sending an Introduction Letter and Postcard (Instrument 1). Contact information is also updated using the National Change of Address service and batch record matching using TransUnion. Contact information from RADS and these additional sources are used for the purposes of administering the survey. This PII is stored separately from the file of survey responses to protect respondent privacy.

The proposed revised ASR 2020 (Instrument 2) collects the first name and birth date of each individual living in the household. This information is used for the purpose of identifying household members that are eligible for the survey and administering the household roster. It is stored separately from other survey responses to protect respondent privacy.

PII (from RADS, the voluntary contact update, or the survey instrument) is not returned to ACF, either on the restricted-use data file or as a separate data file, as a part of the contract.

Information from the ASR is not maintained in a paper or electronic system from which survey data are actually or directly retrieved by an individuals’ personal identifier.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information.

Data Security and Monitoring

The contractor’s security plan shall comply with the NIST SP 800-18, Guide for Developing Security Plans for Federal Information Systems; FIPS 199, Standards for Security Categorization of Federal Information and Information Systems; the Federal Information Processing Standard (FIPS) 200, Recommended Security Controls for Federal Information Systems; the latest version of NIST SP 800-53, Security and Privacy Controls for Federal Information Systems and Organizations applicable baseline requirements; and other applicable NIST guidance as well as HHS and ACF policies and other guidance. The data security plan must specify the following:

• The personal or sensitive information that is collected, including content of interviews, notes for tracking purposes, personally identifiable information, and relevant administrative data that will be accessed.

• The assurances provided to respondents or participants about the privacy of their information.

• How data will be entered and stored, including confirmation data security that includes encryption procedures that meet the HHS standards.

• How potentially sensitive information is transmitted, the personal information included in transmissions, any encryption for electronic transmission, the protections in place for materials handled via mail services, etc. This should include discussions of data among staff within the facility as well as between facility staff and field staff.

• The personal information field staff maintain on cases that are in the field, and how field or relevant program staff are instructed to handle and store case folders on which they are working.

• The training procedures in place for all project staff at all levels of the project and specific instructions to all staff related to handling personal or sensitive information.

• The disposition of the data and information collected at the end of the contract.

As part of the detailed data security plan, the Contractor describes their processes for reporting suspected or confirmed breaches of data security protocol to ACF. All data security breaches, including suspected breaches and breaches of protocol MUST be reported to ACF no later than one hour after the Contractor’s staff is made aware of the breach or the possibility of a breach. The Contractor submits an annual report that outlines their compliance with the data security plan developed for the project, no later than one month after the year has ended.

Data Availability

Data are available in tabulations published in ORR’s Annual Report to Congress.

A restricted-use data file and technical documentation are returned to ACF to enable statistical analyses of the survey data. This file does not contain personal identifiers and birth date is reduced to birth year to protect respondent privacy.

Public-use data files and technical documentation necessary for their use are archived for access by external researchers through ICPSR at The University of Michigan. The goal of this effort is to maximize the files’ utility for research purposes while protecting the privacy of responding households. Information in the public use file undergoes formal disclosure review by the contractor prior to release to reduce the risk of re-identification, according to standards for de-identification and access control set by ICPSR. The results of the disclosure review process are presented to, and approved by, the federal project officer prior to submission of public use data to the archive.

A11. Justification for Sensitive Information ²

The ASR collects self-reported information about receipt of economic assistance from the government and application for legal permanent resident status. ORR is required to collect this information by 8 U.S.C. 1523. Respondents are informed that the provision of this information is voluntary, and that information will be kept private to the extent allowed by law, and that information will only be used for statistical purposes.

This information collection is overseen by the Institutional Review Board of the Urban Institute, the ASR prime contractor. The Urban Institute maintains a Federal wide Assurance for human subjects protections. IRB approval for the study is updated annually, and the current IRB package, which details precautions against the inadvertent release and identifiably of sensitive information, is attached as Appendix C- Current IRB Packet 11.19.18.

A12. Estimates of Annualized Burden Hours and Costs

Explanation of Burden Estimates

The ASR has an annual target of 1,500 respondents. To ensure sufficient statistical power, the survey stays in the field until this target is achieved.

The ASR asks sampled individuals to provide demographic information about each eligible individual in the household (entering during the survey’s reference period and currently aged 16 or older); asks additional questions about their own experiences and perceptions; and collects information about the household as a whole. As such, the burden to the respondent varies by the number of eligible adults in the household. Based on administration of the ASR in previous years, and taking into account revisions proposed to the ASR, we estimate the average time to complete the revised ASR to be about 48 minutes. This is an increase from the previously approved ASRs, which averaged about 30 minutes. The contact update form is estimated to average about 3 minutes per response.

Estimated Annualized Cost to Respondents

Table A3 shows the estimated burden for the redesigned ASR 2020, 2021, and 2022. The estimated average hourly wage of refugees, derived from respondents to the 2018 Annual Survey of Refugees, is $13.16. To account for fringe benefits and overhead, the rate is multiplied by two, which is $26.32. The total annual cost to respondents is estimated to be $33,558.

Table A3. Proposed Burden for ASR 2020, 2021, and 2022

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden (in hours)	Annual Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Introduction Letter and Postcard	4500	1	.05	225	75	$26.32	$1,974
ORR-9 Annual Survey of Refugees	4500	1	.80	3,600	1,200	$26.32	$31,584
Total					1,275		$33,558

A13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There are no additional costs to respondents.

A14. Annualized Costs to the Federal Government

Table A4 is presented for the proposed revision to the ICR, and applies to ASR 2020, 2021, and 2022. The cost estimate is derived from the executed contract, which was awarded via a new competitive procurement.

Table A4. Cost for ASR 2020, 2021, 2022

Cost Category	Estimated Costs
Instrument Development and OMB Clearance (One time)	$194,300
Project Management and Communication (3 Annual)	$370,500
Data Security and Storage (3 Annual)	$ 84,000
Survey Administration (3 Annual)	$1,764,300
Data Preparation, Analysis, and Documentation (3 Annual)	$468,000
Data Archiving (3 Annual)	$237,000
Total costs over the request period	$3,118,100
Annual costs	$1,039,367

A15. Explanation for Program Changes or Adjustments

The number of respondents used in the burden calculation was reduced from the previously-approved 2,000 respondents to 1,500, to reflect what can be reasonably accomplished with the resources the agency has available for this information collection. The target of 1,500 respondents was reached for ASR 2017, 2018, and 2019.

The burden associated with the ASR was increased from 0.5 to 0.80 hours, to reflect estimates of the burden from the revised instrument for a household averaging 2 eligible adults. This marginal increase in individual burden is expected to substantially increase the utility of the data collection for ORR programmatic use and secondary research use to improve understanding of the factors associated with resettlement outcomes.

A16. Plans for Tabulation and Publication and Project Time Schedule

The ASR is fielded yearly in the calendar spring and tabulated by the contractor soon after. Tabulations are published in ORR’s Annual Report to Congress.

Following the publication of the Annual Report to Congress, the public use data file and documentation will be archived on the website of ICPSR at University of Michigan, to enable secondary research use of the data.

A17. Reason(s) Display of OMB Expiration Date is Inappropriate

No request is submitted to seek approval to not display the expiration date.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

Attachments

Instrument 1 - Introduction Letter and Postcard (2020 Proposed)

Instrument 2 - ORR-9 Annual Survey of Refugees (2020 Proposed)

Appendix A - Crosswalk of Current and Revised ASR Instruments

Appendix B - Roundtable Participant List

Appendix C - Current IRB Packet 11.19.18

1 Mercer, A., Caporaso, A., Cantor, D., and Townsend, R. (2015) How Much Gets You How Much? Monetary Incentives and Response Rates in Household Surveys, Public Opinion Quarterly, Vol. 79, Issue 1, Spring 2015, Pages 105–129, https://doi.org/10.1093/poq/nfu059. (See Table 5.)

2 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

12