SSB - ACF Evidence Capacity Support: Survey of SOT Grantees

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

ACF Evidence Capacity Support: Survey of Survivors of Torture Grantees

Formative Data Collections for Program Support

0970 - 0531

Supporting Statement

Part B

April 2022

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Nicole Deterding, Harmanpreet Bhatti

Part B

B1. Objectives

Study Objectives

The Office of Planning, Research, and Evaluation (OPRE) at the Administration for Children and Families (ACF) under the U.S. Department of Health and Human Services (HHS) proposes to conduct an online survey with up to 35 Services for Survivors of Torture (SOT) grantee programs funded by the Office of Refugee Resettlement (ORR). The purpose of the survey is to learn more about SOT program capacity, current data collection and reporting processes, perceptions of reporting requirements, and technical assistance and training needs. Information from the surveys will be used by ORR to assess the feasibility for future evaluations among SOT programs and determine any changes that can be made in future data collection and reporting requirements.

Generalizability of Results

As noted in Supporting Statement A, this study is intended to present a description of SOT programs’ performance management capacity, not to promote statistical generalization to other programs or populations. The findings from the SOT program survey are not meant to be representative of other programs or populations.

Appropriateness of Study Design and Methods for Planned Uses

A survey using a combination of multiple choice and open-ended questions is the best data collection approach to determine any changes that should be made to the current SOT data collection and reporting requirements. Multiple-choice questions will provide descriptive data that is collected in a consistent way across programs. Open-ended survey items provide programs the opportunity to include more detailed and nuanced responses that will add context to quantitative data. Surveys can be completed electronically and are of relatively low burden to respondents but allow for the perspectives of a variety of staff from many programs to be captured and used to inform ORR’s data improvement initiatives for SOT programs and required data collection activities.

This information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.  

B2. Methods and Design

Target Population

The target population for this information collection is SOT program staff with ORR grants. The study team will use non-probability, purposive sampling to contact program directors across all 35 SOT programs. The survey will be completed by staff serving in a variety of roles within the SOT programs, such as program directors and social workers. Due to the anticipated variation in the roles of individuals contributing to the survey responses, survey data may not be representative of SOT program staff as a whole.

Respondent Recruitment

ORR will connect the study team with program directors from each of the 35 SOT programs via email. The study team will then follow up with each program with an invitation to complete the survey. Program directors will identify the appropriate staff to help complete the survey. For more information on outreach procedures, see section B4. Collection of Data and Quality Control.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument

The survey was developed by a team of researchers familiar with the current SOT data collection and reporting requirements and builds on previous work with the SOT program. This previous work included a review of SOT grantee Performance Progress Reports and background documents, a literature review on commonly used performance measures in support for survivors of torture programs, an analysis of program-level administrative data from ORR, and feedback sessions with select SOT program staff and clients.

The survey was reviewed by the study team and project officers and streamlined to include only questions necessary to achieve the objectives of the data collection. See Instrument 1. SOT Program Survey for the full survey instrument.

B4. Collection of Data and Quality Control

Child Trends, an OPRE subcontractor, will be responsible for all survey administration and data monitoring activities.

Outreach to SOT program staff:

All respondent recruitment will take place via email. A representative from ORR will email SOT program directors to provide information about the survey. Child Trends will follow up with the program directors to provide additional information and instructions for completing the survey. Ultimately, each program director will be asked to identify one person from their program to complete the survey using the online survey platform, Alchemer. The individual who is responsible for completing the online survey should seek input from additional staff as needed. The study team will plan to send up to two additional emails to program directors reminding them to complete the survey. Please see Attachment B. SOT Survey outreach emails for all email templates.

Survey administration:

Child Trends will share the survey link with SOT program staff via email. The survey will be programmed into a secure, online platform, Alchemer. SOT program staff will have the opportunity to complete the survey at their convenience within a one-month timeframe. Contact information for a study team member from Child Trends who can answer any questions SOT staff have while completing the survey will be shared with the survey link.

Data monitoring:

The survey will be programmed to safeguard against major data quality issues. For example, for multiple choice questions where there is only one appropriate response, the survey will not allow more than one response option to be selected. The survey also includes a question asking the respondent if the study team may contact them to ask follow-up questions. Staff from Child Trends will plan to review survey responses as they are submitted and follow up with any questions as long as the respondent provided permission for the study team to do so.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

The survey is not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. Response rates will not be reported publicly, however, response rates will be calculated and reported in an internal memo to ORR.

NonResponse

As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated.

B6. Production of Estimates and Projections

The data will not be used to generate population estimates, either for internal use or dissemination. The data collected will only be used for program improvement and to inform future evaluation efforts.

B7. Data Handling and Analysis

Data Handling

All surveys will be collected using a secure, online platform, Alchemer. The survey will be programmed with validation checks to reduce errors (e.g., for questions where respondents should select only one answer, the survey will be programmed such that only one response can be selected). Further, data will be downloaded directly from Alchemer and imported into a data analysis software, Stata, reducing any errors associated with manually transferring data.

Data Analysis

For quantitative survey items, the study team will run descriptive statistics using the Stata software. Where appropriate, the study team will also run cross-tabulations to examine any differences across program characteristics. The study team will then review and summarize program responses to open-ended questions.

Data Use

The research team will not release any data from this collection to the public. The data collected will be for internal ORR use only. The survey findings will be shared with ORR in a memo that will include descriptive statistics and summarize qualitative responses.

B8. Contact Person

Child Trends will lead the data collection and analysis for this information request. The team at Child Trends is led by Sarah Catherine Williams ([email protected]).

Attachments

Attachment A – IRB Determination Letter

Attachment B – SOT Survey outreach emails

Instruments

Instrument 1 – SOT Program Survey

4