NonSub Change - 24 Month Survey, Contacts

To: Kelsi Feltz

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget (OMB)

From: Mary Mueggenborg

Office of Planning, Research and Evaluation (OPRE)

Administration for Children and Families (ACF)

Date: July 19, 2023

Subject: Nonsubstantive Change Request – Supporting Youth to be Successful in Life (SYSIL) Study (OMB #0970-0574)

This memo requests approval of nonsubstantive changes to the approved information collection, Supporting Youth to be Successful in Life (SYSIL) Study (OMB #0970-0574).

Background

OMB approved the impact and implementation studies and instruments for the OPRE Study, Supporting Youth to be Successful in Life (SYSIL) on July 27, 2021, which included a baseline survey and 6-, 12-, and 24-month follow-up surveys.

Currently, the approved study includes the following information collections:

• Four 30-minute web-based surveys: one at baseline (enrollment), and then post-enrollment at 6-, 12-, and 24-months.

• Administrative data from Linked Network of Colorado (LINC) to look at additional youth outcomes.

• Four contact information update web-based surveys at 9-, 15-, 18-, and 21-months.

• Virtual and in-person visits to sites to interview staff and youth.

• Two additional brief check-in calls with staff.

Baseline data collection with youth for the impact study began in September 2021. Data collection for the 6-month and 12-month follow-up surveys began in April 2022 and October 2022, respectively. After 18 months of enrollment, the number of youth enrolled was 63 percent of the expected sample, with monthly enrollment numbers averaging 12 youth per month, as opposed to the projected 19 youth per month. After numerous discussions with the study team and caseworkers to understand the cause of the lower enrollment and to address the challenges, the study team decided to extend enrollment by 6 months and decrease the number of follow-up surveys. These proposed changes are an effort to help increase enrollment into the study.

Overview of Requested Changes

This memo requests the following nonsubstantive changes:

• Replacing the 24-month survey with 24-month administrative data from LINC that will allow us to look at longer term outcomes without the 24-month post-enrollment survey.

• Reducing the number and revising the timing of the contact information requests since there will no longer be a survey at 24 months. We propose a change from 4 requests at 9-, 15-, 18-, and 21-months post-enrollment to 2 requests at 3- and 9-months post-enrollment with the same $5 token of appreciation.

• Replacing the test of the $5 token of appreciation for the contact information requests with a test of using a $5 pre-payment for the 6-month follow-up survey.

Replacing the 24-month follow-up survey

Since enrollment for the study has been slower than expected, the study team has decided to extend enrollment for 6 months. To support this, we propose replacing the 24-month follow-up survey with 24-month administrative data. Administrative data obtained from LINC will include information on education and employment, stable housing, involvement with the criminal or juvenile justice system, access to available system resources, child welfare status, and parenting status.

Reducing the number and revising the timing of the contact information requests

In replacing the 24-month follow-up survey with administrative data, the need for updated contact information after the 12-month survey is no longer necessary. As such, we propose removing the 15-, 18-, and 21-month requests and adding a 3-month request.

Given that the 6- and 12-month follow-up surveys are now our primary sources of survey data we think it is important to improve the consistency in our contact with youth and to ensure we have reliable contact information to use for outreach on the 6- and 12-month surveys. Including a 3-month contact information request will allow us to engage with youth soon after they are enrolled in the study and will enable them to provide updated contact information before the 6-month survey. We propose maintaining the 9-month contact information request to gather updated contact information before the 12-month survey.

Replacing the test of the tokens of appreciation

The test of the $5 token of appreciation is currently structured so the test is the provision of a $5 token at the first contact information update request (9-months post-enrollment). The condition varies by cohort, which is synonymous with month, so all youth in a certain month receive the same token of appreciation for the 9-month contact information update request ($5 or nothing). All youth receive the $5 token with subsequent contact information update requests, with those youth who did not receive $5 at the first request receiving $10 at the final request.

The proposed change in the timing of contact information update requests means the current test does not work. We believe having some youth not receive a token of appreciation with the 3-month contact information update, our first contact post-enrollment, may result in lower response rates for the surveys. Therefore, we would like to eliminate the current test of tokens of appreciation and provide a $5 token to all youth who respond at both the 3- and 9-month contact information requests. There will no longer be a $10 token offered at the final request. We want to ensure youth get into the habit of responding to our data collection requests from the beginning and encourage them to respond to our subsequent requests.

We propose replacing the current test of tokens of appreciation with a test of a two-part payment option for the 6-month follow-up survey token of appreciation. Currently, respondents are supposed to receive $45 for completion of the 6-month follow-up survey. For this new test, cohorts of youth (youth enrolled in a month) will be randomly assigned to either receive a $5 pre-payment for the 6-month survey, or no pre-payment. Those who receive the pre-payment will get the remainder of the token of appreciation ($40) upon completing the 6-month survey. Those who do not receive the pre-payment will get the full token of appreciation ($45) upon completion of the 6-month survey. We will examine whether response rates vary based on their pre-payment status.

Finally, we propose to restructure the token of appreciation for the 12-month survey. Currently, respondents are supposed to receive $50 for completion of the 12-month follow-up survey. We propose all youth receive the $5 pre-payment for the 12-month survey with the remainder of the token of appreciation ($45) upon completion. We will not include a test of this option as there is no variation in whether youth receive the pre-payment.

Below we summarize the rounds of data collection and tokens of appreciation, incorporating the proposed changes:

• Baseline survey: $40 (currently approved)

• 3-month contact information request: $5 (new contact request, amount consistent with other contact update requests currently approved)

• 6-month follow-up survey: $45 (with random assignment of a $5 pre-payment for some cohorts) (overall amount remains the same, new structure to include pre- and post-offerings)

• 9-month contact information request: $5 (currently approved)

• 12-month follow-up survey: $5 pre-payment and $45 post-payment (overall amount remains the same, new structure to include pre- and post-offerings)

Overall, removing the 24-month survey and the adjustments to the contact update requests result in an overall reduction in burden estimates from 461 hours to 413 hours. An updated Supporting Statement A, Supporting Statement B, and revised Appendix A (consent and assent forms) accompany this memo.

Time Sensitivities

We would appreciate approval as soon as possible since data collection is ongoing and we would like to make these adjustments for as many youth as possible as they complete the data collection activities.