NonSub Change Memo - HPOG 2.0 CovidCohortSurvey

DATE: October 14, 2021

TO: Jordan Cohen

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget (OMB)

FROM: Nicole Constance and Lisa Zingman

Office of Planning, Research, and Evaluation (OPRE)

Administration for Children and Families (ACF)

SUBJECT: Request for Non-Substantive Changes to the National and Tribal Evaluation of the 2^nd Generation of the Health Profession Opportunity Grants (HPOG 2.0) (OMB #0970-0462)

This memo requests approval of three non-substantive changes to recently approved information collections for the National and Tribal Evaluation of the 2^nd Generation of the Health Profession Opportunity Grants (HPOG 2.0) (OMB # 0970-0462).

Background

Between August 2015 and July 2019, OMB has approved several information collection instruments, procedures and supporting materials in support of the HPOG 2.0 National and Tribal Evaluation. These include baseline data collection from all study participants, data collection protocols and procedures for the HPOG 2.0 Tribal Evaluation, and several instruments in support of the National Evaluation, including Short-Term and Intermediate Follow-up Surveys to support the HPOG 2.0 National Evaluation impact study. In July 2021, OMB approved Instrument 12a COVID-Cohort Short-term Survey (COVID-19 Cohort STS) and supporting materials.

Since approval, the following events have led to the need for minor changes so some of the currently approved information collection.

• OPRE’s Privacy Analyst reviewed the advance letter (Attachment K) and requested changes.

• While preparing to program the COVID-19 Cohort STS (Instrument 12a), the team identified revisions to language that would help with the clarity of the request. The team also had a change in staffing resulting in a necessary update to contact information.

• The success of the Intermediate Follow-up Survey Critical Items Only (Instrument 18a)—which was still underway at the time the COVID-19 Cohort STS was submitted to OMB for review and approval – led the team to create a similar Critical Items Instrument for the COVID-19 Cohort STS.

Overview of Requested Changes

This section summarizes the specific changes for which ACF seeks approval and the expected benefits.

#1— Revision to Supporting Material-Attachment K-Revised COVID-19 Short-term Survey Advance Letter

In reviewing the materials to be used in support of this information collection and the longer follow-up period, OPRE’s Privacy Analyst requested a revision to the advance letter. The request was to drop one sentence from the previously approved advance letter (Attachment K-Revised.) OPRE’s Privacy Analyst requested the deletion of the last sentence in the paragraph noted below to ensure that OPRE and other researchers have access to the data for potential research in the future. The letter retains the statement that information will be kept private to the extent allowed by law. Additional assurances that identifiable information won’t be shared or revealed in any reports are included in the survey introduction, where a trained interviewer can address any questions in real time.

“Whether you choose to participate in the survey or not will not affect any assistance that you may receive now or in the future. If you choose to participate, any information you provide to us will be kept private to the extent allowed by law. Only the researchers involved in this study will see your responses.”

#2— Revision to Instrument 12a COVID-19 Short-term Survey

In reviewing the COVID-19 Cohort STS in preparation for programming, the evaluation team identified one question (question I4b) that requires minor wording changes for better clarity. We also updated the name and contact information in the Introduction to reflect the change in contact person from Gretchen Locke to Radha Roy.

#3— Adding a new version of Instrument 12a: COVID-Cohort Short-term Follow-up Survey

In June 2020, OMB approved a shorter version of the Intermediate Follow-up Survey, one that captured critical items only, and could be administered in much less time than the full survey (Instrument 18a). This shorter version of the instrument was deployed when a case would otherwise be classified as a final nonrespondent and was a very successful tool, increasing the overall survey response rate for the Intermediate Follow-up Survey by 9.2 percentage points. The shorter instrument also helped to reduce the differential in response rates between the treatment and control groups—from 7.0 percentage points to 3.6 percentage points.

Given the success of the first Critical Items Instrument, ACF is requesting approval to include a similar Critical Items Instrument for the COVID-19 Cohort STS. This shorter instrument would be used only when a case would otherwise be classified as a final nonrespondent. Although ACF is mindful of contextual effects that might be caused by shortening the instrument relative to the original STS, we are also concerned about the potential for increased nonresponse bias given the resurgence of COVID in many communities. By fielding the short instrument as a last resort, we hope to minimize both contextual effects and changes in nonresponse bias that would cloud comparisons of short-term impacts in the pre-COVID and COVID eras. The analysis team will run sensitivity tests to assess how the Critical Items Only version affected comparability between the original Short-term and COVID-19 Cohort STS, in addition to using the same nonresponse analyses and adjustments to minimize contextual effects.¹

The requested changes to the full instrument are included in Instrument 12a_COVID-19_CohortShortTermSurvey_Revised and Instrument 12b COVID-Cohort Short-Term Survey_Critical Items. The short instrument is very similar to the short instrument approved for the Intermediate Follow-up Survey. There is one question in Instrument 12b that does not appear in Instrument 12a: question A3a. To determine if the respondent was answering questions in Section D about their current or most recent job, we need to ascertain if they are currently working.

We have updated the supporting statements to reflect these changes. We have also removed burden related to Instrument 1 (PAGES Participant-level baseline data collection, tribal and non-tribal grantees) and the Intermediate Follow-up Surveys (Instruments 18 and 18a), as those data collections are complete.

Time Sensitivities

ACF requests a response about these changes as quickly as possible. A prompt response will allow the COVID-19 Cohort STS data collection to remain on schedule, including ample time for the evaluation contractor to program the full survey instrument and the shorter critical items only version into its CAPI system and test the programming logic before survey administration begins in December 2021.

1 The ability to use the shorter instrument when needed may also encourage survey completion during a resurgence of COVID-19 in some communities or to mitigate COVID-19 related hesitancy from study members or local data collectors that may pose challenges to planned in-person data collection efforts.

2