SS-A COVID-19 _0990-0475 Public Education Campaign Evaluation Surveys_06-03-22 (002)

Supporting Statement Part A

U.S. Department of Health and Human Services

ASPA COVID-19 Public Education Campaign Evaluation Surveys

(OMB Control No. 0990-0475)

Justification

1. Need and Legal Basis

In accordance with 5 CFR 1320.13, the United States Department of Health and Human Services (HHS) plans to submit an emergency revision request. As a general matter, under sections 1320 (a)(2)(ii) and (2)(iii) of the federal regulations, an emergency processing may be authorized in the occurrence of an unanticipated event and the use of normal clearance procedures is reasonably likely to prevent or disrupt the collection of information. Given the impact of the COVID-19 pandemic on the nation, the Secretary of Health and Human Services (HHS) has also declared a public health emergency effective January 27, 2020, under section 319 of the Public Health Start Printed Page 75022Service Act (42 U.S.C. 247d [1]) and renewed it continually since its issuance.

2. Information Users

The office of the Assistant Secretary for Public Affairs (ASPA) is collecting information through the COVID-19 Attitudes and Beliefs Survey (CABS) and Monthly Outcome Survey (MOS) to evaluate the impact of the HHS COVID-19 Public Education Campaign. Throughout the duration of the campaign, this information will primarily be used by ASPA to determine whether the campaign is having the intended impact on target audiences’ knowledge, attitudes, and beliefs as they relate to COVID-19 and targeted preventative behaviors. This will inform an evaluation of the efficacy of the Campaign subsequent changes to Campaign messaging or strategy. The information will also be used to keep key stakeholders informed of the Campaign’s progress.

To further enhance the robustness of the evaluation, ASPA will conduct two supplements to the longitudinal CABS. The first involves increasing the sample of parents surveyed in the next two waves and by implementing an additional, brief media diet survey to CABS respondents in between the next two waves. Increasing the parent sample will strengthen the ASPA project’s ability to evaluate Campaign impact on parent confidence in getting their eligible children vaccinated and on subsequent child COVID-19 vaccinations, and the inclusion of items to assess respondents’ media channel and platform use will improve the ability to ascertain the extent to which respondent vaccination can be attributed to Campaign exposure.

3. Use of Information Technology

Both the COVID-19 Attitudes and Beliefs Survey (CABS) and the Monthly Outcome Survey (MOS) will be administered online by survey panel vendors (NORC AmeriSpeak and Ipsos KnowledgePanel). Panel members will be invited to the survey primarily through email invitations and reminders and will be asked to log-in to the panel vendor portal to complete the survey. The surveys minimize respondent burden by using clear and easy to read pages with minimal graphics to make it as easy as possible to respond to questions. The surveys also incorporate skip logic so respondents automatically skip past any questions that are not applicable to them.

Members of the NORC AmeriSpeak and KnowledgePanels update their personal information with the panel vendors on a regular basis, meaning that a number of profile variables are available through the vendors for an additional cost. For this effort, ASPA will purchase relevant profile variables from the panel vendors to help minimize the number of questions asked in the survey, thus reducing burden.

4. Duplication of Efforts

There is no duplicative information collection instrument or process.

5. Small Businesses

The outcome evaluation surveys will be completed by individuals and not by any organizations, and therefore will not affect small businesses.

6. Less Frequent Collection

This request is voluntary. Sampled members can choose to respond or not respond to the surveys with no penalty for not responding. Data collected through the CABS and MOS data collection efforts will help determine whether and to what extent the HHS COVID-19 Public Education Campaign is delivering life-saving health information to the general public and to communities who are disproportionately impacted by COVID-19. The surveys must be conducted on a regular basis (CABS: every four months; MOS: every month) so the Campaign can track awareness and impact and determine if Campaign messages or strategy need to be adjusted. Communicating impactful health information in a timely basis is of critical importance during a pandemic, and the Campaign needs to remain continuously informed to be able to make quick and important decisions. If the surveys were conducted less frequently, then the Campaign would not have the information needed to evaluate its progress and determine next steps on an ongoing basis. Additionally, submitting this request through an emergency submission enabled ASPA to execute the first wave of the CABS as a benchmark/baseline, which is typically fielded before or immediately at the start of a campaign and provides a true measure of the environment prior to intervention.

7. Special Circumstances

There are no special circumstances associated with this collection.

8. Federal Register/Outside Consultation

The 60-day FRN for the original package was published 9/13/21 (Federal Register Citation 86 FR 50895) and no comments were received; similarly, the 30-day FRN was published 11/22/21 (Federal Register Citation 86 FR 66319) and no comments were received. The 30-day FRN for this emergency revision request will be published concurrently with submission of the revision request to OMB; the 60-day FRN will follow the 30-day notice.

Before submitting for approval, both the CABS and MOS were reviewed by an expert panel of reviewers including staff from HHS, the Food and Drug Administration, and the Centers for Disease Control and Prevention. The CABS and MOS were each submitted to an external IRB review process and received approval. Both surveys have been in the field for over a year continue to guide the execution and evaluation of the COVID-19 Public Education Campaign.

9. Payments/Gifts to Respondents

Each panel vendor has their own rewards program that provides panel members with points for completing surveys, which can be redeemed for gift cards or merchandise. These small tokens of appreciation will help maximize response rates across a more diverse respondent pool.

For CABS, respondents will receive the equivalent of $10 in points for completing Wave 1 of the survey; they will receive the equivalent of $18 in points for completing each subsequent Wave of the survey. For CABS, providing an incentive is important not just for recruitment of respondents, but for retention of respondents over the course of all six waves of the survey. High rates of attrition would have a detrimental impact on ASPA’s ability to run within-person analyses that capture changes in attitudes and behaviors over the course of the Campaign. Sample members will not be replaced if they do not respond.

For the MOS, respondents will receive the equivalent of $1 in points for completing the survey. Incentives are important for the MOS because the survey fielding window is relatively short and the number of required completes is relatively high. Providing incentives will allow the survey vendor to recruit sufficient number of respondents during the fielding window.

10. Confidentiality

ASPA will comply with all Privacy Act, Freedom of Information laws and regulations that apply to this collection. ASPA will not have access to any identifying information for respondents to the outcome evaluation surveys. Both panel vendors provide their members with confidentiality assurances when they agree to become members of the survey panels and do not release personally identifying information to data users.

The NORC AmeriSpeak confidentiality agreement with panel members outlines that responses will be anonymized, and each participant will be identified by a unique ID. This number is generated for each separate study, so the same number is not used for a specific panelist across all data files. Panel member survey responses are confidential, and personally-identifying information is not revealed to clients unless the panelist gives NORC explicit permission to do so. 

The Ipsos KnowledgePanel member “Bill of Rights” states that responses will be anonymized, and each participant will be assigned a unique ID. Express permission from participants is required for Ipsos to share identifying information with a survey client. Knowledge Panel members are informed that participation in surveys is voluntary and that they do not have to answer any questions they do not wish to answer. 

11. Sensitive Questions

There are few psychological risks to completing cognitive, decision making, and psychological assessments. However, survey questions about COVID-19 may make participants feel uncomfortable or bring up unwanted thoughts or feelings. To minimize these risks, we will emphasize the voluntary and confidential nature of the research, as well as state the limits to confidentiality in the informed consent form, and again at the beginning of each research wave. Additionally, in the event participants experience any distress from this study, we will provide information to the 1) SAMHSA Disaster Distress Helpline (primary), which provides 24/7 crisis counseling and support for individuals experiencing emotional distress caused from natural or human-caused disasters, as well as the 2) Suicide Prevention Lifeline (a 24/7 free and confidential support resource).

12. Burden Estimates (Hours & Wages)

For the MOS, we estimate 5,000 completes per wave with an average response time of 15 minutes. The initial burden estimate accounted for only one year of data collection. The following burden calculation allows for continued data collection of up to 24 additional waves. 24 waves x 5,000 respondents = 120,000 total respondents. 120,000 total respondents x 15 minutes = 1,800,000 minutes or 30,000 hours.

For the CABS, we estimate no more than 3,800 completes per wave with an average response time of 35 minutes. (Note, this is a longitudinal survey that began with 4,000 respondents, and the completes per wave decrease due to attrition.) The initial burden estimate accounted for only one year of data collection. The following burden calculation allows for continued data collection of up to six additional waves. 6 waves x 3,800 respondents = 22,800 total respondents. 22,800 total respondents x 35 minutes = 798,000 minutes or 13,300 hours.

The CABS supplements will incur an additional 2,060 burden hours for the extra parent respondents recruited for the next wave (n=1,350), the additional parents retained in the following wave (n=1,215 estimate due to unknown attrition), and the media diet supplement for all original respondents still in sample and new parent respondents retained (n=4,500, estimated due to unknown attrition). Note the additional parents sample responds to the current 35-minute survey, and the forthcoming media diet supplement is estimated to take approximately 7.5 minutes.

Therefore, the total burden request for this package is equal to 30,000 hours for MOS plus 13,300 hours for CABS plus 2,060 for CABS supplements, or 45,360 total hours as reflected in the table below:

Estimated Annualized Burden Hours

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
MOS	MOS Survey Questionnaire	120,000	1	15/60	30,000
CABS	CABS Survey Questionnaire	3,800	6	35/60	13,300
	Parent Supplement	2,565	1	35/60	1,497
	Media Diet Supplement	4,500	1	7.5/60	563
Total					45,360

13. Capital Costs

There are no direct costs to respondents other than their time to participate in the study.

14. Cost to Federal Government

The total cost to the government is approximately $5.1M to conduct 6 waves of CABS and 24 waves of the MOS as broken down below and an additional $300K for the CABS supplements.

Over the course of a year, the CABS will field 3 waves using a panel vendor and will collect responses from up to 3,800 unique participants in total. The total labor price is approximately $2.5M with approximately $1.5M additional for ODCs. Incentive payments are included in ODCs. Included in these costs are: 1) Development of the annotated questionnaire; 2) Panel vendor services to program and test the survey in English and Spanish, field the survey, staff the survey helpline, and clean and deliver data; 3) Participant incentives; 4) Translation of materials into Spanish; 5) Profile variables from the panel vendor; 5) Preliminary dataset delivery for Wave; 6) IRB review; 7) Analysis and reporting (tri-annual presentations and reports of key findings and recommendations); and 8) Quality control.

Over the course of a year, the MOS will field for 12 waves using a panel vendor and will collect 5,000 completes. The total labor price is approximately $200K with approximately $500K additional for ODCs, which includes incentive payments. Included in these costs are: 1) Development of the MOS annotated questionnaire. 2) Panel vendor services to program and test the survey in English and Spanish, field the survey, staff the survey helpline, and clean and deliver data to FMG; 3) Participant incentives; 4) Translation of materials into Spanish; 5) Profile variables from the panel vendor; 6) Preliminary dataset delivery for Wave 1; 6) IRB approval; 7) Analysis and reporting; and 8) Quality control.

15. Changes to Burden

The information collection request itself (i.e., MOS and CABS) is unchanged except for the CABS supplements; the burden calculation initially submitted only accounted for one year of data collection. This emergency revision increases burden to account for MOS and CABS data collection through the life of the package, expiring in January 2024.

16. Publication/Tabulation Dates

Data collection for the study will occur throughout 2022 and 2023. Extension past January 2024 is not anticipated. The aim of this survey is to evaluate COVID-19 public health communications. As such, the data will be used internally to evaluate these efforts. After each survey wave has been completed, a results briefing will be prepared to evaluate and report the survey findings. We expect to publish peer reviewed articles on an iterative basis.

To reach these goals, we aim to use the following advanced statistical techniques:

• Cost benefit analysis and sensitivity analyses based on computer simulation

• Cluster/Segmentation analysis

• Panel regression analysis/differences-in-differences regression

17. Expiration Date

The CABS will display the OMB approval number and expiration date on all survey materials, including the survey invitations, the survey web screens, and at the bottom of all survey screens. The MOS is an omnibus survey and therefore does not lend itself to the displaying of an OMB approval number and expiration date at the beginning of the survey or on survey communications.

18. Certification Statement

There are no exceptions to item 19 of OMB Form 83-1.

Attachment A: Current COVID-19 Monthly Outcome Survey Questionnaire_Wave 17

Attachment B: Current COVID-19 Attitudes and Beliefs Survey Questionnaire_Wave 5

Attachment C (forthcoming): COVID-19 Attitudes and Beliefs Media Diet Supplement

6