ASPA COVID-19 Public Education Campaign Evaluation Surveys_Supporting Statement Part A_010421

Supporting Statement Part A

U.S. Department of Health and Human Services

ASPA COVID-19 Public Education Campaign Evaluation Surveys

(OMB Control No. ####-New)

Justification

1. Need and Legal Basis

In accordance with 5 CFR 1320.13, the United States Department of Health and Human Services (HHS) plans to submit an emergency processing submission. As a general matter, under sections 1320 (a)(2)(ii) and (2)(iii) of the federal regulations, an emergency processing may be authorized in the occurrence of an unanticipated event and the use of normal clearance procedures is reasonably likely to prevent or disrupt the collection of information. Given the impact of the COVID-19 pandemic on the nation, the Secretary of Health and Human Services (HHS) has also declared a public health emergency effective January 27, 2020, under section 319 of the Public Health Start Printed Page 75022Service Act (42 U.S.C. 247d [1]) and renewed it continually since its issuance.

2. Information Users

The office of the Assistant Secretary for Public Affairs (ASPA) is collecting information through the COVID-19 Attitudes and Beliefs Survey (CABS) and Monthly Outcome Survey (MOS) to evaluate the impact of the HHS COVID-19 Public Education Campaign. Throughout the duration of the campaign, this information will primarily be used by ASPA to determine whether the campaign is having the intended impact on target audiences’ knowledge, attitudes, and beliefs as they relate to COVID-19 and targeted preventative behaviors. This will inform an evaluation of the efficacy of the Campaign subsequent changes to Campaign messaging or strategy. The information will also be used to keep key stakeholders informed of the Campaign’s progress.

At the conclusion of the Campaign, the information collected in these two outcome evaluation surveys will be used to inform a final cost-benefit analysis.

3. Use of Information Technology

Both the COVID-19 Attitudes and Beliefs Survey (CABS) and the Monthly Outcome Survey (MOS) will be administered online by survey panel vendors (NORC AmeriSpeak and Ipsos KnowledgePanel). Panel members will be invited to the survey primarily through email invitations and reminders and will be asked to log-in to the panel vendor portal to complete the survey. The surveys minimize respondent burden by using clear and easy to read pages with minimal graphics to make it as easy as possible to respond to questions. The surveys also incorporate skip logic so respondents automatically skip past any questions that are not applicable to them.

Members of the NORC AmeriSpeak and KnowledgePanels update their personal information with the panel vendors on a regular basis, meaning that a number of profile variables are available through the vendors for an additional cost. For this effort, ASPA will purchase relevant profile variables from the panel vendors to help minimize the number of questions asked in the survey, thus reducing burden.

4. Duplication of Efforts

There is no duplicative information collection instrument or process.

5. Small Businesses

The outcome evaluation surveys will be completed by individuals and not by any organizations, and therefore will not affect small businesses.

6. Less Frequent Collection

This request is voluntary. Sampled members can choose to respond or not respond to the surveys with no penalty for not responding. Data collected through the CABS and MOS data collection efforts will help determine whether and to what extent the HHS COVID-19 Public Education Campaign is delivering life-saving health information to the general public and to communities who are disproportionately impacted by COVID-19. The surveys must be conducted on a regular basis (CABS: every four months; MOS: every month) so the Campaign can track awareness and impact and determine if Campaign messages or strategy need to be adjusted. Communicating impactful health information in a timely basis is of critical importance during a pandemic, and the Campaign needs to remain continuously informed to be able to make quick and important decisions. If the surveys were conducted less frequently, then the Campaign would not have the information needed to evaluate its progress and determine next steps on an ongoing basis. Additionally, submitting this request through an emergency submission will enable ASPA to execute the first wave of the CABS as a benchmark/baseline, which is typically fielded before or immediately at the start of a campaign and provides a true measure of the environment prior to intervention.

7. Special Circumstances

There are no special circumstances associated with this collection.

8. Federal Register/Outside Consultation

Before submitting for approval, both the CABS and MOS were reviewed by an expert panel of reviewers including staff from HHS, the Food and Drug Administration, and the Centers for Disease Control and Prevention. The CABS and MOS were each submitted to an external IRB review process and received approval.

9. Payments/Gifts to Respondents

Each panel vendor has their own rewards program that provides panel members with points for completing surveys, which can be redeemed for gift cards or merchandise. These small tokens of appreciation will help maximize response rates across a more diverse respondent pool.

For CABS, respondents will receive the equivalent of $10 in points for completing Wave 1 of the survey; they will receive the equivalent of $18 in points for completing each subsequent Wave of the survey. For CABS, providing an incentive is important not just for recruitment of respondents, but for retention of respondents over the course of all six waves of the survey. High rates of attrition would have a detrimental impact on ASPA’s ability to run within-person analyses that capture changes in attitudes and behaviors over the course of the Campaign. Sample members will not be replaced if they do not respond.

For the MOS, respondents will receive the equivalent of $1 in points for completing the survey. Incentives are important for the MOS because the survey fielding window is relatively short and the number of required completes is relatively high. Providing incentives will allow the survey vendor to recruit sufficient number of respondents during the fielding window.

10. Confidentiality

ASPA will comply with all Privacy Act, Freedom of Information laws and regulations that apply to this collection. ASPA will not have access to any identifying information for respondents to the outcome evaluation surveys. Both panel vendors provide their members with confidentiality assurances when they agree to become members of the survey panels and do not release personally identifying information to data users.

The NORC AmeriSpeak confidentiality agreement with panel members outlines that responses will be anonymized, and each participant will be identified by a unique ID. This number is generated for each separate study, so the same number is not used for a specific panelist across all data files. Panel member survey responses are confidential, and personally-identifying information is not revealed to clients unless the panelist gives NORC explicit permission to do so. 

The Ipsos KnowledgePanel member “Bill of Rights” states that responses will be anonymized, and each participant will be assigned a unique ID. Express permission from participants is required for Ipsos to share identifying information with a survey client. KnowledgePanel members are informed that participation in surveys is voluntary and that they do not have to answer any questions they do not wish to answer. 

11. Sensitive Questions

There are few psychological risks to completing cognitive, decision making, and psychological assessments. However, survey questions about COVID-19 may make participants feel uncomfortable or bring up unwanted thoughts or feelings. To minimize these risks, we will emphasize the voluntary and confidential nature of the research, as well as state the limits to confidentiality in the informed consent form, and again at the beginning of each research wave. Additionally, in the event participants experience any distress from this study, we will provide information to the 1) SAMHSA Disaster Distress Helpline (primary), which provides 24/7 crisis counseling and support for individuals experiencing emotional distress caused from natural or human-caused disasters, as well as the 2) Suicide Prevention Lifeline (a 24/7 free and confidential support resource).

12. Burden Estimates (Hours & Wages)

For the MOS, we estimate 5,000 completes x 0.17 hours per submission = 850 burden hours associated with this survey each wave. Over 12 waves per year, the annual burden is estimated to be 10,200 total burden hours.

For the CABS, we estimate 4,000 completes x 0.58 hours per submission = 2,320 burden hours associated with this survey each wave. Over 3 waves per year, the annual burden is estimated to be 6,960 total burden hours.

13. Capital Costs

There are no direct costs to respondents other than their time to participate in the study.

14. Cost to Federal Government

For the CABS, the survey will field for 6 waves using a panel vendor and will collect an estimate 16,817 responses from 4,000 unique participants in total. The total labor price is $2,487,768.24 with $1,345,250 additional for ODCs. Included in the ODCs is the estimated $270,000 that will be paid in incentives. Broken down, the price for one wave of the CABS is $414,628.04 in labor costs and $224,083 in ODCs associated with survey fielding, which includes an estimated $45,000 that will be paid in incentives. This price reflects the labor and direct costs to field 6 waves of a 35-minute online survey to 4,000 total participants from the NORC AmeriSpeak Panel. Included in these costs are: 1) Development of the annotated questionnaire; 2) Panel vendor services to program and test the survey in English and Spanish, field the survey, staff the survey helpline, and clean and deliver data; 3) Participant incentives; 4) Translation of materials into Spanish; 5) Profile variables from the panel vendor; 5) Preliminary dataset delivery for Wave; 6) IRB review; 7) Analysis and reporting (tri-annual presentations and reports of key findings and recommendations); and 8) Quality control.

Over the course of the base year, the MOS will field for 12 waves using a panel vendor and will collect 5,000 completes. The total labor price is $185,148.24 with $445,250.00 additional for ODCs. Included in the ODCs is the $120,000 that will be paid in incentives. Broken down, the price for one wave of the MOS is $7,714.51 in labor costs and $37,000 in ODCs associated with survey fielding, which includes $5,000 that will be paid in incentives. This price reflects the labor and direct costs to field 12 waves of a 10-minute online survey to 120,000 total participants from the Ipsos KnowledgePanel. Included in these costs are: 1) Development of the MOS annotated questionnaire. 2) Panel vendor services to program and test the survey in English and Spanish, field the survey, staff the survey helpline, and clean and deliver data to FMG; 3) Participant incentives; 4) Translation of materials into Spanish; 5) Profile variables from the panel vendor; 6) Preliminary dataset delivery for Wave 1; 6) IRB approval; 7) Analysis and reporting; and 8) Quality control.

15. Changes to Burden

This is a new information collection request.

16. Publication/Tabulation Dates

Data collection for the study is likely to occur between January 1, 2021, and February 28, 2023 with potential for further extension. The aim of this survey is to evaluate COVID-19 public health communications. As such, the data will be used internally to evaluate these efforts. After each survey wave has been completed, a results briefing will be prepared to evaluate and report the survey findings. We expect to publish peer reviewed articles on an iterative basis.

To reach these goals, we aim to use the following advanced statistical techniques:

• Cost benefit analysis and sensitivity analyses based on computer simulation

• Cluster/Segmentation analysis

• Panel regression analysis/differences-in-differences regression

17. Expiration Date

The CABS will display the OMB approval number and expiration date on all survey materials, including the survey invitations, the survey web screens, and at the bottom of all survey screens. The MOS is an omnibus survey and therefore does not lend itself to the displaying of an OMB approval number and expiration date at the beginning of the survey or on survey communications.

18. Certification Statement

There are no exceptions to item 19 of OMB Form 83-1.

Attachment A: COVID-19 Attitudes and Beliefs Survey Questionnaire

Attachment B: COVID-19 Attitudes and Beliefs Survey NORC Consent Form

Attachment C: COVID-19 Attitudes and Beliefs Survey Communications

Attachment D: COVID-19 Monthly Outcome Survey Questionnaire

Attachment E: COVID-19 Monthly Outcome Survey Ipsos Omnibus COVID Questions

Attachment F: COVID-19 Monthly Outcome Survey Ipsos Consent Form

Attachment G: COVID-19 Monthly Outcome Survey Communications

5