Supporting Statement A for
Federal COVID Response - Audience Feedback to Inform Ongoing Messaging and Strategies for "Combat COVID"
OMB# 0925-XXXX, exp., XX/XX/XXXX
06/17/21
Contact Information
Ryan Bayha
National Institutes of Health, Office of Science Policy
6705 Rockledge Drive, Suite 750, MSC 7985
Bethesda, MD 20892-7985
TYPE OF COLLECTION: Emergency
Mini Supporting Statement A
A.1 Circumstances Making the Collection of Information Necessary
On February 24, 2021 The White House announced the continuation of the national emergency concerning the COVID-19 pandemic. In response to the ongoing national emergency, the Federal COVID Response (FCR) Team was established, along with the Combat COVID initiative (combatcovid.hhs.gov).
The FCR Team is a cross-agency partnership that includes the U.S. Department of Health and Human Services (HHS), including the National Institutes of Health (NIH) Office of the Director, Centers for Disease Control and Prevention (CDC), the U.S. Food and Drug Administration (FDA), the Biomedical Advanced Research and Development Authority (BARDA), and the U.S. Department of Defense (DOD). The FCR Team oversees the Combat COVID initiative (including the combatcovid.hhs.gov website)—a multifaceted effort to provide the general public and healthcare providers with the latest evidence-based information on COVID-19 treatments and the Accelerating COVID-19 Therapeutic Interventions and Vaccines (ACTIV) clinical trials.
Together with their contractor, the FCR Team is working to:
Address participation barriers and raise awareness of ACTIV clinical trials, and
Ensure the general public’s and health care provider’s needs are met as it pertains to evidence-based information on these trials.
Because the COVID-19 treatment landscape continues to evolve, new evidence-based information continues to come to the forefront, and audience needs continue to change, it is critical for the FCR Team to collect quick audience feedback from the general public (especially from groups who have not historically been well-represented in clinical trials) and healthcare providers to identify these evolving needs. By understanding target audience needs, the FCR team will be able to properly develop and broadly disseminate relevant COVID-19 treatment and ACTIV clinical trial resources.
Given the state of this national public health emergency, the FCR Team is requesting emergency clearance from the Office of Management and Budget (OMB) to ensure the timely collection and application of data that is so critical to supporting efforts to mitigate the pandemic in the Unites States.
The collection is authorized under Title 42 United States Code Section 285.
A.2 Purpose and Use of the Information Collection
The purpose of the information collection is to collect routine feedback from the Combat COVID Initiative’s two target audiences (the general public and healthcare providers) to identify evolving needs and better disseminate relevant information as it relates to COVID-19 treatment and ACTIV clinical trial resources, specifically.
Data collected through this emergency clearance will be used to inform the development and broad dissemination of Combat COVID resources, including new or enhanced message and material concepts (e.g., social media ads, digital display ads, out-of-home ads), and/or web pages (combatcovid.hhs.gov). The FCR team recognizes that this emergency submission only authorizes data to be collected for six months following the approval date and will conform with this regulation.
In order to continue to respond to this national public health emergency through the routine collection of data, the FCR team will begin the process of developing and submitting a separate full clearance package for the OMB’s consideration following the completion of this six-month emergency clearance.
Upon emergency clearance approval, the team will employ two strategies to collect this routine audience feedback:
DATA COLLECTION STRATEGY 1: Bi-monthly 60-minute virtual audience feedback teams sessions (through focus groups, in-depth interviews, online bulletin boards) for rapid qualitative testing of new Combat COVID message and material concepts, including: social media ads, digital display ads, out-of-home ads, and new web pages and content for CombatCOVID.hhs.gov.
DATA COLLECTION STRATEGY 2: 15-minute custom web surveys to quantitatively understand target audience needs, awareness of Combat COVID, and changes over time in knowledge attitudes, and behaviors related to clinical trials for COVID-19 treatments, and to inform ongoing messages and communications strategies.
Submitting for Non-Substantial Changes
Because the COVID-19 landscape continues to evolve, we would like to request that we submit to OMB relevant new stimuli (e.g., social media ad concepts, out-of-home ad concepts, digital display ad concepts, webpage mockups) on a rolling basis for non-substantive change reviews. While our team has carefully crafted the questions needed to be asked via DATA COLLECTION STRATEGY 1 and DATA COLLECTION STRATEGY 2 above, the team cannot confirm all stimuli beyond the first data collection endeavor of DATA COLLECTION STRATEGY 1 and 2 at the time of this initial submission given the quick-changing nature of COVID-19 and the messaging that may be needed in the future as it relates to clinical trials for treatments. In terms of submitting non-substantial changes, the team will plan to prepare and deliver new stimuli to OMB no later than 10 business days prior to any data collection effort. Should OMB have feedback to the new stimuli being tested, the team will integrate feedback and will not move forward with the data collection until approved by OMB.
For DATA COLLECTION STRATEGY 2 (the two surveys), there may be a need to slightly adjust question wording depending on the stimuli that we are asking respondents to react to. For example, “Have you seen this webpage before today?” might need to be replaced with “Have you seen this ad before today?” Changes such as this will not impact the length of the survey or estimated burden. We will; however, plan to submit these relevant changes to OMB as non-substantial change requests, and will not proceed with the data collection until the non-substantial change requests are reviewed and approved by OMB.
Audience Feedback Teams Sessions: The FCR team will use a professional market research firm and their participant opt-in database to recruit using an OMB-approved recruitment script and screener (see Attachments 1 and 2) for four Combat COVID audience feedback teams. These feedback teams will participate in routine qualitative sessions (i.e., online bulletin boards, focus groups, and in-depth interviews [See Attachments 3-5]). Three audience feedback teams will include representation from the general public and include a mix of age, ethnicity/race, gender, geography, and education (n=60 total participants). The fourth feedback team will include health care providers who represent a mix of years in practice, geography, gender, and specialty type (n=20 participants).
By recruiting a cohort of participants at the outset, in lieu of a new cohort of participants for each session, recruitment-related costs and participant recruitment screening burdens are minimized. Participants will be expected to participate in up to 8 qualitative sessions, each no longer that 60 minutes in length.
The professional market research firm will handle all aspects of participant recruitment. The FCR team and its contractor will never have access to participants’ personally identifiable information, such as last name, phone number, mailing address.
Custom Web Survey: The team will use a professional market research/survey panel firm to field two custom online surveys (one for consumers, one for HCPs) to their opt-in survey panels. The market research/survey panel firm maintains a large panel of hundreds of thousands of potential respondents. Surveys will be approximately 15 minutes in length each, fielded at baseline and at four “checkpoints” based on the creation and dissemination of new Combat COVID resources:
One survey with a nationally representative sample of the general public panel members (n = 2,000). Note: to ensure a nationally representative sample, the market research/survey panel firm relies on U.S. Census data to ensure respondents pulled from their opt-in panel comprise a nationally representative sample. At the outset, weighting dashboards are created and tracked in real-time to ensure demographic distributions are aligned with the population of interest.
One survey with healthcare providers panel members who directly treat COVID-19 positive patients (n = 300).
The professional market research/survey panel firm will handle all aspects of survey fielding. The FCR team and its contractor will never have access to respondents’ personally identifiable information, such as last name, phone number, mailing address.
Following the conclusion of each data collection activity, the contractor will submit topline summary presentations to the FCR Team, along with actionable recommendations for the development and/or enhancement of ongoing Combat COVID resources. For example, actionable recommendations may include one or more of the following:
Audience-tailored messages educating those at high-risk for contracting COVID-19 about the availability of COVID-19 treatment clinical trials
Ads (traditional and digital) with graphic treatments/images that resonate with audiences
New ideas for communication/education materials relevant to COVID-19 treatment clinical trials—including topics, formats (i.e., fact sheets, brochures), and outreach strategies
A.3 Use of Information Technology to Reduce Burden
Each of those modes will be deployed using online collaboration tools:
DATA COLLECTION STRATEGY 1 will consist of live discussions held via online video- or audio-conferencing (e.g., Zoom) or online bulletin boards for asynchronous discussions. Participants in virtual qualitative data collection efforts have been found to be no less and often more diverse in terms of geography, education level, and racial and ethnic identity than in-person data collection efforts.1,2 Although participants in virtual data collection tended to have higher overall internet use, virtual modes still successfully recruited participants reporting lower rates of internet use.3 Holding sessions remotely across target audiences helps to minimize participant burden overall (i.e., time it would take for a participant to drive to a research facility). For HCPs especially, the use of virtual data collection and asynchronous collection methods reduces the impact of participation on busy professional schedules.4
DATA COLLECTION STRATEGY 2 will recruit from an opt-in web survey research panel maintained by a market research/survey panel firm. This will allow users to participate on their own time using any device with a web browser. Pages are succinct and easy to navigate. Respondents only see questions that apply to them (i.e., a healthcare provider would not see the questions intended for a consumer).
A.4 Efforts to Identify Duplication
No similar data are gathered or maintained by the agency or available from other sources known to the agency. Data collected under this clearance will focus on ways to inform communications efforts to increase awareness of, knowledge of, and information seeking for COVID-19 clinical trials for treatment (and the associated combatcovid.hhs.gov website). Questions in the surveys about vaccination status, caregiver status, or trusted media sources are included only for their correlation with interest or concerns about participating in a clinical trial for COVID-19 treatment, and only to inform clinical trials for treatment-specific messages for sub-groups within target audiences (such as those who have not received a vaccine). No other effort is collecting data on this niche topic, including the ASPA COVID-19 Public Education Campaign Market Research effort (OMB Control Number 0990-0476), which conducts weekly pulse surveys to track key metrics (such as vaccine confidence, familiarity with and trust in HHS) for COVID prevention.
A.5 Impact on Small Businesses or Other Small Entities
All data collection efforts will be conducted with individuals, thus this will not impact small businesses.
A.6 Consequences of Collecting the Information Less Frequently
Collecting routine information from the general public and healthcare providers will ensure the federal government stays abreast of changing target audience needs as they relate to COVID-19 treatments and ACTIV clinical trials. This ensures needed resources are developed and broadly disseminated, thus maximizing the return on taxpayer dollars. (All data collection efforts, however, are voluntary, and all participants are free to respond, or not respond, to questions posed at any point during this endeavor.)
Combat COVID and the ACTIV trials, in particular, are designed to respond to the evolving nature of the COVID-19 pandemic. The ACTIV trials are designed to rapidly assess the effectiveness of new, promising COVID treatments, and the related Combat COVID messaging will need ongoing updates as new studies come online and studies that don’t show effectiveness are closed. As vaccination rates continue to increase, and the location and demographics of outbreaks change, the needs of the target audiences (and who or where those target audiences are) will also change.
In order to respond to these changes and to avoid over-burdening the public, the FCR will solicit audience feedback through qualitative sessions monthly, as any less frequently may negatively impact Combat COVID outreach strategies and message development.
Data collection used to measure changes over time – using the custom web survey – will be carried out every 3 months, to allow sufficient time for the messages to make an impact, prior to yielding results. If survey data are collected less frequently, Combat COVID message development will not be able to respond to new breakthroughs and information needs from different target audiences on a topic, and the messages will be less effective.
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
In response to the national public health emergency, and because the Federal COVID Response requires timely feedback on the Combat COVID initiative to adapt to the changing circumstances of the ongoing COVID-19 epidemic, we are requesting emergency clearance for this data collection. Thus, it will not go through a 60-day public comment period and will only undergo a 30-day public comment period after clearance has been granted. The team is also submitting an information collection request for full review in order to continue collecting data after the six-month Emergency Clearance approval period.
Emergency Clearance for this data request will allow the FCR team to begin rapid testing of messages their impact over the next six months. As this landscape changes, a full review and clearance will allow the team to keep testing new messages developed for new circumstances, and to show how knowledge of and attitudes toward clinical trials for COVID-19 therapies has changed over time. The full review and clearance will comply with the guidelines as laid out in 5 CFTR 1320.5.
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency
This is an emergency request; an emergency FRN was published on 5/27/20201 No. 86 FR 28633, pages 28633-28634.
With regard to consult with outside agencies, the FCR team is designed to coordinate the work between several agencies. Members of the team described the need for timely data collection to inform the communications strategy for Combat COVID, gave input on the research questions/topics, and reviewed the proposed data collection strategy to provide broad survey and in-depth data from target audiences of the initiative.
These team members included:
Lieutenant Colonel Avon Cornelius (HHS/Office of the Secretary/Immediate Office of the Secretary)
Colonel Brian Burk (HHS/Office of the Secretary/Immediate Office of the Secretary)
Christine Keuthe (HHS/Office of the Assistant Secretary for Preparedness and Response/Biomedical Advance Research and Development Authority)
A.9 Explanation of Any Payment of Gift to Respondents
Research has shown improved participation in qualitative interviews with adults in the general United States population when given a monetary incentive in comparison to a nonmonetary incentive or no incentive.5
Feedback team participants will receive a token of appreciate of $40 for each feedback team activity in which they participate (individual discussion, group discussion, or online bulletin board post prompt), up to a total of $320 (if a participant were to participate in all 8 activities). The proposed incentive amount has been shown to be effective by Federal agencies for minimizing attrition for qualitative research activities, offsetting the challenges for these audiences to participate. This amount is consistent or less than other amounts offered for similar research efforts (virtual qualitative research participation).6,7
Consumer respondents and HCP respondents will receive $3 and $20, respectively, for a complete web survey. The consumer incentive amount is consistent incentives found to increase response rates.8 The difference in incentives for HCPs consistent with studies showing higher response rates from HCP respondents with increased incentives, especially looking for a subset of HCPs who have treated patients with COVID-19.9
A.10 Assurance of Confidentiality Provided to Respondents
No personally identifiable information will be collected. All respondent information will be kept private to the extent permitted by law. Recruitment for the audience feedback teams and custom web survey will be done by a professional research firm who maintains an opt-in panel of respondents. For audience feedback teams, panel members will be contacted and screening using the scripts and screeners in Attachment 1 (for general public) and Attachment 2 (for healthcare providers). For the custom web survey, panel members will complete the screening as part of the survey questions in Attachment 6.
The FCR Team (including NIH, BARDA, CDC, FDA, and DoD) will not have access to any personally identifiable participant/respondent data for neither the audience feedback teams, nor the surveys, at any time during or after this data collection endeavor.
The NIH Office of the Director’s Privacy Policy Branch Chief & Senior Official for Privacy also reviewed the approach for protecting participants’ privacy/avoiding collecting any personal contact information and confirmed in writing that a Systems of Records Notice nor Privacy Act Memo will be needed for the endeavor.
A.11 Justification for Sensitive Questions
To identify which messages are clear, relevant, and actionable for different audiences, this data collection will ask questions about experience with COVID-19 prevention and treatment. Collecting demographic information is necessary to ensure that the messages are effective with members of groups that have historically been underrepresented in clinical trials, fulfilling NIH’s mandate for the inclusion of women and minority groups in all NIH-funded clinical research under the Public Health Service Act sec. 492B, 42 U.S.C. sec. 289a-2. Voluntary demographic information will be collected during fielding to ensure that feedback teams include demographically diverse perspectives and that survey respondents are representative of the national population. Other questions, such as marital status, household composition, or health insurance status have direct relevance to concerns about COVID infection and treatment, and this information will only be used to ensure messages are effective and relevant based on these factors.
Audience feedback team respondents and survey respondents are alerted of this prior to each data collection effort, and are assured that their responses are entirely voluntary. If a question makes a participant/respondent uncomfortable, they are free to “skip” it. Participants/respondents may also choose to stop participation at any time. For demographic reporting purposes, the team also collects information on age, sex, ethnicity and race, employment status, marital status, income range, state of residence, and education. These questions may be seen as sensitive, and thus are programmed to be voluntary and participants are free to decline to respond. Respondents are also assured of the private nature of participating in the research activities at the beginning of each survey, and also at the outset of any qualitative discussion. A person’s first name will never be connected to their demographic information, including race and ethnicity. The information will only ever be reported in summary form.
A.12.1 Estimated Burden Hours
In order to successfully recruit for audience feedback teams, it may be necessary to contact up to 120 members of the general public and 40 HCPs. Screening takes 5 minutes, for a total of 13.33 hours. Each audience feedback team activity (focus group, individual interview, or online bulletin board post) will take 60 minutes; the 60-minute estimation assumes 5 minutes for set-up or technical assistance for any activity. The 80 audience feedback team members (60 general public, 20 HCP) will participate in 8 activities over the project time period (June 2021 through December 2022) for a total of 640 hours. 2,000 general public survey respondents and 300 HCP respondents will complete each 15-minute survey. Surveys will be fielded at five times over the project period (one benchmark and four follow-up), for a total of 2,875 hours.
Table 12-1 below summarizes the estimate of total burden hours over the length data collection for the Combat COVID project, June 2021 through December 2022. Table 16-1, below, shows when those activities will take place over this period.
Table 12-1 Estimated Total Burden Hours
Type of Collection |
No. of Respondents |
No. of Responses per Respondent |
Time per Response (in hours) |
Total Hours |
Consumer Audience Feedback Team Screener (Attachment 1) |
120 |
1 |
5/60 |
10 |
HCP Audience Feedback Team Screener (Attachment 2) |
40 |
1 |
5/60 |
3 |
Consumer Audience Feedback Activity (Attachments 3 & 5) |
60 |
8 |
1 |
480 |
HCP Audience Feedback Activity (Attachments 4 & 5) |
20 |
8 |
1 |
160 |
Benchmark & Follow-Up Web Surveys – Consumer Audience (Attachment 6) |
2,000 |
5 |
15/60 |
2,500 |
Benchmark & Follow-Up Web Survey – HCP Audience (Attachment 6) |
300 |
5 |
15/60 |
375 |
Total |
2,540 |
12,300 |
…. |
3,528 |
A.12-2 TOTAL COST TO RESPONDENTS
A.12-2 Total Cost to the Respondents
Type of Respondents |
Total Annual Burden Hours |
Hourly Respondent Wage Rate* |
Respondent Cost |
General Public |
2,990 |
$25.72 |
$76,902.80 |
Health Care Providers
|
538 |
$40.21 |
$21,632.98 |
TOTAL |
3,528 |
|
$98,535.78 |
*The General Public and HCP hourly rates were obtained from https://www.bls.gov/oes/current/oes_nat.htm.
A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers
No other cost burden is expected.
A.14 Cost to the Federal Government
The costs in table 14-1 below represent the total cost over the 17-month time period. (Total percent effort for Federal Oversight is estimated not to exceed the equivalent of 5% of a 12-month salary period.)
Table 14-1 Costs to Federal Government
Staff |
Grade/Step |
Salary* |
% of Effort |
Fringe (if applicable) |
Total Cost to Gov’t |
Federal Oversight |
|
|
|
|
|
Lieutenant Colonel |
GS-14 Step 7 |
$147,034 |
5% |
|
$7,351.70 |
|
|
|
|
|
|
|
|
|
|
|
|
Contractor Cost |
|
|
|
|
$360,600.00 |
|
|
|
|
|
|
Travel |
|
|
|
|
$0 |
Other Cost |
|
|
|
|
$0 |
|
|
|
|
|
|
Total |
|
|
|
|
$367,951.70 |
*The salary in table above is cited from https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2021/DCB.pdf.
A.15 Explanation for Program Changes or Adjustments
Not applicable.
A.16 Plans for Tabulation and Publication and Project Time Schedule
Results from this data collection will not be published. Data will be used internally for the development and/or enhancement of ongoing Combat COVID resources, including audience-informed messages and traditional and digital ads relevant to educating the general population (and their loved ones) at risk for COVID-19 about clinical trials for treatment.
Table 16-1 below shows how the data collection activities will take place over the course of the FCR Combat COVID initiative; however, only the data collection from June 2021 through December 2021 (6 months) will occur under this emergency clearance. January 2022 through December 2022 data collection will occur under a separate full clearance package, upon OMB approval. These dates are given as approximations, expecting that communication strategies or new media/messages will need to be tested during that time period.
Table 16-1 FCR Data Collection Timeline
|
Jun – Aug 2021 |
Sept – Dec 2021 |
Jan – Mar 2022 |
Apr – Jun 2022 |
Jul – Sept 2022 |
Oct – Dec 2022 |
Benchmark Survey |
X |
|
|
|
|
|
Follow-up Survey 1 |
|
X |
|
|
|
|
Follow-up Survey 2 |
|
|
X |
|
|
|
Follow-up Survey 3 |
|
|
|
X |
|
|
Follow-up Survey 4 |
|
|
|
|
X |
|
Audience Feedback Team Recruitment |
X |
|
|
|
|
|
Audience Feedback Team Activities |
1 |
2 |
3, 4 |
5 |
6, 7 |
8 |
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate
We are not requesting an exemption to the display of the OMB Expiration date. All data collection instruments will display the OMB number and expiration date.
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions
This Emergency Clearance data collection request will comply with the requirements in 5 CFR 1320.9.
Attachments:
Attachment 1: Consumer Recruitment Screener
Attachment 2: HCP Recruitment Screener
Attachment 3: Consumer Audience Feedback Team Moderator’s Guide for Team Discussions and In-Depth Interviews
Attachment 4: HCP Audience Feedback Team Moderator’s Guide for Team Discussions and In-Depth Interviews
Attachment 5: Audience Feedback Team Sample Online Bulletin Board Prompt
Attachment 6: Custom Web Survey with Consumer and Provider Paths
1 Rupert D, Poehlman J, Hayes J, Ray S, Moultrie R. Virtual Versus In-Person Focus Groups: Comparison of Costs, Recruitment, and Participant Logistics. J Med Internet Res 2017;19(3):e80. DOI: 10.2196/jmir.6980
2 Namey E, Guest G, O’Regan A, Godwin CL, Taylor J, Martinez A. How Does Mode of Qualitative Data Collection Affect Data and Cost? Findings from a Quasi-experimental Study. Field Methods. 2020;32(1):58-74. doi:10.1177/1525822X19886839.
3 Rupert et al., 2017.
4 Daniels N, Gillen P, Casson K, Wilson I. STEER: Factors to Consider When Designing Online Focus Groups Using Audiovisual Technology in Health Research. International Journal of Qualitative Methods. January 2019. doi:10.1177/1609406919885786
5 Kelly, B., Margolis, M., McCormack, L., LeBaron, P. A., & Chowdhury, D. (2017). What Affects People’s Willingness to Participate in Qualitative Research? An Experimental Comparison of Five Incentives. Field Methods, 29(4), 333–350. https://doi.org/10.1177/1525822X17698958
6 Kelley et al., 2017.
7 See also: OMB Control No. 0910-0796, ICR Reference No: 202007-0910-005, entitled “Generic Clearance for the Collection of Qualitative Data on Tobacco Products and Communications”
8 Smith, M.G., Witte, M., Rocha, S. et al. Effectiveness of incentives and follow-up on increasing survey response rates and participation in field studies. BMC Med Res Methodol 19, 230 (2019). https://doi.org/10.1186/s12874-019-0868-8
9 Noel H, Huang AR. The Effect of Varying Incentive Amounts on Physician Survey Response. Evaluation & the Health Professions. 2019;42(1):71-81. doi:10.1177/0163278718809844
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Sarah Ward |
| File Modified | 0000-00-00 |
| File Created | 2021-11-11 |