Memo

Request for Approval under the “Generic Clearance for the Collection of Qualitative Feedback on Agency Service Delivery” (OMB Control Number 0990-0459)

TITLE OF INFORMATION COLLECTION:

HHS COVID-19 Public Education Campaign Creative Testing Focus Groups

PURPOSE:

The purpose of this project is to conduct focus groups to inform creation of behavior-change motivating advertisements for targeted audiences who are currently hesitant to receive a COVID-19 vaccination when available.

These focus groups will be conducted on behalf of the U.S. Department of Health and Human Services (HHS). As there has been inconsistent and mixed messaging surrounding COVID-19, it is imperative that messaging and advertisements disseminated to the public are scientifically accurate, evidence-based, and tested among target audiences. As such, the primary purpose of these focus groups will be to test creative concepts with participants. Focus groups will assess creative comprehension, resonance, relevance, and call to action clarity. They will allow the opportunity to unpack and explore ambivalence and ascertain the “why” underpinning attitudes and behaviors. These focus groups will also provide the opportunity to learn what resonates with audiences, why it makes an impact, and what could be better. Findings from each round of testing will be used to refine and enhance creative assets concepts. Ultimately, these findings will help inform the development of messages and materials, which, in turn, will be produced and disseminated to the general public and those disproportionately impacted by COVID-19.

Given the rapidly evolving nature of the COVID-19 pandemic, HHS is asking for an expedited review of this request.

DESCRIPTION OF RESPONDENTS:

For the HHS COVID-19 Public Education Campaign Creative Testing Focus Groups, 18 focus groups (with 4-6 participants per group) will be conducted remotely, among the following demographics:

• Nine focus groups with the general population (mixed gender) who generally believe in vaccines but may be hesitant about the novel COVID-19 vaccines, the “moveable middle”. Groups divided by age range/generation.

  • These groups do not include people who do not see general merit in vaccines and who would never get the COVID-19 vaccine.

• Three focus groups with Hispanic/Latino adults, mixed gender, divided by acculturation. Two groups will be Spanish-speaking

• Three focus groups with Black/African American adults, mixed gender, divided by age range/generation.

• Three focus groups with American Indian/Alaska Native adults, divided by gender, mixed ages.

TYPE OF COLLECTION:

[ ] Customer Comment Card/Complaint Form [ ] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software [ ] Small Discussion Group

[X ] Focus Group [ ] Other: ______________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the federal government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: Monica Vines, Campaign Manager for Research (Project Manager’s name)

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [ ] Yes [X] No

2. If Yes, is the information that will be collected included in records that are subject to the Privacy Act of 1974? [ ] Yes [ ] No

3. If Applicable, has a System or Records Notice been published? [ ] Yes [ ] No

Gifts or Payments:

Is an incentive (e.g., money, reimbursement of expenses or token of appreciation) provided to participants? [ X ] Yes [ ] No If Yes, please describe the incentive and provide a justification for the amount.

OMB guidance currently sets the incentive for in person focus groups at $75. Due to COVID-19, these focus groups must be virtual, which would typically warrant a lower incentive payment. However, HHS requests that participants in these focus groups receive a payment of $75 for participation in one 90-minute virtual focus group.

Providing incentives for recruitment of focus group participants is a critical component of planning a successful recruitment effort. Given the critical need to get consistent messaging out to the public as soon as possible, ensuring incentives are high enough will support successful recruitment of all participants, which is imperative especially for those groups disproportionately affected by COVID-19.

Although initially it may appear as a cost savings to have a lower incentive amount, in the long term it takes more resources to recruit respondents to attend a focus group with a lower incentive. For instance, with a lower incentive it may require 10 calls before securing a participant while a higher incentive may require only half of those calls. A higher incentive also reduces the number of no-shows, which also means we will have less focus group cancellations (we cannot run focus groups with less than 4 participants).

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Burden Hours
Nine focus groups with the general population	54	90 minutes per participant	81
Three focus groups with Hispanic/Latino adults	18	90 minutes per participant	27
Three focus groups with Black/African American adults	18	90 minutes per participant	27
Three focus groups with American Indian/Alaska Native adults	18	90 minutes per participant	27
Totals	108	90 minutes per participant	162

FEDERAL COST: The estimated annual cost to the federal government.

In total, we plan to conduct eight rounds of qualitative creative testing with each round comprised of 18 focus groups. Broken down, the price for one round of qualitative creative testing is $135,111.84 in labor costs and $46,139.22 in other direct costs. The cost of incentives is $10,800; this cost is not included in either of the aforementioned figures.

Included in these costs are:

Development of the focus group screener.

Development of the focus group discussion guide.

• Translation of materials into Spanish.

• IRB review.

• Participant recruitment.

• Participant incentives.

• Execution of the focus groups and moderation.

• Live interpretation of the two Spanish-speaking groups for those in the virtual back room.

• Transcription.

• Analysis and reporting (memo of key findings and recommendations).

• Quality control.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [X ] Yes [ ] No

If the answer is Yes, please provide a description of both below (or attach the sampling plan)? If the answer is No, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

For the HHS COVID-19 Public Education Campaign Creative Testing Focus Groups, 18 remote focus groups will be conducted with up to 108 participants per month, with 4–6 participants in each group. Vendors will recruit participants from their panel databases via invitation emails and a phone screener to participate in the groups.

Administration of the Survey

1. How will you collect the information? (Check all that apply)

[X] Web-based or other forms of social media.

[X ] Telephone.

[ ] In-person.

[ ] Mail.

[ ] Other, explain.

2. Will interviewers or facilitators be used? [X] Yes [ ] No

Please make sure that all survey materials, instructions, and scripts are submitted with the request.