ASPA COVID-19 Public Education Campaign Market Research_SupportingStatement Part B_012121_FINAL

ASPA COVID-19 Public Education Campaign Market Research_SupportingStatement Part B_012121_FINAL.docx

ASPA COVID-19 Public Education Campaign Market Research

OMB: 0990-0476

Document [docx]
Download: docx | pdf

Supporting Statement Part B

U.S. Department of Health and Human Services

ASPA COVID-19 Public Education Campaign Market Research


(OMB Control No. ####-New)



  1. Respondent Universe and Sampling Methods


Current Events Tracker (CET)

The CET will involve cross-sectional weekly surveys. A different sample of adults will be invited to participate in the survey each week. A total of 1,000 respondents are expected to complete the survey each week. The survey will be conducted each week and will track questions including COVID-19 vaccine uptake and intentions, perceptions of COVID-19 vaccines, attitudes toward institutions managing the coronavirus response, sources of information used for awareness, and preventive measures taken.


The population of interest for this study is U.S. adults, aged 18 and older. The survey will be conducted using an online probability-based survey panel, the Ipsos KnowledgePanel. Every week for two years, a sample of U.S. adults aged 18 and over will be selected from the Ipsos KnowledgePanel to participate in the Weekend Ipsos Omnibus survey, which will include the CET questions developed for this study. Ipsos fields their omnibus survey as a way to address the information needs of multiple organizations through a single survey, thus reducing respondents’ time and burden by having participants respond to just one set of questions instead of multiple surveys. The survey vehicle is designed to be flexible to accommodate many different clients. Often the questions fielded as part of the omnibus are topical in nature and could cover topics such as views of events in the news; however, the additional clients and survey questions included in the omnibus survey over the next two years is not known at this time. The CET questions are expected to comprise the majority of the questions in the survey from week to week.


The Ipsos KnowledgePanel contains approximately 60,000 panel members and is nationally representative. Members of the KnowledgePanel were invited to participate in the panel through address-based sampling (ABS) of U.S. households The KnowledgePanel includes hard-to-reach populations and households without landline telephones, without internet access, or whose dominant language is Spanish. Ipsos weights the entire KnowledgePanel according to benchmarks from the March 2020 U.S. Census Bureau Current Population Survey (CPS) so that the weighted distribution of the KnowledgePanel reflects the U.S. adult population. The geodemographic dimensions used for weighting the KnowledgePanel include, but are not limited to, the following dimensions: age, race/ethnicity, education census region, household income, homeownership status, metropolitan area, Hispanic origin, and language dominance (with respect to the Spanish language). 


Foundational Focus Groups

For the HHS COVID-19 Public Education Campaign Foundational Focus Group Discussions, up to eight rounds of 18 focus groups (with 4-6 participants per group) will be conducted remotely with adults who comprise the “movable middle,” those who are undecided about getting a COVID-19 vaccine but are open to the idea; the group excludes those who do not see merit in vaccines in general or who say they would never get the COVID-19 vaccine. These focus groups will be segmented based on participants’ race/ethnicity, age, and/or gender. For the first round, nine groups will be conducted with the general population (a mix of genders and race/ethnicities) segmented by age; three groups will be conducted with Black/African American adults (mixed gender) segmented by age; three groups will be conducted with Hispanic/Latinx adults (mixed gender) segmented by age and primary language; and three groups will be conducted with American Indian/Alaska Native adults, segmented by gender. Participants will be grouped based on their self-reported race and ethnicity. Participants for the American Indian/Alaska Native groups will be asked about their tribal enrollment and/or affiliation and will also be asked some questions about their cultural identity—these criteria are meant to ensure the participants recruited identify with their culture and align with the target audience for the Campaign.


A copy of the screener, which outlines the exact criteria that must be met for participants to qualify for the first round of focus groups, is provided. Specific demographic criteria used for recruiting or segmenting groups may vary slightly from round to round. ASPA will provide updated screeners to OMB for review.


Vendors will recruit participants from their panel databases via invitation emails and a phone screener to participate in the groups.


Copy Testing Surveys

The copy testing survey will involve cross-sectional surveys of 1,000 U.S. adults before each Wave of creative assets are aired. Up to eight waves of surveys will be fielded throughout the duration of the Campaign. The survey measures perceptions of COVID-19 vaccines, intended behaviors related to the COVID-19 vaccines, and the perceived effectiveness of advertisements viewed.


The population of interest for this study is U.S. adults, aged 18 and older, who have not yet received both doses of a COVID-19 vaccine. The survey will be conducted using an online, nonprobability-based, opt-in survey panel managed by Prodege. Prodege offers a panel of over 17 million members and maintains a database of over 1,000 demographic and behavioral attributers that can be used for sample targeting. Members of this panel agree to have their information stored and shared for the purposes of recruiting for research studies. For each Campaign Wave, Prodege will recruit 1,000 participants to complete the survey.



  1. Procedures for the Collection of Information


Statistical Methodology, Estimation, and Degree of Accuracy


Current Events Tracker (CET)

Each wave of the CET survey will be selected using a stratified random sample from Ipsos KnowledgePanel. Strata will be based on key demographic attributes. The sample sizes in each stratum will be determined based on the expected response rate for the strata with the goal of having a final dataset that is representative of the United States adult population.  

  

The CET will be weighted to account for the probability of inclusion in the KnowledgePanel, probability of selection into the CET, the probability of responding to the survey, and any nonresponse bias that may occur using this methodology. Weights will then be calibrated to Census benchmarks for key demographics, such as age, gender, region, education, ethnicity, and race.


The length of the survey is expected to be about 10 minutes to complete the questions asked on behalf of ASPA. Note that Ipsos uses only completed surveys.


Foundational Focus Groups

The Foundational Focus Groups are conducted using qualitative data gathering techniques that do not require statistical metrics of group accuracy. The focus group discussions will be analyzed for thematic findings and exemplar quotes or discussion to include in final reports.


Copy Testing Surveys

Participants in each wave of the copy testing survey will be selected using a stratified random sample from Prodege’s panel. Strata will be based on key demographic variables of interest in each wave. Recruitment efforts will be adjusted in each wave based on expected response rates and target demographics. The copy testing survey will be weighted according to Census benchmarks for key demographics.


Mode of Administration


Current Events Tracker (CET)

The CET will be a self-administered web survey. The web survey will be optimized to ensure that survey participants can complete the questionnaire on mobile devices (e.g., cell phones, tablets, etc.). For the CET, participants will be recruited to take the survey from the Ipsos KnowledgePanel every week. Ipsos will identify a sample of panel members to take the survey based on the demographic information they have provided Ipsos (e.g., age, race, ethnicity). The only eligibility requirement to complete the survey is age (18 years and older) and language preference (English). Since Ipsos includes this information as part of their panel members’ data, no screening of participants is necessary to determine eligibility for the survey. Approximately 2,857 panel members will be invited to take the survey each week to achieve the desired number of completed surveys. Ipsos currently fields an online omnibus survey (Weekend Omni) to 1,000 U.S. adults in their panel every week. Beginning in January 2021, the panel vendor will incorporate the CET into the KnowledgePanel WeekendOmni. During survey fielding, Ipsos will send email communications to invite and remind participants from its KnowledgePanel to participate in their Weekend Omnibus survey. Approximately 1,000 participants will complete each wave for up to 92 waves, totaling 92,000 participants over approximately two years. Reminders will be sent as needed to reach the targeted 1,000 completed surveys during each fielding. Fielding will last three days per week.


Foundational Focus Groups

Up to eight rounds of 18 foundational focus groups will be conducted remotely (with 4-6 participants per group). Professional vendors will recruit the necessary participants. The vendors will be responsible for recruiting participants for the groups from their databases and all participants will be required to go through a screener to ensure their eligibility for the focus groups. Focus groups will be conducted online via Zoom teleconference. Participants will be required to join the focus group discussion through the link provided to them by the recruiting facility using both audio and video throughout the session. To ensure smooth discussion during the group, any participants who are unable to connect with both audio and video will be provided their incentive but will not be permitted to participate in the group. A trained moderator will guide participants through the discussion, using a combination of probing questions and activities throughout the session. Once the session is complete, the respondents will be dismissed, and the online session will be closed. Each focus group will be recorded to help with notetaking and transcription of the discussion.


Copy Testing Surveys

The copy testing survey will be conducted using an online, nonprobability-based survey panel managed by Prodege. The online survey will be optimized to ensure that respondents can complete the questionnaire on mobile devices (e.g., cell phones, tablets). The online survey will use a combination of audio/video clips and static images to assess respondents’ aided Campaign recall. Respondents will be randomly assigned to an ad condition in which they are shown one subset of advertisements.


For each Campaign Wave, Prodege will recruit 1,000 participants to complete the survey. Prodege will recruit respondents from their panel database via invitation emails. Respondents will be targeted based on basic demographic variables, such as age, race and ethnicity, in order to achieve a diverse sample that closely reflects the distribution of these demographics within the population of interest. Respondents will also be screened by a set of requirements provided by the first few questions on the survey. These requirements include not working for the U.S. Federal government or media marketing fields, not having participated in a COVID-19 vaccine trial, and not having received two doses of the COVID-19 vaccine. Those who do not meet the eligibility criteria for the survey will be screened out.  


  1. Methods to Maximize Response Rates and Data Reliability Response Rates


Current Events Tracker (CET)

Ipsos recruits panel members for each study using email for contact. Sample members will receive an initial email invitation and email reminders will be sent to those who do not respond to the initial invitation. To encourage response, sample members who complete the survey will receive an incentive equivalent to $1 in the form of points that panel members can redeem for merchandise or gift cards. The panel recruitment response rate for Ipsos’ KnowledgePanel is 12% and the cooperation rate for each survey wave is expected to be between 35% and 38%. Ipsos will take eligibility for the survey into account when sampling panel members for this survey; therefore, additional screening questions are not necessary.


Foundational Focus Groups

Up to eight rounds of 18 foundational focus groups will be conducted remotely, with 4-6 participants per group, for a total of up to 864 participants over the course of the Campaign. Vendors will recruit 8 participants for each group to ensure enough participants attend each group. An incentive of $75 will be provided to those who participate in the focus groups to help ensure adequate recruitment of participants and recruit a more diverse group of participants for each session. Vendors who recruit participants for the groups will follow-up with participants prior to the start of their scheduled group to remind them of their participation and ensure adequate show rates for each group.


Copy Testing Surveys

The cooperation rate for each wave of the copy testing survey is expected to be between 30% and 35%. Potential participants will receive an invitation email from Prodege inviting them to take the survey. Participants who complete the survey will receive an inventive equivalent to $5.00 in the form of points that may be redeemed for gift cards to popular retailers (online and offline) or used for donations toward charities.



Issues of Non-Response


Current Events Tracker (CET)

The Ipsos Weekend Omnibus typically has a high cooperation rate (exceeding 35%) for a weekly survey. Factors impacting non-response will be assessed after each administration and addressed in survey weighting to reduce non-response bias.


Foundational Focus Groups

For each of eight rounds of focus group, vendors will recruit and enroll 144 potential subjects (8 per group) to be able to complete the focus groups among 72-108 subjects total (4–6 participants per group). Vendors will be instructed to recruit 8 potential subjects for each focus group with the goal of including 4-6 participants per group. This practice is intended to mitigate the impact of “no shows” and ensure a variety of perspectives are represented through the groups. Ultimately, the focus groups are meant to be a qualitative means of collecting data and are not designed to be representative of the entire population. Results and findings will be presented in a way that reflects the overall qualitative design and its associated limitations.


Copy Testing Surveys

Prodege estimates the copy testing survey will have a high cooperation rate (exceeding 30%) based on their work with similar video viewing studies. Factors impacting non-response will be assessed after each administration and addressed in survey weighting to reduce non-response bias. In later waves, estimates may be adjusted to factor in the rate of vaccination amongst the general population, which may impact nonresponse rates if our target sample (i.e., individuals who have not yet received a COVID-19 vaccine) changes.






Reliability of Data Collection


Current Events Tracker (CET)

The panel vendor will program the survey and will check all aspects of the programming. This will include verifying the text of the instructions, questions, and response options are correct and that the skip patterns and other aspects of survey logic are programmed appropriately.


After fielding, the vendor will eliminate sample members who provided poor quality data by speeding through the survey, “straight-lining,” or skipping most of the questions. FMG will perform an independent evaluation to confirm the decisions to exclude respondents due to poor data quality. After data collection, the vendor will provide FMG with the raw data as collected from the respondents as well as the final data after any necessary editing has been completed. FMG will have two analysts independently verify that data edits are correct, and that the final data are as expected and align with the questionnaire that was used for programming.


Foundational Focus Groups

Focus group vendors will screen potential participants based on the provided screener to ensure they meet the definition of the “movable middle” and any additional subgroup criteria (e.g., race/ethnicity).


Copy Testing Surveys

The panel vendor will program the survey and will check all aspects of the programming. This will include verifying the text of the instructions, questions, and response options are correct and that the skip patterns and other aspects of survey logic are programmed appropriately. FMG will independently verify these points as well to ensure the questionnaire is programmed correctly.


After fielding, the vendor will eliminate sample members who provided poor quality data by speeding through the survey, “straight-lining,” or skipping most of the questions. FMG will perform an independent evaluation to confirm the decisions to exclude respondents due to poor data quality. After data collection, the vendor will provide FMG with the raw data as collected from the respondents as well as the final data after any necessary editing has been completed. FMG will have two analysts independently verify that data edits are correct, and that the final data are as expected and align with the questionnaire that was used for programming.





Research Goals and Intended Use of Data Collected


Current Events Tracker (CET)

For the CET, our primary research goals include determining the following research questions:

  • Vaccine Uptake and Confidence: To what degree have audiences already received some or all doses of a COVID-19 vaccine? Do audiences intend to receive a COVID-19 vaccine, and what decisions and perceptions drive their intended behavior?

  • Vaccine Trust and Information: To what degree do audiences trust the safety and efficacy of the vaccine development process, as well as its public health and political messengers? To what degree do audiences trust individual vaccines, pharmaceutical companies, and/or vaccine development companies? To what degree does politicization or historical episodes affect the audiences’ perceptions of the COVID-19 vaccines? What other sources of information affect audiences’ degree of vaccine confidence?

  • HHS Trust/Campaign Brand Equity: To what extent do audiences trust HHS as a source of information on COVID-19? How does this compare to other information sources?

  • Knowledge: To what extent do audiences understand the vaccine development and testing process, vaccine effectiveness, and vaccine safety? To what extent can audiences accurately identify COVID-19 misinformation and rumors?


The CET will serve as a measure for both attitudes and behavior related to COVID-19 vaccines and preventive behaviors among the U. S. population as a whole and key subpopulations. Measures of COVID-19 vaccine uptake and confidence offer both a way to ascertain campaign impact in terms of change in targeted outcomes that are associated with fewer COVID-19 infections, as well as to point campaign leadership to areas in greater need for adjusting messaging content, delivery, or coverage. The CET provides a means of quickly examining the general public’s key attitude positions, health vulnerabilities, and trust in experts to help inform and guide the campaign. Those questions that are fielded consistently from week to week will be examined over time to see how the public’s attitudes are changing during the course of the campaign; however, this survey will not be used to determine campaign-attributable change in attitudes or behaviors. This survey is designed to understand how American’s information needs may change during the course of the campaign, so that messaging can adapt to meet these needs. The flexibility of the chosen omnibus survey will also allow ASPA to collect findings on emergent information needs with a very short lead time, ensuring the Campaign can best respond to current events and milestones without require extensive coordination.


A copy of the Wave 1 questionnaire is included as part of this submission (Attachment A). We expect that approximately half of the questions in the Wave 1 survey will be fielded throughout the duration of the Campaign. The other questions are designed to capture emergent research needs and are designed to be changed from week to week as needed. For example, it may be important to understand how current events, such as vaccine developments or rollout, impact vaccine confidence and uptake. These questions will be identified and added to the survey as relevant; we cannot anticipate these questions so must adapt the survey accordingly over time. ASPA will provide OMB with updated questionnaires for subsequent waves of fielding throughout the duration of the study.


Foundational Focus Groups

The purpose of the foundational focus groups is to explore key audiences’ perceptions and attitudes toward the COVID-19 pandemic and what preventive measures they take daily to avoid viral transmission. The success of the HHS COVID-19 Public Education Campaign depends on the ability to uncover and identify the target audiences’ understanding of the pandemic and their wants, needs, preferred messaging channel and source preferences when it comes to learning more about COVID-19. In addition, we want to better understand their usage of preventive measures to slow the spread of the virus.


A copy of the moderator guide for the first round of focus groups is included as an attachment to this submission (Attachment B). The moderator guide will change from round to round based on current events and the environment surrounding COVID-19 and vaccination; however, primary research questions will focus around participants’ experiences and perceptions with COVID-19 and their attitudes, experiences, and choices with respect to the COVID-19 vaccine. Questions will be adjusted based on how the pandemic has changed over time and how the vaccine rollout has progressed. ASPA will provide the final moderator guides for subsequent rounds of focus groups to OMB.


The primary purpose of these focus groups will be to inform the HHS COVID-19 Public Education Campaign strategy about audience risk knowledge, perceptions, current behaviors, and barriers and motivators to healthy behaviors (including getting the COVID-19 vaccine). Ultimately, these findings will also help inform the development of messages and materials, which, in turn, will be produced and disseminated to the general public.


Copy Testing Surveys

The primary purpose of the copy testing surveys is to test whether and to what extent the advertising is having the intended effect on the target audience’s attitudes and behavioral intentions. Key measures on the survey will include creative comprehension, resonance, relevance, and attitude and behavior change. Findings from the survey will be used internally to refine and enhance creative assets. Ultimately, these findings will help inform the development of messages and materials, which, in turn, will be disseminated to the general public.


A copy of the Wave 1 survey instrument is included in this submission (Attachment C). We anticipate the survey instrument will be largely unchanged from Wave to Wave, with the key exception of modifying the stimuli that are being tested. ASPA will provide OMB with updated survey instruments for subsequent rounds of copy testing that identify any changes from Wave to Wave.


4. Tests of Procedures or Methods


Current Events Tracker (CET)

Cognitive testing was conducted on the CET prior to finalizing the survey instrument. The cognitive testing was used to refine the survey questions and improve utility. A total of six individuals participated in the testing and debriefs for the CET instrument. Participants were recruited to ensure diversification by age, gender, race/ethnicity, and educational attainment.


Trained moderators conducted the cognitive interviews virtually using the Zoom platform. Participants used screen sharing to scroll through a mock online survey and were asked to use the think-aloud method to describe their thought process and response selection as they proceeded through the test survey. After completing the survey instrument, the interviewer asked participants a series of debriefing questions that focused on the survey overall and specific survey questions. The debriefing questions focused on items that participants expressed questions or confusion with during completion of the mock survey, as well as specific items and wording that the researchers had identified in advance.


All interviews were recorded to capture participant comments on the survey, and any questions that required revisions. The data collection did not focus on the actual survey instrument responses but rather on how well the questions were understood and how well they conveyed the research intentions. All identifiable information was kept private to the extent allowable by law and destroyed subsequent to the conclusion of all interviews. Revisions to the questionnaire were made based on the results of the survey instrument testing.


Foundational Focus Groups

Trained moderators will conduct the foundational focus groups using the Zoom platform. Moderators will participate in trainings provided by the research team prior to the focus groups in to review in detail project information and the focus group materials. They will also receive training in using the online Zoom platform to ensure that they can conduct the groups effectively with few issues. Dry runs of each round of focus groups will be conducted by the moderators to become familiar with the focus group protocol and activities, making sure that they are comfortable managing the amount of time dedicated to each topic in the guide.


Copy Testing Surveys

A subset of questions on the copy testing survey was tested prior to finalizing the survey instrument. The cognitive testing was used to refine the survey questions and improve utility. Trained moderators conducted the cognitive interviews virtually using the Zoom platform. Participants used screen sharing to scroll through a mock online survey and were asked to use the think-aloud method to describe their thought process and response selection as they proceeded through the test survey. After completing the survey instrument, the interviewer asked participants a series of debriefing questions that focused on the survey overall and specific survey questions. The debriefing questions focused on items that participants expressed questions or confusion with during completion of the mock survey, as well as specific items and wording that the researchers had identified in advance.


All interviews were recorded to capture participant comments on the survey, and any questions that required revisions. The data collection did not focus on the actual survey instrument responses but rather on how well the questions were understood and how well they conveyed the research intentions. All identifiable information was kept private to the extent allowable by law and destroyed subsequent to the conclusion of all interviews. Revisions to the questionnaire were made based on the results of the survey instrument testing.



5. Individuals Consulted on Statistical Methods


The following persons outside of HHS contributed to, reviewed, and/or approved the design, instrumentation and sampling plan:


Name

Affiliation

Telephone Number

Leah Hoffman, MPH

Fors Marsh Group

571-444-1694

Erik Lund

Fors Marsh Group

612-743-4480

Scott Vanderbilt

Fors Marsh Group

571-858-3821

Lindsey Strausser

Fors Marsh Group

571-858-3773



Attachment A: HHS COVID-19 Public Education Campaign Current Events Tracker Wave 1 Questionnaire

Attachment B: HHS COVID-19 Public Education Campaign Foundational Focus Groups Discussion Guide

Attachment C: HHS COVID-19 Public Education Campaign Copy Testing Survey Questionnaire

File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleSupporting Statement – Part B
SubjectHIP 2.0; Healthy Indiana Plan, CMS; Supporting Statement
AuthorCenters for Medicare & Medicaid Services
File Modified0000-00-00
File Created2021-10-07

© 2024 OMB.report | Privacy Policy