Message Testing for Tobacco Communication Activities (MTTCA)
Request for Revision (OMB No. 0920-0910, exp. 5/31/2021)
Supporting
Statement: Part B
Program Official/Contact
Michelle O’Hegarty, PhD
Office on Smoking and Health
National Center for Chronic Disease Prevention and Health Promotion
Centers for Disease Control and Prevention
4770 Buford Highway, NE MS F-79
Atlanta, Georgia 30341
770-488-5582
FAX:770-488-5939
Email: [email protected]
TABLE OF CONTENTS
B. STATISTICAL METHODS
Respondent Universe and Sampling Methods
Procedures for the Collection of Information
Methods to Maximize Response Rates and Deal with No response
Tests of Procedures or Methods to be Undertaken
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
LIST OF ATTACHMENTS
Attachment 1: Public Health Service Act
Attachment 1a: PPHF
Attachment 2: Federal Register Notice
Attachment 2a: Public Comment #1 on 60-day Federal Register Notice
Attachment 2b: Public Comment #2 on 60-day Federal Register Notice
Attachment 2c: CDC Response to Public Comments
Attachment 3: Overview of Planned Information Collections
Attachment 3a: Example Data Collection Instrument – Rough-Cut Testing Survey Screener
Attachment 3b: Example Data Collection Instrument – Rough-Cut Testing Survey
Attachment 3c: Example Data Collection Instrument – Creative Concept Testing Survey Screener
Attachment 3d: Example Data Collection Instrument – Creative Concept Testing Survey
Attachment 3e: Example Data Collection Instrument –Creative Concept Testing Focus Group Screener
Attachment 3f: Example Data Collection Instrument – Creative Concept Testing Focus Group Moderator’s Guide
Attachment 3g: Example Data Collection Instrument – Screenshots Message Platform Testing Screener
Attachment 3h: Example Data Collection Instrument – Screenshots Message Platform Testing Survey
Attachment 3i: Example Data Collection Instrument – Moderator’s Guide For Adults Who Smoke Cigarettes
Attachment 3j: Example Data Collection Instrument – Moderator’s Guide For Adults Who Smoke Cigarettes and Use E-cigarettes (Dual Use)
Attachment 4: Example Institutional Review Board Approval Letter
Attachment 5: Example Informed Consent Form
B.1. Respondent Universe and Sampling Methods
The Centers for Disease Control and Prevention (CDC) requests OMB approval for a three-year period, as a revision, to the Message Testing for Tobacco Communication Activities (MTTCA, OMB No. 0920-0910, exp. 5/31/2021) clearance. The purpose of this generic clearance is to collect information among people who smoke in regard to their attitudes and perceptions, and to pretest draft messages and materials for clarity, salience, appeal, and persuasiveness. Information collected will be used to develop tobacco-related health messages, including messages related to the National Tobacco Education Campaign (NTEC).
The primary NTEC audience is adults who smoke ages 18 through 54. The existing clearance also includes youth ages 13-17 years old. There are no proposed changes to the populations of interest from the last extension request to MTTCA. Respondents will not include members of the state or local government.
There is a need to continue to test prevention and cessation messages related to combustible tobacco products and expand the noncombustible products to include heated tobacco. As these information collections are considered part of formative work for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. A variety of qualitative and quantitative information collection strategies are supported through this mechanism, including in-depth interviews; in-person focus groups; online focus groups; in-person, or telephone interviews; and online surveys.
Qualitative: Qualitative methods are not intended to yield results that are statistically projectable or used to derive quantitative estimates. However, these methods allow tobacco communication messages and/or programs to be designed and marketed with specific audiences in mind (e.g., people who smoke ages 18-54 years old). In qualitative studies, quota sampling is often used to select a convenience sample of individuals who meet certain qualifications that reflect characteristics typical of the target audience. Response rate is not applicable to quota sampling because this type of sampling results in a nonprobability sample which is not representative of the population. In qualitative studies, respondents can be initially contacted by telephone, through the mail, or online; over-recruiting is done to compensate for non-respondents.
Quantitative: Where quantitative methods are used, information collection activities will target particular audiences with statistical sampling procedures employed to identify potential survey respondents. Online surveys for message testing will seek a convenience sample (e.g. through an online panel) that nonetheless has an acceptable degree of diversity in key demographic characteristics such as age, gender, education, and race/ethnicity. CDC does not intend to generate nationally representative results or precise estimates of population parameters using these surveys.
CDC will request Office of Management and Budget (OMB) approval for each data collection by submitting an Information Collection Request that describes project purpose, use, and methodology. Sampling methods will vary based on the target audiences and methodology for each round of data collection.
B.2. Procedures for the Collection of Information
The methodologies planned for use in this submission will follow standard state-of-the-art approaches adapted from marketing and communications research. Electronic data collection methods will be employed to minimize COVID-19 exposure risk. If interviews and focus groups are conducted in-person, additional precautions will be put into place to ensure staff, contractors, and participants are protected from COVID-19 exposure. Mitigation strategies could include a) enacting social distancing protocols (i.e. six feet of space between persons to eliminate close contact), b) requiring appropriate personal protective equipment (PPE) use by all parties (e.g. N95 or three-layer cotton masks), c) symptom screening prior to data collection, and d) disinfection of surfaces between sessions/interviews. The precautions may be revised to add or remove strategies as needed. Lastly, precautions may vary from site to site based on conditions on the ground and the timing of when activities are conducted. Project-specific plans will be described in each GENIC.
In this context, the term pretesting refers to testing messages, strategies, and communication materials before they are finalized and fielded. Questions in all pretesting methodologies include standard measures of communications that are designed to assess to what degree the message was successful in communicating information, including perceived effectiveness, main idea recall, comprehension, believability, personal relevance, motivation to quit smoking, and likes and dislikes. Additional questions may be added to address any specific concerns regarding a message or advertisement, such as how a respondent views a particular logo or caption related to the ad. The following describes examples of the types of methodologies to be used.
Screening: Brief screening questionnaires will be used to ensure that individuals who complete data collection activities meet approved eligibility criteria. The screening questions will confirm demographic and tobacco use or non-combustible product use, as appropriate to the specific data collection activity.
In-Depth Interviews: Individual in-depth interviews can be conducted in-person or online. These interviews are used to collect information from key informants to elicit attitudes and perceptions that offer insight into critical influences on individual’s belief structures or for pretesting message concepts, draft materials, and communication strategies. Individual in-depth interviews are ideal when the information in question requires in-depth probing or when individual, rather than group, responses are considered more appropriate. This methodology is appropriate for determining target audience attitudes, beliefs, and feelings.
Focus Groups: Focus groups or group interviews can also be conducted in-person or online. Focus groups are used to obtain insights into target audience perceptions, beliefs, and attitudes in the early stages of the communication process (i.e., in concept, strategy and materials development) and to understand how individuals discuss a message or advertisement with each other. Focus groups are usually composed of 8 - 12 people who have characteristics similar to the target audience or subgroups of the target audience. The groups are conducted by a professional moderator who keeps the session on track while allowing respondents to talk openly and spontaneously. The moderator uses a structured discussion outline. Focus groups are valuable in exploring consumer reactions to message concepts before additional resources are put into their development.
Surveys: Surveys can be conducted using self-administered online, telephone, or paper-and-pencil questionnaires. A sample of consenting participants is recruited from the target audience; respondents will be asked to respond to questions regarding their reactions to messaging or ads with respect to the main message, believability, comprehension, perceived effectiveness, and whether it would impact their behavioral intentions regarding tobacco use, non-combustible product use or secondhand smoke. This method of formative testing is not designed to generate nationally representative results or estimates of population parameters but rather is used to test whether messages or ads are credible, comprehensible and persuasive. Surveys can also be conducted for recruitment of real people to appear in advertisements used in future national tobacco education campaigns.
Online bulletin boards: Online bulletin boards may be used to aid in identifying messages and themes that resonate the most with audience members, as well as to identify gaps for the development of new creative materials. Bulletin board sessions typically run over a two- to three-day period, and respondents can respond at their convenience during the session, while a trained live moderator will monitor responses throughout the duration of the bulletin board.
For all methodologies, professionally recognized procedures will be followed in each information collection activity to ensure high quality data. Some examples of these procedures include the following:
Training sessions, supervision and monitoring will be conducted for all data collection efforts in which moderators/interviewers interact with respondents (e.g., focus groups).
Observers will monitor focus groups, focus group proceedings will be recorded, and online technical support will be made available, should the need arise.
An institutional review board approval will be requested, when needed.
Online survey procedures require that respondents answer or explicitly decline each presented question before moving forward. This maximizes the utility of data collected.
Data submitted through online surveys will be subjected to statistical validation techniques (such as disallowing out-of-range values).
All data collection and analysis will be performed in compliance with OMB, Privacy Act, and Protection of Human Subjects requirements.
B.3. Methods to Maximize Response Rates and Deal with Non-response
In the case of data collection activities that involve interviews or surveys conducted in-person, online, or on the telephone, several procedures can be used to increase responses. Below are a few examples.
Interviewers will participate in thorough training sessions. Training topics will include study objectives, question-by-question reviews of data collection instruments, strategies for engaging respondents, role playing, and techniques for fostering respondent cooperation and survey completion.
Experienced, highly trained staff will moderate all focus groups and bulletin board sessions in-person and online.
After an initial online qualifying screener, all respondents will be rescreened and confirmed via telephone. Prior rounds of data collection indicate that this is a very effective method for keeping high response and show rates for bulletin boards and focus groups.
For bulletin boards, if a respondent has not joined the discussion, they will typically be sent a follow-up email at the start of Day 2.
Potential respondents will be informed through a variety of methods, such as email messages, about the importance of these projects and encouraged to participate.
Content and layout of email invitations will be assessed for clarity to ensure robust response rates.
After the original invitation, respondents who have not completed the survey after 48 hours may receive a reminder invitation.
Creative and attractive graphics will be used to attract the attention of respondents, where relevant.
CDC, a credible agency that serves the public good, will be identified as the agency of record.
Participants will be given incentives, which can include cash, prepaid gift cards, or points that can be redeemed for other items, such as Amazon gift cards; they will be made aware of the type and amount of incentive prior to participating in the study.
B.4. Tests of Procedures or Methods to be Undertaken
This project involves the collection of quantitative and qualitative information. This request is a revision of a previously approved generic clearance package. Similar procedures have already been used to conduct testing through this clearance. Additionally, contractors or CDC staff may pretest or pilot test the instrument(s) and method of data collection, when time permits and if deemed necessary. Lessons from the pretest or pilot test will be identified, and changes, as necessary, will be incorporated into the instrument and method. Pretests and pilot tests will typically involve between 10 and 30 individuals unless OMB clearance is sought for a larger number.
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Primary responsibility for data collection and analysis through this clearance will be conducted by Qualtrics and Battelle, whose information is listed below.
Qualtrics
400 Qualtrics Drive
Provo, UT 84604
Phone (800) 340-9194
Email: [email protected]
Battelle
505 King Avenue
Columbus, Ohio 43201
Phone: (800) 201-2011
Email: [email protected]
Individuals consulted at CDC on the study design are listed below.
Centers
for Disease Control and Prevention 4770 Buford Highway, N.E MS F-79 Atlanta, GA 30341 |
||
Brian Armour |
Associate Director for Science |
Phone: 404.498.3014 E-mail: [email protected] |
Elizabeth Courtney-Long |
Health Scientist |
Phone: 404.498.0264 E-mail: [email protected] |
Diane Beistle |
Chief, Health Communication Branch |
Phone: 770.488.5066 E-mail: [email protected] |
Lauren Boyle-Estheimer |
Health Communication Specialist |
Phone: 404.498.2283 Email: [email protected] |
Lindsey McCarter |
Health Communication Specialist |
Phone: 770.488.4239 Email: [email protected] |
Michelle O’Hegarty |
Health Communication Specialist
|
Phone: 770.488.5582 E-mail: [email protected] |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Goggans, Julie (CDC/DDNID/NCCDPHP/OSH) (CTR) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |