Menthol-Flavored Tobacco Product Sales Restriction Policy Evaluation
[OMB No. if applicable. 0920-xxxx] [OMB expiration date]
Supporting Statement B
Program Official Contact
Kelly McAleer, MSPH
Health Scientist
National Center for Chronic Disease Prevention and Health Promotion
Centers for Disease Control and Prevention
P: 404-498-4840
5/17/2024
TABLE OF CONTENTS
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 3
B1. Respondent Universe and Sampling Methods 3
B2. Procedures for the Collection of Information 9
B3. Methods to Maximize Response Rates and Deal with No Response 10
B4. Tests of Procedures or Methods to be Undertaken 11
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 11
1. Public Health Service Act [42 U.S.C. 241]
2a. Survey recruitment language
2b. Focus group recruitment language
3a. Consent forms – Survey consent to participate form
3b. Consent forms – Focus group consent to participate form
4a. Survey eligibility screener
4b. Focus group eligibility screener
5a. Information collection instruments – Survey
5b. Information collection instruments – Focus group moderator guide
6. Screenshots of web-based data collection pages
7a. 60-day FRN publication
7b. 60-day FRN public comments and response
8a. Deloitte IRB letter of approval
8b. CDC IRB approval
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
This information collection will examine tobacco-related knowledge, attitudes, and behaviors among adults (aged ≥18 years) who have ever used tobacco products and reside in selected cities with and without menthol-flavored tobacco sales restriction policies. This information collection focuses on groups who have higher prevalence of commercial menthol tobacco product use including people who identify as lesbian, gay, bisexual, transgender, queer, or another sexual or gender minority (LGBTQ+) and racial and ethnic minority groups (persons identifying as American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific Islander, Hispanic or Latino, or multiracial). CDC optimized site selection to have a large enough sample size from each of these two groups to understand the impacts of tobacco sales restriction policies on these populations.
Data collection consists of a web panel survey (attachment 5a) and focus group discussions (attachment 5b). Screenshots of the web panel survey are in attachment 6. All participants must provide consent to participate in the survey or the focus group discussions, have access to an electronic device (e.g., smartphone, tablet, desktop computer), and be able to participate in a survey or focus group conducted in English. Participant consent forms are found in attachments 3a and 3b.
Power Analysis
CDC generated power estimates for this information collection by reviewing extant literature to determine effect size/differences between cities with and without implemented menthol sales restriction policies. WhileCDC did not find any studies that used the exact design and outcomes proposed for this evaluation, we were able to estimate the effect size of menthol tobacco product sales restrictions on similar outcomes from related studies. For example, one of the studies examined the impact of a country-wide policy in England (implemented in May 2020) that restricts the sale of menthol tobacco products.1 This study compared the prevalence of menthol tobacco use among youth in England to that of Canada, which had an existing ban, and the United States, which did not have a ban.2 The study found a nine percentage point reduction in menthol tobacco use prevalence among youth in England following the implementation of the ban (3.0% in August 2020 vs. 12.1% in February 2020; p < .001) compared to a stable menthol use rate in both Canada (2.3%–3.1%) and the United States (33.6%–36.9%) over the full study period (August 2018–August 2020). Another recent study looked at the effects of menthol tobacco product restrictions in Canada and found that after a ban was implemented, people who used menthol tobacco products daily or occasionally were more likely to report that they quit smoking than people who used non-menthol tobacco products (24% and 20% vs. 14%; p=0.014) or made a quit attempt (63% and 62% vs. 43%; p<0.001).3 Similarly, Fong et al., in another Canadian study found that after the implementation of a policy restricting the sale of menthol tobacco products, people who used menthol tobacco products were more likely than people who used non-menthol tobacco products to quit smoking when they had been smoking daily (difference=8.0%; 95% CI: 2.4%–13.7%, p=0.005) and among people who were smoking daily and not daily (difference=7.3%; 95%CI: 2.1%–12.5%, p=0.006).4
After analyzing the studies described above and considering the differences among the implemented policies (geographic region and study populations), CDC concluded that policies prohibiting menthol tobacco products can impact tobacco quit attempts of one day or longer. CDC estimated a conservative effect size for this impact to be seven percentage points. CDC based the power analysis on this metric (quit attempts for one day or longer) because tobacco cessation is an key outcome measure of this evaluation. CDC conducted a power analysis to determine the sample size needed to detect a seven-percentage point difference in the rate of quit attempts between cities with and without menthol restriction policies among the general population.
This information collection will compare data from all selected cities that have menthol sales restriction policies with all selected cities that do not have menthol sales restriction policies. CDC did not power the information collection for site-level comparisons within and between individual sites. However, it was powered to answer the research questions about tobacco product use behavior differences between communities with a policy restricting the sale of menthol tobacco and those without such policies. Based on the power analysis conducted, the sample size for the general population included in this information collection (n=5,366) is sufficient to detect a seven-percentage point or 12 percentage point difference with a power of .99 for a one- or two-sided test, respectively. While there is less power to detect differences between the subgroups of interest, the sample size should have sufficient representation among LGBTQ+ and racial and ethnic minority groups to detect a 12-percentage point difference with a power of at least .8 using a one-sided test.
Recruitment
CDC will recruit 5,366 individuals to complete the web panel survey and 75 individuals to participate in the focus group discussions for a total sample size of 5,441 participants. In the United States, about 40% of individuals who currently smoke use menthol cigarettes.5 As a result, CDC estimates that about 40% of the 5,366-person survey sample will use menthol tobacco products (n=2,147).
Qualtrics, a market research services firm, will recruit individuals who meet the evaluation’s inclusion criteria from pre-enrolled Qualtrics web panels. The inclusion criteria for the web panel survey are individuals that are over 18 years of age, live within the city limits of selected sites, and currently use or formerly used tobacco products. The inclusion criteria for the focus group discussions are individuals that are over 18 years of age, live within the city limits of a city with an implemented menthol tobacco sales restriction policy, have access to a device with video call capability, and currently use or formerly used tobacco products. Qualtrics will recruit individuals using their online web panel partnerships and will pre-target participants for the information collection’s respondent universe based on geographic and demographic data. For this collection, Qualtrics has pre-targeted potential participants to only include individuals who currently live in the cities selected for this evaluation (Atlanta, GA; Baltimore, MD; Boston, MA, Cleveland, OH; Columbus, OH; Detroit, MI; Milwaukee, WI; Oakland, CA; Portland, OR; San Diego, CA; the Twin Cities (Minneapolis, MN and St. Paul, MN); and Washington, DC).1 Qualtrics’s pre-targeting also includes demographic profile data to further limit potential participants to those who currently use or formerly used tobacco products.
Qualtrics will send recruitment messaging out to all individuals in the respondent universe. The recruitment language is found in attachments 2a and 2b. Qualtrics will invite eligible participants to partake in the web panel survey or the virtual focus group discussions. The eligibility screening questionnaires are found in attachments 4a and 4b. Those who agree to sign up for the information collection make up the sampling frame. Qualtrics estimates an 80% response rate for individuals in the sampling frame. Qualtrics will administer the survey to the entire sampling frame to maximize the final sample size for the information collection.
In addition to analyzing data for all respondents, CDC will stratify analyses by subgroup to identify if there are differences between individuals living in cities with and cities without menthol sales restriction policies among these subgroups. The first subgroup of interest are persons who currently or formerly used tobacco products who identify as being in a racial or ethnic minority population. The second subgroup are people who currently or formerly used tobacco products who identify as LGBTQ+.
The tables below show the respondent universe and sampling numbers for the general population, as well as for racial and ethnic minority and LGBTQ+ subgroups. In Table 1, the number of people in the respondent universe represents all respondents that could be included in this convenience sample. This includes individuals who meet the information collection’s inclusion criteria and are currently registered with a Qualtrics panel. While the Qualtrics panel aggregation will yield a nonprobability sample in each community, this approach allows for electronic, low-cost, and rapid community-level data collection which would otherwise be cost prohibitive and time intensive to field in 12 jurisdictions.
The sampling frame includes Qualtrics’s estimate for the number of people who would show an interest in participating in the survey but may not necessarily take or complete the survey. The desired number in the final sample represents the minimum number of people needed to meet the information collection’s sampling criteria and statistical power. Qualtrics estimates a 75–90% response rate (which varies by city) among the individuals in the sampling frame. CDC will distribute a survey to everyone in the sampling frame to maximize the response rate. Therefore, the sampling fraction is 100%.
Respondent Universe Tables
Table 1. General population (adults (aged ≥18 years) who currently use tobacco products or have ever used tobacco products pre-enrolled in Qualtrics panels) |
||||
City Groups |
No. in Respondent Universe |
Sampling Frame |
Desired No. in Final Sample |
Expected Response Rate1 |
With Policy |
5,600 |
3,005 |
2,438 |
81.1% |
Oakland |
611 |
316 |
250 |
79.1% |
Twin Cities |
463 |
227 |
194 |
85.5% |
Washington, DC |
1,264 |
713 |
568 |
79.7% |
Boston |
863 |
467 |
360 |
77.1% |
Portland |
1,044 |
576 |
524 |
91.0% |
Columbus |
822 |
442 |
331 |
74.9% |
San Diego |
533 |
264 |
211 |
79.9% |
Without Policy |
4,400 |
2,436 |
1,923 |
78.9% |
Detroit |
1,038 |
581 |
455 |
78.3% |
Atlanta |
1,339 |
764 |
610 |
79.8% |
Milwaukee |
699 |
378 |
325 |
86.0% |
Baltimore |
726 |
394 |
295 |
74.9% |
Cleveland |
598 |
319 |
238 |
74.6% |
1 Response rates presented are rounded estimates based on Qualtrics’s historical data. Actual response rates for the information collection may vary.
Table 2 outlines the number of respondents in the universe, the sampling frame, the desired number in the final sample, and the expected response rate for the racial and ethnic minority sub-population. Table 3 outlines these estimates for the LGBTQ+ sub-population. Qualtrics will recruit racial and ethnic minority and LGBTQ+ populations from the sampling frame for the general population shown in Table 1. As a result, the general population sampling frame becomes the respondent universe for these two populations. The sampling frame for the racial and ethnic minority sub-population is anyone from the respondent universe who identifies as a person from a racial or ethnic minority group. The sampling frame for the LGBTQ+ sub-population is anyone from the respondent universe who identifies as LGBTQ+. The predicted sampling frame for the racial and ethnic minority group respondent universe is based on Qualtrics’s estimate of the percent of their web panel population that identifies as a member of a racial or ethnic minority group. Similarly, the predicted sampling frame for the LGBTQ+ respondent universe is based on Qualtrics’s estimate of the percent of their web panel population that identifies as a member of the LGBTQ+ community. Like the general population sample, CDC will survey every individual in the racial and ethnic group and LGBTQ+ population sampling frames to maximize the sample size for these populations. Therefore, the sampling fractions are 100%.
Table 2. Racial and Ethnic Minority groups (adults (aged ≥18 years) who currently use tobacco products or have ever used tobacco products pre-enrolled in Qualtrics panels) |
||||
City |
No. in Respondent Universe |
Sampling Frame |
Desired No. in Final Sample |
Expected Response Rate1 |
With Policy |
3,005 |
398 |
316 |
79.4% |
Oakland |
316 |
53 |
42 |
79.2% |
Twin Cities |
227 |
27 |
21 |
77.8% |
Washington, DC |
713 |
120 |
96 |
80.0% |
Boston |
467 |
68 |
54 |
79.4% |
Portland |
576 |
22 |
17 |
77.3% |
Columbus |
442 |
64 |
51 |
79.7% |
San Diego |
264 |
44 |
35 |
79.5% |
Without Policy |
2,436 |
374 |
298 |
79.7% |
Detroit |
581 |
102 |
81 |
79.4% |
Atlanta |
764 |
123 |
98 |
79.7% |
Milwaukee |
378 |
40 |
32 |
80.0% |
Baltimore |
394 |
59 |
47 |
79.7% |
Cleveland |
319 |
50 |
40 |
80.0% |
1 Response rates presented are rounded estimates based on Qualtrics’s historical data. Actual response rates for the information collection may vary.
Table 3. LGBTQ+ (adults (aged ≥18 years) who currently use tobacco products or have ever used tobacco products pre-enrolled in Qualtrics panels) |
||||
City |
No. in Respondent Universe |
Sampling Frame |
Desired No. in Final Sample |
Expected Response Rate1 |
With Policy |
3,005 |
321 |
255 |
79.4% |
Oakland |
316 |
30 |
24 |
80.0% |
Twin Cities |
227 |
15 |
12 |
80.0% |
Washington, DC |
713 |
62 |
49 |
79.0% |
Boston |
467 |
67 |
53 |
79.1% |
Portland |
576 |
44 |
35 |
79.5% |
Columbus |
442 |
75 |
60 |
80.0% |
San Diego |
264 |
28 |
22 |
78.6% |
Without Policy |
2,436 |
266 |
213 |
80.1% |
Detroit |
581 |
56 |
45 |
80.4% |
Atlanta |
764 |
70 |
56 |
80.0% |
Milwaukee |
378 |
26 |
21 |
80.8% |
Baltimore |
394 |
65 |
52 |
80.0% |
Cleveland |
319 |
49 |
39 |
79.6% |
1 Percentages are rounded to the nearest whole number. Response rates presented are rounded estimates based on Qualtrics’s historical data. Actual response rates for the information collection may vary.
CDC will use the Qualtrics platform to administer and conduct the web panel survey, the focus group discussion screening questionnaires, and the consent forms. CDC will conduct the focus group discussions through Zoom, a video conferencing software. CDC will use the Qualtrics platform to gather data from screening questionnaires, consent forms, and surveys. They will use Zoom to collect audio and video data from focus groups. Trained evaluators from Deloitte Consulting LLP will conduct the focus groups.
Respondents will respond to screening questions for the web panel survey to confirm they meet the inclusion criteria. Potential focus group participants will also answer screening questions to confirm eligibility.
There is one consent form for the Community Web Panel Survey and another for the Community Focus Group. The consents forms will introduce and describe the purpose of the information collection to all participants and ask for participant consent prior to data collection. Only individuals who meet the inclusion criteria, based on the screening questionnaires, will receive consent forms.
Survey data collection will occur at one point in time. Participants will complete the survey screening questionnaire and will immediately receive the Community Web Panel Survey. Focus group data collection will occur at two points in time. Participants will complete the focus group screening questionnaire and then participate in the focus group at a later time. The survey questionnaire will take approximately 15 minutes, and the focus group discussions will take about 60 minutes. Both activities are anonymous. CDC will not capture participants' personal identifiable information (PII) or protected health information (PHI). The survey questionnaire consists of 76 questions. However, no participant will be asked all the survey questions due to survey logic that tailors presented questions based on previous survey responses. The focus group discussion interview guide has 13 questions.
CDC will use Qualtrics’ web panel recruitment services to recruit individuals for the web panel survey and focus group discussions. Qualtrics has an online panel distribution feature that allows the routing of data collection tools to a targeted population that has already consented and agreed to receive regular invitations to participate in survey research on a wide range of topics. Qualtrics also aggregates many online panel networks to offer sponsored surveys to a large pool of panel participants. To maximize response rates, Qualtrics offers survey respondents an incentive in the form of award points after completing the survey. The number of award points received may vary depending on the compensation arrangement of the specific Qualtrics panel. On average, the award points are equivalent to an estimated $10 per survey response, with a maximum of $15 per survey response. Qualtrics will email a $45 Visa gift card to focus group participants upon completion of the focus group discussion.
The projected response rate for this sample is 80%. Qualtrics’s sample feasibility estimates assume a non-response and termination rate of 20% due to non-qualification based on survey screening requirements. To ensure a proper sample size for this information collection, Qualtrics will continue to recruit individuals from web panels until responses meet the sampling threshold needed for the collection. It is estimated that recruitment and data collection will take up to three months total for the web panel survey and focus group discussions. A completed response represents a panelist completing and submitting the web panel survey or participating in the focus group discussion throughout the duration of the moderated session.
CDC reviewed the data collection procedures and tools used in this information collection, and Solutions Institutional Review Board (IRB) (Deloitte's IRB) approved the tools and procedures. Deloitte’s IRB letter of approval is in attachment 8.
The web panel survey leverages validated items from existing national surveys. Before finalizing the web panel survey and focus group guide, individuals in CDC’s Office on Smoking and Health (OSH) piloted each data collection tool to verify and improve question clarity, as well as to beta-test the technical functionality of the web-based survey. Nine OSH staff participated in piloting the web panel survey using the Qualtrics platform (the platform that will host the survey instrument during the data collection period). Six OSH staff participated in piloting the focus group discussion over Zoom (the platform that will host focus group sessions during the data collection period).
CDC provided the participants piloting the survey with the opportunity to provide feedback within the Qualtrics platform following each survey question. CDC provided the participants piloting the focus group with the opportunity to provide input over Zoom and via email following the conclusion of the focus group discussion pilot. CDC also provided pilot volunteers for both the survey and focus group with an instruction document before conducting the pilot test, which included prompts to consider when sharing feedback. Deloitte conducted a final pilot test of the final survey with 9 CDC staff. This final pilot test helped CDC determine an estimate of the time burden for the survey.
The individuals in the table below are responsible for designing and managing the Menthol-Flavored Tobacco Products Policy Evaluation data collection tools. The Deloitte Consulting staff listed in the table below are also responsible for data collection and analysis following OMB approval.
Name |
Contact Information |
Affiliation |
Role |
Katherine Moran |
CDC, Office on Smoking and Health Evaluation Team |
Designed the data collection tools and will analyze the data |
|
Kelly McAleer |
CDC, Office on Smoking and Health Evaluation Team |
Designed the data collection tools and will analyze the data |
|
Briana Oliver |
CDC, Office on Smoking and Health Evaluation Team |
Designed the data collection tools and will analyze the data |
|
Kevin Caron |
CDC, Office on Smoking and Health Evaluation Team |
Will analyze the data |
|
Inara Valliani |
CDC, Office on Smoking and Health Evaluation Team |
Designed the data collection tools and will analyze the data |
|
Phil Rosenbaum |
CDC, Office on Smoking and Health Evaluation Team |
Will analyze the data |
|
Cathy Lesesne |
Principle Investigator, Deloitte Consulting, Menthol-Flavored Tobacco Product Policy Evaluation Team |
Designed the data collection tools, will collect the data, and will analyze the data |
CDC consulted the two individuals listed in the table below on the statistical aspects of the data collection tools. These individuals will also consult on data analysis and interpretation of the evaluation findings.
Name |
Contact Information |
Affiliation |
Role |
Xu Wang |
CDC, Office on Smoking and Health Research Team |
Consulted on statistical aspects of the design, will consult on quantitative data analysis and interpretation |
|
Iris Alcantara |
CDC, Office on Smoking and Health Research Team |
Consulted on statistical aspects of the design, will consult on focus group data analysis and interpretation |
REFERENCES
1 The state of Ohio recently passed a preemption law overruling Columbus’s menthol tobacco product sales restriction policy. As a result, Columbus may be replaced in the final evaluation. For any other city with a newer menthol tobacco product sales restriction policy, if court litigation results in removal of the policy prior to implementation of the protocol, CDC will replace the city with a comparable city that has an established menthol tobacco product sales restriction policy.
1 East, Katherine A. “Outcomes of the Menthol Cigarette Ban in England, 2018-2020.” JAMA Network Open, JAMA Network, 3 May 2022, jamanetwork.com/journals/jamanetworkopen/fullarticle/2791805.
2 East, Katherine A. “Outcomes of the Menthol Cigarette Ban in England, 2018-2020.” JAMA Network Open, JAMA Network, 3 May 2022, jamanetwork.com/journals/jamanetworkopen/fullarticle/2791805.
3 Chaiton, Michael O, et al. “Ban on Menthol-Flavoured Tobacco Products Predicts Cigarette Cessation at 1 Year: A Population Cohort Study.” Tobacco Control, BMJ Publishing Group Ltd, 1 May 2020, tobaccocontrol.bmj.com/content/29/3/341.
4 Fong, Geoffrey T, et al. “Impact of Canada’s Menthol Cigarette Ban on Quitting among Menthol Smokers: Pooled Analysis of Pre-Post Evaluation from the ITC Project and the Ontario Menthol Ban Study and Projections of Impact in the USA.” Tobacco Control, U.S. National Library of Medicine, 28 April 2022, pubmed.ncbi.nlm.nih.gov/35483720/.
5 “Menthol: Facts, Stats and Regulations.” Truth Initiative, 22 Apr. 2022, truthinitiative.org/research-resources/traditional-tobacco-products/menthol-facts-stats-and-regulations.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |