U.S. Consumer Product Safety Commission (CPSC)
Volume I
Supporting Statement
Portable Electric Heater (PEH)
2023 Standard-Hybrid Focus Group Study
OMB# XXXX-XXXX
March 2023
1) Submittal-Related Information 3
2) Background And Study Rationale 3
3) Recruitment And Data Collection 3
4) Consultations Outside the Agency 7
5) Justification For Sensitive Questions 8
7) Assurance of Confidentiality 8
8) Estimate of Hourly Burden 9
9) Cost To Federal Government 10
This material is being submitted under OMB Control No.: XXXX-XXXX.
The title of the information collection is: Portable Electrical Heater Focus Groups.
According to the US Consumer Product Safety Commission (CPSC), an estimated annual average of 1,400 fires can be attributed to the use of portable electric heaters between 2016 and 2018.1 This risk is likely disproportionately shared. For example, FEMA and NFPA, in their reports “Fire Risk in 2019” and “Poverty and the Risk of Fire ,” respectively, identified multiple factors that contribute to fire deaths rates higher than the national averages, including race, age, gender, and location. More specifically in the reports, fire death rates are higher with African Americans, people living in poverty, adults ages 55 or older, young children, those with less formal education, and those who live in rural areas.2,3 Furthermore, the reports state that many of the states with higher percentages of those at risk of injury or death due to fire are in the Midwest and South. Higher rates of death due to fire in southern states may be due to in part the intermittent need for heating and resulting higher rates of using of portable heating devices as their primary heat.
The Portable Electrical Heater Focus Groups will provide qualitative research on consumer perception, behavior, and experience with the purchase and use of indoor portable electrical heaters (PEH). Findings from the qualitative research will be documented in a final report that will include an analysis of the data collected regarding the safety and risks of PEHs to consumers and the public, and it will include recommendations and strategies to enhance communication to consumers on the safety and risks of these products. Findings from the study are not intended to be nationally representative.
No statistical methods are identified because the focus groups are qualitative in nature and are not representative of the population.
For this effort, EurekaFacts will recruit a diverse cross-section of PEH users from the EurekaFacts participant database. The EurekaFacts participant database includes tens of thousands of individuals nation-wide and is constantly refreshed and updated through independent participant outreach methods in both English and Spanish. These efforts ensure that the research participants recruited by EurekaFacts are not “professional research respondents,” instead they are individuals with limited to no experience with qualitative research who can provide fresh and actionable information for stakeholders.
The recruitment strategy integrates multiple outreach/contact methods and resources, such as Internet ads, individual emails, telephone recruiting, and social media recruiting. Using multiple outreach methods helps to ensure we reach sufficient numbers of the demographics of interest (see Table 1 below). EurekaFacts will inform CPSC when recruitment commences and will provide updates on screening and recruitment numbers on a weekly basis throughout the recruitment period.
EurekaFacts will recruit 78-84 participants, specifically Black, Indigenous, or persons of color (BIPOC), Hispanic (Bilingual), Parent or legal guardian with young children (age < 6), Older Adults (age >55), and Low-income communities. Furthermore, efforts will focus on recruiting individuals from the Midwest and Southern states, supplemented, as needed, by a national outreach.
Table 1. Demographic Recruitment Target
Participant Demographics |
Total Screened |
Participant Total |
BIPOC |
150 |
16 |
Hispanic (Bilingual) |
150 |
16 |
Parent or legal guardian with young children |
150 |
10 |
Older Adults (age >55) |
150 |
10 |
Low income |
150 |
10 |
General Population |
150 |
16 |
Total |
900 |
78 |
The plan is to over-recruit to account for cancellations and no-shows. In order to successfully recruit 78-84 focus group participants from diverse socio-demographic backgrounds, EurekaFacts will target 1300 participants for recruitment and 650 screening. Note that these targets are subject to adjustment and are for planning purposes only.
All focus groups will be held online via the Zoom meeting platform.
Data Collection Process
EurekaFacts will utilize both the traditional focus group methodology as well as a hybrid cognitive interviewing and focus group approach. The hybrid approach helps to control for any distortion or bias that may arise due to participants not wanting to appear unaware about PEH warnings and precautions. Using this proven approach, the evaluation focuses on a few critical processes that are well-correlated with behavioral compliance, such as comprehension, recall, and effect on perceived risk of injury or harm. This comprehensive approach will be conducted within a study design involving a hybrid evaluation session that incorporates two different methods: cognitive or contextual interviews and focus group discussion. These two methods will achieve two different research goals:
Knowledge and Practices: Cognitive interviews will explore consumer understanding and perceptions of risks involved in using PEHs, usage patterns, and consumer knowledge about the maintenance and safety of PEHs. The number and type of PEHs owned by participants will also be identified.
Decision Making: Focus group discussions will further examine past experiences with PEHs, including any incidents or problems with PEHs; explore decision making processes for purchasing a PEH; and help to generate ideas for caution or warning communications that meet the needs of different audiences.
While contextual interviews provide insight into the setting and the environment in which the PEH is being used, the cognitive interviewing method is a common technique used to test comprehension of written/digital communication materials, assessments, general documentation, and surveys. This method allows the interviewer to explore how respondents understand, process and act upon information presented in the materials, targeting the same cognitive processes involved in processing warnings/cautions: comprehension, recall, and judgment. Thus, cognitive interviewing is particularly appropriate for evaluating decision making processes, including behavioral compliance with instructions and cautions. This method was proven reliable and useful during assessments of warning effectiveness in several studies.4
The hybrid approach takes no longer than a typical focus group session, and therefore does not result in any additional burden on the respondent. However, it offers much richer and multi-dimensional data. Past research conducted by EurekaFacts staff compared the effectiveness of cognitive interviewing versus focus groups, and demonstrated that cognitive interviews may yielded more detailed, relevant, and useful data, particularly in terms of revealing participant lack of knowledge on a topic and participant acknowledgement of failing to review warning or caution messages. However, focus groups were found to be better at generating creative ideas for messaging. Thus, conducting both types would provide a broad range of information and responses on the topic.5
To conduct the evaluation of consumers’ perceptions of risk and decision-making processes related to the purchase and use of PEHs, EurekaFacts will conduct online focus groups utilizing the traditional/standard focus group methodology coupled with online sessions utilizing the hybrid methodology, for a total of 12 online focus groups, with 78-84 participants (maximum 84). Best practices indicate that the ideal size of an online focus group is 5-8 participants, in order to give time for each participant to speak. Furthermore, the hybrid data collection approach will consist of two parts: The first part will consist of concurrent short one-on-one structured cognitive interviews that will evaluate participants’ knowledge and practices (30-35 minutes); the second part will follow traditional focus group methodology and explore the participants' decision-making processes as a group (45-50 minutes), as shown in Table 2 below. All focus groups, regardless of methodology, will last no more than 90 minutes.
The focus group sessions will be organized as follows:
Three standard focus groups with 5-9 participants each (approx. n=24), focusing on knowledge and practices
Three standard focus groups with 5-9 participants each (approx. n=24), focusing on decision –making, including the influence of caution warnings, and generating ideas for caution messaging
Six hybrid sessions, with five (5) participants each (n=30), focusing on knowledge and practices as well as decision making processes, including the influence of caution warnings, and generating ideas for caution messaging.
The hybrid sessions have fewer participants in each session to compensate for the time while conducting the concurrent cognitive interviews.
Table 2. Organization of Focus Group Methodology
Methodology |
Knowledge and Practices |
Decision Making |
Number of Standard Online Focus Group Sessions |
3 Focus Groups (n=8 each) |
3 Focus Groups (n=8 each) |
Number of Online Hybrid Groups: Cognitive Interviews and Focus Groups |
6 Hybrid groups (n=5 each) |
|
Part 1: Cognitive Interviews |
Part 2: Focus Group |
|
This hybrid approach will allow a reliable and comprehensive evaluation of perceptions of the risks associated with the use of PEHs and to control for any distortion or bias that may arise due to participants not wanting to appear unaware of the risks associated with PEHs. Focus groups are the best method to brainstorm and generate creative ideas for messaging that can meet expectations and needs of different customers or users; however, it is well documented that a group discussion may unnaturally influence some participants’ opinions and is not conductive to honest disclosure of opinions due to social desirability bias. Hence, the benefit of conducting one-on-one cognitive interviews in addition to the focus groups.
For evaluations in which there is a need to capture and explore participant understanding and knowledge of concepts, such as risk or warning communication, cognitive interviews are a more reliable research approach. In-depth interviews are especially conducive for exploration of sensitive topics. In a one-on-one interview, participants are more likely to admit that they have difficulties understanding a concept or warning message and will more truthfully indicate their expected level of compliance with a warning message than they will in a group setting. Also, the one-on-one interview format allows for in-depth exploration of issues with perception, comprehension, and risk judgment of communication materials and, therefore, provides a more comprehensive evaluation of participant’s level of general knowledge about PEHs and their understanding of risk, behavioral practices, and messaging.
Furthermore, EurekaFacts has found that participants may be more open addressing sensitive topics when they are in homogenous groups6,7. As a result, the focus groups for this study will be organized by the demographics of interest: BIPOC, Latino/Latina (bilingual), Parents, Older Adults (>55), and Low Income. One additional group will be scheduled for the general population of current PEH owners (owned a minimum of 7 years/used PEH within the past 24 months).
The table below presents the breakout of the focus groups in more detail regarding which demographic groups participates in a standard or hybrid focus group based on the assessment of groups that will benefit the most from the hybrid setting.
Table 3. Sample Size for Focus Group Breakdown
Focus Group Number |
Population |
Approach (Standard or Hybrid) |
Number of Participants |
1a |
BIPOC |
Standard |
8 |
1b |
BIPOC |
Standard |
8 |
2a |
Hispanic (bilingual) |
Standard |
8 |
2b |
Hispanic (bilingual) |
Standard |
8 |
3a |
Parents or legal guardians with young children |
Hybrid |
5 |
3b |
Parents or legal guardians with young children |
Hybrid |
5 |
4a |
Older Adults |
Hybrid |
5 |
4b |
Older Adults |
Hybrid |
5 |
5a |
Low Income |
Hybrid |
5 |
5b |
Low Income |
Hybrid |
5 |
6a |
General Population |
Standard |
8 |
6b |
General Population |
Standard |
8 |
|
TOTAL |
78 |
|
In order to successfully garner participation from 78-84 focus group participants from diverse socio-demographic backgrounds, EurekaFacts plans to hold 6 focus groups, with 5-9 participants in each.
Note: EurekaFacts will over-schedule participants for each group as some participants may not show up on the day of the focus group session.
EurekaFacts, LLC is located in Rockville, Maryland. It is an established for-profit research and consulting firm, offering facilities, tools, and staff to collect and analyze both qualitative and quantitative data. EurekaFacts is working as a subcontractor to CPSC to recruit participants and collect qualitative data. In addition, EurekaFacts staff will facilitate the focus group sessions and provide a report.
The collection is voluntary.
The collection is low burden for respondents and low-cost for the federal government.
The collection is non-controversial and does not raise issues of concern to other federal agencies.
Information gathered will not be used to substantially inform influential policy decisions.
The collection is targeted to the solicitation of opinions from respondents.
COVID-19 Procedures:
EurekaFacts, LLC has safety protocols in place which aim to protect the safety of their employees, participants, and their families. EurekaFacts protocol follows CDC guidance for COVID-19. Safety strategies used to protect EurekaFacts staff working in the office include use of hand sanitizer, wearing masks, and socially distancing when appropriate. The focus group will be conducted virtually.
The Portable Electric Heater Focus Groups will not pose any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, or matters commonly considered private.
To help ensure participation and to thank participants for their time and effort, EurekaFacts will provide an e-gift card from a major credit card company. Each participant will receive a $100 virtual gift card via email. The incentive provided will encourage participation in the focus groups. In marketing and social science research, providing participant incentives is a well-established and accepted standard of practice. The incentive is paid for a number of reasons. Paying an incentive demonstrates that people’s time is valuable. The incentive can help offset the opportunity costs of participants that have competing obligations. Additionally, incentives help ensure that we are able to recruit respondents from a variety of backgrounds, specifically those in the targeted sub-populations, and that participants are available on the stipulated date and time of the virtual focus group.
Furthermore, a review of the literature reveals an incentive rate of $50 for a 30-minute interview,8 a range from $50 to several hundred dollars for focus groups,9 and a rate of $75 for a 90 minute focus group with general consumers.10 Another source cites an average incentive rate of about $80 for focus groups with general consumers.11,12 The sample needed for the current project is considered a “hard to reach” population; therefore, providing the slightly higher rate of $100 for a 90 minute session is necessary to ensure response of enough qualified participants to achieve the numbers required by the contract.
The e-gift card will be sent to the email address of the respondent from Virtual Incentives (www.virtualincentives.com/products/global-egiftcards/) after their participation in the focus group.
Focus group respondents will be informed that their participation is voluntary and that none of the participants will be specfically identified to the CPSC by their full name.
All respondents will be assigned a unique identifier (ID), which will be created solely for data file management and used to keep all materials for each respondent together. Data collected will be deidentified (e.g., the respondent ID will not be linked to the respondent’s name). In both the confirmation emails and in the welcome screen to the focus groups, participants will be asked to sign in using only their first names. Prior to recording the session and if any of the participants signed in with their full names, the moderator and tech support will ensure that only the participants’ first names will be visible on the recording. To facilitate open conversations, individuals’ first names will be used during the session and therefore may be recorded on audio or video tapes.
Audio and video tapes will not be released to the public. All demographic information on focus group participants will be generalized in the final report to CPSC, so that it cannot be attributed to any specific individual.
Prior to the online focus group session, participants will receive a digital Consent Form. Participants will be asked to read and digitally sign the consent form. Once they “submit” the form, EurekaFacts staff will automatically be notified and will document receipt. The consent form outlines the general purpose of the study and the activities the study may require. The consent form also informs participants that they will be recorded during the study and that their participation is voluntary.
Any information collected from respondents, as well as recordings, will be destroyed within 60 days of the end of the study. All data collection and analysis will be performed in accordance with OMB standards and guidance, Privacy Act, and Protection of Human Subject requirements.
The estimated burden for recruitment assumes attrition throughout the focus group process. Assumptions for approximate attrition rates for direct participant recruitment from initial contact to follow-up are 80 percent, and from follow-up to confirmation 33 percent. All focus groups sessions will be scheduled for no more than 90 minutes.
Table 4. Estimate of Hourly Burden Hours for the Focus Group Sessions
Type of Respondent |
Form Name |
Number of Respondents |
Number of Responses |
Hours per Respondent |
|
BIPOC |
Screening |
150 |
150 |
.15 |
22.5 |
Follow-Up Contact |
60* |
60 |
.05 |
3 |
|
Consent Form |
20* |
20 |
.1 |
2 |
|
Confirmation |
20* |
20 |
.1 |
2 |
|
Focus Group Participation |
16* |
16 |
1.5 |
24 |
|
Hispanic (Bilingual) |
Screening |
150 |
150 |
.15 |
22.5 |
Follow-Up Contact |
60* |
60 |
.05 |
3 |
|
Consent Form |
20* |
20 |
.1 |
2 |
|
Confirmation |
20* |
20 |
.1 |
2 |
|
Focus Group Participation |
16* |
16 |
1.5 |
24 |
|
Parent |
Screening |
100 |
100 |
.15 |
15 |
Follow-Up Contact |
40* |
40 |
.05 |
2 |
|
Consent Form |
20* |
20 |
.1 |
2 |
|
Confirmation |
20* |
20 |
.1 |
2 |
|
Focus Group Participation |
16* |
16 |
1.5 |
24 |
|
Older Adults (age <55) |
Screening |
150 |
150 |
.15 |
22.5 |
Follow-Up Contact |
60* |
60 |
.05 |
3 |
|
Consent Form |
20* |
20 |
.1 |
2 |
|
Confirmation |
20* |
20 |
.1 |
2 |
|
Focus Group Participation |
16* |
16 |
1.5 |
24 |
|
Low income |
Screening |
100 |
100 |
.15 |
15 |
Follow-Up Contact |
40* |
40 |
.05 |
2 |
|
Consent Form |
20* |
20 |
.1 |
2 |
|
Confirmation |
20* |
20 |
.1 |
2 |
|
Focus Group Participation |
16* |
16 |
1.5 |
24 |
|
Total Burden |
250.5 |
||||
*Subset of participants screened, and therefore is not double counted in the total number of respondents.
The total cost of the study is $160,120. This cost includes recruitment, data collection, analyses, report writing, and participant incentives.
EurekaFacts will begin recruiting and screening potential participants upon receiving OMB clearance. Focus groups are tentatively scheduled to begin by May 1, 2023, depending on when OMB clearance is authorized.
Activity |
Tentative Dates |
Recruitment |
Mid-April – May 2023 |
Focus group sessions |
May 2023 |
Draft report with recommendations |
July 2023 |
1 Miller, D., 2016–2018 Residential Fire Loss Estimates, U.S. National Estimates of Fires, Deaths, Injuries, and Property Losses from Unintentional Fires (July 2021); U.S. Consumer Product Safety Commission, Bethesda, Maryland
2 Fire Risk in 2019, Topical Fire Report Series (October 2021, Volume 21, Issue 8); US Fire Administration, Emmitsburg, Maryland.
3 Fahy, R. Ph.D. and Maheshwari, R., Poverty and the Risk of Fire (July 2021); NFPA Research, National Fire Protection Association, Quincy, MA
4 Wolf, M.S., et al. (2007). To err is human: Patient misinterpretations of prescription drug label instructions." Patient Education and Counseling, 67(3), 293-300.
5 Sugovic, M., Nooraddini, I., Sherehiy, B. (2016). Evaluation of safety label design: Comparison between cognitive interviewing versus focus group methods. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1), 1632-1636.
6 Freeman, T. (2006). ‘Best practice’ in focus group research: Making sense of different views. Journal of Advanced Nursing 56(5), 491-497.
7 Fern, E.F. (2001). Advanced focus group research. Thousand Oaks, CA: Sage Publications.
8 OMB.report Justification for Providing Incentives for Participation in Marketing Research. OMB: 0938-1247
9 Stewart DW and Shamdasani PN (2015). Focus Groups: Theory & Practice, 3rd Edition. Los Angeles: Sage.
10ICF (2022). Consumer Focus Groups Relating to Prepaid Products Rulemaking and Model Form Development – Request for OMB approval. OMB: 3170-0022.
11 Boyd, C. (2022, August 25). The Ultimate Guide to User Research incentives. The Ultimate Guide to User Research Incentives. Retrieved from https://www.userinterviews.com/blog/the-ultimate-guide-to-user-research-incentives
12 The 2022 Research Incentives Report. (2023, February 1). Retrieved February 24, 2023 from https://www.userinterviews.com/blog/research-incentives-report
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Alanis Irigoyen |
| File Modified | 0000-00-00 |
| File Created | 2023-07-29 |