Test Trial Focus Groups on Child Resistant ATV Ignition System Functionality and Usability Study

TITLE OF INFORMATION COLLECTION:

Test Trial Focus Groups on Child Resistant ATV Ignition System Functionality and Usability Study

The general objective of this project is to provide the Consumer Product Safety Commission (“CPSC”) with data to determine if the child-resistant ignition lock system prototype is effective at preventing children from operating the ATV and assess the acceptability of the system for adult users.

Test Trial focus groups will employ a hands-on interaction with the child-resistant ATV ignition system prototype (also “CR ignition” or “child-resistant ignition”), permitting child and adult participants to experience both a standard and child-resistant prototype ignition so that they can provide a more systematically valid evaluation of the technology and its functionality. The focus group discussions will be enhanced by the exposure of each participant to a working system that will enable each participant to provide a concrete basis for his or her opinions on the effectiveness, usability, and acceptability of the system, in addition to parameters that would maximize belt use and safety benefits of this technology.

PURPOSE:

All-terrain vehicles (“ATVs”) are motorized vehicles, designed for off-road use, with four low-pressure tires (less than 10 pounds per square inch), a seat designed to be straddled by the operator, and handlebars for steering. ATVs have been associated with a high number of deaths and injuries to children and teens. Approximately one-fourth of ATV-related fatalities reported to CPSC staff involve children younger than 16, and about 40 percent of these are children younger than 12 (2012 Annual Report for ATV-Related Deaths and Injuries). In the 31-year period ending in December 2012, there were almost 3,000 reported ATV-related fatalities of children younger than 16. Naturally, there are many more injuries for every fatality reported; the 2012 Annual Report estimates 26,500 ATV-related, emergency department-treated injuries in 2012 for children younger than 16. There are numerous aspects to this problem, but critical among them is the ability of under-age children to start and operate an adult ATV.

CPSC staff believes that a child-resistant ignition on ATVs may have the potential to reduce the likelihood that a child could start an adult ATV, and consequently, reduce the number of injuries and deaths to children in this class of vehicle. Virginia Polytechnic Institute and State University’s Industrial and System Engineering department designed a prototype child-resistant ignition. The prototype requires the ATV driver to depress buttons on the handle bars, depress a foot pedal, and sit toward the rear of the vehicle seat to start the vehicle. However, there often is a trade-off between the effectiveness of an operational device in influencing the child’s behavior and the willingness of adult users to accept the device. The CPSC is interested in whether the physical requirements of this child-resistant ignition prototype can prevent children under a certain age from starting the ignition of an ATV without affecting adult use. More broadly, the issue is whether ATVs can be designed so that a child would be unable to start the vehicle; and if this is possible, how would such an ignition system be used and accepted by the adult riding population.

To establish the effectiveness of the child-resistant ignition prototype in reducing the number of child injuries and deaths related to ATVs, CPSC staff requires a thorough evaluation, in qualitative and quantitative terms, of the successes and problems of various groups of children, so that subsequent decisions can be data-driven and supported objectively. If the child-resistant ignition prototype is successful at preventing children from operating an ATV, but also presents the same issues for adult users, the perceived usability and acceptance by consumers may negatively impact their decision to purchase a vehicle with this feature. On the other hand, the success of the system could increase the likelihood that adults would look for strategies that will disable it. Therefore, the study will include adults and children, and more specifically, subgroups of larger children and smaller adults. In addition, the experimental design also needs to address adult annoyance and acceptability issues that may provide consumer acceptance barriers.

To determine the effectiveness of the ignition prototype in preventing children from operating the ATV and to assess adult user acceptance and usability, CPSC staff requires:

• information on ATV use patterns for both children and adults;

• the likelihood that the prototype will effectively inhibit children from activating the ignition;

• the likelihood that the prototype will prevent adult users from operating the ATV;

• the strategies used to operate the prototype (for children and adults);

• the degree of adult user-acceptance and perceived annoyance.

CPSC’s contractor will execute all necessary preparatory and logistical activities to recruit for, and conduct, all the focus groups, including developing the test trial protocol, preparing moderator guides, securing facilities, maintaining and shipping the ATV (if necessary), preparing support materials for the focus groups, and recruiting participants.

The contractor will conduct test trial focus groups with recruited adults and children. The recruitment strategy will include targeting smaller-stature adults and larger-stature children to assist in the exploration of the physical boundaries within each group that may impair usability or increase the likelihood that an individual can successfully use the ignition system. The focus groups will employ a hands-on interaction with both the standard and child-resistant ignition prototype. Observing the children as they interact with the standard and child-resistant prototype will enable researchers to assess how effective the prototype is, relative to the standard ignition, at inhibiting children from using an ATV. Observing the children will also allow researchers to identify strategies used by the children to operate the ignition, which could be used to develop other child-resistant requirements. Permitting adult participants to operate the ignition will enable them to provide a more systematically valid evaluation of the technology and its functionality. At no point will the ATV actually be running; a series of lights will indicate whether the ignition systems have been activated. The focus group discussions will be enhanced by the exposure of each participant to a working system that will enable each participant to provide a concrete basis for his or her opinions on the effectiveness of this particular countermeasure strategy, usability, acceptability, and design features and parameters that would maximize the safety benefits of this technology.

Once the participant has had an opportunity to start the ATV using the standard and child-resistant ignition prototype, the contractor would like to conduct a small focus group (no more than three participants at a time) facilitating discussions that will address the following topics:

Small focus group discussions will address the following topics for children:

• ATV experience

• Experience with other child-resistance technologies and other child-resistant objects

• Description of their strategies when attempting to activate the ignition

• Perceived difficulties

• Other strategies they may use to disable the child-resistance feature

Small focus group discussions will address the following topics for adults:

• ATV experience

• Experience of adults with unauthorized use of their ATVs

• Perceived difficulties activating the prototype ignition

• User acceptance and perceived annoyance

• Strategies used to operate the prototype ignition

• Other usability related issues with the child-resistant ignition

• Likelihood of disabling or overriding the ignition system

• Would it affect their desire to purchase a new ATV if it had this type of ignition system

• Specific design features that might increase user acceptability

• Likelihood that a child-resistant ignition will prevent a child from starting the ATV

• Strategies a child may use to disable the prototype ignition

At the conclusion of the test trial focus groups, the contractor will develop a report that will also summarize insights gained regarding the overall performance of the child-resistant ignition prototype. The report will identify any strengths, limitations, and strategies used by children and adults to override the system; physical characteristics of children who were able to defeat the system; usability considerations for adult riders; and acceptance by particular user groups. Finally, the information gathered will be used to make recommendations on the effectiveness of the overall system and conditions under which users are more willing to accept the feature, as well as highlight specific design features that might increase user acceptability.

DESCRIPTION OF RESPONDENTS:

The recruitment goal is to identify 180 children between the ages of 7 and 15 years old and 50 adults who are at least 18 years of age. In accordance with the study objectives, half of the children will have experience with ATVs, and half will not. All adults will have experience riding or driving ATVs. In each of the two age groups, a portion of the participants will represent the more extreme percentiles regarding physical size. That is, approximately 10 children will be within the 85^th percentile regarding physical size, and approximately 10 adults will be within the 15^th percentile for physical size. Participants will be selected carefully, targeting individuals representing the conditions outlined above.

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [X - $50] Yes [ ] No

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Burden
Focus Group Participants	230	150 minutes per participant	575 hours
Parents of child participants	180	150 minutes per parent	450 hours
Totals	410	150 minutes per participant	1025 hours

FEDERAL COST: The estimated annual cost to the federal government is $248,731

Total estimated cost to the government for conducting the data collection is as follows:

Number of Participants 410

Total estimated cost of conducting survey $248,731

Cost per completed Participant $607

This estimate is based on the total cost of the awarded research contract, divided by the specified number of completed participants.

CHILD-RESISTANT IGNITION FUNCTIONALITY AND USABILITY STUDY

Project Number: 8914.05: Test Trial Focus Groups

Moderator Guide: Adult Participants

1. Review Purpose, Objective, and Scope of the Focus Group

Introductions and rules

Moderator introduces self and aides

The purpose of this focus group is for us to learn more about how people are using their all-terrain vehicles (ATVs) and to test a new child-resistant (CR) feature on the ATV. The CR ignition prototype is intended to help prevent children who do not meet certain height and weight requirements from starting the ATV. The U.S. Consumer Product Safety Commission would like to explore the system’s potential effectiveness, as well as user acceptance issues, and factors that would increase the likelihood of adult drivers accepting this type of device in their vehicles. The overall objective of the study is to understand whether this ignition prototype works, whether there are improvements that could be made to increase the feature’s effectiveness, as well as users’ acceptance of the child-resistant feature.

You have been selected to participate in this focus group because you indicated that you are a regular user of an ATV. We would like to provide you with the opportunity to try out the CR-ignition prototype that we have installed on an ATV that we have here and ask you to share your opinions with us.

How many of you have taken part in a focus group?

Before we begin our discussion, I would like to review some basic issues and guidelines.

1. Focus groups have certain rules or protocols that we follow:

1. No one will be judging your responses.

2. We need to hear about your feelings and opinions. We are not here to reach consensus, but rather, to hear and discuss a range of views. There are no “right” or “wrong” answers.

3. We want to encourage crosstalk among group members. We are not here to hear from moderator. The moderator will merely guide the discussion to cover the topics we need to hear about.

4. We want to give everyone the opportunity to speak–it is important to hear from everyone.

5. The session is being recorded for offline analysis; participation is voluntary.

6. Refreshments, restrooms, and breaks are available and planned.

2. We hope that you will feel free to be completely honest in this discussion. Your specific responses will NOT be shared with anyone other than people working on the project. Your name, and any other identifying information about you, will not be used in any report that we prepare about these focus groups.

3. Please respect the privacy of the other people in this group by not discussing what is said here with anyone outside the group or in public.

4. Our objective is to gain insight on how people would feel about this type of system. As we go through the session, I will guide the group along various topics; but you are the experts, and you will be doing most of the talking.

2. Self-Introductions and Ice Breaker Exercise

Go around to each person in turn.

Round of introductions of participants (first names) –

1. What is the make and model of the ATV that you own or use most frequently?

2. How often do you use it?

3. What kind of features does it have? Have you made any special modifications to it?

4. Can you describe ways that you use it?

• Recreational

• Work

• Both

• Other

3. Pre-Ride Discussion

Driving Behavior

1. What types of terrain do you usually travel over?

   1. Surfaces (grass, mud, sand, gravel, ruts and bumps)

   2. Slopes (up and down, along the side)

2. What kinds of maneuvers do you make while driving your ATV?

3. What are your typical speeds?

4. Where do you store your ATV? Do other members of your family have access to it?

5. How often do you ride alone vs. with other riders?

6. Do you have children or grandchildren?

   1. Do you ride with children?

      1. How often do you ride with them?

      2. Are your children/grandchildren always passengers, or do they drive the ATV on occasion?

      3. Are they allowed to ride alone/unsupervised?

      4. Has your child/grandchild ever tried to use the ATV unsupervised or without your permission?

   2. Do you ever talk about ATV safety with your children/grandchildren?

Child-Resistance Feature

1. Can you think of any products that have child-resistant features?

2. In general, how do you feel about child-resistant features on any type of product, not just ATVs?

3. What have been your experiences with child-resistance technology?

4. Hands-On Field Task/Anthropometric Measurements

Now, I would like each of you to have the opportunity to experience starting the ATV with the standard and child-resistance ignition prototype. Each of you will go with ___________ and have an opportunity to try the system out one at a time. I will ask that you do not speak to each other about your experiences with the ATV until everyone has had an opportunity to try.

Once you all have participated in this exercise, we will come back here, and I will ask you to provide some feedback on your experiences.

To have a better idea of whether some of the ATV ignition systems work for people of all shapes and sizes, we need to collect a few measurements from you. The items we want to measure are your weight, your height while standing, your arm length, your leg length, the length of your hands and feet, the length of your torso, and how far you can reach.

To save time, while you are waiting for your turn, ____________ will measure your height, weight, arm length, leg length, torso length, foot length, finger length, and the distance of your reach. Measurements will be taken on one person at a time.

See document titled, “Hands-On Field Task/Anthropometric Measurements Script.”

5. Post-Task Discussion

General discussion on the task and performance

1. Please tell us about your experience with the different ATV ignition types?

   1. Standard Ignition?

   2. CR Ignition Prototype?

2. Which system (Standard vs. Prototype) was more challenging to work with?

3. Did the child-resistance feature impact your ability to start the ATV?

4. Did you use any strategies to start the child-resistant ignition? What strategies did you use?

5. Before the method was demonstrated to you, was it clear to you which parts of the ATV needed to be activated to overcome the child-resistance feature?

Child-Resistant Ignition

1. In your opinion, what is the likelihood that the child-resistant ignition will prevent a child age 7 or 8 from starting the ATV? How about a child age 9-11? What about children ages 12-15?

2. Can you identify any strategies a child (perhaps your child or grandchild) might use to start the ATV with the child-resistance feature or to overcome the child-resistance feature? [PROBES: Heavy objects, friends, jumping up and down on the seat?]

Acceptance

1. Were you ever frustrated attempting to start the ATV with the child-resistance feature?

   1. If so, why?

2. How do you think the presence of the child-resistance feature would influence your ability to use your ATV?

3. How would you feel about the child-resistant system as a feature in your ATV?

   1. What are the pros?

   2. What are the cons?

   3. Are there any unintended consequences?

4. How would the presence of the child-resistant feature influence your decision to purchase this type of vehicle?

5. What might make you change your mind about purchasing a vehicle with the child-resistance feature?

6. What changes to the child-resistant system design would improve your impression of the system and your overall acceptance?

   1. Why?

7. If there was a child-resistant feature on your ATV, would you try to disable it?

   1. How?

Alternative Ideas

1. Can you suggest any other ideas that might improve the design of the child-resistant ignition on the ATV?

6. Wrap-Up

Does anyone have anything else they would like to mention about the child-resistant feature before we conclude the focus group? Are there any issues that we have not raised?

Closing Remarks

Thank you for your time. What we have heard and learned about today will help us assess the feasibility of this system as an effective way to prevent children from starting ATVs.

• Instruct participants about how they will be reimbursed for their time.

CHILD-RESISTANT IGNITION FUNCTIONALITY AND USABILITY STUDY

Project Number: 8914.05: Test Trial Focus Groups

Moderator Guide: Child Participants (w/ATV Experience)

1. Review Purpose, Objective, and Scope of the Focus Group

Introductions and rules

Moderator introduces self and aides

Have any of you ever participated in a focus group before?

A focus group is simply a group of people sitting around a table talking about a specific subject or issue. We just want to know how you feel about something or your opinion.

The purpose of this focus group is for us to learn more about how people are using their all-terrain vehicles (ATVs) and to test a new feature. The U.S. Consumer Product Safety Commission is our client and has asked us to help them to try to understand whether this feature works, and if there are ways to improve it or make it better.

You have been selected to participate in this focus group because your parents told us that you have experience with ATVs. Your family may own one, or you may ride on an ATV. We would like you to try out this specific feature on our ATV and then share your opinions with us or tell us how you feel about this feature.

Before we begin our discussion, I would just like to review some basic issues and guidelines.

1. Focus groups have certain rules that we follow:

   1. No one will be judging your responses.

   2. We need to hear about your feelings and opinions. There are no “right” or “wrong” answers.

   3. We want to give everyone the opportunity to speak–it is important to hear from everyone.

   4. The session is being recorded so that we can review the recordings later; you are here because you volunteered, and you are free to leave at any time.

   5. Refreshments, restrooms, and breaks are available and planned.

2. We hope that you will feel free to be completely honest in this discussion. Your specific responses will NOT be shared with anyone other than the people working on the project. Your name will not be used in any report that we prepare about these focus groups.

3. Please respect the privacy of the other people in this group. Please do not talk about what the other children say with anyone outside the room.

4. Our purpose is to understand how people would feel about this type of system. As we go through the session, I will guide us along various topics; but you are the experts and will be doing most of the talking.

2. Self-Introductions and Ice Breaker Exercise

Go around to each person in turn.

Round of introductions of participants (first names) –

1. Tell me about the last time you were on an ATV, and what did you do?

   1. Where were you riding?

   2. Who were you with?

   3. Were you the driver or the passenger?

3. Pre-Ride Discussion

Driving Behavior

1. How often do you ride on an ATV? How long have you been riding?

2. Do you ride as the driver, the passenger, or both?

1. When you drive, does anyone else ever ride on the ATV with you?

3. Where do you ride?

4. Does your family own an ATV?

   1. How many ATVs does your family own?

   2. Can you tell us the make and model of your family’s ATV(s)?

5. Do you have your own ATV?

6. Can you tell us the make and model of your ATV(s)?

7. Do other members of your family use your ATV?

8. Have you ever tried to start an ATV?

   1. An ATV made for an adult?

   2. An ATV made for a child?

9. Have your parents or guardian or any adult ever discussed ATV safety with you?

10. Do you ever ride alone or without supervision?

11. What do you like to do when you ride on the ATV?

12. How fast do you usually ride?

4. Hands-On Field Task/Anthropometric Measurements

Now, I would like each of you to have a chance to experience our ATV. Each of you will go with ___________ and have a chance to try to start the ATV. I will ask that you do not speak to each other about your experience with the ATV until everyone has had a chance to try it.

Once you all have had a chance to try the ATV, we will come back here, and I will ask you to tell us about your experiences.

To have a better idea of how people of all shapes and sizes work together with ATVs, we need to collect a few measurements from you. We would like to measure your weight, your height while standing, your arm length, your leg length, the length of your hands and feet, the length of your torso, and how far you can reach. (Demonstrate on your own body which measurements will be made).

While you are waiting for your turn, ____________ will take some measurements, including measurements of your height, weight, arm length, leg length, and a few other measurements. Measurements will be taken on one person at a time.

See document titled, “Hands-On Field Task/Anthropometric Measurements Script.”

5. Post-Task Discussion

General discussion on the task and performance

1. Please tells us a bit about your experience with the ATV?

2. Can you tell me what you did to try to start the ATV?

   1. How did you try to start the ATV?

   2. Can you tell me what you did? What did you try first? Why?

3. We only allowed you ___ minutes to try to start the ATV. If you were at home, what other things might you have done to start the ATV?

6. Wrap-Up

Does anyone have anything else they would like to mention about their experience before we end the focus group? Are there any issues that we have not raised?

Closing Remarks

Thank you for your time. What we have heard and learned about today will help us understand this system for ATVs.

• Instruct participants about how they will be reimbursed for their time.

CHILD-RESISTANT IGNITION FUNCTIONALITY AND USABILITY STUDY

Project Number: 8914.05: Test Trial Focus Groups

Moderator Guide: Child Participants (w/out ATV Experience)

1. Review Purpose, Objective, and Scope of the Focus Group

Introductions and rules

Moderator introduces self and aides

Have any of you ever participated in a focus group before?

A focus group is simply a group of people sitting around a table talking about a specific subject or issue. We just want to know how you feel about something or get your opinion.

The purpose of this focus group is for us to learn more about how people are using their all-terrain vehicles (ATVs) and to test out a new feature. Some of you may not know what an ATV is. That’s okay. We will describe what an ATV is, what it looks like, and how an ATV is used a bit later. The U.S. Consumer Product Safety Commission is our client and has asked us to help them to try to understand whether this feature works, and whether there are ways to improve it.

You have been asked to participate in this focus group because your parents told us you have never been on an ATV before. We want you to try out this feature on our ATV and let us know how you feel about it and share your opinions with us.

Before we begin our discussion, I would like to review some basic issues and guidelines.

1. Focus groups have certain rules that we follow.

   1. No one will be judging your responses.

   2. We need to hear about your feelings and opinions. There are no “right” or “wrong” answers.

   3. We want to give everyone the opportunity to speak–it is important to hear from everyone.

   4. The session is being recorded so that we can review the recordings later; you are here because you volunteered, and you are free to leave at any time.

   5. Refreshments, restrooms, and breaks are available and planned.

2. We hope that you will feel free to be completely honest in this discussion. Your specific responses will NOT be shared with anyone other than the people working on the project. Your name will not be used in any report that we prepare about these focus groups.

5. Please respect the privacy of the other people in this group. Please do not talk about what the other children say with anyone outside the room.

6. Our purpose is to understand how people would feel about this type of system. As we go through the session, I will guide us along various topics, but you are the experts, and will be doing most of the talking.

2. Self-Introductions and Ice Breaker Exercise

Go around to each person in turn.

Round of introductions of participants (first names) –

1. What sort of outdoor activities do you like to do? (Ride bikes, rollerblade, play soccer)

   1. When was the last time you _________?

   2. Where do you do these activities?

   3. Who were you with?

3. Pre-Ride Discussion

Prior ATV Knowledge

1. Do you know what an ATV is? [After participants share their thoughts, describe an ATV and show a picture of one.]

1. An ATV is an all-terrain vehicle that has a seat that is straddled by the rider, along with handlebars for steering. All terrain means that this vehicle can be ridden on a lot of different types of ground (dirt, gravel, sand).

2. Do you know anyone who has an ATV? If so, have you ever seen them start or ride it?

3. What do you think about ATVs?

4. Hands-On Field Task/Anthropometric Measurements

Now, I would like you each to have a chance to experience our ATV. Each of you will go with ___________ and have a chance to try to start the ATV. I will ask that you do not speak to each other about your experience with the ATV until everyone has had a chance to try it.

Once you all have had a chance to try the ATV, we will come back here, and I will ask you to tell us about your experiences.

To have a better idea of how people of all shapes and sizes work together with ATVs, we need to collect a few measurements from you. We would like to measure your weight, your height while standing, your arm length, your leg length, the length of your hands and feet, the length of your torso, and how far you can reach. (Demonstrate on your own body which measurements will be made).

While you are waiting for your turn, ____________ will take some measurements, including your height, weight, arm length, leg length, and a few others. Measurements will be taken on one person at a time.

See document titled, “Hands-On Field Task/Anthropometric Measurements Script.”

5. Post-Task Discussion

General discussion on the task and performance

1. Please tell us a bit about your experience with the ATV?

2. Can you tell me what you did to try to start the ATV?

   1. How did you try to start the ATV?

   2. Can you walk me through the process? What did you try first? Why?

3. We only allowed you ___ minutes to try to start the ATV. If you were at home, what other ways might you use to start the ATV?

6. Wrap-Up

Does anyone have anything else they would like to mention about their experience before we end the focus group? Are there any issues that we have not raised?

Closing Remarks

Thank you for your time. What we have heard and learned about today will help us understand how well this system might work for ATVs.

• Instruct participants about how they will be reimbursed for their time.