1625-0003 Terms of Clearance Response

TERMS OF CLEARANCE: This ICR is approved for one year. Prior to resubmission, USCG will seek and address feedback from key stakeholders to assess potential data utility and data quality concerns. USCG will also test and refine the form's layout, instructions, and information collection elements.

RESPONSE: The Coast Guard Auxiliary Association (CGAuxA) was awarded a grant from the United States Coast Guard (Coast Guard) to examine and recommend appropriate changes to the present Boating Accident Report (BAR) Form (CG-3865 [Rev. 12-06]. CGAuxA worked with a nationally known expert in forms design, Dr. Deborah J. Mayhew, to recommend changes to the present BAR form. Dr. Mayhew consulted with the USCG, the National Association of State Boating Law Administrators and other stakeholders and developed a new and easier-to-use BAR form, while considering data quality and data utility concerns. The current and recommended forms are both constrained by the requirements of 33 CFR 173 and 174. The recommended form:

• Captures the requirements now contained in the Code of Federal Regulations

(CFR),

• Incorporates many of the suggestions offered by key partners such as members of National Association of State Boating Law Administrators (NASBLA), the

National Boating Safety Advisory Council (NBSAC), and the United States

Power Squadrons (USPS®) who read this report in draft,

• Is based on proven design principles to increase form readability and user-friendliness,

• Reflects lessons learned in the usability test, and

• Is slightly longer (largely because of increased white space and a more organized and consistent layout), but significantly easier to read and understand than the present form.

Usability Test

CGAuxA conducted a limited (seven subjects, all boaters and some fisherman as well)

usability test as part of this work and made revisions to the recommended form based on the results of the usability test. The intent of the usability test was to identify any “serious” flaws in the design, rather than to draw statistical inferences. The majority of the individuals sampled were male (6 of 7), college educated (7 of 7), roughly half (3) were 41 – 55 years of age, the other half were older, all were experienced boaters, most with around 50 years of experience.

The usability test provided information on respondents views of the present and recommended forms. More important, it identified questions/fields that were ambiguous or difficult to understand, and permitted us to make appropriate revisions.

Despite the limited sample size, the results of the usability test were interesting.

For example the test found that:

• Nearly half (3 of 7) of the respondents did not know that it was necessary for the operator/owner to complete a BAR form and most (5 of 7) did not know under what circumstances the form is required (i.e., the definition of a reportable accident). Most (6 of 7) did not know where to get blank forms when needed or where to submit them. This certainly highlights the need to educate recreational boaters on the requirements for submitting the form.

Design Research Engineering (30 August 2007 memorandum) was pleased that a usability test was conducted. However, they stated “The draft form should be pre-tested with law enforcement officials, who frequently complete the form, especially for boating accidents involving serious injury or fatality. Since it is likely that law enforcement will find the draft form ill-suited to their needs and knowledge-level, an optimal course of action is to begin designing a form specifically for the law enforcement community. We respectfully disagree with the Design Principles assertion that the regulations do not allow this.” We do not purport to provide legal analysis of the CFR. Our brief was to develop a form that could be completed by the operator/owner of the boat. We do agree that the development of any subsequent forms should be field tested with the population who will be filling out the form. If a two-tier system is implemented, then it will be appropriate to field test the form with the target populations.

• The recommended form was consistently preferred to the present form in several respects. For example, respondents reported that it was easier to determine if a report was required, to whom to send the report, more likely to be completed, easier to understand and complete accurately, had an improved layout (including adequate space for responses), and the font and reduced capitalization were preferred. It is interesting to note that some respondents actually believed that the recommended BAR form was shorter than the present form—a subjective assessment.

• Respondents were unsure about the meaning of certain terms (e.g., medical treatment beyond first aid, vessel documentation number, hull identification number, dam/lock, failure to vent, external navigational aid, and gunwale). We made some changes based on this result, but retained certain terms, such as vessel documentation number and hull identification number even though one or more respondents did not know the meaning of these terms.

• Respondents felt answers to some questions were unclear or subjective. For example, possible answers relative to wind and water conditions were questioned by respondents. Regarding water conditions the use of the descriptors “Calm,” “Choppy,” “Rough,” and “Very rough” were subjective—even though we defined each of these terms (e.g., the calm water condition was defined as up to 6 in. waves). We agree that the descriptive terms, by themselves, admitted to various interpretations—what is choppy relative to the operator of a 60-ft sailing vessel might differ if an 8-ft rowboat were being used. As a compromise, we placed the quantitative description first, followed by a qualitative characterization in parentheses, for example, “Up to 6 in. waves (calm).

• Respondents were irritated by certain questions. For example, they were irritated

that “age” and “date of birth” were both included as fields. Thus, we eliminated “date of birth” and retained “age.” As a second example, respondents queried the necessity of including “inches” as well as “feet” in characterizing the length and beam of their boats; we changed the fields to feet only and changed the header to read “Size estimates.” In these and other cases we made changes to the form to address their concerns, unless doing so would compromise compliance with the CFR requirements.

• The recommended form still requires time and effort to complete. We made several revisions to the recommended form based on the usability test, but some difficulties remain. In our opinion these remaining difficulties can only be resolved by eliminating some of the essential elements of information EEIs now required by the CFR or relaxing the present requirement that the form be completed by the operator/owner. Sensitive to the negative impression a longer form might make on operators/owners, we limited the length of the initial draft of the recommended form to six pages - two pages more than the current form, but in fact still fewer pages than many state forms.

We revised the initial draft of the recommended form to produce a final draft based on the comments and results of the usability test. However, we held the length of the final draft of the recommended BAR form to six pages while still improving the layout, readability, and impression of the form. In so doing we made several compromises. For example:

• Both internal reviewers and some usability test respondents indicated a desire for more room in the accident description entry area on page 1 of our initial draft of the recommended form. However, this (as well as other legitimate suggestions) would have required adding pages to the recommended form, so we chose not to take this suggestion at this time. Instead in this case, we added an instruction line indicating that respondents could attach additional pages on which to continue their accident description if necessary. This seemed like a reasonable compromise in the timeframe we had to complete the revision of the form.

• It is worth noting however, that during the usability test, some respondents clearly

indicated they would prefer a longer form that was better organized, better laid out and easier to understand, to a shorter form that achieved brevity by sacrificing these things. Some indicated it really did not matter at all how long the form was if it was truly and optimally “user friendly”. In addition, some test respondents actually thought the recommended form (six pages) they filled out was shorter than the current form (four pages), and in addition they thought it took less time to fill out in spite of the fact that it actually took slightly longer. We recommend that in any next revision of the form, consideration be given to the possibility of lengthening the form in order to incorporate more potentially useful redesign ideas learned during this current revision process. If this is done, it would be important to run another usability study on the lengthened and redesigned form to validate the hypothesis that these changes in fact resulted in positive benefits.