Usability Testing Results

CVH MIS

Content Discussion and Usability Study Result Summary

Overview

At the Cardiovascular Health Heart Disease and Stroke Prevention (CVH HDSP) Evaluation Meeting on December 4, 2003, twenty four (24) representatives from Basic Implementation States gathered to participate in CVH HDSP Management Information System (MIS) usability exercises and content discussions. The participants were divided into four groups of six. Each group participated in four different information gathering sessions. The sessions consisted of the following:

1. Usability exercises

2. Post usability questionnaires

3. Program information, burden definition, work plan discussions

4. Infrastructure and capacity discussions.

The purpose of these sessions was to validate the proposed content for selected sections of the MIS, to verify the overarching navigation and framework of the MIS, verify functionality and terminology used, and identify any potential usability or design issues.

Once collected, the feedback from these sessions underwent analysis which resulted in 12 recurrent themes. The themes were then categorized as being either “content” or “usability” related. A strategy session was then held to formulate recommendations for addressing each theme as it relates to the MIS. The themes and their associated recommendations were presented to the CDC for comment and will now be used to inform the ongoing development of the CVH MIS and its content.

Below is a brief overview of the themes and their recommendations.

General Comments

Overall, most participants were excited about using the MIS and expressed a desire to become more involved in the design and development of the system. After working through the usability exercises using the MIS prototype, participants were asked what they liked most about the prototype. The following is a sampling of comments:

• “Consistent and easily managed.”

• “Nice to have partners all in one place.”

• “Central repository for our work plan.”

• “Clean look.”

• “A way to keep up with data for reporting purposes.”

Content Themes & Recommendations

The content themes focused on the specific content areas of the MIS - the questions to be asked in the MIS and the state’s ability to provide the information.

• The purpose of the MIS is not clear to the States

Based on input from the states, it is recommended that a vision statement be created for the CVH HDSP MIS project and shared with states funded for CVH programs, and other stakeholders. A vision statement is critical for clearly articulating the understanding and purpose of the MIS. It will also guide and support the current development of the CVH HDSP MIS as well as future development and enhancements to the system. Currently, an effort is underway to develop a vision statement.

• The MIS should generate the Interim Progress report

Since the MIS collects information that the states use when developing their Interim Progress Reports (progress report and the continuation application), states felt strongly that the MIS should be the tool to also generate the report. As such, the recommendation was made to ensure that the MIS collected information relevant to the Interim Progress Report so it could be printed from the MIS. The generation of the Interim Progress Report within the MIS will promote a standard format for the report, which will improve reporting consistency. In addition, this recommendation also provides states with a central repository for CVH programmatic information, as well as, an ‘everyday’ tool for tracking the progress of their CVH program activity.

• Only appropriate, relevant and important information should be collected

States offered many suggestions for revisions to the content of the MIS. Due to the time constraints of the meeting, however, not all suggestions were discussed completely. As such, it is recommended that a collaborative work group be established. Work groups have a positive impact on productivity, creativity, direction, attaining project goals, and ensuring that the content and the design of the system is in line with the needs of both CDC CVH and state programs. Recently, a CVH MIS Work Group was established. This work group will consist of representatives from both the CDC and state programs. The responsibilities of the workgroup will be to:

• Establish priorities for the MIS

• Ensure that the vision of the MIS is adhered to

• Act as a communication liaison between the MIS development team, other CVH Branch staff and the states

• Make recommendations on MIS content and design decisions

• Participate in user and usability testing

Usability Themes & Recommendations

• Titles to sections of pages and field labels are not clear enough

After completing the required tasks in the usability exercises, states remarked that the tab and field labels were confusing or easily misinterpreted. It is recommended that tab and field labels be re-evaluated to ensure that they are clear, direct the user navigationally and provide appropriate level of detail. Additionally, it was discovered that response options on several questions were either too vague or too detailed. As such, response options should be all inclusive and mutually exclusive.

• More help text is needed

While working through the usability exercises, states asked for definitions or examples of some field titles and response options for more clarity. As a result, it is recommended to incorporate contextual and in-line help, particularly for program-specific language; develop a glossary of terms; and provide end user training on the MIS to clarify the intent of the information requested in the CVH MIS.

• Screens are too dense

States expressed concern over the amount of information collected on one screen. Typically, the screens involve a lot of scrolling information which increased the chances that information would be overlooked. As an alternative to these long, scrolling pages, it is recommended that the information be grouped into smaller, more manageable sections. Not only will this avoid important information from being overlooked, but it will also minimize eye fatigue and avoid feelings of being overwhelmed when entering information into the MIS.

• The relationship between sections of the MIS are unclear

While working through the usability exercises, it became apparent that the relationships between different sections in the MIS were not clearly apparent. In order to reiterate the existing relationships between program elements, it is recommended that the MIS user interface be refined to visually communicate the relationships. Positioning related information in closer proximity will ensure that the MIS reinforces the existing CVH program structure that states are currently familiar with, and ensures a navigational flow that clearly communicates the relationship between program elements. For example, one change may include allowing the user to add an activity associated with an objective within the context of the objective.