0990-HHS Web Site UsabilitySurvey (2)

HHS Web Site Customer Satisfaction Survey

United States Department of Health and Human Services

INTRODUCTION

This is a request for approval by the Office of Management and Budget (OMB), under the federal Paperwork Reduction Act of 1995, for clearance of a Web-based remote-user feedback survey to assess and improve the quality of information services the U.S. Department of Health and Human Services (HHS) provides to customers through its World Wide Web sites. Remote surveys will collect data about how participants interact with HHS’s Web sites electronically. Users will take the survey at their home or work computers. Participants in a usability survey are reflective of a Web site’s target audience.

A. JUSTIFICATION

A-1. Circumstances Making the Collection of Information Necessary

Executive Order 12862 (Attachment 1) directs Federal agencies that provide significant services directly to the public to survey customers to determine the kind and quality of services they need and their level of satisfaction with existing services. The U.S. Department of Health and Human Services (HHS) seeks to obtain approval to conduct usability surveys on pages of the HHS Web sites on an http:///ongoing basis.

This collection of information is necessary to enable HHS to respond to The President's Management Agenda, which calls for government reform that is citizen-centric, results-oriented and market-based and requires agencies to participate in e-Government initiatives that provide high quality customer service, provide citizens with readier access to government services, increase access for persons with disabilities to agency Web sites and make government more transparent and accountable (http://www.whitehouse.gov/omb/budget/fy2002/mgmt.pdf, pp. 4 and 25).

Collecting voluntary customer feedback is the least burdensome, most effective way for HHS to determine whether or not its public Web sites are useful to and used by its customers. In fact, the American Customer Satisfaction Index says the collection of such feedback may be more important in the public sector than in the private sector:

The dissatisfied recipient of Government services can rarely punish a faltering service provider by taking his or her business elsewhere, or by demanding a lower price. There is no market feedback (with the possible exception of elections) that forces the [public sector] service provider to improve, or signals what to improve (ACSI Commentary: Federal Government Scores, http://www.theacsi.org/government/govt-01c.html).

HHS is requesting a 3-year generic clearance in order to carry out its mission. Generic clearance is needed to ensure that HHS can continuously improve its Web sites though regular surveys developed from these pre-defined questions.

Surveying HHS Web sites on a regular, ongoing basis will help ensure that users have an effective, efficient, and satisfying experience on any of our Web sites, maximizing the impact of the information and resulting in optimum benefit for the public. The surveys will ensure that this communication channel meets customer and partner priorities, builds HHS’s brands, and contributes to HHS health and human services impact goals.

HHS employees, full-time contractors, or contract vendors will collect the data for these surveys. The data collected will include background participant information that does not identify individuals, task identification, and overall satisfaction measures. The data will be collected electronically and stored digitally on a secure site or computer.

This survey is authorized under the Public Health Service Act (42 USC 241) Section 301. A copy of the legislation is included (Attachment 2).

A-2. Purpose and Use of the Information Collection

The entire HHS Web site contains over two million pages of information, products, guidelines, and training focused on diseases, health conditions, public health, and human services. The HHS Web sites are comprised of multiple major divisional sites and smaller sites, and most usability surveys will focus on evaluating just one portion of these cites. The HHS Web sites are accessible to everyone on the World Wide Web and has many different audiences, including public health and human services professionals, physicians, media, policy makers, and the general public.

By collecting Web site usability information, HHS will be able to serve and respond to the ever-changing demands of its Web site users. Additionally, we will be able to determine the best way to present messages on the HHS Web site. The HHS site is one of HHS’s primary channels to raise awareness of important issues for its target audiences.

These users include individuals (such as patients, the elderly and disabled, educators, students, etc.), interested communities, partners, healthcare providers, service providers, and businesses. Survey information will augment current Web content, delivery, and design surveys which are used to understand the Web user, and more specifically, the HHS user community.

The purpose of such usability surveys is to judge the Web content and presentation though which HHS communicates scientific, health and public assistance information to its target audiences to help ensure that health and human services impacts are maximized through the delivery of a useful, efficient, and effective Web site.

Primary objectives are to determine whether the HHS Web site:

(1) Meets the wants, preferences, and needs of its target audiences.

(2) Is an effective vehicle for sending messages to target audiences.

(3) Delivers existing services at a satisfying level of quality.

(4) Provides users with the kind and quality of services they need.

Findings will help HHS to:

(1) Understand the user community and how to better serve HHS Web site users.

(2) Identify areas of the Web site requiring improvement in either content or delivery.

(3) Determine the kind and quality of services our target audiences need.

(4) Determine how to align Web offerings with identified user need(s).

(5) Explore new or refined methods for offering, presenting and delivering information most effectively, to enable us to present messages as well as serve the needs of people who are already coming to seek particular information or to learn about a particular topic.

The data collected from this effort will allow us to answer critical usability questions, including:

• What are the Web needs and preferences for our target audiences?

• How often and for what purposes (there can be several simultaneously) do our target audiences typically use the HHS Web sites?

• How satisfied are they with their experience on HHS Web sites?

• What difficulties do they experience when trying to complete typical tasks on HHS Web sites?

• In what ways can we improve their speed and ability to find the information they want, expect or need on HHS Web sites?

• Were messages on the site presented in such a way that they are noticeable, easy to understand, easy to remember, and have an impact on the viewer’s behavior plans?

• How does their awareness of, knowledge of, and opinions on a health topic change after viewing HHS Web sites?

• Did they find information/messages about health and human service issues they weren’t initially looking for when viewing information on the site? Did the message have an effect, e.g. change their behavior plans?

• Are they satisfied with the services offered through HHS Web sites?

• What improvements would the user like to see made to the existing services on HHS Web sites?

• What other services do they need?

The survey will help ensure that HHS Web sites meet site visitor and agency needs, build HHS’s brands, and contribute to health and human service impact goals. Feedback from the user base is necessary to fully judge the performance of HHS’s Web sites. All data collected through the survey will be used to determine whether HHS should revise content, labels, structure, or layout of its Web pages. If indicated, revisions would be intended to increase the success rate of information–seeking Web site visitors.

A-3. Use of Improved Information Technology and Burden Reduction

All data will be collected electronically to reduce the burden to the respondent.

For most questions in the survey, the respondent will click on a “radio button” or checkbox that corresponds to their response. For open-ended questions in usability surveys, the respondent would be told to enter their answer in the provided text box. We have attempted to keep the format of the survey simple with short questions and clearly labeled and scaled answer choice-sets.

We will keep the survey short, with no more than 12 questions at any one time, including no more than two open-ended questions.

The set of survey questions included in this package were gathered from (1) previous usability surveys conducted at HHS or (2) recommended usability questions used by other usability professionals in other organizations and are considered best practices. In determining which questions to include in the package, usability professionals across HHS were consulted and questions that had poor performance in the past or were not considered best practices were discarded. Because we are requesting a 3-year generic clearance for a wide variety of possible usability surveys on HHS Web sites, the list of questions is large enough that this package can cover all potential survey scenarios needed. However, as stated above, each survey is limited to a specific number of questions and HHS staff will not incorporate every question in any one survey.

A-4. Efforts to Identify Duplication and Use of Similar Information

No similar information exists, although some usability surveys have been conducted on a few, specific pages of the HHS Web site. Approval of this package will greatly expand HHS’s ability to perform usability surveys on the HHS Web site.

A-5. Impact on Small Businesses or Other Small Entities

There is no burden on small businesses or small entities.

A-6. Consequences of Collecting the Information Less Frequently

There are a number of potential negative consequences if these data are not collected. In addition, if the collection is not conducted on an ongoing basis, we will not have valuable data needed to routinely revise messages and reorganize online health information in a way that is most easily understood and accessed by Web site visitors. Specifically, without this data there would be:

• No performance measures by which to determine effectiveness of the HHS Web sites as a tool for our visitors and message channels for the department. This results in lowered user satisfaction, fewer return visits, and decreased information dissemination.

• No user data to include in Web site design decision-making to ensure that user experience on our site is efficient, effective, and enjoyable. This results in an unfocused approach to Web design in which we are unable to determine whether our site is useful or not.

• Vital feedback regarding customer and/or partner satisfaction with various aspects of the HHS’s services will be unavailable.

Usability surveys will only be conducted at intervals considered appropriate to measure the impact of HHS Web site changes and to monitor the level of performance. We are only expecting one-time responses from respondents. Therefore, it is not possible to ask participants to fill out the survey less frequently.

A-7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this information collection package. This request fully complies with the guidelines of 5 CFR 1320.5.

A-8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

The Agency’s 60-day notice appeared in the Federal Register on Friday, July 13, 2007, in Volume 72, page 38950 as required by 5 CFR 1320.8 (D). No public comments were received in response to the notice.

A-9. Explanation of Any Payments or Gift to Respondents

There sill be no payments or gifts to respondents.

A-10. Assurance of Confidentiality Provided to Respondents

The HHS Privacy Officer reviewed this submission and determined that the Privacy Act does not apply to data collections conducted according to procedures described in this application. All questions for the surveys to be conducted under this OMB approval are included within this Information Collection Request.

All participants will be informed at the beginning of the survey that their responses will be treated in a secure manner, that all data will be safeguarded closely, and that no individual identifiers are planned to be used in survey reports.

All data will be stored in secured electronic files for at least two years and no longer than 10 years.

This project is exempt from IRB requirements.

A-11. Justification for Sensitive Questions

Questions concerning Race and Ethnicity (language preference) may be considered sensitive by a portion of respondents. Race and Ethnicity questions are included in the set of Demographic questions that may be asked of respondents. Where relevant to the evaluation of Web site usability, Race and Ethnicity data will be collected consistent with HHS policy and standard OMB classifications.

A-12. Estimates of Annualized Burden Hours and Costs

The remote survey will last approximately 12 minutes and take place at the participant’s computer. These estimates were determined through analysis of times from previous usability surveys using similar questions and a survey of usability professionals to ascertain average times for users to perform tasks.

Estimate of survey respondents was based on an estimate of the ideal number of usability surveys that HHS would conduct over a 3 year period. Because HHS has not been able to conduct these types of surveys at the level needed previously, it is anticipated that most of HHS’s Web sites will require some sort of usability survey.

Estimates of Annual Burden

	Survey Type	Number of Respondents	Frequency of Response per Respondent	Avg. Burden Per Response (hrs.)	Burden Hours
	Remote Surveys	48,000	One time	12/60	9600

An average hourly salary of approximately $18.09 is assumed for all respondents, including clinicians and scientific users, based on the Department of Labor (DOL) National Compensation Survey. Because of the scope of this generic clearance and the variety of the types of participants, the average salary was utilized rather than attempting to estimate salaries for groups of audiences. With a maximum annual respondent burden of 9,600 hours, the overall annual cost of respondents’ time for the proposed interviews is estimated to be a maximum of $173,664. There will be no direct costs to the respondents other than their time to participate in each survey.

Total Respondent Hours	Hourly pay rate	Total Respondent Burden
9600	$18.09	$173,664.00

A-13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no additional costs to the respondents. There is no burden to record keepers.

A-14. Annualized Cost to the Federal Government

Usability surveys will be prepared by contractors or HHS staff (FTE). An FTE manager will review all surveys. Usability teams will vary across HHS web teams but typically an FTE and contractor will work together on survey preparations, coding the surveys electronically, conducting the surveys, and analyzing of data. Additionally, a senior level FTE will typically review and approve the activities. The amount of time staff and contractors spend on surveys will vary depending on the number of participants for each survey, the number of questions, and the site being surveyed. An average number of 400 web page survey sites a year were assumed for estimation purposes. Overall time spent by HHS staff and contractors is lessened as this package provides tasks and questions to be used in the survey; thus, reducing time staff normally would have spent developing these questions.

Staff or Contractor	Average Hours per Study	Average Hourly Rate	Average Cost
Contractor instrument preparation, conduction, analysis (GS-12/GS-13 equivalent)	3	$36.00	$108.00
FTE survey preparation, conduction, analysis (GS-13)	10	$39.00	$390.00
FTE manager survey review (GS-14)	1	$45.00	$45.00
Average Costs per survey			$543.00
Average 1 Year Cost			$217,200.00

A-15. Explanation for Program Changes or Adjustments

This is a new data collection for HHS and is essential to ensuring that visitors to the HHS Web site are able to expeditiously find information they need.

A-16. Plans for Tabulation and Publication and Project Time Schedule

Activity	Time Schedule
1. Determine which Web site will be surveyed. 2. Determine survey questions. 3. Determine target quotas.	Ongoing
4. Completion of surveys.	Ongoing
5. Analysis of surveys.	Ongoing
6. Adjustment of Web site based on results of the survey.	Ongoing

A-17. Reason(s) Display of OMB Expiration Date is Inappropriate

Exemption is not being sought. The OMB expiration date will be displayed.

A-18. Exceptions to Certification for Paperwork Reduction Act Submission

There are no exceptions to certification.