Part A

12

Supporting Statement for OMB Approval

HUD.GOV Customer Satisfaction Website Assessment Survey

Part A: Justification

A.1 Explanation of the Circumstances that Make Data Collection Necessary

Executive Order 12862 directs federal agencies that provide significant services directly to the public to survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services (see Appendix A). The Office of Departmental Operation and Coordination (ODOC) of the Department of Housing and Urban Development (HUD) thereby requests approval of a customer satisfaction survey of its customers, the users of the Department’s primary website, www.hud.gov.

The HUD.GOV Internet site is the Department’s primary method for public dissemination of information, services, and data from HUD. The Department's internet presence serves over 2.1 million unique visitors a month who register approximately 3.75 million visits, providing information, services, and access to over 50 different applications 24 hours a day, 7 days a week to its customers. It is a critical component in how the Department conducts daily business and carries out its mission.

There is a clear need to measure user satisfaction with the HUD.GOV website. In addition to compliance with GPRA and Executive Order 12862, actual audience feedback is needed for a planned redesign of the website, marketing the website, and targeting information and services that are needed by the public. In light of this, a customer satisfaction survey is proposed aimed at evaluating user satisfaction with the content, navigation and overall usability of the website.

The proposed survey’s core questions seek to obtain numerical ratings from respondents on their satisfaction with various aspects of the website. For example, there will be questions on satisfaction with the usefulness (including clarity and ease of reading) of information on the site and use of the navigation tools. There are questions regarding how the website is used by the respondent. Visitors will be asked how easy it was to find what they were looking for, and if what they found was helpful. All respondents will be asked for suggestions for improvement. Questions on the respondent’s demographics are included, which will enable the Department to get a better understanding of the audience HUD.GOV serves.

Response to the survey questions will enable the Department to determine which aspects of the website are in need of improvements, as well as determine the barriers to usage and identify future site improvements. A rating system is intended to aid the in identification of satisfaction in particular areas and establish a baseline for follow-up surveys.

A.2 Use of the Data

As previously discussed, the main purpose of the proposed customer satisfaction survey is to obtain feedback in order to provide a website that is easy to access, easy to navigate, clear, informative, and useful. Specifically, the research will examine whether the information is presented in an easily accessible format and whether it meets the needs of users of www.hud.gov. The survey will also provide a means to characterize site visitors, how to serve them, and determine what they are looking for. It will serve as the guideline for future enhancements to the website.

In addition to customer satisfaction, the surveys will ask respondents to identify the strengths and weaknesses of the HUD.GOV website. This level of detail will assist in identifying needed improvements and assist with marketing, outreach, and redesign efforts.

The respondent universe is composed of all visitors to the website during a period of no more than six weeks. Participation in the customer satisfaction survey is completely voluntary, so respondents are self-selected, and it is hoped that any biases resulting from self-selection will be overridden by a large number responding. Since an attempt will be made to obtain a census of users, inferential statistical analysis will not be needed. As such, statistical power is not germane to this project.

A.3 The Extent of the Use of Technological Collection Techniques or Other Forms of Information Technology

Automated information technology will be used to collect and process data. Responses to the survey instrument presented in Appendix B and C will be sent over the Internet. The survey will be available online for at least two weeks, but not more than four.

A.4 Efforts to Identify Duplication

There is no duplication with other existing data collection instruments, nor is the data to be collected through the surveys available elsewhere. Currently, there is no existing quantitative research project that collects feedback from customers of the website. This will be the first data collection for users of the HUD.GOV website.

A.5 Collection of Information Impacts on Small Businesses or Other Small Entities

There is no requirement that small businesses or other small entities respond to the proposed surveys. While individuals working in small businesses may be respondents to the survey, it would only be the result of their voluntary choice to do so while visiting the website. In any case, the impact on any respondent should be small due to the short length of time required to respond to the survey instrument, usually less than two minutes.

A.6 Describe the Consequences to Federal Program or Policy Activities if the Collection is Not Conducted or is Conducted Less Frequently

The Government Performance Results Act of 1993 (GPRA) requires all Federal agencies to focus on results, services, and customer satisfaction.¹ If the collection is not conducted, compliance with GPRA will be diminished. Valuable information that could be used to improve the website would not be available if the surveys are not conducted.

A.7 Explain Any Special Circumstances that Would Cause Information Collection to be Conducted in a Special Manner

There are no known or anticipated special circumstances that would require data collection in a manner inconsistent with OMB guidelines and the Privacy Act.

A.8 Publication and Consultation

A.8.1 Required Publication in the Federal Register

Publication in the Federal Register was not required.

A.8.2 Consultation with Persons Outside of the Agency

Mainstreet Technologies, Inc. is currently under contract to HUD to support this effort, including all programming needed to field the surveys. This company prepared the survey instruments in accordance with industry-standard website evaluation techniques and submitted them to HUD for review and approval.

A.9 Payments or Gifts to Respondents

There will be no offer of payment or gifts to respondents who choose to take part in the customer satisfaction survey.

A.10 Assurance of Confidentiality

The survey will be confidential and no attempt will be made to identify or track individual respondents. At the end of the survey, individuals will be given the option of participating in a user Task Scenario to assess the usability of the HUD.GOV web site. This is not required portion of the survey. If the user chooses to participate in a user Task Scenario they will be asked to provide their name, e-mail address, phone number. This information will only be used in conjunction with the user Task Scenario and destroyed after completion. Confidentiality of responses will be maintained throughout the data collection. Any demographic information supplied by respondents will be used for analytical purposes only. The user task scenario is estimated to take no more than 15 minutes. Participants will be asked to complete 1 to 3 tasks related to a specific user group such as prospective homebuyer, a current homeowner, or an agency seeking a grant.

A link to an OMB burden statement will be provided to the respondent prior to respondent going to the survey online. The OMB burden statement will include average response time and the voluntary nature of the survey as well as explain the use of cookies and how the data will be used. The statement is presented in Appendix E below.

A.11 Sensitive Information

No questions of a sensitive nature will be asked of respondents.

A.12 Burden and Cost

A.12.1 Estimate of Hour Burden of Information Collection on Respondents

It is estimated that participants will require no more than an average of two minutes to complete the survey. Actual time required will vary based on participant reading speed and level. This length is based upon current estimates and will be verified by a survey pilot test. The survey will be online for a period of at least two weeks but not more than four weeks starting December 2008.

The annual burden table below shows a total annual burden of 140 hours to participants.

	Website Survey
Average Number of visits per month	3,750,000
Number of Respondents	4,200
Average Time for Respondent to Complete	.03 hours (2 minutes)
Total Hours of Respondent Burden	140

Table A.12.1: Estimate of Respondent Burden

A.12.2 Approval for a Single Form

This request for OMB approval is for a single customer satisfaction survey.

A.12.3 Estimate of Costs to Respondents for Burden of Information Collection

There is no cost to the respondents to complete the survey.

A.13 Costs to Respondents for Record Keeping

There are no costs to the respondents for record keeping.

A.14 Annualized Costs to the Federal Government

The total cost is $93,880. This is for staff time for research design, development of the survey instrument, pretesting, development of a draft Office of Management and Budget clearance package, data collection, data entry, data analyses, development of a final report, and project management. It was initially estimated that 320 hours would be needed for these activities.

A.15 Reason for any Program Changes or Adjustments Reported in A.13 or A.14

This is a new data collection.

A.16 Analysis Plan and Time Schedule for Project

A.16.1 Analysis Plan

The survey will be administered in December 2008/January 2009. The field period for this study is planned for two weeks, but may extend to four weeks if responses fall short of the desired goal of 10,000 completed surveys. Analysis will focus on user satisfaction, an overall profile of users, and effectiveness of the website. There will be special attention to suggestions for improvement.

During the field period, response rates and raw responses will be monitored to ensure achieving the desired response rate and high quality of data. After the end of the field period, collected data will be tabulated, cleaned and prepared for analysis. The objective of the analysis is to develop recommendations that will help the Department with prioritizing website enhancement and improvement efforts. The first analysis will examine frequencies of all survey question responses. This will assist in developing a profile of respondents and to obtain a general understanding of users’ attitudes towards the website, its features and offered products and services. The second analysis will be to perform cross-tabulation of all items pertaining to satisfaction by users and any other variables of interest. Cross-tabulation of data will provide more insight into the different groups of users and how their level of satisfaction with the website and level of engagement in different operations with the website vary. Last, open-ended response to the “comments and suggestions” question will be read and their comments will be analyzed and categories of suggestions and comments by the users will be reported.

A.16.2 Time Schedule for Project

The schedule for major activities in this project will be the following:

Activity	Date
Award Contract # S8QACCA0071 for HUD.GOV Website Assessment	September 30, 2008
Pretest of survey	December 17, 2008
Survey activated online	December 22, 2008
Determination as to whether or not to extend survey availability by two weeks	January 5, 2009
Submission of Final Report	February 20, 2009

Table A.16.2: Schedule for Project

A.17 Seeking Approval Not to Display OMB Expiration Date

No approval is sought.

A.18 Consistency with 5 CFR 1320.6

The data collection will be conducted according to the guidelines specified in 5 CFR 1320.6. No special circumstances that would require inconsistency with these guidelines are known or anticipated.

Part B: Collection of Information Employing Statistical Methods

B. 1.1 Potential Respondent Universe

All visitors to the HUD.GOV website in December 2008/January 2009 have the potential to be a respondent. Approximately 10% of visitors will randomly be selected to be offered the opportunity to complete the online survey during their visit to the website.

B. 1.4 Expected Response Rate

The www.hud.gov website receives approximately 3.75 million visits each month. Since we anticipate offering the survey for a two-week period, this reduces the pool of visits/visitors to 1.875 million. Approximately 10% (187,500) will randomly be selected to receive the survey and of these we expect a 5% response rate or approximately 10,000 completed surveys. Should we not receive 10,000 completed surveys within the two-week period, we would extend the survey for up to an additional two weeks to ensure a robust sample. Existing literature suggests that web assessment surveys have response rates from eight to twenty percent, depending on the type of audience (see Appendix E). However, a similar survey was performed to assess the website of the National Library of Medicine (NLM) home page in 2002, which yielded a response rate of about five percent.¹

B.2.1 Procedure for Data Collection

The website survey will be live for a period of two weeks, but it will not exceed four weeks. Not every website visitor will be shown the survey. The survey will be randomly generated, both in terms of visitors being asked to take the survey and where in the site the invitation is presented to them. This invitation will be shown requesting feedback from the visitor (see Appendix F). There will be an option to decline the survey. If a visitor accepts the invitation to the survey, he/she will be shown the survey. Those declining to complete the survey will continue with their normal tasks without interruption.

B.3 Describe Methods to Maximize Responses Rates

The questionnaire shown in Appendixes B and C has been developed to provide a maximum response rate among potential participants. To achieve high response rates several steps will be taken:

First, the web survey is designed to be customer friendly. The navigation through the survey is simple and average response time will be less than two minutes.

Second, to reduce the burden on respondents, all users will be given the choice to not take the survey.

Third, the questions will be customized to the website. Questions pertinent to their visit to www.hud.gov only will be displayed to the users.

These procedures are consistent with contemporary strategies in the field of web survey research (see Appendix E). Many visitors to the HUD.GOV website have a personal or professional interest in the success of their ability to retrieve information or conduct business, which should enhance their willingness to respond.

B.4 Describe Tests of Procedures

Once OMB approval is obtained, and prior to programming the final survey questionnaires into the website, the HUD will pilot test it with 10 to 15 respondents.

B.5 Statistical Consultants and Information Collection Agents

The HUD.GOV Customer Satisfaction Website Assessment survey has been developed by HUD in conjunction with a HUD contractor, Mainstreet Technologies, Inc.. This small business is responsible for all data collection and data analysis.

Appendix A: Executive Order 12862

Title:Executive Order 12862: Setting Customer Service Stds.
Author: The White House
Date: 11 Sept 1993
Content-Type: text
Content-Length: 4113
Apparently-To: [email protected]
Status: RO
Subject: Executive Order of Sept 11, 1993 Setting Customer Service Stds.

THE WHITE HOUSE

Office of the Press Secretary

For Immediate Release
September 11, 1993

EXECUTIVE ORDER

SETTING CUSTOMER SERVICE STANDARDS

Putting people first means ensuring that the Federal Government provides the highest quality service possible to the American people. Public officials must embark upon a revolution within the Federal Government to change the way it does business. This will require continual reform of the executive branch's management practices and operations to provide service to the public that matches or exceeds the best service available in the private sector.

NOW, THEREFORE, to establish and implement customer service standards to guide the operations of the executive branch, and by the authority vested in me as President by the Constitution and the laws of the United States, it is hereby ordered:

Section 1. Customer Service Standards.

In order to carry out the principles of the National Performance Review, the Federal Government must be customer-driven. The standard of quality for services provided to the public shall be: Customer service equal to the best in business. For the purposes of this order, "customer" shall mean an individual or entity who is directly served by a department or agency. "Best in business" shall mean the highest quality of service delivered to customers by private organizations providing a comparable or analogous service.

All executive departments and agencies (hereinafter referred to collectively as "agency" or "agencies") that provide significant services directly to the public shall provide those services in a manner that seeks to meet the customer service standard established herein and shall take the following actions:

1. identify the customers who are, or should be, served by the agency;

2. survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services;

3. post service standards and measure results against them;

4. benchmark customer service performance against the best in business;

5. survey front-line employees on barriers to, and ideas for, matching the best in business;

6. provide customers with choices in both the sources of service and the means of delivery;

7. make information, services, and complaint systems easily accessible; and

8. provide means to address customer complaints.

Sec. 2. Report on Customer Service Surveys.

By March 8, 1994, each agency subject to this order shall report on its customer surveys to the President. As information about customer satisfaction becomes available, each agency shall use that information in judging the performance of agency management and in making resource allocations.

Sec. 3. Customer Service Plans.

By September 8, 1994, each agency subject to this order shall publish a customer service plan that can be readily understood by its customers. The plan shall include customer service standards and describe future plans for customer surveys. It also shall identify the private and public sector standards that the agency used to benchmark its performance against the best in business. In connection with the plan, each agency is encouraged to provide training resources for programs needed by employees who directly serve customers and by managers making use of customer survey information to promote the principles and objectives contained herein.

Sec. 4. Independent Agencies.

Independent agencies are requested to adhere to this order.

Sec. 5. Judicial Review.

This order is for the internal management of the executive branch and does not create any right or benefit, substantive or procedural, enforceable by a party against the United States, its agencies or instrumentalities, its officers or employees, or any other person.

WILLIAM J. CLINTON

THE WHITE HOUSE,
September 11, 1993

Appendix D: OMB Burden Statement

The Government Paperwork Elimination Act (GPEA) (P.L. 105-277, Title XVII) allows citizens to use electronic technologies when filing information with, or retrieving it, from the Federal Government. The online mode of information collection will assist HUD with complying with the provisions of GPEA. Public reporting burden for this collection of information is estimated to average two minutes per response. Responses are voluntary. Every effort will be made to maintain confidentiality, which is explained below. This agency may not collect, and you are not expected to respond, unless the survey displays a currently valid OMB control number.

Privacy Policy and Use of Cookies:

Many websites use "cookies" to help personalize a respondent's online experience. A cookie is a text file that is placed on the hard drive of a computer by a web page server. Cookies cannot be used to run programs or deliver viruses to your computer. Cookies are uniquely assigned to you, and can only be read by a web server in the domain that issued the cookie to you. You have the ability to accept or decline cookies. Most web browsers automatically accept cookies, but you can usually modify your browser setting to decline cookies if you prefer.

The Customer Satisfaction Survey regarding www.hud.gov does not use cookies in any way.

Appendix E: Bibliography

Comley, P. (2000). Pop-up surveys. What works, what doesn’t work and what will work in the future. Proceedings of the ESOMAR worldwide Internet conference Net Effects 3. Publication series - Volume 237. Amsterdam, NL: ESOM

Hogg, Allen, and Jaci Jarrett Masztal: Drop-Down, Radio Button, or Fill-in-the-Blank? Effects of Attribute Rating Scale Type on Web Survey Responses. ESOMAR Congress, Rome, September 2001.

Ethan Rapp: The Catch-22 of Online Survey Research

Kenneth W. Mentor: Internet-Based Survey Research. Presented at the Annual Meetings of the Academy of Criminal Justice Sciences, March 2002, Anaheim, CA

Tedesco, H., R. L. Zuckerberg, and E. Nichols, 1999. “Designing Surveys for the Next Millenium: Web-Based Questionnaire Design Issues.” Proceedings of the Third ASC International Conference, Edinburgh, September, pp 103-12.

Couper, M P. 2000, “Web Surveys - A Review of Issues and Approaches.” Public Opinion Quarterly 64:464-494

Dillman, Don A., Robert D. Tortora, and Dennis Bowker. In Press. “Influence of Plain vs. Fancy Design on Response Rates for Web Surveys.” Proceedings of Survey Methods Section, 1998 Annual Meetings of the American Statistical Association, Dallas, Texas.

Dillman, Don A. and Dennis K. Bowker. 2001. The Web Questionnaire Challenge to Survey Methodologists in Dimensions of Internet Science. Edited by Ulf-Dietrich Reips & Michael Bosnjak..

Appendix F: Web Survey Invitation

Thank you for visiting www.hud.gov.

We need your feedback and assistance so we can make our website more useful to you. Would you please take a short survey? The survey should take about 1-2 minutes of your time. The feedback you provide will help us enhance our site and serve you better in the future. All results will be kept strictly confidential.

If you have any questions or concerns about this survey, please email us at [email protected].

Yes, I will help

Decline

1 Government Performance Results Act of 1993, Section 2b, (3). Web citation: www.whitehouse.gov/OMB/mgmt-gpra/gplaw2m.html

1 Minutes Of The Board Of Regents Meeting on September 10-11, 2002. National Library Of Medicine. National Institutes Of Health, Department Of Health And Human Services