ICR 2535-0116 Part A

ICR 2535-0116 Part A.doc

Generic Customer Satisfaction Surveys

OMB: 2535-0116

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 2535-0116 can be found here:

Document [doc]

Download: doc | pdf

Supporting Statement for OMB Approval

HUD.GOV Customer Satisfaction Website Assessment Survey

Part A: Justification

A.1 Explanation of the Circumstances that Make Data Collection Necessary

Executive Order 12862 directs federal agencies that provide significant services directly to the public to survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services (see Appendix A). The Office of Departmental Operation and Coordination (ODOC) of the Department of Housing and Urban Development (HUD) thereby requests approval of a customer satisfaction survey of its customers, the users of the Department’s primary website, www.hud.gov.

The HUD.GOV Internet site is the Department’s primary method for public dissemination of information, services, and data from HUD. The Department's internet presence serves over 2.1 million unique visitors a month who register approximately 3.75 million visits, providing information, services, and access to over 50 different applications 24 hours a day, 7 days a week to its customers. It is a critical component in how the Department conducts daily business and carries out its mission.

There is a clear need to measure user satisfaction with the HUD.GOV website. In addition to compliance with GPRA and Executive Order 12862, actual audience feedback is needed for a planned redesign of the website, marketing the website, and targeting information and services that are needed by the public. In light of this, a customer satisfaction survey is proposed aimed at evaluating user satisfaction with the content, navigation and overall usability of the website.

The proposed survey’s core questions seek to obtain numerical ratings from respondents on their satisfaction with various aspects of the website. For example, there will be questions on satisfaction with the usefulness (including clarity and ease of reading) of information on the site and use of the navigation tools. There are questions regarding how the website is used by the respondent. Visitors will be asked how easy it was to find what they were looking for, and if what they found was helpful. All respondents will be asked for suggestions for improvement. Questions on the respondent’s demographics are included, which will enable the Department to get a better understanding of the audience HUD.GOV serves.

Response to the survey questions will enable the Department to determine which aspects of the website are in need of improvements, as well as determine the barriers to usage and identify future site improvements. A rating system is intended to aid the in identification of satisfaction in particular areas and establish a baseline for follow-up surveys.

A.2 Use of the Data

As previously discussed, the main purpose of the proposed customer satisfaction survey is to obtain feedback in order to provide a website that is easy to access, easy to navigate, clear, informative, and useful. Specifically, the research will examine whether the information is presented in an easily accessible format and whether it meets the needs of users of www.hud.gov. The survey will also provide a means to characterize site visitors, how to serve them, and determine what they are looking for. It will serve as the guideline for future enhancements to the website.

In addition to customer satisfaction, the surveys will ask respondents to identify the strengths and weaknesses of the HUD.GOV website. This level of detail will assist in identifying needed improvements and assist with marketing, outreach, and redesign efforts.

The respondent universe is composed of all visitors to the website during a period of no more than six weeks. Participation in the customer satisfaction survey is completely voluntary, so respondents are self-selected, and it is hoped that any biases resulting from self-selection will be overridden by a large number responding. Since an attempt will be made to obtain a census of users, inferential statistical analysis will not be needed. As such, statistical power is not germane to this project.

A.3 The Extent of the Use of Technological Collection Techniques or Other Forms of Information Technology

Automated information technology will be used to collect and process data. Responses to the survey instrument presented in Appendix B and C will be sent over the Internet. The survey will be available online for at least two weeks, but not more than four.

A.4 Efforts to Identify Duplication

There is no duplication with other existing data collection instruments, nor is the data to be collected through the surveys available elsewhere. Currently, there is no existing quantitative research project that collects feedback from customers of the website. This will be the first data collection for users of the HUD.GOV website.

A.5 Collection of Information Impacts on Small Businesses or Other Small Entities

There is no requirement that small businesses or other small entities respond to the proposed surveys. While individuals working in small businesses may be respondents to the survey, it would only be the result of their voluntary choice to do so while visiting the website. In any case, the impact on any respondent should be small due to the short length of time required to respond to the survey instrument, usually less than two minutes.

A.6 Describe the Consequences to Federal Program or Policy Activities if the Collection is Not Conducted or is Conducted Less Frequently

The Government Performance Results Act of 1993 (GPRA) requires all Federal agencies to focus on results, services, and customer satisfaction.^¹ If the collection is not conducted, compliance with GPRA will be diminished. Valuable information that could be used to improve the website would not be available if the surveys are not conducted.

A.7 Explain Any Special Circumstances that Would Cause Information Collection to be Conducted in a Special Manner

There are no known or anticipated special circumstances that would require data collection in a manner inconsistent with OMB guidelines and the Privacy Act.

A.8 Publication and Consultation

A.8.1 Required Publication in the Federal Register

Publication in the Federal Register was not required.

A.8.2 Consultation with Persons Outside of the Agency

Mainstreet Technologies, Inc. is currently under contract to HUD to support this effort, including all programming needed to field the surveys. This company prepared the survey instruments in accordance with industry-standard website evaluation techniques and submitted them to HUD for review and approval.

A.9 Payments or Gifts to Respondents

There will be no offer of payment or gifts to respondents who choose to take part in the customer satisfaction survey.

A.10 Assurance of Confidentiality

The survey will be confidential and no attempt will be made to identify or track individual respondents. At the end of the survey, individuals will be given the option of participating in a user Task Scenario to assess the usability of the HUD.GOV web site. This is not required portion of the survey. If the user chooses to participate in a user Task Scenario they will be asked to provide their name, e-mail address, phone number. This information will only be used in conjunction with the user Task Scenario and destroyed after completion. Confidentiality of responses will be maintained throughout the data collection. Any demographic information supplied by respondents will be used for analytical purposes only. The user task scenario is estimated to take no more than 15 minutes. Participants will be asked to complete 1 to 3 tasks related to a specific user group such as prospective homebuyer, a current homeowner, or an agency seeking a grant.

A link to an OMB burden statement will be provided to the respondent prior to respondent going to the survey online. The OMB burden statement will include average response time and the voluntary nature of the survey as well as explain the use of cookies and how the data will be used. The statement is presented in Appendix E below.

A.11 Sensitive Information

No questions of a sensitive nature will be asked of respondents.

A.12 Burden and Cost

A.12.1 Estimate of Hour Burden of Information Collection on Respondents

It is estimated that participants will require no more than an average of two minutes to complete the survey. Actual time required will vary based on participant reading speed and level. This length is based upon current estimates and will be verified by a survey pilot test. The survey will be online for a period of at least two weeks but not more than four weeks starting December 2008.

The annual burden table below shows a total annual burden of 300 hours to participants.

	Website Survey
Average Number of visits per month	3,750,000
Number of Respondents	10,000
Average Time for Respondent to Complete	.03 hours (2 minutes)
Total Hours of Respondent Burden	300

Table A.12.1: Estimate of Respondent Burden

A.12.2 Approval for a Single Form

This request for OMB approval is for a single customer satisfaction survey.

A.12.3 Estimate of Costs to Respondents for Burden of Information Collection

There is no cost to the respondents to complete the survey.

A.13 Costs to Respondents for Record Keeping

There are no costs to the respondents for record keeping.

A.14 Annualized Costs to the Federal Government

The total cost is $93,880. This is for staff time for research design, development of the survey instrument, pretesting, development of a draft Office of Management and Budget clearance package, data collection, data entry, data analyses, development of a final report, and project management. It was initially estimated that 320 hours would be needed for these activities.

A.15 Reason for any Program Changes or Adjustments Reported in A.13 or A.14

This is a new data collection.

A.16 Analysis Plan and Time Schedule for Project

A.16.1 Analysis Plan

The survey will be administered in December 2008/January 2009. The field period for this study is planned for two weeks, but may extend to four weeks if responses fall short of the desired goal of 10,000 completed surveys. Analysis will focus on user satisfaction, an overall profile of users, and effectiveness of the website. There will be special attention to suggestions for improvement.

During the field period, response rates and raw responses will be monitored to ensure achieving the desired response rate and high quality of data. After the end of the field period, collected data will be tabulated, cleaned and prepared for analysis. The objective of the analysis is to develop recommendations that will help the Department with prioritizing website enhancement and improvement efforts. The first analysis will examine frequencies of all survey question responses. This will assist in developing a profile of respondents and to obtain a general understanding of users’ attitudes towards the website, its features and offered products and services. The second analysis will be to perform cross-tabulation of all items pertaining to satisfaction by users and any other variables of interest. Cross-tabulation of data will provide more insight into the different groups of users and how their level of satisfaction with the website and level of engagement in different operations with the website vary. Last, open-ended response to the “comments and suggestions” question will be read and their comments will be analyzed and categories of suggestions and comments by the users will be reported.

A.16.2 Time Schedule for Project

The schedule for major activities in this project will be the following:

Activity	Date
Award Contract # S8QACCA0071 for HUD.GOV Website Assessment	September 30, 2008
Pretest of survey	December 17, 2008
Survey activated online	December 22, 2008
Determination as to whether or not to extend survey availability by two weeks	January 5, 2009
Submission of Final Report	February 20, 2009

Table A.16.2: Schedule for Project

A.17 Seeking Approval Not to Display OMB Expiration Date

No approval is sought.

A.18 Consistency with 5 CFR 1320.6

The data collection will be conducted according to the guidelines specified in 5 CFR 1320.6. No special circumstances that would require inconsistency with these guidelines are known or anticipated.

Part B: Collection of Information Employing Statistical Methods

B. 1.1 Potential Respondent Universe

All visitors to the HUD.GOV website in December 2008/January 2009 have the potential to be a respondent. Approximately 10% of visitors will randomly be selected to be offered the opportunity to complete the online survey during their visit to the website.

B. 1.4 Expected Response Rate

The www.hud.gov website receives approximately 3.75 million visits each month. Since we anticipate offering the survey for a two-week period, this reduces the pool of visits/visitors to 1.875 million. Approximately 10% (187,500) will randomly be selected to receive the survey and of these we expect a 5% response rate or approximately 10,000 completed surveys. Should we not receive 10,000 completed surveys within the two-week period, we would extend the survey for up to an additional two weeks to ensure a robust sample. Existing literature suggests that web assessment surveys have response rates from eight to twenty percent, depending on the type of audience (see Appendix E). However, a similar survey was performed to assess the website of the National Library of Medicine (NLM) home page in 2002, which yielded a response rate of about five percent.^¹

B.2.1 Procedure for Data Collection

The website survey will be live for a period of two weeks, but it will not exceed four weeks. Not every website visitor will be shown the survey. The survey will be randomly generated, both in terms of visitors being asked to take the survey and where in the site the invitation is presented to them. This invitation will be shown requesting feedback from the visitor (see Appendix F). There will be an option to decline the survey. If a visitor accepts the invitation to the survey, he/she will be shown the survey. Those declining to complete the survey will continue with their normal tasks without interruption.

B.3 Describe Methods to Maximize Responses Rates

The questionnaire shown in Appendixes B and C has been developed to provide a maximum response rate among potential participants. To achieve high response rates several steps will be taken:

First, the web survey is designed to be customer friendly. The navigation through the survey is simple and average response time will be less than two minutes.

Second, to reduce the burden on respondents, all users will be given the choice to not take the survey.

Third, the questions will be customized to the website. Questions pertinent to their visit to www.hud.gov only will be displayed to the users.

These procedures are consistent with contemporary strategies in the field of web survey research (see Appendix E). Many visitors to the HUD.GOV website have a personal or professional interest in the success of their ability to retrieve information or conduct business, which should enhance their willingness to respond.

B.4 Describe Tests of Procedures

Once OMB approval is obtained, and prior to programming the final survey questionnaires into the website, the HUD will pilot test it with 10 to 15 respondents.

B.5 Statistical Consultants and Information Collection Agents

The HUD.GOV Customer Satisfaction Website Assessment survey has been developed by HUD in conjunction with a HUD contractor, Mainstreet Technologies, Inc.. This small business is responsible for all data collection and data analysis.

Appendix A: Executive Order 12862

Title:Executive Order 12862: Setting Customer Service Stds.
Author: The White House
Date: 11 Sept 1993
Content-Type: text
Content-Length: 4113
Apparently-To: [email protected]
Status: RO
Subject: Executive Order of Sept 11, 1993 Setting Customer Service Stds.

THE WHITE HOUSE

Office of the Press Secretary

For Immediate Release
September 11, 1993

EXECUTIVE ORDER

SETTING CUSTOMER SERVICE STANDARDS

Putting people first means ensuring that the Federal Government provides the highest quality service possible to the American people. Public officials must embark upon a revolution within the Federal Government to change the way it does business. This will require continual reform of the executive branch's management practices and operations to provide service to the public that matches or exceeds the best service available in the private sector.

NOW, THEREFORE, to establish and implement customer service standards to guide the operations of the executive branch, and by the authority vested in me as President by the Constitution and the laws of the United States, it is hereby ordered:

Section 1. Customer Service Standards.

In order to carry out the principles of the National Performance Review, the Federal Government must be customer-driven. The standard of quality for services provided to the public shall be: Customer service equal to the best in business. For the purposes of this order, "customer" shall mean an individual or entity who is directly served by a department or agency. "Best in business" shall mean the highest quality of service delivered to customers by private organizations providing a comparable or analogous service.

All executive departments and agencies (hereinafter referred to collectively as "agency" or "agencies") that provide significant services directly to the public shall provide those services in a manner that seeks to meet the customer service standard established herein and shall take the following actions:

identify the customers who are, or should be, served by the agency;
survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services;
post service standards and measure results against them;
benchmark customer service performance against the best in business;
survey front-line employees on barriers to, and ideas for, matching the best in business;
provide customers with choices in both the sources of service and the means of delivery;
make information, services, and complaint systems easily accessible; and
provide means to address customer complaints.

Sec. 2. Report on Customer Service Surveys.

By March 8, 1994, each agency subject to this order shall report on its customer surveys to the President. As information about customer satisfaction becomes available, each agency shall use that information in judging the performance of agency management and in making resource allocations.

Sec. 3. Customer Service Plans.

By September 8, 1994, each agency subject to this order shall publish a customer service plan that can be readily understood by its customers. The plan shall include customer service standards and describe future plans for customer surveys. It also shall identify the private and public sector standards that the agency used to benchmark its performance against the best in business. In connection with the plan, each agency is encouraged to provide training resources for programs needed by employees who directly serve customers and by managers making use of customer survey information to promote the principles and objectives contained herein.

Sec. 4. Independent Agencies.

Independent agencies are requested to adhere to this order.

Sec. 5. Judicial Review.

This order is for the internal management of the executive branch and does not create any right or benefit, substantive or procedural, enforceable by a party against the United States, its agencies or instrumentalities, its officers or employees, or any other person.

WILLIAM J. CLINTON

THE WHITE HOUSE,
September 11, 1993

Appendix B: Web Customer Satisfaction Survey

HUD.GOV Website Assessment Survey

Thank you for agreeing to participate in the survey. Please answer the following questions as they relate to your experience with www.hud.gov:

General Questions

1. Please rate the layout of this site.

Poor				Excellent	No Opinion
1	2	3	4	5
0	0	0	0	0	0

2. Please rate the ease of reading this site.

Extremely difficult				Very Easy	No Opinion
1	2	3	4	5
0	0	0	0	0	0

3. Please rate how well the information is organized.

Poorly				Extremely well	No Opinion
1	2	3	4	5
0	0	0	0	0	0

4. Please rate your ability to find the information you want on this site.

Extremely difficult				Very easy	No Opinion
1	2	3	4	5
0	0	0	0	0	0

5. Please rate the speed of loading the page on this site.

Extremely slow				Very quickly	No Opinion
1	2	3	4	5
0	0	0	0	0	0

6. What is your overall satisfaction with this site?

Not at all satisfied				Extremely satisfied	No Opinion
1	2	3	4	5
0	0	0	0	0	0

7. How well does this site meet your expectations?

Not at all				Very well	No Opinion
1	2	3	4	5
0	0	0	0	0	0

8. How likely are you to return to this site?

Not likely				Extremely likely	No Opinion
1	2	3	4	5
0	0	0	0	0	0

9. How likely are you to recommend this site to someone else?

Not likely				Extremely likely	No Opinion
1	2	3	4	5
0	0	0	0	0	0

10. What specific information were you looking for? (Note: This will be a free-text area)

11. Did you find the specific information that you were looking for?

_____ Yes

_____ No

_____ Don’t Know

12. Please rate the ease of navigation on this site.

Extremely Difficult				Very easy	No Opinion
1	2	3	4	5
0	0	0	0	0	0

13. Please rate the accuracy of information on this site.

Extremely inaccurate				Very accurate	No Opinion
1	2	3	4	5
0	0	0	0	0	0

14. Please rate the quality of information on this site.

Poor				Excellent	No Opinion
1	2	3	4	5
0	0	0	0	0	0

15. Please rate the usefulness of the information provided on this site.

Not at all useful				Very useful	No Opinion
1	2	3	4	5
0	0	0	0	0	0

16. Please rate your ability to accomplish what you wanted to on this site.

Not able to				Was able to with ease	No Opinion
1	2	3	4	5
0	0	0	0	0	0

17. Was the number of steps to get where you want acceptable?

_____ Yes

_____ No

_____ Don’t Know

18. Please rate the ease of finding the information you want on this site.

Extremely difficult				Very easy	No Opinion
1	2	3	4	5
0	0	0	0	0	0

19. What can we do to improve HUD.gov? (Note: This will be a free text area)

20. I would be willing to participate in a user Task Scenario to assess the usability of this web site. The user task scenario is estimated to take 15 -30 minutes. Participants will be asked to complete 1 to 3 tasks related to a specific user group such as prospective homebuyer, a current homeowner, or an agency seeking a grant.

If you would like to participate, please provide your: 1) name; 2) e-mail address and 3) phone number.

Appendix C: Web Customer Satisfaction Survey

Demographic Survey Questions

Highest Education Level
1. Some High School
2. High school degree
3. Trade or Vocational certificate
4. Some college (community or higher)
5. College degree (associate level)
6. College degree (bachelor level)
7. Some graduate school
8. Graduate Degree (Master or Higher)
9. Professional Degree (e.g. law, medicine, business, finance, engineering, pharmacy, public health)
Age
1. Under 18,
2. 18-25
3. 26-35
4. 36-45
5. 46-55
6. 56-62
7. 63-65
8. 66-75
9. 76-85
10. Over 85
Gender
1. M
2. F
Race
1. Spanish, Hispanic, Latino
2. Mexican, Mexican-American, Latino
3. Puerto Rican
4. Other Spanish/Hispanic/Latino
5. White
6. Black, African-American, Negro
7. American Indian or Alaska Native
8. Asian Indian
9. Chinese
10. Other Race (identify)
My Native Language is:
1. English
2. Arabic
3. Chinese
4. French
5. German
6. Polish
7. Portuguese
8. Russian
9. Spanish
10. Vietnamese
11. Other (identify)
What is your employment status?
1. Employed Full-Time
2. Employed Part-Time
3. Self Employed
4. Not Employed but Looking for Work
5. Not Employed not Looking for Work
6. Retired
7. Student
8. Military
9. Homemaker
10. Prefer not to answer
Are you a U.S. Veteran?
1. Yes
2. No
Which of the following categories best describes your total household income, before taxes?
1. Less than $12,000
2. $12,001 - $20,0001
3. $20,001 - $30,000
4. $30,001 - $40,000
5. $40,001 - $50,000
6. $50,001 - $60,000
7. $60,001 - $70,000
8. $70,001 - $80,000
9. $80,001 - $90,000
10. $90,001 - $100,000
11. More than $100,000
12. Prefer not to answer
What State do you live in?
1. All States will be listed
2. Other (identify)
Please identify your HUD Customer Group (more than one answer may apply):
1. Homebuyer
2. Homeowner
3. Landlord
4. Military or Veteran
5. Person with Disabilities (1 or more)
6. Public Safety (e.g., Law enforcement, fire department, medical)
7. Native American
8. Researcher
9. Senior (over 62)
10. Student
11. Tenant
12. Victim of Discrimination
13. Youth (under 18)
14. Other, please specify (in a textbox)
Please identify the Housing Industry group that applies (more than one answer may apply):
1. Appraiser
2. Housing Developer
3. Housing Agency (e.g., Public Housing Agency)
4. Housing Inspector
5. Lender (e.g., a mortgage bank, credit union or other lending institution)
6. Tribe (e.g., Native American, Native Alaskan)
7. Real Estate or Broker Agent
8. Not Applicable (not a member of the housing industry)
9. Other, please specify (in a textbox)
Please identify your HUD Partner Group membership (select at least one response):
1. Auditor
2. Congress
3. Elected Government Official (state or local)
4. Faith-Based or Community Organization
5. Fair Housing
6. Hospital
7. HUD Grantee (i.e, recipient or applicant for a HUD Grant)
8. Investor
9. Non-Profit Organization
10. Small Business
11. Not Applicable (not a member of any of these HUD Partner Groups)
12. Other, please specify (in a textbox)

Appendix D: OMB Burden Statement

The Government Paperwork Elimination Act (GPEA) (P.L. 105-277, Title XVII) allows citizens to use electronic technologies when filing information with, or retrieving it, from the Federal Government. The online mode of information collection will assist HUD with complying with the provisions of GPEA. Public reporting burden for this collection of information is estimated to average two minutes per response. Responses are voluntary. Every effort will be made to maintain confidentiality, which is explained below. This agency may not collect, and you are not expected to respond, unless the survey displays a currently valid OMB control number.

Many websites use "cookies" to help personalize a respondent's online experience. A cookie is a text file that is placed on the hard drive of a computer by a web page server. Cookies cannot be used to run programs or deliver viruses to your computer. Cookies are uniquely assigned to you, and can only be read by a web server in the domain that issued the cookie to you. You have the ability to accept or decline cookies. Most web browsers automatically accept cookies, but you can usually modify your browser setting to decline cookies if you prefer.

The Customer Satisfaction Survey regarding www.hud.gov does not use cookies in any way.

Appendix E: Bibliography

Comley, P. (2000). Pop-up surveys. What works, what doesn’t work and what will work in the future. Proceedings of the ESOMAR worldwide Internet conference Net Effects 3. Publication series - Volume 237. Amsterdam, NL: ESOM

Hogg, Allen, and Jaci Jarrett Masztal: Drop-Down, Radio Button, or Fill-in-the-Blank? Effects of Attribute Rating Scale Type on Web Survey Responses. ESOMAR Congress, Rome, September 2001.

Ethan Rapp: The Catch-22 of Online Survey Research

Kenneth W. Mentor: Internet-Based Survey Research. Presented at the Annual Meetings of the Academy of Criminal Justice Sciences, March 2002, Anaheim, CA

Tedesco, H., R. L. Zuckerberg, and E. Nichols, 1999. “Designing Surveys for the Next Millenium: Web-Based Questionnaire Design Issues.” Proceedings of the Third ASC International Conference, Edinburgh, September, pp 103-12.

Couper, M P. 2000, “Web Surveys - A Review of Issues and Approaches.” Public Opinion Quarterly 64:464-494

Dillman, Don A., Robert D. Tortora, and Dennis Bowker. In Press. “Influence of Plain vs. Fancy Design on Response Rates for Web Surveys.” Proceedings of Survey Methods Section, 1998 Annual Meetings of the American Statistical Association, Dallas, Texas.

Dillman, Don A. and Dennis K. Bowker. 2001. The Web Questionnaire Challenge to Survey Methodologists in Dimensions of Internet Science. Edited by Ulf-Dietrich Reips & Michael Bosnjak..

Appendix F: Web Survey Invitation

Thank you for visiting www.hud.gov.

We need your feedback and assistance so we can make our website more useful to you. Would you please take a short survey? The survey should take about 1-2 minutes of your time. The feedback you provide will help us enhance our site and serve you better in the future. All results will be kept strictly confidential.

If you have any questions or concerns about this survey, please email us at [email protected].

Yes, I will help

Decline

1 Government Performance Results Act of 1993, Section 2b, (3). Web citation: www.whitehouse.gov/OMB/mgmt-gpra/gplaw2m.html

1 Minutes Of The Board Of Regents Meeting on September 10-11, 2002. National Library Of Medicine. National Institutes Of Health, Department Of Health And Human Services

File Type	application/msword
File Title	NOTES:
Author	Laurent V. Hodes
Last Modified By	h13462
File Modified	2008-12-12
File Created	2008-12-12

ICR 2535-0116 Part A