American Fact Finder Plan

The Census Bureau plans to conduct a usability test of the new American FactFinder (AFF) Web site under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). The objective of this research is to identify issues that are problematic and frustrating to the user and to compare the results to the baseline usability testing that was conducted on the old AFF system. AFF is a free online tool that allows users to find, customize and download Census information. It is available to the public, and a multitude of diverse users search the site seeking a vast range of information.

Over the past two years the developers of the AFF Web site worked with stakeholders and the usability lab to release a new web site. The release occurred in mid-February 2011. This study is intended to be a “baseline” on the new version of the site to understand how it performs from the user’s perspective with respect to the old version. The study will use the same tasks and protocol that was used on the baseline of the old system so that we will be able to compare the user performance on the new site with respect to user performance on the old site. The Census Bureau is interested in continuing iterative testing to tweak the new site. The previous study was submitted to OMB for approval in a letter dated October 9, 2008.

A selection of the usability findings from the study of the old version of the AFF follows:

• The Search function was not helpful to users. People said that they wanted the search function to help them when they could not find answers, but it seldom did. Users often created a google type query using plain language and were frustrated when the search returned a “no results” answer.

• The maps were difficult for users. The grey writing on the maps against the grey background was difficult for users to read. Three of the novice users tried to obtain a city map by typing in the city name in the address box but were frustrated when they could not access the city map and instead were prompted for a zip code.

• Terminology issues confused users once they had navigated off the main page. Some of the terminology that confused users included NAICS, SHP, ASM, GS, data revision notices, 2004 value and 2004 percent share.

Many of these issues were known issues as usability testing had been conducted throughout the years on the AFF Website. The new design of the AFF attempted to address these issues in a general way. The usability testing of the new AFF site will highlight if these issues remain as well as what new usability issues were created with the re-design of the site.

During April and May, staff from the Census Bureau’s usability lab will interview 20 external participants from the Washington DC metropolitan area. We will recruit two different user groups: novice users of census data who have a minimum of one year of Internet experience and use the Internet at least three times a week to search for information; and expert users of census data who use census data for work purposes. Novice participants will be recruited from the Usability Lab database, which is composed of people from the metropolitan DC area who volunteered to participate after responding to a craigslist.com posting or an ad in a local newspaper. Expert users will be recruited both by the Usability Lab database, and by recommendations from contacts made by the Census Regional Offices or census partners. Participants will come to the Usability Lab at the Census Bureau for the study.

Participants will be given a paper questionnaire about their Internet experience and some demographic characteristics. Then each participant will be given a set of tasks on paper (the same set of tasks that was used in the initial baseline study) for the Web site. Tasks will be randomized such that no participant will receive the tasks in the same order. Participants will be asked to think-aloud while they are working on the tasks and they will be prompted to think-aloud when they fall silent. At the conclusion of the study, participants will be asked to complete a paper questionnaire designed to measure their satisfaction with the new version of the AFF site. Subjective satisfaction ratings will be collected for such design elements as the layout of page, ease of finding information, and use of Census jargon. Finally, participants will provide verbal feedback about the Web site during a debriefing with the test administrator at the conclusion of the session. A copy of the initial questionnaire, the tasks set, the satisfaction questionnaire, and the debriefing questionnaire are enclosed.

Respondents will be informed that their involvement is voluntary and that the information they provide is confidential and will be seen only by employees involved in the research project. Participants will be videotaped and the recordings will be seen only by employees involved in the research project. This data is being collected under the authority of Title 13. Participants will be compensated $40.00 for their participation.

We estimate that users will spend one hour on average taking the study, including time spent working on the demographic and satisfaction questions, the tasks and the debriefing. Thus, the total estimated respondent burden for this test is 20 hours.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Erica Olmsted-Hawala

Center for Survey Methods Research

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-4893

[email protected]