Supporting Statement

Department of Commerce (DOC)

National Institute of Standards and Technology (NIST)

Generic Clearance for Usability Data Collections

OMB Control No. 0693-0043

Expiration Date: 10/31/2012

Character String Typing Study Task Evaluation

Supporting Statement

1. Explain who will be surveyed and why the group is appropriate to survey.

The Information Technology Laboratory’s (ITL) Computer Security Division (CSD) of the National Institute of Standards and Technology (NIST) would like to update the password sections of NIST Special Publication 800-63 Electronic Authentication Guideline. Currently the password sections discuss only security considerations. CSD would also like to include usability considerations in the revision. These usability considerations should be based on empirical data on memory, character string and password usage. This protocol is a first step in collecting empirical data to complement the security data.

As part of a usability study, we intend to recruit 200 individuals using an existing contract, which provides a database of over 10,000 people that have participated in previous usability studies. The participants will perform a character string typing task where they are asked to memorize ten character strings and then type them into a software package where we measure speed and accuracy. A statistical power analysis was completed to determine the ability of the test to detect when (1) typing time increases as character string length increases; (2) typing accuracy drops to below 20% when the character string length is longer than 10 characters. The analysis concluded that 200 individuals was a sufficient number to detect the effect. The survey data to be collected from each individual will identify which strings were the most difficult to memorize.

2. Explain how the survey was developed including consultation with interested parties, pretesting, and responses to suggestions for improvement.

This questionnaire has been developed based on standard templates used by our usability group. Similar questions were piloted, and validated in a previous approved usability studies. We have incorporated the feedback and suggestions from the previous studies into the task evaluation questions and believe the form is highly usable in its current form.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

Prior to participation, all participants sign a consent form that fully explains the study and the survey. The participants will be given a Pre-Task Questionnaire to determine usage and familiarity with smart phone and keyboard functions. After each participant has memorized the ten strings and typed them into the system, a paper copy of the survey instrument, Character String Typing Study Task Evaluation Questionnaire (Post-Task) will be provided to the participant for completion The questions are based on standardized methods in the usability field, specifically the Questionnaire for User Interaction Satisfaction (QUIS) (http://www.lap.umd.edu/QUIS/index.html).

The expected response rate will be 100% since each participant will be provided the survey by the test facilitator and will complete the survey as part of the overall usability test.

The estimated response time for the Pre- and Post-Task Questionnaires is 15 minutes each,

200 respondents X 2 responses X 15 minutes = 100 hours.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The data in the task evaluations will be subjected to statistical analysis and form a primary outcome of the experiment. We intend to perform an analysis of variance to identify the sources of variability from one or more of the factors. By varying the factors in a predetermined pattern and analyzing the output, we plan to make an accurate assessment as to the cause of the variation in the accuracy and speed of the typing of the character strings. The data collected will be used to update the guidance on password length and the use of special characters in passwords for password policies in NIST Special Publication 800-63.