0693.0043.SuppStatement.030113

SUPPORTING STATEMENT

U.S. Department of Commerce

National Institute of Standards and Technology

Generic Clearance for Usability Data Collections

OMB CONTROL NO. 0693-0043

A. JUSTIFICATION

This is a request to extend the Office of Management and Budget (OMB) approval.

1. Explain the circumstances that make the collection of information necessary.

In accordance with the Office of Management and Budget’s (OMB) regulations at 5 CFR 1320 implementing the Paperwork Reduction Act (PRA), the Government Performance and Results Modernization Act of 2012, and its mission …

“To promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life.”

the National Institute of Standards and Technology (NIST), a non-regulatory agency of the Department of Commerce (DOC), proposes to conduct a variety of usability data collections under this generic collection.

Individual information collection requests submitted under this generic collection may involve usage of technological devices, such as web sites, handheld computers, cell phones and robots. Collecting this type of information will allow NIST researchers to study human-computer interactions and help establish guidelines and standards for effective and efficient interactions.

An example of a previously approved data collection: NIST collected information from the medical community in an effort to evaluate and determine the usability of various commercial Electronic Health Record (EHR) products available for use.

NIST will limit its inquiries to data collections that solicit strictly voluntary opinions and information, and will not – under this PRA clearance request – collect information that is required (mandatory) or regulated.

For each proposed request using the generic clearance, NIST will submit the actual instrument and any related documents (letters, emails to respondents, scripts, etc.) to OMB along with responses to the following questions:

1. Explain who will be surveyed and why the group is appropriate to survey.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The following collections have been previously approved previously, ongoing, and part of this request.

Below is a summary of lessons learned from each individual ongoing ICs:

NIST, ITL, IAD, Fingerprint Directional Symbols Study

Pilot testing began on September 19, 2012. The testing started on October 22, 2012 and the plan is to complete the remainder of the testing at a date to be determined. Thus we have no data to report at this time.

Public Safety Imaging Systems – Human Perceptions Testing

The initial perception test has shown to be successful: the expected correlation between task performance and perception has been established, and a formula relating human perception and imager performance characteristics to task performance established. The survey results (from questions 1.d, 1.e, and 1.f) have allowed us to assess the quality and/or applicability of the images presented to the PTS and, in conjunction with responses to question 3, whether there is a need to modify future perception test protocols and/or imagery. The aggregate response to these questions indicates that no change needs to be made to the protocols although more imagery may need to be added to represent a greater possibility of scenarios. Inputs to questions 1.a, 1.b, and 1.c provided general information on the collective experience of the PTS to provide demographic information on the PTS population relative to the emergency response population at large.

The results indicate that our group is representative. The response to question 2 was used to understand if extenuating circumstances may have affected the perception test; none were discovered.

Usability Benchmarks for Voting Systems Research Project

Additional studies in several regions of the U.S. showed some usability performance differences with this test. Preliminary analysis suggests that voter experience over time affects the performance. An initial study with voters with disabilities showed wide variations in performance and confirmed that the performance test metrics are not suitable for this population due to the difficulties in recruiting sufficiently large numbers of people with similar disabilities for testing. The Election Assistance Commission and its Technical Guidelines Development Committee has asked NIST to investigate the effect of a broader voter demographic on the usability performance testing. In future studies, NIST will recruit individuals across a broader age and education range and include some subjects with vision and dexterity disabilities to investigate additional metrics for evaluation.

NIST Usability of Biometric System

In a usability test to determine if fingerprint self-capture is possible, we found that the real-time feedback fingerprint system is a highly usable system and shows evidence of great potential for fingerprint self-captures. By following the on-screen, real-time instructions, participants quickly learned, felt comfortable and confident to capture their own fingerprints without any assistance. Although the overlay guide did not show consistent advantages or disadvantages with respect to performance, it was perceived as helpful with hand positioning and provided visual feedback about where users’ hands were in relation to the scanner area. The use of the overlay guide is recommended for use in the fingerprint self-capture process since it does not negatively impact performance and is preferred by users.

NIST, ITL, Mobile Character String Typing Task Evaluation

The first evaluation was performed on a mobile phone device. The data shows that the typing of special characters which requires multiple virtual screens was more difficult to type for the participants than regular alphabetic characters. Further analysis is required to determine the possible relationship between errors and character string length. Additional studies are necessary to compare and contrast the difficulties with larger mobile devices such as tablets.

Interacting with Social Media Websites

Pilot testing has begun, however the full data collection will not start until February 2013 due to availability of the usability lab.   Thus we have no data to report on at this time.

NIST In-depth Interview: Perceptions of Online Security

From our first data collection, it is clear that the participants had the awareness about online safety and certainly security had their attention with respect to privacy and the protection of information. They also have some understanding and knowledge about online security but they lack a skill set to protect their identities and information on-line. Providing a skill set will allow them to develop complete mental models that will help them to make predictions and shape the appropriate behaviors to approaching online security.

Our findings also indicate that the participants were primarily aware of or concerned with confidentiality. Yet Federal Information Processing Standards Publication 199, Standards for Security Categorization of Federal Information and Information Systems, defines three security objectives confidentiality, integrity and availability as follows:

1. A loss of confidentiality -- the unauthorized disclosure of information

2. A loss of integrity-- the unauthorized modification or destruction of information and

3. A loss of availability – the disruption of access to or use of information or an information system

Few, if any participants, discussed or described the attributes or characteristics of the objectives of integrity and availability. Even when discussing the work environment the focus was on the disclosure of information.

Future research should identify the skills that will assist users to build the appropriate cyber-security mental models. By doing so, users would be able to replace their partial mental models built from the coping mechanisms and thought processes that they carry from the physical world that are incomplete or inappropriate for the virtual world. As a result, they would be able to build more concrete and complete mental models which would change their cyber-security behaviors.

A comprehensive approach to awareness, education, and training must emphasize the three dimensions of cyber-security. Moreover, by providing an educational component or campaign that provides online and computer users with a clear skill set, the National Initiative Cybersecurity Education can make a large contribution to securing cyber-space.

Study of Electronic Health Record Usability PART 1 – Web Surveys of EHR Usability

Using this survey, NIST’s contractor has collected data on 559 individuals from a single academic medical center that has developed its own EHR system. This survey has helped to gain insight on demographic information about representative EHR users at this institution, the kinds of tasks EHR users perform, and user opinions and suggestions regarding EHR usability.

We plan to collect data at other healthcare organizations to expand the representativeness of this data.

Study of Electronic Health Record Usability PART 2 – Structured EHR Usability Interviews

NIST has not gained insight from this survey to date because it has not yet begun data collection.

Study of Electronic Health Record Usability PART 3 – Direct observations of EHR Users

NIST has not gained insight from this survey to date because it has not yet begun data collection.

NIST , ITL, Electronic Health Record Usability. PART 4 – Usability testing of commercial EHR products

NIST has not gained insight from this survey to date because it has not yet begun data collection.

NIST Feedback Form For P25 ISSI Network Simulation Tool

From the responses received so far, we have been able to identify some improvements in the documentation provided to help the users understand the software tool. It has also been used to inform the users who wanted to be contacted about other related tools developed by NIST.

2. Explain how, by whom, how frequently, and for what purpose the information will be used. If the information collected will be disseminated to the public or used to support information that will be disseminated to the public, then explain how the collection complies with all applicable Information Quality Guidelines.

Data collected by NIST researchers will be used to help plan the direction of future research. The information collected will not be directly disseminated to the public but aspects or portions of the information collected may be used to support research published in various journals and conferences. There will be no attribution to individuals in the analyzed data. Since the data collected from this collection, may involve Human Subjects, these individual collection efforts will need to be vetted through and approved by the NIST Institutional Review Board (IRB) as needed or required.

This information collection and dissemination will comply with the NIST Chief Information Officer (CIO) Information Quality Guidelines and Standards. Quality will be ensured and established at levels appropriate to the nature and timeliness of the information to be disseminated and will include all pre-dissemination reviews, as required by the Information Quality Guidelines and Standards.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological techniques or other forms of information technology.

Whenever possible, improved information technology will be used to reduce burden on NIST’s customers. In addition to traditional data collection methods, NIST will, whenever possible, offer electronic response options via the NIST Internet website and by fax.

The planned use of transactional and electronic web site surveys will substantially contribute to the number of projected responses and associated burden hours.

4. Describe efforts to identify duplication.

NIST has an internal review process that will examine each survey or data collection effort to be conducted under this generic clearance – to prevent internal duplication of effort and to ensure that appropriate data collection instruments are developed. By examining the three-year, NIST-wide requirements and combining those requirements into a generic clearance request, NIST is, to the greatest extent possible, centralizing the administration of its customer’s data collections. This will provide for a more consistent and comprehensive approach. While there may be other surveys or data collections that become the subject of separate clearance requests, NIST is confident that the procedures in place ensure that there will be no duplication. Due the nature of NIST’s unique mission and programs to further, no similar data exists.

Individuals participating in these types of usability data collection efforts may be asked to sign an informed consent document (meaning participants are informed of the data collection and are providing their consent to participate) and no individual will be surveyed more than once for the same research study.

5. If the collection of information involves small businesses or other small entities, describe the methods used to minimize burden.

Some small businesses and other small entities may be involved in these efforts, but NIST will keep the burden to them-as well as on any business, organization, or individual-at a minimum by asking for opinions on a strictly voluntary basis and by asking for only the minimum amount of information needed to evaluate usability and utility of NIST research for measurement and standardization work.

6. Describe the consequences to the Federal program or policy activities if the collection is not conducted or is conducted less frequently.

If these surveys and other data collections were conducted less frequently or not conducted, NIST researchers would not have important information that may hinder the future direction and scope of NIST research as well as achieving NIST’s mission.

.

7. Explain any special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines.

The data collections conducted under this generic clearance will be conducted in accordance with the guidelines stated in 5 CFR 1320.05.

8. Provide citation information for the PRA Federal Register notice that solicited public comments on the information collection prior to this submission. Summarize the public comments received in response to that notice and describe the actions taken by the agency in response to those comments. Describe the efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

A Federal Register Notice soliciting public comment was published on Wednesday, May 16, 2012 (Vol. 77, No. 95, pg. 28853). No comments were received.

9. Explain any decisions to provide payments or gifts to respondents, other than remuneration of contractors or grantees.

NIST will not provide any payment or gift to respondents to any written, telephone, comment card, or other such information collection.  However, if respondents must leave their home or place of business to travel to a specific location, NIST will, on a case-by-case basis, consider a modest remuneration for the participant’s travel expenses.  In such cases, the remuneration may range from $25-$75 per individual, dependent on the data collection and the level and length of participation required of the participants. 

Respondents that participate in information collections conducted in a “laboratory” type-setting (such as in-depth interviews, and usability testing, etc.) under this clearance may receive a small stipend to offset the cost of travel expenses. This practice has proven effective when recruiting subjects to participate in specific research, and is employed by other Federal cognitive laboratories. The average incentive for participation in a one-on-one interview or usability is approximately $40.

10. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation, or agency policy.

Individuals will be assured of confidentiality to the extent permitted by law, including the Freedom of Information Act (FOIA), and if applicable, will be given notice of this via an informed consent document tailored for each individual information collection which is approved by the NIST (IRB) prior to the collection.  All information collected will be stored anonymously without identifiers.  There will be no attribution to individuals in the analyzed data.  The Code of Federal Regulations Title 15, Part 27, Protection of Human Subjects, describes the general requirements for informed consent.  However, all surveys and, therefore, information provided by respondents will be completely voluntary.

The Privacy Act (5 U.S.C. § 552a €(3)) only requires a Privacy Act Statement when collecting personal information that will be placed or stored in a system of records. NIST will not collect personally identifiable information from respondents that participate in these usability collections.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.

Not applicable, no sensitive data will be collected.

12. Provide an estimate in hours of the burden of the collection of information.

The estimated total for the three-year period of approval is 8,500 responses and 5,000 hours.

13. Provide an estimate of the total annual cost burden to the respondents or record-keepers resulting from the collection (excluding the value of the burden hours in

Question 12 above).

Not applicable.

14. Provide estimates of annualized cost to the Federal government.

It is difficult to estimate the possible Federal costs in advance. The availability of funds and staff resources will impact NIST’s ability to conduct all the proposed activities. Consequently, it is uncertain of the number of activities that will be undertaken. In addition, it is difficult to project the number of estimated responses that will be received for data collections. Most of the surveys will be conducted and analyzed using in-house resources, though NIST may contract some of the surveys to be conducted.

15. Explain the reasons for any program changes or adjustments.

Not applicable.

16. For collections whose results will be published, outline the plans for tabulation and publication.

The results of the data collections may be used in papers published in research journals and presented at conferences. There will be no attribution to individuals in the analyzed data.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons why display would be inappropriate.

All written and electronic material will display the expiration date of the OMB approval of the information collection. All written and electronic surveys (with the exception of transactional surveys in the form of business reply cards where space will not permit) will also display the following notification:

This collection of information contains Paperwork Reduction Act (PRA) requirements approved by the Office of Management and Budget (OMB). Notwithstanding any other provisions of the law, no person is required to respond to, nor shall any person be subject to a penalty for failure to comply with, a collection of information subject to the requirements of the PRA unless that collection of information displays a currently valid OMB control number. Public reporting burden for this collection is estimated to be XXXXXXX minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed and completing and reviewing the collection of information. Persons wishing to comment on the burden estimate or any aspect of this collection of information, or offer suggestions for reducing this burden, to the National Institute of Standards and Technology, Attn: XXAdd-Individual-Point-of-Contact_InformationXX.

OMB Control No. 0693-0043

Expiration Date: XX-XX-XXXX

18. Explain each exception to the certification statement.

Not applicable.

9