Memo

Memorandum

To: Office of Management and Budget

From: Lin Wang, Center for Survey Measurement, U.S. Census Bureau

Subject: Mobile UI Design Framework for Survey Operations – Phase 1, Study 1

Date: April 11, 2016

Background

Mobile devices have become an important tool in survey data collection and other survey operations at the U.S. Census Bureau. Because of the nature of small screen size and touch interface, we are challenged with usability in mobile software, i.e., some mobile software is not as easy to use as one expects. The user interface (UI) design of mobile software is crucial to user/respondent experience, and consequently survey quality, because the UI is the interface through which the user/respondent receives and provides survey information. Poor UI design has resulted in errors in task performance, prolonged time in completing a task, and unsatisfied user experiences. In addition, it is costly to re-design and re-develop user interfaces. To improve user experience in using the Census Bureau’s mobile software and to reduce survey measurement error, the Census Bureau needs a framework of comprehensive mobile UI design that is based on scientific evidence of user research, for developing human-centered mobile software.

Under its Improving Operational Efficiency (IOE) program, the Census Bureau initiated the project of Mobile User Interface Design Framework for Survey Operations (aka, Mobile UI Framework) in 2015. The purpose of the project is to develop (1) standards for basic parameters in mobile software user interface and (2) guidelines for major aspects in mobile UI design. The standards and guidelines to be developed in this project concern three types of mobile software: (1) self-administrated web survey instruments, (2) field operation applications, and (3) data dissemination websites. The project is to be carried out in three phases. In Phase I, standards and guidelines for self-administrated web survey instruments are developed, Phase II for field operation applications, and Phase III for data dissemination websites.

Phase I includes three studies: Study 1 develops standards and generates empirical evidences supporting the standards; Study 2 develops guidelines and collects evidences supporting the guidelines; Study 3 develops a demonstration web survey instrument in two versions, with one version using the current UI design practice (comparison) and the other version using the standards and guidelines (S&G). The present memorandum addresses human performance data collection in Study 1.

Purpose of research

The Census Bureau plans to conduct usability testing (Testing) under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). The purpose of the Testing is to provide behavioral evidences that support the mobile UI design standards.

Research design

Standards are to be developed for three categories of basic mobile UI parameters: (1) Touch target size and spacing, (2) Text entry and display, and (3) UI luminance and color. For categories 1 and 2, the following approach is taken to establish the standards: (1) Define a list of basic mobile UI parameters; (2) define user performance metrics for each parameter; (3) design an experiment for each mobile UI parameter or a combination of parameters; (4) collect user performance data through behavioral experiments; (5) establish standards based on the statistical analysis of user performance data collected in (4). For category 3, standards are formed through literature review because considerable research has been done on the subject of luminance and color perception. Usability testing will be conducted only for categories 1 and 2. Table 1 lists the parameters to be investigated.

Table 1. Basic Mobile UI Parameters for Standards

Parameters / Parameter Combinations	Standard to be established
Width of square touch target	Minimum width of a square touch target
Spacing surrounding a square touch target	Minimum space surrounding a square touch target with a given width
Diameter of round touch target	Minimum diameter of a round touch target
Spacing surrounding a round touch target	Minimum space surrounding a round touch target with a given diameter
Height of a text field	Minimum height of a text field
Vertical spacing above and below a text field	Minimum vertical space above and below a text field
x height of text display	Minimum x height of text display

Five experiments were designed to collect behavioral evidences which the standards will be developed upon. Experiments 1 to 4 concern data entry while Experiment 5 data display. All experiments are within-subject design. Four groups of 30 participants will participate in the study, each group being tested in Experiment 1, 2, 3, and Experiments 4 and 5 combined, respectively. Our past mobile usability studies showed marked individual differences in task performance. We thus determined that a sample size of 30 is needed to improve statistical power. Each experiment session lasts about 90 minutes. Experiments will be conducted in the Usability Lab at the Census Bureau Headquarters.

Refer to Enclosure 1 for detailed experimental design.

Participants

One hundred and twenty (30x4, 60 males and 60 females) participants will be recruited from the public through advertisement or an existing participants pool. Each participant must meet all the inclusion criteria below. The age specification is based on the rationale that older users have more difficulties in operating a mobile device due to age-related mobility. If an UI design is workable for a older user, it should be readily workable for a younger user; but not necessarily the other way around.

Participants’ inclusion criteria:

1. Age between 60 and 70 years old

2. Binocular habitual near-vision around 20/20

3. Normal luminance contrast sensitivity

4. Education of 8^th grade or equivalent

5. Fluent English speaker

6. Smartphone user with 12 months experience

Burden

We estimate that each participant will spend 90 minutes to participate in the study, including time for consent, collecting demographic information, practice, and post-test debriefing. Total burden for 120 participants is estimated 180 hours (1.5 hour x 120 participants).

Incentives

The participant will receive $60.00 to offset the costs of participation in the study.

Sponsor

Improving Operational Efficiency (IOE) program, the U.S. Census Bureau.

New items

The standards are new and yet to be tested.

Enclosures

The following materials will be used in the study and are attached with this letter:

1. Study Plan for Mobile UI Design Framework for Survey Operations

2. Demographic questionnaire

3. Mobile experience questionnaire

4. Post-test debriefing materials

5. Experimental protocol

Contact

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Lin Wang

Center for Survey Methods Research

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-9069

[email protected]

3