Study Plan

Study Plan for Mobile UI Design Framework for Survey Operations

(Phase I, Study 1)

Draft: March 20, 2016

Last revision: April 11, 2016

Version 1.0

Study Team

IOE Project Team

Sponsor

U.S. Census Bureau

Human Factors and Usability Research Group

Center for Survey Measurement

Associate Directorate for Research and Methodology

U.S. Census Bureau

1. Introduction

The purpose of the IOE project (2015-01), Mobile User Interface Design Framework for Survey Operations (aka, Mobile UI Framework), is to develop (1) standards for basic parameters in mobile software user interface (mobile UI) and (2) guidelines for major aspects in mobile UI design. The standards and guidelines to be developed in this project concern three types of mobile software: (1) self-administrated web survey instruments, (2) field operation applications, and (3) data dissemination websites. The project is to be carried out in three phases. In Phase I, standards and guidelines for self-administrated web survey instruments are developed, Phase II for field operation applications, and Phase III for data dissemination websites.

The present document is a study plan for Phase I, Study 1, Experiments 1 -5.

2. Specific Aims in Phase I

1. Develop empirical-evidence-based standards for basic mobile UI parameters in self-administrated web survey instruments.

2. Develop evidence-based guidelines for mobile UI design of self-administrated web survey instruments.

3. Develop a demonstration mobile website of self-administrated web survey instrument using the standards and guidelines generated in Aims 1 and 2 (Demo Survey), and a corresponding website using current practices (Comparison Survey).

4. Compare respondents’ performance of completing the Demo Survey with completing the Comparison Survey.

3. Aim I: Standards for Basic UI Parameters

Standards are developed for three categories of basic mobile UI parameters: (1) Touch target size and spacing, (2) Text entry and display, and (3) UI luminance and color. For categories 1 and 2, the following approach is taken to establish the standards: (1) Define a list of basic mobile UI parameters; (2) define user performance metrics for each parameter; (3) design an experiment for each mobile UI parameter or a combination of parameters; (4) collect user performance data through behavioral experiments; (5) establish standards based on the statistical analysis of user performance data collected in (4). For category 3, standards are formed through literature review because considerable research has been done on the subject of luminance and color perception.

3.1. Basic mobile UI Parameters

Table 1 lists the parameters or parameter combinations for which standards are to be established.

Table 1. Basic Mobile UI Parameters for Standards

Parameters / Parameter Combinations	Standard to be established
Width of square touch target	Minimum width of a square touch target
Spacing surrounding a square touch target	Minimum space surrounding a square touch target with a given width
Diameter of round touch target	Minimum diameter of a round touch target
Spacing surrounding a round touch target	Minimum space surrounding a round touch target with a given diameter
Height of a text field	Minimum height of a text field
Vertical spacing above and below a text field	Minimum vertical space above and below a text field
x height of text display	Minimum x height of text display

3.2. Experiment 1 – Square Touch Target

Performance measures. User performance of operating a square touch target is to be assessed in this experiment. Three trial-level (see Stimulus presentation) metrics are used to measure user performance: Success (Success), Task completion time (Time), and Perceived task difficulty (Difficulty). Each metric is defined in Table 2.

Table 2. User Performance Metrics for Experiments 1 and 2

Metric (Abbreviation)	Definition
Success (Success)	A dichotomous indicator, 1 = hitting a target, 0 = missing a target
Task completion time (Time)	Duration between the onset of task and the end of the task
Perceived task difficulty (Difficulty)	Subjective rating of perceived difficulty level in performing the task, using a 5-point rating scale: very easy, easy, not easy nor difficult, difficult, very difficult

Experimental factors. Two factors are manipulated in this experiment: Target width (Width) and space surrounding the target (Space). There are 10 levels of width, ranging from 2 mm to 11 mm, with 1 mm increment. For each target width, there are certain levels of space, starting from 1 mm and, with 1 mm increment, ending with a value, M, that satisfies the following equation:

Width + (M x 2) = 17 or 18

Table 3 lists all the combinations of width and space levels to be tested in Experiment 1, totaling 55 combinations. In addition to the 55 combinations, there are 10 more conditions, one for each target, where spacing is infinite (no surrounding objects). Consequently, the experiment has 65 width-space combinations.

Table 3. Combinations of Touch Target Width and Space Levels for Experiment 1

Width (mm)	Min space (mm)	Max space (mm)
2.0	1	8
3.0	1	7
4.0	1	7
5.0	1	6
6.0	1	6
7.0	1	5
8.0	1	5
9.0	1	4
10.0	1	4
11.0	1	3

Stimuli. Four-by-four arrays of squares with different width-space combinations as defined in Experimental factors, with one square highlighted as a touch target. The touch target randomly occupies one of the four locations in the middle of the array.

Stimulus presentation. The experiment includes one session. The session is comprised of 260 trials. Each trial consists of one touching task as described in the Experimental paradigm. For the first 65 trials, each of the 65 width-space combinations as defined in Experimental factors is randomly assigned to one and only one trial. The same randomized 65-trial sequence is repeated four times without interruption within the session, resulting in 260 trials of pseudo-randomized width-space combinations.

Experimental paradigm. The participant performs a touching task in this experiment. The task starts with the participant holding a smartphone with one hand and placing the index figure of the other hand at a starting point on the smartphone screen. After the index finger is placed at the starting point, a stimulus as designed in Stimuli appears on the screen. As soon as seeing the touch target, the participant moves the index finger away from the starting point to touch the target. Once the finger lands on the screen, the stimulus disappears and the task is over. Figure 1 shows a sketch of the user interface of the experimental software.

---------------------------------

Place Figure 1 around here

---------------------------------

Experimental procedure. Each participant participates in one experimental session. The session is estimated to last 90 minutes. A high-level experimental procedure is described as follows:

1. The participant is greeted by a test administrator (TA), and seated with comfort in a quite room with bright ambient light.

2. The participant is informed with the nature and purpose of this study.

3. The participant is given opportunities to ask questions and to sign a consent form.

4. The participant receives a vision screening and anthropometric measurement of index finger tip.

5. The TA reads instructions to the participant.

6. The participant practices the task until feel comfortable.

7. The TA starts video and audio recording.

8. The participant performs the task.

9. The participant rates the task difficulty.

10. The TA debriefs the participant.

11. The participant receives an honorary and signs the receipt.

12. The session ends.

Participants. Fifteen males and 15 females are to be recruited from the public through advertisement or existing participants pool. Each participant must meet all the inclusion criteria as follows:

1. Age between 60 and 70 years old

2. Binocular habitual near-vision around 20/20

3. Normal luminance contrast sensitivity

4. Education of 8^th grade or equivalent

5. Fluent English speaker

6. Smartphone user with 12 months experience

3.3. Experiment 2 – Round Touch Target

The design for Experiment 2 is the same as Experiment 1 except for the following: The square is replaced with a circle, and square width with circle diameter.

3.4. Experiment 3 – Text Field Height (data entry)

Performance measures. User performance of typing in a text field is to be assessed in this experiment. Four trial-level metrics are used to measure user performance: Cursor placement (Cursor), Typing errors (Error), Task completion time (Time), and Perceived task difficulty (Difficulty). Each metric is defined in Table 4.

Table 4. User Performance Metrics for Experiment 3

Metric (Abbreviation)	Definition
Cursor placement (Cursor)	Number of attempts made to place the cursor in the text field
Typing errors (Error)	Number of errors made during data entering (typing)
Task completion time (Time)	Duration between successfully placing the cursor to typing the last character
Perceived task difficulty (Difficulty)	Subjective rating of perceived difficulty level in performing the task, using a 5-point rating scale: very easy, easy, not easy nor difficult, difficult, very difficult

Experimental factors. Two factors are manipulated in this experiment: Text field height (Height) and space above and below the text field (Space). There are 5 levels of width, ranging from 3 mm to 7 mm, with 1 mm increment. For each text field height, there are certain levels of space, starting from 1 mm and, with 1 mm increment, ending with a value, M, that satisfies the following equation:

Height + (M x 2) = 13 or 14

Table 5 lists all the combinations of height and space levels to be tested in Experiment 3, totaling 21 combinations. In addition to the 21 combinations, there are 5 more conditions, one for each text field height, where spacing is infinite (no surrounding objects). Consequently, the experiment has 26 height-space combinations.

Table 5. Combinations of Text Field Height and Space Levels for Experiment 3

Height (mm)	Min space (mm)	Max space (mm)
3.0	1	5
4.0	1	5
5.0	1	4
6.0	1	4
7.0	1	3

Stimuli. (1) Text fields with height-space combinations as defined in Experimental factors. (2) Twenty six sentences with the length of 20-25 words at 6th-grade literacy level to be typed in the text fields. See Appendix A for the content of the 26 sentences.

Stimulus presentation. The experiment includes one session. The session is comprised of 26 trials. Each trial consists of one data entry task as described in the Experimental paradigm. Each of the 26 sentences is randomly assigned to one and only one trial. Each of the 26 height-space combinations defined in Experimental factors is randomly assigned to one and only one trial.

Experimental paradigm. The participant performs a data entry task in this experiment. The task starts with the participant holding a smartphone with one hand and placing the index figure of the other hand at a starting point on the smartphone screen, with a stimulus sentence text displayed within a viewing distance. After the index finger is placed at the starting point, a blank text field with a given width-space combination (see Experimental factors) appears on the screen. As soon as seeing the text field, the participant moves the index finger away from the starting point to touch the text field. After the cursor is successfully placed in the text field, the participant types the stimulus sentence text into the text field. Once the last character is typed in, the task is over. Figure 2 shows a sketch of the user interface of the experimental software.

---------------------------------

Place Figure 2 around here

---------------------------------

Experimental procedure. Same as Experiment 1.

Participants. Same as Experiment 1.

3.5. Experiment 4 – Text Field Height (data editing)

The design for Experiment 4 is the same as Experiment 3 except for Experimental paradigm as described below.

Experimental paradigm. The participant performs a data editing task in this experiment. The task starts with the participant holding a smartphone with one hand and placing the index figure of the other hand at a starting point on the smartphone screen. After the index finger is placed at the starting point, a text field with a given width-space combination (see Experimental factors in Experiment 3, same text field design as Experiment 3) appears on the screen. The text field is populated with one of the 26 sentences used in Experiment 3. The participant is then asked to change a word and 2 particular characters in the middle of a word. After the two actions are completed, the task is over.

3.6. Experiment 5 – Character Height (data display)

Performance measures. User performance of reading text displayed on a smartphone screen is to be assessed in this experiment. Three trial-level metrics are used to measure user performance: Reading errors (Error), Reading time (Time), and Perceived task difficulty (Difficulty). Each metric is defined in Table 6.

Table 6. User Performance Metrics for Experiment 5

Metric (Abbreviation)	Definition
Reading errors (Error)	Number of errors made during reading
Reading time (Time)	Duration between pronouncing the first word and pronouncing the last word in the reading text
Perceived task difficulty (Difficulty)	Subjective rating of perceived difficulty level in performing the task, using a 5-point rating scale: very easy, easy, not easy nor difficult, difficult, very difficult

Experimental factors. One factor is manipulated in this experiment: Character x height (x-height). There are 5 levels of x height, ranging from 1 mm to 3 mm, with 0.5 mm increment. Spacing between the characters is determined by typeface. Table 7 lists 5 x-height to be tested in Experiment 5.

Table 7. x height for Experiment 5

X height (mm)

1.0

1.5

2.0

2.5

3.0

Stimuli. Ten reading texts at 6^th-grade level (IReST, International Reading Speed Texts) displayed in the typeface of Source Sans Pro and in x-heights defined in Experimental factors.

Stimulus presentation. The experiment includes one session. The session is comprised of 10 trials. Each trial consists of one reading task as described in the Experimental paradigm. Each of the 10 reading texts is randomly assigned to one and only one trial. For the first 5 trials, each of the 5 x-heights defined in Experimental factors is randomly assigned to one and only one trial. The same randomized 5-trial x-height sequence is repeated two times without interruption within the session, resulting in 10 trials of pseudo-randomized x-heights.

Experimental paradigm. The participant performs a reading task in this experiment. The task starts with the participant holding a smartphone with one hand and placing the index figure of the other hand at a starting point on the smartphone screen. After the index finger is placed at the starting point, a stimulus text with a given x height (see Experimental factors) appears on the screen. As soon as seeing the text, the participant starts to read aloud the text. Once the last word is pronounced, the task is over. Figure 3 shows a sketch of the user interface of the experimental software.

---------------------------------

Place Figure 3 around here

---------------------------------

Experimental procedure. Same as Experiment 1.

Participants. Same as Experiment 1.

Figure 1.

Figure 2.

Figure 3.

Appendix A

6th-grade sentences for Experiment 3 and 4

Though Janet had been working eagerly on her art submission for weeks, her mother wasn’t sure what the project involved.

My mom and I grabbed the signs and we posted them along our street as well as the surrounding streets in the neighborhood.

There are two kinds of prairie grasses, one that grows a lot in the spring, another that grows a lot during the summer.

Given Joe’s history with hobbies, it was no surprise that Joe’s mom was hesitant to buy a magician’s kit for his birthday.

There is a lot to learn about the plants people eat, such as how to grow them in addition and how to prepare them for eating.

Michael likes the view that he gets from the family’s log cabin up in the mountains, and he enjoys hiking in the forest.

When Jennifer reached her back steps that afternoon, she could see an inviting light coming from the kitchen windows.

Mother has some of the potatoes that you gave us last week, and she told us we can use them to grow more potatoes in our yard.

Soon Momma’s gooey, sweet Caramel Melt Cake had risen in the pans, and Niesha sat down in the kitchen waiting to eat it.

Although Joe’s beans looked healthy, they did not grow as fast or climb as high as any of the other students’ bean plants.

The small island was filled with many trails winding through the thick underbrush, a small lake, and a dangerous wild pig.

In the summertime, there was so much tall grass in the land that people called it a sea of grass that grew as tall as people.

When Joe’s beans began to form, they were large, flat, and wide, unlike the pencil-thin beans that dangled from the fence.

She thought of the smell of Momma’s kitchen, the safety she felt at home, but most importantly, Momma’s supportive words.

People traveled in groups, and each family would buy a covered wagon, which is a big wooden wagon with a kind of tent on it.

He drove all the way from Indiana to get us, and when we saw his car pull up we ran outside as we were excited to go to the game.

Jane started the school year as the shyest girl on the speech team, but I ended the school year as the blue ribbon winner.

The communities would grow into towns, and then some of these would expand into cities, from a wagon train to a metropolis.

Thousands of bison and hundreds of birds and other animals that used to live in this area are gone, but they did not migrate.

The rocky outer crust of Earth is surprisingly not solid; rather, it is broken up into gigantic pieces called plates.

Surrounding the Pacific Ocean is a circle of volcanoes that includes more than half of the world’s active volcanoes.

The people on the islands picked the buds; the buds were pink when people picked them, and then they dried and turned dark.

This means that the species are protective of the areas that they claim and they generally do not share with other tigers.

Janet made all the soap and candles her family used, and she also prepared the sugar from the sugar-trees on their farm.