Attachment U - Usability Test Round 1 Report

Attachment U. Final Analysis Report and
Recommendations for Revisions to the FoodLogger
from the First Round of Usability Testing

.

Report on FoodLogger usability evaluation (Round 1)
Lin Wang, Shelley Feuer, Heather Ridolfo
FoodAPS-2 project team of Census Bureau and NASS
June 16, 2021

Executive Summary
The Round-1 FoodLogger usability evaluation was conducted from May 6 to May 24, 2021. Six
primary food shoppers from six households participated in the study, with two being 60+ old and the
other four being recipients of food assistance programs. The evaluation design calls for an 8-day study
including three major parts: FoodLogger use training, 7-day food acquisition reporting, and lab-based
usability testing. FoodLogger of version-2 was the data collection instrument under evaluation. Three
use cases (a combination of Food-at-home (FAH) event and a Food-away-from-home (FAFH) event, a
FAFH event, and a school-meal event) and 18 critical tasks (a task such that failure to complete it would
result in measurement errors) were tested. Qualitative and quantitative methods were applied for data
analysis. Usability issues will be classified as high (H), medium (M), and low (L) priority. H issues are
those that prevent a task from being completed; M issues prolong task completion; L issues do not
impact effectiveness and efficiency of task completion but may affect user’s satisfaction.
Participants attained adequate skills through training, carried out 7-day food acquisition
reporting in a daily living setting, and completed use cases in the lab-based usability testing. In general,
participants expressed positive experiences in using FoodLogger. Four usability issues with high priority,
4 with medium priority, and 5 with low priority were reported. Details were documented in the body of
the report.

FoodLogger evaluation report - 1

1

1. Evaluation Objective
The objective for the present usability evaluation of the FoodLogger native smart phone app
(FoodLogger) is to assess how potential respondents enter food acquisition data into the FoodLogger in
terms of effectiveness, efficiency, and satisfaction. The effectiveness of the FoodLogger will be
measured by the success of data entry and the accuracy of entered data; efficiency will primarily be
measured by the time taken to enter data; and satisfaction will be measured by respondent-reported
satisfaction which includes a user’s perception of difficulty, the extent to which their expectations are
met, and a user’s emotional response to data entry. It is hypothesized that effective data entry will
prevent missing or erroneous data and minimize measurement errors, efficient data entry will reflect
lower respondent burden, and satisfaction with the data entry experience will help sustain respondents’
participation in the Second National Food Acquisition and Purchase Survey (FoodAPS-2).

2. Evaluation Methods
Participating households
Six households participated in the study. Participants were recruited via advertisement and
word-of-mouth. The characteristics of the participating households are listed in Table 1. All participants
were given a Disclaimer that states the purpose of the study, data to be collected, rights as a participant,
and the statutory authority under which the study was conducted. A written consent to participating the
study was obtained from each participant prior to the commence of the study.

Table 1. Participants Demographic Characteristics

Household Size
(person)
School Children < 11year-old (person)
School Children 11-15year-old (person)
WIC Recipient
SNAP or Other
Government Food
Assistance
Residence RUCA code
Primary Shopper Age
(year)
Primary Shopper
Gender

Household
1
2

Household
2
4

Household
3
7

Household
4
2

Household
5
1

Household
6
4

1

0

1

0

0

1

0

0

0

1

0

2

No
Yes

No
No

Yes
Yes

No
Yes

No
Yes

No
Yes

1
47

1
64

8
29

1
42

2
66

2
33

F

F

F

F

F

F

FoodLogger evaluation report - 1

2

Primary Shopper
Education

Associate’s
degree

Graduate
degree

Some
college, no
degree

Some
college, no
degree

Some
college, no
degree

Some
college, no
degree

Race

Black or
African
American

White

American
Indian or
Alaska
Native

Black or
African
American

White

White

Hispanic Ethnicity
Smartphone Use
History (length)
Smartphone Use
Frequency
Using Map on
Smartphone
Three most frequent
uses of smartphone

No
≥ 2 years

No
≥ 2 years

No
≥ 2 years

No
≥ 2 years

No
≥ 2 years

No
≥ 2 years

Everyday

Everyday

Everyday

Everyday

Everyday

Everyday

Yes

Yes

Yes

Yes

Yes

Yes

email,
Texting,
Call

Call,
Messaging,
GPS

Call,
Texting,
email

Call, email,
Social
media

Texting,
Taking
photos,
Making
telemedici
ne
appointme
nt

Call,
Texting,
email

Product evaluated
The product evaluated was FoodLogger version 2, developed by Westat. FoodLogger is a native
mobile application that serves as a data collection instrument for FoodAPS-2.

Evaluation design
The evaluation design calls for an 8-day study including three major parts: FoodLogger use
training, 7-day food acquisition reporting, and lab-based usability testing. Table 2 shows the timeline of
the study. Due to COVID-19 pandemic, the study was conducted virtually via MS Teams, emails, and
telephone calls. Specific methods for each of the three parts will be presented in its dedicated section
below.

Table 2. Timeline of major testing activities
Component
Being introduced to the study
Receiving a Disclaimer
Signing a Consent Form
FoodLogger evaluation report - 1

Day
1
x
x
x

Day
2

Day
3

Day
4

Day
5

Day
6

Day
7

Day
8

3

Completing a demographic
questionnaire
Installing FoodLogger
Data entry training
Field data entry
Field data entry debriefing
Lab-based usability testing
Signing incentive voucher

x
x
x
x

x

x

x
x

x

x

x

x
x

3. FoodLogger Training
Training design
The FoodLogger training consisted of three components in sequence: Basic concepts in
FoodAPS-2 data entry, FoodLogger installation, and data entry practices. A training courseware in the
form of MS PowerPoint was developed by the study team (Appendix A). Only primary shoppers received
training. The training started with introducing the basic concepts. Then, under trainer’s guidance, the
trainee downloaded FoodLogger to his/her smartphone. Lastly, the trainee practiced entering specific
food information (e.g., PLU code) into FoodLogger. One debriefing on FoodLogger installation and
another on the rest of the training were conducted to assess training effectiveness.
For primary shoppers who have a 11-15-year-old school child, he/she was instructed to train the
child on data entry by her/himself using the courseware.

Major findings and recommendations
Basic concepts:
1. Participants could understand most concepts covered in training.
2. Four out of six participants could not clearly distinguish the difference between “Stop”
and “Food event.” Recommendation: Reconcile the two concepts into one. Alternatively,
develop effective instructions to help respondents to understand the two concepts.
3. Two out of six participants were not clear about “Food acquisition.” Recommendation:
Emphasize the concept in training.
4. Two out of six participants were less clear about “Combo meal.” Recommendation:
Emphasize the concept in training.
5. Average training time on the Basic Concepts part is 12.7±2.6 min.
FoodLogger installation:
1. Participants could successfully install FoodLogger on their smartphone under trainer’s
guidance.
2. The participants would have encountered difficulties without trainer’s guidance,
particularly with respect to configuration.
FoodLogger evaluation report - 1

4

3. Recommendation: A step-by-step written instruction may facilitate installation.
Data entry practice:
1. Participants could generally carry out basic data entry tasks right after learning the basic
concepts about FoodLogger. Those basic tasks include text entry, bar code scanning, PLU
code entry, taking a picture of and uploading a paper receipt.
Training School children:
1. The primary shopper was asked to train their 11-15-year-old school children on data
entry using FoodLogger. Though there is no direct assessment on training quality,
observation of data entry performance by the children suggests that the training did not
effectively prepare the children to successfully enter food data into FoodLogger. In
addition, children had few opportunities to use FoodLogger during the study period.
2. Recommendation: All eligible household members are to be trained by qualified trainers,
rather than a household member.

4. Lab-based Usability Testing
Testing design
USE CASES: Three use cases were tested in the lab-based usability testing: (1) A combination of
Food-at-home (FAH) event and a Food-away-from-home (FAFH) event (Appendix B), (2) a FAFH event
(Appendix C), and (3) a school-meal event (Appendix D). The three use cases are designed such that each
critical task (described below) will be performed at least once during testing. All primary-shopper
participants were tested on use cases 1 and 2. Primary shoppers with school children younger than 11
years old were also tested on use case 3 as a proxy for a school-age child under 11 years old. School-age
participants of 11 years old or older were tested on use case 3.
CRITICAL TASKS: A critical task refers to a task such that failure to complete it would result in
measurement errors (e.g., scanning a bar code). Eighteen critical tasks, as listed in Table 3, were
identified for successful data entry using FoodLogger, and were tested in the lab-based usability testing.

Table 3. Critical tasks for data entry using FoodLogger
Task #
1
2
3
4
5
6

Task
Start a day
Select a food stop from a list of stops identified by
FoodLogger
Add a food stop manually
Select a food event
Add a food event manually
(FAH food item:)
Enter item name

FoodLogger evaluation report - 1

Sub-task

barcode, PLU, text
5

7
8
9
10
11
12
13
14
15
16
17
18

Enter weight/volume/size
Enter number of items
Enter payment
(FAFH combo food item:)
Select "combo meal" button
Enter meal name
Enter payment
Enter number of items
Enter individual meal items
(FAFH individual food item:)
Select "individual item" button
Enter meal name
Enter number of items
Enter payment

pay by single item or multiple items;
payment methods

payment methods

payment methods

PERFORMANCE MEASURES: The following metrics were used to assess participants’ data entry
performance.
a) Data entry accuracy – The extent to which the entered data are correct.
b) Data entry time – Duration between the start and end of a use case.
c) Navigation – The extent to which participant’s actual navigation path deviates from the
optimal path.
DATA COLLECTION: A protocol was followed to carry out the lab-based usability testing
(Appendix E). Methods for data collection include:
a) Passive observation
b) Thinking aloud
c) Retrospective Debriefing – Focused on critical design components, e.g., language
comprehension. A debriefing guide was followed to cover critical actions of interest.
DATA ANALYSES: Quantitative data were analyzed with descriptive and/or inferential statistics
accordingly. Qualitative data were summarized to identify common usability issues and their causes.
Usability issues will be classified as high (H), medium (M), and low (L) priority. H issues are those that
prevent a task from being completed; M issues prolong task completion; L issues do not impact
effectiveness and efficiency of task completion but may affect user’s satisfaction (e.g., imperfect text
formatting).

Summary performance measures
All participants were able to complete assigned use cases. Table 4 shows quantitative measures
for food information entry performance. Those quantitative measures are defined as follows:

FoodLogger evaluation report - 1

6

•
•
•
•
•
•
•

•

•

Use-Case-1 completion time (min): The time between starting a food event and before
submitting a receipt.
Use-Case-1 Average time for entering a single food item (min): (Use Case 1 completion
time)/(Number of food items in Use Case 1).
Use-Case-1 paper receipt upload time (min): The time between selecting “Yes, I have a paper
receipt” and completing uploading the image of the paper receipt.
Use-Case-1 Deli reported: A dichotomous indicator on reporting the deli purchase, with Y
indicating the purchase being reported, and N otherwise.
Use-Case-2 completion time (min): The time between starting a food event and before
submitting a receipt.
Use-Case-3 completion time (min): The time between starting a food event and completing the
event.
Optimal food name entry rate: A ratio of the number of food items entered into FoodLogger
using optimal name entry method over the number of total food items entered into FoodLogger
(e.g., barcode scanning for a barcoded item is an optimal method while text entry not)
Optimal packaging selection rate: A ratio of the number of food items entered into FoodLogger
with optimal packaging selection over the number of total food items entered into FoodLogger
(e.g., Caton category is optimal for milk)
Optimal weight/volume selection rate: A ratio of the number of food items entered into
FoodLogger with optimal weight/volume selection over the number of total food items entered
into FoodLogger (e.g., Pound/ounce is optimal category for bulk coffee)
Table 4. Food information entry performance summary

Use-Case-1
completion time
(min)
Use-Case-1
Average time for
entering a single
food item (min)
Use-Case-1 paper
receipt upload
time (min)
Use-Case-1 Deli
reported
Use-Case-2
completion time
(min)

Household
1
40

Household
2
45

Household
4
31

Household
5
data
confounded
with tech
glitches

Household
6
34

2.4

Household
3
18 (data
entered
directly
from
receipt)
NA

2.1

1.6

NA

1.8

1

1

1

1

1

1

Y

N

N

N

Y

Y

16

32 (with
tech
glitches)

7

11

2

17

FoodLogger evaluation report - 1

7

Use-Case-3
completion time
(min)
Optimal food name
entry rate
Optimal packaging
selection rate
Optimal
weight/volume
selection rate

5

NA

4 (by child)

NA

5 (by child)

1.00

5 (entered
a wrong
path in
first try)
0.17

0.78

1.00

0.22

0.94

1.00

0.75

1.00

0.69

1.00

1.00

0.76

0.82

0.89

0.72

0.89

0.88

Usability issues with HIGH priority
Select/add a food event
Issues: For one “food event venue” (e.g., a grocery store), there can be only one event. Adding a
second food event at the same venue would lead to adding food items into the first event. For example,
after reporting a FAH event at a grocery store, one cannot add another FAFH event at the same grocery
store. This behavior (1) confused the participant and (2) created difficulties for data entry, e.g., FAH and
FAFH could not be appropriately handled at the same time. The current algorithm contradicts the
concepts of Stop, which permits multiple food events at the same Stop by definition.
Recommendations: Make all options for FAH and FAFH available for all “food events.” Review
the design from stop selection to FAH/FAFH selection.
Enter food item name – Barcode/PLU
Issues: (1) Not all barcodes/PLU had corresponding food item names available in the database. If
scanning a barcode returns with no food name, the workflow breaks. The participant was not presented
with a graceful and intuitive screen to proceed to the next step. This broken workflow could very likely
lead to data entry errors. (2) Error messages were not visually salient enough to grab participant’s
attention.
Recommendations: (1) Add a screen with a text field and a clear instruction asking the
respondent to type in the food name. (2) Make error message text more visually salient.
Make packaging selection
Issues: It was not unusual for the participant to select a category which is not intended by the
designer, resulting in a cascade of undesirable behaviors: wrong measurement and measurement unit.
Its root causes appear to be (1) the categories are not exhaustive nor mutually exclusive, and (2)
categorization is more or less subjective judgement.
Recommendations: Since this is not essential information of interest and is error-prone, it may
be better to be eliminated. Alternatively, make all possible measurement units available for all
packaging categories.
Enter weight/volume/piece
Issues: It was frequently observed that the participant was unable to select an appropriate
measurement unit. The reasons appear to be one or a combination of the following: (1) the participant
didn’t have credible information, and (2) the appropriate unit was not on the list of choices. The
FoodLogger evaluation report - 1

8

omission of appropriate units could be because (1) the list was not exhaustive or (2) the participant was
on a wrong path. Some participants did not know how to report this info for multi-item package.
Recommendations: Make the list more comprehensive and add a free text field to allow the
participant to make an ad-hoc addition of measurement unit. Place unit choice list above the quantity
text field, enabling the participant to decide whether an appropriate measurement unit exists. Review
multi-item entry design.

Usability issues with MEDIUM priority
Select/add a food stop
Issues: (1) One participant experienced difficulty adding a missing stop between detected stops.
(2) Food acquisition type was contingent on stop type, which may cause certain food acquisition types to
be missing, e.g., selecting a restaurant eliminates the food acquisition type of “online order for home
delivery.”
Recommendations: (1) Eliminate the links of adding a missing stop between detected stops.
However, add ONE (and only one) link of adding a missing stop at the end of the detected stop list in a
prominent font to attract participants’ attention and facilitate tapping the link. (2) Add a free-text field
for adding a food delivery type.
Select/add a food event
Issues: Approximately 50% of participants had difficulty comprehending the conceptual
difference between “stop” and “food event.”
Recommendations: Replace “stop” and “food event” with one term/concept. Alternatively,
develop effective instructions to help respondents to understand the two concepts.
Enter food item name - Barcode
Issues: Participants did not always seem to notice the display of a food item name when the
scanning picked up the item and would continue to scan it multiple times before they finally noticed its
having been already picked up.
Recommendations: Make the text of food item name more visually salient.
Upload an electronic receipt
Issues: At least 4 out of 6 participants encountered some degree of difficulties with downloading
or/and uploading an electronic receipt.
Recommendations: Forgo this task, ask respondents to email the receipts to the survey team.

Usability issues with LOW priority
Enter food item name – Barcode/PLU
Issues: (1) Sometimes, “packaging selection” and “weight/volume/piece” were skipped by
FoodLogger after barcode was scanned or PLU was entered, presumably by design to reduce
respondents’ burden if the information is already in the database. (2) Typos in the database.

FoodLogger evaluation report - 1

9

Recommendations: (1) Need further investigation. Was this intentionally designed? If so, it
would be helpful to display the information on the screen to assure the respondents that the
information is there, and to provide a consistent user experience to the respondents. (2) Correct typos.
Enter number of items
Issues: One participant complained that the plus and minus buttons were too light to see them.
Recommendations: Make the text display more salient.
Enter combo meal
Issues: A few participants entered the name of an item as the combo meal name. This resulted
in at least one participant adding sandwich toppings because the app asked her “what was in ‘chicken
sandwich’. That is, instead of adding combo meal items, she thought she had to enter toppings. A few Ps
were also unsure if they should enter condiments like BBQ sauce and ketchup.
Recommendations: Make instructions clearer in training. In particular, what is a combo meal?
How to name a combo meal? Should condiments be included?
Enter a long list of items
Issues: Some participants expressed the concern over entering a long list of food items (40 or
more). It would be time consuming and cause drop-offs, particularly for busy parents.
Recommendations: Two participants suggested that the app pre-populate food names by
uploading a receipt before entering food item names.
Response choice design
Issues: Some response choices are not consistent with convention, e.g., “-“ instead of circle or
square on the screen of packaging choices.
Recommendations: Use conventional response choices design.

5. 7-Day Food Acquisition Reporting
Study design
This component was executed in the participant’s daily living setting on his/her own, without
TA’s observation. The participant was instructed (Appendix E) to enter information for all their acquired
foods into FoodLogger every day, and to log all the problems and difficulties encountered during data
entry. A standard log form was provided to the participant. The TA was available to provide assistance
over phone if needed, though none of the participants contacted the TA. Upon completion of the 7-Day
reporting, a semi-structured debriefing session was carried out to collect participants’ experiences in
food acquisition reporting using FoodLogger in a daily living setting.

Summary of findings from debriefing
Table 5 summarizes responses from the participants on debriefing questions. See Appendix F for
debriefing questions.
Table 5. Participants’ feedback on 7-day food acquisition experiences
FoodLogger evaluation report - 1

10

Confirming a Stop (person)
Manually adding a Stop
(person)
Adding a Food Event (person)
Scanning a barcode (person)
Entering a PLU (person)
Entering a food item name
(person)
Entering size/weight/volume
(person)
Reporting school meal
(person)
Entering cost for a food item
(person)
Entering cost for entire food
event (person)
Choosing method of payment
(person)

Extremely
easy
6
2

Three Greatest challenges

Neutral

3

1

5
4
5
5

2
1
1

1

2

Difficult

Extremely
difficult

NA

1
1

3

3
5

3
1

6
6

Yes
5

No
1

Not at all
3

Somewhat
2

Moderately
1

Very

Extremely

Household
1
Trivial
notification
s (e.g., stop
lights),
measurem
ent unit
conversion,
rememberi
ng not to
report nonfood items

Household
2
Combo
meal,
finding a
receipt,

Household
3
Time
consuming,
FoodLogger
slow in
response

Household
4
None

Household
5
Getting
used to it,
not
knowing
how to
correct a
mistake
initially

Household
6
Keeping up
with
logging
informatio
n, keeping
clicking no
to get the
stops off
the log

Free
1

$5 / 15 min
1

$5
1

$7
1

$10
1

$20
1

Use of "Type-ahead" (person)

Bothered by notifications?
(person)

Easy

Adequate amount of
incentive (person)
FoodLogger evaluation report - 1

11

Completely
not

Somewhat
not
2

Neither yes
or no

Somewhat
yes

Completely
yes
4

Household
1
x

Household
2
x

Household
3
x

Household
5
x

x

x

x

Household
4
x
x
x

x

x
x

Comfortable with sharing
GPS locations during study
period (person)
Food acquisition in the past
30 days:
Grocery store, in-person
Grocery store, order online
Big Box Store, in-person
Big Box Store, order online
Restaurant, eat in
Restaurant, order online
Friend or family member’s
house
Food from charity
Other

x

Household
6
x
x
x

x
x

x
x

x

x
x

6. Round-2 Usability Evaluation Planning
Based on findings from the Round-1 study, we plan to adopt Round-1 study design for the
Round-2 usability evaluation, with the following modification:
1. The lab-based usability session will be conducted on the second day of the study period,
i.e., the next day following the training session. This modification is based on the rationale
that, (1) Round-1 results show that participants generally attain the skills of using
FoodLogger after the training session, (2) conducting the lab-based usability session earlier
in the 7-day period can better assess participants’ data entry performance, and (3)
conducting the lab-based usability session on a different day from the training avoids fatigue
due to prolonged study time.
2. Two households with the primary shopper of 60 years old or older and four households
with a school child of 11-15 years old having his/her own smartphone will be recruited. In
the Round-1 study, performance of using FoodLogger by school children of 11-15 years old is
not satisfactory. Assessment of this age group needs to be enhanced.
3. Participating school children of 11-15 years old will be required to attend the same
training as the primary shopper in person. This modification is designed to investigate if a
full-scale training would improve performance. In Round-1, we were unable to monitor and
assess the quality of training conducted by the primary shopper.

7. Limitations

FoodLogger evaluation report - 1

12

Participant’s performance of using FoodLogger is associated with training they received. The
training was developed and conducted by the study team as requested and may be different from the
training potential respondents will receive in the FoodAPS-2 field test or formal survey. Thus, findings
presented in this report may not be generalizable to a population receiving different training and need
to be interpreted with caution.

FoodLogger evaluation report - 1

13

Appendix A: Training Courseware

FoodLogger evaluation report - 1

14

FoodLogger Training
FoodAPS‐2 Study Team of U.S. Census Bureau and NASS

This presentation is released to inform interested parties of research and to encourage discussion.  
The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau.

1

1

Basic concepts
1. What is FoodLogger?
• A mobile app running on a smartphone
• A survey instrument – You will use it to report all the foods you will acquire over the next 7 days

2. What are foods?
• Anything that you can eat or drink

3. The concept of food acquisition
•
•
•
•

Any food items that you get from outside of your home
Foods can be purchased or free
You may or may not eat those acquired foods while you are participating the study
In the next seven days, If you eat a food item (e.g., a bagel) that was acquired before today, this 
item cannot be counted in food acquisition.

2

2

1

Basic concepts (con’t)
DAY

STOP

Day 1

Stop 1

Day 2

Day 3

Day 4

Stop 2

Day 5

Day 6

Day 7

Stop F

FOOD EVENT

Food 
Event 1

FOOD ITEM
Food 
Item 1

Food Event 2

Food 
Item 2

Food Event K

Food 
Item N

3

3

Basic concepts (con’t)
A food item has:

• Name (e.g., Apple)
• Weight/Volume/Size (e.g., 5 lb, 20 fl oz, 1 serving)
• Cost (free item has the cost of $0)

Multiple same food items also have:

• Count/quantity (e.g., 5 bottles of water)

A food event has:

• Payment (free event has a payment of $0)

4

4

2

Data Entry Workflow

5

5

Installing FoodLogger: iPhone Users
1.
2.
3.
4.
5.
6.
7.
8.
9.

Go to the App Store on your phone
Search FoodLogger
Download the free app
Open the FoodLogger app
Enter your household’s unique PIN to login
Review terms and conditions
Select your name from the dropdown menu and tap “Continue”
When asked, give FoodLogger permission to use your location
If asked, allow the FoodLogger to send you notifications 
6

6

3

Download FoodLogger: iPhone 
Users (con’t)

7

7

Installing FoodLogger: Android Users
1.
2.
3.
4.
5.
6.
7.
8.

Go to the Google Play store on your phone
Search FoodLogger
Download the free app
Open the FoodLogger app
Enter your household’s unique PIN to login
Review terms and conditions
Select your name from the dropdown menu and tap “Continue”
When prompted, give FoodLogger permission to access your device’s 
location
9. If asked, allow the FoodLogger to send you notifications 
8

8

4

Download FoodLogger: Android 
Users (con’t)

9

9

Data Entry Workflow

10

10

5

Practice (I)
1.
2.
3.
4.

Start a day
Add a food stop manually
Add a food event manually
Enter item name
• Barcode
• PLU
• Text

5. Enter weight/volume
6. Enter number of items
7. Enter payment
• Single item
• Multiple items
• Payment modes

11

11

Practice (II)

12

12

6

Practice (III)
1. Breakfast at Starbucks
• One Blueberry Scone ($2.40)
• One Banana ($0.90)
• One Caffe Latte (tall, $3.50)

2. School lunch (combo, free)
•
•
•
•

Chicken Caesar Wrap
Celery Sticks
One Fresh Orange
1% White Milk (Half Pint)

13

13

Exercise
Pick 5 items from your pantry, and enter the food information 
into the FoodLogger

14

14

7

Profile and Income Information
Complete these two 
questionnaires by yourself.
Use FAKE data to complete 
these to questionnaires.
Do not enter your actual 
personal information! 
15

15

Assignment
Teach your child participant how to use FoodLogger to enter the 
foods he/she acquires, for example, school meals.

16

16

8

Appendix B: Use Case 1
Note: Actual food items may vary slightly among participating households depending on store inventory
at the time of food purchase. However, the same critical tasks were covered across households.

FoodLogger evaluation report - 1

15

Use Case 1: Food-at-Home Event plus Food-Away-from-Home Event
Purpose: To test FAH + FAFH
Critical tasks tested:
• Start a day
• Select a food stop from a list of stops identified by the online map
• Select a food event
• Enter food item name:
o Text
o Barcode
o PLU
• Enter weight/volume/etc
• Enter number of items (quantity)
• Enter payment information
o Pay by single mode
o Pay by multiple modes
Event Set-up:
• Food came from supermarket (see delivery slip for more information)
Scenario:
Today you went to a supermarket during lunch break to buy groceries for the next few days. You
paid for your groceries with your EBT card (or food stamps) and debit card. You have the receipt for the
groceries. While you were at the supermarket, you also bought a prepared lunch from the deli that you
ate there. You paid for your lunch with cash, but you were in a hurry and forgot to take your receipt from
the deli. You remember that you paid about $6.50 for lunch. Now you have the groceries in front of you.
Please enter the information about this stop into the FoodLogger.
Groceries:

o

o

o
o

Pre-packaged food (with barcode)
 Pasta
 Crackers
 Blueberries
 Couscous
Produce (with PLU)
 2 apples
 3 limes
 1 pepper
 1 carrot (no PLU)
Food from the bakery
 1 cookie
 1 loaf of bread (made in-store, store brand)
Bulk food
 Coffee beans
 Granola

Food with store-specific barcodes (e.g., store brand pre-packaged food)
 Coconut water
 Salt
 Oats meal
 Eggs
o Multiple items packaged together
 Mineral water (4 bottles)
 Juice (4 cups)
Non-food items:
o 1 facial tissues box
o 1 soap
o

Food from the Deli (not present, just described/pictured):
o Caesar salad (small) $2.25
o Bread roll (small) $0.50
o Cup of soup (~8 oz) $2.00
o Bottle of juice (11 fl oz) $1.75

Appendix C: Use Case 2

FoodLogger evaluation report - 1

16

Use Case 2: Food-Away-from-Home Event
Critical tasks tested:
• Start a day
• Select a food stop from a list of stops identified by the online map
• Add a food event manually
• FAFH Combo meal:
o Select "combo meal" button
o Take a picture
o Enter meal name
o Enter meal price
o Enter number of items (quantity)
o Enter individual meal items
o Enter total event cost or price
o Select payment type
o Take a picture of a receipt and upload it.
• FAFH Individual food item:
o Select "individual item" button
o Enter meal item name
o Enter number of items (quantity)
o Enter item price
o Enter event cost or price
o Select payment type
o Upload receipt
Event Set-up
• The food came from McDonald’s. See delivery receipts for more information.
Scenario:
This evening you ordered food from McDonald’s for your family’s evening meal using the
restaurant’s website. You placed the order at home, and had the food delivered to your home to eat with
your family. You paid with your credit card and have an electronic receipt in your email. Please enter the
information regarding this meal into the FoodLogger app.
•

Food from McDonald’s:
o A Big Mac
o A milk jug
o A chicken sandwich combo meal with fries and drink
 Large fries
 Medium drink
 4 Ketchup packs
o 10-piece Chicken nuggets with 3 barbeque sauce packs
o A Happy Meal (hamburger, apples, fries, milk)
o Baked Apple Pies (3)

Appendix D: Use Case 3

FoodLogger evaluation report - 1

17

Use Case 3: School Meal
Scenario 1: Not Free – Proxy Report
Critical tasks tested:
• Start a day
• Add a food stop to FoodLogger in the text field
• Add a food event manually
• Combo meal:
o Select "combo meal" button
o Enter meal name
o Enter meal price
o Enter individual meal items
 “Other” option (those that do not conform to pre-defined list of
combo items)
o Enter payment type
Event Set-up
• Your child got this food at their school
While your child was at school today, [he/she] was served lunch. School lunches cost $3.50 and
he/she paid for the meal with her pre-loaded lunch card. [He/She] told you that [he/she] had a carton of
milk, mashed potatoes, gravy, one slice of beef the size of his/her hand, and a side of corn. Please enter
this event and food into the FoodLogger app.

Scenario 2: Free – Proxy Report
Critical tasks tested:
• Start a day
• Add a food stop to FoodLogger in the text field
• Add a food event manually
• Combo meal:
o Select "combo meal" button
o Enter meal name
o Enter meal price
o Enter number of items (quantity)
o Enter individual meal items
 “Other” option (those that do not conform to pre-defined list of
combo items)
Event Set-up
• Your child got this food at their school
While your child was at school today, [he/she] had lunch there and didn’t pay for it. [He/She] told
you that, in the lunch box, there were a carton of milk, mashed potatoes, gravy, one slice of beef the size
of his/her hand palm, and a side of corn. Please enter this event and foods into the FoodLogger.

Scenario 3: Not Free – Self Report
Critical tasks tested:
• Start a day
• Add a food stop to FoodLogger in the text field OR add stop from GPS
• Add a food event manually
• Combo meal:
o Select "combo meal" button
o Enter meal name
o Enter meal price
o Enter number of items (quantity)
o Enter individual meal items
 “Other” option (those that do not conform to pre-defined list of
combo items)
o Enter payment type
Event Set-up
• You got this meal at your school
While you were at school today, you were served lunch. School lunches cost $3.50 and you paid
for the meal with you pre-loaded lunch card. The lunch you received had a carton of milk, mashed
potatoes, gravy, one slice of beef the size of his/her hand, and a side of corn. Please enter this event and
food into the FoodLogger app.

Scenario 4: Free – Self Report
Critical tasks tested:
• Start a day
• Add a food stop to FoodLogger in the text field OR add stop from GPS
• Add a food event manually
• Combo meal:
o Select "combo meal" button
o Enter meal name
o Enter payment type
o Enter number of items (quantity)
o Enter individual meal items
 “Other” option
Event Set-up
• You got this meal at your school
While you were at school today, you had lunch there and didn’t pay for it. The lunch box you got
had a carton of milk, mashed potatoes, gravy, one slice of beef the size of your hand palm, and a side of
corn. Please enter this event and food into the FoodLogger.

Appendix E: Instructions for Field Data Entry

FoodLogger evaluation report - 1

18

Instructions for Field Data Entry
In order for us to properly evaluate FoodLogger, we ask you to use FoodLogger every day for 7 days,
starting from today, to report your household’s food acquisition. It is very important that you use
FoodLogger to log all of the foods you either purchased or received for free during this 7-day period. You
should report your food by the end of each day. Don’t skip days or wait until the last day.
The foods that you should report include all the food items you will have either purchased or got for
free, regardless whether the foods are eaten or not during the 7-day period. Do not report any food that
you acquired before today. For example, if this morning you ate a bagel you bought last week, you
should NOT report that food. However, if you went to a friend’s house and had a breakfast there, you
SHOULD report that food.
If you have any problems or run into difficulties while reporting your food in FoodLogger, please log
those problems in the form we provided.

Include the date, time, and short description of the issue. This information will be very helpful for us. If
you need help with using FoodLogger, call this number, , between 7:00 am and
10:00 pm Eastern Time for assistance.

Appendix F: Debriefing Questionnaire on 7-Day Field Data Entry

FoodLogger evaluation report - 1

19

Debriefing guide for field data entry

Please rate your opinion on how easy/difficult it was to enter the following information into the
FoodLogger:
Stops:
Confirm a food stop
1: Extremely easy

2

3

4

5: Extremely difficult

Add a stop that was not automatically captured
1: Extremely easy

2

3

4

5: Extremely difficult

2

3

4

5: Extremely difficult

2

3

4

5: Extremely difficult

2

3

4

5: Extremely difficult

3

4

5: Extremely difficult

Events:
Add a food event
1: Extremely easy

Food items:
Scan a barcode
1: Extremely easy

Enter a PLU
1: Extremely easy

Enter a food item name
1: Extremely easy

2

When you typed text, did you notice the “type-ahead” feature? Did you use it? What is your
opinion?

Sometimes the app asks you to enter information on the size/weight/volume of your food
items. How easy or difficult was it to…

Enter the size/weight/volume for food items
1: Extremely easy

2

3

4

5: Extremely difficult

Enter information for school meals
1: Extremely easy

2

3

4

5: Extremely difficult

Payment: For each food event, the app asks you for the price of the entire purchase, and
sometimes it asks for the price of each food item. How easy or difficult was it to….

Enter the price for individual food items (item level)
1: Extremely easy

2

3

4

5: Extremely difficult

Enter the cost of the entire purchase (event level)
1: Extremely easy

2

3

4

5: Extremely difficult

Choose the method of payment
1: Extremely easy

2

3

4

5: Extremely difficult

[Whenever the user does not select “Extremely easy” ask: Can you please elaborate on why
you selected _(rate)___ for _(activity) (For example, why you selected 5 for entering
information for school meals)]

Did you encounter any problems with reporting a combo meal? If so, what problems?

Up to this point, what has been your overall experience with the FoodLogger?

How do you feel about the length of time it takes you to enter food information?

How do you feel about the amount of effort that is required to report the food acquisition so
far?

[If needed] What do you believe could improve this process?

FoodLogger sends notifications to you periodically. Are you bothered by the notifications?
1.
2.
3.
4.
5.

Not at all
Somewhat
Moderately
Very
Extremely

What are the three greatest challenges you have encountered using FoodLogger in the past
seven days?

Compared to the first three days, do you feel more or less comfortable with using FoodLogger
in the last three days?

Do you have any other comments or thoughts about your experiences with the FoodLogger
over the past seven days?

I have a few more questions:
You are paid $5 a day for reporting food acquisition information, for seven days. Do you think
that’s an adequate amount for your effort? [If not: How much money would you think
appropriate?]

How comfortable were you in sharing your GPS location during this study?
1. Completely uncomfortable
2. Somewhat uncomfortable

3. Neither uncomfortable or comfortable
4. Somewhat comfortable
5. Completely comfortable
Here is a list of places people can get food, please say Yes to those places where you acquired
food for yourself or family members in the past 30 days, to the best of your memory.
1. Grocery store – in-person shopping
2. Grocery store – order online for pickup or delivery
3. Big Box Store or Warehouse Club Stores (e.g., Walmart, Target, Costco) – in-person
shopping
4. Big Box Store or Warehouse Club Stores (e.g., Walmart, Target, Costco) – order
online for pickup or delivery
5. Restaurant – eat in
6. Restaurant – order online or by phone for carry out or delivery
7. Friend or family member’s house
8. Food from a church, a food pantry, a food bank, or eat-in soup kitchen
9. Other

Now, I have a few questions for your child. Can you ask him/her come?

Hello, ! How are you doing? I have a few questions to ask you about using
FoodLogger:

1. How do you like using FoodLogger?
2. On a 5-point scale, 1 being most difficult and 5 being easiest, how do you rate your
experience using FoodLogger?
3. What is the most difficult thing you encountered when using FoodLogger?
4. Is there anything else about FoodLogger do you want to tell us?