appendix 1 - Study Sample

OMB Control Number: 0560-0286

OMB Expiration date: 11/30/2020

Farmers.gov Tree Testing Study Plan

Objectives

The objective of this study is to evaluate the findability of content through the Farmers.gov navigation hierarchy and labeling system. The research question for this study is:

RQ: Does the current structure and labeling system of the Information Architecture support the way users look for information in order to complete priority tasks?

Methods

A Tree Test is a task-based research method that quantifies the findability of priority content on a website by using only the labels and structure of the navigation. Participants will be asked to find priority information by only using a navigational tree, without any design elements or search functions.

The tests will be performed remotely via the online tool Tree Jack and will be unmoderated. The entire study should take no more than 20 min. Users will be able to complete it on their own time using a desktop computer with internet connection. Some of the metrics we will be able to gather from Tree Jack include:

• Task by task success rates

• Time to complete task successfully

• Path analyses

• Destination table with probable problem areas

Priority User Tasks (will be randomized)

1. A hurricane has impacted your cattle. See if there are any resources that can help you with that.

   1. Success = Recover -> Livestock Indemnity Program

   2. Success = Recover -> Emergency Assistance for Livestock, Honeybees, and Farm-Raised Fish Program

   3. Success = Recover -> Disaster Assistance Discovery Tool

2. You need to schedule time to meet with your local USDA service center representative. Find where you can do that.

   1. Success = Connect -> Make an Appointment

3. You want to enlarge your existing farm and you want to find out if the USDA can help you with the necessary finances. See where you can find this information.

   1. Success = Fund -> Farm Loans - > Farm Ownership Loan Program

4. You are new to crop farming and you want to find out the most important USDA deadlines (crop insurance deadlines etc.) in one place. Where can you find this information?

   1. Success = Connect -> Search the Calendar

5. You have decided that you want to become a farmer and you want to find out what economic assistance is available to you. See where you can find this information.

   1. Success = Fund -> Targeted Loans -> Beginning Farmers and Ranchers Loans

6. Protecting your farm land and the environment is important to you as a farmer and you want to learn how you can do that. See where you can find that information.

   1. Success = Conserve -> Explore Conservation Practices

7. Your vines were impacted by excessively strong winds last year and you want to know if you can receive any help from USDA. Where can you find this information?

   1. Success = Recover -> 2017 Wildfires and Hurricanes Indemnity Program

   2. Success = Recover -> Tree Assistance Program

   3. Success = Recover -> Disaster Assistance Discovery Tool

8. You are looking for help with your farm’s irrigation system. Where can you find that ?

   1. Success = Conserve -> Learn about funding for conservation -> Environmental Quality Incentives Program

   2. Success = Recover -> Environmental Quality Incentives Program

9. You want to find out about field days coming up near you. See where you can locate this information.

   1. Success = Connect -> Calendar

10. You are interested in finding trade data for your crops. See where you can find that.

    1. Success = Manage -> MARS Market Prices

11. You are ready to step down and hand over the day-to-day operations of your farm to someone else and you want to learn about the process. See where you can find this information.

    1. Success = Manage-> Retirement Information

Post-Task Questions

After each task:

Task completion confidence on a 1-7 scale

• 1 = Not at all confident

• 7 = Extremely confident

Task difficulty on a 1-7 scale

• 1 = Not Difficult at All

• 7 = Extremely Difficult

Post-Test Questions

At the end of the test:

What was the most difficult thing about this exercise and why?

(List of all tasks to aid user cognition)

Recruitment

The ideal sample size for this study is 35-50 participants. We will recruit participants through the use of a recruitment survey published on the live farmers.gov site. Participants will need to meet certain criteria to be eligible for this study (see Farmers.gov Usability Testing Screener language section).

Farmers.gov team members will be responsible for identifying potential candidates and communicating with them about the study. Participants will be sent a link to the Tree Test to complete on their own time before the due date. There will be no incentive offered.

Farmers.gov Usability Testing Screener

1. How familiar are you with farmers.gov

1. I’ve never heard of it

2. I’m aware but have never used it

3. I’ve used it a few times

4. I use it often

2. Which category best describes you?

   1. I’m a beginning farmer

   2. I’m a mid-career farmer

   3. I’m an established farmer

   4. Neither

3. What type of farming do you do?

   1. Open ended

4. Do you have access to a computer with reliable internet connection?

   1. Yes

   2. No

5. How old are you?

   1. Under 18

   2. 18 -23

   3. 24 - 44

   4. 45 - 54

   5. 55-65 +

6. Please select timeslots when you are available for a one-hour discussion session

Grid with timeslots

7. What is your contact email address or phone number (whichever you prefer)?

1. Open ended

Reporting

The results of this study will be presented in a findings and recommendation report shortly after the completion of the study.

Scheduling

Scheduling will be done manually by MetroStar System team members via email.

Recruitment Message

We need help from producers like you. Please take a few minutes to answer a few questions to see if you are eligible to participate in a study with farmers.gov

PII and Data Management

All PII and personal data from potential participants will be stored in the federally-approved survey software (e.g., Qualtrics).

Any use of data stored in the survey software will be anonymized and aggregated. Documents containing PII that are exported from the survey software, such as Microsoft Excel files, will be password protected or encrypted.