American Community Survey (ACS) Survey Mail Materials Eye-tracking Study

American Community Survey Mail Materials Redesign

Eye-tracking Protocol

INTERVIEWER’S NAME: ______________________

DATE: ____ / ____ / 2019

START TIME: ______: ______ AM / PM

END TIME: ______: ______ AM / PM

Materials

• Demographic questionnaire

• Closed-ended recall questionnaires

  • Administer the recall questionnaires in the order corresponding to the participant’s assigned group in Table 1

Table 1: Testing material files
	1st letter	2nd letter	3rd letter	4th letter	Final
Group 1	Control	Test 3	Test 2	Test 1	C, 3, 2, 1
Group 2	Test 2	Control	Test 1	Test 3	2, C, 1, 3
Group 3	Test 3	Test 1	Control	Test 2	3, 1, C, 2
Group 4	Test 1	Test 2	Test 3	Control	1, 2, 3, C

1. Introduction

• Thank you for taking the time to meet today.

• I am [name] of the U.S. Census Bureau.

• [I’m/we’re] meeting with people like yourself to look at some new materials we will be using in the future.

• Here in our lab, we evaluate how well our survey materials work. What works well, we keep. When potential recipients, such as you, have difficulty with something, we have an opportunity to fix it before we send them out to the public.

• Today I will ask you to review some draft materials as we use eye-tracking equipment to see what grabs your attention as you look at them.

2. Interview consent

• Before we start, I would like you to read over the document in front of you.

• This document explains a little bit about this interview and provides information about your rights as a participant.

• Our session today is confidential, so all information you provide is protected and we won’t use your name in any report.

• Your participation in this study is completely voluntary. You may decline to answer any particular question.

• This form also asks for your permission to have this session audio recorded. That way, I can focus on what is going on rather than having to concentrate on taking notes or having to rely on my memory later.

• Please ask me any questions you may have about this document.

• Once you have finished reading the document, please sign it.

• (After Rs and interviewer sign consent form) I will now turn on the audio recorder and we will start the eye-tracking equipment.

3. Calibration

Before we get started, I need to calibrate the equipment. When I start the calibration, please keep your head still and follow the dot with your eyes.

• Conduct calibration procedure before the start of every letter.

• Repeat calibration if value is below _%

• Remind the participant as needed to keep their head still and maintain the same distance from the monitor.

4. Eye-tracking

Viewing letters

Open a new project and select the file corresponding to the participant’s assigned group.

I’m going to show you four letters, one at a time. I’ll start the first letter in just a moment.

Please review this letter as you might if you had received it in the mail, and let me know when you are finished.

Start letter. Note any meaningful comments and other data (e.g., frown, non-verbal utterances, etc.).

Now that your are finished, I will ask you some questions about the letter you just looked at.

Open-ended recall

Make sure the letter is gone from the monitor, then ask:

What do you remember about the letter?

What else do you remember?

Anything else? (Repeat until no new information is offered)

If no information cited, e.g., only impressions of the visual elements are cited:

Do you remember any specific information?

What else do you remember?

Anything else? (Repeat until no new information is offered)

Close-ended recall questionnaire

After each letter, give the participant the corresponding closed-ended questionnaire.

Now I’d like you to answer the questions on this page. Please keep in mind that this is not a test of how well you read or anything like that.

Preferences

After the participant completes the closed-ended recall questionnaire for each letter, ask:

If you received this letter in the mail, how likely do you think you would be to take the survey?

Start next letter and repeat the procedures, starting with calibration.

After reviewing all letters

Show the final image with all four letters and ask:

Thinking about the four letters you reviewed, which do you prefer? Why?

Which do you like least? Why?

5. Wrap-up

Do you have any other thoughts about the letter that we haven’t talked about?

Anything else you’d like us to know? (Repeat until no new information about the letters is offered)

6. Demographic questionnaire

Give participant the demographic questionnaire.

Before we wrap up, I’d like you to complete this brief questionnaire that asks for basic demographic information.

7. End the interview

Thanks very much for your time today! As promised, I have $40 for you as a token of appreciation for your help with our project.

I’ll just need you to fill out this voucher where it’s highlighted. Print your name and address, then sign and enter today’s date.

Give participant the voucher, and then the incentive.

8. Notes on interview

Any impressions or observations about the session?

Any impressions or observations about the participant?

Did the participant appear to read the text or skim? Cite observational evidence.