Form Appendices Eye Tracking study procol

Appendix A: Structure of the Question Format Survey

Structure of Survey

Survey	Block	Contents
1	Block 1	List: Y/N question 1-3 List: Mark all questions 4-6
	Block 2	List: Mark all questions 1-3 List: Y/N questions 4-6
2	Block 3	Grid: Grid questions 1-3 Grid: Individual questions 4-6 (each “individual question” is really a page containing a series of 10 questions, to correspond to the 10 items in the grid)
	Block 4	Grid: Individual questions 1-3 Grid: Grid questions 4-6

• For the Y/N, Mark all, and Grid questions, each question will be on one page. For the individual version of the grid questions, all the questions will appear on the same page, to parallel the grid version of the question

• To counterbalance the presentation of questions and format, participants will see questions 1-3 of each type in one format and questions 4-6 in the other.

• To minimize order effects, we have built two separate surveys.

  • In the first survey, participants will be randomly assigned to either block 1, which contains Yes/No questions 1-3 and Mark all questions 4-6, or to block 2, which contains Yes/No questions 4-6 and Mark all questions 1-3.

  • At the end of the first survey, SurveyMonkey will automatically take participants to the second survey, which uses the same strategy for randomizing the grid and individual questions. Due to limitations of the randomization features of SurveyMonkey, it wasn’t possible to randomize the order of the surveys.

  • Within an individual Yes/No, Mark all, and grid question, the items (rows) will be randomized.

  • Likewise, the 10 questions for each “Individual” grid condition will be randomized within a page.

  • Within each of the two surveys, all the questions (or groups of 10 “Individual” grid question) are randomized.

Appendix B: Survey Questions for the Question Format Study

List Questions

Lists Question 1: Cell phone apps

Mark all version: Which of the following cell phone apps do you use? (Mark all that apply)

Yes/No version: Do you use the following cell phone apps?

• Banking services

• Camera

• Email

• Games

• Mapping / Navigating

• News services

• Podcast player

• Social media

• Streaming music

• Streaming video

• Weather

• Web Browser

Lists Question 2: Languages

Mark all version: What languages are you able to read or speak at even a basic level? (Mark all that apply)

Yes/No version: Can you read or speak the following languages at even a basic level?

• Chinese

• English

• French

• German

• Hindi

• Italian

• Japanese

• Korean

• Portuguese

• Spanish

• Swahili

• Urdu

Lists Question 3: Purchases

Mark all version: Which of the following items have you purchased in the last month? (Mark all that apply)

Yes/No version: Have you purchased any of the following items in the last month?

• Automobile

• Large kitchen appliances

• Housing (rent or mortgage payments)

• Utilities (e.g., gas, water, electricity)

• Furniture

• Cell phone service

• Digital book reader or tablet

• Clothing or shoes

• Tuition or child care

• Food at a restaurant

• Food for consumption at home

• Public transportation

Lists Question 4: Purchases

Mark all version: Which of the following items have you purchased in the last month? (Mark all that apply)

Yes/No version: Have you purchased any of the following items in the last month?

• Personal care products (e.g., hair care products, skin care products)

• First aid products

• Over the counter medicine

• Housekeeping supplies (cleaners, paper goods)

• Hardware or other home maintenance supplies

• Household services (housecleaning, home maintenance, or yard work)

• Entertainment (e.g., movies, music, tickets to concerts or sporting events)

• Long distance travel (not local commuting)

• Books or magazines (including digital and print editions)

• Charitable donations

• Gasoline

• Dry cleaning or tailoring

Lists Question 5: Exercise

Mark all version: Which of the following types of exercise have you engaged in anytime in the last year? (Mark all that apply)

Yes/No version: Have you engaged in the following types of exercise anytime in the last year?

• Biking / Spinning

• Dancing

• Martial Arts

• Rowing

• Running

• Skating

• Swimming

• Tai Chi

• Tennis

• Walking / Hiking

• Weightlifting

• Yoga

Lists Question 6: Sports

Mark all version: Which of the following sports do you enjoy watching? (Mark all that apply)

Yes/No version: Do you enjoy watching the following sports?

• Auto Racing

• Baseball

• Basketball

• Boxing

• Ice Hockey

• Cricket

• Football

• Golf

• Horse Racing

• Lacrosse

• Soccer

• Tennis

Grid vs Individual Questions

Grid Question 1: Frequency

Grid: How often do you do the following?

Individual: How often do you <activity>?

• Spend an evening with friends

• Attend a sporting event

• Attend a class of any kind

• Play a computer game

• Exercise

• Go to the library

• Volunteer for charity

• Cook a meal

• Read a book for pleasure

• Watch TV

Options:

• Every day

• Several times a week

• Several times a month

• Several times a year

• Never

Grid Question 2: Importance

Grid: At work, how important is it to have the following?

Individual: At work, how important is it to have <item>?

• Services such as a post office or bank

• Music in the common areas

• Opportunities for advancement

• Time off

• Benefits other than time off

• Windows in your workplace

• Parking

• A cafeteria

• Manageable commute

• Good location

Options:

• Extremely important

• Very important

• Moderately important

• Slightly important

• Not at all important

Grid Question 3: Enjoyment

Grid: How much do you enjoy listening to the following types of music?

Individual: How much do you enjoy listening to <genre>?

• Alternative

• Blues

• Classical

• Country

• Electronic

• Hip-hop

• Jazz

• Pop

• R&B

• Rock

Options:

• Enjoy a lot

• Enjoy a little

• Neither enjoy nor dislike

• Dislike a little

• Dislike a lot

Grid Question 4: Frequency

Grid: How often do you travel by the following modes?

Individual: How often do you travel by <mode>?

• Commercial Airline

• Private Airplane

• Automobile

• Commuter Train

• Amtrak Train

• Boat

• Taxi/Uber/Lyft

• Bicycle

• Motorcycle

• Bus

Options:

• Everyday

• Several times a week

• Several times a month

• Several times a year

• Never

Grid Question 5: Importance

Grid: For good health, how important is it to do the following?

Individual: For good health, how important is it to <do activity>?

• Exercise

• Eat healthy foods

• Meditate

• Spend time with friends

• Spend time with family

• Run a marathon

• Limit time watching television

• Read books

• Do crossword puzzles

• Go to bed early

Options:

• Extremely important

• Very important

• Moderately important

• Slightly important

• Not at all important

Grid Question 6: Enjoyment

Grid: How much do you enjoy the following types of movies?

Individual: How much do you enjoy <genre> movies?

• Action / Adventure

• Animated

• Comedy

• Documentary

• Drama

• Musical

• Mystery

• Romance

• Science Fiction

• Western

Options:

• Enjoy a lot

• Enjoy a little

• Neither enjoy nor dislike

• Dislike a little

• Dislike a lot

Appendix C: Envelopes and Forms

Envelope 1:

Envelope 2:

Envelope 3:

You’ve been selected to participate in the May 2017 Occupational Employment Statistics (OES) report.

See inside for how your participation provides vital wage and employment data.

Form 1:

Form 2:

Appendix D: Protocol for OES Envelopes and Forms Study

Imagine that you are a busy administrator for a business in Alabama and receive a stack of postal mail each day. On the next few pages, you will be asked to imagine you received the following envelopes at your desk. Please view each envelope at your own pace, and then we will ask you some follow-up questions about them later.

--page break--

[insert envelope 1]

--page break--

[insert envelope 2]

--page break--

[insert envelope 3]

--page break--

Now we’d like you to imagine you are a busy administrator working for a hospital in Colorado. On the next few pages, you will view some forms telling you more information about participating in the Occupational Employment Statistics report. Please view each form at your own pace, and then we will ask you some follow-up questions about them later.

--page break--

[insert form 1]

--page break--

[insert form 2]

--page break--

Next we’d like you to answer some questions about the envelopes you viewed earlier.

[insert pictures of all 3 envelopes, labeled, randomized order]

What were your reactions this envelope [insert envelope 1 / envelope 2 / envelope 3 individually for each] ______________

Which envelope do you most prefer?

• Envelope 1

• Envelope 2

• Envelope 3

Which envelope do you think businesses would be most likely to open?

• Envelope 1

• Envelope 2

• Envelope 3

-- page break --

Which logo do you prefer to appear on the envelope?

• [insert DOL logo]

• [insert BLS logo]

What language do you prefer to appear above the address window?

• “Information about submitting data inside”

• “Official government correspondence”

What language do you prefer to appear below the address window?

• “Official government correspondence”

• “For more information visit: https://www.bls.gov/respondents/oes”

What language do you prefer to appear on the left side of the envelope, below the return address?

• “U.S. Bureau of Labor Statistics Occupational Employment Statistics Report Enclosed”

• “You’ve been selected to participate in the May 2017 Occupational Employment Statistics (OES) report. See inside for how your participation provides vital wage and employment data.”

-- page break --

Please look over the first form below again:

[insert form 1]

What were your reactions to this form? _____________

How effective do you think this form would be at getting businesses to complete the OES report?

• Not at all effective

• Slightly effective

• Moderately effective

• Very effective

• Extremely effective

-- page break --

Please click ONCE on the words, images, or figures that you find the most persuasive to participate in the OES report.

[insert form 1]

-- page break --

Please look over the second form below again

[insert form 2]

What were your reactions to this form?: _________________

How effective do you think this form would be at getting businesses to complete the OES report?

• Not at all effective

• Slightly effective

• Moderately effective

• Very effective

• Extremely effective

-- page break –

Please click ONCE on the words, images, or figures that you find the most persuasive to participate in the OES report.

[insert form 2]

OMB Control Number: 1220-0141

Expiration Date: April 30, 2018

Appendix E: Consent Form

The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you and your activity on the computer will be recorded, and you may be observed. In addition, we will record your eye movements using an eye tracker. This will help us understand how you are processing the content in the study. The eye tracker follows reflections of light on your eyes to determine what you are looking at on the monitor.

If you do not wish to be recorded, you still may participate in this research.

We estimate it will take you an average of 45 minutes to participate in this research (ranging from 30 minutes to 60 minutes).

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141 and expires April 30, 2018.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974 (DOL/BLS – 14 BLS Behavioral Science Research Laboratory Project Files (81 FR 47418)), as amended (5 U.S.C. 552a) , you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. Per the Federal Cybersecurity Enhancement Act of 2015, Federal information systems are protected from malicious activities through cybersecurity screening of transmitted data.