Eye Tracking Study for OOH

September 17, 2010

NOTE TO THE REVIEWER OF: OMB No. 1220-0141

Cognitive and Psychological Research

From: Bill Mockovak

Chief, Behavioral Science Research Center

Office of Survey Methods Research

Subject: Submission of materials for an eye-tracking study of

prototypes of a template for the Occupational Outlook

Handbook and differently formatted cover letters

Please accept the enclosed materials for approval under OMB clearance package #1220-0141, “Cognitive and Psychological Research.”

In accordance with our agreement with OMB, we are submitting a brief description of the research, and the materials to be used in the research at least two weeks prior to the beginning of the study.

A maximum of 12 burden hours is estimated for this project.

If there are any questions regarding this project, please contact Bill Mockovak at (202) 691-7414.

I. Introduction

The Occupational Outlook Handbook (OOH) is in the process of being redesigned so that its content is optimized for use on the Web. To date, the redesign process has proceeded through two design phases and usability evaluations, with the objective of arriving at a template that can be used for all occupational profiles.

The evaluation has now progressed to Phase 3. Currently, two prototypes are under consideration as the template for occupational profiles. Although standard usability testing will also be used in this phase, the purpose of the current research is to use eye-tracking to determine how elements of the alternative template designs affect what people look at, for how long, and in what order. Although we will look at all design elements, our primary objective will be to determine the impact of including a Table of Contents on search strategies, effectiveness, and efficiency.

In addition, since the eye-tracking tasks are not expected to last a full session (60 minutes), another eye-tracking study will be piggybacked onto this effort. This second eye-tracking study will investigate the impact of using different visual designs (formats) for survey cover letters (see Attachment 2 for the two versions being compared) on reading patterns and efficiency.

II. Methodology

Twelve adult participants (21 or older) will be recruited from the database maintained by OSMR. We will attempt to recruit as diverse a group as possible based on education, age, and work status. A minimum requirement will be proficiency using the Internet. Each participant will be asked to complete simple information-seeking tasks using the two prototypes (Chef and Childcare worker).

Tasks. Participants will be randomly assigned to two groups of 6 each: Group 1 will work with Prototype A first, Group 2 will work with Prototype B first. Although participants will use multiple pages of the prototype, the only page of interest is the initial or summary page (see Attachment 1). Participants will be asked to complete the following tasks for both prototypes.

A. Questions for the Childcare worker Prototype

1. In which job setting are childcare workers required to obtain a Child Development Associate (CDA) credential?

2. Is the following statement true or false? “The job market for childcare workers is good.”

B. Questions for the Chef Prototype

1. How much can chefs and head cooks expect to earn in a year?

2. Who do sous chefs and other cooks generally work for in a kitchen?

Following this study, participants will be asked to read two survey cover letters displayed as pdf files. Group 1 will read the Information Mapping Version first, followed by the Control version. Group 2 will read the letters in reverse order.

The following dependent measures will be used for each study.

Dependent Measures for the OOH tasks. The key measures will be the following:

• Scanning pattern. We want to determine how inclusion of the Table of Contents affects scanning/reading behaviors on the Summary Page.

• Effectiveness & Efficiency. We will calculate a success rate for the tasks, as well as how long participants take to complete each task (we realize the small sample size precludes making any generalizations about effectiveness or efficiency).

Dependent Measures for the Cover Letter. The key measures will be the following:

• Scanning pattern. We want to determine how the different formats affect scanning/reading behaviors on the Summary Page.

• Time to read each letter. We will compare average times for reading each letter.

In addition, a short knowledge test will be given to each participant after the first reading to see if key information was acquired (see Attachment 4).

III. Burden hours

This study will take place in the cognitive lab, and last an average of 45-60 minutes. Although a relatively small number of tasks is included in this study, eye-tracking studies take longer to set up and run because the equipment has to be calibrated (and re-calibrated, if necessary). Since there will be 12 participants, the total estimated respondent burden hours is a maximum of 12 hours.

IV. Data Confidentiality

Participants will be informed as to the voluntary nature of the study, and told that they are contributing to the redesign of the Occupational Outlook Handbook, as well as more effective cover letters. Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice. A copy of the consent form is included as Attachment 3.

V. Attachments

1. Attachment 1 – Prototypes for Childcare Workers and Chefs

2. Attachment 2 – Alternative cover letter formats

3. Attachment 3 – Consent form

4. Attachment 4 – Knowledge test

Attachment 1 – Prototypes for Childcare Workers and Chefs

Attachment 2 – Text for Information Mapping Cover Letter

Dear Handbook Purchaser	We are sending this survey to you because you, or your organization, purchased a copy of the Occupational Outlook Handbook in the past.
What is the purpose of this survey?	Because use of the print version of the Occupational Outlook Handbook has decreased dramatically over the past decade, BLS is considering ending production of the print version and producing only an online version better suited for (or “optimized for”) the Web. However, before stopping production, we would like to hear from purchasers of the Handbook about how this decision will impact them or others.
What if I do not use the Handbook?	Please give the questionnaire to someone who uses the Handbook. If no one is available who uses the Handbook, please return the blank questionnaire.
How long will this take?	This brief survey should take less than 5 minutes.
Two ways to respond: Mail or Web	You can mail back the questionnaire in the enclosed envelope (no postage necessary), or You can complete the form online: http://www.surveymonkey.com/S/H1
When should I mail back the questionnaire?	Please return the questionnaire within 2 weeks.
Will my responses be kept confidential?	Your responses will be kept confidential, and we will not link your name with your opinions. The OMB Approval Number for this study is 1225-0059, expiring 11/30/2012.
Whom can I contact with questions or comments?	Please contact Dr. William Mockovak, 202-61-7414, [email protected].

Text for Control Cover Letter

Dear Handbook Purchaser,

We are sending this survey to you because you, or your organization, purchased a copy of the Occupational Outlook Handbook in the past.

Because use of the print version of the Occupational Outlook Handbook has decreased dramatically over the past decade, BLS is considering ending production of the print version and producing only an online version better suited for (or “optimized for”) the Web. However, before stopping production, we would like to hear from purchasers of the Handbook about how this decision will impact them or others.

This brief survey should take less than 5 minutes.

1. You can mail back the questionnaire in the enclosed envelope (no postage necessary), or

2. You can complete the form online: http://www.surveymonkey.com/S/M2

If you do not use the Handbook, please give the questionnaire to someone who does. If no one is available who uses the Handbook, please return the blank questionnaire.

Please return the questionnaire within 2 weeks.

Your responses will be kept confidential, and we will not link your name with your opinions. The OMB Approval Number for this study is 1225-0059, expiring 11/30/2012.

If you have questions or comments, please contact Dr. William Mockovak, 202-61-7414, [email protected].

Sincerely,

William P. Mockovak, Ph.D.

Office of Survey Methods Research

Bureau of Labor Statistics

Attachment 3 - Consent Form

The Bureau of Labor Statistics (BLS) is conducting research to improve the quality of the Occupational Outlook Handbook. This study will examine how you use the Handbook to find information about occupations, and how different design features affect your use of the Handbook. In addition, you will be asked to read some cover letters to assess the impact of different formats.

The BLS, its employees, agents, and partner statistical agencies will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio and/or videotaped, and you will be observed. If you do not wish to be taped, you still may participate in this research.

We estimate it will take you an average of 60 minutes to participate in this research.

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141 and expires February 29^th, 2012.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: [02/29/12]

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

Attachment 4 – Knowledge Test

After the first reading only, the participants will be asked the following multiple choice questions:

1. What is the purpose of the survey? To determine …

   1. If you would be interested in purchasing another copy of the Occupational Outlook Handbook

   2. How stopping production of a print version of the Occupational Outlook Handbook will affect you

   3. If you enjoy using the optimized, online (Web) version of the Occupational Outlook Handbook

   4. Why people are not purchasing printed copies of the Occupational Outlook Handbook

2. Is the survey voluntary?

   1. Yes

   2. No

   3. It’s not clear

3. When should the questionnaire be mailed back?

   1. Within 5 days

   2. Within 1 week

   3. Within 2 weeks

   4. Within 3 weeks

   5. Within 4 weeks

   6. No time limit

4. How long should it take to complete the survey?

   1. Less than 1 minute

   2. Less than 2 minutes

   3. Less than 3 minutes

   4. Less than 4 minutes

   5. Less than 5 minutes