Form Attachments 1 thro Attachments 1 thro BLS.gov testing protocol

Attachment 1 - Illustrations of Drop-Down Menus

Prototype A – Click on/off Option

• Need to click on tab to open, and click on X to close.

Prototype B – Compact Menus

• Dragging mouse over menu will open it

• Menu closes when you move mouse off it (no clicking necessary)

• Higher-level categories are displayed (not as many sub-categories displayed)

Prototype C – Same as current menu, but with hover delay

• Menu drops, with slight delay, when you move mouse over it

• Moving mouse off menu makes it disappear (no clicking necessary)

• Have to scroll to see entire menu

Prototype D – Current menu (baseline)

• Current menu on BLS.gov

• Menu drops immediately (no delay) with mouseover

• Moving off menu makes it disappear quickly

• Have to scroll to see entire menu

ATTACHMENT 2: Parent/Guardian Consent Form

Parental Consent Form

The Bureau of Labor Statistics (BLS) is conducting research to improve the quality of its web-based materials. This specific study is designed to determine the effectiveness of menus provided for navigation throughout the BLS.gov website.

During this research your child will be observed by a BLS researcher, comments will be audiotaped, and a computer program will be used to record entries and timing data for later analysis. Please note that this is a not a test of your child’s abilities. Instead, we are interested in how well the website is designed and how easy it is to use. Any difficulty your child might encounter with certain tasks will provide us with information we can use to redesign the website to make it easier to use.

Participation in this research project is voluntary, and your child has the right to stop at any time. We estimate the session will last an average of 45-60 minutes. As a thank you for making the effort to come into our lab and participate, we will give your child a check for $40. If you would like to talk with a researcher before giving your permission, please see the contact information below. If you agree to allow your child to participate, please sign below.

The BLS, its employees, agents, and partner statistical agencies, will use the information your child provides for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your child’s responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. The OMB control number is 1220-0141 and expires February 28, 2015.

-------------------------------------------------------------------------------------------------------------------------------

I have read and understand the statements above, and I give my consent for my child to participate in this study.

___________________________________ ___________________________

Parent’s or Guardian’s signature Date

___________________________________ Contact Information for BLS Researchers:

Parent’s or Guardian’s printed name Dr. Bill Mockovak, 202-691-7414

OMB Control Number: 1220-0141

Expiration Date: 2/28/2015

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

ATTACHMENT 3: Participant Consent Form

Consent Form

The Bureau of Labor Statistics (BLS) is conducting research to improve the quality of its web-based materials. This specific study is designed to determine the effectiveness of menus provided for navigation throughout the BLS.gov website.

During this research you will be observed by a BLS researcher, comments will be audiotaped, and a program will be used on the computer to record entries and timing data for later analysis.

We estimate it will take you an average of 45-60 minutes to participate in this research. Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Please note that this is a not a test of your abilities. Instead, we are interested in how well the website is designed and how easy it is to use. Any difficulty you might encounter with certain tasks will provide us with information we can use to redesign the website to make it easier to use.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. The OMB control number is 1220-0141 and expires February 28, 2015.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: 2/28/2015

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

Attachment 4 - Protocol for In-House and Online Usability Testing

In-House Introduction

• Thank you for coming in today. Before we begin, have you participated in one of our studies before? (a yes answer reduces the amount of general background information about the purpose of lab studies and BLS that is provided).

• Background. Let me give you some background for what you will be doing today. The Bureau of Labor Statistics, or BLS, would like to make it as easy as possible for people to find information on its website. Therefore, I am going to ask you to search for some information on the BLS website. You will be presented with a series of tasks and asked to search for a specific section on the website or find information to answer a question.

• Now this next point is important. It’s easy to think that we’re evaluating you because we’re going to ask you to complete tasks, and I’ll be watching, but we’re really not evaluating you. We’re evaluating the website. We might have done a great job designing the website, or we might have done a poor job. The only way we can find out is to have a wide variety of people use it. So, please keep that in mind. We are not evaluating you. So relax and try to have fun today.

• Also, when some people come into our lab, they view whatever we ask them to do as a personal challenge, and they really want to do their best. That’s a good attitude up to a point, but what we’re trying to do is simulate what you would do if you were at home, or somewhere else, and you came to the BLS website to find some information. We want you to try as hard as you would if you were at home, but if you reach a point where if you were at home you would just skip the task and go to another website, please do that. This is important information for us. If you skip a task, it tells us that we haven’t made the website easy enough to use.

• Also, we use software on the computer to record what you are doing. For example, where you move the mouse, what you click on, and so forth. This just helps us summarize what people did. When we produce reports, we do not identify individuals.

• Talk aloud. One more point. As you search for information on the website, it would really be helpful to me if you would talk out loud to explain what you are doing and why. Some people feel very self conscious doing this, but it really helps us determine what you are thinking, and it gives us clues for why a task may be difficult. We have found it helps to give people a little practice doing this to warm up, so before we begin, let’s practice a bit. What I want you to do is sit back now and describe what you see when you unlock the door to your apartment or home. Just describe the first room you see after you open the door, including tables, chairs, pictures on the wall, and so on. (Allow description)

• Do you have any questions before we begin?

Consent Form

• Before we begin, I have a consent form that I would like you to read and sign. This acknowledges that you are participating freely in this study.

Tasks

Now I’m going to walk you through a couple of practice tasks on the computer, so you can see how they will be presented.

Complete 2 practice tasks (mention not to use search box)

Okay, from this point on, you are on your own, so work at your own pace, but please remember to talk out loud about. Tell me what you are thinking and why you are doing something. If you get stuck, feel free to ask me a question, and if you get frustrated and would skip a task, go ahead and skip it.

One more thing before we begin, remember not to use the search box. We’re not interested in that today.

Any questions before we begin?

Online Protocol

Note: Trymyui testers are thoroughly trained in “talking aloud.” Also, no training is required on using their system.

Introduction for Tasks

The Bureau of Labor Statistics (BLS) would like to make it as easy as possible for people to find information on its website. This voluntary study is being collected under OMB No. 1220-0141 and it is estimated it will take you no longer than 20 minutes to complete. You will be presented with a series of tasks and asked to answer a question or search for a specific page on its website. However, this next point is important. When looking for something, please do not use the search box. Make believe it’s not even there. Also, if you find that you cannot complete a task, stop and move to the next task. Have fun!

Tasks

1. Find a publication called the Occupational Outlook Quarterly. This is an online magazine about jobs and careers.

2. Find the section of this website that provides useful information for students and teachers.

3. What is the average yearly wage for all people working in New York state?

4. BLS counts the number of mass layoff events in the United States. Find out how many mass layoff events occurred during the month of January 2012

5. Find information comparing the number of people unemployed in different countries.

6. Find the section that provides information about efforts to measure the number of green jobs in the U.S.

7. Find the section that presents information about the number of injuries and illnesses in the workplace.

8. Find the section that presents information about the number of strikes by workers or lockouts of workers by employers.

Written Feedback

You can ask for written feedback on four questions at the conclusion of the tasks.

1. Overall, how easy or difficult were these tasks to complete? Were they very easy, easy, neither easy nor difficult, difficult, or very difficult? How easy do you think these tasks would be for an average American citizen using this website?

2. Please tell us what you think about the drop-down menus that appear at the top of each page. For example, did you like the way they worked? Did you like the formatting and the choice of colors?

3. Please give us more reactions to the drop-down menus at the top of each page. Did the lists that were shown make sense to you? How do these menus compare with other websites you have used?

4. Overall, how easy or difficult were the menus to use? Would you say they were very easy, easy, neither easy nor difficult, difficult, or very difficult?

Attachment 5 – Debriefing and Rating Questions for In-House Participants

1. Overall, how easy or difficult were these tasks to complete?

 Very easy

 Very

 Neither easy nor difficult

 Difficult

 Very difficult

2. How easy or difficult was it to navigate, or move around, on this site?

 Very easy

 Very

 Neither easy nor difficult

 Difficult

 Very difficult

3. Overall, how easy or difficult were the menus to use?

 Very easy

 Very

 Neither easy nor difficult

 Difficult

 Very difficult

Debriefing Questions

1. Please tell us what you think about the drop-down menus that appear at the top of each page. For example, did you like the way they worked? Did you like the formatting and the choice of colors?

2. Please give us more reactions to the drop-down menus at the top of each page. Did the lists that were shown make sense to you? How do these menus compare with other websites you have used?

System Usability Scale

Strongly Strongly

disagree agree

1. I think that I would like to use this

website frequently

2. I found the website unnecessarily

complex

3. I thought the website was easy

to use

4. I think that I would need the

support of a technical person to

be able to use this website

5. I found the various functions in

this website were well integrated

6. I thought there was too much

inconsistency in this website

7. I would imagine that most people

would learn to use this website

very quickly

8. I found the website very

cumbersome to use

9. I felt very confident using the

website

10. I needed to learn a lot of

things before I could get going

with this website

Using SUS

The SUS scale is generally used after the respondent has had an opportunity to use the website being evaluated, but before any debriefing or discussion takes place. Respondents should be asked to record their immediate response to each item, rather than thinking about items for a long time.

All items should be checked. If a respondent feels that they cannot respond to a particular item, they should mark the center point of the scale.

Attachment 6 – Checklist for Evaluating Online Testing

Prototype - __________

Task	Location of First Click	Total Number of Clicks	Success 1=Yes 2=No	Time to Complete (success only)	# of Errant Clicks
1
2
3
4
5
6
7
8
9
10

Location of First Click:

SA –Subjects Areas

DT – Database & Tools

P- Publications

ER – Economic Release

B – Beta

E - Elsewhere