May 12, 2010
NOTE TO THE REVIEWER OF: |
OMB No. 1220-0141 Cognitive and Psychological Research
|
FROM: |
William Mockovak, Director Behavioral Sciences Research Center, Office of Survey Methods Research (OSMR)
|
SUBJECT: |
Usability Testing of Search Prototypes for the Redesigned Occupational Outlook Handbook Website
|
Please accept the enclosed materials for approval under the OMB clearance package “Cognitive and Psychological Research.”
In accordance with our agreement with OMB, we are submitting a brief description of the research, and the materials to be used in the research at least two weeks prior to the beginning of the study.
In this phase of developing an Occupational Outlook Handbook redesigned for the Web, two versions of a homepage will be tested that offer a variety of search functions to Handbook users. In addition, different prototypes for “landing pages” and an advanced search function for occupations will be tested.1
Our plan is to recruit participants from several different groups that might use the Handbook, including high school students, college students, working adults, unemployed or nonworking adults, and high school and college career counselors. Since we do not know how successful our recruiting efforts will be within each group, the sample sizes identified in this request reflect our best estimates at this time. Most testing will be done by BLS staff but BLS will also contract with a firm that specializes in web testing for some of the testing. We anticipate this research will require 60 burden hours.
If there are any questions regarding this project, please contact Bill Mockovak (202) 691-7414.
Introduction and Purpose
The BLS Employment Projections program produces a variety of outputs, ranging from technical materials to career information for kids. Its most widely used product is the Occupational Outlook Handbook, which accounts for about one-third of BLS.gov website's traffic (the Web version receives about one million separate visits monthly). The decision was made several years ago to redesign the Handbook for the Web. This research is another step in that process.
Previous research conducted with the online Occupational Outlook Handbook (OOH) first focused on identifying major groups that use the OOH. Once these groups were identified, focus groups and usability testing were used to produce both qualitative and quantitative assessments of the current website (http://www.bls.gov/oco/) and to determine user requirements. These requirements were then used to develop prototypes for a new OOH website.
In the next round of testing, the prototypes were evaluated, a final prototype was selected for the redesigned Handbook, and rewriting of the occupational profiles started using the new template.
The primary objective of this phase of the research and development process is to evaluate a new homepage that offers a variety of search functions to Handbook users. In addition, different prototypes for “landing pages” will be tested. “Landing Pages” are used to display information for a set of related occupations, for example, different types of teachers, engineers, or physicians.
Volunteers from different user groups will be asked to evaluate the new homepage, search functions, and landing pages using both in-house and remote-usability testing in the OSMR lab. In addition some testing will take place through a contract by a firm specializing in web testing.
II. Data Collection Methodology
The primary objective of the usability testing is to compare the performance of different user groups on the new homepage, search functions, and landing pages. Both quantitative and qualitative measures of performance will be collected. For example, quantitative measures will include time to complete a task and success rate. Qualitative measures will include ratings of ease of use, preference ratings, and open-ended feedback about preferences for different versions, aesthetics, layout, formatting, etc.
OOH User Populations to Be Tested
The following groups will be tested in the May through July 2011 timeframe:
Group |
Estimated Number |
High school students |
25 |
College students |
10 |
High school career counselors |
6 |
College career counselors |
6 |
Adults - working |
6 |
Adults – unemployed or not currently in labor force |
7 |
Total |
60 |
Although we are unlikely to exceed the total number of participants, the exact breakdown by user group reflects a best guess at this time. Actual numbers will depend on the success of recruiting efforts.
Also, most of the adults (working or unemployed) will be tested in the BLS lab. All of the counselors will participate using the remote testing setup. The website testing firm will recruit and test from each of the groups (except for counselors).
Importance of Incentives for Testing
Given ongoing budgetary constraints, our intent is to solicit voluntary (non-incentive) participation whenever possible. However, we know from past attempts to organize focus groups and to recruit for both internal and external usability testing that recruiting representatives from some groups can be difficult, and the costs of recruiting can easily exceed the costs of incentives. For example, in past evaluations we have conducted focus groups with high school and college career counselors but when we tried to recruit counselors for remote usability tests, our success rate was only 10 percent (we have not yet tested students remotely) with a $25 incentive. We discovered that counselors are very busy people – hence the need for a $40 incentive.
As far as students are concerned, our past experience has clearly revealed that we need to offer an incentive to obtain participants from this group. Current plans are only to test students in the lab. To summarize, offering an adequate incentive has the following benefits:
There is less likelihood people will miss a scheduled session.
We can usually recruit a more representative group of participants.
The first point is especially important because it takes time to prepare for a testing sessions (internal or remote). Since we always encourage members of the development team to observe testing sessions, it is extremely frustrating for everyone involved when a participant does not show. The cost of wasted staff time easily exceeds the cost of the incentive.
Examples of Tasks
Participants will be asked to search for information starting from the redesigned homepage of the Occupational Outlook Handbook. Sample tasks will be similar to the following, although the exact occupations may vary:
Find career information about being a nurse
Find career information about being an electrical engineer
Find career information about being a hazardous waste material specialist
What are the three fastest growing occupations (in terms of number of total jobs)?
Which occupations pay the best?
Search for any occupation that you are interested in learning more about?
What occupations are other visitors to this site interested in?
In-Person Protocol
When usability testing is conducted in a face-to-face setting, participants will:
Be briefed about the study, read a description of the study (or both), and complete a consent form. Consent forms will be obtained from the parents or guardians of students, ages 14-17 (see Attachment 2). In addition, students will be asked to sign a standard consent form before the study is started (see Attachment 3).
Complete a brief questionnaire to obtain some basic demographic and background information about each participant (Attachment 1A).
Complete a series of usability tasks.
A general debriefing will follow.
The in-person testing protocol is shown in Attachment 1A. The procedure will be modified as follows for remote testing.
Remote Testing
Since we need to have consent before we can proceed with a remote testing session, we provide participants with information ahead of time so that we can obtain either verbal or e-mail acknowledgement of consent prior to the start of a session.
One possible approach would be to send the consent form to participants, ask them to read and sign it, and then have them return it prior to their scheduled session. However, we want to avoid this step because of the additional time it takes. We often schedule testing sessions on a short time schedule, with very little advanced notice, so the additional days required to ask a participant to mail back a consent form, and hoping a participant will do so in a timely manner (even with prompts) can seriously disrupt our schedule.
The following steps summarize our procedure.
Remote testing will be conducted using an application, call Webex, developed by Cisco. Use of Webex has been approved by BLS security for this purpose. Another possibility is that we will use Loop11, a completely automated tool for remote usability testing that we have used in the past.
Participants will be contacted in advance and mailed a consent form and, if an incentive is involved, they will also be mailed a BLS administrative Form (SF-1034) at the same time. This form is required by BLS to confirm receipt of convenience checks.
To avoid asking a participant to mail back the consent form and SF-1034 in separate envelopes, in sessions involving a facilitator, the BLS researcher will obtain either taped verbal consent or ask the participant to respond to an e-mail stating that he or she is participating voluntarily in the scheduled session.
After the session is completed, participants will return the signed consent form and Form SF-1034 in a pre-addressed envelope.
Once OSMR receives the SF-1034 and signed consent form, we will mail a $40 check to the participant (unless the participant is a child, in which case the reimbursement will be limited to $25.)
If we use Loop11, the procedure can be simplified because a facilitator is not interacting directly with the participant. In this case, the forms are mailed to participants as described above, and then once participants acknowledge consent via e-mail, they will be able to access the Loop11 application at their convenience. Once the usability testing session is compete, participants will mail back the SF-1034 and consent form, and OSMR will mail them a check.
The remote testing protocol is shown in Attachment 1B.
Analysis will focus on time taken to complete tasks, task success, comments made during the tasks, and the follow-up debriefing session.
A maximum of 60 burden hours is expected. We estimate a session will last an average of 45 minutes, and not exceed an hour.
IV. Data Confidentiality
Participants will be informed as to the voluntary nature of the study. Participants will be informed that the study will be used for internal purposes to improve the design of the Occupational Outlook Handbook (OOH). In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
V. Attachments
Attachment 1A: In-house testing protocol
Attachment 1B: Remote testing protocol
Attachment 2: Parent/Guardian Consent Form
Attachment 3: Participant Consent form
ATTACHMENT 1A - Protocol for In-House Usability Testing
Thank participant for coming.
Ask if person has participated in previous studies in BLS.
Introduce any observers.
Goal for today. We are in the process of redesigning our website that provides detailed career information about hundreds of occupations. It’s called the Occupational Outlook Handbook. Have you heard about it by any chance? Have you ever visited the website for the Handbook?
As I mentioned, we’re in the process of redesigning this website but we’re not sure what the best design is for the homepage, so today I’m going to ask you to search for some information about different occupations using different homepage designs we are considering. After you have finished the tasks, we’ll discuss the different versions, and you can tell me which design you prefer, and why.
Reminder. An important point to keep in mind is that we are not evaluating you today. We are evaluating different designs for the homepage, so if you have problems with a task or one of the designs, that’s actually a good thing because then we know we have to make changes. Since this isn’t a test, try to relax. We don’t care how much of a computer whiz you are. Try to make believe you are working on your home computer with no one looking at you.
Procedures & Consent Form. As part of what you’ll be doing today, we have software on the computer that captures your keystrokes and tells us where you clicked on the screen. This software does a lot of work for us, so we don’t have to worry about missing something. Although we collect this information, we will not share it with anyone. We use it only for statistical purposes, and we don’t link your name with it. Now, before I ask you to actually do anything, I’d like you to read and sign this form which states that you are participating voluntarily in this research (Obtain signed consent form).
Start. Start test.
Note. Order of prototypes will vary.
General Instructions – I am going to ask you to look for information today about different occupations. In each case, your search will start on one of two versions of the homepage that we have developed.
For some of the tasks, all you have to do is go to a web page, and you’ll be done, but other tasks will ask you to find some information on the page and then answer a question. Please remember that this is not a test. We’re interested in seeing how easy it is for you to find the answer to a question, so it’s not important if you get the right answer or not. In fact, if a lot of people come up with the wrong answer, that’s very useful information for us because it tells us that our current approach is not the best one, and that the information is too difficult to find.
I also want to point out that you will not be working with the actual Occupational Outlook Handbook today. Instead, we have set up a simplified version of the web site, and some of the actual pages you will be seeing are from our old Handbook before it was redesigned for the Web. However, you will also be looking at some pages that have a new design. I don’t think you will have any problem figuring out which ones these are.
Talk aloud. As you complete the form, I would like you to talk out loud to explain what you are doing and why you are doing it. Some people feel self conscious doing this, but it really helps us determine what you are doing and why. We have found it helps to give people a little practice doing this to warm up, so before we begin, let’s practice a bit. What I want you to do is sit back now and describe what you see when you unlock the door to your apartment or home. Just describe the first room you see after you open the door, including tables, chairs, pictures on the wall, and so on. (Allow time for description) Now, instead of describing a room, I want to describe your thoughts and reactions as you complete our online form in a couple of minutes.
Explain how UTE will present tasks, and work through practice tasks.
Instructions: You are going to work through two practice tasks to give you the chance to get comfortable with what we are doing, and then you will complete 6 tasks that ask you to look for information. After the sixth task, you will see a homepage that is formatted differently. I’ll alert you when this happens so the change doesn’t surprise you. You will then complete six more tasks with the second version of the homepage.
Tasks
The following tasks will be used in the usability test. The order of the homepages (1 or 2) will be changed for each participant. For example, Participant 1 will work with Version A first, Participant 2 will work with Version B first, but each participant will work with both versions. P1 and P2 are practice tasks.
Order 1
|
Homepage Version |
Question |
Answer |
P1 |
A |
Assume you are a math major in college. Since you like math, you are interested in finding out which occupations require math skills, even though you don’t want to be a pure mathematician. Where would you start looking? |
Click Math under Groups of Occupations tab, and then Mathematicians on landing page |
P2 |
A |
Assume you are good with your hands. You also like machines and tools, and are interested in being some type of mechanic. Where would you look to find out what occupations mechanics could work in? |
|
1 |
A |
Your cousin has told you he is thinking about becoming a physical therapist, and was wondering what he could expect to earn, on average (median) annually. What is the answer to his question? |
$72,790 annually |
2 |
A |
A friend asked you if you knew what occupation was growing the fastest. What would you tell her? |
Biomedical engineers |
3 |
A
|
A family friend thinks she would like to work in healthcare, and asks you if you could help her find out what general types of occupations are in that field. Go to the pages that provide this information. |
Under “Groups of Occupations,” click Healthcare Practitioners and Healthcare support (use Detailed Landing Page) |
4 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, find out what the annual pay of audiologists is (figure is for 2008). |
$44,342 (use Photo Grid Landing Page) |
5 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the most growth in the number of jobs between 2008 and 2018? |
Speech-language pathologists (use Growth Landing Page) |
1 |
B |
A member of your family is changing jobs and said he is thinking about becoming an athletic trainer, and wonders what he could expect to earn each year. What is the answer to his question? |
$39,640 |
2 |
B |
A friend asked you if you knew which occupation had the most new job openings. What would you tell her? |
Registered nurses |
3 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, what is the projected growth rate in jobs for audiologists between 2008 and 2018? |
Minus 28% (use Photo Grid Landing Page) |
4 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the biggest decline in the number of jobs between 2008 and 2018? |
Dentists (-56%) (use Growth Landing Page) |
5 |
B |
A family member is leaving the military and wonders what jobs might be open to him that require only a high school education, but still pay at least $50,000 a year. What would you tell him? |
Railroad conductors & yardmasters, rotary drill operators, subway/streetcar operators |
Order 2
|
Homepage Version |
Question |
Answer |
P1 |
B |
Assume you are a math major in college. Since you like math, you are interested in finding out which occupations require math skills, even though you don’t want to be a pure mathematician. Where would you start looking? |
Click Math under Groups of Occupations tab. |
P2 |
B |
Assume you are good with your hands. You also like machines and tools, and are interested in being some type of mechanic. Where would you look to find out what occupations mechanics could work in? |
|
1 |
B |
A member of your family is changing jobs and said he is thinking about becoming an athletic trainer, and wonders what he could expect to earn each year. What is the answer to his question? |
$39,640 |
2 |
B |
A friend asked you if you knew which occupation had the most new job openings. What would you tell her? |
Registered nurses |
4 |
B
|
A family friend thinks she would like to work in healthcare, and asks you if you could help her find out what general types of occupations are in that field. Go to the pages that provide this information. |
Under “Groups of Occupations,” click Healthcare Practitioners and Healthcare support (use Detailed Landing Page) |
5 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, what is the projected growth rate in jobs for audiologists between 2008 and 2018? |
Minus 28% (use Photo Grid Landing Page) |
6 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the biggest decline in the number of jobs between 2008 and 2018? |
Dentists (-56%) (use Growth Landing Page) |
1 |
A |
Your cousin has told you he is thinking about becoming a physical therapist, and was wondering what he could expect to earn, on average (median) annually. What is the answer to his question? |
$72,790 annually |
2 |
A |
A friend asked you if you knew what occupation was growing the fastest. What would you tell her? |
Biomedical engineers |
5 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, find out what the annual pay of audiologists is (figure is for 2008). |
$44,342 (use Photo Grid Landing Page) |
6 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the most growth in the number of jobs between 2008 and 2018? |
Speech-language pathologists (use Growth Landing Page) |
3 |
A |
A family member wants to change jobs and wonders what jobs might be open to him that only require only a high school education, pay at least $25,000 a year, and have a growth rate between 8 and 15 percent over the next 10 years. What would you tell him? |
Bus drivers |
Debriefing Questions
The debriefing questions are described next. For easier administration, these have been combined into two attachments: one for the general population, and one for counselors.
Homepage comparison (paper copies of the homepages will be handed to participant)
We used two different versions of the homepage today. Please look them over again, and tell me which version you prefer.
Version A
Version B
No preference
Why do you prefer that version?
Occupation Groups
As you can see, we varied the format of Occupation Groups a bit. On Version A, they are listed in a column down the left hand side of the page. And, on Version B, they are shown in two columns. Do you have a preference for either version?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
Location of Filter & Number of Options
Another thing we varied between the two versions is this feature that allows you to sort occupations by different criteria like pay, required education, and the number of projected new jobs. That feature appears here on Version A, and here on Version B. Which version do you prefer?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
Looking at the sorting feature again, Version A offers five ways to sort occupations, and Version B offers three ways. Which version do you prefer?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
When we were thinking about this sorting feature, we came up with five characteristics of occupations that we thought would be of most interest. These are: education level, the projected number of new jobs, pay in 2010, the training required, and the anticipated growth rate.
Can you rank order these from 1 to 5, where 1 is the most important characteristic to you.
Rank 1 ___
Rank 2 ___
Rank 3 ___
Rank 4 ___
Rank 5 ___
Search Boxes
We also varied the number of search boxes between the two versions. Version A has one search box up located up here, and Version B has two search boxes: one in the same location, and one farther down the page (point them out). Which version do you prefer?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
Fastest Growing Jobs (projected) & Most New Jobs (projected)
We also added some links that we thought would be of general interest to people visiting this website. For example, you can click on the highest paying jobs, the fastest growing jobs, the jobs that most visitors to this site are looking at (most viewed), and jobs with the most new job openings.
I wanted to talk to you about this link: Fastest Growing Jobs (projected). Can you tell me what that means in your own words?
We also have the link Most New Jobs (projected). Can you tell me what the difference is between these two links?
Finally, I wanted to get your opinion about the Most Viewed link (point it out). How likely do you think it is that visitors to this website will find that link useful. Would you say it was (read categories):
Very likely
Likely
Neither likely nor unlikely
Unlikely
Very unlikely
Filter Occupations - “Click go” instruction
Version A has this Click go when done instruction following the heading “Filter Occupations By” (point this out). Version B does not have this instruction.
Did you notice this instruction?
Yes
No
Not sure
Did you think it’s helpful?
Yes
No
Not sure
OOH Occupation Finder Page
Do you remember this page?
Yes
No
Not sure
Did you read these instructions (Filter & Sort)?
Yes
No
Not sure
How clear would you say these instructions are?
Very clear
Clear
Neither clear nor unclear
Unclear
Very unclear
Landing Page comparisons (paper copies of the landing pages will be handed to participant)
In some career fields, there are related types of occupations. For example, today you saw occupations that fell under Healthcare Practitioners and Healthcare Support. We used three different formats for presenting this information. I want you to look over these versions and then order them from 1 to 3, where 1 is the version you like most, 2 is the next preferred, and 3 is the one you like the least.
Rank 1 _____
Rank 2 _____
Rank 3 _____
What do you like about the version you ranked first?
What do you dislike about the version you ranked last?
OOH Occupation Finder (a paper copy of the advanced search occupation finder will be handed to participant)
We developed this table to help you search quickly for occupations that met certain criteria, for example, jobs that pay more than a certain amount of money, jobs that are expected to experience a lot of growth, and jobs that require a certain level of education.
How easy or difficult was this table to use?
Very easy
Easy
Neither easy nor hard
Difficult
Very difficult
Did not use
If you had encountered this table on the website, how likely or unlikely is it that you would have used it?
Very likely
Likely
Neither likely nor unlikely
Unlikely
Very unlikely
Do you have any comments about the table that would help us improve it or make it easier to use?
Background Questions (These will be collected on the computer after the session)
Before we begin, there are a few questions about yourself that I would like you answer.
How would you rate your level of experience using the Internet?
I am fairly new to it
I have some experience, but still much to learn
I have moderate (about average) experience
I have much (considerable) experience
I use the Internet a lot and understand its intricacies
What is your age?
18-21
22-25
26-30
31-40
41-50
51-60
61 or older
What is the highest level of education you have completed?
Some high school
Some high school, but with technical training
High school graduate or GED
Some college, but no degree
Technical training past high school
Associate college degree (2-year program, for example, A.A. or A.S.)
College degree (for example, B.A. or B.S.)
Some post-graduate education, but no degree
Post-graduate degree (for example, M.A., M.S., Ph.D., or professional
degree)
A. Homepage comparison (hand paper copies of the homepages to participant)
We used two different versions of the homepage today. Please look them over again, and tell me which version you prefer.
Version A
Version B
No preference
Why do you prefer that version?
B. Occupation Groups
As you can see, we varied the format of Occupation Groups a bit. On Version A, they are listed in a column down the left hand side of the page. And, on Version B, they are shown in two columns. Do you have a preference for either version?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
C. Location of Filter & Number of Options
Another thing we varied between the two versions is this feature that allows you to sort occupations by different criteria like pay, required education, and the number of projected new jobs. That feature appears here on Version A, and here on Version B. Which version do you prefer?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
Looking at the sorting feature again, Version A offers five ways to sort occupations, and Version B offers three ways. Which version do you prefer?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
When we were thinking about this sorting feature, we came up with five characteristics of occupations that we thought would be of most interest. These are: education level, the projected number of new jobs, pay in 2010, the training required, and the anticipated growth rate.
Please rank order these from 1 to 5, where 1 is the most important characteristic to you.
Rank 1 ___
Rank 2 ___
Rank 3 ___
Rank 4 ___
Rank 5 ___
D. Search Boxes
We also varied the number of search boxes between the two versions. Version A has one search box up located up here, and Version B has two search boxes: one in the same location, and one farther down the page (point them out). Which version do you prefer?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
E. Fastest Growing Jobs (projected) & Most New Jobs (projected)
We also added some links that we thought would be of general interest to people visiting this website. For example, you can click on the highest paying jobs, the fastest growing jobs, the jobs that most visitors to this site are looking at (most viewed), and jobs with the most new job openings.
I wanted to talk to you about this link: Fastest Growing Jobs (projected). Can you tell me what that means in your own words?
We also have the link Most New Jobs (projected). Can you tell me what the difference is between these two links?
Finally, I wanted to get your opinion about the Most Viewed link (point it out). How likely do you think it is that visitors to this website will find that link useful. Would you say it was (read categories):
Very likely
Likely
Neither likely nor unlikely
Unlikely
Very unlikely
F. Filter Occupations - “Click go” instruction
Version A has this Click go when done instruction following the heading “Filter Occupations By” (point this out). Version B does not have this instruction.
a. Did you notice this instruction?
Yes
No
Not sure
b. Did you think it’s helpful?
Yes
No
Not sure
G. OOH Occupation Finder Page
Do you remember this page?
Yes
No
Not sure
Did you read these instructions (Filter & Sort)?
Yes
No
Not sure
How clear would you say these instructions are?
Very clear
Clear
Neither clear nor unclear
Unclear
Very unclear
H. Landing Page comparisons (hand paper copies of the landing pages to participant)
In some career fields, there are related types of occupations. For example, today you saw occupations that fell under Healthcare Practitioners and Healthcare Support. We used three different formats for presenting this information. I want you to look over these versions and then order them from 1 to 3, where 1 is the version you like most, 2 is the next preferred, and 3 is the one you like the least.
Rank 1 _____
Rank 2 _____
Rank 3 _____
a. What do you like about the version you ranked first?
b. What do you dislike about the version you ranked last?
I. OOH Occupation Finder (hand a paper copy to participant)
We developed this table to help you search quickly for occupations that met certain criteria, for example, jobs that pay more than a certain amount of money, jobs that are expected to experience a lot of growth, and jobs that require a certain level of education.
How easy or difficult was this table to use?
Very easy
Easy
Neither easy nor hard
Difficult
Very difficult
Did not use
If you had encountered this table on the website, how likely or unlikely is it that you would have used it?
Very likely
Likely
Neither likely nor unlikely
Unlikely
Very unlikely
a. Do you have any comments about the table that would help us improve it or make it easier to use?
A. Homepage comparison (had paper copies of the homepages to participant)
We used two different versions of the homepage today. Please look them over again, and tell me which version you prefer.
Version A
Version B
No preference
Why do you prefer that version?
B. Occupation Groups
As you can see, we varied the format of Occupation Groups a bit. On Version A, they are listed in a column down the left hand side of the page. And, on Version B, they are shown in two columns. Do you have a preference for either version?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
C. Location of Filter & Number of Options
Another thing we varied between the two versions is this feature that allows you to sort occupations by different criteria like pay, required education, and the number of projected new jobs. That feature appears here on Version A, and here on Version B. Which version do you prefer?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
Looking at the sorting feature again, Version A offers five ways to sort occupations, and Version B offers three ways. Which version do you prefer?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
When we were thinking about this sorting feature, we came up with five characteristics of occupations that we thought would be of most interest. These are: education level, the projected number of new jobs, pay in 2010, the training required, and the anticipated growth rate.
Please rank order these from 1 to 5, where 1 is the most important characteristic to you.
Rank 1 ___
Rank 2 ___
Rank 3 ___
Rank 4 ___
Rank 5 ___
D. Search Boxes
We also varied the number of search boxes between the two versions. Version A has one search box up located up here, and Version B has two search boxes: one in the same location, and one farther down the page (point them out). Which version do you prefer?
Version A – Is there a reason you prefer this version?
Version B – Is there a reason you prefer this version?
No preference
E. Fastest Growing Jobs (projected) & Most New Jobs (projected)
We also added some links that we thought would be of general interest to people visiting this website. For example, you can click on the highest paying jobs, the fastest growing jobs, the jobs that most visitors to this site are looking at (most viewed), and most new jobs.
a. Do you think your students (clients) will know the difference between fastest growing jobs (projected) and most new jobs (projected)?
b. Do you have a preference for the order in which these links are shown?
Yes – if yes, what is your preferred order?
No
We debated between using percentage change versus adjective descriptors for describing the growth rates for occupations. Which do you prefer and why?
Percentage changes
Adjectives (average, faster than average, etc.)
Other - please explain
Finally, as a counselor, how useful do you think this link will be to your students (clients)?
Very useful
Somewhat useful
A little useful
Not at all useful
F. Filter Occupations - “Click go” instruction
Version A has this Click go when done instruction following the heading “Filter Occupations By” (point this out). Version B does not have this instruction.
a. Did you notice this instruction?
Yes
No
Not sure
b. Did you think it’s helpful?
Yes
No
Not sure
c. Do you have a preference for the order in which the characteristics that you can sort by are
shown?
Yes – if yes, what is your preferred order?
No
H. Landing Page comparisons (hand paper copies to participant)
In some career fields, there are related types of occupations. For example, today you saw occupations that fell under Healthcare Practitioners and Healthcare Support. We used three different formats for presenting this information. I want you to look over these versions and then order them from 1 to 3, where 1 is the version you like most, 2 is the next preferred, and 3 is the one you like the least.
Rank 1 _____
Rank 2 _____
Rank 3 _____
a. What do you like about the version you ranked first?
b. What do you dislike about the version you ranked last?
This version uses the largest photos. One drawback to this approach is that the occupations cannot be sorted, so that occupations that start with A, B, C, etc., will always be at the top of the page. Do you see any problems with this?
Yes – if yes, please explain
No
This version presents some general job summary information about each occupation. How useful is this information compared to other items that could be shown here?
Very useful
Somewhat useful
A little useful
Not at all useful
a. What other items do you think could be shown here?
I. OOH Occupation Finder (hand a paper copy to participant)
We developed this table to help you search quickly for occupations that met certain criteria, for example, jobs that pay more than a certain amount of money, jobs that are expected to experience a lot of growth, and jobs that require a certain level of education.
Do you remember this page?
Yes
No
Not sure
Did you read these instructions (Filter & Sort)?
Yes
No
Not sure
How clear would you say these instructions are?
Very clear
Clear
Neither clear nor unclear
Unclear
Very unclear
Do you have a preference for the order in which the characteristics that you can search by are shown?
Yes – if yes, what is your preferred order?
No
How easy or difficult was this table to use?
Very easy
Easy
Neither easy nor hard
Difficult
Very difficult
Did not use
If your students encounter this table on the website, how likely or unlikely is it that you think they will use it?
Very likely
Likely
Neither likely nor unlikely
Unlikely
Very unlikely
a. Do you have any comments about the table that would help us improve it or make it easier to use?
ATTACHMENT 1B - Protocol for Remote Usability Testing
Protocol for Remote Usability Testing
Introduction for when WebEx is used (see Attachment for separate Loop11 protocol)
Thank you for agreeing to participate today.
Ask if person has participated in previous studies in BLS.
Introduce any observers.
Note: The facilitator and participant will share the computer screen, but not see each other. Communication will be via telephone.
Background. Before we begin, let me give you some background for what you will be doing today. We are in the process of redesigning our website that provides detailed career information about hundreds of occupations. It’s called the Occupational Outlook Handbook. Have you heard about it by any chance? Have you ever visited the website for the Handbook?
As I mentioned, we’re in the process of redesigning this website but we’re not sure what the best design is for the homepage, so today I’m going to ask you to search for some information about different occupations using different homepage designs we are considering. After you have finished the tasks, I’ll ask you to rate how easy or difficult it was to find information, and then at the end of this session we’ll discuss the different versions, and you can tell me which design you prefer, and why.
Reminder. An important point to keep in mind is that we are not evaluating you today. We are evaluating different designs for the homepage, so if you have problems with a task or one of the designs, that’s actually a good thing because then we know we have to make changes. Another thing I want you to do is relax. This isn’t a test. We’re don’t care how much of a computer whiz you are. Try to make believe you are working on your home computer with no one observing you.
Procedures & Consent Form. As part of what you’ll be doing today, we have software on the computer that captures your keystrokes and tells us where you clicked on the screen. This software does a lot of work for us, so we don’t have to worry about missing something. Although we collect this information, we will not share it with anyone. We use it only for statistical purposes, and we don’t link your name with it.
Reminder. E-mail or phone call should have been received acknowledging voluntary participation.
General Instructions – I am going to ask you to look for information today about different occupations. In each case, your search will start on one of two versions of the homepage that we have developed.
For some of the tasks, all you have to do is go to a web page, and you’ll be done, but other tasks will ask you to find some information on the page and then answer a question. Since this isn’t a test, we’re interested in seeing how easy it is for you to find the answer to a question, so it’s not important if you get the right answer or not. In fact, if a lot of people come up with the wrong answer, that’s very useful information for us because it tells us that our current approach is not the best one, and that the information is too difficult to find.
I also want to point out that you will not be working with the actual Occupational Outlook Handbook today. Instead, we have set up a simplified version of the web site, and some of the actual pages you will be seeing are from our old Handbook before it was redesigned for the Web. You will be looking at some pages that have a new design. I don’t think you will have any problem figuring out which ones these are.
Talk aloud. As you complete the form, I would like you to talk out loud to explain what you are doing and why you are doing it. Some people feel self conscious doing this, but it really helps us determine what you are doing and why. We have found it helps to give people a little practice doing this to warm up, so before we begin, let’s practice a bit. What I want you to do is sit back now and describe what you see when you unlock the door to your apartment or home. Just describe the first room you see after you open the door, including tables, chairs, pictures on the wall, and so on. (Allow description) Now, instead of describing a room, I want to describe your thoughts and reactions as you complete our online form.
Start. Start test.
Note. Order of prototypes will vary.
Explain how UTE will present tasks, and work through practice tasks.
Tasks
The following tasks will be used in the usability test. The order of the homepages (1 or 2) will be changed for each participant. For example, Participant 1 will work with Version A first, Participant 2 will work with Version B first, but each participant will work with both versions. P1 and P2 are practice tasks.
Order 1
|
Homepage Version |
Question |
Answer |
P1 |
A |
Assume you are a math major in college. Since you like math, you are interested in finding out which occupations require math skills, even though you don’t want to be a pure mathematician. Where would you start looking? |
Click Math under Groups of Occupations tab, and then Mathematicians on landing page |
P2 |
A |
Assume you are good with your hands. You also like machines and tools, and are interested in being some type of mechanic. Where would you look to find out what occupations mechanics could work in? |
|
1 |
A |
Your cousin has told you he is thinking about becoming a physical therapist, and was wondering what he could expect to earn, on average (median) annually. What is the answer to his question? |
$72,790 annually |
2 |
A |
A friend asked you if you knew what occupation was growing the fastest. What would you tell her? |
Biomedical engineers |
3 |
A
|
A family friend thinks she would like to work in healthcare, and asks you if you could help her find out what general types of occupations are in that field. Go to the pages that provide this information. |
Under “Groups of Occupations,” click Healthcare Practitioners and Healthcare support (use Detailed Landing Page) |
4 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, find out what the annual pay of audiologists is (figure is for 2008). |
$44,342 (use Photo Grid Landing Page) |
5 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the most growth in the number of jobs between 2008 and 2018? |
Speech-language pathologists (use Growth Landing Page) |
1 |
B |
A member of your family is changing jobs and said he is thinking about becoming an athletic trainer, and wonders what he could expect to earn each year. What is the answer to his question? |
$39,640 |
2 |
B |
A friend asked you if you knew which occupation had the most new job openings. What would you tell her? |
Registered nurses |
3 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, what is the projected growth rate in jobs for audiologists between 2008 and 2018? |
Minus 28% (use Photo Grid Landing Page) |
4 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the biggest decline in the number of jobs between 2008 and 2018? |
Dentists (-56%) (use Growth Landing Page) |
5 |
B |
A family member is leaving the military and wonders what jobs might be open to him that require only a high school education, but still pay at least $50,000 a year. What would you tell him? |
Railroad conductors & yardmasters, rotary drill operators, subway/streetcar operators |
Order 2
|
Homepage Version |
Question |
Answer |
P1 |
B |
Assume you are a math major in college. Since you like math, you are interested in finding out which occupations require math skills, even though you don’t want to be a pure mathematician. Where would you start looking? |
Click Math under Groups of Occupations tab. |
P2 |
B |
Assume you are good with your hands. You also like machines and tools, and are interested in being some type of mechanic. Where would you look to find out what occupations mechanics could work in? |
|
1 |
B |
A member of your family is changing jobs and said he is thinking about becoming an athletic trainer, and wonders what he could expect to earn each year. What is the answer to his question? |
$39,640 |
2 |
B |
A friend asked you if you knew which occupation had the most new job openings. What would you tell her? |
Registered nurses |
4 |
B
|
A family friend thinks she would like to work in healthcare, and asks you if you could help her find out what general types of occupations are in that field. Go to the pages that provide this information. |
Under “Groups of Occupations,” click Healthcare Practitioners and Healthcare support (use Detailed Landing Page) |
5 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, what is the projected growth rate in jobs for audiologists between 2008 and 2018? |
Minus 28% (use Photo Grid Landing Page) |
6 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the biggest decline in the number of jobs between 2008 and 2018? |
Dentists (-56%) (use Growth Landing Page) |
1 |
A |
Your cousin has told you he is thinking about becoming a physical therapist, and was wondering what he could expect to earn, on average (median) annually. What is the answer to his question? |
$72,790 annually |
2 |
A |
A friend asked you if you knew what occupation was growing the fastest. What would you tell her? |
Biomedical engineers |
5 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, find out what the annual pay of audiologists is (figure is for 2008). |
$44,342 (use Photo Grid Landing Page) |
6 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the most growth in the number of jobs between 2008 and 2018? |
Speech-language pathologists (use Growth Landing Page) |
3 |
A |
A family member wants to change jobs and wonders what jobs might be open to him that only require only a high school education, pay at least $25,000 a year, and have a growth rate between 8 and 15 percent over the next 10 years. What would you tell him? |
Bus drivers |
Homepage comparison
We used two different versions of the homepage today. Let me show them to you again (Versions will be shown on computer). Which version do you prefer?
Version A
Version B
No preference
Why do you prefer that version?
Landing Page comparisons
In some career fields, there are related types of occupations. For example, today you saw occupations that fell under Healthcare Practitioners and Healthcare Support. We used three different formats for presenting this information. Let me show them to you again (Versions will be shown on computer). As I do this, please determine which you liked best, next best, and least of all.
Rank 1 _____
Rank 2 _____
Rank 3 _____
What do you like about the version you ranked first?
What do you dislike about the version you ranked last?
OOH Occupation Finder
We developed this special table to help you search quickly for occupations that met certain criteria (table will be shown on computer), for example, jobs that pay more than a certain amount of money, jobs that are expected to experience a lot of growth, and jobs that require a certain level of education.
How easy or difficult was this table to use?
Very easy
Easy
Neither easy nor hard
Difficult
Very difficult
Did not use
If you had encountered this table on the website, how likely or unlikely is it that you would have used it?
Very likely
Likely
Neither likely nor unlikely
Unlikely
Very unlikely
Do you have any comments about the table that would help us improve it or make it easier to use?
Background Questions (These will be collected on the computer after the test)
Before we begin, there are a few questions about yourself that I would like you answer.
How would you rate your level of experience using the Internet?
I am fairly new to it
I have some experience, but still much to learn
I have moderate (about average) experience
I have much (considerable) experience
I use the Internet a lot and understand its intricacies
What is your age?
18-21
22-25
26-30
31-40
41-50
51-60
61 or older
What is the highest level of education you have completed?
Some high school
Some high school, but with technical training
High school graduate or GED
Some college, but no degree
Technical training past high school
Associate college degree (2-year program, for example, A.A. or A.S.)
College degree (for example, B.A. or B.S.)
Some post-graduate education, but no degree
Post-graduate degree (for example, M.A., M.S., Ph.D., or professional
degree)
Attachment – Loop11 Protocol (Loop11 allows a usability test to be conducted without any direct human facilitation)
Introduction. Thank you for participating in this evaluation. Your feedback will help improve the functionality of the Occupational Outlook Handbook website. We estimate that this exercise will last about 20 minutes. Please read all instructions carefully.
Before you begin, please note that you will not be working with the actual Occupational Outlook Handbook today. Instead, a simplified version of the web site has been set up, and some of the pages you will be seeing are from our old Handbook before it was redesigned for the Web. You will also be looking at some pages that have a new design. I don’t think you will have any problem figuring out which ones these are.
NOTE: This survey is being administered by Loop11.com and resides on a server outside of the BLS Domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response.
Loop11 Instructions.
Start. Start test.
Note. Order of prototypes will vary.
Explain how Loop11 will present tasks
We would now like you to complete some tasks and then answer several questions relating to your experience with each task. In each task you will be asked a question, such as "What is the median salary for a plumber?" In this example, your task would be to use the tools on the website to navigate to a webpage that has this information on it. We are interested in whether you can find the information, how you get there, and how long it takes. Once you believe you have found the answer click the Complete Task button.
Approach using our site as if you were at home, school, etc. and if, after expending a certain amount of effort, you believe you would “bail out” and go elsewhere to find the information, then that is what you should do. This isn’t a test of your ability, it’s a test of the website, so if you do abandon the task, that’s useful information for us because we know we have to make improvements.
*** NOTE: Please be patient and wait for each webpage to load. Because some of the pages are still in development, they may take a few seconds to load properly. Do not click the Task Complete or Abandon Task button until the webpage has loaded and you have completed or attempted the task.
When you are ready to begin, click OK and then click Next.
Tasks. The following tasks will be used in the usability test. The order of the homepages (A or B) will be changed for each participant. For example, Participant 1 will work with Version A first, Participant 2 will work with Version B first, but each participant will work with both versions.
Note: Since Loop11 runs without a facilitator, to avoid having the session run too long (20 minutes is probably the maximum), participants may be asked to complete the 8 tasks (2 practice, 6 regular tasks) for only one version. However, a couple of tasks will also be presented with the alternative version.
Order 1
|
Homepage Version |
Question |
Answer |
P1 |
A |
Assume you are a math major in college. Since you like math, you are interested in finding out which occupations require math skills, even though you don’t want to be a pure mathematician. Where would you start looking? |
Click Math under Groups of Occupations tab, and then Mathematicians on landing page |
P2 |
A |
Assume you are good with your hands. You also like machines and tools, and are interested in being some type of mechanic. Where would you look to find out what occupations mechanics could work in? |
|
1 |
A |
Your cousin has told you he is thinking about becoming a physical therapist, and was wondering what he could expect to earn, on average (median) annually. What is the answer to his question? |
$72,790 annually |
2 |
A |
A friend asked you if you knew what occupation was growing the fastest. What would you tell her? |
Biomedical engineers |
3 |
A
|
A family friend thinks she would like to work in healthcare, and asks you if you could help her find out what general types of occupations are in that field. Go to the pages that provide this information. |
Under “Groups of Occupations,” click Healthcare Practitioners and Healthcare support (use Detailed Landing Page) |
4 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, find out what the annual pay of audiologists is (figure is for 2008). |
$44,342 (use Photo Grid Landing Page) |
5 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the most growth in the number of jobs between 2008 and 2018? |
Speech-language pathologists (use Growth Landing Page) |
1 |
B |
A member of your family is changing jobs and said he is thinking about becoming an athletic trainer, and wonders what he could expect to earn each year. What is the answer to his question? |
$39,640 |
2 |
B |
A friend asked you if you knew which occupation had the most new job openings. What would you tell her? |
Registered nurses |
3 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, what is the projected growth rate in jobs for audiologists between 2008 and 2018? |
Minus 28% (use Photo Grid Landing Page) |
4 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the biggest decline in the number of jobs between 2008 and 2018? |
Dentists (-56%) (use Growth Landing Page) |
5 |
B |
A family member is leaving the military and wonders what jobs might be open to him that require only a high school education, but still pay at least $50,000 a year. What would you tell him? |
Railroad conductors & yardmasters, rotary drill operators, subway/streetcar operators |
Order 2
|
Homepage Version |
Question |
Answer |
P1 |
B |
Assume you are a math major in college. Since you like math, you are interested in finding out which occupations require math skills, even though you don’t want to be a pure mathematician. Where would you start looking? |
Click Math under Groups of Occupations tab. |
P2 |
B |
Assume you are good with your hands. You also like machines and tools, and are interested in being some type of mechanic. Where would you look to find out what occupations mechanics could work in? |
|
1 |
B |
A member of your family is changing jobs and said he is thinking about becoming an athletic trainer, and wonders what he could expect to earn each year. What is the answer to his question? |
$39,640 |
2 |
B |
A friend asked you if you knew which occupation had the most new job openings. What would you tell her? |
Registered nurses |
4 |
B
|
A family friend thinks she would like to work in healthcare, and asks you if you could help her find out what general types of occupations are in that field. Go to the pages that provide this information. |
Under “Groups of Occupations,” click Healthcare Practitioners and Healthcare support (use Detailed Landing Page) |
5 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, what is the projected growth rate in jobs for audiologists between 2008 and 2018? |
Minus 28% (use Photo Grid Landing Page) |
6 |
B |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the biggest decline in the number of jobs between 2008 and 2018? |
Dentists (-56%) (use Growth Landing Page) |
1 |
A |
Your cousin has told you he is thinking about becoming a physical therapist, and was wondering what he could expect to earn, on average (median) annually. What is the answer to his question? |
$72,790 annually |
2 |
A |
A friend asked you if you knew what occupation was growing the fastest. What would you tell her? |
Biomedical engineers |
5 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, find out what the annual pay of audiologists is (figure is for 2008). |
$44,342 (use Photo Grid Landing Page) |
6 |
A |
Starting at the link for Healthcare Practitioners and Healthcare Support, which occupation is expected to see the most growth in the number of jobs between 2008 and 2018? |
Speech-language pathologists (use Growth Landing Page) |
3 |
A |
A family member wants to change jobs and wonders what jobs might be open to him that only require only a high school education, pay at least $25,000 a year, and have a growth rate between 8 and 15 percent over the next 10 years. What would you tell him? |
Bus drivers |
Homepage Preference. We would like you to take a minute or two to review each of the homepages we showed you earlier. Browse long enough to develop an impression of each page. When you are ready to begin, click OK and then click Next. (Note: We will include links to the two homepages).
Review Homepage Version A
Review Homepage Version B
Which version do you prefer?
Version A
Version B
No preference
Why do you prefer that version?
Landing Page comparisons
In some career fields, there are related types of occupations. For example, today you saw occupations that fell under Healthcare Practitioners and Healthcare Support. We used three different formats for presenting this information. Please look at them again and determine which you liked best, which you liked the next best, and then which you liked least of all.
Review Version A
Review Version B
Review Version C
Which version do you most prefer?
Version A, with list of occupations, columns with bar chart/histogram and median wages
Version B, with small pictures in first (left) column
Version C, with larger pictures, 4 across each row
No preference
What do you like about the version you ranked first?
What do you dislike about the version you ranked last?
OOH Occupation Finder
We developed a special table to help you search quickly for occupations that met certain criteria, for example, jobs that pay more than a certain amount of money, jobs that are expected to experience a lot of growth, and jobs that require a certain level of education. Please look at this table again now.
Review Advanced Search Table
How easy or difficult was this table to use?
Very easy
Easy
Neither easy nor hard
Difficult
Very difficult
Did not use
If you had encountered this table on the website, how likely or unlikely is it that you would have used it?
Very likely
Likely
Neither likely nor unlikely
Unlikely
Very unlikely
Do you have any comments about the table that would help us improve it or make it easier to use?
ATTACHMENT 2: Parent/Guardian Consent Form
Parental Consent Form
The Bureau of Labor Statistics (BLS) is conducting research to improve the quality of its web-based materials. This specific study is intended to determine the effectiveness of a redesigned homepage that allows multiple ways to search for career information in the Occupational Outlook Handbook.
During this research your child will be observed by a BLS researcher, comments will be audiotaped, and a computer program may be used to record entries and timing data for later analysis. Please note that this is a not a test of your child’s abilities. Instead, we are interested in how well the website is designed and how easy it is to use. Any difficulty your child might encounter with certain tasks will provide us with information we can use to redesign the website to make it easier to use.
Participation in this research project is voluntary, and your child has the right to stop at any time. We estimate the session will last an average of 45 minutes. As a thank you for participating, we will give your child a check for $25. If you would like to talk with a researcher before giving your permission, please see the contact information below. If you agree to allow your child to participate, please sign below.
The BLS, its employees, agents, and partner statistical agencies, will use the information your child provides for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your child’s responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.
Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. The OMB control number is 1220-0141 and expires February 29th, 2012.
-------------------------------------------------------------------------------------------------------------------------------
I have read and understand the statements above, and I give my consent for my child to participate in this study.
___________________________________ ___________________________
Parent’s or Guardian’s signature Date
___________________________________ Contact Information for BLS Researchers:
Parent’s or Guardian’s printed name Dr. Bill Mockovak, 202-691-7414
OMB Control Number: 1220-0141
Expiration Date: 2/29/2012
In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
ATTACHMENT 3: Participant Consent Form
Consent Form
The Bureau of Labor Statistics (BLS) is conducting research to improve the quality of its web-based materials. This specific study is intended to determine the effectiveness of a redesigned homepage that allows multiple ways to search for career information in the Occupational Outlook Handbook.
During this research you will be observed by a BLS researcher, comments will be audiotaped, and a program may be used on the computer to record entries and timing data for later analysis.
We estimate it will take you an average of 45 minutes to participate in this research. Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.
Please note that this is a not a test of your abilities. Instead, we are interested in how well the website is designed and how easy it is to use. Any difficulty you might encounter with certain tasks will provide us with information we can use to redesign the website to make it easier to use.
The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.
Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. The OMB control number is 1220-0141 and expires February 29th, 2012.
------------------------------------------------------------------------------------------------------------
I have read and understand the statements above. I consent to participate in this study.
___________________________________ ___________________________
Participant's signature Date
___________________________________
Participant's printed name
___________________________________
Researcher's signature
OMB Control Number: 1220-0141
Expiration Date: 2/29/2012
In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
1 “Landing Pages” display information for a set of related occupations, for example, different types of teachers.
| File Type | application/msword |
| File Title | October 3, 2007 |
| Author | BLS User |
| Last Modified By | mockovak_w |
| File Modified | 2011-05-23 |
| File Created | 2011-05-23 |