Note to Reviewer - K to 12

April 24, 2014

NOTE TO THE REVIEWER OF:	OMB CLEARANCE 1220-0141
FROM:	Robin Kaplan Research Statistician Office of Survey Methods Research
SUBJECT:	Submission of Materials for the BLS K-12 Website Feedback Study

Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 In accordance with our agreement with OMB, we are submitting a brief description of the study.

The total estimated respondent burden hours for this study are estimated to be 33.3.

If there are any questions regarding this project, please contact Robin Kaplan at

202-691-7383 or Jean Fox at 202-691-7370.

1. Introduction and Purpose

The Bureau of Labor Statistics (BLS) published its K-12 website on September 30^th, 2013 (link: http://www.bls.gov/k12/). The website contains resources to teach students about BLS products through activities and games and provide information about careers. The website is divided into various sections. The first section, Games and Quizzes, contains activities to appeal to younger students and teach them about BLS facts, terminology, and different occupations. The second section, Student Resources, draws from the popular Occupational Outlook Handbook, and provides kid friendly content regarding career information, employment projections, and a glossary of BLS terms. The final section provides a history of BLS that contains a timeline of major events throughout history and biographies of BLS commissioners.

Since its launch, the BLS K-12 website has been very popular, receiving approximately 100,000 visits per month. However, no usability testing has been conducted to get students’ feedback about the overall appeal and usability of the website. This type of testing is crucial to determine what portions of the website need improvement, whether students are learning from the content, and what features of the website are most appealing to students.

To gain insight on the issues listed above, we seek to solicit feedback from students attending the annual Department of Labor (DOL) event “Take Your Daughters and Sons to Work Day” on Thursday, April 24, 2014. We have already obtained permission from the event coordinators to conduct usability sessions, which will be incorporated into the activities planned for children attending this event.

2. Research Design

We plan to get feedback from children attending “Take Your Daughters and Sons to Work Day” at the DOL. This event will have an estimated 80 children, ranging in age from elementary school students to high school students. The sessions are scheduled to last 25 minutes. We are scheduled for one session of elementary school students, two sessions of middle school students, and one session of high school students. See Appendix A for the complete study protocol.

The coordinators of “Take Your Daughters and Sons to Work Day” at the DOL have planned our session to take place in a large computer lab at DOL. We plan to use an informal type of usability testing, where we will give a brief demo of the website, allow the participants to try the site, then ask for feedback from the kids on their experience. We chose this method because as our first attempt at user testing for the site, it will allow us to get some general feedback and some ideas about where to focus our attention in the future. In addition, it will be well suited to involve the kids who are visiting DOL for “Take Your Daughters and Sons to Work Day.”

The event planners have scheduled four groups of about 20 students each to participate in our program. Parents will receive the consent forms as part of the packet they receive at the start of the day. They will sign a consent form to allow their children to participate in this research (see Appendix B for a copy of the consent form).

During the session with the children, we will introduce ourselves and BLS. We will then discuss the purpose of the session, that we are trying to improve the site and need their help to do that. We will talk about the purpose of the BLS K-12 website and give a brief demonstration of the five sections of the site (Games and Quizzes, Student Resources, Teacher’s Desk, History of BLS, and FAQs)

For the session with the middle and high school students, we’ll start off by asking the kids if they thought about what they wanted to do when they grew up (or “after they graduated high school,” for the older kids). We will ask them to go to the “Career Exploration” portion of the site, and look for information related to jobs they might be interested in. After spending a few minutes with that, we will ask them to go to the Games portion, and we will direct them to games that might be relevant for their age group. During this time, we will walk around the room and talk with the students about their experiences and what they liked and didn’t like. For the elementary school students, we’ll use the same approach, but focus only on the Games portion of the site.

At the end of the session, we will take about five minutes to talk with the group as a whole to get their input on the site. Given the size of the groups and the ages of the students, we expect this will be easier than conducting any kind of debriefing with individual students.

We plan to be flexible during the test sessions. We may not get through all the activities, or we may find that the participants are more interested in one aspect of the site than another. Therefore, we want to be able to accommodate the participants’ preferences and reactions during the test.

3. Aims of the Research

Children’s feedback of the BLS K-12 website is crucial to determine how effectively the site meets its mission to create an appealing, easy-to-use, informative website. The K12 development team will use this feedback to identify any problems children have using the website and improve on them. We can also use this feedback to develop new games and features of the website that will appeal to children and determine whether the current activities are engaging and promote learning.

4. Participants and Burden Hours

Our goal is to obtain feedback from up to 80 children, depending on how many children are in attendance and how many participate. Each session is expected to last 25 minutes, for a maximum total of 33.3 burden hours (80 participants X .41 hours).

5. Data Confidentiality

Recruiting of participants will be done through the DOL “Take Your Daughters and Sons to Work Day.” Parents/guardians will be informed that the study will be used to improve the design of the BLS K-12 website, and that all feedback provided will be used statistical purposes only and will hold the information in confidence to the full extent permitted by law in accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws.

Attachments

Appendix A – Usability Test Protocol

Appendix B – Consent Form

Appendix A: BLS K-12 Website Feedback Protocol

Elementary School Students Session

Group	Session Time and Location	Participants
3	10:00 – 10:25 am; room S-5215	16 Elementary School Students

Introduction [4 minutes]:

Hi everyone! Welcome to our session on the Bureau of Labor Statistics K-12 webpage. We’re excited to be here and tell you more about what we do at BLS. BLS provides information about different types of jobs and the economy. How many of you know what you want to be when you grow up, or want to know more about what your parents do? BLS has this type of information, so we are excited to show you the new BLS K-12 website.

[Brief demo:] The goal of the BLS K-12 website is to teach students more about career information, occupations, and information we publish about the economy. The BLS K-12 website is divided into 5 sections. The first is Games and Quizzes, which has games, such as memory and word searches, that teach you about BLS facts, terminology, and different occupations. The Student Resources section has information about careers, and the sections are divided into categories like science, law, and math. The Teacher’s Desk has classroom activities that teachers can use to teach students about careers and the economy, The History of BLS section has a timeline of major events that happened, like when BLS started. Finally, there is Frequently Asked Questions section that includes more information about what we do at BLS and who works there.

Our goal for today is to have you try out the BLS K-12 website, then get your feedback so we can improve the stie. We’ll be walking around the room to see what you like or dislike about the site, if you’re learning anything, and if you find them fun.

Games [16 minutes]:

Click on the “Games and Quizzes” section of the website. Select any game that you would like to try, and again we will walk around the room and ask some questions about what you think of the games.

Follow-up questions for individual students as we are walking around:

• What game are you playing/did you try?

• Was that game fun?

• What did you like or dislike about it?

• Did you learn anything?

• Point to the face that matches how you feel about the website:

Debriefing [5 minutes]:

Thank you for participating in our session today and trying the games on the BLS K-12 website. We hope you had fun and learned more about careers and what we do at BLS. Your feedback will help us improve the website. What did you like best about the site? What could we do to improve it? Thanks for all your help!

Middle School and High School Sessions

Group	Session Time	Participants
7	10:30 – 10:55 am; room S-5215	21 High School Students
4	11:00 – 11:25 am; room S-5215	21 Middle School Students
6	11:30 – 11:55 am; room S-5215	21 Middle School Students

Introduction [4 minutes]:

Hi everyone! Welcome to our session on the Bureau of Labor Statistics K-12 webpage. We’re excited to be here and tell you more about what we do at BLS. We’re part of the Department of Labor, and we provide information about different types of jobs and the economy. BLS has one of the most popular websites for people to research different types of careers. We also provide economic information, such as what things people like to buy, how much things cost, and information about the employment rate.

[Brief demo:] The purpose of the BLS K-12 website is to teach students more about career information, occupations, and information we publish about the economy. The BLS K-12 website is divided into 5 sections. The first is Games and Quizzes, which has games, such as memory and word searches, that teach you about BLS facts, terminology, and different occupations. The Student Resources section has information about careers, and the sections are divided into categories like science, law, and math. The Teacher’s Desk has classroom activities that teachers can use to teach students about careers and the economy, The History of BLS section has a timeline of major events that happened, like when BLS started. Finally, there is Frequently Asked Questions section that includes more information about what we do at BLS and who works there.

Our goal for today is to show you the BLS K-12 website, and then get your feedback on it. We’ll be walking around the room to see what you guys like or dislike about the website, what you’d change about it, if you’re learning anything, and if you find it fun.

Careers [8 minutes]:

How many of you have thought about what you’d like to do after graduating from high school? The first section we’d like get your feedback on is the “Student Resources” section. Click on that section of the website. Go ahead and select a career that you find interesting on the website. We will be walking around the room, and might ask you some questions about what you think of this part of the website.

Follow-up questions for individual students:

• What career are you interested in? Where did you click to find it?

• Are you learning anything about [career]?

• What do you think of this page, do you like or dislike it?

• Would you change anything?

Games [8 minutes]:

Okay, now we are going on to the Games section. Like we said, this section has different games designed to help you learn more about BLS. Click on the “Games and Quizzes” section of the website. Select any game that you would like to try, and again we will walk around the room and ask some questions about what you think of the games.

Follow-up questions for individual students:

• What game are you playing/did you try?

• Was that game fun? What did you like or dislike about it?

• Did you learn anything?

Debriefing [5 minutes]:

Thank you for participating in our session today and looking at the Student Resources and Games section of the BLS K-12 website. We hope you had fun and learned more about careers and what we do at BLS. Your feedback will help us improve the website. What did you like best about the site? What could we do to improve it? Thanks for all your help!

Appendix B – Consent Form

CONSENT FORM

Title of Study: BLS K-12 Website Feedback Study

Purpose: The Bureau of Labor Statistics seeks to get feedback on the content of its website: http://www.bls.gov/k12/

Description of Study: The Bureau of Labor Statistics recently launched its first K-12 website. Children’s feedback of the BLS K-12 website is crucial to determine how effectively the site meets its mission to create an appealing, easy-to-use, informative website. The K12 development team will use this feedback to identify any problems children have using the website and improve on them. We can also use this feedback to identify potential new games and features of the website that will appeal to children and determine whether the current activities are engaging and promote learning.

As part of the 2014 “Take Your Daughters and Sons to Work Day,” we would like to ask students for their feedback on different portions of the BLS K-12 website. We are looking for general impressions of the website and feedback on what things they would like to see on the website. We may also ask participants to look up some information, perform basic tasks on the website, and play some of the games.

The BLS, its employees and agents, will use the information you and your child provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

We estimate it will take an average of 25 minutes to participate in this research.

Participation in this research project is strictly voluntary, and your child has the right to stop at any time. If you agree to have your child participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141, and expires February 28, 2015.

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to allow my child to participate in this study.

___________________________________ April 24, 2014

Parent or Guardian’s signature Date

___________________________________

Parent or Guardian’s printed name

___________________________________

Child’s name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: February 28, 2015

Page 10 of 10