Note to Reviewer - SOII DJTR

8

September 19, 2011

NOTE TO THE REVIEWER OF:	OMB CLEARANCE #1220-0141 “Cognitive and Psychological Research”
FROM:	Jennifer Edgar Office of Survey Methods Research
SUBJECT:	Submission of Materials for the SOII Days with Job Transfer or Restriction IDCF Usability testing

Please accept the enclosed materials for approval under the OMB clearance package #1220-0141, “Cognitive and Psychological Research.” In accordance with our agreement with OMB, I am submitting a brief description of the research and the materials to be used in the research at least two weeks prior to the beginning of the study.

The website to be tested is currently under development, and is not expected to be finished until October. Given tight resources, both time and technical staff, we must complete usability testing quickly after the site is completed to allow for changes to be made prior to production deadlines. This requires that we have OMB approval and are ready to begin usability testing as soon as the site is completed. Therefore, we can only attach a draft protocol, and specific usability assessment tasks are not included because the prototypes to be tested do not yet exist.

Additionally, as with all usability testing, depending on issues identified during the testing process, changes to the site may be made and additional testing required. We have built this potential second round of testing into the burden hour estimate, but again cannot provide specific usability assessment tasks or a final protocol as the changes to be evaluated are currently unknown. Given the tight time frame resulting from production schedules, we are asking for a generic clearance to cover usability testing on the IDCF SOII site.

We anticipate this research will require 22 burden hours for all rounds of testing.

If there are any questions regarding this project, please contact Jennifer Edgar at (202) 691-7528.

1. Introduction and Purpose

The purpose of this study is to evaluate the usability of the revised SOII data collection instrument on the Internet Data Collection Facility (IDCF). Previously, respondents only had to report Days Away from Work in the SOII Survey. Starting in 2012, a small sample of respondents will also have to report the number of Days with Job Transfer or Restriction (DJTR) cases. To collect this information, several changes were made to the web instrument.

Additionally, some minor changes were made to the layout of the IDCF survey pages and the wording of the error messages. This study will evaluate those changes as well, looking to identify any confusion or problems participants encounter as they complete the web survey.

2. Participants

The participants will be a maximum of 20 SOII participants. Establishments will be selected based on their known DJTR status. Should a participant with a disability be selected, we will make necessary accommodations to allow them to participate.

3. Methodology

BLS will identify a sample of respondents who are reliable reporters and are known to have DJTR cases. Participants will be recruited by phone (Attachment 1) to participate in a remote testing session. Participants agreeing to participate will be sent instructions for the remote testing site and will set up a date and time to conduct the study.

Remote Testing

1. Remote testing will be conducted using an application, call Webex, developed by Cisco. Use of Webex has been approved by BLS security for this purpose.

2. Participants will be contacted in advance and emailed or mailed a consent form. A letter will also be sent explaining the purpose of the study and the participant’s role (see Attachment 2).

3. To avoid asking a participant to mail back the consent form, the BLS researcher will obtain either taped verbal consent or ask the participant to respond to an e-mail stating that he or she is participating voluntarily in the scheduled session.

After the remote testing session has been set up, participants will be asked to complete a series of tasks and answer some questions (Attachment 3). Quantitative measures will assess success rates on predefined tasks (e.g., logging into the system, entering required SOII information, navigating correctly, and submitting the data) and efficiency (e.g., How long does it take to complete tasks?). Participants will also be debriefed after testing sessions to obtain qualitative feedback about the prototypes. For example, qualitative measures will include ratings of ease of use, intuitiveness of the design, and open-ended feedback about layout, formatting, aesthetics, etc.

4. Burden Hours

We estimate that each survey interview will take a maximum of 60 minutes for an estimated maximum of 20 burden hours (60 x 20 participants). Each participant will also receive one recruitment phone call, which is estimated to take less than 5 minutes for a maximum total of 2 additional burden hours. Therefore, the total estimated burden hours for this study is 22 hours.

5. Data Confidentiality

Participants will be informed that all information received by them will be kept confidential at every step in the process. They will be sent a consent form (Attachment 4) prior to the study, and the following statement will be read to participants prior to their participation in the study.

“Your participation is voluntary, and you may skip any questions you do not want to answer. Your answers will be kept strictly confidential and will be used only for research purposes.”

The participant will be asked to give their verbal consent before starting the study.

6. Attachments

1. Attachment 1: Advance Call Script (Draft)

2. Attachment 2: Confirmation email/letter text

3. Attachment 3: Testing Outline (Draft)

4. Attachment 4:Consent form

Attachment 1: Advance Call Script

1. Hello. My name is ______________. I am calling from the Bureau of Labor Statistics. The Bureau is working to improve the Survey of Occupational Injuries and Illnesses. We have made some changes to the website and were hoping that you would be willing to help us evaluate them.

2. Our records show that you are the person that completes the Survey of Occupational Injuries and Illnesses. Are you the person who prepares this report?

If No	If Yes
Would you be able to tell me who completes the survey? I can contact them directly to see if they’d be willing to participate in this study.	Great! Would you be willing to help us with this study? The interview will only take about 30 minutes, or maybe a bit longer depending on your answers. Yes: Thanks! Let’s set up a date and time that you’d be able to conduct the interview. You’ll need access to your SOII records, a computer with high-speed internet access, and a telephone. No: Thanks for your time.

3. Get participant’s email address or mailing address to send advance letter.

4. Confirm date, time and telephone number with Participant.

5. Give participant the researcher’s telephone number.

6. Thank participant.

Attachment 2: Advance Letter

Dear ,

Thank you for agreeing to participate! As I explained on the phone, the Bureau of Labor Statistics (BLS) is looking to improve the Survey of Occupational Injuries and Illness’s web survey. Your participation in our study will help us evaluate the changes we’ve made so far, and identify any other changes that should be made to make the website as easy as possible for respondents to use.

On _____ at _____ I will be calling you for your interview. At that time, you’ll log into a website which will allow us to work together to complete some SOII tasks. I’ll be able to see what you’re doing on the website as you work. I will give you the information you need to log in at the time of our interview.

<<Insert remote testing site>>

In preparation for the study, please pull out your SOII form from last year. We’ll use that as we work. I am also attaching a consent form for your review prior to our interview time. I will ask for your verbal consent before we begin, you do not have to sign and return the form.

If you have any questions, or need to reschedule, please contact me at 202-691-7528 or [email protected].

Thank you in advance for your help with this study!

Jennifer Edgar, PhD

Office of Survey Methods Research

Bureau of Labor Statistics

Washington DC 20212

202-691-7528

[email protected].

Attachment 3: Testing Outline

• Background. Before we begin, let me give you some background for what you will be doing today. The Bureau of Labor Statistics is trying to improve the website that you use to enter SOII data. Our purpose today is to find out how easy or difficult the web survey is to complete and to identify ways to improve it. You can help us identify any problems, so please take your time completing it. I’m going to ask you to enter your SOII data from last year, and then complete some tasks on the SOII website. I’ll also ask you some questions to get your feedback.

• Think aloud instructions. As you complete the survey, I would like you to think out loud to explain what you are doing and why. What I’m going to ask you to do is tell me what you’re thinking as you complete the survey. It may feel strange at first, but it’s a valuable way for us to learn about your experience as you move through the website. If you forget, I may remind you to think aloud as you work during the survey.

• SOII DJTR. Now that you understand what the survey is about, the next thing I’m going to ask you to do is complete the survey on the Internet. The email I sent last week has the website you will need, you’ll need to go to that site now. provide site and login information

• I am going to ask you to complete several tasks on the survey site <<tasks to be entered based on development>>

• Debriefing questions. Now I have a few general questions to ask you about your experience completing the survey. Non-scripted questions will be added based on testing observations and additional post-testing questions will be added based on tasks.

1. What would you say was the most difficult step involved in completing the survey? If you personally didn’t have a problem, what do you think would be the most difficult step for others?

2. BLS is now asking for respondents to tell us about both Days Away from work (Column H) and Days with Job Transfer or Restriction (Column I). What is your reaction to this?

3. Did you have any trouble entering the Days with Job Transfer or Restriction information?

4. Were there any places where the instructions could be improved or were needed?

5. What are your general impressions of the online survey?

Attachment 4: Consent Form

Consent Form

The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.

We estimate it will take you an average of 45 minutes and not to exceed 1 hour to participate in this research.

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: [02/29/12]

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

8