Attachment IV: Interviewer Protocol/Test Overview
Background. Before we begin, let me give you some background about what you will be doing today. We are trying to improve a website that companies use to report work-related injuries and illnesses. Our purpose today is to find out how easy or difficult the website is to use and to identify ways of improving it. To do this, I’m going to ask you to act as if you were a respondent in the Survey of Occupational Injuries and Illnesses (SOII) for a fictional company, and your job is to report information about any injuries and illnesses that employees in this company have had over the past year. I’ll provide you with all the information you’ll need to complete the SOII survey for this fictional company, so you’ll just be taking that information and trying to use the website to enter it in. We’ll run through a few different tasks to examine different sections of the website, and I’ll also ask you some questions as we go along to get your feedback.
So, again, our purpose here today is to evaluate how people use this website. Our purpose is not to evaluate you. If the website is easy to use, that’s great. But if you have any difficulties using it, we assume that it is a problem with the way the site is designed, not with the user. We’ll take what we learn from this study to work on design changes that we hope will make it easier to use the website in the future. Okay? Any questions?
[If not] Alright, to get us started, I want you to assume that you are an employee of a fictional company called Ski Solutions, Inc. For the purposes of this study, imagine that Ski Solutions is a small, independently-owned company that operates three ski slopes in Alaska. I’d like you to imagine that you are the Safety Manager for Ski Solutions, and that your company has been selected to participate in the Survey of Occupational Injuries and Illnesses. You’ll be reporting for the company using a BLS website called the Internet Data Collection Facility or IDCF.
We’ll go through several tasks using the survey website, as I mentioned, and you’ll be using this packet of information to help you with each task. [Hand the packet to the participant.] On each page of the packet there is a brief description of the task and some information about the company that you’ll need to complete the task. We’ll go through the tasks one at a time – you’ll start by first reading the task description. Once you’ve reviewed the material, I’ll launch the SOII website so you can begin entering the company information, and when you’re done with each task we’ll have a few brief questions about your experience using the site.
Think aloud instructions. Finally, as you complete each task, I would like you to think out loud to explain what you are doing and why. What I’m going to ask you to do is tell me what you’re thinking as you try to fill out the information on the SOII website. It may feel strange at first, but it’s a valuable way for us to learn about your experience as you move through the website. If you forget, I may remind you to think aloud as you work during the survey.
Do you have any questions for me before we begin?
Attachment V: Tasks and Debriefing Protocol – Test Group
Task 1: Add Two Establishments
For the purposes of this task and those that follow, you will be reporting for the calendar year 2012.
You have received three mailings indicating the surveys that you need to report for. You have just logged in to the IDCF website using the account for establishment ID 02-88811107-0. In this task, you need to make sure all three surveys are entered in the system. Please use the survey forms to do that, and don’t forget to think out loud as you go through the task!
Now you want to report for establishment number 02-888111007-0, please show me how you would do that.
Post-task debriefing probes:
(Show Screen 1)
How difficult was it for you to add establishments?
Was there any language on the website that confused you?
Were there any features of the website layout that you found confusing or frustrating, or which you think could be improved?
How difficult was it for you to locate your establishment ID on your survey mailing?
Task 2: Enter Case Data (establishment with cases)
Now, for one of your establishments you need to enter additional data. You have been provided copies of the forms that employers are required to use to keep track of workplace injuries and illnesses. Please use the information that has been entered in these forms to complete Sections 2 and 3 of the survey. If you have any questions as you work please let me know, and remember to talk out loud as you work.
Debriefing probes:
(Show Screen 3)
How easy or difficult was it to indicate that you needed to enter information about a case?
On this screen, what did you expect the “Enter Case Data 1” button to do?
What about that “Continue” button, what did you think that would do?
Does the number of rows in the table mean anything to you?
How do you think the computer decided how many rows to show?
What if you had an additional case to add, what would you do?
Was there anything confusing about this page?
Task 3: Enter Case Data (establishment without cases)
Now, for another one of your establishments you don’t have any cases to enter data for.
In Section 2, enter 0 (zero) for “Total number of cases with days away from work” and 1 for “Total number of other recordable cases”.
Under “Injury and Illness Types” enter 1 in #1 to indicate that there was one injury. Press “Save & Continue” to proceed to Section 3.
When you get to Section 3 a ski lift supervisor comes by and tells you he remembers two cases that should have been recorded, but were mistakenly left off the forms. After making the corrections, he gives you the revised forms that should be used to respond to the survey. Use this new set of forms to make any appropriate changes for this establishment.
Debriefing probes:
(Show Screen 3)
On this screen, what did you expect the “Back” button to do?
What about that “Continue” button, what did you think that would do?
The screen originally said “In Section 2 you reported: 0 cases with days away from work” (show screen shot), what did that mean to you?
Was it clear that you did not need to submit a detailed case form for the Other Recordable Case?
Was there anything confusing about this page?
Comparison Probes
Again, we have an alternate version of this page, take a look and let me know what features you like better or worse in this version
(Show Screen A3)
On this screen, what would you expect the “Enter Case” button to do?
What about the “Continue” button, what did you think that would do?
What do you think the “Remove” button would do?
What if you had an additional case to add, what would you do?
Attachment V: Tasks and Debriefing Protocol – Control Group
Task 1: Add Two Establishments
For the purposes of this task and those that follow, you will be reporting for the calendar year 2012.
You have received three mailings indicating the surveys that you need to report for. You have just logged in to the IDCF website using the account for establishment ID 02-88811107-0. In this task, you need to make sure all three surveys are entered in the system. Please use the survey forms to do that, and don’t forget to think out loud as you go through the task!
Now you want to report for establishment number 02-888111007-0, please show me how you would do that.
Post-task debriefing probes:
(Show Screen A1)
What did you think when you first saw this screen?
How difficult was it for you to add establishments?
Was there any language on the website that confused you?
Were there any features of the website layout that you found confusing or frustrating, or which you think could be improved?
How difficult was it for you to locate your establishment ID on your survey mailing?
Task 2: Enter Case Data (with cases)
Now, for one of your establishments you need to enter additional data. You have been provided copies of the forms that employers are required to use to keep track of workplace injuries and illnesses. Please use the information that has been entered in these forms to complete Sections 2 and 3 of the survey. If you have any questions as you work please let me know, and remember to talk out loud as you work.
Debriefing probes:
(Show Screen A3 )
How easy or difficult was it to indicate that you needed to enter information about a case?
On this screen, what did you expect the “Enter Case” button to do?
What about that “Continue” button, what did you think that would do?
What if you had an additional case to add, what would you do?
Was there anything confusing about this page?
Task 3: Enter Case Data (without cases)
Now, for another one of your establishments you don’t have any cases to enter data for.
In Section 2, enter 0 (zero) for “Total number of cases with days away from work” and 1 for “Total number of other recordable cases”.
Under “Injury and Illness Types” enter 1 in #1 to indicate that there was one injury. Press “Save & Continue” to proceed to Section 3.
When you get to Section 3 a ski lift supervisor comes by and tells you he remembers two cases that should have been recorded, but were mistakenly left off the forms. After making the corrections, he gives you the revised forms that should be used to respond to the survey. Use this new set of forms to make any appropriate changes for this establishment.
Debriefing probes:
(Show Screen A4)
The screen said “In Section 2 you reported: 0 cases with days away from work” (show screen shot), what did that mean to you?
Was it clear that you did not need to submit a detailed case form for the Other Recordable Case?
Was there anything confusing about this page?
Comparison Probes
Again, we have an alternate version of these pages, take a look and let me know what features you like better or worse in this version
(Show Screens 3 and 4)
On this screen, what would you expect the “Enter Case 1” button to do?
What about that “Continue” button, what did you think that would do?
What do you think the “Remove” button would do?
Does the number of rows in the table mean anything to you?
How do you think the computer decided how many rows to show?
What if you had an additional case to add, what would you do?
Which set of pages do you think would be easier to use?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | kincaid_n |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |