Form Attachments Cognitive Testing Protocol and Form

Attachment A: Cognitive Interview Protocol

1. In-person Interviews

• Thank you for participating today.

• I am …... [This is my colleague ( ) who will be taking notes for us today]

• Explain cognitive testing

  • The goal of our session is for us to evaluate a new form we are developing for PPI. We want to see how you complete the form. We are looking for what works well and what is more difficult.

  • During this session, we will give you several simulated forms to complete, along with scenarios that provide the information you will need to complete the forms. The items/services in the forms will be similar to what you already price for us. However, the scenarios may or may not match actual situations in your business.

  • Please use the information in the scenarios to complete the forms. If a scenario is significantly different from what you would normally experience, please let us know.

  • We are testing the form, not you, so any problems you have are problems with the form that we need to fix. We would like to hear all your comments and suggestions so we can improve the form as much as possible.

  • The information you provide today will only be used for the purposes of improving the form. It will not be used for any other purpose, and it is not part of the regular PPI data collection.

• Administer consent form.

• Have the participant complete the scenarios.

• Conduct the debriefing (see Attachment D).

• Thank the participant.

2. Remote Participants

Instructions during initial call

• Thank you for agreeing to participate.

• Explain cognitive testing

  • The goal of our session is for us to evaluate a new form we are developing for PPI. We want to see how you complete the form. We are looking for what works well and what is more difficult.

  • We will send you several simulated forms to complete, along with scenarios that provide the information to complete the forms. The items/services in the forms will be similar to what you already price for us. However, the scenarios may or may not match actual situations in your business.

  • Please use the information in the scenarios to complete the forms. If a scenario is significantly different from what you would normally experience, please let us know.

  • We will also send a consent form, which states that participation is voluntary, and that you may stop at any time.

  • As you are entering the data, please note how long it takes to complete each form.

  • We would like you to fax back the completed PPI forms along with the consent form.

  • We will then arrange a call with you, so you we can get feedback on your experience with the new form.

• Set up time to call back

Introduction during the test session

• Thank you for participating today.

• I am… [My colleague(s) <names> will also be listening today]

• Explain cognitive testing

  • We have a series of questions to ask you about the forms and your experience.

  • We are testing the form, not you, so any problems you have are problems with the form that we need to fix.

  • As a reminder, your participation is completely voluntary and you can decline to answer any question at any time.

  • The information you provide today will only be used for the purposes of improving the form. It will not be used for any other purpose, and it is not part of the regular PPI data collection.

  • We would like to hear all your comments and suggestions so we can improve the form as much as possible.

• Conduct the debriefing (see Attachment D).

• Thank the participant.

Attachment B: Sample Scenarios

Note: These samples were taken from a cognitive test for the PPI form we conducted using BLS employees. For these tests, we created scenarios to cover a wide range of conditions. In the test with actual respondents, we plan to tailor the scenarios to the types of items or services they report.

Scenario A

Your company produces grain combines. Sales of model number A123 have been low in the past year and your company discontinued producing this combine on March 1, 2012. The replacement grain combine, model B 1234, has an additional two rows and the cab has a premium windshield wiper and washer system. The price for model B1234 is $427,000. The mark up of both combines is the same. Model A123 cost you $150,000 to manufacture, while model B1234 costs you $200,000.

Scenario B

Your company produces a variety of lighting products, including flashlights. Over the last month your company has modified the flashlight’s housing, so it is now 4 ½ inches in diameter, and 7 inches long. The new model number is AB12, and the price of the new item is $53.

Scenario C

You are a large broker-dealer who specializes in making a marker in investment grade corporate bonds. You report to BLS the bid-ask spread for several corporate bonds. Several times throughout the year, you replace the bond that you report with a new bond from a similar company.

Using the information below, please fill out the form for BLS:

New Bond:

Marathon Oil Corp

Coupon Rate: 7.5%

Maturity: 2/15/2019

Credit rating: Ba

Ask: 118.19

Bid: 117.41

Attachment C: Sample Form

Attachment D: Debriefing Items

Complete questions 1 – 4 for each item or service.

For the in-person participants, we will ask these four questions after they complete each form, then start with question 5 (Q5) after they have finished all the forms. The remote participants will start with Q5, then insert the item-specific questions after Q13.

Q What steps did you take to complete the form? (Probe: What information did you report on the form?

Q Did you have any difficulty completing the form? (Probe: Was anything particularly confusing or challenging?)

• Yes

• No (go to Q4)

• Don’t Know/Not Sure/No Opinion (go to Q4)

Q What was difficult?

Q (this question will only be used as needed. The test moderator will review the form to determine if there are any other questions to ask, such as how the participant decided on a specific entry).

Q I’d like to begin by asking about your initial reaction to the form. (Probes: When you first saw the form, what did you think about it?)

Q How easy or difficult was it to complete the form? Very Easy, Easy, Neither Easy nor Difficult, Difficult, Very Difficult.

• Very Easy

• Easy

• Neither Easy nor Difficult

• Difficult

• Very Difficult

• No Opinion/Don’t Know/Not Sure

Q Did you look at the cover sheet at all? (check one)

• Yes

• No (go to Q10)

• Don’t Know / Not Sure / No Opinion (go to Q10)

Q How helpful was the cover sheet? Not At All Helpful, Slightly Helpful, Somewhat Helpful, or Very Helpful.

• Not At All Helpful

• Slightly Helpful

• Somewhat Helpful

• Very Helpful

Q When did you look at the coversheet? (check all that apply)

• Before starting

• When I wasn’t sure what to do

• Other, please specify

• Don’t Know / Not Sure / No Opinion

Q What was your general reaction to the coversheet? (Probes: When you first saw the coversheet, what did you think about it?)

Q Was there anything about the cover sheet that was particularly confusing or difficult to understand?

Q Was there anything about the cover sheet that was particularly helpful?

Q Do you have any other comments on the cover sheet?

For remote participants, Q1-Q4 will be here, repeated for each form

Instructions for remote participants: “Next I’d like to look at the form itself. Let’s start/continue with <brief item description & item code>.”

Q [Additional question for remote participants] Approximately how long did it take you to complete this form?

Q If you were completing these forms with real information, how much additional time would it take for you to locate the information you need for a single form?

Q Overall, what didn’t you like about the form?

Q Overall, what did you like about the form?

Q (this question will only be used as needed. The test moderator will review the discussion so far to determine if there are any other questions to ask).

Q Do you have any other comments that would help us improve the form?