Phase 6 Round 3 Testing Protocol

Cognitive Interviewing Protocol

Small Business Pulse Survey – Phase 6, Round 3

6.17.21 draft

Interviewer notes are in red.

Research Questions to Address:

• Are respondents able to answer the questions as intended?

• Are there any items that cause respondents problems?

  • What can be done to fix those problems?

• Estimate of average time to complete?

Introduction:

• Hello. Thank you for your time today. We are looking to obtain feedback on a revised version of a short survey that has been designed to learn more about the effects of the Coronavirus pandemic on small businesses.

• Our purpose for speaking with you today is to see how respondents answer the survey questions and to evaluate how the questions work or don’t work.

• We are not testing you– we only want to evaluate the questionnaire.

• Today, we will have you complete the short questionnaire, then we will review some of the questions with you to get your feedback.

• Email link to Business Pulse survey to respondent.

• I just sent you an email. Can you please click on the link to the Qualtrics survey and read over the first page? This study is being conducted under the authority of Title 13 USC. We plan to use your feedback to improve the survey questions and make sure they make sense to respondents like you. Do you agree to participate? Thank you. Please click on the checkbox at the bottom of the screen, but do not advance the screen just yet.

Before the Questionnaire:

• Before we begin, could you tell me a little bit about your business -- what types of goods or services does this business provide?

• Thank you. And what is your role in the business? (What kind of responsibilities do you have?)

• Since they were sampled before, should we ask if they ever responded to the survey in the past? Ask if they remember getting a request for this?

Have Respondent Complete the Questionnaire:

Now I’d like for you to go through the survey question by question. Please tell me your answer to each question as you go along. After you are done, we will go back and discuss some of the questions in more detail.

Document responses to each question and the time respondent took to complete survey.

___ minutes

Review the Questionnaire:

Thank you for completing the survey. On your screen should be a summary of your responses. You can use this as we talk a little bit more about some of the questions you have just answered.

First, what is your overall impression of this survey?

Item 16 (Export revenues)

Now let’s go back and look at Item 16.

Q16. In the next 6 months, do you think this business will have export revenues (revenues from the sale of goods or services to foreign customers or clients)?

Select only one:

• Yes

• No

• Not applicable, this business does not usually have export revenues

• Please tell me in your own words what this question is asking.

• How did you come up with your answers to this question?

• What did you think about when answering this question?

• What does “export revenues” mean to you?

• Do you find the definition in parentheses helpful or not helpful?

• What would you include when answering this question? Exclude?

• What time period were you thinking about when answering this question? What does “In the next 6 months” mean to you?

• How confident are you in your answer? (If not confident, why?)

• How easy or difficult was this question to answer? (If difficult, why?)

• Do you have any suggestions on how to improve this question?

Now I would like you to think about a slightly different version of this question.

Alt 16. In the next 6 months, do you think this business will have revenues from exports (selling products or services internationally)?

Select only one:

• Yes

• No

• Not applicable, this business does not usually have revenues from exports

• What is your reaction to this version of the question?

• In your own words, what is the difference between the two versions?

• Export revenues v. Revenues from exports?

• The sale of v. Selling?

• Goods or services v Products or services?

• To foreign customers or clients v. Internationally?

• Do you prefer this version, or the original version? Why?

Item 17 (Six months, business needs)

Next, let’s look at Item 17.

Q17. In the next 6 months, do you think this business will need to do any of the following?

Select all that apply:

• Obtain financial assistance or additional capital

• Identify new supply chain options

• Develop online sales or websites

• Increase marketing or sales

• Identify and hire new employees

• Identify potential markets for exporting goods or services

• Permanently close this business

• None of the above

• Please tell me in your own words what this question is asking.

• How did you come up with your answers to this question?

• What did you think about when answering this question?

• What does “Identify potential markets for exporting goods or services” mean to you? Is there another way to word that?

• If you were looking to begin exporting or to increase exporting, what might you want to do/learn about in the next six months?

• Do you have any suggestions on how to improve this question?

Other Items

Are there any other questions you would like to go back and talk about?

(Review other items that raised concerns during survey completion.)

Wrap Up:

We are almost done. I just have a few more questions.

• Overall, how easy or difficult was it to answer these questions?

• Do you have any other comments or suggestions about anything we have discussed today?

That is all the questions I have for you today. Thank you very much for your time today. We really appreciate your help.

8