Protocol

Draft 10-30-15

2015 Annual Survey of Entrepreneurs

Cognitive Interview Protocol

Research Questions to Address:

• Are respondents able to answer the questions as intended?

• Are there any items that cause the respondents problems? What can be done to fix those problems?

• Are there any order/context effects?

• How much time do respondents estimate they will need to complete the survey?

• How does the web-only reporting option affect the response process? What substitute for a mailed form should be provided, if any?

Materials:

• Copies of questionnaire testing draft

• Recorder, batteries

• Consent form

Procedure:

• Note whether R responded to the 2014 ASE. (Some questions will require either past or future tense depending on whether R is previous ASE reporter.)

• Go through the Introduction and Before the Questionnaire questions.

• Then provide R with the draft questionnaire and ask R to fill it out. As R completes each question, ask the general probes and any item-specific follow-up probes.

• Note how R navigates the questionnaire and if they appear to be having any difficulty with the questions. If R appears to have any problems (Utterances like “Hmm”, confused looks, etc.), probe on this.

• Go through probes about Web-Only Reporting and Wrap-up Questions

Introduction:

• Explain purpose of meeting: to understand the process of answering questions, and to evaluate how new questions work or don’t work.

• Some questions will seem odd and/or obvious, but I don’t want to assume I know what you are thinking.

• Explain that we are not testing the respondent – we only want to evaluate the questionnaire

• Structure of meeting: Understand your business and your role in it, then review the questionnaire and ask you questions as you complete it.

• Permission to record discussion? Solely for note-taking purposes. Have R sign consent form.

Before the Questionnaire:

• What types of services does this business provide?

• What is your role in the company? What kind of responsibilities do you have?

• Have you completed other surveys from the Census Bureau or other agencies? If so, which ones?

• (If previous ASE respondent) What was your role in the process for responding to this questionnaire? (Gather data? Enter data? Consult with data providers? Etc.)

Questionnaire test

Now let’s take a look at the draft questions. I would like you to go through and answer the questions as if you were filling out an actual form. If you don’t have answers any particular questions, you can just make a best guess. I will ask you some questions as we go through them.

These questions are intended for [name of business as listed in recruiting file].

General Probes:

• What does this question mean to you?

• How would you go about answering this question?

• Would you be able to answer this question yourself? (If no) Who would be the appropriate person to answer it?

• (If R appears to have problem/confusion) What are you thinking about?

• Reflect R’s specific answer: “You said ___. Can you tell me more about that?”

Existing ASE questions

• General probes

Management practices

Q1

• What does “service or production problem” mean to you?

• What are some examples of problems encountered at your business, if any?

Q2

• What does “key performance indicator” mean to you?

• (If one or more selected) What are some examples of the indicators you were thinking of?

Q3

• (For any responses besides Never) Can you tell me more about the type(s) of review you had in mind when you selected your response(s)?

Q4

• What does “business, service, or production targets” mean to you?

• What are some examples of such targets used at your business, if any?

• What does “time frame” mean to you?

• Note whether R indicates any confusion between “targets” and “performance indicators” in Q2-3.

Q5

• How did you decide which answer to select?

Use of alternate forms of labor

Q8

• What does each category listed here mean to you?

Q9

• What does “how intensive” mean to you in the context of this question?

Q10

• How would you distinguish between each of these categories?

• Note how R distinguishes between “some overlap”, “significant overlap”, and “complete overlap.”

Record-keeping

Q11

• What does “record-keeping” mean to you?

• What does each of these categories mean to you?

• How would you respond if none of these categories applied to your business?

Q12

• What does “format of record-keeping” mean to you?

• What does “general purpose software” mean to you? Can you give me an example?

• What does “software designed for record-keeping mean to you? Can you give me an example?

Q13

• Please tell me in your own words what each of these answer categories means to you.

• (For each category selected) What were you thinking about when you selected that category? How does your business use data to affect decisions about that category?

Web-Only Reporting

• Adjust the tense of these questions according to whether R responded to the 2014 ASE.

• This survey only has an internet reporting option. How [would/did] that work for you?

• [Will/did] it make any difference that we [will/did] not send a paper form?

• Thinking about the questions we have been going over today, would you have any need for something to download or print to assist you in preparing your responses?

  • (If yes) What sort of downloadable/printable document would you want to have? What would it look like? What would you do with it? What level of detail should it contain?

• Do you want to keep copies of your response to this survey for your records?

  • (If yes) What format would you prefer?

• What has been your experience with using internet reporting systems for surveys or other reporting?

Wrap-up Questions

• (If previous R) About how long did it take you to complete the survey before?

• How long do you think it would take you to complete these questions?

• Any other feedback you would like to give us on these questions or your experiences?

Thanks for your time!

Existing Qs to test?

• New Funding Relationships

• Avoidance of additional financing – new screener and follow-up

• Customer locations

5