Draft Cognitive Interview Protocol, 8/14/19
(Kristin Stettler and Krysten Mesner)
Research Questions:
Do respondents understand what information they are being asked to provide with these new questions?
Do respondents understand the terminology used in the questions?
Are respondents able to answer the questions that are being asked?
Is the requested information available in respondent’s records?
Are the instructions helpful to respondents?
Expected Length of Interview: 30-60 minutes
Materials Needed:
Draft questions
New definitions from Classification Manual
Consent forms
Digital recorder
General probes that may be used throughout the interview:
Reflect back on respondent’s answer: “you said…”
In your own words, what is this question asking?
How did you arrive at this number/answer this question?
What records (if any) did you look at? What line or lines were of interest?
Specifically, what did you include in this number? What did you exclude?
Would you consult other people to obtain this answer?
Explain purpose of meeting: get feedback on potential redesigned questions for the Quarterly Summary of State and Local Government Tax Revenue (F-72).
There may be times throughout this interview that this seems like a test of you, I assure you that it is not. The purpose of these interviews is to ensure that we are communicating the data requests clearly with you, so in that way, these interviews are actually a test of us. On that note, we want your honest feedback throughout, and if it’s not clear to you how to answer, please do your best. There will be ample opportunities for you to let us know how to make improvements so that you could do a better job answering.
First, I’d like to learn a bit about each of you, and your roles at your organization.
What is your job title? What is your background, in terms of education and job experience?
What is your role in completing government surveys?
How much experience do you have completing the QTAX survey?
Do you work with anyone else in your organization to get the data for the QTAX survey?
How are those other people involved?
How many people are involved?
Before we get started with our draft questions, I want to ask you to about your financial records.
What sorts of financial records are available?
What type of financial records systems do you have?
[If in person] Are you able to show us your system(s)?
Thank you for that background information! Now I’d like to have you review the current version of the QTAX survey, just to refresh your memory.
What was your experience completing this survey?
How long would you estimate it takes to complete, each quarter, including gathering data, etc.? What takes the longest?
Have you looked through the Classification Manual? Just once, or do you refer to it regularly?
Do you have any comments or suggestions for us regarding the current form?
Next, I am going to show you the revised version that we have drafted, along with some new definitions for the Classification Manual. Glancing through the form, what is your first impression?
OK, now let’s look at Item 2c, Online sales and gross receipts taxes.
In your own words, what are we asking for here?
How would you answer this question?
Where would the data come from in your records?
Do you have this information in your records?
What difficulties might you have retrieving this data?
What types of things would you be including here?
What does “Online sales and gross receipts taxes” mean to you?
What does “Portion of 2a collected from Internet-based retailers and remote sellers” mean to you?
Do you have any other terms for this?
Is there any additional information you would like to see provided here?
Would you be likely to look at the definition in the Classification Manual?
Looking at the definition in the Classification Manual, what is your reaction?
Would you like any additional information or clarifications?
Could anything be deleted?
Would this have already been included in 2a, State imposed taxes above?
Do you have any other thoughts or suggestions on this item?
Now let’s look at Item 4g, Sports betting.
In your own words, what are we asking for here?
How would you answer this question?
Where would the data come from in your records?
Do you have this information in your records?
What difficulties might you have retrieving this data?
What types of things would you be including here?
What does “Sports betting” mean to you?
Do you have any other terms for this?
How does this relate to Pari-mutuels and Amusements, above?
Is there any additional information you would like to see provided here?
Would you be likely to look at the definition in the Classification Manual?
Looking at the definition in the Classification Manual, what is your reaction?
Would you like any additional information or clarifications?
Could anything be deleted?
Where was this being recorded in previous forms?
Do you have any other thoughts or suggestions on this item?
Next let’s look at Items 4h, i, and j, Medical marijuana sales, Recreational marijuana sales and Total marijuana sales (h plus i).
In your own words, what are we asking for here?
How would you answer these questions?
Where would the data come from in your records?
Do you have this information in your records?
What difficulties might you have retrieving this data?
What types of things would you be including here?
What does “medical marijuana” and “recreational marijuana” mean to you?
Do you have any other terms for this?
Is there any additional information you would like to see provided here?
Would you be likely to look at the definition in the Classification Manual?
Looking at the definition in the Classification Manual, what is your reaction?
Would you like any additional information or clarifications?
Could anything be deleted?
If you were not able to provide medical and recreational marijuana separately, would you be able to provide the total, in j?
Where was this being recorded in previous forms?
Do you have any other thoughts or suggestions on this item?
OK, let’s look at Item 5h, Marijuana licenses.
In your own words, what are we asking for here?
How would you answer this question?
Where would the data come from in your records?
Do you have this information in your records?
What difficulties might you have retrieving this data?
What does “Marijuana licenses” mean to you?
Do you have any other terms for this?
What types of things would you be including here?
Are individual licenses included here, or somewhere else?
Is there any additional information you would like to see provided here?
Would you be likely to look at the definition in the Classification Manual?
Looking at the definition in the Classification Manual, what is your reaction?
Would you like any additional information or clarifications?
Could anything be deleted?
Where was this being recorded in previous forms?
Do you have any other thoughts or suggestions on this item?
Feedback on Other Items
Thanks for your input on the new items. I wanted to review a couple of the current items to get your feedback on them as well. Is that ok?
In 4f, Amusements, what are you including?
In 2b, Collection of locally imposed supplements to State general sales taxes (net of State –retained amount), what does this mean to you? How have you been answering this?
In Item 4, where are you including vaping? Would it help to clarify where it should be? For example, “Tobacco/vaping products”? Are there other terms you use for this?
Classification Manual
Now I would like to review the current Classification Manual.
In general, what do you think of the definitions?
Are there additional definitions/instructions you think would be helpful?
Are there any definitions/instructions you think are NOT necessary?
Do you have any suggestions for improvement?
Wrap-up
We are almost done, and I just have a few wrap-up questions.
Overall, what did you think about the new questions we reviewed?
Do you have any suggestions for improvement?
Do you think these questions would be easy to answer, difficult to answer, or somewhere in between? Why?
Could you provide an estimate of how long you think it would take you to answer just the questions we reviewed today? What is driving that estimate?
Compared to the current version of the QTAX form, what do you think of this revised version? Would this version be easier or harder for you to complete?
Is there anything else we haven’t touched on today, that you think is relevant?
Do you have any questions for us?
Thank you very much for your time today!
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kenneth Herrell (CENSUS/ESMD FED) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |