Cognitive Interview Protocol (Paper)

ACS Respondent Burden Income Testing

Cognitive Interview Protocol

Mode

Paper

Introduction

Hello, my name is ______________ and I work here at Westat. Thank you for taking the time to participate in this research study.

Westat is working to develop new questions for the American Community Survey, which is sponsored by the U.S. Census Bureau. Before surveys are conducted, it’s important to try out questions with the help of people such as yourself. The survey asks questions about you and your household.

It is important that the questions make sense, are easy to answer, and that everyone understands the questions the same way. If you agree to take part in this study, I will hand you a questionnaire to answer. After you complete it, I will ask you some questions about the answers you gave. There are no right or wrong answers. Our purpose is not to compile information about you. Instead, your interview along with those of others will show us how to improve these questions for a later survey.

Informed Consent

Before we get started, there are a few things I should mention. This is a research project, and your participation is voluntary. If you prefer not to answer any questions just say so, and you can go on to the next one. It’s also okay if you change your mind after starting and would rather not participate.

All your answers, everything you say, will be kept confidential. We will not use your name in any reports. The interview will take about 40 minutes and you will receive $40. We will also need to audio record our conversation. This helps me so I can listen to what you are saying and won’t have to take a lot of detailed notes while you are talking; it will also help when we write up a summary of this interview. Only project staff will have access to the recording and other project materials, and those materials will be stored according to Title 13 requirements for protecting the identity of individual respondents. {Finally, some of the researchers developing the questions are here today observing our interview to learn if there are things that might need to be changed.}

HAND CONSENT FORMS TO RESPONDENT, This form contains all of the things I just told you about your rights in this interview. Please read it over and sign both copies if you are willing to take part in the study.

HAVE R SIGN TWO CONSENT FORMS, KEEP ONE AND GIVE ONE TO RESPONDENT.

TURN ON RECORDER. The date and time is ____________. Now that the recorder is running, let me ask again, is it okay with you if we record this interview?

Protocol

In a moment, I am going to hand you the questionnaire. I’d like you to fill it out as if you received it in the mail at home. If you had received this form at home, your address would be located on the front of the questionnaire in the white area.

After you have finished answering the questions, we will talk about some of the answers you gave. Before we begin, do you have any questions about the process?

HAND QUESTIONNAIRE TO RESPONDENT: Please go ahead and start filling out the questionnaire.

SELECTION OF PERSONS TO PROBE ON:

WRITE RESPONDENT’S FIRST NAME AS PERSON #1 BELOW.

SELECT ONLY ADULTS AGES 15 OR OLDER AS PERSONS 2-3.

PRIORITIZE SPOUSE, THEN NON-RELATIVES, THEN RELATIVES FROM Q2 OF HOUSEHOLD ROSTER

Person 1 First Name _________________________

Person 2 First Name _________________________

Person 3 First Name _________________________

weeks worked: Q40a-b, page 10

Research Questions:

Does asking Hours Worked first provide helpful context for calculating the number of weeks worked?

How do respondents calculate number of weeks they have worked? If converting from months to weeks, how well does that conversion align with the actual number of weeks they worked?

Do respondents interpret the reference period as intended?

Do respondents accurately report the number of weeks they worked during this reference period?

How easy or difficult is it to recall weeks worked during the reference period?

PERSON 1 PROBES

Let’s look at question 40a. How did you decide on your answer here?

Was it easy or difficult to answer this question? What made it easy or difficult?

What time period were you thinking of when you answered this question?

IF ANSWERED 40B: Now let’s look at question 40b. How did you come up with your answer to the number of weeks you worked?

IF NEEDED: How did you decide what weeks to include or exclude from your answer?

IF NEEDED: Did you include part-time jobs? Did you include paid time off?

Would it be easier or more difficult to answer this question in months instead of weeks? What makes you feel that way?

How confident are you in the answer you gave? Explain.

Before you answered the question about the number of weeks you’d worked, you answered a question about the number of hours you usually work each week. How helpful was it to answer first about the hours you worked then about the weeks you worked? Would you have preferred a different order? Explain.

PERSON 2 PROBES (PAGE 14)

How easy or difficult was this for you to answer the number of weeks worked for (PERSON 2)? What made it easy/difficult?

How confident are you in the answer you provided for (PERSON 2)?

PERSON 3 PROBES (PAGE 18)

How easy or difficult was this for you to answer the number of weeks worked for (PERSON 3)? What made it easy/difficult?

How confident are you in the answer you provided for (PERSON 3)?

INCOME: Q47-Q48, page 11

Research Questions:

Do respondents interpret the reference period as intended?

Can respondents respond accurately at the end of the year about the previous calendar year?

Can respondents respond accurately at the beginning of the year even if they haven’t done their taxes yet?

Do respondents perceive reporting income for the prior calendar year to be less burdensome than reporting for the past 12 months?

Do respondents include all applicable income types in total income?

Could respondents have provided dollar amounts for each yes response in Q47? Would respondents reference tax documents if responding at home?

How closely did respondents pay attention to the instructions at Q48, and are there any comprehension difficulties?

How easy or difficult is it to provide total income in this format?

When reporting income for a spouse, how do respondents handle joint income?

Let’s now look at question 47.

[IF YES TO Q47a] You said yes to 47a. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

You said [READ RESPONSE FROM Q47b] to 47b. Tell me how you decided on your answer to this question.

IF NEEDED: What type of income were you thinking about here?

[IF YES OR LOSS IN Q47b] What time period were you thinking about? [IF NEEDED: From what date to what date?]

IF YES OR LOSS IN 47b: What do you think was meant by the category for “loss”?

You said [READ RESPONSE FROM Q47c] to 47c. Tell me how you decided on your answer to this question.

IF NEEDED: What type of income were you thinking about here?

[IF YES OR LOSS IN Q47c] What time period were you thinking about? [IF NEEDED: From what date to what date?]

IF YES OR LOSS IN 47c: What do you think was meant by the category for “loss”?

[IF YES TO Q47d] You said yes to 47d. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47e] You said yes to 47e. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47f] You said yes to 47f. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47g] You said yes to 47g. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47h] You said yes to 47h. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

In (2015/2016), were there any other types of income that you earned that were not listed in question 47?

How easy or difficult is it for you to remember what types of income you had in (2015/2016)? [IF NEEDED, REPEAT YES CATEGORIES.] Explain.

How certain are you of your answers to these questions? Explain.

Now let’s look at Q48. Tell me how you came up with your answer.

What time period were you thinking about?

How easy or difficult was it to come up with your answer?

How accurate would you say your answer is to question 48?

If you were filling out this survey at home, how might you have come up with your answer? (IF NECESSARY IN SET 1: Would you have looked at last year’s tax returns?)

[IF MORE THAN ONE “YES” RESPONSE IN Q47] What types of income did you include in your answer?

[IF R DOES NOT MENTION ALL OF THE CATEGORIES MARKED IN Q47] In the previous question, you mentioned (SOURCES OF INCOME). How did you decide whether or not to include that income in your answer in Q48?

[IF R MENTIONED ANY OTHER TYPE OF INCOME NOT LISTED IN Q47] A few minutes ago, you mentioned that you also earned income from a source not listed in question 47. Did you include that amount in your answer to question 48?

IF NEEDED: If we had asked for it, could you have provided a dollar amount for each type of income you said yes to in question 47?

[IF Q47b or 47c=”Loss”] Earlier you said you had a loss in (CATEGORY). How did you account for that when you were coming up with the total income in Q48? (Explain)

Did you notice the instructions in Q48?

What did you think these instructions were telling you to do?

How closely would you say you paid attention to those instructions?

Would you prefer to provide income amounts for each of the categories in Q47, and then add them up to get a total, or would you prefer to just provide the total in Q48? What makes you say that?

PERSON 2 PROBES (PAGE 15)

How easy or difficult was this for you to answer the income questions for (PERSON 2)? What made it easy/difficult?

[IF Q47b or 47c=”Loss”] Earlier you said [PERSON 2] had a loss in (CATEGORY). How did you account for that when you were coming up with [PERSON 2’s] total income in Q48? (Explain)

How confident are you in the answers you provided for (PERSON 2)?

IF NEEDED (AND P2 IS R’S SPOUSE): How did you decide where to report joint income that you and (PERSON 2) earn together?

PERSON 3 PROBES (PAGE 19)

How easy or difficult was this for you to answer the income questions for (PERSON 3)? What made it easy/difficult?

[IF Q47b or 47c=”Loss”] Earlier you said [PERSON 3] had a loss in (CATEGORY). How did you account for that when you were coming up with [PERSON 3’s] total income in Q48? (Explain)

How confident are you in the answers you provided for (PERSON 3)?

Closing and Incentive

IF OBSERVERS ARE PRESENT, CHECK TO SEE IF THEY HAVE FURTHER QUESTIONS.

Those are all the questions I have for you. Is there anything we haven't discussed that you would like to mention?

DISCUSS ANY RESPONDENT COMMENTS.

Thank you for your time.

STOP TAPE RECORDER.
GIVE INCENTIVE AND HAVE RESPONDENT SIGN RECEIPT.

OY1 ACS Respondent Burden Testing – Weeks Worked and Income Series: FINAL Protocol: Paper 11