Cognitive Testing Protocol (CAI)

ACS Respondent Burden Income Testing

Cognitive Interview Protocol

Mode

CAI

Introduction

Hello, my name is ______________ and I work here at Westat. Thank you for taking the time to participate in this research study.

Westat is working to develop new questions for the American Community Survey, which is sponsored by the U.S. Census Bureau. Before surveys are conducted, it’s important to try out questions with the help of people such as yourself. The survey asks questions about you and your household.

It is important that the questions make sense, are easy to answer, and that everyone understands the questions the same way. If you agree to take part in this study, I will read you the questions as if I were an interviewer contacting you at your home. Afterwards, I will ask you some questions about the answers you gave. There are no right or wrong answers. Our purpose is not to compile information about you. Instead, your interview along with those of others will show us how to improve these questions for a later survey.

Informed Consent

Before we get started, there are a few things I should mention. This is a research project, and your participation is voluntary. If you prefer not to answer any questions just say so, and we’ll go on to the next one. It’s also okay if you change your mind after starting and would rather not participate.

All your answers, everything you say, will be kept confidential. We will not use your name in any reports. The interview will take about 40 minutes and you will receive $40. We will also need to audio record our conversation. This helps me so I can listen to what you are saying and won’t have to take a lot of detailed notes while you are talking; it will also help when we write up a summary of this interview. Only project staff will have access to the recording and other project materials, and those materials will be stored according to Title 13 requirements for protecting the identity of individual respondents. {Finally, some of the researchers developing the questions are here today observing our interview to learn if there are things that might need to be changed.}

HAND CONSENT FORMS TO RESPONDENT, This form contains all of the things I just told you about your rights in this interview. Please read it over and sign both copies if you are willing to take part in the study.

HAVE R SIGN TWO CONSENT FORMS, KEEP ONE AND GIVE ONE TO RESPONDENT.

TURN ON RECORDER. The date and time is ____________. Now that the recorder is running, let me ask again, is it okay with you if we record this interview?

Protocol

In a moment, I am going to begin asking you the questions. I’d like you to answer them as you would if I contacted you at home. Also, you will notice that some questions ask about “this address” which refers to your home, not the building we are sitting in today. After I have finished asking you the questions, we will talk about some of the answers you gave. Before we begin, do you have any questions about the process?

BEGIN AMERICAN COMMUNITY SURVEY ROSTER

ASK ENTIRE ROSTER, ROW BY ROW (ASKING ALL SUB-ITEMS FOR A QUESTION BEFORE MOVING TO THE NEXT PERSON), THEN STOP FOR PERSON SELECTION.

SELECTION OF PERSONS TO PROBE ON:

WRITE RESPONDENT’S FIRST NAME AS PERSON #1 BELOW.

SELECT ONLY ADULTS AGES 15 OR OLDER AS PERSONS 2-3.

PRIORITIZE SPOUSE, THEN NON-RELATIVES, THEN RELATIVES FROM Q2 OF HOUSEHOLD ROSTER

Person 1 First Name _________________________

Person 2 First Name _________________________

Person 3 First Name _________________________

weeks worked: Q40a-C, page 4

Research Questions:

Does asking Hours Worked first provide helpful context for calculating the number of weeks worked?

Do respondents prefer to answer in weeks or months worked?

Does the option to answer in months lower burden for respondents?

Do respondents accurately report the number of weeks or months they worked during this reference period?

How do respondents calculate number of weeks or months they have worked?

Do respondents interpret the reference period as intended?

How easy or difficult is it to recall weeks worked during this reference period?

PERSON 1 PROBES

Earlier I asked some questions about the weeks you work. First, I asked, “During the 52 weeks covering (2015/2016), did you work EVERY week? Count paid vacation, paid sick leave, and military service as work.” And you said [REPEAT R’S ANSWER TO 40A].

How did you decide on your answer here?

Was it easy or difficult to answer this question? What made it easy or difficult?

What time period were you thinking of when you answered this question?

IF ANSWERED 40B: Next I asked you, “During the 52 weeks covering (2015/2016), how many WEEKS did you work? Include paid time off and include weeks when (you/Name) only worked for a few hours.” And I said “If you would rather give your answer in months, please say so.” And you answered [REPEAT R’S ANSWER FOR Q40B/C].

First of all, how did you decide whether you wanted to answer in weeks or months?

How did you come up with your answer to the number of [weeks/months] you worked?

IF NEEDED: How did you decide what [weeks/months] to include or exclude from your answer?

IF NEEDED: Did you include part-time jobs? Did you include paid time off?

[IF R ANSWERED IN WEEKS] Would it have been easier or more difficult to answer this question in months instead of weeks? What makes you feel that way?

[IF R ANSWERED IN MONTHS] Would it have been easier or more difficult to answer this question in weeks instead of months? What makes you feel that way?

How confident are you in the answer you gave? Explain.

Before you answered the question about the number of weeks you’d worked, you answered a question about the number of hours you usually work each week. How helpful was it to answer first about the hours you worked then about the weeks you worked? Would you have preferred a different order? Explain.

PERSON 2 PROBES (PAGE 10)

How easy or difficult was this for you to answer the number of weeks worked for (PERSON 2)? What made it easy/difficult?

IF ANSWERED 40B for (PERSON 2): How did you decide whether to answer in weeks or months for (PERSON 2)?

How confident are you in the answer you provided for (PERSON 2)?

PERSON 3 PROBES (PAGE 16)

How easy or difficult was this for you to answer the number of weeks worked for (PERSON 3)? What made it easy/difficult?

IF ANSWERED 40B for (PERSON 3):How did you decide whether to answer in weeks or months for (PERSON 3)?

How confident are you in the answer you provided for (PERSON 3)?

INCOME: Q47-Q48, pageS 5-7

Research Questions:

Do respondents interpret the reference period as intended?

Can respondents respond accurately at the end of the year about the previous calendar year?

Can respondents respond accurately at the beginning of the year even if they haven’t done their taxes yet?

Do respondents perceive reporting income for the prior calendar year to be less burdensome than reporting for the past 12 months?

Do respondents include all applicable income types in total income?

Could respondents have provided dollar amounts for each yes response in Q47? Would respondents reference tax documents if responding at home?

How closely did respondents pay attention to the instructions at Q48, and are there any comprehension difficulties?

How easy or difficult is it to provide total income in this format?

When reporting income for a spouse, how do respondents handle joint income?

Next let’s talk about the question when I asked you about different types of income that you received last year.

[IF YES TO Q47a] You said yes to wages or salary . Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47b] You said yes to tips, bonus or commission. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

You said [READ RESPONSE FROM Q47c] to the question that asked if you received any self-employment income in (2015/2016)? I also said to consider income from own businesses (farm or non-farm) including proprietorships and partnerships. Tell me how you decided on your answer to this question.

IF NEEDED: What type of income were you thinking about here?

[IF YES TO Q47c] What time period were you thinking about? [IF NEEDED: From what date to what date?]

IF YES TO 47c: What do you think was meant by the follow-up question I asked about self-employment income, “Was this income a loss?”

You said [READ RESPONSE FROM Q47e] to the question that asked if you received any interest or dividends in (2015/2016) and to consider even small amounts credited to an account. How did you decide on your answer to this question?

IF NEEDED: What type of income were you thinking about here?

What time period were you thinking about? [IF NEEDED: From what date to what date?]

You said [READ RESPONSE FROM Q47f] to the question that asked if you received any net rental income in (2015/2016). How did you decide on your answer to this question?

IF NEEDED: What type of income were you thinking about here?

[IF YES TO Q47f] What time period were you thinking about? [IF NEEDED: From what date to what date?]

IF YES TO 47f: What do you think was meant by the follow-up question I asked, “Was this income a loss?”

[IF YES TO Q47h] You said yes to royalty income or income from estates and trusts. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47i] You said yes to Social Security or Railroad Retirement benefits. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47j] You said yes to Supplemental Security Income payments. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47k] You said yes to public assistance or public welfare payments from the state or local welfare office. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47l] You said yes to retirement, survivor, or disability pensions. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

[IF YES TO Q47m] You said yes to any other sources such as Department of Veterans Affairs payments, unemployment compensation, child support, or alimony. Tell me how you decided on your answer to this question.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

In (2015/2016), were there any other types of income that you earned that I did not ask you about?

How easy or difficult is it for you to remember what types of income you had in (2015/2016)? [IF NEEDED, REPEAT YES CATEGORIES.] Explain.

How certain are you of your answers to these questions? Explain.

Next I asked you to tell me what your TOTAL income was in (2015/2016) from all sources. And I said [READ APPROPRIATE INSTRUCTIONS THAT WERE READ WITH THE QUESTION]. Tell me how you came up with your answer.

What time period were you thinking about? [IF NEEDED: From what date to what date?]

How easy or difficult was it to come up with your answer?

How accurate would you say your answer is?

If you were answering these questions at home, how might you have come up with your answer? (IF NECESSARY IN SET 1: Would you have looked at last year’s tax returns?)

[IF MORE THAN ONE “YES” RESPONSE IN Q47] What types of income did you include in your total income answer?

[IF R DOES NOT MENTION ALL OF THE CATEGORIES THAT WERE YES IN Q47] Earlier, you mentioned (SOURCES OF INCOME). How did you decide whether or not to include that income in your answer for total income?

[IF R MENTIONED ANY OTHER TYPE OF INCOME NOT LISTED IN Q47] A few minutes ago, you mentioned that you also earned income from a source that I hadn’t asked you about before. Did you include that amount in your answer for total income?

IF NEEDED: If we had asked for it, could you have given a dollar amount for each of the types of income you said yes to?

[IF Q47d or 47g=“Yes” (Loss)] Earlier you said you had a loss in (CATEGORY). How did you account for that when you were coming up with the total income amount? (Explain)

Would you prefer to provide income amounts for each of the different types of income categories that you said yes to, and then add them up to get a total, or would you prefer to just provide the total income amount? What makes you say that?

PERSON 2 PROBES (PAGE 11-13)

How easy or difficult was this for you to answer the income questions for (PERSON 2)? What made it easy/difficult?

[IF Q47d or 47g=“Yes” (Loss)] Earlier you said [PERSON 2] had a loss in (CATEGORY). How did you account for that when you were coming up with [PERSON 2’s] total income amount? (Explain)

How confident are you in the answers you provided for (PERSON 2)?

IF NEEDED (AND P2 IS R’S SPOUSE): How did you decide where to report joint income that you and (PERSON 2) earned together?

PERSON 3 PROBES (PAGE 17-19)

How easy or difficult was this for you to answer the income questions for (PERSON 3)? What made it easy/difficult?

[IF Q47d or 47g=“Yes” (Loss)] Earlier you said [PERSON 3] had a loss in (CATEGORY). How did you account for that when you were coming up with [PERSON 3’s] total income amount? (Explain)

How confident are you in the answers you provided for (PERSON 3)?

Closing and Incentive

IF OBSERVERS ARE PRESENT, CHECK TO SEE IF THEY HAVE FURTHER QUESTIONS.

Those are all the questions I have for you. Is there anything we haven't discussed that you would like to mention?

DISCUSS ANY RESPONDENT COMMENTS.

Thank you for your time.

STOP TAPE RECORDER.
GIVE INCENTIVE AND HAVE RESPONDENT SIGN RECEIPT.

OY1 ACS Respondent Burden Testing – Weeks Worked and Income Series: FINAL Protocol: CAI 13