Research Goals, Recruiting Requirements, and Question Wording for the Second Round of Testing

American Community Survey Cognitive Testing to Reduce the Burden and Difficulty of Questions

Research Goals, Recruiting Requirements, and Question Wording for the Second Round of Testing

October 18, 2016

Weeks Worked and Hours Worked

Research Goals

Sequence of Hours Worked and Weeks Worked: Continue to evaluate asking Hours Worked before Weeks Worked to see if asking Hours Worked first provides helpful context in framing the work done in the past year. Unlike the first round of testing, we will not show them a different version but would like to probe on if respondents felt that this sequence effected the burden of this question.

Months Worked: Evaluate the explicit option to respond to weeks worked in months in the interviewer-administered mode. We would like to know (1) if respondents choose this option, (2) if this option lessens burden, and (3) if respondents report accurately when reporting this way (we are concerned that respondents who work 2 weeks a month will report with 12 months, when the correct answer would be 24 weeks).

Reference Period: Examine modifications to clarify the reference period. To coincide with the income series, we are changing the reference period to be the past calendar year. We would like to know (1) if respondents can comprehend the reference period/reference period clarity, (2) if respondents report accurately based on this reference period, and (3) if the revised reference period affects respondent burden (are respondents able to easily remember their weeks worked and hours worked for the past calendar year both at the end of the year and at the beginning of the year?)

Recruiting Requirements

1. Because the reference period will be 2015 for the December testing, around 2/3 of respondents should have worked in that year. For the January testing, we would like respondents who worked in 2016.

2. Include respondents who work in “irregular” schedule jobs (worked less than 52 weeks a year, don’t work each week or the same hours each week, have some seasonal patterns to the work, such as agricultural areas or substitute teachers, those in the food industry, retail workers, etc.)

Proposed Wording

Paper

CATI/CAPI

38. When did this person last work, even for a few days? Within the past 12 months 1 to 5 years ago* Over 5 years ago or never worked -> Skip to question 47 39. During the 52 weeks covering 2015**, that is from January 1, 2015 to December 31, 2015, in the WEEKS WORKED, how many hours did this person usually work each WEEK? Usual hours worked each WEEK:___ 40a. During the 52 weeks covering 2015, did this person work EVERY week? Count paid vacation, paid sick leave, and military service as work. Yes -> skip to question 41 No b. During the 52 weeks covering 2015, how many WEEKS did this person work? Include paid time off and include weeks when this person only worked for a few hours. WEEKS: ____

When did (you/name) last work, even for a few days? Within the past 12 months Between 1 to 5 years ago* Over 5 years ago or never worked ( Skip to question 47) During the 52 weeks covering 2015**, that is from January 1, 2015 to December 31,2015, in the weeks you worked, how many hours did (you/Name) usually work each week? Usual hours worked each week:_____ During the 52 weeks covering 2015, did (you/Name) work EVERY week? Count paid vacation, paid sick leave, and military service as work. Yes (skips next 2 questions) No During the 52 weeks covering 2015, how many WEEKS did (you/Name) work? Include paid time off and include weeks when (you/Name) only worked for a few hours. If you would rather give your answer in months, please say so. Weeks ____ (skip next question) Would rather answer in months How many months did you work in 2015? Months ____

* There is normally a skip instruction here. If respondents choose this option, they skip over the hours worked and weeks worked questions. However, because we changed the reference period to be the last calendar year instead of the last 12 months, some people in this category should get the weeks worked and hours worked questions. To resolve that issue, we are removing the skip instruction for this round of cognitive testing. We recognize that by doing so, some people who did not work in 2015 will now get these questions. For the CAI mode for this testing, we could create an instruction for the interviewer to skip the Weeks Worked and Hours Worked questions if the respondents mention that the last worked between 2010 and 2014. Westat could also focus the recruiting effort on those who worked in 2015 or sooner and those who last worked 5 years ago or never worked. If testing shows this reference period as a positive change, we will come up with a better solution for question 38.

** Year will be 2016 for the January testing.

2017 Production Versions:

Paper:

Interviewer-Administered:

WKW

During the PAST 12 MONTHS or 52 weeks, did (<Name>/you) work 50 or more weeks? Count paid time off as work.

 1. Yes

 2. No

WKW2

How many weeks DID (<NAME>/you) work, even for a few hours, INCLUDING paid vacation, paid sick leave, and military service? Was it:

• Read all answer categories.

❍ 1. 50 to 52 weeks

❍ 2. 48 to 49 weeks

❍ 3. 40 to 47 weeks

❍ 4. 27 to 39 weeks

❍ 5. 14 to 26 weeks

❍ 6. 13 weeks or less

WKH

During the weeks worked IN THE PAST 12 MONTHS, how many hours did (<Name>/you) usually work each week?

2016 Content Test Version of Weeks Worked:

Paper:

Interviewer-Administered:

WKW

During the PAST 12 MONTHS or 52 weeks, did <(Name)/you> work EVERY week? Count paid vacation, paid sick leave, and military service as work.

 1. Yes SKIP NEXT QUESTION

 2. No

WKW2

During the PAST 12 MONTHS or 52 weeks, how many WEEKS did <(Name)/you> work? Include paid time off and include weeks when <(Name)/you> only worked for a few hours.

weeks ________

Income

Research Goals

Reference Period: Examine changes to the reference period. We would like to know (1) if respondents can comprehend the reference period/reference period clarity, (2) if respondents who report at the end of the year can accurately report income for the previous calendar year, (3) if respondents reporting at the beginning of the year before they do their taxes can report for the previous calendar year, and (4) if the revised reference period affects respondent burden (Is reporting income for the last full calendar year easier than income for the past 12 months?)

Including all income amounts in total: Evaluate whether respondents include all of the income type that they replied “Yes” to in their calculation of total income. Several of the income types are a part of question 47 as a reminder for respondents to include these types of income in their calculation in question 48. We want to know if respondents will still include these types of income in their total calculation even if we do not ask them to list the specific amounts in question 47.

Being able to report specific amount for the income types: Probe respondents about whether they could have given specific amounts for 47a through 47h if the survey would have asked for them. We also want to know if respondents would have used references such as tax forms or pay stubs to look up their income information if they were completing the survey at home.

Instructions: Examine respondents’ use and understanding of the instruction text. We changed the wording and placement of some the income instructions, and now question 48 has significantly more instruction test. We want to know if respondents take the time to read the instructions and if they understand them.

Respondent burden: Explore the effects of this question format on respondent burden. This research goal is not mutually exclusive from the others. We believe that omitting the requirement to report income amounts for question 47 will reduce respondent burden, but if this makes it more difficult to calculate total income and if the revised instructions are more cumbersome, then the main goal of this research was not met.

Recruiting Requirements

1. Respondents who do not work or are retired along with those who have not worked in the past five years (in recognition that we need mostly workers for weeks worked, this can be a smaller number of respondents).

2. Respondents who are currently working (employed with a wage/salary job)

3. Respondents who receive each of these income types:

- Self-employment income

- Interest or dividend

- Net rental income

- Pension or 401k

- Social security

- Supplemental security income

- Public assistance income

4. Respondents who are married and receive self-employment income from a business or interest and dividends, or net rental income (we want to know how they report joint income)

Proposed Wording

Paper

CATI/CAPI

47. INCOME RECEIVED IN 2015* Consider income received from January 1, 2015 to December 31, 2015. For income received jointly, if you know the appropriate amount for each person, mark (X) “Yes” for each person. If not, mark (X) “Yes” for only one person and mark (X) “No” for the other person. a. In 2015, did this person receive wages, salary, commissions, bonuses, or tips? Yes No b. In 2015, did this person receive self-employment income from own nonfarm businesses or farm businesses, including proprietorships and partnerships? If the net income was a loss, mark (X) the "Loss" box. Yes No Loss c. In 2015, did this person receive interest, dividends, net rental income, royalty income, or income from estates and trusts? Consider even small accounts credited to an account. If the net income was a loss, mark (X) the "Loss" box. Yes No Loss d. In 2015, did this person receive Social Security or Railroad Retirement? Yes No e. In 2015, did this person receive Supplemental Security Income (SS)? Yes No f. In 2015, did this person receive any public assistance or welfare payments from the state or local welfare office? Yes No g. In 2015, did this person receive retirement, survivor, or disability pensions? This does NOT include Social Security. Yes No h. In 2015, did this person receive any other sources of income received regularly such as Veterans’ (VA) payments, unemployment, compensation, child support or alimony? This does not include lump sum payments such as money from an inheritance or the sale of a home. Yes No 48. What was this person’s total income in 2015 from all sources? Include income amounts for questions 47a to 47h that were marked (X) "Yes." If "Yes" for 47a, include amount from all jobs before deductions for taxes, bonds, dues, or other items. If "Yes" for 47b, include NET income after business expenses. If "Loss" for 47b or 47c, subtract amount from total income. For income received jointly, include the appropriate share for this person in the total. If net income was a loss, enter the amount and mark (X) the "Loss" box next to the dollar amount.

The next few questions are about income received in 2015*, that is from January 1, 2015 to December 31, 2015… Did (you/name) receive any wages or salary? Yes No Did (you/name) receive any [FILL if yes to previous question: "additional"] tips, bonuses or commissions in 2015? Yes No Did (you/name) receive any self-employment income in 2015? (Consider income from own businesses (farm or non-farm) including proprietorships and partnerships.) Yes No (skips next question) Was this income a loss? Yes No [FILL(1) if person didn’t work in the past 5 year or never worked so this is first question in the series: The next few questions are about income in 2015, that is from January 1, 2015 to December 31, 2015 …] Did (you/name) receive any interest or dividends [FILL(2) if other FILL(1) is false: in 2015 ]? Consider even small amounts credited to an account. Yes No Did (you/name) receive any net rental income in 2015? Yes No (skips next question) Was this income a loss? Yes No Did (you/name) receive any royalty income or income from estates and trusts in 2015? Yes No Did (you/name) receive any Social Security or Railroad Retirement benefits in 2015? Yes No Did (you/name) receive any Supplemental Security Income (SSI) payments in 2015? Yes No Did (you/name) receive any public assistance or public welfare payments from the state or local welfare office in 2015? Yes No Did (you/name) receive any retirement, survivor, or disability pensions in 2015? Yes No Did (you/name) receive income on a REGULAR basis from any other sources such as Department of Veterans Affairs (VA) payments, unemployment compensation, child support or alimony in 2015? Yes No What was (you/name) TOTAL income in 2015 from all sources? (If “Yes” to EARNX or TIPSX, add this instruction:) Include amount from all jobs before taxes and other deductions. (If “Yes” to SEMX add this instruction:) For self-employment income, include net income after operating expenses. Report earnings as a tenant farmer or sharecropper. (If “Yes” to EARNX or TIPSX and “Yes” to SEMX add this instruction:) Include amount from all jobs before taxes and other deductions. For self-employment income, include net income after operating expenses. Report earnings as a tenant farmer or sharecropper.

* Year will be 2016 for January testing.

2017 Production Versions:

Paper:

CAI:

EARNX

The next few questions are about the income DURING THE PAST 12 MONTHS, that is from <current month, year – 1> to <month – 1, current year>…

Did [<Name/you] receive any wages or salary?

Yes

No

EARN

How much did [<Name>/you] receive in wages and salary from all jobs before taxes and other deductions?

TIPSX

Did [<Name>/you] receive any [EARN=1, fill with "additional"] tips, bonuses or commissions DURING THE PAST 12 MONTHS?

TIPS

How much did [<Name>/you] receive in tips, bonuses, or commissions from all jobs before taxes and other deductions?

SEMX

Did [FILL1: <Name>/you] receive any self-employment income DURING THE PAST 12 MONTHS?" (Report income from own businesses (farm or non-farm) including proprietorships and partnerships.)

Yes

No

SEM

"What was the amount?"(Report net income after operating expenses. Include earnings as a tenant farmer or sharecropper.)

INTRX

[FILL1: The next few questions are about income DURING THE PAST 12 MONTHS, that is from <current month, year- 1> to <month -1, current year>…] Did [FILL2: <Name>/you] receive any interest or dividends [FILL3: DURING THE PAST 12 MONTHS]? Report even small amounts credited to an account.

Yes

No

INTR

"What was the amount?"

RENTX

Did [<Name>/you] receive any net rental income DURING THE PAST 12 MONTHS?

Yes

No

RENT

What was the net amount?

ROYALX

Did [<Name>/you] receive any royalty income or income from estates and trusts DURING THE PAST 12 MONTHS?

Yes

No

ROYAL

What was the amount?

SSX

Did [FILL1: <Name>/you] receive any Social Security or Railroad Retirement benefits DURING THE PAST 12 MONTHS?"

Yes

No

SS

What was the amount?

SSIX

Did [FILL1: <Name>/you] receive any Supplemental Security Income (SSI) payments DURING THE PAST 12 MONTHS?

Yes

No

SSI

What was the amount?

PAX

Did [FILL1: <Name>/you] receive any public assistance or public welfare payments from the state or local welfare office DURING THE PAST 12 MONTHS?

Yes

No

PA

What was the amount?

RETX

Did [FILL1: <Name>/you] receive any retirement, survivor, or disability pensions DURING THE PAST 12 MONTHS?

Yes

No

RET

What was the amount?

OIX

Did [FILL1: <Name>/you] receive income on a REGULAR basis from any other sources such as Department of Veterans Affairs (VA) payments, unemployment compensation, child support or alimony DURING THE PAST 12 MONTHS?

Yes

No

OI

What was the amount from all sources?

TI

What was [FILL1: <Name>'s/your] TOTAL income during the PAST 12 MONTHS?

TICALC

[FILL1: According to my calculations [FILL2: <Name>/you have/<blank>] received $<sum of EARN, TIPS, SEM, INTR, RENT, ROYAL, SS, SSI, PA, RET, OI> from all income sources/<blank>] [FILL3: I have recorded that/<blank>] [FILL4: <Name> has/you have/<blank>] received no income] DURING THE PAST 12 MONTHS. Is this correct?"

Yes

No

TIEST

What is your best estimate of the total income [FILL1: <Name> has/you have] received from all sources DURING THE PAST 12 MONTHS?