Proposal Topics

03 - 2022 ACS Content Test - Cognitive Testing - Kickoff Meeting - Proposals.pdf

Generic Clearance for Questionnaire Pretesting Research

Proposal Topics

OMB: 0607-0725

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0607-0725 can be found here:

2022-10-14 - Extension without change of a currently approved collection

Document [pdf]

Download: pdf | pdf

2022 American Community Survey
Content Test
Cognitive Testing Kickoff Meeting
Census Bureau’s Materials Inventory
Proposals, N = 10

2022 ACS Content Test – Cognitive Testing
TOPIC – Household Roster
Wednesday, February 19, 2020
Subcommittee Lead:
Elizabeth Poehler, U.S. Census Bureau
Subcommittee Team Members:
Derek Breese, U.S. Census Bureau
Sandy Clark, U.S. Census Bureau
Mary Davis, U.S. Census Bureau
Sarah Heimel, U.S. Census Bureau
Eric Jensen, U.S. Census Bureau
Grace Kena, Bureau of Justice Statistics
Kathleen Kephart, U.S. Census Bureau
Scott Konicki, U.S. Census Bureau
Dave Raglin, U.S. Census Bureau
Andrew Roberts, U.S. Census Bureau
Mary Frances Zelenak, U.S. Census Bureau
Objective:
The objective is to test a new set of rostering instructions and coverage questions to improve the
within household coverage of the American Community Survey (ACS). There are known
coverage issues with census and survey data, such as the undercount of young children. In
addition, while the complexity of household living arrangements in the United States has
increased, the instructions used for the ACS have not changed since the late 1990s. Prior
research has demonstrated that respondents find the household roster portion of the survey
confusing and hard to understand (see Clark,2017; DeMaio and Hughes, 2003; Ashenfelter, et al,
2011; Ashenfelter et al, 2013). 1
Cognitive Testing Research Goals:
•
•
•
•

Which version helps respondents understand better that they should include unrelated people
on the roster?
Do respondents notice and comprehend all of the text about who to include or exclude
outside of the main question stems? Are they utilizing this text when responding?
Which version of the paper form yielded the more accurate roster?
(For paper) Was one format preferred by respondents?

Clark, S. (2017) “Analysis of the Household Roster Questions on the ACS” U.S. Census Bureau. Accessed
5/3/2019 at https://www.census.gov/library/working-papers/2017/acs/2017_Clark_01.html.
DeMaio, T. and Hughes, K. (2003) “Report of Cognitive Research on the Residence Rules and Seasonality
Questions on the American Community Survey (ACS)”, U.S. Census Bureau, Survey Methodology #2003-08.
Ashenfelter, K., Holland, T., Quach, V., Nicols, E. and Lakhe, S. (2011). “ACS Internet 2011 Project: Report for
Rounds 1 and 2 of ACS Wireframe Usability Testing and Round 1 of ACS Internet Experiment Mailing Materials
Cognitive Testing.” U.S. Census Bureau Survey Methodology #2012-01.
Ashenfelter, K., Holland, T., Quach, V. and Nichols, E. (2013). “Final Report for the Usability Evaluation of ACS
2011 Online Instrument Rounds 4a and 4b” U.S. Census Bureau. Survey Methodology #2013-04

2022 ACS Content Test – Cognitive Testing
TOPIC – Household Roster
Wednesday, February 19, 2020
•
•
•

•
•
•

(For CAPI version 1) What do the terms “short visit,” “short time,” and “overnight stay”
mean to respondents?
(For CAPI version 2) What does the term “short time” mean to respondents?
Were respondents hesitant or unsure about including anyone that lives or stays with them?
− If so, what are the relationships and/or living situations of those people?
− What was the reason they were hesitant or unsure to include them?
− Were they unsure how long someone had to live there to be included?
Do respondents have privacy concerns about reporting certain household members?
− If so, what are the relationships and/or living situations of those people?
− What wording increased or decreased their concerns?
(For CAPI only) Did the respondents express annoyance or confusion about why we were
asking the series of coverage questions?
If the respondent doesn't have a complex living situation now (they didn’t add or delete
people via the coverage questions), have they ever have had one (lived with someone else or
have someone else live with them)? How would they have answered the questions about that
situation?

Cognitive Testing Recruiting Requirements:
•

•
•
•
•
•
•
•
•

Households with multiple families cohabitating
Households with related subfamilies
Households with unrelated subfamilies/individuals
Households where children are in the subfamily, especially children aged 0-4
Households with children in custody arrangements
Households with foster children
Households with no one related to each other
Families with someone in the Armed Forces and on deployment (lower priority)
Families with children who live away at college (lower priority)

2022 ACS Content Test – Cognitive Testing
TOPIC – Household Roster
Wednesday, February 19, 2020
Paper

CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

2022 ACS Content Test – Cognitive Testing
TOPIC – Household Roster
Wednesday, February 19, 2020
Computer Assisted Interview Script
CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

Original Roster

I’m going to be asking some questions
about everyone who is living or staying
at this address. First let’s create a list of
the people, starting with you.

Let’s create a list of everyone, including
people not related to you, living or
staying at this address.

Let’s create a list of everyone living or
staying at this address, even if they are
not related to you.

What is your name?

(What is the name of the next person
living or staying here?)

What is your name? What is the name of
the next person living or staying here?
*Enter first name. If there are no
additional people to list, enter 999 to
continue.

 Make sure to include the respondent
if he/she is staying there.

 Make sure to include the respondent if
he/she is staying here.

Notes or Comments
1.
2.

Subcommittee is interested in using a flashcard
for V1. Does contractor agree?
Overcount Follow-Up will be at the end of the
interview testing Household Roster (Group 1,
Versions 1 and 2).

2022 ACS Content Test – Cognitive Testing
TOPIC – Household Roster
Wednesday, February 19, 2020

Undercount -1

Undercount Probe (Single Question)

Undercount -1

The following questions are to make sure
this list is as complete as possible…Does
anyone else live or stay here, such as
roommates, foster children, boarders, or
live-in employees?

The following questions are to make sure
everyone is included.

The people you have listed so far are…
 Read all names.

Other than the people you have already
mentioned, are there any children living
or staying here, such as babies,
grandchildren, or foster children? These
children could be related or unrelated to
you.

*Enter first name. If there are no
additional people to list, enter 999 to
continue.

Other than these people, is there anyone
else staying here such as…
• Babies
• Grandchildren
Undercount -2
• Foster children
Is there anyone else staying here even for • Any other children, related or
unrelated to you
a short time, such as a friend or relative?
• Roommates
*Do not include overnight or weekend
• People or families who have no other
guests who have a residence somewhere
place to stay?
else.
*Enter first name. If there are no
additional people to list, enter 999 to
continue.

Undercount -2
Other than the people you have already
mentioned, is there anyone else staying
here, such as roommates and other
people or families who have no other
place to stay?
 Do not include overnight or weekend
guests who have a residence somewhere
else.

 Do not include overnight or
weekend guests who have a
residence somewhere else.
 List additional names to the right.

2022 ACS Content Test – Cognitive Testing
TOPIC – Household Roster
Wednesday, February 19, 2020

Overcount - 1
The next questions are to help refine this
list. I have listed….
(read names)
*Read all bolded names.
Are any of these people away NOW for
more than two months, like a college
student or someone in the military?
*DO NOT select children in boarding
school or summer camp.
*Select children in shared custody who
are not currently staying at the sample
address, regardless of the length of stay.
*Select the persons who are away NOW
for more than two months.
*Enter all that apply, separate with
commas.
*The two-month period is not anchored
by a specific reference date, but can
encompass the two months prior to
survey contact or the two months
following the survey contact date.
*If no one is away, enter 999 to continue.

Overcount - 1
 Copy names from Original Roster,
Undercount Probe.

Overcount - 1
 Copy names from Original Roster,
Undercount – 1, and Undercount – 2.

The following questions are to make sure
everyone we have listed should be
included in this survey.

I have listed…
 Read all names.

I have…
 Read all names.

Do any of these people live somewhere
else, such as a college student or someone
in the Armed Forces on deployment?

Do any of these people live somewhere
else now, such as a college student or
someone in the Armed Forces on
deployment?

 MARK “No” for:
(1) children in boarding school or
summer camp

 MARK “Yes” for:
(1)children in shared custody who are
not currently staying at the sample
address, regardless of the length of
stay;
(2)persons who are away NOW for
MORE than two months^

 MARK “Yes” for:
(1) children in shared custody who are
not currently staying at
the
sample address, regardless of the length
^ - The two-month period is not anchored
of stay;
by a specific reference date, but can
(2) persons who are away NOW for
encompass the two months prior to the
more than two months^
interview or the two months following the
interview date.
^ - The two-month period is not anchored
by a specific reference date, but can

2022 ACS Content Test – Cognitive Testing
TOPIC – Household Roster
Wednesday, February 19, 2020

Overcount – 2

encompass the two months prior to the
interview or the two months following the
interview date.

Overcount – 2

 Copy names from Original Roster,
Undercount – 1, and Undercount – 2,
but OMIT anyone with a “Yes”
response to Overcount – 1.

(Do you/Do any of these people)…
*Read all bolded names.
have some other place where they
usually stay?
*DO NOT select children in shared
custody who are currently staying at the
sample address, regardless of where they
usually stay.
*DO NOT select commuter workers who
stay in some other residence closer to
work when their family residence is the
sample address.
*Select commuter workers who stay at
the sample address to be closer to work.
*Select the persons who have some other
residence.
*Enter all that apply, separate with
commas.
*If no one has another place where they
usually stay, enter 999 to continue.

 Copy names from Original Roster and
Undercount Probe, but OMIT anyone
with a “Yes” response to Overcount – 1.

I have listed…
 Read all remaining names.

Are any of these people staying here for
a short time?

Are any of these people staying here for
a short visit or for an overnight stay?

 MARK “No” for:
(1) children in shared custody who are
currently staying at the sample
address, regardless of where they
usually stay;
(2) commuter workers who stay in
some other residence closer to work
when their family residence is the
sample address.

 MARK “No” for:
(1) children in shared custody who are
currently staying at the sample address,
regardless of where they usually stay;
(2) commuter workers who stay in some
other residence closer to work when
their family residence is the sample
address.

 MARK “Yes” for:
(1) commuter workers who stay at the
sample address to be closer to work;
(2) persons who have some other
residence.

 MARK “Yes” for:
(1) Commuter workers who stay at the
sample address to be closer to work;
(2) Persons who have some other
residence.

2022 ACS Content Test – Cognitive Testing
TOPIC – Household Roster
Wednesday, February 19, 2020

More than 2 Months Question

(Are / Are you/ Is

 Copy names with a “Yes” response to
Overcount – 2.

I have listed…
 Read names with a “Yes” response to
Overcount – 2.

Are you/Is staying here for
MORE than two months^?

 MARK “Yes”:
(1) persons who are staying MORE than
two months^;
(2) children in shared custody who are
currently staying at the sample address,
regardless of where they usually stay;
(3) commuter workers who stay at the
sample address to be closer to work

 MARK “Yes”:
(1) persons who are staying MORE than
two months^
(2) children in shared custody who are
currently staying at the sample address,
regardless of where they usually stay
commuter workers who stay at the
sample address to be closer to work

^ - The two-month period is not
anchored by a specific reference date,
but can encompass the two months prior
to the interview or the two months
following the interview date.

*
staying here for MORE than two
months?
*Select the persons who are staying
MORE than two months.
*Select children in shared custody who
are currently staying at the sample
address, regardless of where they usually
stay.
*Select commuter workers who stay at
the sample address to be closer to work.
*Enter all that apply, separate with
commas.
*The two month period is not anchored
by a specific reference date, but can
encompass the two months prior to
survey contact or the two months
following the survey contact day.
*If no one is staying more than two
months, enter 999 to continue.

2022 ACS Content Test – Cognitive Testing
TOPIC – Household Roster
Wednesday, February 19, 2020

Roster Check Screen
*If you have keyed in a name in error,
delete the name by selecting it below.
*Enter all that apply, separate with
commas.
*If you do not need to delete anyone,
enter 999 to continue.

 Generate the final roster using the first
 Generate the final roster using the first 11-6 names remaining in the order they
6 names remaining in the order they
were mentioned:
were mentioned.
- Copy names from Original Roster and
- Copy names from Original Roster,
Undercount Probe
Undercount – 1 and Undercount - 2
- OMIT anyone with a “Yes” response to
- OMIT anyone with a “Yes response to
Overcount – 1
Overcount - 1
- OMIT anyone with a “No” response to
- OMIT anyone with a “No” response to
More than 2 Months Question
More than 2 Months Question
Overcount Follow-up

Overcount Follow-up

We are conducting research to
understand why people stay in more
than one place. Earlier in the survey you
indicated that [Auto fill all names from
overcount 1 and 2] sometimes live(s)
somewhere else or is (are) only staying
here for a short time. Could you briefly
explain [Autofill all names from
overcount 1 and 2]'s living situation?

We are conducting research to
understand why people stay in more
than one place. Earlier in the survey you
indicated that [Auto fill all names from
overcount 1 and 2] sometime(s) live(s)
somewhere else or is (are) only staying
here for a short time. Could you briefly
explain [Autofill all names from
overcount 1 and 2]'s living situation?

2022 ACS Content Test – Cognitive Testing
TOPIC – Educational Attainment
Wednesday, February 19, 2020
Subcommittee Lead:
Kurt Bauman, U.S. Census Bureau
Subcommittee Team Members:
Maribel Aponte, Department of Veterans Affairs
Chris Chapman, Department of Education
Tamara Lee, Department of Veterans Affairs
Laura Nixon, U.S. Census Bureau
Broderick Oliver, U.S. Census Bureau
David Raglin, U.S. Census Bureau
Erik Schmidt, U.S. Census Bureau
Kristi Scott, Social Security Administration
Stephen Simoncini, U.S. Census Bureau
Objective:
A relatively high percentage of people are selecting the response category “No schooling
completed” in the self-response modes of the American Community Survey, including
adults who have probably completed some level of schooling.
Our goal is to improve respondents’ understanding of the category that currently is called
“no schooling completed.” Our intent is that this category would include only those
people who have not completed as much as a single grade of formal schooling. That is,
they never completed formal schooling at any level to the point where they would be
qualified for the next level of instruction (e.g. going from kindergarten to first grade).
“Home school” that follows a curriculum would count as formal schooling in this sense.
Cognitive Testing Research Goals:
•
•
•
•

Do respondents understand this question and the overall meaning?
Use of the term “grade” (V1) rather than “level” (V2) of school in the base question Is one easier to understand)?
How do people currently enrolled in school answer this question? Does the
instruction help them answer?
Use of “less than grade 1” (V1) versus expanded three categories “less than 1 year of
school completed,” “nursery school or preschool,” and “kindergarten” (V2) - Does
one version better help people understand who should select these response options?
Specifically, does a broader category (V1) help people respond? How do those with
no schooling, schooling in levels below first grade, and others respond? This
difference in wording between versions also results in differences in the main
headings that the specific categories fall under. Do the heading differences affect
respondents’ understanding of the categories?
1

2022 ACS Content Test – Cognitive Testing
TOPIC – Educational Attainment
Wednesday, February 19, 2020
•
•

Unlike the current version, both test versions remove the heading and subheading
“No schooling completed.” Do those with no schooling know how to respond?
Do individuals with homeschooled children understand the question and response
categories and know where their child should be classified?

Cognitive Testing Recruiting Requirements:
•
•
•

Parents/legal guardians of homeschooled children who have not attended regular
school
Individuals aged 25 or over who do not have a high school diploma or a GED or
equivalent
Parents/legal guardians of at least one child aged 3 to 5

2022 ACS Content Test – Cognitive Testing
TOPIC – Educational Attainment
Wednesday, February 19, 2020
Paper
CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

Notes or Comments
1. Probe to see if
respondents read the
italicized instruction.
2. Probe to see if
respondents understand
the “less than grade1” and
“less than 1 year of school
completed” phrases. Do
respondents of Young
children in pre school and
K check the less than 1
year of school completed?

2022 ACS Content Test – Cognitive Testing
TOPIC – Educational Attainment
Wednesday, February 19, 2020
Computer Assisted Interview Script
CURRENT PRODUCTION VERSION
[Fill 3: Using this list, what/ What] is the
highest degree or level of school [Fill 2:
(you have/ has)] COMPLETED?
51. No schooling completed
52. Nursery school
53. Kindergarten
54. Grade 1
55. Grade 2
56. Grade 3
57. Grade 4
58. Grade 5
59. Grade 6
60. Grade 7
61. Grade 8
62. Grade 9
63. Grade 10
64. Grade 11
65. Grade 12, no diploma
66. Regular high school diploma
67. GED or alternative credential
68. Some college, no degree
69. Associate’s degree (for example:
AA, AS)
70. Bachelor’s degree (for example:
BA, BS)

TEST VERSION 1

TEST VERSION 2

NOTES OR COMMENTS

Using this list, what is the highest grade
of school or degree you have/
has COMPLETED? If you are/
is currently enrolled, select the previous
grade or highest degree received.

Using this list, what is the highest level of 1. Flashcards will be used.
school or degree you have/ has
COMPLETED? If you are/ is
currently enrolled, select the previous
grade or highest degree received.

Less than grade 1
Grade 1
Grade 2
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Grade 9
Grade 10
Grade 11
Grade 12, no diploma
Regular high school diploma
GED or alternative credential
Some college, no degree
Associate’s degree (for example: AA, AS)
Bachelor’s degree (for example: BA, BS)
Master’s degree (for example: MA, MS,
MEng, MEd, MSW, MBA)

Less than 1 year of school completed
Nursery school or preschool
Kindergarten
Grade 1
Grade 2
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Grade 9
Grade 10
Grade 11
Grade 12, no diploma
Regular high school diploma
GED or alternative credential
Some college, no degree
Associate’s degree (for example: AA, AS)
Bachelor’s degree (for example: BA, BS)

2022 ACS Content Test – Cognitive Testing
TOPIC – Educational Attainment
Wednesday, February 19, 2020
71. Master’s degree (for example: MA,
MS, MEng, MEd, MSW, MBA)
72. Professional degree beyond a
bachelor’s degree (for example:
MD, DDS, DVM, LLB, JD)
73. Doctorate degree (for example:
PhD, EdD)
74. Vocational or technical license

Professional degree beyond a bachelor’s
degree (for example: MD, DDS, DVM,
LLB, JD)
Doctorate degree (for example: PhD, EdD)

Master’s degree (for example: MA, MS,
MEng, MEd, MSW, MBA)
Professional degree beyond a bachelor’s
degree (for example: MD, DDS, DVM,
LLB, JD)
Doctorate degree (for example: PhD, EdD)

 Do not read “Vocational and technical
license.”

2022 ACS Content Test – Cognitive Testing
TOPIC – Health Insurance
Wednesday, February 19, 2020
Subcommittee Lead:
Sharon Stern, U.S. Census Bureau
Subcommittee Team Members:
Edward Berchick, U.S. Census Bureau
Richard Chard, Social Security Administration
Robin Cohen, National Center for Health Statistics
Ken Finegold, U.S. Department of Health and Human Services, Office of the Assistant Secretary
for Planning and Evaluation
Sarah Heimel, U.S. Census Bureau
Paul Jacobs, Agency for Health Research and Quality
Tamara Lee, National Center for Veterans Analysis and Statistics
Toks Oluwole, Social Security Administration
Jonathan Vespa, U.S. Census Bureau
Jessica Vistnes, Agency for Health Research and Quality
Objective:
The objective is to improve the accuracy of the ACS Health Insurance Coverage question,
especially Medicaid and direct-purchase coverage.
Revisions to the health insurance question would enhance question reliability and validity. Prior
research has found that Medicaid and other means-tested programs are underreported in the ACS
and that direct-purchase coverage is overreported (Boudreaux, Ziegenfuss, Graven, Davern, &
Blewett, 2011; Boudreaux, Call, Turner, & Fried, 2014; Boudreaux, Call, Turner, Fried, &
O'Hara, 2013; Lynch, Kenney, Haley, & Resnick, 2011; O'Hara, 2010). Data are used “to enable
the Department of Health and Human Services (HHS) and other federal agencies to more
accurately distribute resources and better understand state and local health insurance needs”
(U.S. Census Bureau, 2007). The proposed revisions would help to capture changes to the health
insurance landscape that occurred with the passage of the Patient Protection and Affordable Care
Act (United States Congress, 2010). The primary objectives of revising the health insurance
coverage question (Q16) is to do a better job measuring public coverage, to improve accuracy of
direct purchase coverage, and to reduce over-count of single-service insurance plans.
Note: Currently we only have one set of text that we will test in both version 1 and version 2.
Cognitive Testing Research Goals:
•
•
•

How are respondents reporting Medicaid and direct-purchase plans?
Are the additional instructions (“Do NOT include plans that cover only one type of
insurance, such as dental, drug, or vision plans”) clear to respondents and are respondents
including single-service health insurance plans in their responses?
How do people with Medicare Advantage plans classify their health insurance coverage?
1

2022 ACS Content Test – Cognitive Testing
TOPIC – Health Insurance
Wednesday, February 19, 2020
•
•
•
•

How do people with Marketplace coverage classify their coverage?
How do respondents with household members who have coverage through the state
Children’s Health Insurance Program (CHIP) classify the child’s coverage?
For respondents who check more than one option, are they double reporting single coverage
or do they have more than one type?
What types of insurance or health plans do respondents associate with “current or former
employer, union, or professional association”?

Cognitive Testing Recruiting Requirements:
We are requesting that the cognitive testing has a sufficient sample of:
• Persons aged 65 or older
• Persons enrolled in Medicaid
• Persons with household members enrolled in CHIP
• Marketplace enrollees (state or federal marketplace)

2022 ACS Content Test – Cognitive Testing
TOPIC – Health Insurance
Wednesday, February 19, 2020
Paper
CURRENT PRODUCTION VERSION

TEST VERSIONS 1 + 2

2022 ACS Content Test – Cognitive Testing
TOPIC – Health Insurance
Wednesday, February 19, 2020
Computer Assisted Interview Script
CURRENT PRODUCTION VERSION

TEST VERSIONS 1 + 2

I am now going to ask you some questions about [Fill 1: your/’s] health
insurance and health coverage.

I am now going to ask you some questions about your/ health insurance
and health coverage. Do NOT include plans that cover only one type of service, such
as dental, drug or vision plans.

[Fill 2: Are you/Is ] currently covered by health insurance through a current or
former employer or union of [Fill 3: yours or another family member/ or
another family member]?
1. Yes
2. No

Are you/Is currently covered by health insurance through a current or
former employer, union, or professional association of yours or another family
member/<[him/her] or another family member>?
Yes
No

[Fill 1: Are you/Is ] currently covered by health insurance purchased directly
from an insurance company by [Fill 2: you or another family member/ or
another family member]?
1. Yes
2. No
[Fill 1: Are you/Is ] currently covered by Medicare, for people age 65 or older or
people with certain disabilities?

Are you/Is currently covered by Medicare, for people age 65 or older or
people with certain disabilities?
Yes
No
Are you/Is currently covered by Medicaid, the Children’s Health
Insurance Program (CHIP) or any kind of government-assistance plan for those
with low incomes or a disability?
Yes
No

1. Yes
2. No
[Fill 1: Are you/Is ] currently covered by Medicaid, Medical Assistance, or any
kind of government-assistance plan for those with low incomes or a disability?
1. Yes
2.
No

Are you/Is currently covered by health insurance purchased directly from
an insurance company, through a State or Federal Marketplace, HealthCare.gov, or
a similar website by you or another family member/<[him/her] or another family
member>?
Yes
No

2022 ACS Content Test – Cognitive Testing
TOPIC – Health Insurance
Wednesday, February 19, 2020
[Fill 1: Are you/Is ] currently covered by TRICARE or other military health
care?

Are you/Is currently covered by TRICARE or other military health care?
Yes
No

1. Yes
2. No
[Fill 1: Are you/Is ] currently covered through the VA or enrolled for VA health
care?

Are you/Is currently covered through the VA or enrolled for VA health
care?
Yes
No

1. Yes
2. No
[Fill 1: Are you/Is ] currently covered through the Indian Health Service?

Are you/Is currently covered through the Indian Health Service?
Yes
No

1. Yes
2. No
[Fill 1: Are you/Is ] currently covered by any other health insurance or health
coverage plan?

Are you/Is currently covered by any other health insurance or health
coverage plan?
Yes
No

1. Yes
2. No

2022 ACS Content Test – Cognitive Testing
TOPIC – Disability
Wednesday, February 19, 2020
Subcommittee Lead:
Sharon Stern, U.S. Census Bureau
Subcommittee Team Members:
Maribel Aponte, Department of Veterans Affairs
Jenny Berkley, U.S. Census Bureau
Karen Brummond, Equal Employment Opportunity Commission
Fran Chevarley, Agency for Healthcare Research and Quality
Douglas Conway, U.S. Census Bureau
Romella El Kharzazi, Equal Employment Opportunity Commission
Jason Fields, U.S. Census Bureau
Erika Harrell, Bureau of Justice Statistics
David Kashihara, Agency for Healthcare Research and Quality
Lan Liang, Agency for Healthcare Research and Quality
Jennifer Madans, National Center for Health Statistics
Matt Messel, Social Security Administration
Jennifer Ortman, U.S. Census Bureau
Amy Steinweg, U.S. Census Bureau
Hilary Waldron, Social Security Administration
Julie Weeks, National Center for Health Statistics
Objective:
The objective is to improve accuracy of disability measures and increase comparability across
other major surveys.
We want to determine if respondents understand the questions, what is being asked for, and if
they can easily determine what response category they should select. It is important to evaluate
how the proposed questions perform when a person is responding for themselves and when the
person is serving as a proxy reporter. We want to understand how respondents select a response
to the proposed four answer response categories and ensure that respondents understand the
proposed questions without creating undue burden.
Cognitive Testing Research Goals:
•
•

Modified response categories in both versions to be more detailed than current “Yes/No”
response categories: How do respondents distinguish between categories when selecting a
category?
Introductory text in Computer Assisted Instrument (CAI) before Q18: Does having
introductory text, “The next questions ask about difficulties [NAME] may have doing
certain activities.” help transition from the previous set of questions to the disability
section?

2022 ACS Content Test – Cognitive Testing
TOPIC – Disability
Wednesday, February 19, 2020
•
•

•
•

•

Q19a: Do respondents think there is a difference between “stairs” (Version 1) and “steps”
(Version 2)? Does one term help respondents understand the question better than the other?
Q19b: Do adults think this question only applies to people with dementia? Do respondents
focus on the term “remembering,” on the term “concentrating,” or both? How do people with
a learning disability (or people answering for someone who has one) answer this question?
Do respondents associate particular health conditions with difficulty remembering or
concentrating?
Q19c: Do respondents understand the text “washing all over” (Version 2) as being a more
broad (i.e., inclusive) description of bathing? Do respondents find “washing all over” to be
odd or too personal? Do respondents find “bathing” (Version 1) to be odd or too personal?
19d: What types of disabilities/conditions are being identified for children when the response
indicates difficulty with communication? Does the question makes sense for younger
children who are not old enough to talk? Do respondents understand that this question is not
referring to mastering English as a second language (particularly those who do not speak
English very well)?
Q20: In Version 2, the question preamble, “Because of a physical, mental, or emotional
condition,…” is included in the beginning of the question. Does this preamble result in a
better understanding that this question is asking about disability limitations regarding doing
errands? What types of problems/issues do respondents mention when discussing “difficulty
doing errands alone such as visiting a doctor’s office or shopping”?
Do respondents age 50+ interpret the difficulties in this series of questions as part of the
normal aging process? If so, do respondents age 50+ underreport?

Cognitive Testing Recruiting Requirements:
•
•
•
•

Parents/legal guardians of children with disabilities
People with disabilities, especially learning disabilities
People whose first language is not English and who do not speak English very well
People 50 years or older

2022 ACS Content Test – Cognitive Testing
TOPIC – Disability
Wednesday, February 19, 2020
Paper
CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

2022 ACS Content Test – Cognitive Testing
TOPIC – Disability
Wednesday, February 19, 2020

2022 ACS Content Test – Cognitive Testing
TOPIC – Disability
Wednesday, February 19, 2020
CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

2022 ACS Content Test – Cognitive Testing
TOPIC – Disability
Wednesday, February 19, 2020

Computer Assisted Interview Script
CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

For all people…
I am now going to ask some questions about difficulty you The next questions ask about difficulties you/
may have doing certain activities.
may have with ordinary daily activities.
Is this person deaf or does he/she have serious difficulty
hearing?



Yes
No

Is this person blind or does he/she have serious difficulty
seeing even when wearing glasses?



Yes
No

The next questions ask about difficulties you/
may have doing certain activities.

Do you/does have difficulty seeing, even if
wearing glasses?

No difficulty
Some difficulty
A lot of difficulty
Cannot do at all

Do you/does have difficulty hearing, even if using Do you/does have difficulty hearing, even if using
a hearing aid?
a hearing aid?
No difficulty
Some difficulty
A lot of difficulty
Cannot do at all

No difficulty
Some difficulty
A lot of difficulty
Cannot do at all

2022 ACS Content Test – Cognitive Testing
TOPIC – Disability
Wednesday, February 19, 2020
For people 5 and over…
Because of a physical, mental, or emotional condition, does
this person have serious difficulty concentrating,
remembering, or making decisions?
-+
 Yes
 No
Does this person have serious difficulty walking or
climbing stairs?



Yes
No

Does this person have difficulty dressing or bathing?



Yes
No

Do you/does have difficulty walking or climbing
stairs?

Do you/does have difficulty walking or climbing
steps?

No difficulty
Some difficulty
A lot of difficulty
Cannot do at all

Do you/does have difficulty remembering or
concentrating?

No difficulty
Some difficulty
A lot of difficulty
Cannot do at all

Do you/does have difficulty bathing or dressing?

Do you/does have difficulty washing all over or
dressing?

No difficulty
Some difficulty
A lot of difficulty
Cannot do at all

Using your/ usual language, do you/does
have difficulty communicating, for example understanding
or being understood?
No difficulty
Some difficulty
A lot of difficulty
Cannot do at all

2022 ACS Content Test – Cognitive Testing
TOPIC – Disability
Wednesday, February 19, 2020
For people 15 and over…
Because of a physical, mental, or emotional condition,
does [NAME] have difficulty doing errands alone
such as visiting a doctor’s office or shopping?



Yes
No

Do you/does have difficulty doing errands
alone such as visiting a doctor’s office or shopping?
No difficulty
Some difficulty
A lot of difficulty
Cannot do at all

Because of a physical, mental, or emotional condition,
do you/does have difficulty doing errands
alone such as visiting a doctor’s office or shopping?
No difficulty
Some difficulty
A lot of difficulty
Cannot do at all

2022 ACS Content Test – Cognitive Testing
TOPIC – Means of Transportation to Work
Wednesday, February 19, 2020
Subcommittee Lead:
Brian McKenzie, U.S. Census Bureau
Subcommittee Team Members:
Dorothy Barth, U.S. Census Bureau
Charlynn Burd, U.S. Census Bureau
Michael Burrows, U.S. Census Bureau
Ken Cervenka, Bureau of Transportation Statistics
Slagan Locklear, U.S. Census Bureau
Jasmy Methipara, Bureau of Transportation Statistics
Clara Reschovsky, Bureau of Transportation Statistics
Objective:
The prevalence of and usage of ride-hailing services has increased considerably in recent years,
and these services are expected to become more prevalent in the coming years. Our objective is
to capture these services in our ACS means of transportation question in a direct and
unambiguous way. The current ACS travel mode question includes several categories (“taxicab,”
“carpooling,” and “other”) that may potentially accommodate this mode of transportation, but
none of these categories explicitly reference ride hailing. We would like to concentrate ridehailing responses into one category, combining it with traditional taxi services.
We also would like to take the opportunity to probe respondents and gain information that could
be useful in helping improve the transportation to work question in the future. The question
currently only allows for one response; however, respondents may use multiple modes and we
may want to capture them. Additionally, in a rapidly changing transportation landscape, we may
be missing some modes that we could consider adding (for example, electric scooters).
Cognitive Testing Research Goals:
•
•
•
•

For all respondents, is the meaning of the “taxi or ride-hailing services” category clear? If
not, what descriptive words would have made the meaning clearer? Are there descriptive
words missing?
Among those who chose ride-hailing as their primary means of transportation to work, what
is their second most common mode of work travel?
Among those who did not choose ride-hailing as their primary means of transportation to
work, do they ever use ride-hailing services to travel to or from work? How often? In what
context?
Do respondents have any travel modes they use to get to work that are not represented in the
ACS travel mode question? What are they?

2022 ACS Content Test – Cognitive Testing
TOPIC – Means of Transportation to Work
Wednesday, February 19, 2020
Cognitive Testing Recruiting Requirements:
Our recruiting requirement goal is to target workers who are likely to either use a ride-hailing
service or consider using one. Respondents should satisfy the following criteria:
•
•

Individuals who use Lyft or Uber to get to work
Individuals who rely on alternative means of transportation to get to work, but do not use
public transportation, such as rides from other individuals (i.e., people who do not typically
work at home, but use a carpool, vanpool, slug line, etc.)

2022 ACS Content Test – Cognitive Testing
TOPIC – Means of Transportation to Work
Wednesday, February 19, 2020
Paper
CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

2022 ACS Content Test – Cognitive Testing
TOPIC – Means of Transportation to Work
Wednesday, February 19, 2020
Computer Assisted Interview Script
CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

NOTES OR COMMENTS
1.

"{Fill 2: Using this list,} LAST WEEK, how did
{Fill 3: /you} USUALLY get to work?

Using this list, LAST WEEK, how did
you/ USUALLY get to work?

(If {Fill 4: he/she/)/you} usually used more
than one method of transportation during the trip,
report the one used for most of the distance.)

(If ^you/^ usually used more than
one method of transportation during the trip,
report the one used for most of the distance.)

(If ^you/^ usually used more than one
method of transportation during the trip, report
the one used for most of the distance.)

(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)

Car, truck, or van
Bus
Subway or elevated rail
Long-distance train or commuter rail
Light rail, streetcar, or trolley
Ferryboat
Taxi or ride-hailing services
Motorcycle
Bicycle
Walked
Worked from home
Other Method

Car, truck, or van
Bus
Subway or elevated rail
Long-distance train or commuter rail
Light rail, streetcar, or trolley
Ferryboat
Ride-hailing services (including taxi)
Motorcycle
Bicycle
Walked
Worked from home
Other Method

Car, truck, or van
Bus
Subway or elevated rail
Long-distance train or commuter rail
Light rail, streetcar, or trolley
Ferryboat
Taxicab
Motorcycle
Bicycle
Walked
Worked from home
Other Method

Flashcards will be used.

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
Subcommittee Lead:
Jonathan Rothbaum, U.S. Census Bureau
Subcommittee Team Members:
Maribel Aponte, National Center for Veterans Analysis and Statistics
Dorothy Barth, U.S. Census Bureau
Lynn Fisher, Social Security Administration
Gloria Guzman, U.S. Census Bureau
Tamara Lee, National Center for Veterans Analysis and Statistics
Carey Patrick, Bureau of Labor Statistics
Kirby Posey, U.S. Census Bureau
Mark Prell, Economic Research Service
Dave Shoffner, Social Security Administration
Adam Smith, U.S. Census Bureau
Objective:
The Census Bureau has been investigating the use of administrative records to reduce respondent
burden and improve data quality in the American Community Survey (ACS) (see
https://www.census.gov/programs-surveys/acs/operations-and-administration/agility-inaction.html). As a part of using administrative records, potential changes to the ACS income
section have been evaluated with cognitive testing. We believe this process should continue, with
additional changes evaluated in the 2022 ACS Content Test.
What we are testing in this study is a continuation of prior testing that was done with Westat (see
the report at: https://www.census.gov/content/dam/Census/library/workingpapers/2017/acs/2017_Westat_01.pdf). We will build off the findings from the Westat study, and
test an alternative form that incorporates Westat’s recommendations and aims to address their
concerns. To align better with administrative data, we will also test changing the reference period
from the last 12 months to the previous calendar year.
In addition to changes to the income question, we also would like to test changing the reference
period for the SNAP benefit question to last year (from “last 12 months”). Because the questions
near the SNAP question have a different reference period than last year, we propose testing the
question in two different places – its current location (version 1) and at the end of the housing
section (version 2).
Finally, we want to test some changes to the questions in the weeks worked series. The version
we will test, like income, builds off findings from the Westat test, and includes the same change
to the reference period.

Cognitive Testing Research Goals:
1

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
Income
• Do the respondents report income for the appropriate reference period (prior year)?
• Are the respondents reporting income accurately, especially keeping in mind the following
changes being made to question or instructional wording:
− Total Income Amount - Adding “include all sources”
 Does the respondent report “all sources” or do they leave out some?
− Self-Employment Income - Adding “including work paid for in cash”
 Does the respondent report all self-employment income (including side jobs
that they may not report as income for tax purposes)?
− Net Rental Income – splitting up category as its own question (paper)
 Does splitting up the categories make it easier for the respondent to recall the
amounts and report accurately?
 Does having Net Rental Income as its own category (on Paper) make
respondents who are reading quickly misreport their monthly rent to a
landlord (instead of rental income)?
 If a respondent indicates that they did not make any rental income, we want to
learn more.
 How would respondents who did make rental income answer if they had lost
money or broke even on a rental property?
− Public Assistance Income - new wording and instructions
 Does the new wording (and additional instructions) help the respondent to
report the amounts and the type of income that we intend to be reported with
this question, or do they include or exclude certain types?
Weeks Worked
• Do the changes to the WEEKS WORKED series (in addition to the year change) obtain the
appropriate information for that year?
• Does question 39 set up the universe for the WEEKS WORKED questions properly? (Or
conversely, do respondents get confused by the additional question (39b) and SKIP patterns
to 39?)
• Adding “for at least one day” to 41b is supposed to let the respondent know that we consider
a week being worked even if they just worked one day in that week. Does the respondent
seem to understand this concept?
SNAP
• Which placement of the SNAP question was better for respondents (current location in
version 1, or end of housing section in version 2)?
• Is one better than the other at getting respondents to understand the reference period and how
it differs from other questions?
• Does the placement of the question at the end of the housing section result in it getting
overlooked?

Cognitive Testing Recruiting Requirements:
2

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
•
•
•
•
•

Individuals who were irregular workers (such as partial year workers, irregular schedules,
multiple jobs) in the prior year
Individuals with regular work schedules in the prior year
Individuals who did not work in the prior year
Individuals who earned retirement income, self-employment income, net rental income, or a
commission/bonus/tips in the prior year
Individuals who received SNAP benefits or public assistance in the prior year

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
Paper  Income
CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

NOTES OR COMMENTS
Note: The reference period for SNAP Test
V1 + V2 is “2019” in the final paper
questionnaires, not “2018.”
Re: SNAP questions:
1. Test V1 = Keep SNAP question
where it currently is in the survey. It
corresponds to question 18 in the
V1 of the paper questionnaire.
2. Test V2 = Move SNAP question to
end of housing section to reduce
impact of change in reference
period for the respondent. It
corresponds to question 27 in the
V2 of the paper questionnaire.

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020

Note: The reference periods for INCOME
Test V1 + V2 is “2019” in the final paper
questionnaires, not “2018.”

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
Computer Assisted Interview Script  Income
CURRENT PRODUCTION VERSION
IN THE PAST 12 MONTHS, did you or
any member of this household receive
benefits from the Food Stamp Program
or SNAP (the Supplemental Nutrition
Assistance Program)? Do NOT include
WIC, the School Lunch Program, or
assistance from food banks.

[Instruction to be read]
The next few questions are about
income DURING THE PAST 12
MONTHS, that is from to …

TEST VERSION 1

TEST VERSION 2

NOTES OR COMMENTS

IN THE PAST 12 MONTHS, did you or
any member of this household receive
benefits from the Food Stamp Program
or SNAP, the Supplemental Nutrition
Assistance Program? Do NOT include
WIC, the School Lunch Program, or
assistance from food banks.

IN THE PAST 12 MONTHS, did you or
any member of this household receive
benefits from the Food Stamp Program or
SNAP, the Supplemental Nutrition
Assistance Program? Do NOT include
WIC, the School Lunch Program, or
assistance from food banks.

Note: The reference period for SNAP and
INCOME Test V1 + V2 is “2019” in the final
CAI scripts, not “2018.”

Yes
No
 Complete Questions A – M for each
person before moving onto the next
person.
The next few question are about all types
of income, taxable and non-taxable,
received in 2018 (from January 1, 2018
to December 31, 2018).

1. Did [/you] receive any wages
or salary?
<1> Yes
<2> No

If yes:

For income received jointly, report the
appropriate share for each person – or, if
that’s not possible, report the whole
amount for only one person and do not
report the income for the other person.

Re: SNAP questions:
1. Test V1 = Keep SNAP question where it

currently is in the survey. It corresponds
to question 18 in the V1 of the paper
questionnaire.
2. Test V2 = Move SNAP question to end of
 Complete Questions A – M for each person
housing section to reduce impact of
before moving onto the next person.
change in reference period for the
respondent. It corresponds to question 27
in the V2 of the paper questionnaire.
The next few question are about all types of
income, taxable and non-taxable, received
in 2018 (from January 1, 2018 to December
31, 2018).
Yes
No

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
How much did [/you] receive in
wages and salary from all jobs before
taxes and other deductions?
2. Did you receive any tips, bonuses or
commissions DURING THE PAST 12
MONTHS?
<1> Yes
<2> No

If yes:
How much did [/you] receive in
tips, bonuses, or commissions from all
jobs before taxes and other deductions?

3. Did [/you] receive any selfemployment income DURING THE PAST
12 MONTHS? Report income from own
businesses (farm or non-farm) including
proprietorships and partnerships.
<1> Yes
<2> No

If yes:
What was the amount? Report net income
after operating expenses. Include earnings
as a tenant farmer or sharecropper.

A. Did you/ receive any wages
or salary in 2018?

A. Did you/ receive any wages or
salary in 2018?

If yes: How much did you/
receive in wages and salary from all jobs
before deductions for taxes, bonds, dues
or other items?

If yes: How much did you receive in wages
and salary from all jobs before deductions
for taxes, bonds, dues or other items?

B. Did you/ receive any
commissions, bonuses, or tips in 2018?
If yes: How much did you/
receive in tips, bonuses, or commissions
from all jobs before deductions for taxes,
bonds, dues, or other items?

C. Did you/ receive any selfemployment income in 2018? Report
income from own businesses (farm or
non-farm) including proprietorships and
partnerships.

C. Did you/ regularly receive any
self-employment income in 2018, including
work paid for in cash? Report income from
own businesses (farm or non-farm)
including proprietorships and
partnerships.

If yes: What was the amount? Report
NET income after business expenses.

If yes: What was the amount? Report NET
income after business expenses.

4. Did [/you] receive any interest D. Did you/ receive any interest D. Did you/ receive any interest
or dividends in 2018? Report even small or dividends in 2018? Report even small
or dividends DURING THE PAST 12
amounts credited to an account.
amounts credited to an account.
MONTHS? Report even small amounts
credited to an account.
<1> Yes
<2> No

If yes: What was the amount?

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
If yes:
What was the amount?

5. Did [/you] receive any net
rental income DURING THE PAST 12
MONTHS? Net rental income is the

total amount after expenses.
<1> Yes
<2> No

E. Did you/ receive any net
E. Did you/ receive any net rental
rental income in 2018? Net rental income income in 2018? Net rental income is the
is the total amount after expenses.
total amount after expenses.
If yes: What was the net amount?

If yes: What was the net amount?

What was the net amount?

F. Did you/ receive any royalty F. Did you/ receive any royalty
income or income from estates and trusts income or income from estates and trusts
in 2018?
in 2018?

6. Did [/you] receive any

If yes: What was the amount?

G. Did you/ receive any Social
Security or Railroad Retirement benefits
in 2018?

G. Did you/ receive any Social
Security or Railroad Retirement benefits in
2018?

If yes: What was the amount?

H. Did you/ receive any
Supplemental Security Income (SSI)
payments in 2018?

If yes: What was the amount?

I. Did you/ receive public
assistance or public welfare payments
from the state or local welfare office in
2018?

I. Did you/ receive any financial
assistance from the state or local welfare
office in 2018?

If yes:

royalty income or income from estates and
trusts DURING THE PAST 12 MONTHS?
<1> Yes
<2> No

If yes:
What was the amount?

7. Did [/you] receive any
Social Security or Railroad Retirement
benefits DURING THE PAST 12
MONTHS?
<1> Yes
<2> No

If yes:
What was the amount?

8. Did [/you] receive any
Supplemental Security Income (SSI)
payments DURING THE PAST 12
MONTHS?

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
<1> Yes
<2> No

If yes:
What was the amount?
9. Did [/you] receive public

assistance or public welfare payments
from the state or local welfare office
DURING THE PAST 12 MONTHS?
<1> Yes
<2> No

If yes:
What was the amount? (Do not include
SSI.)

10. Did [/you] receive any
survivor or disability income DURING THE
PAST 12 MONTHS?
<1> Yes
<2> No

If yes:
What was the amount? (Do not include
Social Security.)
11. Did [/you] receive a
pension or any retirement income from a
previous employer or union, or any regular
withdrawals or distributions from
retirement accounts such as 401(k),
403(b), IRA, Roth IRA, or other accounts
designed specifically for retirement
DURING THE PAST 12 MONTHS?
<1> Yes
<2> No

If yes: What was the amount? Do not
include SSI.
J. Did you/ receive any
survivor or disability income in 2018?
If yes: What was the amount? Do not
include Social Security.
K. Did you/ receive a pension
or any retirement income from a
previous employer or union, or any
regular withdrawals or distributions
from retirement accounts such as 401(k),
403(b), IRA, Roth IRA, or other
accounts designed specifically for
retirement in 2018?
If yes: What was the amount? Do not
include Social Security.
L. Did you/ receive income on a
REGULAR basis from any other sources
such as Department of Veterans Affairs
(VA) payments, unemployment
compensation, child support, or alimony
in 2018?

If yes: What was the amount? Do NOT
include non-cash benefits, such as energy
or housing assistance, The Food Stamp
Program, or SNAP.
J. Did you/ receive any survivor
or disability income in 2018?
If yes: What was the amount? Do not
include Social Security.
K. Did you/ receive a pension or
any retirement income from a previous
employer or union, or any regular
withdrawals or distributions from
retirement accounts such as 401(k), 403(b),
IRA, Roth IRA, or other accounts designed
specifically for retirement in 2018?
If yes: What was the amount? Do not
include Social Security.
L. Did you/ receive income on a
REGULAR basis from any other sources
such as Department of Veterans Affairs
(VA) payments, unemployment
compensation, child support, or alimony in
2018?

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
If yes:
What was the amount? (Do not include
Social Security.)

If yes: What was the amount from all
sources? Do not include lump sum
payments such as money from an
inheritance or sale of a home.

12. Did [/you] receive income
on a REGULAR basis from any other
sources such as Department of Veterans
Affairs (VA) payments, unemployment
compensation, child support or alimony
DURING THE PAST 12 MONTHS?
<1> Yes
<2> No

M. What is the best estimate of the
TOTAL income you/ received
from all sources in 2018?

If yes: What was the amount from all
sources? Do not include lump sum
payments such as money from an
inheritance or sale of a home.
M. What is your best estimate of the
TOTAL income you/ received
from all sources in 2018?

If yes:
What was the amount from all sources?
(Do not include lump sum payments such
as money from an inheritance or sale of a
home.)
13. What is your best estimate of the
TOTAL income [FILL1: 's/your]
received from all sources DURING THE
PAST 12 MONTHS?

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020

Paper  Weeks Worked

CURRENT PRODUCTION VERSION

TEST VERSIONS 1 + 2

NOTES OR COMMENTS
Note: The reference period for WEEKS WORKED Test
V1 + V2 is “2019” in the final paper questionnaires, not
“2018.”

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020
Computer Assisted Interview Script  Weeks Worked
CURRENT PRODUCTION VERSION
When did {Fill 1: / you} last work, even for a
few days?
___ Within the past 12 months [goto WKWX]
___ Between 1 to 5 years ago [goto COWCP]
___ Over 5 years ago or never worked -> [goto INTRX]
During the PAST 12 MONTHS or 52 weeks, did {Fill
1: / you} work EVERY week? Count paid
vacation, paid sick leave, and military service as work.
____ Yes [goto WKH]
____ No [goto WKWN]

TEST VERSIONS 1 + 2
A. When did you/ last work, even for a few
days?
Within the past 12 months
1 to 5 years ago
Over 5 years ago or never worked
 If response is “Over 5 years ago or never worked,” SKIP
to INCOME.
 Otherwise, go to B.
B. In 2018, did you/ work at a job or business
at any time, even for a few days?

During the PAST 12 MONTHS or 52 weeks,
how many WEEKS did {Fill 1: / you}
work? Include paid time off and include weeks
when {Fill 1: / you} only worked for a
few hours.

Yes
No
 If response is “No,” SKIP C – E.
 Otherwise, go to C.

During the weeks worked IN THE PAST 12
MONTHS, how many hours did {Fill 1:
/ you} usually work each week?

C. During the weeks that you/ WORKED in
2018, how many hours did you/ usually work
each week?

NOTES OR COMMENTS
Note: The reference period for Weeks Worked Test V1 +
V2 is “2019” in the final CAI scripts, not “2018.”

2022 ACS Content Test – Cognitive Testing
TOPIC – Income
Wednesday, February 19, 2020

D. In 2018, did you/ work EVERY week?
Include all jobs and count paid vacation, paid sick
leave, and military service as work.

Yes
No
 If response is “Yes,” SKIP E.
 Otherwise, go to E.
E. Of the 52 weeks in 2018, how many WEEKS did
you/ work for at least one day? Include all
jobs, paid time off, and weeks when you/ only
worked for a few hours.
 If person worked in the past 5 years, go to TYPE OF
EMPLOYMENT.
 Otherwise, SKIP to INCOME.

2022 ACS Content Test – Cognitive Testing
TOPIC – Sewer
Wednesday, February 19, 2020
Subcommittee Lead:
Evan Brassell, U.S. Census Bureau
Subcommittee Team Members:
Shawn Bucholtz, Department of Housing and Urban Development
Molly Cromwell, U.S. Census Bureau
Alexander Daily, Energy Information Administration
Zach Lowenstein, Environmental Protection Agency
Gaida Mahgoub, Environmental Protection Agency
Broderick Oliver, U.S. Census Bureau
Mark Prell, Economic Research Service
Barry Ramsey, U.S. Department of Agriculture
Max Zarate-Bermudez, Centers for Disease Control and Prevention
Objective:
Consistent data on the decentralized wastewater infrastructure status in rural and other
communities is needed to protect public health, water quality, and to understand and meet the
country's growing infrastructure needs.
Regular data updates are required to address changes in housing development and to support
regular planning and funding cycles at the local, state, and national level. An annual survey such
as the ACS would accurately capture new construction rates of septic systems.
Existing data sources do not provide current, reliable, location-specific data on the use of septic
systems. The ACS is the only available survey that can provide this data in a timely, consistent,
standardized, and a location-specific manner.
When testing the questions, recruiting from areas known to not have public sewer, such as small
towns, may give insight into reporting error. Recruiting in dense urban areas, not likely to have
septic systems, would provide similar insight.
Cognitive Testing Research Goals:
•
•
•

Do people know the difference between a septic system or cesspool versus one connected to
public sewer?
Do people have more than one type of disposal system? If so, how do they decide to answer?
What type of system do those who choose “Other type of sewage disposal” (version 1) or
“No, use other type of system” (version 2) have?

Cognitive Testing Recruiting Requirements:
•
•

Individuals living in housing units with septic tanks only
Individuals with homes connected to public sewer
1

2022 ACS Content Test – Cognitive Testing
TOPIC – Sewer
Wednesday, February 19, 2020
Paper

TEST VERSION 1

TEST VERSION 2

NOTES OR COMMENTS
1. The word “cesspool” will not be
translated in Spanish version (Rounds 2
and 3).

2022 ACS Content Test – Cognitive Testing
TOPIC – Sewer
Wednesday, February 19, 2020
Computer Assisted Interview Script
TEST VERSION 1
What is the MAIN type of sewage
disposal for this ?
Public sewer
Septic system or cesspool, or
Another type of sewage disposal

TEST VERSION 2
A. Is this
connected to a public sewer?
Yes
No
 If response is “Yes,” SKIP B.
 Otherwise, go to B.
B. Is it connected to a septic system or
cesspool?
Yes
No

NOTES OR COMMENTS
1. The word “cesspool” will not be translated
in Spanish version (Rounds 2 and 3).

2022 ACS Content Test – Cognitive Testing
TOPIC – Electric Vehicles
Wednesday, February 19, 2020
Subcommittee Lead:
Evan Brassell, U. S. Census Bureau
Subcommittee Team Members:
Molly Cromwell, U.S. Census Bureau
Carolyn Hronis, Energy Information Administration
Kristin Kerns, U.S. Census Bureau
Larry Locklear, U.S. Census Bureau
Lindsay Longsine, U.S. Census Bureau
Danni Mayclin, Energy Information Administration
Brian McKenzie, U.S. Census Bureau
Clara Reschovsky, Bureau of Transportation Statistics
Objective:
The U.S. Energy Information Administration (EIA) is singularly tasked under PL93-275 and
PL95-91 with carrying out independent analysis of energy supply, demand, and technology
relevant to the adequacy of energy resources to meet demands in the near and long term for the
Nation’s social and economic needs. Energy technology, sources, and uses are projected to
change as much in the next 10-20 years as they have in the last 100.
A diverging and growing number of state and local governments’ energy and tax policies, and
energy sector and other incentives to meet renewable energy and other goals, influence the
quality of EIA’s energy projections. EIA can observe the result of policies and programs in the
aggregate but lacks sufficient information to understand drivers and impact from consuming
units, e.g., households, buildings, and industry.
The technology change and capital necessary to modernize the interconnected U.S. electric
power grid is poorly observed at the local level through the energy consumer. In particular, the
differential adoption of energy producing technologies such as rooftop solar (i.e., photovoltaic
generating capacity, PV) as well as energy consuming technologies such as electric or hybridelectric plug-in vehicles (EVs) creates new, more variable demands and more potential (e.g.,
mobile storage provided by battery-enabled vehicles) for energy infrastructure for and in U.S.
households.
The American Community Survey would be the only data source at the level of granularity
necessary for EIA to adequately project future energy sources, infrastructure, and consumer
needs during this era of rapid change.

2022 ACS Content Test – Cognitive Testing
TOPIC – Electric Vehicles
Wednesday, February 19, 2020
Cognitive Testing Research Goals:
•
•
•
•

Are respondents reporting hybrid vehicles that do not require connecting to an electrical
source for charging?
Are respondents going to consider every household member that might own an eligible
vehicle?
Are respondents reporting vehicles that they are currently leasing?
(Version1) What other type of electric vehicles do people have when they respond “Yes” to
14b?

Cognitive Testing Recruiting Requirements:
•
•
•

Individuals who have purchased vehicles in the last 10 years
Owners/leasers of electric vehicles that require plugging in to an electrical source for
charging
Owners/leasers of any other type of electric vehicles (including hybrid) that do not
require plugging in to an electric source for charging

2022 ACS Content Test – Cognitive Testing
TOPIC – Electric Vehicles
Wednesday, February 19, 2020
Paper

TEST VERSION 1

TEST VERSION 2

2022 ACS Content Test – Cognitive Testing
TOPIC – Electric Vehicles
Wednesday, February 19, 2020
Computer Assisted Interview Script

TEST VERSION 1

TEST VERSION 2

Are any of the following types of electric vehicles
kept at home for use by members of this
household?

At this , do you or any
member of this household own or lease any of the
following types of electric vehicles?

 Read one at a time:

A plug-in electric vehicle?

Yes
No

Another type of electric vehicle?

A hybrid electric vehicle?

Yes
No

2022 ACS Content Test – Cognitive Testing
TOPIC – Home Heating Fuel
Wednesday, February 19, 2020
Subcommittee Lead:
Evan Brassell, U.S. Census Bureau
Subcommittee Team Members:
William Chapin, U.S. Census Bureau
Cha-Chi Fan, Federal Communications Commission
Carolyn Hronis, Energy Information Administration
Lindsay Longsine, U.S. Census Bureau
Danni Mayclin, Energy Information Administration
Objective:
We are trying to make responding easier by amending two of the response options in the “gas”
categories. In the first response option, we change “gas” to “natural gas” since the term natural
gas is commonly used. In the second response option, we add “butane” to the list of other gas
examples. We think this will help respondents be more accurate in listing their main heating fuel
type for gas.
Cognitive Testing Research Goals:
•
•
•
•

Do respondents understand what “natural gas” (response category 1) is as opposed to “gas”
(response category 2)?
Do the words “butane” or “propane” help respondents find the correct heating fuel category?
(Version 1) What is the impact of removing “bottled or tank” from the second response
option?
Are respondents only reporting the fuel they use the “MOST”?

Cognitive Testing Recruiting Requirements:
•
•
•

People living in occupied housing units using natural gas as fuel MOST used
People living in occupied housing units using butane/propane as fuel MOST used
People who use some other type of heating fuel

2022 ACS Content Test – Cognitive Testing
TOPIC – Home Heating Fuel
Wednesday, February 19, 2020
Paper
CURRENT VERSION

TEST VERSION 1

TEST VERSION 2

2022 ACS Content Test – Cognitive Testing
TOPIC – Home Heating Fuel
Wednesday, February 19, 2020
Computer Assisted Interview Script
CURRENT PRODUCTION VERSION

TEST VERSION 1

TEST VERSION 2

HFLA:
To heat this [FILL BUILDING TYPE] which fuel
do you use MOST—
Gas, electricity, fuel oil or kerosene, coal or coke,
wood, solar energy or some other fuel?
<1> Gas (go to HFLB)
<2> Electricity
<3> Fuel oil or kerosene
<4> Coal or coke
<5> Wood
<6> Solar energy
<7> Some other fuel
<8> No fuel used

HFLA. To heat this ,
which fuel do you use MOST—

Gas,
Electricity,
Fuel oil or kerosene,
Coal or coke,
Wood,
Solar energy,
or Some other fuel?

 If response to HFLA is “Gas,” go to HFLB.
 Otherwise, SKIP to Question HFLC.

 If response to HFLA is “Gas,” go to HFLB.
 Otherwise, SKIP HFLB and HFLC.

HFLB:

HFLB. Is it natural gas from underground pipes
serving the neighborhood?

HFLB. Is it natural gas used from underground
pipes serving the neighborhood?
Yes
No

Is the gas used from underground pipes serving the
neighborhood?
<1> Yes
<2> No (go to HFLC)
HFLC:
Is it bottled, tank, or LP gas?
<1> Yes
<2> No

Yes
No
 If response to HFLB is “Yes,” SKIP HFLC.
 Otherwise, go to HFLC.

 If response to HFLB is “Yes,” SKIP HFLC.
 Otherwise, go to HFLC.

HFLC. Is it a gas such as propane or butane?

HFLC. Is it bottled or tank gas, such as propane
or butane?

Yes
No

2022 ACS Content Test – Cognitive Testing
TOPIC – Solar Panels
Wednesday, February 19, 2020
Subcommittee Lead:
Evan Brassell, U.S. Census Bureau
Subcommittee Team Members:
Chip Berry, Energy Information Administration
William Chapin, U.S. Census Bureau
Jeremy Engelberg, U.S. Census Bureau
Carolyn Hronis, Energy Information Administration
Michael Risley, U.S. Census Bureau
Objective:
Renewable energy as a source for home heating fuel is a growing industry. Currently the only
way to track it is through limited data on solar panel sales. By asking this question, we will be
able to obtain data for operational solar panels on a housing unit level across the country. This
information will help the Energy Information Administration match energy consumption to
energy production across the United States.
The technology change and capital necessary to modernize the U.S. electrical power grid is
poorly observed at the local level and through the energy consumer. The differential adaptation
of energy producing technologies such as rooftop solar (i.e. photovoltaic generating capacity,
PV) creates new, more variable demands and new potential for energy infrastructure for and in
U.S. households.
Cognitive Testing Research Goals:
•
•

How do people understand the terms, “solar panel” (version 1) and “solar power” (version
2)? Do they view them as meaning the same thing or something different? How do they
decide if they have solar power/panels?
Do people understand the term “photovoltaic”?

Cognitive Testing Recruiting Requirements:
•
•

Individuals in housing units with solar/photovoltaic panels
Individuals in single family home without solar panels

2022 ACS Content Test – Cognitive Testing
TOPIC – Solar Panels
Wednesday, February 19, 2020
Paper
TEST VERSION 1

TEST VERSION 2

2022 ACS Content Test – Cognitive Testing
TOPIC – Solar Panels
Wednesday, February 19, 2020
Computer Assisted Interview Script
TEST VERSION 1

TEST VERSION 2

Does this use
solar panels that generate electricity?

Does this use
solar power or photovoltaic panels that
generate electricity?

Yes
No

File Type	application/pdf
Author	Beverly Marie Pratt (CENSUS/ACSO FED)
File Modified	2020-02-13
File Created	2020-02-13