2019 ACS Due Dates Test NSC

ACS Methods Panel 2019 Due Dates Test NSC.pdf

American Community Survey Methods Panel Tests

2019 ACS Due Dates Test NSC

OMB: 0607-0936

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0607-0936 can be found here:

Document [pdf]

Download: pdf | pdf

OMB Nonsubstantive Change Request
Department: Commerce
Agency: U.S. Census Bureau
Title: American Community Survey Methods Panel Tests, 2019 Due Dates Test
OMB Control Number: 0607-0936
Expiration Date: 08/31/2021
Request: The U.S. Census Bureau plans to conduct additional research under the American
Community Survey (ACS) Methods Panel Tests clearance to test the use of due dates in six
experimental treatments for the American Community Survey. The due date is designed to
create a sense of urgency, communicate the importance of the survey, and provide a timeframe
for response (which may be beneficial, for example, to those with a busy schedule). The
purpose of the test is to determine if including a due date increases self-response rates. The
Census Bureau proposes to conduct a field test as part of production ACS, in either the
September or October 2019 panel. This will require a change in the mail materials for a subset
of the sampled addresses. No additional burden is expected.
Background: Research has shown that when a request has a sense of urgency, people are more
likely to comply with the request (Kotter, 2008; Gunelius, 2009). One way of creating a sense of
urgency is through deadline or due date messaging. In a recent mail-package focus group
conducted for the Census Bureau by Reingold, Inc. (2014), participants indicated that “a stated
deadline or due date would be a strong motivator for them to respond in a timely fashion.”
A due date may have an additional benefit. Research suggests that a due date can reduce
respondent burden, if it aligns with the recipient’s mail prioritization process (Dillman, Smyth, &
Christian, 2014).
Prior Census Bureau research has tested the use of deadline messaging on mail materials for
the decennial census. As to the potential effect of increasing self-response, the results of
experiments have been varied.
A 2003 National Census Test study (Bouffard, J., Brady, S., & Stapleton, C., 2004) found that a
due date on the outgoing envelope on the initial questionnaire mailing had no effect on
response rates, but did increase the speed of the mail response.
A 2006 Decennial Census study (Martin, E., 2009) found that giving people a deadline and a
shorter interval (by one week) to complete the census form led to a higher rate of mail
response. The researchers believed that the effect of a compressed mailing schedule may be
important for the deadline date to be effective, as it creates a sense of urgency. A follow-up
study in 2010 (Stokes, S., Reiser, C., Bentley, M., & Meier, A., 2011) found that the compressed
schedule, alone and in combination with deadline message treatments, did not differ on the
overall national-level mail return rates compared to control.
The purpose of the “2019 American Community Survey Due Dates Test” is to determine the
effect on self-response of using due date messaging in the ACS mail contact materials.

Timeline: The test is being planned as part of the ACS September 2019 or October 2019 panel,
adhering to the same data collection protocols as production ACS.
Method: The ACS sends up to five mailings to sampled addresses to obtain a self-response. At a
minimum all sampled addresses receive the first two mailings. Nonrespondents after the
second mailing receive the third and fourth mailing. Nonrespondents after the fourth mailing
receive the fifth mailing. The mailings are: 1) an initial mailing package, 2) a reminder letter, 3) a
paper questionnaire package, 4) a reminder postcard, and 5) a final reminder. None of these
mailings contain an explicit deadline or due date. They do however contain implied due date
messages, such as “…please complete the survey online as soon as possible” and “If you have
not already responded, please do so now.” This test will assess the effect of an explicit due date
message in the fifth mailing on self-response.
This test will include six experimental treatments, including one control treatment. Sampled
addresses that are not part of the test will receive production ACS mail materials. Unlike other
ACS tests, however, those cases will not serve as the control and will not be used in the
analysis.
In January 2020, the Census Bureau is proposing to update the production ACS mail materials
(as a result of the 2018 Mail Materials Test). Treatments 1 through 6 will use these updated
materials in the first four mailings (see Attachment A for facsimiles of these mail materials for
mailings one through four). However, the content of the fifth mailing will differ per treatment.
Treatment 1 will serve as the control treatment for this test. The content of the fifth mailing will
vary depending on the experimental treatment, as described below.
Three elements of the fifth mailing will change in the experimental treatments. In Treatment 1,
the control treatment, these elements will not include a due date. In Treatments 2 through 6,
these elements will contain due date messaging.
• A call-out box in the letter:

•
•

Text in the letter that reads:
“If you do not respond promptly, a Census Bureau interviewer may contact you
with a personal visit to complete the survey.”
A box on the envelope:

Treatments 2 through 6 will incorporate due date messaging in these elements. All of these
treatments will include a due date in the call-out box in the letter. Treatment 2 isolates the
effect of adding a due date to the call-out box. Treatments 3, 4, and 5 add additional
information in the text of the letter about the due date. Treatment 6 tests including a due date
on the outside of the envelope.
•

Experimental call-out box - Treatments 2 through 6 will use a modified call-out box that
adds the due date:

•

Experimental Text in the Letter - A paragraph after the call-out box will be added in
Treatments 3, 4, and 5 using different variations of experimental text (note that the
entire paragraph is new to the letter; the bolded text is bold in the mail materials):
An “add” message (in Treatment 3):
“Because your response is critically important to your local community and to
your country, a Census Bureau interviewer may come to your home to complete
the survey in person. If you do not respond by October 26, 2019, we will add
you to our schedule for a visit.”
A “neutral” message (in Treatment 4):
“Your response is critically important to your local community and to your
country. Respond by October 26, 2019, or a Census Bureau interviewer may
come to your home to complete the survey in person.”
A “remove” message (in Treatment 5):
“Because your response is critically important to your local community and to
your country, a Census Bureau interviewer may come to your home to complete
the survey in person. Respond by October 26, 2019 to be removed from our
schedule for a visit.”

•

Experimental Box on the Envelope - Treatment 6 will use a date on the envelope:

See Attachment B for facsimiles of the mail materials for the fifth mailing for all treatments.
Table 1 summarizes the experimental design.

Table 1. Experimental Treatments for the 2019 ACS Due Dates Test
Treatment

Due Date Message

Envelope Text

T1
(Control)

None

Final Notice
Respond Now

Call-out Box Only

Final Notice
Respond Now

Call-out Box + Add Message

Final Notice
Respond Now

Call-out Box + Neutral Message

Final Notice
Respond Now

Call-out Box + Remove Message

Final Notice
Respond Now

Call-out Box Only

Final Notice
Response Due:
(date)

Analysis Metrics: The Census Bureau will compare self-response rates and final response rates
between treatments to assess the impact of the due date elements. The following comparisons
will be made among the sampled addresses sent the fifth mailing:
•

To determine if there is an impact on self-response of adding a due date in the call-out
box, the Census Bureau will compare Treatment 1 versus Treatment 2.

•

To assess the additional impact of adding a due date message in the text of the letters,
Treatments 3, 4, and 5 will be compared against Treatment 2.

•

To assess the impact of the variations of the due date messages in the text of the
letters, Treatments 3, 4, and 5 will be compared to each other.

•

To assess the impact of adding a due date to the envelope, Treatment 6 will be
compared to Treatment 2.

•

The overall impact of a treatment will be assessed by comparing Treatments 2 through 6
to Treatment 1. The Census Bureau will also assess the impact of each of the
experimental treatments on data collection costs.

Sample: The monthly ACS production sample of approximately 295,000 addresses is divided
into 24 groups, where each group contains approximately 12,000 addresses. Each group is a
representative subsample of the entire monthly sample, and each monthly sample is
representative of the entire yearly sample and the country. The Census Bureau will use two

randomly selected groups for each treatment. Hence, each treatment will have a sample size of
approximately 24,000 addresses. In total, approximately 144,000 addresses will be used for the
six treatments. The remaining sample will receive the current production materials. The sample
size will be able to detect differences of approximately 1.74 percentage points between the
self-response return rates of two treatments. The power of the test, which is 80 percent and
where α=0.1, assumes a 50 percent response rate.
Use of Incentives: None.
Burden: There is no change in burden to the public associated with this test. The test is being
conducted using production ACS sample. No additional contacts are being made.
Attachments:
Attachment A – Treatment 1 Mail Materials
Attachment B – Treatments 2-6 Due Date Treatments
Attachment C – ACS Study Plan: 2019 American Community Survey Due Dates Test
Attachment D – ACS Methods Panel Burden Hour Tracker
Attachment E – 30 Day Federal Register Notice
Contact Information:
Please contact the following individuals for further information about this test:
Data Collection:
Jennifer Ortman
American Community Survey Office
U.S. Census Bureau
Washington, D.C. 20233
(301) 763-5750
[email protected]

Statistical Aspects:
Elizabeth Poehler
Decennial Statistical Studies Division
U.S. Census Bureau
Washington, D.C. 20233
(301) 763-9305
[email protected]

Attachment A: Mailings 1 through 4 for all Treatments
Mailing 1
Production
(Not part of
the test)

Production
Materials As Is

Treatment 1
(Control)

Envelope:
ACS-46IM(2020)

Mailing 2

Pressure Seal
Letter:
ACS-20(L)(2020)

Mailing 3

Envelope:
ACS-46DDT(2020)

Instruction Card:
ACS-34IM

Survey Form:
ACS-1(DDT)

Letter:
ACS-13(L)(2020)

Letter:
ACS-14(L)(2020)

Multilingual
Brochure:
ACS-9 (2020)

Return Envelope
6385-47(2014)

Mailing 4

Reminder
Postcard:
ACS-29(2020)

Treatment 2
(Box Only)

(same as above)

Treatment 3
(Add
Statement AS)
Treatment 4
(Neutral
Statement OR)

(same as above)

Treatment 5
(Remove
Statement RS)

(same as above)

Treatment 6
(Box +Date on
Envelope)

(same as above)

See Attachment B for the fifth mailing for the treatments.

Attachment A: Mailings 1 through 4 for all Treatments
I.
II.
III.
IV.

Mailing 1: Initial Mailing
Mailing 2: First Reminder
Mailing 3: Questionnaire Package
Mailing 4: Second Reminder

Mailing 1: Initial Mailing
Envelope:
Instruction Card:
Letter:
Multilingual Brochure:

ACS-46IM(2020) (02-07-2019)
ACS-34IM (2-13-2019)
ACS-13(L)(2020) (03-01-2019)
ACS-9 (2020)

Treatments 1-6: Initial Mailing Envelope
PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau
National Processing Center
1201 E. 10th St.
Jeffersonville, IN 47132
OFFICIAL BUSINESS
Penalty for Private Use $300

OPEN IMMEDIATELY
The American Community Survey

YOUR RESPONSE IS
REQUIRED BY LAW

ACS-46IM(2020) (02-07-2019)

Treatments 1-6: Initial Mailing Instruction Card

American Community Survey
Go to https://respond.census.gov/acs to complete the American Community Survey online.

(Vea el otro lado
para español.)

IMPORTANT: You will need information from the address label on this card to log in. If you need help or have
questions about the American Community Survey, call the toll-free number 1-800-354-7271.

American Community Survey

Vaya a https://respond.census.gov/acs para completar la Encuesta sobre la Comunidad Estadounidense por
Internet en español.

ATENCIÓN: Necesitará información que aparece en la etiqueta en el otro lado de esta tarjeta para iniciar la sesión.
Si usted necesita ayuda para llenar la encuesta o tiene preguntas acerca de la Encuesta sobre la Comunidad
Estadounidense, llame sin cargo al 1-877-833-5625.

See other side for English.
ACS-34 IM (2-13-2019)

Treatments 1-6: Initial Mailing Letter

DC
U.S. Census Bureau
Washington, DC 20233
Ofﬁce of the Director

A message from the Director of the U.S. Census Bureau:
Your address has been randomly selected by the U.S. Census Bureau to participate in the
American Community Survey. The Census Bureau conducts this survey each year to give our
country an up-to-date picture of how we live – including our education, housing, and jobs.
Communities across the country rely on information from this survey to decide where important
services are needed, including:
● Improving roads and reducing trafﬁc
● Building schools
● Planning for the health care needs of the elderly

Respond now at https://respond.census.gov/acs
Your response is required by U.S. law.
Because your household has been asked to participate on behalf of your community, it is vital
that you complete this survey to help meet critical needs in your area.
If you are unable to complete the survey online, we will send you a paper questionnaire in a
few weeks. The Census Bureau is using the internet to collect this information to conserve natural
resources, save taxpayers’ money, and process data more efﬁciently.
If you need help completing the survey or have questions, please call our toll-free number
(1–800–354–7271).
Thank you in advance for your prompt response.
Sincerely,

Steven D. Dillingham
Enclosures

American Community Survey data help determine the annual distribution of
more than
ACS-13(L)(2020) (03-01-2019)

$675 billion

in federal funds to communities nationwide.
census.gov

Will my response be conﬁdential?
Yes. The U.S. Census Bureau is required by law to protect this information. The Census Bureau is
not permitted to publicly release your responses in a way that could identify you. We are
conducting this survey under the authority of Title 13, United States Code, Sections 141 and
193. Federal law protects your privacy and keeps your answers conﬁdential (Title 13, United
States Code, Section 9). Per the Federal Cybersecurity Enhancement Act of 2015, your data are
protected from cybersecurity risks through screening of the systems that transmit your data.
Am I required to ﬁll out the survey?
Yes. Your response to this survey is required by law (Title 13, U.S. Code, Sections 141, 193, and
221). Title 13, as changed by Title 18, imposes a penalty for not responding. As a randomly
selected representative of your community, you are the voice of your neighbors and peers. To
create an accurate picture of your community, it is critical that you respond.
How will the Census Bureau use the information I provide?
By law, the Census Bureau can only use your responses to produce statistics. Your information
will be used in combination with information from other households to produce data for your
community. Similar data will be produced for communities across Puerto Rico and the United
States.
We may combine your answers with information that you gave to other agencies to enhance the
statistical uses of these data. This information will be given the same protections as your survey
information. Based on the information that you provide, you may be asked to participate in
other Census Bureau surveys that are voluntary.

Treatments 1-6: Initial Mailing Multilingual Brochure

Important
Information From the
U.S. Census Bureau

La Oficina del Censo de los Estados
Unidos está realizando la Encuesta
sobre la Comunidad Estadounidense
En unos días, recibirá por correo un
cuestionario de la Encuesta sobre la
Comunidad Estadounidense. Como usted
está viviendo en los Estados Unidos, la ley
exige que usted responda a esta encuesta.
Si tiene preguntas sobre el cuestionario,
llámenos al 1-877-833-5625 para hablar
con uno de nuestros empleados que habla
español. La llamada es gratis. El empleado
podrá contestar sus preguntas o usted
podrá completar la encuesta por teléfono.

Yes. The U.S. Census Bureau is
required by law to keep your
information confidential. The Census
Bureau is not permitted to publicly
release your responses in a way that
could identify you. Per the Federal
Cybersecurity Enhancement Act of
2015, your data are protected from
cybersecurity risks through screening
of the systems that transmit your data.

Sí. La Oficina del Censo de los
EE.UU. está obligada por ley a mantener
confidencial su información. A la Oficina
del Censo no se le permite divulgar sus
respuestas de manera que este hogar
pudiera ser identificado. En conformidad
con la Ley para el Fortalecimiento de la
Seguridad Cibernética Federal del 2015,
sus datos están protegidos contra los
riesgos de seguridad cibernética mediante
los controles aplicados a los sistemas que
trasmiten su información.

Issued 2-5-2019
ACS-9 (2020)
Connect with us @uscensusbureau

是的。 根据法律规定，美国人口普查
局将对您的信息保密。人口普查局不得
以可识别您的身份的方式公开发布您的
回复。依据2015年联邦增强网络安全法
案，通过监察传输您资料的系统，来确
保您个人资料受到保护，避免网络安全
风险。

Có. Cục Thống Kê Dân Số Hoa Kỳ được pháp
luật yêu cầu bảo mật thông tin của quý vị. Cục
Thống kê không được phép công bố công khai
các phản hồi của quý vị theo cách có thể nhận
diện quý vị. Theo Luật Tăng Cường An Ninh
Mạng của Liên Bang 2015, số liệu của quí vị sẽ
được bảo vệ để tránh khỏi các nguy cơ về an
ninh mạng qua cách kiểm duyệt các hệ thống
chuyển số liệu của quí vị.

Да. По закону Бюро переписи
населения США обязано соблюдать
конфиденциальность ваших данных. Ему
запрещено публично разглашать Ваши
ответы таким образом, чтобы по ним
можно было установить Вашу личность.
Защиту Ваших данных от кибер-рисков
регулирует федеральный закон «О
повышении кибербезопасности» от 2015
года, в соответствии с которым регулярно
проводится проверка систем передачи
данных.

네. 미국 인구조사국은 법에 따라 귀하의
정보를 비밀로 유지해야 할 의무가 있습니다.
미국 인구조사국은 귀하가 응답한 정보를
귀하의 신상을 알 수 있는 형태로 일반에 공개할
수 없으며, 2015년 연방 사이버보안강화법에
따라, 귀하의 데이터는 데이터 전송
시스템의 철저한 검사를 통해 사이버 보안의
위험으로부터 보호됩니다.

Attachment A: Mailings 1 through 4 for all Treatments
I.
II.
III.
IV.

Mailing 1: Initial Mailing
Mailing 2: First Reminder
Mailing 3: Questionnaire Package
Mailing 4: Second Reminder

Mailing 2: First Reminder
Pressure Seal Letter:

ACS-20(L)(2020) (03-01-2019)

Treatments 1-6: First Reminder Letter

DC
U.S. Census Bureau
Washington, DC 20233
Ofﬁce of the Director

A message from the Director of the U.S. Census Bureau:
A few days ago, you should have received instructions for completing the American Community
Survey online. Local communities depend on information from this survey to decide where schools,
highways, hospitals, and other important services are needed. If you have not already responded,
please do so now.

Respond now at https://respond.census.gov/acs
Log in using this user ID:
If we do not receive your response online, we will mail a paper questionnaire to your address.

Your response to this survey is required by law.
Your response is critically important to your local community and your country. Responding
promptly will prevent you from receiving additional reminder mailings, phone calls, or personal visits
from Census Bureau interviewers.
If you need help completing the survey or have questions, please call our toll-free number
(1–800–354–7271).
Thank you in advance for your prompt response.
Sincerely,

Steven D. Dillingham

census.gov

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau
National Processing Center
1201 E 10th Street
Jeffersonville IN 47132
OFFICIAL BUSINESS
Penalty for Private Use $300

ACS-20(L)(2020) (03-01-2019)

Attachment A: Mailings 1 through 4 for all Treatments
I.
II.
III.
IV.

Mailing 1: Initial Mailing
Mailing 2: First Reminder
Mailing 3: Questionnaire Package
Mailing 4: Second Reminder

Mailing 3: Questionnaire Package
Envelope:
Letter:
Survey Form:
Return Envelope:

ACS-46DDT (2020) (04-03-2019)
ACS-14(L)(2020) (03-01-2019)
ACS-1(DDT) (05-06-2019)
6385-47(2014) (10-2013)

Treatments 1-6: Questionnaire Package Envelope

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

OPEN IMMEDIATELY
U.S. CENSUS FORM ENCLOSED

YOUR RESPONSE IS
REQUIRED BY LAW

Treatments 1-6: Questionnaire Package Letter

DC
U.S. Census Bureau
Washington, DC 20233
Ofﬁce of the Director

A message from the Director of the U.S. Census Bureau:
The U.S. Census Bureau recently sent you a request to complete the American Community Survey
online. If you have not already responded to this important survey, please do so now.
Complete the survey using ONLY ONE of the following options:

● Respond online at https://respond.census.gov/acs
● If you are unable to respond online, please ﬁll out and
mail back the enclosed questionnaire.
You are required by U.S. law to respond to this survey.
The Census Bureau has randomly selected your address to receive this survey as part of a nationally
representative sample. Because you will be providing important information on behalf of your
community, it is vital that you complete this survey to help meet critical needs in your area – including
determining where to locate new schools, hospitals, and ﬁre stations.
If you do not respond promptly, a Census Bureau interviewer may contact you to complete the survey.
The Census Bureau is required by law to keep your information conﬁdential.
The Census Bureau is not permitted to publicly release your responses in a way that could identify
you or your household.
If you need help completing the survey or have questions, please call our toll-free number
(1–800–354–7271).
Thank you for your prompt response.
Sincerely,

Steven D. Dillingham
Enclosures

ACS-14(L)(2020) (03-01-2019)

census.gov

Treatments 1-6: Questionnaire

13199013

The American Community Survey

Start Here

➜

Month

You have two ways to respond:
Respond online today at:
https://respond.census.gov/acs

Please print today’s date.

➜

Day

Year

Please print the name and telephone number of the person who is
filling out this form. We will only contact you if needed for official
Census Bureau business.
Last Name

Complete this form and mail it
back as soon as possible.

First Name

Your response is required by law.
The American Community Survey is conducted by
the U.S. Census Bureau. This survey is one of only
a few surveys for which all recipients are required
by law to respond. The U.S. Census Bureau is
required by law to protect your information.

Area Code + Number

—

➜

How many people are living or staying at this address?
• INCLUDE everyone who is living or staying here for more than 2 months.
• INCLUDE yourself if you are living here for more than 2 months.
• INCLUDE anyone else staying here who does not have another place to
stay, even if they are here for 2 months or less.
• DO NOT INCLUDE anyone who is living somewhere else for more than
2 months, such as a college student living away or someone in the
Armed Forces on deployment.
Number of people

➜

Fill out pages 2, 3, and 4 for everyone, including yourself, who is
living or staying at this address for more than 2 months. Then
complete the rest of the form.

If you need help or have questions about
completing this form, please call
1-800-354-7271.
Telephone Device for the Deaf (TDD):
Call 1–800–582–8330.
¿NECESITA AYUDA? Llame sin cargo alguno al
1-877-833-5625.
For more information about the American Community
Survey, visit our website at: https://www.census.gov/acs

ACS-1(DDT)

FORM
(05-06-2019)

§.4{.¤

OMB No. 0607-0810
OMB No. 0607-0936

Treatments 1-6: Questionnaire

13199286

Mailing
Instructions
➜ Please make sure you have...

• listed all names and answered the questions on
pages 2, 3, and 4
• answered all Housing questions
• answered all Person questions for each person.
➜ Then...

• put the completed questionnaire into the postage-paid
return envelope. If the envelope has been misplaced,
please mail the questionnaire to:
U.S. Census Bureau
P.O. Box 5240
Jeffersonville, IN 47199-5240
• make sure the barcode above your address shows
in the window of the return envelope.
Thank you for participating in
the American Community Survey.

For Census Bureau Use
POP

EDIT

EDIT CLERK

PHONE

TELEPHONE CLERK

JIC1

JIC2

JIC3

JIC4

The Census Bureau estimates that, for the average
household, this form will take 40 minutes to complete,
including the time for reviewing the instructions and
answers. Send comments regarding this burden estimate
or any other aspect of this collection of information,
including suggestions for reducing this burden, to:
Paperwork Project 0607-0810 and 0607-0936,
U.S. Census Bureau, 4600 Silver Hill Road, AMSD – 3K138,
Washington, D.C. 20233. You may e-mail comments to
[email protected]; use "Paperwork Project
0607-0810 and 0607-0936" as the subject. Please
DO NOT RETURN your questionnaire to this address.
Use the enclosed preaddressed envelope to return your
completed questionnaire.
Respondents are not required to respond to any
information collection unless it displays a valid approval
number from the Office of Management and Budget.
This 8-digit number appears in the bottom right on the
front cover of this form.
Form ACS-1(DDT) (05-06-2019)

§.4}w¤

Treatments 1-6: Questionnaire Package Return Envelope
AN EQUAL OPPORTUNITY EMPLOYER

OFFICIAL BUSINESS
Penalty for Private Use $300

NO POSTAGE
NECESSARY
IF MAILED
IN THE
UNITED STATES

6385-47(2014) (10-2013)

BUSINESS REPLY MAIL
FIRST-CLASS MAIL

PERMIT NO. 16081

WASHINGTON DC

POSTAGE WILL BE PAID BY THE U.S. CENSUS BUREAU

DIRECTOR
US CENSUS BUREAU
PO BOX 5240
JEFFERSONVILLE IN 47199-5240

FFATTFADDDTAADTFADDFAFTDFFAADFFFAFDTFDTAFATFDFATADTTFDFDTFFFFATTF

Attachment A: Mailings 1 through 4 for all Treatments

I.
II.
III.
IV.

Mailing 1: Initial Mailing
Mailing 2: First Reminder
Mailing 3: Questionnaire Package
Mailing 4: Second Reminder

Mailing 4: Second Reminder
Postcard:

ACS-29(2020) (02-08-2019)

Draft 2 (2/8/19)

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau

National Processing Center
1201 E. 10th St.
Jeffersonville, IN 47132

OFFICIAL BUSINESS
Penalty for Private Use $300

YOUR RESPONSE IS
REQUIRED BY LAW

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau

National Processing Center
1201 E. 10th St.
Jeffersonville, IN 47132

OFFICIAL BUSINESS
Penalty for Private Use $300

YOUR RESPONSE IS
REQUIRED BY LAW

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

YOUR RESPONSE IS
REQUIRED BY LAW

Treatments 1-6: Second Reminder (Postcard)

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

Draft 2 (2/8/19)

The U.S. Census Bureau has sent you repeated requests to complete the
American Community Survey. If you have not already responded, it is
imperative that you complete this survey now.

Complete and mail back your paper questionnaire now.
Or respond at https://respond.census.gov/acs

You are required by U.S. law to respond to this survey (Title 13, U.S. Code,
Sections 141, 193, and 221).

If you do not respond now, a Census Bureau interviewer may contact you to
complete the survey.

If you need help completing the survey or have questions, please call our
toll-free number (1–800–354–7271).

If you need help completing the survey or have questions, please call our tollfree number (1–800–354–7271).

Thank you.

ACS-29(2020) (02-08-2019)

The U.S. Census Bureau has sent you repeated requests to complete the
American Community Survey. If you have not already responded, it is
imperative that you complete this survey now.

Complete and mail back your paper questionnaire now.
Or respond at https://respond.census.gov/acs

You are required by U.S. law to respond to this survey (Title 13, U.S. Code,
Sections 141, 193, and 221).

If you do not respond now, a Census Bureau interviewer may contact you to
complete the survey.

If you need help completing the survey or have questions, please call our
toll-free number (1–800–354–7271).

Thank you.

Thank you.
ACS-29(2020) (02-08-2019)

ACS-29(2020) (02-08-2019)

Attachment B: Fifth Mailing
Experimental Due Date Treatments
Mailing 5: Final Reminder

Treatment 1: Control (no due date)

ACS-23(LX)(2020) (03-01-2019)

Treatment 2: Call-out box only

ACS-23(LX) DDT-BO

Treatment 3: Call-out box + Add Message

ACS-23(LX) DDT-AS

Treatment 4: Call-out box + Neutral Message

ACS-23(LX) DDT-OR

Treatment 5: Call-out box + Remove Message

ACS-23(LX) DDT-RS

Treatment 6: Call-out box + Due Date on Envelope ACS-23(LX) DDT-BOD

Treatment 1: Control (no due date)

DC
U.S. Census Bureau
Washington, DC 20233
Ofﬁce of the Director

A message from the Director of the U.S. Census Bureau:
The U.S. Census Bureau has sent you several requests to complete the American
Community Survey. If you have not already done so, now is the time to respond.

Respond now at https://respond.census.gov/acs
Log in using this user ID:
OR complete and mail back your paper questionnaire.
Your response is required by law.
If you do not respond promptly, a Census Bureau interviewer may contact you with a
personal visit to complete the survey. If you would like to complete the survey by
telephone or need assistance, please call our toll-free number (1–800–354–7271).
Thank you.
Sincerely,

Steven D. Dillingham

census.gov

Treatment 1: Control (no due date)

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau
National Processing Center
1201 E 10th Street
Jeffersonville IN 47132
OFFICIAL BUSINESS
Penalty for Private Use $300

FINAL NOTICE
RESPOND NOW

ACS-23(L)(2020) (03-01-2019)

Treatment 2: Call-out box only

DC
U.S. Census Bureau
Washington, DC 20233
Ofﬁce of the Director

A message from the Director, U.S. Census Bureau ...
The U.S. Census Bureau has sent you several requests to complete the American
Community Survey. If you have not already done so, now is the time to respond.

Due: October 26, 2019
Respond now at https://respond.census.gov/acs
Log in using this user ID:
OR complete and mail back your paper questionnaire.
Your response is required by law.
If you do not respond promptly, a Census Bureau interviewer may contact you with a
personal visit to complete the survey. If you would like to complete the survey by telephone
or need assistance, please call our toll-free number (1–800–354–7271).
Thank you.
Sincerely,

Steven D. Dillingham

census.gov

Treatment 2: Call-out box only

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau
National Processing Center
1201 E 10th Street
Jeffersonville IN 47132
OFFICIAL BUSINESS
Penalty for Private Use $300

FINAL NOTICE
RESPOND NOW

ACS-23(LX)DDT-BO (05-09-2019)

Treatment 3: Call-out box + Add Message

DC
U.S. Census Bureau
Washington, DC 20233
Ofﬁce of the Director

Due: October 26, 2019
Respond now at https://respond.census.gov/acs
Log in using this user ID:
OR complete and mail back your paper questionnaire.
Your response is required by law.
Because your response is critically important to your local community and to your country, a
Census Bureau interviewer may come to your home to complete the survey in person. If you
do not respond by October 26, 2019, we will add you to our schedule for a visit.
If you would like to complete the survey by telephone or need assistance, please call our
toll-free number (1–800–354–7271).
Thank you.
Sincerely,

Steven D. Dillingham

census.gov

Treatment 3: Call-out box + Add Message

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau
National Processing Center
1201 E 10th Street
Jeffersonville IN 47132
OFFICIAL BUSINESS
Penalty for Private Use $300

FINAL NOTICE
RESPOND NOW

ACS-23(LX)DDT-AS (05-09-2019)

Treatment 4: Call-out box + Neutral Message

DC
U.S. Census Bureau
Washington, DC 20233
Office of the Director

Due: October 26, 2019
Respond now at https://respond.census.gov/acs
Log in using this user ID:
OR complete and mail back your paper questionnaire.
Your response is required by law.
Your response is critically important to your local community and to your country. Respond
by October 26, 2019, or a Census Bureau interviewer may come to your home to
complete the survey in person.
If you would like to complete the survey by telephone or need assistance, please call our
toll-free number (1–800–354–7271).
Thank you.
Sincerely,

Steven D. Dillingham

census.gov

Treatment 4: Call-out box + Neutral Message

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau
National Processing Center
1201 E 10th Street
Jeffersonville IN 47132
OFFICIAL BUSINESS
Penalty for Private Use $300

FINAL NOTICE
RESPOND NOW

ACS-23(LX)DDT-OR (05-09-2019)

Treatment 5: Call-out box + Remove Message

DC
U.S. Census Bureau
Washington, DC 20233
Ofﬁce of the Director

Due: October 26, 2019
Respond now at https://respond.census.gov/acs
Log in using this user ID:
OR complete and mail back your paper questionnaire.
Your response is required by law.
Because your response is critically important to your local community and to your country, a
Census Bureau interviewer may come to your home to complete the survey in person.
Respond by October 26, 2019 to be removed from our schedule for a visit.
If you would like to complete the survey by telephone or need assistance, please call our
toll-free number (1–800–354–7271).
Thank you.
Sincerely,

Steven D. Dillingham

census.gov

Treatment 5: Call-out box + Remove Message

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau
National Processing Center
1201 E 10th Street
Jeffersonville IN 47132
OFFICIAL BUSINESS
Penalty for Private Use $300

FINAL NOTICE
RESPOND NOW

ACS-23(LX)DDT-RS (05-09-2019)

Treatment 6: Call-out box only + Due date on Envelope

DC
U.S. Census Bureau
Washington, DC 20233
Ofﬁce of the Director

Steven D. Dillingham

census.gov

Treatment 6: Call-out box only + Due date on Envelope

PRESORTED
FIRST-CLASS MAIL
POSTAGE & FEES PAID
U.S. Census Bureau
Permit No. G-58

U.S. Census Bureau
National Processing Center
1201 E 10th Street
Jeffersonville IN 47132
OFFICIAL BUSINESS
Penalty for Private Use $300

FINAL NOTICE
RESPONSE DUE:
OCTOBER 26, 2019

ACS-23(LX)DDT-BOD (05-09-2019)

Attachment C – Study Plan
American Community Survey Research and Evaluation Program

ACS Research & Evaluation Analysis Plan (REAP)

2019 American Community Survey
Due Dates Test

REAP Revision Log
Version
0.3
0.4
1.0

Date

Description
Initial Draft for Feedback
Second Draft for Approval
Final REAP

U.S. Census Bureau

TABLE OF CONTENTS
1.

Introduction ..................................................................................................................... 1

Background ...................................................................................................................... 1

Literature Review ............................................................................................................. 2

Research Questions and Methodology ............................................................................. 5
4.1
4.2
4.3

Experimental Design ..................................................................................................... 5
Research Questions ...................................................................................................... 7
Methodology................................................................................................................. 8
4.3.1
4.3.2
4.3.3
4.3.4
4.3.5
4.3.6

Sample Design ................................................................................................ 9
Self-Response Return Rates ........................................................................... 9
Final Response Rates .................................................................................... 10
Standard Error of the Estimates ................................................................... 10
Form Completeness ...................................................................................... 11
Cost Analysis ................................................................................................. 11

Assumptions and Limitations .......................................................................................... 11

Table Shells..................................................................................................................... 12

Potential Changes to ACS ................................................................................................ 13

References ...................................................................................................................... 13

Appendix A: CTL (Control) – No Due Date in Letter Call-Out Box ............................................ 15
Appendix A: CTL (Control) – No Due Date in Envelope Call-Out Box....................................... 16
Appendix B: BOX – Due Date in Letter Call-Out Box ............................................................... 17
Appendix B: BOX – No Due Date in Envelope Call-Out Box .................................................... 18
Appendix C: ENV – Due Date in Letter Call-Out Box ............................................................... 19
Appendix C: ENV– Due Date in Envelope Call-Out Box ........................................................... 20
Appendix D: NTL – Due Date in Letter Call-Out Box and a “Neutral” Message ....................... 21
Appendix D: NTL – No Due Date in Envelope Call-Out Box ..................................................... 22
Appendix E: ADD – Due Date in Letter Call-Out Box and an “Add” Message .......................... 23
Appendix E: ADD – No Due Date in Envelope Call-Out Box .................................................... 24
Appendix F: RMV – Due Date in Letter Call-Out box and a “Remove” Message ..................... 25
Appendix F: RMV – No Due Date in Envelope Call-Out Box .................................................... 26

U.S. Census Bureau

1. INTRODUCTION
Research has shown that when a request has a sense of urgency, people are more likely to
comply (Kotter, 2008; Gunelius, 2009). One way of creating a sense of urgency in a mail contact
survey request is to use a due date. In a recent mail-package focus group conducted for the U.S.
Census Bureau by Reingold, Inc., several participants volunteered that a stated deadline or due
date would be a strong motivator for them to respond in a timely fashion, especially when
coupled with the “required by law” notice (i.e., Title 13, U.S. Code, Sections 141 and 193)
(Reingold, 2014, p.17).
A due date may have an additional benefit. Research suggests that a due date can reduce
respondent burden, if it aligns with the recipient’s mail prioritization process (Dillman, Smyth,
and Christian, 2014). For example, many people sort their bills by due date, which helps them
to prioritize bill payments. A survey request with no due date does not align with this
prioritization process, placing an added burden on the survey recipient.
The American Community Survey (ACS) mail contact materials do not contain an explicit
deadline or due date. They do however contain implied due date messages, such as, “…please
complete the survey online as soon as possible” and “Respond online today…”
The purpose of the 2019 ACS Due Dates Test is to determine the effect on self-response of
using a due date in the ACS mail contact materials—specifically in the last mailing (a pressure
seal letter). This test will determine the effect of a due date:
•

In the callout box in the body of the letter

•

In the callout box on the envelope

•

Contextual message in the body of the letter that ties the due date to the possibility of a
visit from a Census Interviewer

2. BACKGROUND
The ACS is an ongoing, nationwide survey conducted by the Census Bureau to collect detailed
social, economic, housing, and demographic information from the population living in housing
units and group quarters. The ACS uses a mail contact strategy to encourage residents in
sampled addresses to self-respond.
To encourage self-response, the Census Bureau sends up to five mailings to a mailable sampled
address. The list of mailable sampled addresses is updated twice during this process to remove

U.S. Census Bureau

households that have already responded; thus minimizing the number of mail contacts received
by those who have already responded.
The Census Bureau sends the first mailing (initial mailing) to all mailable addresses in the
sample. The initial mailing invites recipients to complete the ACS online or wait for a paper
questionnaire. About seven days later, the Census Bureau sends all of these addresses a followup reminder letter.
About 14 days after the reminder letter is sent, the Census Bureau sends a third mailing to all
addresses for which a response was not received at the time of the first update of the sampled
addresses. This mail package includes a paper questionnaire. About four days later, the Census
Bureau sends these addresses a follow-up reminder postcard.
About 18 days after the reminder postcard is sent, the Census Bureau sends a fifth and final
mailing—a reminder letter to the remaining addresses for which a response was not received at
the time of the second update of the sampled addresses.
Between 17 and 24 days after the fifth mailing is sent, the Computer-Assisted Personal
Interviewing (CAPI) nonresponse follow-up operation begins wherein a subset of the remaining
nonresponders are surveyed in person. The CAPI operation lasts about four weeks. The selfresponse phase, which continues throughout CAPI, lasts between eight and nine weeks. See
Figure 1 for a pictorial representation of the current ACS self-response mail contact. 1
Figure 1. Current ACS Self-Response Mail Contact Strategy

3. LITERATURE REVIEW
Census research suggests that the inclusion of a due date can potentially boost survey selfresponse. A 2006 Decennial Census study (Martin, 2009) found that giving people a deadline
and a shorter interval (by one week) to complete the census form (2000 version) led to a higher
rate of mail response. The researchers believed that the effect of a compressed mailing
1

See the “American Community Survey Design and Methodology” (2014) for detailed information.

U.S. Census Bureau

schedule might be important for the deadline date to be effective, as it creates a sense of
urgency.
The Martin (2009) study had two major shortcomings: (1) it was not able to differentiate the
effects of the deadline messages from the compressed schedule, and (2) it was not able to
analyze the effects of deadline messaging on the speed of mail returns. For these reasons, the
Census Bureau conducted a follow-up study in 2010 (Stokes, Reiser, Bentley, and Meier, 2011).
This follow-up study tested four types of deadline messages: Mild, Progressive, Nonresponse
Followup (NRFU) Motivation, and Cost Savings. These themes were tested on the advance
letter, the cover letter and outgoing envelope of the initial questionnaire mailing package, and
the reminder postcard.
As an example, the deadline message on the initial mailing outgoing envelope for the Mild,
NRFU Motivation, and Cost Savings treatments is shown in Figure 2. For these treatments, the
message, “Mail by April 5” was used.
Figure 2. Deadline Message on outgoing envelope for the
Mild, NRFU Motivation, and Cost Savings treatments

The deadline message on the initial mailing outgoing envelope for the Progressive treatment is
shown in Figure 3. For this treatment, the message, “Deadline is April 5” was used.

U.S. Census Bureau

Figure 3. Deadline Message on outgoing envelope for the
Progressive treatment

The deadline message displayed in the cover letter of the initial questionnaire mailing package
for each of these themes is provided below:
•

Mild: “Please complete and mail back the enclosed census form by April 5.”

•

Progressive: “The deadline to complete and mail back the enclosed census form is April 5.”

•

NRFU Motivation: “Please complete and mail your census form by April 5 so that you can
avoid a personal visit from an interviewer.”

•

Cost Savings: “Please complete and mail your census form by April 5. Mailing your census
form on time saves money that would otherwise be used to follow up with you.”

Two major findings from the Stokes et al. (2011) study are provided below:
•

Finding 1: The overall national-level mail return rate for the Mild, NRFU Motivation, and
Cost Savings deadline message panels was significantly higher than control.

•

Finding 2: The Progressive deadline message panel’s replacement mailing return rate was
significantly lower than control.

Loss aversion theory teaches us that people would rather avoid a loss than reap a reward
because the pain of losing is psychologically more powerful than the pleasure of gaining.
For example, telling customers that they could miss out on a great discount if they do not act
now is more effective than offering them additional benefits if they act now. This reaction to
loss is explained by the ownership effect. People are averse to letting go of something they

U.S. Census Bureau

perceive as owning (Kahneman and Tversky, 1979; Kahneman and Tversky, 1992; Barberis,
2013; Kay, 2016).
In mail contact surveys, a stated due date coupled with a contextual message inspired by loss
aversion theory has the potential to enhance cooperation. These types of messages tap into
people’s tendency to avoid a sense of loss at the expense of a potential gain. Some people are
less risk averse and may be more inspired by a message that implies a gain.

4. RESEARCH QUESTIONS AND METHODOLOGY
4.1

Experimental Design

The 2019 ACS Due Dates Test will be conducted using the September 2019 ACS production
sample. Five experimental treatments and one control treatment will be used to test the
effectiveness of using due dates. We will test due date messaging in only the fifth mailing.
Originally we wanted to include due dates in the second and third mailings as well. However,
the number of potential treatments became too large and we had to make cuts.
In January 2020, the Census Bureau is proposing to update the production ACS mail materials as
a result of the 2018 Mail Materials Test (Risley and Berkley, forthcoming). For the 2019 ACS Due
Dates Test, the fifth mailing for the control treatment (CTL) will incorporate the following new
elements from the 2018 Mail Materials Test:
•

A call-out box in the letter (pressure seal letter):

•

Text in the letter that reads:
If you do not respond promptly, a Census Bureau interviewer may contact you with
a personal visit to complete the survey.

•

A box on the envelope:

U.S. Census Bureau

Five experimental treatments will test due date modifications of these three elements, in an
incremental manner:
•

Box treatment (BOX) will test the effect on self-response of a due date in the callout box
in the body of the letter.

•

Envelope treatment (ENV) will test the effect on self-response of a due date in the
callout box in the body of the letter and the callout box on the envelope.

•

Neutral treatment (NTL) will test the effect on self-response of a due date in the callout
box in the body of the letter coupled with a contextual message that tells the ACS
recipient that an interviewer may pay them a personal visit if their response is not
received by the due date (i.e., a neutral message—see exact wording below):
o Your response is critically important to your local community and to your
country. Respond by October 26, 2019, or a Census Bureau interviewer may
come to your home to complete the survey in person.

•

Add treatment (ADD) will test the effect on self-response of a due date in the callout
box in the body of the letter coupled with a contextual message inspired by loss
aversion theory that tells the recipient that they will be added to a schedule for a
personal visit if their response is not received by the due date (i.e., an add message—
see exact wording below):
o Because your response is critically important to your local community and to
your country, a Census Bureau interviewer may come to your home to complete
the survey in person. If you do not respond by October 26, 2019, we will add
you to our schedule for a visit.

•

Remove treatment (RMV) will test the effect on self-response of a due date in the
callout box in the body of the letter coupled with a contextual message inspired by loss
aversion theory that tells the recipient that they will be removed from a schedule for a
personal visit if they respond by the due date (i.e., a remove message—see exact
wording below):
o Because your response is critically important to your local community and to
your country, a Census Bureau interviewer may come to your home to complete
the survey in person. Respond by October 26, 2019 to be removed from our
schedule for a visit.

The mail materials for the BOX and ENV treatments most resemble the CTL treatment
materials, except for the inclusion of a due date in the callout boxes. The NTL, ADD, and RMV
6

U.S. Census Bureau

treatments differ from the CTL by the inclusion of a due date in the letter callout box and an
accompanying contextual due date message (bolded for emphasis). See Appendices A-F for
facsimiles of these mail materials.
Table 1 provides an overview of the differences among the six treatments with respect to the
call-out box in the body of the letter, the callout box on the envelope, and the contextual due
date message in the body of the letter.
Table 1. Experimental Treatments for the 2019 Due Dates Test
Treatment
CTL
BOX
ENV
NTL
ADD
RMV

4.2

Due Date Message in Letter
None
Call-out Box Only
Call-out Box Only
Call-out Box + “Neutral” Message
Call-out Box + “Add” Message
Call-out Box + “Remove” Message

Text in Envelope Callout Box
Final Notice Respond Now
Final Notice Respond Now
Final Notice Response Due: October 26, 2019
Final Notice Respond Now
Final Notice Respond Now
Final Notice Respond Now

Research Questions

The use of due dates has the potential to increase self-response return rates and lower ACS
data collection costs. Table 2 outlines the research questions that the 2019 ACS Due Dates Test
will answer.

U.S. Census Bureau

Table 2. Research Questions

Research Question
1. What is the impact on self-response
return rates of using a due date in the
letter callout box?
2. What is the impact on self-response
return rates of using a due date in in the
envelope callout box in addition to the due
date in letter callout box?
3. What is the impact on self-response
return rates of using a contextual due date
message in the body of the letter, tying the
due date to the possibility of a visit from a
Census interviewer, in addition to a due
date in the callout box?
4. What is the difference in self-response
return rates of the respondent being told
that they are being added to a schedule vs
being removed from a schedule vs a neutral
message?
5. What is impact on self-response rates
and final response rates of each of the
experimental treatments? (Overall and by
mode).
6. What is the impact on form
completeness of using a due date?**
7. What is the impact on costs, relative to
current production, of implementing each
of the experimental treatments? (Overall
and by mode)

Treatment Comparison When to Compare
BOX vs CTL
 Date fifth mailing is mailed*
 Date CAPI begins
ENV vs BOX




Date fifth mailing is mailed
Date CAPI begins

NTL vs BOX
ADD vs BOX
RMV vs BOX




Date fifth mailing is mailed
Date CAPI begins

ADD vs NTL
RMV vs NTL
RMV vs ADD




Date fifth mailing is mailed
Date CAPI begins

All experimental
treatments vs CTL




Date fifth mailing is mailed
Date CAPI begins

All experimental
treatments vs CTL
All experimental
treatments vs CTL



Closeout



Closeout

*Comparisons made at the date of the fifth mailing are to verify that there are no differences between treatments before our
experimental changes (due dates) are implemented.
** A due date may compel some recipients to enter scant data simply to meet the deadline

4.3

Methodology

All self-response analyses, except for the cost analysis, will be weighted using the ACS base
sampling weight (the inverse of the probability of selection). The CAPI response analysis will
include a CAPI subsampling factor that will be multiplied by the base weight. We will use a
significance level of α=0.1 when determining significant differences between treatments. For
analysis that involves multiple comparisons, we will adjust for the Type I familywise error rate
using the Hochberg method (Hochberg, 1988).

U.S. Census Bureau

4.3.1

Sample Design

The monthly ACS production sample of approximately 295,000 addresses is divided into 24
groups, where each group contains approximately 12,000 addresses. Each group is a
representative subsample of the entire monthly sample and each monthly sample is
representative of the entire yearly sample and the country. We will use two randomly selected
groups for each treatment. Hence, each treatment will have a sample size of approximately
24,000 addresses. In total, approximately 144,000 addresses will be used for the treatments
(five experimental treatments and one control treatment).
The sample sizes are chosen so that we may conduct 90-percent confidence level hypothesis
tests to determine if the differences between control and experimental treatment selfresponse return rates are greater than 1.74-percentage points. In calculating our sample size,
we assume that the survey achieves a 50-percent response rate and that our statistical test has
a power value (discernment of this 1.74-percent point difference) of 80-percent.
4.3.2

Self-Response Return Rates

To evaluate the effectiveness of the experimental treatments, we will calculate self-response
return rates. Using the entire (mailable and deliverable) sample universe could possibly mask
the effect of a due date in the fifth mailing. Therefore, we will use the smaller, nonresponding
universe that was sent the fifth mailing as our denominator for the rates calculated at the date
CAPI begins (formula 1). However, because the comparisons in question 5 (all treatments vs
CTL) inform whether we will conduct cost and form completeness analysis, we will also
calculate rates using the complete, mailable universe at this point in time (formula 2). For the
rates calculated at the date of the fifth mailing and at closeout, we will use formula 2.
Formula 1: Self-Response Rate using nonresponding addresses as the universe
2

A blank form is a form in which there are no persons with sufficient response data and there is no telephone
number listed on the form.

U.S. Census Bureau

Formula 2: Self-Response Rate using all mailable and deliverable addresses as the universe: 3

4.3.3

Final Response Rates

To evaluate the effect of the experimental treatments on overall response to the survey, we
will calculate final overall response rates using the formula below.

4.3.4

Standard Error of the Estimates

We will estimate the variances of the point estimates using the Successive Differences
Replication (SDR) method with replicates – the standard method used in the ACS (see U.S.
Census Bureau, 2014, Chapter 12). In calculating the self-response return rates and final
response rates, we will use replicate base weights, which only account for sampling
probabilities. We will calculate the variance for each rate and for the difference between rates
using the formula below:
80

4
Var (X0 ) =
� (Xr - X0 )2
80
r=1

Where:

Xr = the estimate calculated using the rth replicate
X0 = the estimate calculated using the full sample

The standard error of the estimate (X 0 ) is the square root of the variance.

We remove addresses where the United States Postal Service (USPS) returns the initial mail package or paper
questionnaire package as undeliverable as addressed (UAA) and a response is not received

U.S. Census Bureau

4.3.5

Form Completeness

There is a concern that adding a due date could cause potential respondents to submit an
incomplete form rather than taking time to fill it out completely and risk missing the due date.
For this reason, we will compare the form completeness of the experimental treatments to CTL.
If none of the experimental treatments have higher self-response return rates than CTL, then
we will not conduct an analysis of form completeness.
For each response, the denominator is the number of questions that should have been
completed (after adjusting for skip patterns based on responses or removing cases that did not
provide a response to an earlier dependent question). The numerator is the number of these
items that were actually completed.
4.3.6

Cost Analysis

In evaluating the different experimental treatments, it is not sufficient to only compare their
self-response return rates and final response rates. A treatment’s data collection costs, if
adopted, are also important. If one or more of the experimental treatments sees an increase in
earlier self-response, the CAPI workloads, which cost more per case to complete than selfresponse cases, would be smaller.
For each experimental treatment, we will evaluate impacts under the following scenarios:
•

•

Maintain the current sample size: this scenario will apply the results from this test to a
full year of ACS sample to evaluate the effect on the cost of using the given
experimental treatment methodology for an entire ACS data collection year.
Maintain costs: this scenario applies the results from this test to determine how much
the sample size would need to decrease or how much it could increase to collect the
ACS data using the test strategies within the fiscal year 2019 budget.

If none of the experimental treatments has higher self-response return rates than CTL, we will
not perform a cost analysis.

5. ASSUMPTIONS AND LIMITATIONS
During the cognitive testing phase, it was noted that many participants misinterpreted the
“remove” statement in the RMV treatment. Specifically, they read the statement to mean that
if they did nothing (did not respond), then they would be removed from the interviewer
schedule (the exact opposite of what the statement says). We decided to go ahead with this
treatment, because we wanted to test the impact on response rates of being “removed” from
something versus being “added” to something.

U.S. Census Bureau

The envelope we are testing for the ENV treatment is a composite of two envelopes that were
cognitively tested. We took elements from both envelopes that performed well in testing to
create the final version for the ENV treatment (this composite version was not cognitively
tested).
We assume that the treatment printing and processing costs are comparable for the six
treatments for the test.

6. TABLE SHELLS
Below are samples of tables that will be used in the final report to show results from this test.
Table 3. Sample Table for Total Self-Response Return Rates
Point in Data Collection
Cycle
Treatment X
Treatment Y
Before the Fifth Mailing
####
####
Date CAPI begins
####
####

Difference
####
####

P-Value
####
####

Source: U.S. Census Bureau, American Community Survey, 2019 Due Dates Test. DRB Approval Number: CBDRB-FYXX-RAGLINBXXXX.
Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a
statistically significant result. Significance was tested based on a two tailed t-test (Treatment X ≠ Treatment Y) at the α=0.1
level.

Table 4. Final Response Rates
Rate
Control
####
Experimental
####

Experimental - Control
--####

Difference P-Value
----####
####

Table 5. Final Response Rates by Mode
Treatment X
Production
Overall Response
####
####
Internet
####
####
Mail
####
####
CAPI
####
####

Difference
####
####
####
####

P-Value
####
####
####
####

U.S. Census Bureau

7. POTENTIAL CHANGES TO ACS
This research could potentially result in the inclusion of due date messaging in the ACS mail
contact materials, as a relatively inexpensive means of boosting self-response.

8. REFERENCES
Barberis, N. C. (2013). “Thirty Years of Prospect Theory in Economics: A review and
Assessment.” Journal of Economic Perspectives, 27(1), 173-96.
Kahneman, D., & Tversky, A. (1979). “Prospect theory: An analysis of Decision under Risk.”
Econometrica, 47, 263-291.
Dillman, D., Smyth, J., & Christian, L. (2014). Internet, Phone, Mail, and Mixed-Mode Surveys:
The Tailored Design Method (4th ed.). Hoboken, NJ: Wiley.
Gunelius, S. (2010). 3 Ways to Move Customers to Action. Retrieved on October 1, 2018 from
https://www.entrepreneur.com/article/205692.
Hochberg, Y. (1988). “A Sharper Bonferroni Procedure for Multiple Tests of Significance,”
Biometrika, 75 (4), 800-802. Retrieved on January 17, 2017 from
http://www.jstor.org/stable/2336325?seq=1#page_scan_tab_contents
Kahneman, D. & Tversky, A. (1992). "Advances in Prospect Theory: Cumulative Representation
of Uncertainty". Journal of Risk and Uncertainty, 5 (4): 297–323.
Kay, M. (2016). “You can Implement These Tips to Your Site or You Can Keep Losing Subscribers
Every Day: The Story of Loss Aversion.” Psychology for Marketers. Retrieved on May 14,
2019 from http://psychologyformarketers.com/loss-aversion/
Kotter, J.P. (2008). A Sense of Urgency. Boston, MA: Harvard Business Press.
Martin, E. (2009). “Can a Deadline and Compressed Mailing Schedule Improve Mail Response in
the Decennial Census?” Public Opinion Quarterly, 25(2), 361-367.
Reingold, Inc. (2014). “ACS Mail Package Research: Focus Groups and Interviews.” 2014
American Community and Evaluation Report Memorandum Series #ACS 14-RER-25.
Risley, M. & Berkley, J. (forthcoming). “2018 Mail Materials Test”. Washington DC, U.S. Census
Bureau.
Stokes, S., Reiser, C., Bentley, M., & Meier, A. (2011). “2010 Census Deadline Messaging and
Compressed Mailing Schedule Experiment,” U.S. Census Bureau.
Martin, E. (2009). “Can a Deadline and Compressed Mailing Schedule Improve Mail Response in
the Decennial Census?” Public Opinion Quarterly, 25(2), 361-367.
13

U.S. Census Bureau

U.S. Census Bureau. 2014. “American Community Survey Design and Methodology.” Retrieved
on May 04, 2019 from http://www2.census.gov/programssurveys/acs/methodology/design_and_methodology/acs_design_methodology_report_
2014.pdf.

U.S. Census Bureau

Appendix A: CTL (Control) – No Due Date in Letter Call-Out Box

U.S. Census Bureau

Appendix A: CTL (Control) – No Due Date in Envelope Call-Out Box

U.S. Census Bureau

Appendix B: BOX – Due Date in Letter Call-Out Box

U.S. Census Bureau

Appendix B: BOX – No Due Date in Envelope Call-Out Box

U.S. Census Bureau

Appendix C: ENV – Due Date in Letter Call-Out Box

U.S. Census Bureau

Appendix C: ENV– Due Date in Envelope Call-Out Box

U.S. Census Bureau

Appendix D: NTL – Due Date in Letter Call-Out Box and a “Neutral” Message

U.S. Census Bureau

Appendix D: NTL – No Due Date in Envelope Call-Out Box

U.S. Census Bureau

Appendix E: ADD – Due Date in Letter Call-Out Box and an “Add” Message

U.S. Census Bureau

Appendix E: ADD – No Due Date in Envelope Call-Out Box

U.S. Census Bureau

Appendix F: RMV – Due Date in Letter Call-Out box and a “Remove” Message

U.S. Census Bureau

Appendix F: RMV – No Due Date in Envelope Call-Out Box

U.S. Census Bureau

Attachment D

ACS Methods Panel Burden Tracker

3-year American Community Survey Methods Panel Tests (0607-0936) (August 2018 - August 2021)
Total Burden Hours Requested over 3 years: 348,167

Methods Panel Submissions for Tests in
August 2018-August 2021

Field Date

Sample
Size (HUs)

Mail Materials Test
2019 Census Test

September 2018
June 2019

480,000

2019 ACS Due Dates Test

September 2019

144,000

Total

Estimated
Number of
Respondents

Cost to
Respondent

Burden
Estimated
Hours
Burden Hours
Requested
Used

241,440

$0
$0

0
40,000

72,432

40,000

Notes
There is no change in burden to the public
associated with this test. The test is being conducted
using production ACS sample
0
40,240
There is no change in burden to the public
associated with this test. The test is being conducted
using production ACS sample
0

40,240

5/30/2019

Attachment E- 30 Day Federal Register Notice

jbell on DSK3GLQ082PROD with NOTICES

20848

Federal Register / Vol. 84, No. 92 / Monday, May 13, 2019 / Notices

call, at 12:00 p.m. (EDT) Thursday, June
6, 2019. The purpose of the planning
meeting is to continue project planning
for a future briefing meeting on the
Committee’s civil rights project that
examines the intersection of
homelessness, mental health and the
criminal justice system, including a
review of the DC Mental Health Court.
DATES: Thursday, June 6, 2019 at 12:00
p.m. (EDT).
ADDRESSES: Public Call-In Information:
Conference call number: 1–877–260–
1479 and conference call ID number:
1929821.
FOR FURTHER INFORMATION CONTACT: Ivy
L. Davis, at [email protected] or by phone
at 202–376–7533.
SUPPLEMENTARY INFORMATION: Interested
members of the public may listen to the
discussion by calling the following tollfree conference call number: 1–877–
260–1479 and conference call ID
number: 1929821. Please be advised that
before placing them into the conference
call, the conference call operator may
ask callers to provide their names, their
organizational affiliations (if any), and
email addresses (so that callers may be
notified of future meetings). Callers can
expect to incur charges for calls they
initiate over wireless lines, and the
Commission will not refund any
incurred charges. Callers will incur no
charge for calls they initiate over landline connections to the toll-free
telephone number herein.
Persons with hearing impairments
may also follow the discussion by first
calling the Federal Relay Service at 1–
800–877–8339 and providing the
operator with the toll-free conference
call number: 1–877–260–1479 and
conference call ID number: 1929821.
Members of the public are invited to
make statements during the Public
Comments section of the meeting or to
submit written comments. The
comments must be received in the
regional office by Monday, July 8, 2019.
Comments may be mailed to the Eastern
Regional Office, U.S. Commission on
Civil Rights, 1331 Pennsylvania
Avenue, Suite 1150, Washington, DC
20425 or emailed to Evelyn Bohor at
[email protected]. Persons who desire
additional information may contact the
Eastern Regional Office at 202–376–
7533.
Records and documents discussed
during the meeting will be available for
public viewing as they become available
at: https://gsageo.force.com/FACA/
FACAPublicViewCommitteeDetails?id=
a10t0000001gzlKAAQ. Please click the
‘‘Meeting Details’’ and ‘‘Documents’’
links. Records generated from this
meeting may also be inspected and

VerDate Sep<11>2014

16:29 May 10, 2019

Jkt 247001

reproduced at the Eastern Regional
Office, as they become available, both
before and after the meeting. Persons
interested in the work of this advisory
committee are advised to go to the
Commission’s website, www.usccr.gov,
or to contact the Eastern Regional Office
at the above phone numbers, email or
street address.
Agenda
Thursday, June 6, 2019, at 12:00 p.m.
(EDT)
I. Welcome and Rollcall
II. Discuss Project and Hearing Planning
III. Other Business
IV. Next Planning Meeting
V. Public Comments
VI. Adjourn
Dated: May 7, 2019.
David Mussatt,
Supervisory Chief, Regional Programs Unit.
[FR Doc. 2019–09746 Filed 5–10–19; 8:45 am]
BILLING CODE P

DEPARTMENT OF COMMERCE
Census Bureau
Submission for OMB Review;
Comment Request
The Department of Commerce will
submit to the Office of Management and
Budget (OMB) for clearance the
following proposal for collection of
information under the provisions of the
Paperwork Reduction Act.
Agency: U.S. Census Bureau.
Title: American Community Survey
Methods Panel Tests, 2019 Due Dates
Test.
OMB Control Number: 0607–0936.
Form Number(s): ACS–1, ACS
internet.
Type of Request: Non-substantive
Change Request.
Number of Respondents: 144,000.
Average Hours per Response: 40
minutes.
Burden Hours: No additional burden
hours are requested under this nonsubstantive change request.
Needs and Uses: The American
Community Survey (ACS) collects
detailed socioeconomic data from about
3.5 million housing units in the United
States and 36,000 in Puerto Rico each
year. The ACS also collects detailed
socioeconomic data from about 195,000
residents living in Group Quarter (GQ)
facilities. An ongoing data collection
effort with an annual sample of this
magnitude requires that the ACS
continue research, testing, and
evaluations aimed at reducing
respondent burden, improving data

PO 00000

Frm 00003

Fmt 4703

Sfmt 4703

quality, achieving survey cost
efficiencies, and improving ACS
questionnaire content and related data
collection materials. The ACS Methods
Panel is a research program designed to
address and respond to issues and
survey needs.
Residents of sampled housing units
are invited to self-respond to the ACS
through a series of up to five mailings.
These mailings are sent to respondents
over approximately six and a half
weeks. Research has shown that when a
request has a sense of urgency, people
are more likely to comply with the
request (Kotter, 2008; Gunelius, 2009).
One way of creating a sense of urgency
is through deadline or due date
messaging. The current ACS mailings
contain implied due date messages,
such as ‘‘. . . please complete the
survey online as soon as possible’’ or ‘‘If
you have not already responded, please
do so now’’ but do not provide an
explicit due date. The purpose of this
test is to assess the effect on selfresponse of an explicit due date message
in the fifth mailing.
Six experimental treatments are
proposed. One treatment will serve as
the control and will not contain an
explicit due date. The other treatments
will vary the use and placement of a due
date.
This test will study the impact on
self-response and cost of including a
due date in the fifth mailing. To field
this test, the Census Bureau plans to use
the ACS production sample (clearance
number: 0607–0810, expires 06/30/
2020). Thus, there is no increase in
burden from this test since each
treatment will result in the same burden
estimate per interview (40 minutes). The
Census Bureau proposes to test the use
of due date messages as part of the ACS
September or October 2019 panel,
adhering to the same data collection
protocols as production ACS.
The ACS sample design consists of
randomly assigning each monthly
sample panel into 24 groups of
approximately 12,000 addresses each.
Each group, called a methods panel
group, within a monthly sample is
representative of the full monthly
sample. Each monthly sample is a
representative subsample of the entire
annual sample and is representative of
the sampling frame. The Census Bureau
proposes to use two randomly selected
methods panel groups for each
treatment. Hence, each treatment will
have a sample size of approximately
24,000 addresses. In total,
approximately 144,000 addresses will
be used for the six experimental
treatments. The remaining sample will
receive production materials.

E:\FR\FM\13MYN1.SGM

13MYN1

Federal Register / Vol. 84, No. 92 / Monday, May 13, 2019 / Notices
The Census Bureau proposes to
evaluate treatment comparisons by
comparing self-response rates. For each
comparison a two-tailed test will be
used so that the Census Bureau can
measure the impact on the evaluation
measure in either direction with 80
percent power, at the a = 0.1 level. The
sample size will be able to detect
differences of approximately 1.74
percentage points between the selfresponse return rates between two
experimental treatments. Additionally, a
cost analysis will also be conducted.
Affected Public: Individuals or
households.
Frequency: One-time test as part of
the monthly American Community
Survey.
Respondent’s Obligation: Mandatory.
Legal Authority: Title 13, United
States Code, Sections 141, 193, and 221.
This information collection request
may be viewed at www.reginfo.gov.
Follow the instructions to view
Department of Commerce collections
currently under review by OMB.
Written comments and
recommendations for the proposed
information collection should be sent
within 30 days of publication of this
notice to OIRA_Submission@
omb.eop.gov or fax to (202) 395–5806.
Sheleen Dumas,
Departmental Lead PRA Officer, Office of the
Chief Information Officer, Commerce
Department.
[FR Doc. 2019–09767 Filed 5–10–19; 8:45 am]
BILLING CODE 3510–07–P

Bureau of Economic Analysis
[Docket No. 1903292999–9299–01]
RIN 0691–XC090

BE–9: Quarterly Survey of Foreign
Airline Operators’ Revenues and
Expenses in the United States
Bureau of Economic Analysis,
Commerce.
ACTION: Notice of reporting
requirements.
AGENCY:

By this Notice, the Bureau of
Economic Analysis (BEA), Department
of Commerce, is informing the public
that it is conducting the mandatory
survey titled Quarterly Survey of
Foreign Airline Operators’ Revenues
and Expenses in the United States (BE–
9). The data collected on the BE–9
survey are needed to measure U.S. trade
in transport services and to analyze the
impact of U.S. trade on the U.S. and
foreign economies. This survey is

jbell on DSK3GLQ082PROD with NOTICES

VerDate Sep<11>2014

16:29 May 10, 2019

Jkt 247001

FOR FURTHER INFORMATION CONTACT:

Christopher Stein, Chief, Services
Surveys Branch (BE–50), Balance of
Payments Division, Bureau of Economic
Analysis, U.S. Department of
Commerce, 4600 Silver Hill Road,
Washington, DC 20233; phone (301)
278–9189; or via email at
[email protected].
SUPPLEMENTARY INFORMATION: Through
this Notice, BEA publishes the reporting
requirements for the BE–9 survey form.
As noted below, all entities required to
respond to this mandatory survey will
be contacted by BEA. Entities must
submit the completed survey forms
within 45 days after the end of each
calendar quarter. This Notice is being
issued in conformance with the rule
BEA issued on April 24, 2012 (77 FR
24373), establishing guidelines for
collecting data on international trade in
services and direct investment through
notices, rather than through rulemaking.
Additional information about BEA’s
collection of data on international trade
in services and direct investment can be
found in the 2012 rule, the International
Investment and Trade in Services
Survey Act (22 U.S.C. 3101 et seq.), and
15 CFR part 801. Survey data on
international trade in services and direct
investment that are not collected
pursuant to the 2012 rule are described
separately in 15 CFR part 801. The BE–
9 survey form and instructions are
available at www.bea.gov/ssb.
Reporting

DEPARTMENT OF COMMERCE

SUMMARY:

authorized by the International
Investment and Trade in Services
Survey Act.

Frm 00004

survey forms and instructions, which
contain complete information on
reporting procedures and definitions,
can be downloaded from www.bea.gov/
ssb and submitted through mail or fax.
Form BE–9 inquiries can be made by
phone to BEA at (301) 278–9303 or by
sending an email to [email protected].
When To Report: Reports are due to
BEA 45 days after the end of each
calendar quarter.
Paperwork Reduction Act Notice
This data collection has been
approved by the Office of Management
and Budget (OMB) in accordance with
the Paperwork Reduction Act and
assigned control number 0608–0068. An
agency may not conduct or sponsor, and
a person is not required to respond to,
a collection of information unless it
displays a valid control number
assigned by OMB. Public reporting
burden for this collection of information
is estimated to average 6 hours per
response. Additional information
regarding this burden estimate may be
viewed at www.reginfo.gov; under the
Information Collection Review tab, click
on ‘‘Search’’ and use the above OMB
control number to search for the current
survey instrument. Send comments
regarding this burden estimate to
Director, Bureau of Economic Analysis
(BE–1), U.S. Department of Commerce,
4600 Silver Hill Rd., Washington, DC
20233; and to the Office of Management
and Budget, Paperwork Reduction
Project 0608–0068, 725 17th Street NW,
Washington, DC 20503, or via email at
[email protected].
Authority: 22 U.S.C. 3101–3108.

Notice of specific reporting
requirements, including who is to
report, the information to be reported,
the manner of reporting, and the time
and place of filing reports, will be
mailed to those required to complete
this survey.
Who Must Report: (a) Reports are
required from U.S. offices, agents, or
other representatives of foreign airline
operators that had total reportable
revenues or total reportable expenses
that were $5 million or more during the
prior year, or are expected to be $5
million or more during the current year.
(b) Entities required to report will be
contacted individually by BEA. Entities
not contacted by BEA have no reporting
responsibilities.
What To Report: The survey collects
information on foreign airline operators’
revenues and expenses in the United
States.
How To Report: Reports can be filed
using BEA’s electronic reporting system
at www.bea.gov/efile. Copies of the

PO 00000

20849

Fmt 4703

Sfmt 4703

Paul W. Farello,
Associate Director for International
Economics, Bureau of Economic Analysis.
[FR Doc. 2019–09790 Filed 5–10–19; 8:45 am]
BILLING CODE 3510–06–P

DEPARTMENT OF COMMERCE
Bureau of Economic Analysis
[Docket No. 190329314–9314–01]
RIN 0691–XC099

BE–577: Quarterly Survey of U.S.
Direct Investment Abroad—
Transactions of U.S. Reporter With
Foreign Affiliate
Bureau of Economic Analysis,
Commerce.
ACTION: Notice of reporting
requirements.
AGENCY:

By this Notice, the Bureau of
Economic Analysis (BEA), Department
of Commerce, is informing the public

SUMMARY:

E:\FR\FM\13MYN1.SGM

13MYN1

File Type	application/pdf
Author	Kathryn A Cheza (CENSUS/ACSO FED)
File Modified	2019-05-30
File Created	2019-05-20