Recruitment Plan

Option Year 4 Cognitive Testing of American Community Survey Recipients During a Decennial Year

Recruitment Plan

Recruitment Overview

For Option Year 4 (OY4) of the American Community Survey (ACS) Respondent Burden Testing contract, Westat will recruit enough participants to complete 48 English-language cognitive interviews. Interviews will be conducted with self-respondents and non-respondents to the ACS during the 2020 Census. The research objectives are to assess 1) recipients’ reactions to receiving both the ACS and the 2020 Census, and 2) their impressions of modified ACS mail materials communicating that their household needs to complete both surveys. The Census Bureau will provide Westat with lists of ACS self-respondents and non-respondents from which Westat will recruit cognitive interview participants.

Table 1 presents the required participant characteristics. In addition:

• Participants must have been mailed both the ACS and the 2020 Decennial Census.

• Participants must handle the mail for their household, either alone or in combination with other household members.

• At least 10 participants must have less than a Bachelor’s degree.

• Aside from education level, there are no minimum requirements for demographic characteristics.

  • For ACS self-respondents, we still strive for a diverse mix of race/ethnicity, age, sex, and household size.

  • For ACS non-respondents, there are no demographic requirements.

Table 1. Participant Characteristics for OY4 Cognitive Testing of ACS Recipients during a Decennial Year

Characteristics	Target
March ACS panel	16
April ACS panel	16
May ACS panel	16
Self-respondents to the ACS	42 (at least 5 from each panel who self-responded after the 3rd mailing)
Non-respondents to the ACS	6 (at least 2 from each panel)
Less than a bachelor’s degree	10
Live in a rural area	6

In order to keep the recall period between receipt of the mailings and the interview about those mailings short, the only recruitment method for this testing effort will be contacting by phone those on the sample lists provided by the Census Bureau. Attempts to screen and schedule participants are therefore limited to those cases with a phone number on the file.

Interview locations

The interviews will be conducted within a two-hour driving radius of Westat cognitive interviewing staff in the following geographic areas:

• Rockville, MD

• Raleigh, NC

• Ft. Collins, CO

• Orange County, CA

• Cleveland, OH

• Lancaster, PA

For cost considerations, we will focus recruiting efforts on the first four areas, using Cleveland and Lancaster as back-up areas.

Interviews will be conducted in a location convenient to the respondent, such as a public library meeting room.

Sample

The Census Bureau will provide the sample for this testing effort. The sample will be delivered in an Excel file via a secure transfer site (Kiteworks) on a weekly or more frequent basis as needed during the data collection period. The sample will be limited to ACS self-respondents and non-respondents in the designated geographic areas. Self-respondents are those who have responded to the ACS (whether or not they have responded to the 2020 Census) via Internet, Mail, or Telephone (TQA) prior to the CAPI launch, and who have been sent one or more of the three mailings of interest for this testing (initial, pressure seal, third). Non-respondents are those who have been exposed to the full ACS and 2020 Census mailing strategies without responding to the ACS (whether or not they have responded to the 2020 Census) and who have been not been sampled into the CAPI effort.

Columns that will appear on the self-respondent sample file include:

• Response designation (Self)

• Name on ACS cover page

• Phone number(s) from ACS cover page or the lookup process

• Street address/city/state/zip associated with the case

• Geographic location designation (WDC/RAL/FTC/ORC/CLE/LAN)

• Rural designation (y/n)

• Demographic characteristics of person whose name is on the cover page

  • Race/ethnicity

  • Age

  • Sex

  • Household size

  • Education level

• Number of ACS mailings sent

• Response date

• Response mode

Columns that will appear on the non-respondent sample file include:

• Response designation (Non)

• Street address/city/state/zip associated with the case

• Geographic location designation (WDC/RAL/FTC/ORC/CLE/LAN)

• Rural designation (y/n)

• Phone number(s)

Interview Dates

Interviews will take place between approximately April 14, 2020 and August 31, 2020. Table 2 presents the interviewing schedule across ACS panel and respondent type.

Table 2. Interviewing Schedule

ACS panel	ACS respondent type	Dates
March	Self-respondent	April 14 – May 29, 2020
	Non-respondent	May 11 – June 30, 2020
April	Self-respondent	May 11 – June 30, 2020
	Non-respondent	June 8 – July 31, 2020
May	Self-respondent	June 8 – July 31, 2020
	Non-respondent	July 8 – August 31, 2020

Self-respondents will be interviewed as soon as possible after responding to the ACS. To achieve this goal, Westat will begin contacting potential interview participants the next business day after receiving each refreshed sample list from the Census Bureau. After the initial few interviews with the March panel, Westat will filter subsequent sample files on recruitment characteristics that are still needed (e.g., rural, education level) until those cells are filled.

Non-respondents will be interviewed after they have been sent all ACS and Census mailings.

Screening and Scheduling

The screener appears in Attachment 1. It captures data on all characteristics of interest in ACS self-respondents and non-respondents as described in the Recruitment Overview section and Table 1. It also asks basic demographic questions such as age, gender, and education level.

In addition to identifying eligible and willing participants, the overall goal of the screening effort is to schedule new interviews as soon as possible after each sample list is received. We will use the filtering technique described in the previous section to “pre-screen” for the cases most likely to fulfill target characteristics. Most of the time, we anticipate scheduling those who agree to participate at the end of the screening call. There may be some screened cases about which we will need to consult with Census before scheduling. We will inform those participants that someone will be in touch with them within three business days.

Interview Progress Spreadsheet

The Interview Progress Spreadsheet (delivered as a separate document with this recruitment plan) is the primary tool by which we will track progress in fulfilling the recruiting criteria. Within this spreadsheet, the master selection grid worksheet records each respondent’s answers to every screener question, and will also include respondent characteristics provided by Census in the sample file. This worksheet also tracks respondents’ basic demographic data, along with date and time of interview, interview location, and assigned interviewer. During the screening period, we will update this worksheet daily.

The Interview Progress Spreadsheet will be used to communicate to the Census Bureau which eligible cases have been selected for scheduling, which have been scheduled, and which have been completed. Westat will upload this document for the Census Bureau on a weekly basis to the secure transfer site during the screening and interviewing period.

The Interview Progress Spreadsheet will allow tracking of quota fulfillment by showing completed interviews by target characteristic. This worksheet will be updated at the end of each day of interviewing.

Risk Mitigation

The most important risk mitigation tool is frequent communication between Westat and Census about recruitment progress, quota fulfillment, interview schedule updates, and interview completion. We anticipate and are prepared for daily or more frequent exchanges on these topics during the screening and interviewing period. Additional risks and mitigation strategies are described below.

1. Participant memory decay. The main purpose of the interviews is to understand as much as possible about ACS recipients’ reactions to mailings they have already received at the time they received them. Because we are relying on participants’ memories, the goal is to interview them as soon as possible after they have been exposed to the mailings of interest. We will mitigate the risk of participant memory decay by streamlining the selection, screening, and scheduling process. These efforts include, but are not limited to filtering on needed characteristics to better ensure we reach who we need to reach; beginning the calls within one business day of receiving the sample file; using programming and automation to quickly make updates in the tracking tools; scheduling interviews at the screening stage whenever possible; and having interviewers poised to conduct interviews at each participant’s earliest convenience. Census will support this effort by providing the sample files on a rolling basis and with as much detail as possible. Detail on the file facilitates Westat’s efforts to quickly identify the needed target characteristics of potential participants.

2. Altering non-respondent behavior. It is not the objective of this study to get participants to respond to the ACS or the 2020 Census. We will use neutral language at screening to avoid inadvertently encouraging them to complete either survey before the in-person interview.

3. Falling short of recruitment goals. Many of the techniques already described will help mitigate this risk, such as the sample file containing as much detail as possible (including phone numbers of non-respondents whenever possible); filtering on needed characteristics; re-screening on characteristics in the sample file. In addition, we will provide daily or more frequent updates to Census as needed, so that the team is alerted as early as possible to possible shortfalls. Any changes to the methods or criteria described in this document (e.g., relaxing the past participation requirement) will only be made after consultation with and approval from Census.

4. Possibility of contract not receiving 6-month no-cost extension. We will conduct the May panel interviews as early as possible. If we become aware soon enough, and with Census approval, we may redistribute the interviews so that fewer are conducted with cases from the May panel and more are conducted with cases from the earlier panels.

Attachment 1

OY4 ACS Cognitive Testing Recruiting Screener

OY4 ACS Cognitive Testing

Recruiting Screener

IF VOICEMAIL REACHED, LEAVE MESSAGE ONCE: Hello, this is [NAME] from Westat calling for [NAME] on behalf of the Census Bureau about a study to find out what people think about the mail they receive. To learn more about participating in this study, please call me at [NUMBER] at your earliest convenience. Thank you.

IF LIVE PERSON ANSWERS: Hello, this is [NAME] from Westat. May I please speak with [NAME]?

IF ASKED THE REASON FOR THE CALL: We’re working with the U.S. Census Bureau to find out what people think about the mail they receive.

ONCE CORRECT PERSON IS ON THE LINE, ADMINISTER EITHER SELF-RESPONDENT OR NON-RESPONDENT SCRIPT AS APPROPRIATE.

SELF-RESPONDENTS: I’m calling about a study we’re conducting for the U.S. Census Bureau to find out what people think about the mail they receive.

IF ASKED FOR MORE INFORMATION: The Census Bureau recently received your household’s response to the American Community Survey and they have asked us to follow-up to gather your reactions to the survey and related mailings.

If you are interested and eligible, we will invite you to a one-hour in-person interview at a location convenient to you. At the end of the in-person interview you will receive $40 cash. To make sure you are eligible for the study, I need to ask you a few questions. This will only take about 5 minutes.

NON-RESPONDENTS: I’m calling about a study we’re conducting for the U.S. Census Bureau to find out what people think about the mail they receive.

IF ASKED FOR MORE INFORMATION: They are conducting a national survey called the American Community Survey. You may or may not remember receiving some mailings about the survey. The Census Bureau is interested in your reactions to those mailings.

If you are interested and eligible, we will invite you to a one-hour in-person interview at a location convenient to you. At the end of the in-person interview you will receive $40 cash. To make sure you are eligible for the study, I need to ask you a few questions. This will only take about 5 minutes.

1. How old are you?

________

REF

DK

IF UNDER 18, REF OR DK, INELIGIBLE. THANK AND END.

2. How many people live in your household including yourself?

___________________

REF

DK

3. (IF MORE THAN ONE PERSON IN HH) Who typically handles the mail in your household? Would you say it’s...

Only you,

You and other members of your household, or

Someone else in your household other than you?

REF

DK

IF “SOMEONE ELSE IN YOUR HOUSEHOLD OTHER THAN YOU”, REF OR DK, INELIGIBLE. THANK AND END.

4. What is your sex?

FEMALE

MALE

OTHER

DK

REF

5. What is the highest grade or level of education you have completed?

LESS THAN HIGH SCHOOL

HIGH SCHOOL DIPLOMA/GED

SOME COLLEGE (INCLUDES ASSOCIATE’S)

COLLEGE GRADUATE (BA, BS)

GRADUATE/PROFESSIONAL DEGREE (MA, MS, MD, JD, PhD, etc.)

REF

DK

6. Are you of Hispanic, Latino, or Spanish origin?

YES

NO

REF

DK

7. What is your race? You may say more than one. (IF NECESSARY, READ CATEGORIES ALOUD)

MARK ALL THAT APPLY.

White,

Black or African American,

American Indian/Alaskan Native,

Asian, or

Native Hawaiian and Other Pacific Islander?

OTHER ______________________

REF

DK

IF NOT ELIGIBLE TO SCHEDULE IMMEDIATELY, SKIP TO CONTACT INFORMATION.

IF ELIGIBLE TO SCHEDULE IMMEDIATELY: Thank you. We would like to go ahead and set up the interview with you as soon as possible.

Contact Information

STORE THE INFORMATION ON THIS DOCUMENT AS HARD COPY ONLY!

KEEP THIS DOCUMENT IN A LOCKED, SECURE LOCATION!

Project: U.S. Census Bureau

Name ______________________________________________________________________

Address ____________________________________________________________________

City, State, ZIP________________________________________________________________

Phone Number _______________________________________________________________

Email Address________________________________________________________________

IF NOT SCHEDULING IMMEDIATELY, READ CLOSING TEXT BELOW TO THANK AND END THE CALL

Thank you for answers to these questions. If you’re eligible to participate in our study, Westat will be back in touch with you within about 3 days.

____________________________________________________________________________________

IF R IS SELECTED FOR AN INTERVIEW, ASK AUDIO RECORDING QUESTION BEFORE SCHEDULING THE DATE AND TIME.

We’d like to audio record the interview. This allows us to carefully review the discussion and draw the proper conclusions. Will that be alright with you?

Yes

No DO NOT SCHEDULE

Date: __________________________

Time: __________________________

Can you suggest a place where you’d be comfortable participating in the interview?

Yes COLLECT LOCATION INFORMATION

No SUGGEST NEAREST PUBLIC LIBRARY IN PARTICIPANT’S GEOGRAPHIC AREA

We would like to conduct the interview at __________. The address is _____________. Will this location work for you?

Yes

No IDENTIFY NEW LOCATION AND RECONTACT