Contact Strategies Testing

Attachment L

Department of Commerce

United States Census Bureau

OMB Information Collection Request

2017 Economic Census

OMB Control Number 0607-XXXX

Contract Strategies Testing

Component of the Collection Strategy	Survey Collection and Treatment(s) Tested		Summary of Results			Implementation Decision
Advance notice 1	2012 Economic Census: Advance letter tested among SUs in industries with historically low response rates. Advance request to verify contact information tested among MUs with fewer than 1,000 employees.		Differences in check-in rates were not statistically significant for either experiment.			No. However, Account Managers will contact selected large MUs in advance of mailout. (See Section 3 subsection on “Outreach”)
Full-scale pilot of later mailout and due date	2015 ASM: Initial mailout date moved to Late January of year following reference year rather than Late December of reference year. Selected large MUs were offered a later due date (in May 2016) than SUs and smaller MUs, whose due date was mid-March 2016.		May 2016 due date for large MUs resulted in later 2015 ASM responses compared to 2014 ASM reporting.			Partial implementation, consisting of late January mailout and mid-March due date for all units.
Due date reminder 2	Quarterly Survey of Business Professional & Classification (SQ-CLASS) (reference period = 2014 2nd qtr): Reminder letter mailed 3 weeks prior to survey due date 2014 ARTS: Reminder letter mailed 2 weeks prior to survey due date		Improved timeliness and statistically significant increase in response maintained through to the end of the collection period.			Yes. Improved check-in rate and increased timeliness of response provides data collection cost savings since fewer cases require more expensive follow-up techniques (e.g., certified mail and telephone follow-up).
Accelerated follow-up with and without due date reminder 2	2014 ARTS: 1st nonresponse post-due-date follow-up reminder letter mailed 2 weeks earlier than traditional mail follow-up		Improved timeliness and statistically significant increase in response maintained through to the end of the collection period.			Yes. Improved check-in rate and increased timeliness of response provides data collection cost savings, reducing number of cases requiring more expensive follow-up (e.g., certified mail and telephone).
Red ink on envelopes 2	2014 AWTS: Using red ink versus standard black ink for imprinted due date / past due notice on the envelope. Applied in initial mail and all NR follow-up reminders.		Overall difference in check-in rates not statistically significant. However, statistically significant interaction effects of red ink treatment with selected subgroups: Increased check-in rate among prior nonrespondents compared to prior respondents Increased check-in rate among selected industries			Yes. Using red ink for imprinted due date / past due notices on envelopes appears to improve response rates among certain subgroups, particularly prior NRs, without reducing response from other subgroups, and it is cost neutral.
Half-page envelope size 2	2015 ARTS: Findings from focus groups with past EC respondents suggested that a larger envelope may get respondents’ attention more effectively. Half-page-sized envelopes compared with standard letter-sized envelopes used in all mail contacts.		Some statistically significant results, but of no practical significance: Statistical significance in check-in rates only at due date, but not at close-out of data collection. Statistically significant difference of ½ day, on average, between mail-out and receipt.			No. Differences, if any, in overall or subgroup response of no practical significance.
Messaging	2014 COS/ASM: Emphasis on electronic reporting options versus standard messaging in letters Emphasis on electronic. reporting options explained in letters versus placed in flyers.		General improvement in uptake of electronic mode and decrease in requests for paper forms; statistical significance varies depending on whether cases are in COS only, in both COS and ASM, as well as employment size.			Yes. Electronic reporting will be emphasized in letters, along with mandatory requirement, confidentiality pledges, purpose & uses of data collected, per OMB requirements.
Flyers 2		2015 SAS: Three treatments consisted of different flyers, each with a different type of motivational message, enclosed with initial and follow-up mailings.	No statistically significant effect of flyers on check-in rates amongst prior NR or respondents. Some small statistically significant, but inconsistent, improvements in check-in rates or response times for different flyers amongst different subgroups.	No. Including flyers in all mail contacts did not improve overall response. Mixed results of different flyers with different industry subgroups is inefficient and not cost effective to implement in production.
Certified mail for targeted subsample of SU nonrespondents3	2015 ASM: Compare use of certified mail follow-up amongst a targeted subsample of SU nonrespondents with non-targeted SU NR follow-up using regular 1st-class mail. Compare approach that combines the two approaches (targeted certified plus regular mail for the remainder) with full SU NR follow-up using regular 1st-class mail only.		The combined approach improved data quality.		A one-time, targeted certified mail follow-up will be implemented amongst SU NRs for the 2017 Econ Census.  The nonresponding SUs not selected to receive a certified follow-up will be sent follow-up letters using regular 1st-class mail.
Pressure-sealed envelopes	2016 SQ-Class, Refile, and ASM: Proposed use for due-date reminders and NR follow-up mailings. They will not be used for initial mailout / contact. Using pressure-sealed will reduce time lag between producing mailing lists and mailout, improving ability to remove responding cases prior to mailout. Concern that pressure-sealed envelopes may be perceived as “junk” mail and discarded or ignored, reducing effectiveness of mail contacts.		TBD			TBD, pending results of SQ-Class, Refile, ASM research

¹ Marquette, Erica, Michael E. Kornbau, and Junilsa Toribio. 2015. Testing Contact Strategies to Improve Response in the 2012 Economic Census.  In JSM Proceedings, Government Statistics Section. Alexandria, VA: American Statistical Association. 2212-2225. 

² Tuttle, Alfred D. 2016. Experimenting with Contact Strategies to Aid Adaptive Design in Business Surveys. In JSM Proceedings, forthcoming. Alexandria, VA: American Statistical Association.

³ Kaputa, et al., 2016 ICES-V