Non_substantive Change Request & Justification Memo

TO: Jennifer Park, Office of Management and Budget

FROM: Timothy Wojan DATE: 5/6/2014

SUBJECT: OMB CONTROL NUMBER: 0536-0071

Non-substantive changes to Rural Establishment Innovation Survey (REIS) main study based on pilot study results.

This memo describes seven proposed changes to the survey design, recruitment protocol, and instrument of REIS main study based on the results from a pilot study that was carried out from November 2013 through February 2014. Results of REIS pilot study are presented in a separate document entitled “Pilot Study Data Report 13-084” submitted by the Social and Economic Sciences Research Center (SESRC) at Washington State University. ERS response to SESRC’s recommendations for main study is discussed in another memo named “ERS Response to SESRC Full Study Recommendation” (ERS Response).

The proposed non-substantive changes for the administration of the main data collection for the REIS are outlined below.  The list provides references to the said SESR report and ERS memo for a fuller description and justification of the proposed changes.

1.  Statistical Design:  The initial sample size for the full study will be doubled to 60,000 from the original request of 30,000.  This is necessary to ensure an adequate number of completed surveys given a response rate roughly half of what was anticipated.  In addition, large establishments with 100 employees or more will be oversampled by a factor of 2. See page 1 of ERS Response and page 30 of Pilot Study Report 13-084.  Details regarding pilot study response rates are found on pages 11-13 of Pilot Study Report 13-084 and details of lower response rate for larger establishments are on page 22.  (Note: Under the proposed sample design change, respondent burden reduced by elimination of the telephone prescreening for the high quality BLS sample and respondent preference for more time efficient modes should accommodate the doubling of sample size while staying within the original approved burden estimate.  Details regarding burden calculation are found in the revised Supporting Statement Part A.)

2. Change in Prescreening Protocol:  Telephone prescreening to identify a specific contact within the establishment and verify eligibility was originally proposed for the full study.  The pilot study revealed that the high quality of the Bureau of Labor Statistics (BLS) Business Registry used as a sample frame contained very few ineligible businesses and response rates were not improved by identifying a specific contact.  Given the additional cost of doubling the sample with a minimal benefit from telephone prescreening it is proposed that prescreening of the BLS sample not be done, that eligibility be determined in the course of data collection, and that the ineligible rate for nonrespondents be determined by the ineligible for respondents.  However, the quality of the proprietary sample required for 5 states not included in the BLS sample has not been determined.  These cases will go through a telephone prescreening to verify eligibility.  See page 1 of ERS Response and page 30 of Pilot Study Report 13-084.

3. Change Incentives from $2 to $1:  The majority of mode sequences tested in the pilot included a $2 token incentive included in two mailings.  The mode that did not include an incentive had a response rate roughly 10% lower.  In order to maintain the incentive payments at the level proposed in the original supporting statement the token incentive will be reduced to $1 included in two mailings.  See page 2 of ERS Response and page 31 of Pilot Study Report 13-084.

4. Use Web First Contact Sequence for Full Study:  The pilot study tested five different contact sequences and the sequence which worked best included an early web link to the survey and used telephone contact in the latter half of the data collection period.  See page 2 of ERS Response and pages 31-32 of Pilot Study Report 13-084.  An in-depth evaluation of the effectiveness of contact sequences is found on pages 26-29. 

5.  Add Question to End of Survey Instrument:  A question to confirm whether the respondent agrees to be contacted if there is a need to clarify a response is requested to address possible data quality issues.   The proposed question and its placement on the last page of the mail questionnaire are found on page 34 of Pilot Study Report 13-084.

6.  Revise Advance Letter to Comport with Selected Contact Sequence:   The advance letter sent to all respondents includes a web link to the survey.  A copy is provided in Attachment D.

7. Revise Supporting Letter signed by ERS Administrator Mary Bohman to Comport with Selected Contact Sequence:  The previous supporting letter indicated a questionnaire would be sent in the coming weeks.  The letter will now be sent with the Advance Letter.  A copy is provided in Attachment D.   

Supplementary Documents Submitted Along with This Memorandum:

Pilot Study Data Report 13-084

Pilot Study Report 13-084 Appendix A Frequency Listings

Pilot Study Report 13-084 Appendix B Cluster Analyses

Pilot Study Report 13-084 Appendix C Open-ended Comments

Pilot Study Report 13-084 Appendix D Survey Instruments

ERS Response to SESRC Full Study Recommendation

Supporting Statement – Part A (Revised)

Supporting Statement – Part B (Revised)

Attachment A - Rural Establishment Innovation Survey Questionnaire-mail version (Revised)

Attachment B - Rural Establishment Innovation Survey Questionnaire-CATI version (Revised)

Attachment C - Rural Establishment Innovation Survey Questionnaire-Web version (Revised)

Attachment D – REIS Advance letter & intro materials (Revised)

2