Justification for Non-substantive Change

TO: OMB

FROM: CPSC

RE: SCOA Survey – Memo on Response Rate and Raising Incentives

DATE: June 12, 2019

The following memo outlines the specific challenges of recruiting participants for the Smoke and Carbon Monoxide (SCOA) survey and provides justification for raising the monetary incentive.¹

1. Introduction

2. SCOA Survey recruitment efforts and the challenges encountered

3. Understanding survey recruitment challenges and proposing solutions

   1. Increasing incentive to inflation adjusted levels and realistic compensation for time and intrusion

   2. The public is increasingly concerned about personal privacy and social trust, and requires greater confidence and motivation to participate in in-home surveys

4. Conclusions

1. Introduction

CPSC’s survey vendor, EurekaFacts, has been administering the SCOA survey since January 2019. For the last 5 months, EurekaFacts had significant difficulty recruiting participants for the in-home portion of the SCOA survey, which is directly impacting the schedule. To improve data collection efforts and to finish this project within the timeframe of the contract, we request approval from OMB to increase the incentive value from $25 for in-home interviews to $75. Justification for this change provided below.

There is ample precedent to make an incentive change equal to the requested value. For two recent CPSC product studies that involved in-home face-to-face data collection, the methodology in the OMB PRA package proposed $75 incentives for participating households. One of these research topics includes an evaluation of cordless window covers. The window cover study and the SCOA survey require in-home face-to-face data collection where participants allow researchers into private rooms within their homes.

2. Survey Recruitment Efforts and the Challenges Encountered

After several non-substantial modifications to the sampling and recruitment efforts, the response rate results are unchanged and are inadequate in meeting the schedule and the current contract with CPSC. Using the same or close methodology in the previous 1992 CPSC Smoke Alarm Operability Survey, the current cooperation rate to schedule participants for in the in-home survey equals 0.23% (or less than one quarter of 1%) for two combined primary sampling units where the survey has fielded. This is substantially less than the reported response rate in the 1992 survey, which was about 35 percent.

Here is a summary of the vendor’s recruitment efforts. From Jan. 1-May 31, 2019, EurekaFacts recruited respondents by phone using an ABS of households to participate in the SCOA survey. Data collection efforts focused on primary sampling units in Charlotte and New Bern, NC. The sample included 480 households from which approximately 60% of households had working phone numbers for a total of 288 households. Following a five-call design, households were recruited by phone using the approved protocol. The results of this recruitment effort for the in-home portion of the survey totaled 0.23%.

The first sample selection was calculated based on an expected response rate of 25%. Following recruitment efforts from the first sample selection, the available contacts were exhausted by mid-February. With OMB approval, a new sample was selected based on an expected 1% response rate. For this sample, data collection efforts also focused on the same PSUs. The sample included 11,023 households and approximately 60% had phone numbers for a total of 6,614 households to recruit by phone. Following a five-call design, each of the households were recruited using the approved screening protocol. The results of this recruitment effort for the in-home portion of the survey totaled 0.23%, on par with the first round of sampling.

In an effort to improve cooperation rate, on May 13, EurekaFacts implemented a revised non-substantial change to the screening instrument that emphasized reasons to participate in the survey and incorporated several refusal aversion strategies. A pre/post comparison of the Jan. 1-May 13, 2019 screening and May 13-31, 2019 screening found that the cooperation rate for in-home participation was unchanged. In a pre/post comparison, an estimated 0.23% agreed to participate in time 1 and 0.23% in time 2. In total, 23,702 calls have been made in the Charlotte and New Bern metro area. From these calls, a total of 2,079 were hard refusals.

3. Understanding Survey Recruitment Challenges and Proposing Solutions

With all other considered factors or variables the same as the 1992 survey, two variables that may be potentially impacting the response rate are the incentive amount and views on personal privacy and social trust.

1. Increasing Incentive to Inflation Adjusted Levels and Realistic Compensation for Time and Intrusion
   
   

As background, the incentive amount in the 1992 survey was $25. Considering the cost of living and the value of a person’s time, the 1992 $25 incentive would be equal to approximately $46 today. In the 1992 SCOA survey, the incentive value equaled $25. Adjusted for inflation, $25 in 1992 has a real dollar value of close to $45.54 in 2019. For the 1992 poll, the household response rate was in line with industry standards for the time and the project was completed within the contract timeframe. Polling methods have changed a lot since 1992 and in-home face-to-face polling is far more challenging and response rates much lower. Respondents need to feel that they are being compensated in an equitable way for their time, attention and for opening up their homes to researchers. Inflation is one factor to consider in determining equitable compensation.

2. Americans are Increasing Concerned About Personal Privacy and Social Trust, and Require Greater Confidence and Motivation to Participate in In-Home Surveys

The inflation adjusted dollar value is one of several factors that may be impacting response rates. Public attitudes on personal privacy and social trust have also changed dramatically since the 1990s, and in survey research, this impacts the willingness of potential participants to accept researchers into their homes no matter how societally beneficial the research goals may be. The General Social Survey and other public polls have found a steady decline in social trust and a corresponding increase in anxieties about privacy in the last 25+ years. Building trust, confidence and easing anxieties about privacy require effective recruitment tools to establish the legitimacy of a government survey and are central to boosting response rate. Moreover, the incentive to $75 above $46 due to inflation is a reasonable compensation of participants for their time, effort and willingness to allow researchers into their homes to achieve a satisfactory response rate.

4. Conclusions

For these reasons, EurekaFacts respectfully requests to increase the incentive from $25 to $75 that will match today’s public demand for an acceptable exchange for their time and for their openness to relinquish some of their privacy. This increase in incentive would be implemented for a 3-month period to determine if it will sufficiently improve the cooperation rate to meet the objectives of the survey within the established timeframe. At the end of the 3-months, the cooperation rate will be analyzed, and a determination will be made to either continue with the $75 incentive or explore other options to increase the response rate.

1 OMB CONTROL NUMBER: 3041-0180

ICR REFERENCE NUMBER: 201806-3041-002

2