Nonsubstantive change request and justification

TO: Jennifer Park, Office of Management and Budget

Statistical and Science Policy,

Office of Information and Regulatory Affairs

FROM: John Pender DATE: 2/6/15

Economic Research Service

SUBJECT: OMB CONTROL NUMBER: 0536-0072

Request for Non-substantive changes to sampling approach and procedure for contacting potential respondents in the health care provider survey component of the Survey on Rural Community Wealth and Health Care Provision (SRCWHCP)

We have completed the pilot phase survey of health care providers in 12 communities for the Survey on Rural Community Wealth and Health Care Provision (see attached pilot study report, henceinafter the “Pilot Report”). The mean response rate found in the pilot study was 25.3%, substantially less than the 80% response rate predicted in the Information Collection Request (ICR) for this survey. The lower than anticipated response rate in the pilot study raises concerns about the statistical power and possible non-response bias of the results of the full survey. To address these concerns, we request the following non-substantive changes to the sampling approach and recruitment protocol for potential respondents in the survey:

1. Increase the maximum number of providers sampled in each community from 10 to 32. Due to the lower response rate, this change will still result in an expected maximum of about 8 survey respondents per community, as proposed in the ICR, and will increase the total number of providers sampled from 1,500 to 1,859. There will be no burden increase associating with this change. A comparison of currently approved and proposed revision for estimated burden is shown in Table 1 below. More detailed explanations are provided in section IV.3 of the Pilot Report.

2. Adding a follow-up email with a link to the web survey in the recruitment protocol, where email addresses are available. (A document containing the proposed script for this proposed email follow up is submitted along with this memo.) This new recruitment protocol differs slightly from that proposed in the ICR and used in the pilot study, which did not include use of a follow up email to the provider (only follow-up phone calls were proposed). Because of the difficulty of reaching health care providers directly by phone, follow-up phone calls alone are not effective in increasing response rates. It is anticipated that use of follow-up emails will increase the response rate to as much as 30%. More discussion about this proposed change can also be found in section III.1 of the Pilot Report.

The completed surveys in the pilot study indicated very few problems of item non-response, with less than 1.2% of the data missing. We judge that the survey questionnaire is adequate for the study objectives, and propose no changes to the questionnaire design.

Statistical Power and Non-Response Bias

Using the results of the pilot study and the proposed new sampling approach, we have estimated the predicted statistical power of the full study. Considering the pilot study responses to five survey questions judged to be critical to the objectives of the study and that represent questions for which the response variance is likely to be highest, the power to detect an effect size of 0.10 was estimated to be above 90% for all questions, and close to 100% for four of the five questions. Hence, the study will have sufficient statistical power to achieve the objectives proposed in the ICR, with the proposed changes.

As proposed in the ICR, we will investigate the potential for non-response bias in analyzing the results of the full survey by testing for significant differences in response rates and survey responses across observed characteristics of potential respondents and the study communities (e.g., provider type, study region or state, whether the community has a hospital, and the degree of rurality of the community). If significant differences in both response rates and survey responses are found to be associated with such characteristics, the mean responses from the survey will be corrected using propensity scores accounting for sample differences in the predicted propensity to respond to the survey, based on those characteristics.

Revised Burden Estimate

Based on the results of the pilot study and the proposed changes to sampling approach and protocol for contacting the respondents, we have revised the burden estimate (Table 1). The total revised burden estimate for the health care provider survey is 663.4 hours, slightly less than the original burden estimated in the ICR for this component of the project (675.0 hours).

Table 1. Health Care Provider Survey Revised Burden Calculations

Study Phase	Burden Hours Approved			Burden Hours under Proposed Revisions for Main Study
	Number of Providers	Minutes	Total	Number of Providers	Minutes	Total
		Per Case	Hours		Per Case1	Hours
Pilot Study (12 Communities) Completed	24	15	6	64	15	16
Non-Respondent Burden
Respondent Burden (Completed Surveys)	96	30	48	20	30	10
Main Study (138 Communities) Proposed	276	15	69	1237	17	350.5
Non-Respondent Burden
Respondent Burden (Completed Surveys)	1104	30	552	538	32	286.9
TOTAL PROJECT SAMPLE	1500		675	1859		663.4

¹The time allocation for each Completed Survey includes 15 minutes for reviewing the request and reading
enclosed materials and 15 minutes for completing the survey. For the Main Study, 2 additional minutes are
included due to incorporating an email reminder.

Summary

The pilot study results indicate that the survey questions are working effectively and are expected to provide the data needed for analysis. The overall procedure specified in the ICR for the Health Care Provider component is basically sound. However, the response rate is significantly less than anticipated and two changes are proposed to help increase both the number of completed surveys and the overall response rate. The proposed changes are (1) to increase the maximum number of providers per community in the sample from 10 to 32, and (2) to increase the response rate by adding an email follow-up component that will serve as a reminder and provide easy access to the online survey for providers. Implementing these two changes will result in a final data set with the necessary statistical power for analysis, and will maintain an estimated burden below that described in the ICR. Concerns about possible non-response bias will be addressed by testing for significant differences in response rates and survey responses across observed characteristics of the providers and their communities, and if differences are found, correcting for these differences in the estimation using propensity scores.

3