Supporting Statement PartB REVISED 11-19-12

Supporting Statement – Part B
Surveys of Physicians and Home Health Agencies to Assess Access Issues for Specific
Medicare Beneficiaries as Defined in Section 3131(d) of the ACA
CMS-10429, OMB 0938-New
Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling
or other respondent selection method to be used. Data on the number of entities (e.g.,
establishments, State and local government units, households, or persons) in the universe covered
by the collection and in the corresponding sample are to be provided in tabular form for the
universe as a whole and for each of the strata in the proposed sample. Indicate expected response
rates for the collection as a whole. If the collection had been conducted previously, include the
actual response rate achieved during the last collection.

Survey of home health agencies (HHAs). The respondent universe for the survey of
home health agencies includes all home health agencies in the U.S. that served
Medicare beneficiaries in 2010. In order to arrive at an estimate of the size of this
respondent universe, we used available Medicare program data (the 2009 Standard
Analytic File and home health utilization data from 2010-Q2). Home health utilization was
only included if the from date and end date were in 2009.
Using these files, there are 9,228 home health agencies (HHAs) with more than 10
referrals in 2009; these HHAs constitute our sampling frame. Of these, there are 2,727
HHAs where more than half of the beneficiary episodes of care for the agency are for
dual eligibles; this constitutes 29.6 percent of the total number in the universe. Because
of the size of this subgroup, it is not necessary to oversample. The expected yield from a
random sample will be sufficient to conduct subgroup analyses and allow estimates of
proportions of the target population with specific characteristics or behaviors without
applying sampling weights. However, it is straightforward to make population estimates
with a self-weighting sample since each observation would have the same weight. The
weight would be calculated so that the sum of the weights would equal the universe of
the target population found in the CMS claims from which the sample will be drawn.
Table 1.1 on the following page summarizes the survey of HHAs serving Medicare
beneficiaries.

Table 1.1 Survey of Home Health Agencies serving Medicare beneficiaries
1

Universe
Number
(percent)
All HHAs with more than 10
episodes in 2009
Subgroup of interest—HHAs
where more than 50% of
episodes of care delivered
are for dually-eligible
beneficiaries

9,228
(91.6%)
2,727
(29.6%)

Sample

Expected
sample yield

Target response
rate

925
simple random
sample

600

65%

273
expected yield
from SRS

177

Survey of physicians. The respondent universe for the survey of physicians includes
physicians meeting the following criterion: referred at least 25 Medicare beneficiaries for
home health services in 2010 where the beneficiaries are members of the ACA priority
populations, defined as either living in a medically underserved area or dually eligible for
Medicare and Medicaid. For planning purposes, we are using 2009 data and designation
as a Health Professional Shortage Area (HPSA) to proxy Medically Underserved Area
(MUA) status—we find 8,007 physicians who meet this criterion, representing
approximately 2.9 percent of all physicians who referred Medicare beneficiaries for home
health services in 2009. Table 1.2 below summarizes the survey of Physicians serving
Medicare beneficiaries from the ACA priority populations.
Table 1.2 Survey of Physicians serving Medicare beneficiaries from ACA priority
populations
Universe
Number
(percent)
All physicians who refer
277,385 (100%)
Medicare beneficiaries
Physicians who refer 25 or
more beneficiaries for home
health services annually
23,602 (8.5%)
Physicians who refer 25 or
more dually-eligible
beneficiaries for home health
3,189 (1.1%)
services annually
Physicians who refer 25 or
more beneficiaries living in
8,007 (2.9%)
HPSAs (*) annually
Target population for survey—
Physicians who refer 25 or
more beneficiaries who are
8,007 (2.9%)
either dually eligible OR living
in HPSA (*) annually
(*) defined as HPSA with score in top 50%

Sample

Expected
sample yield

Target
response rate

275

60%

no data collection

no data collection

no data collection

no data collection
460
Simple random
sample

2

As described above, these sample specifications were produced using 2009 data, and
with the data we had available at the time, to identify the ACA priority populations. The
actual sample will be drawn from the 2010 data and we will have available information on
the LIS status of the population. LIS status will potentially be used to more broadly define
the ACA priority population. The final algorithm used to select the sample will be based
on the actual distribution of referrals to that population and can be provided to OMB once
it is finalized. The data provided above still represents our best approximation of the final
sample.
2. Describe the procedures for the collection of information including:
- Statistical methodology for stratification and sample selection,
- Estimation procedure,
- Degree of accuracy needed for the purpose described in the justification,
- Unusual problems requiring specialized sampling procedures, and
- Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
The survey’s sample of physicians will be a simple random sample with no stratification;
the sample of HHAs will be stratified by rural-urban status to ensure proportional
representation in rural areas. Medicare data will be used to identify HHAs and physicians;
specifically we will use the home health utilization data to identify Medicare beneficiaries
and link (by beneficiary and date) to the Standard Analytic File to identify physicians who
made home health referrals. Should more recent data from the sources listed above
become available prior to fielding the surveys, the research team will update the sampling
to reflect this prior to conducting field work. We anticipate additional data that will allow a
more refined way to identify the populations of interest —e.g., the availability of Census
tract data on medically underserved areas as well as data identifying beneficiaries who
receive low-income-subsidies. Any new information introduced into the sampling process
will be based on the research conducted about this population by the team pursuant to
the home health study.
Although we will use a simple random sampling method for physicians, we plan to review
the results of this drawing for severe underrepresentation of any group of potential
interest. Checking for severe underrepresentation among the subgroups of interest, after
drawing the sample, will help qualify our analysis and inform any limitations of the data.
The HHA data will be used to make univariate estimates for the entire respondent
population (N=600). This sample of HHAs will yield an estimate that is approximately plus
or minus 2 to 4 percentage points at the .05 level of significance. We also anticipate
being able to make comparisons between two subgroups of interest, though the ability to
detect differences will depend on a number of factors including the sample sizes for each
3

of the two subgroups and where the estimate is in the distribution. The ability to make
these comparisons will also depend on actual sample yield and will not be made for more
than two subgroups at a time. Table 2.1 below shows the main comparisons likely to be
made for the HHAs. If we are comparing two subgroups—for example, with 200 HHAs
serving ACA populations and 400 other HHAs—we will be able to report that a difference
of 9 to 12 percentage points is statistically different.
Table 2.1 Possible comparison groups for survey of home health agencies
Comparison

Location of HHA—Rural vs.
Urban
Ownership—Proprietary vs.
Voluntary/Non-profit/Gov’t
Population served: Primarily ACA
populations vs. Others
Size, no. episodes or revenue—
greater than or less than median

Anticipated
sample size

Detectable difference at 80% power, in
percentage points
True proportions less than
True proportions
20% or greater than 80%
approximately 50%

125 vs. 475

10

14

435 vs. 165

9

13

200 vs. 400

9

12

300 vs. 300

8

12

The physician survey data will be used to make univariate estimates only. The sample of
physicians will yield an estimate that is plus or minus 4 to 6 percentage points at the .05
level of significance. No subgroup comparisons are planned.
The HHA survey will be stratified by rural-urban status whereas the physician survey will
not rely on stratification. Stratification is used if a random sample will not result in a
sufficient number of a given type of cases. However, there are no specific subgroups of
critical interest that are not sufficiently represented in the physician population. It should
be remembered as well that any stratification to increase the yield of one type of case will
decrease the yield of another type of case. Moreover, a stratified sample will be less
efficient, resulting in design effects that decrease effective sample size and require the
calculation and use of sampling weights for analysis.
3. Describe methods to maximize response rates and to deal with issues of non-response. The
accuracy and reliability of information collected must be shown to be adequate for intended uses.
For collections based on sampling, a special justification must be provided for any collection that
will not yield 'reliable' data that can be generalized to the universe studied.
The response rate is affected by a number of factors including the salience and
complexity of the instrument, method and amount of payment, skill and training of
interviewers and procedures for converting non-respondents. The resources allocated
should be sufficient to obtain a response rate ranging from the low 50s to about 60
percent, which is our target.

4

Physicians will receive a prepaid incentive of $50.00; a number of studies have shown
that prepaid incentives increase response rates for physicians. (Because of the potential
problems in providing an incentive payment to an employee, we will not offer incentives
to HHAs.) The literature on monetary incentives for physician surveys is substantial, and
clearly indicates that incentives increase response rates and that higher incentives result
in higher response rates. Table 3.1 below provides references to several studies that
examined the impact of different incentives on response rates, and includes a column
that inflation adjusts these incentives. Both the NCI and the Malin et. al studies tested
incentives with questionnaires that required 15 minutes or less to complete. The CDC
study included a longer survey, since the study is from 1981; while we can infer the
importance of relative incentive payments it is difficult to draw any conclusion from the
absolute size of the incentive.
Table 3.1 Impact of Payment Incentives on Response Rates
Title/Sponsor

Incentive

Incentive
($2012)

Response
Rate

Citation
Center for Studying Health System Change. 2009. “HCS
2008 Health Tracking Physician Survey Methodology
Report”. Technical Publication No. 77. Retrieved from:
http://www.hschange.org/CONTENT/1085/
Keating, N.L., Zaslavasky, A.M., Goldstein, J., West,
D.W., Ayanian, J.Z. 2008. “Randomized trial of $20 versus
$50 incentives to increase physician survey response
rates”. Medical Care. 46(8) 878-881.
Malin, J.L., Rideout, J., Ganz, P.A. 2000. “Tracking
managed care: the importance of a cash incentive for
medical director response to a survey”. American Journal
of Managed Care. 6(11)1209-1214.

Center for Studying
Health System Change

$75
$50

$80.09
$53.39

65%
60%

National Cancer Institute
(NCI), National Institutes
of Health (NIH)

$50
$20

$53.21
$21.28

68%
52%

RAND

$50
$0

$66.52
$0

66%
13%

The Center for Disease
Control (CDC) and
opinion Research
Corporation (ORC)

$50
$25

$126.02
$63.01

77%
69%

Gunn, W.J., Rhodes, I.N. 1981. “Physician response rates
to a telephone survey: effects of monetary incentive level.”
Public Opinion Quarterly. 45:109-115.

RAND

$25
$20

$33.26
$26.61

66%
59%

Collins, R.L., Ellickson, R.D., Hays, R.D., Mccaffrey, D.F.
2000. “Effects of incentive size and timing on response
rates to a follow-up wave of a longitudinal mailed survey”.
2000. Evaluation review. 24(4):347-363.

Additional references on prepaid incentives:
Berk, M., Edwards, W. and Gay, N. “The Use of a Prepaid Incentive to Convert
Nonresponders on a Survey for Physicians,” Evaluation and the Health Professions 16, 2
(1993).
Berk, M., Mathiowetz, N., Ward, E., and White, L. "The Effect of Prepaid and Promised
Incentives: Results of a Controlled Experiment," Journal of Official Statistics 3 (1987).
5

Berk, M. “Interviewing Physicians: The Effect of Improved Response Rate," American
Journal of Public Health (November 1985).
Singer E., Van Hoewyk, J. and M.P. Maher (2000) “Experiments with Incentives on
Telephone Surveys.” Public Opinion Quarterly 64: 171-188.
In addition to the physician incentive, the data collection team will implement a number of
procedures to maximize response rates, including telephone prompting for participants
who fail to complete and return the mailed questionnaire within the designated time
period. During the phone prompt, interviewers will encourage participants to return the
questionnaire by mail or fax and will offer to complete the survey over the phone. Further,
survey packets mailed to respondents will be sent via FedEx or USPS Priority mail in
order to catch the attention of the sample person. The packet will include a clear and
concise cover letter describing the purpose and the policy importance of the survey as
well as instructions for completing and returning questionnaire and a pre-stamped
business-reply envelope. Subsequent follow-up mailings (up to two) will be made using
USPS Priority mail services. Reminder postcards will be sent to those participants who
have not responded or who have misplaced or lost their packets, followed by a second
packet and, if necessary, a third packet. The survey instrument itself has been kept brief
and it will be formatted and printed so as to minimize respondent burden. We will also
provide options for submitting the questionnaire via mail, fax, or over the telephone if
requested.
A nonresponse analysis will be conducted comparing characteristics of responding HHAs
and physicians to non-responders, using those characteristics available from the
sampling frames. We will be able to compare HHAs (responders and non-responders)
with respect to size (measured by episodes or revenue), proportion of population served
(accounted for by ACA priority populations), location (region and rural vs. urban), and
ownership. The data available to compare physician responders and non-responders will
be: (1) specialty; (2) number of home health referrals; and (3) proportion of home health
referrals that are for ACA priority populations. We anticipate that larger HHAs and
proprietary HHAs may be somewhat less likely to respond. For physicians, we expect
that higher income physicians (proceduralists) may be somewhat less likely to respond
than primary care physicians and that those with fewer home health referrals may be
somewhat less likely to respond than those who have a greater interest in home health
care (evidenced by a greater number of referrals). We do not think the level of bias will be
severe.
We note that nonresponse results in bias in survey estimates only to the extent that
nonresponders differ from responders with respect to the analytic variables of interest.
As such, the adjustment corrects for nonresponse only to the extent that responders with
specific characteristics respond like the nonresponders would have responded. In other
words, the nonresponse adjustment assumes that the available variables are correlated
with non-response bias.
6

While we do not think the level of bias will be severe, based on the nonresponse
analysis, we will construct a nonresponse adjustment. For the survey of HHAs, we will
base the nonresponse adjustment on a small number of variables; with only 600 cases in
total, using all of the available variables simultaneously would result in the weights
becoming unstable and highly variable. We will select the two characteristics from those
included in the nonresponse analysis that exhibit the greatest degree of nonresponse for
adjustment. These variables will be used to create sub-groups containing respondents
and non-respondents. Weights will then be calculated based on the proportions in each
sub-group and applied to the respondents to reflect the total sample population.
Comparisons on key variables will be analyzed between the unadjusted and weightingclass adjusted respondents. If clear differences are detected, we will use the adjusted
(weighted) estimates.
Depending upon the variability in the weights once the weights are finalized, we will
determine whether it is necessary to use SUDAAN to account for this variability in
estimating standard errors.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an
effective means of refining collections of information to minimize burden and improve utility.
Tests must be approved if they call for answers to identical questions from 10 or more
respondents. A proposed test or set of tests may be submitted for approval separately or in
combination with the main collection of information.
Throughout the development period, we have consulted with members of the technical
expert panel (TEP) established under a recently completed project addressing the
Section 3131(d) mandate. The TEP was convened to provide expertise regarding the
home health industry and input into how best to identify and measure home health
access issues. The TEP members represented HHAs, national home health care
associations, state and federal agencies, consumer advocacy organizations, home health
physicians, and home health research experts. Our TEP consultations to test the
appropriateness of the survey instrument were with a number of physicians and home
health experts all involved in some way in the planning or delivery of home health
services. TEP members also sought input from colleagues who reviewed the
questionnaire and provided feedback on question wording, response categories, and
overall length.
The research team is also planning to conduct a limited pilot test aimed at ensuring that
questions cover the range of potential issues and use accepted terminology. It is the
team’s experience that even a small number of test cases can reveal any possible
problems in questionnaire wording or flow. This pilot test will be conducted with
approximately 5 to 9 friendly respondents. While additional pre-testing could result in a
somewhat improved instrument, this would be at the expense of other survey activities most likely reducing the level of resources available for phone follow-up and thereby
likely resulting in a lower response rate.
7

5. Provide the name and telephone number of individuals consulted on statistical aspects of the
design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will
actually collect and/or analyze the information for the agency.

Jacob Feldman, PhD, Senior Statistician, Social & Scientific Systems was consulted on
statistical aspects of the design. (phone number: 301-628-0416)

8