0990-0330 SS_B OMHA Final OMB Package _ passback 7 27 18

Supporting Statement for OMB Clearance for
Office of Medicare Hearings and Appeals (OMHA)
Appellant Climate Survey

Part B: Collections of Information Employing
Statistical Methods

May 21, 2018

Carla McGregor

U.S. Department of Health & Human Services

Office of the Secretary
Office of Medicare Hearings and Appeals

2511 Jefferson Davis Highway, Suite 2001

Arlington, VA 22202

Telephone: 571-777-2749

Email: [email protected]

Table of Contents

Part B. Collections of Information Employing Statistical Methods B-1

B.1 Respondent Universe and Sampling Methods B-1

B.2 Procedures for the Collection of Information B-3

B.3 Methods to Maximize the Response Rates and to Deal with Non-Response B-5

B.4 Test of Procedures or Methods to be Undertaken B-7

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting
and/or Analyzing Data B-7

Tables

Table B1. Random Stratified Sample per Year B-3

Table B2. Total Number of Appellants OMHA Contacted Given an 80% Response Rate B-3

Table B3. Research Team Contact Information B-7

Appendices

Appendix B1. Appellant Climate Survey-English

Appendix B2. Appellant Climate Survey-Spanish

Appendix C1. Introductory Letter-English

Appendix C2. Introductory Letter-Spanish

Appendix C3. Reminder Phone Script-English

Appendix C4. Reminder Phone Script-Spanish

Appendix C5. Reminder Letter-English

Appendix C6. Reminder Letter-Spanish

2. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1 Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

B.1.a Sampling

The purpose of this study is to improve the service that the Office of Medicare Hearings and Appeals (OMHA) provides to its appellants. The survey will gauge appellants’ satisfaction with this service along with the overall appeals experience. By identifying areas of success as well as areas for improvement, OMHA will be able to continuously improve its existing processes. The information obtained could lead to reallocation of resources, revisions in certain agency processes, and development of guidance related to the agency’s customer services. OMHA will collect information from two appellant types: (1) individuals disagreeing with and appealing payment decisions regarding Medicare coverage (beneficiaries), and (2) providers or suppliers appealing payment decisions regarding Medicare coverage for items and services furnished to beneficiaries (non-beneficiaries).

The potential respondent universe includes a total of approximately 6,400 appellants compiled from OMHA administrative records of appeal filings. Each appellant may have filed one or multiple appeals during the fiscal year measured.¹ The sample for this survey was selected to include only appellants with a “closed” appeal during the first 6 months of the fiscal year (October through March), resulting in a sample of approximately 3,200 appellants. OMHA is focusing on a non-redundant appellant universe so that individuals or entities filing one or two appeals are represented in the sample as well as large entities that account for the vast majority of appeals (i.e., providers or suppliers, who file approximately 82 percent of appeals; typically, a single provider or supplier files multiple appeals throughout the year) are included.

Based on initial analysis of the population, there are roughly 3,200 non-redundant appellants whose cases have been closed in the first 6 months of the fiscal year. The focus of this study is on appellant satisfaction with the existing process, and therefore it is important to treat each appellant—regardless of type—as an individual entity.

B.1.b Expected Response Rates

OMHA expects to achieve a response rate of 80 percent or higher for the Appellant Climate Survey. Assuming that 80 percent respond to the Appellant Climate Survey, as expected, the total estimated sample size needed is 500 to ensure there are 400 respondent appellants. The sample size of 500 includes an expected 400 respondents and 100 non-respondents. The resulting respondent sample will include approximately 200 beneficiaries and 200 non-beneficiaries given that the sample will be selected so that there are 250 beneficiaries and 250 non-beneficiaries.

B.2 Describe the procedures for the collection of information including:

B.2.a Statistical methodology for stratification and sample selection

In order to ensure that the appellant sample is representative of the target universe described in the previous section, OMHA randomly stratifies the appellant sample by appellant type and Medicare part. An equal number of appellants will be sampled from each appellant type. The appellant sample will also be selected so that the number in each Medicare stratum is reflective of population proportions of Appellants by Medicare Part. There are five types of Medicare Parts: (1) Part A (hospital insurance), (2) Part B (medical insurance), (3) Part C (private health insurance plans), (4) Part D (prescription drug coverage), and (5) Other (a combination of Income Related Monthly Adjustment Amounts and “Entitlement” Hearings). Given the percentage distribution of the entire population by these factors, a final stratified random sample includes the number of completed surveys listed below (see Table B1). A total sample of 500, including beneficiaries and non-beneficiaries (i.e., providers and suppliers), will be recruited each year (2018, 2019, and 2020) for participation in the survey. Given an 80 percent response rate, we expect to obtain completed interviews or questionnaires from a total of 400 Appellants (Table B2).

Table B1. Random Stratified Sample per Year²

		Appellant Type
		Beneficiary	Non-Beneficiary	Total
Medicare Part	A	23	103	126
	B	78	119	197
	C	105	2	107
	D	41	24	65
	Other	3	2	5
	Total	250	250	500

Table B2. Total Number of Appellants OMHA Contacted Given an 80 Percent Response Rate

		Appellant Type
		Beneficiary	Non-Beneficiary	Total
Medicare Part	A	18	82	100
	B	63	95	158
	C	84	2	86
	D	33	19	52
	Other	2	2	4
	Total	200	200	400

B.2.b Estimation procedure

Data collected from the sample of appellants will be used to produce estimates that are representative of all appellants in the target universe. Sampling weights which account for the stratified sampling design and adjust for potential nonresponse and coverage errors will be used to calculate estimates and standard errors and conduct hypothesis testing.

B.2.c Degree of accuracy needed for the purpose described in the justification

OMHA derived its total sample size to ensure adequate statistical power for calculating estimates representative of the target universe. OMHA will select a sample of 500 appellants and expects to obtain data from 400 respondents. This sample is expected to produce representative estimates for the overall sample (all 400 respondents), with 95 percent confidence intervals of +/- 5 percentage points or less depending on the outcome. For subgroup analyses, occurring at the beneficiary and non-beneficiary levels, the sample is expected to produce estimates with 95 percent confidence intervals of +/- 7 percentage points or less depending on the outcome.³

B.2.d Unusual problems requiring specialized sampling procedures

OMHA does not anticipate unusual problems that require the usage of specialized sampling procedures.

B.2.e Any use of periodic (less frequent than annual) data collection cycles to reduce burden

OMHA does not plan to use data collection cycles that are less frequent than annual. Data collection will be conducted annually.

B.2.f General data collection procedures

A new sample is drawn in each survey cycle. Appellants may be randomly selected in one or more of the years that the survey is conducted. A stratified random sample based on the two factors identified above—appellant type and Medicare Part—is used to obtain the most representative data possible.

The data collection process will occur in the following manner:

1. At the beginning of each survey cycle, OMHA obtains a list of all unique appellants who have had an appeal closed during the first 6 months of the fiscal year. The sample will be stratified by appellant type (beneficiary and non-beneficiary) and Medicare Part (Part A, Part B, Part C, Part D, and Other). A total sample size of 500 will be randomly selected, including estimated sample sizes of 250 beneficiaries and 250 non-beneficiaries.

2. Appellants selected for participation in the Appellant Climate Survey will be notified of the study by mail (Appendices C1 and C2) and provided information necessary to complete the web survey. These letters contain official signatures and labels to verify the organization’s intent and professionalism.

3. If the appellant does not complete the web survey within 1 week, appellants may be contacted up to 14 times by telephone (Appendices C3 and C4) and receive up to 3 voicemails (Appendix C3 and C4) as reminders and encouragement to complete the Appellant Climate Survey via the web. The Appellant Climate Survey may be completed by telephone during a reminder telephone call attempt, depending on the appellant’s preference and availability.

4. Appellants who do not complete the survey after 1 month of call attempts will be sent a reminder letter (Appendices C5 and C6) that will include a hard copy of the Appellant Climate Survey and a return envelope with prepaid postage. The letter will encourage the appellant to complete the hard copy version of the Appellant Climate Survey and return the survey by mail.

5. Telephone and web-based responses will be stored electronically and downloaded into a Microsoft Excel or SPSS database. Mailed responses will be manually entered into the database.

B.3 Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Overall response rate projections are presented above in section B.1.b. Achieving the specified response rates involves using procedures described below to secure participation once the potential respondents have been identified. The Appellant Climate Survey is designed to achieve the highest practical rates of response, commensurate with the importance of survey uses, time constraints, respondent burden, and data collection costs. In the event that an 80 percent response rate is not obtained, a non-response bias analysis will be conducted.

The recruitment procedures designed to maximize the number of sampled appellants who complete the Appellant Climate Survey are described below.

• The introductory letters inviting appellants to participate in the Appellant Climate Survey will be carefully developed to emphasize the importance of this study and how the information will help OMHA improve the appeals process, and to maximize cooperation through advance warning. In addition, letters will be placed on official U.S. Department of Health and Human Services (HHS) OMHA letterhead signed by the Chief Administrative Law Judge.

• A toll-free number and study email address will be provided so appellants can receive assistance with the survey. English and Spanish speakers will be available to respond to questions and concerns.

• Sampled appellants will have the option of completing the Appellant Climate Survey as a web, telephone, or mail-in survey.

• A training for telephone interviewers will be conducted. The training, specific to this study, will include an overview of the project, a review of the research questions the study will address, a primer on interviewing practices and procedures, and techniques for encouraging respondent candor.

• A core set of interviewers with experience conducting telephone interviews, particularly interviewers who have proven their ability to obtain cooperation from a high proportion of sample members, will be employed.

• Follow-up attempts will be made by telephone with all sampled appellants who do not complete the survey 1 week after the start of data collection. The primary purpose of the call will be to urge them to complete the survey. At that point, if the appellants prefer to complete the survey or remaining sections of the survey over the telephone, an interviewer will administer the full survey or any remaining parts of the survey over the telephone.

• Call scheduling procedures that are designed to call numbers at different times of the day (between 8 a.m. and 6 p.m.) and days of the week (Monday through Saturday) will be used to improve response rates.

• A second mailed reminder will be sent to appellants who have not yet completed the survey after the first month of data collection. All letters will include official HHS OMHA letterhead signed by the Chief Administrative Law Judge to increase study credibility.

B.4 Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The data collection instruments were reviewed for understandability (e.g., confusing wording or layout, questions that were difficult to comprehend or respond to) by a total of four OMHA stakeholders. Instruments were revised, as needed, to incorporate feedback provide by the OMHA stakeholders. All data collection materials are reviewed by OMHA staff prior to the start of data collection in an effort to minimize respondent burden and improve utility. This study is a continuation of previous data collection and utilizes previously tested instruments.

B.5 Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The Contractor, 2M Research, will conduct the Appellant Climate Survey in 2018. The contact information provided below may change depending on the Contractor selected to administer the survey in 2019 and 2020. See Table B3 for current study contact information.
Table B3. Contact Information

Name	Affiliation	Telephone Number	E-mail
Renée A. Johnson	HHS OMHA	(571) 777-2741	[email protected]
Douglas Sahmel	HHS OMHA	(571) 777-2733	[email protected]
Paul Ruggiere	Project Manager, 2M Research Services	(817) 856-0871	[email protected]
Nick Beyler	Corporate Officer in Charge, 2M Research Services	(202) 796-3955	[email protected]
Maxwell Matite	Associate Project Manager, 2M Research Services	(817) 856-0891	[email protected]
Amy Wieczorek	Data Collection Lead, 2M Research Services	(817) 856-0866	[email protected]
James Murdoch	Statistician/Programmer, 2M Research Services	(817) 856-0869	[email protected]
Peyton McGee	Statistician/Programmer, 2M Research Services	(817) 856-0877	[email protected]

1 The actual number of appeal cases submitted each fiscal year varies; thus, the sample universe will fluctuate each fiscal year but should be similar to 6,400.

2 As in previous OMHA Appellant Climate Survey data collection efforts, an additional sample may need to be added if response counts are lower than expected.

3 The precision calculations assume a binary outcome of 50 percent (most conservative assumption) and a design effect due to weighting of 1.05.