CMS-10197 PRA supporting statement 3-24-10 REDO_am

Section 302(b) of The Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (Public Law 108-173) (MMA) requires the Centers for Medicare and Medicaid Services (CMS) to begin a program of competitive bidding for Durable Medical Equipment, Prosthetics, Orthotics and Supplies (DMEPOS) in 10 Competitive Acquisition Areas (CAAs) in 2007. MMA Section 303(d) requires a Report to Congress evaluating the program. Baseline data for the Report to Congress were collected in 2007. On July 15, 2008, the Medicare Improvements for Patients and Providers Act (MIPPA) amended section 1847 of the Social Security Act to make certain changes to the Medicare DMEPOS Competitive Bidding Program. Section 154(a) of MIPPA delayed competition under the program. MIPPA also postponed the Report to Congress on the program evaluation to July 1, 2011. In mid-2009 CMS established a new timeline for the program, which is now scheduled for implementation January 1, 2011. In the meantime, Medicare payment rates for DMEPOS have been reduced and suppliers are now required to attain accreditation in order to continue serving Medicare beneficiaries. As a result of the delay in the effective date of the competitive bidding program, data collected for the evaluation project in 2007 are obsolete and not usable as a baseline against which to measure impacts of the competitive bidding program. We are submitting this information request as a revision of the original one (OMB number 0938-1015, submitted January 3, 2007, and approved April 4, 2007) due to several changes in the survey design and information collection forms. We received several comments on the revision, and so this document reflects modifications we have made to the data collection plan following those comments; thus the current document is submitted for a second, 30-day public comment period.

To collect information for the Report to Congress, we will use the following three (3) data collection methods: beneficiary surveys; focus groups with suppliers and referral agents; and key informant discussions with beneficiary advocates, CMS officials and contractors, referral agents and suppliers. This data collection will take place in a subset of the first nine (9) CAAs and in comparison areas not transitioning to competitive bidding. To measure change before and after the implementation of the DMEOS competitive bidding program, beneficiary surveys will take place before program implementation, and again at least one year after implementation; focus groups will take place before program implementation and again at least one year after implementation; and key informant interviews will take place before implementation and during three additional waves after implementation.

Our 2007 baseline data collection included a supplier survey. We have eliminated the supplier survey from the evaluation plans due to low response rates and technical problems encountered in collecting information on the models of equipment provided to beneficiaries.

Part B: Justification

B.1 Explanation of Circumstances That Make Collection of Data Necessary

Section 302(b) of the MMA requires CMS to begin a program of competitive bidding for DME, supplies, certain orthotics, and enteral nutrients and related equipment and supplies. Following the legislative action of 2008, the bidding program is scheduled to be implemented in 9 CAAs in 2011, an additional 70 CAAs after competition in those areas occurs in 2011, and additional CAAs thereafter. The initial CAAs will be based in 9 of the largest Metropolitan Statistical Areas (MSAs).

MMA Section 303(d) requires a Report to Congress on the program, covering program savings, reductions in cost sharing, impacts on access to and quality of affected goods and services, and beneficiary satisfaction. This project’s purpose is to provide information for this Report to Congress. Due to substantial legislative and regulatory delays in program implementation, the Report to Congress in 2011 will be released just as the program is being implemented, and before the evaluation is complete. This project will continue after the Report to Congress, to evaluate the impact of the program on beneficiaries, on Medicare costs, and on changes in the Medicare DMEPOS market. In addition to evaluating the impact of the program in the first 9 CAAs, interim site visit information and other data collected in the course of this project may be helpful to CMS as it makes plans to implement the second phase (adding 70 large CAAs).

This evaluation’s objectives are both summative (is the program having identifiable impacts on costs, access, and quality?) and formative (what early implementation lessons learned can improve the bid and award process as the program expands from 9 CAAs in 2011 to 70 in the second phase?).

Copies of Section 302(b) and 303(d) of the MMA, and relevant sections of the MIPPA, can be found in Appendix A.

B.2 How the Information Will Be Collected, by Whom, and For What Purpose

The following data will be collected to support a pre-post evaluation design comparing the first CAAs with other (comparison) areas:

Beneficiary survey in four (4) study CAAs and three (3) comparison areas.
Focus groups with suppliers and referral agents in four (4) study CAAs.
Key informant discussions with beneficiary groups or advocates; CMS officials and CMS’s bidding contract managers; referral agents; and suppliers in the four (4) CAAs.
Key informant discussions over the telephone with beneficiary groups or advocates, referral agents, and suppliers in the three (3) comparison areas.

In this submission, we are requesting Office of Management and Budget (OMB) approval for the data collection instruments and materials pertaining to the beneficiary survey, focus groups and key informant discussions. Please see Exhibit 1 for the research questions each data source will address.

Exhibit 1: Data Sources to Answer Key Research Questions

Domain	Research Questions	Medicare Claims	Beneficiary Surveys	Focus Groups & Discussions	Program Data
Medicare expenditures & savings	Unit prices				x
	Utilization	x
	Program Costs				x
Beneficiary cost-sharing	Estimated Savings in beneficiary out-of-pocket costs	x	x
Access	Number and geographic accessibility of suppliers	x	x	x
	Finding a supplier		x	x
	Transition issues	x	x	x
Quality of goods and services	Quality of service		x	x
Quality of goods and services	Product quality		x	x
Consumer satisfaction	Consumer satisfaction		x
Administrative operations & stakeholder impacts	Program dissemination and outreach			x
	Bidding process			x	x
	Transition issues			x
	Quality standards			x

Statistical Data Collection

Prior to the delay in program implementation, in 2007 Abt Associates conducted a baseline (pre) evaluation survey of beneficiaries using five types of DME equipment, as well as a baseline survey of DMEPOS suppliers. The collected data are no longer useful for baseline evaluation purposes because too much time will have elapsed between 2007 and the revised program implementation date of 2011. The 2007 survey, while not useful for the evaluation due to the lengthy delay, was informative and is being used to redesign the evaluation data collection.

Response to the 2007 supplier survey was unacceptably low and led to a biased sample. Several of the large national DME supplier chains decided not to participate, and instructed their local outlets/franchises not to participate. As a result, the few respondents were all ‘independent’ suppliers and did not reflect the industry as a whole. The low response rate was due in part to controversy around Congress’s decision to implement competitive bidding for Medicare home medical equipment, with most supplier groups strongly opposed. Because attitudes about the competitive bidding program in the supplier community have not changed during the two years since that earlier survey attempt, we believe that another survey would yield the same biased results. Such a survey would therefore not be useful for a national evaluation, and we have removed the supplier survey from the evaluation design. We will instead obtain supplier insights about the program through focus groups and interviews (described below).

The beneficiary survey, however, must be repeated in 2010 (pre), and in 2012 (post) and is essential for the program evaluation. The beneficiary survey will focus on the users of five types of DMEPOS: oxygen, hospital beds, powered mobility devices, walkers, and continuous positive airway pressure (CPAP) machines. We have used the 2007 response rates in calculating sample sizes for the 2010 and 2012 beneficiary surveys.

CMS administrative data sources (including Medicare DMEPOS claims data and other administrative data), and supplier bids, will be analyzed to address other research questions. Abt Associates convened a Technical Expert Panel (TEP) to provide guidance on some aspects of the evaluation, specifically on how to assess the impact of competitive bidding on beneficiaries who are oxygen users.

Non-Statistical Data Collection

In 2007 focus groups were conducted with referral agents and DMEPOS suppliers, as were in-depth interviews with a variety of stakeholders. The information collected is no longer useful for evaluation purposes for the same reasons that we find the 2007 survey data no longer useful—that is, too much time will have elapsed between data collection and program commencement. The evaluation design calls for baseline focus groups and interviews. Therefore, in mid-2010 we will conduct the baseline data collection activities, in mid-2011 we will conduct interim interviews with key informants, and in mid-2012 we will conduct follow-up data collection activities. We will also conduct interviews shortly after the competitive bidding program goes into effect in January 2011 (see further discussion of this wave of data collection later in this section).

The key informant and focus group recruits (referral agents and suppliers) will be selected to meet different data collection aims. Referral agents included in the key informant discussions and focus groups will be selected based on extensive experience and their active involvement referring Medicare beneficiaries to DMEPOS suppliers. Referral agent focus group participants will be selected from different health system types (e.g., urban, rural, public, private) and roles (e.g., discharge planner, physical therapist) to provide a diversity of perspectives.

The suppliers for focus groups and key informant discussions will be selected for their involvement and familiarity with the local DMEPOS market and years of experience in the selected metropolitan areas. The supplier focus group participants will include suppliers with a significant part of their business providing at least one of the DMEPOS items chosen for competitive bidding. In the 2010 baseline focus groups, suppliers will not yet know whether their bids are successful; in subsequent follow-up focus groups and interviews, we will focus mainly on those that are successful and continue serving Medicare beneficiaries.

The referral agent focus group participants may include discharge planners, case managers, social workers, home health care workers, physical therapists and nurses, and potentially other professionals who refer patients to DMEPOS suppliers. The aim is to select referral agents from a diverse array of health care organizations. The referral agent and DMEPOS supplier focus group participants will be asked to assess how the competitive bidding program affected them as well as their perceptions about the impact of competitive bidding on Medicare beneficiaries.

In mid-2010, mid-2011 and mid-2012, key informant discussions will be conducted with beneficiary groups/advocates, CMS officials and bidding program managers, referral agents, and suppliers. The transition to competitive bidding in nine communities on January 1, 2011, will be the first of several transitions; 70 additional communities will undergo a similar transition two years later, and many more after that. CMS plans an extensive outreach and education campaign during the first transition in 2011 and the evaluation will team will be collecting information about the level of effort and volumes of public education contacts to the various audiences and stakeholders in order to round out the description of the public education campaign. Much can be learned about the success of these efforts in 2011 to inform future transitions. We have therefore added key informant interviews with patient advocates, CMS officials and other stakeholders during the immediate transition period in early 2011.

Individuals selected for key informant discussions may represent beneficiary and industry perspectives in the CAAs, the comparison areas, or at the State or Federal levels. The information gathered through the key informant discussions will provide a deeper understanding of program impacts from individuals who are highly engaged with the DMEPOS market and have extensive experience in their roles.

Trained researchers experienced in moderating focus groups and conducting key informant discussions will collect the non-statistical data.

B.3 Use of Improved Information Technology to Reduce Burden

The self-administered pencil-and-paper beneficiary surveys may alternatively be administered by telephone upon respondent request. Telephone data collection is also used to follow up with non-respondents. The telephone data collection is supported by a full-featured computer-assisted telephone interviewing (CATI) system with automated case delivery and built-in quality control. We expect 30 percent of sampled beneficiaries will use the CATI system to respond to the survey.

B.4 Efforts to Identify and Avoid Duplication

CMS has planned two kinds of data collection among beneficiaries for purposes of evaluating the new national DMEPOS competitive bidding program. One data collection mailed by the competitive bidding implementation contractor (CBIC) addresses beneficiary satisfaction along several dimensions with a short one-page survey (OMB # 0938-1016). The format of the questions is a rating scale rather than specific response categories and the questions are designed to elicit beneficiary experiences, regardless of the type of DME equipment used. While the sample size eventually will be large, the information gathered will be brief and more general than the program evaluation survey. This is in line with the purpose of the beneficiary satisfaction survey, which is: to monitor regularly the beneficiary experience under the program after it gets underway, to identify suppliers who may have difficulties serving beneficiaries adequately, and, potentially, to identify overall program issues that may arise from time to time.

In contrast, the evaluation survey uses a pre-post design in both CAAs and comparison areas. The purpose of the evaluation survey is to detect possible changes in beneficiary cost sharing, access, quality of goods and services and customer satisfaction as a result of the new program. Questions are structured with specific sets of response categories that address issues in beneficiary cost sharing, access, beneficiary choice of equipment, service quality, training, and customer support. Response categories are also specific to the type of DME each respondent uses.

To avoid duplication, data collection instruments were developed to ask respondents only about information that cannot be found in extant databases. We have identified the following as extant data sources that will be used in this evaluation:

CMS databases, including DMEPOS claims data and other administrative data.
National Supplier Clearinghouse (NSC) file data.

B.5 Efforts to Minimize Burden on Small Business or Other Entities

There will be no survey data collected from small businesses. Small businesses or other small entities may constitute a portion of the respondents in the supplier focus groups and key informant discussions. As we explained on page 3 (Section B.2), we have dropped a supplier survey from the original design. However, we do require input from suppliers; this will be obtained through focus groups and discussions with key informants. The supplier focus groups will take up to 90 minutes and the key informant discussions will take up to 60 minutes. Focus groups will be held in the locality where participants work, and key informant discussions will take place via telephone, to minimize inconvenience for participants.

B.6 Consequences of Less-Frequent Data Collection

Neither the statistical data (the beneficiary survey) nor non-statistical data (the focus groups and key informant discussions) collected during this evaluation can be collected less frequently. First, the beneficiary and supplier surveys must be administered at two (2) points in time, to allow a measurement of the competitive bidding program’s effects in a pre-post analysis. The non-statistical data collection will be conducted at four (4) time points, the first prior to program implementation, the second during the immediate transition period in early 2011, the third 3 to 6 months after implementation, and the fourth approximately 12 months after program implementation. Non-statistical data must be collected this frequently to allow sufficient understanding of competitive bidding program impacts from the perspective of the key stakeholders at these important implementation times.

B.7 Special Circumstances Requiring Collection of Information in a Manner Inconsistent with Section 1320.5(d)(2) of the Code of Federal Regulations

There are no special circumstances associated with this data collection.

B.8 Federal Register Comments and Persons Consulted Outside the Agency

The 30-day Federal Register Notice for the current submission was published [OSORA/PRA WILL INSERT DATE].

The current submission follows a PRA approval we received in 2007. The data collection package was approved on April 4, 2007 (OMB number 0938-1015). For the original request, CMS submitted a notice to the Federal Register announcing the agency’s intention to request an OMB review of data collection activities. The notice was published on June 16, 2006, in Volume 71, Number 116, Page 34928 and provided a 60-day period for public comments. A copy of the Federal Register notice for this information is included in Appendix E. Comments were received from Abbott Laboratories and a response was issued. Modifications to the instruments and evaluation design were incorporated into the resubmission. We then provided for a 30-day public comment period which led to the April 4, 2007, OMB approval.

On December 18, 2009, we published a revised PRA for 60 day public comment, and several comments were received. This submission responds to those comments, and is for a second 30-day public comment period.

Because the current submission is a modification of the original one approved by OMB in 2007, we present below the original persons consulted.

This study’s design phase involved convening and consulting with a TEP that provided and continues to provide essential information related to the design of the data collection instruments. The TEP consists of six (6) experts, including:

A former DMERC medical director.
An individual with expertise in research design.
A pulmonologist.
Two (2) oxygen equipment industry representatives.
An expert on mobility products.

The first TEP teleconference was held on November 29, 2005, to discuss issues relevant to oxygen therapy, competitive bidding, and the structure of the beneficiary and supplier surveys. Follow-up meetings were held on December 9, 2005, and January 9, 2006, to continue refining individual survey items. Subsequent contacts occurred with individual TEP members for advice and consultation before we finalized the original survey instruments.

Exhibit 2 contains a complete listing of the TEP members who participated in discussions about the 2007 surveys, and their affiliations.

Exhibit 2: Technical Expert Panel Members

Kent Christopher, PhD, 9086 East Colorado Circle, Denver, CO, 303-337-8080

Doran Edwards, PhD, Advanced Healthcare Consulting, LLC, 803-865-9225

Laraine Forry, LMF Consulting Services, LLC, 717-579-2638

Bruce Friedman, PhD, Dept of Community and Preventative Medicine, University of Rochester, Rochester, NY, 585-273-2618

Joseph Lewarski, Inogen, Eastlake, OH, 440-269-8046

Jean Minkel, Minkel Consulting, New Windsor, NY, 845-496-5022

In 2009 TEP members were asked to review the revised questionnaires; additional comments were provided by Laraine Forry, Kent Christopher, Bruce Friedman and Doran Edwards.

B.9 Payments to Respondents

Honoraria will be provided to the following respondents:

Focus groups: focus groups will be conducted with referral agents (e.g., discharge planners, rehabilitation therapists, home care staff) and with suppliers and/or supplier group representatives. The focus groups will be held in the evening and participants will be offered dinner and a $75 honorarium.
No honoraria will be offered to participants in the in-depth interviews.
No honoraria will be offered to beneficiaries who complete the survey.

B.10 Assurance of Confidentiality

We pledge confidentiality to the extent provided by law. Having said that, Abt Associates has conducted numerous projects involving sensitive information; consequently, facilities and procedures have been developed to maintain this confidentiality. To begin, all staff assigned to Abt projects sign confidentiality agreements. In addition, access to the data processing areas is controlled, with only authorized personnel allowed in the computer rooms and the computer tape libraries. All databases are password protected, with only the data administrators having write authority over files. Finally, electronic data transferred via diskette or CD-ROM to clients are aggregated, encrypted, and password-protected before shipping via a bonded courier.

Abt’s wholly owned subsidiary, Abt-SRBI, will conduct the beneficiary survey. Abt-SRBI uses the Confirmit Suite of products to conduct both Confirmit Fusion CATI (Computer Assisted Telephone Interviewing) and Confirmit CAPI (Computer Assisted Personal Interviewing) projects. The Confirmit suite of products is an integrated application that allows Abt to design surveys and manage sample and interview quotas. The Suite has robust sample management and advanced quota control, and also handles multilingual interviewing. The Confirmit Suite allows the centralized distribution of CATI surveys across interviewer desktops via Abt’s Wide Area Network. It also permits centralized sample management and case distribution.

The focus groups and key informant discussions will not be audiotaped or videotaped. The written notes collected from the focus groups and discussions will be labeled with a study code and will not include the name of the participant or his/her organization. The materials will be stored in secured locations and electronic materials will be stored in a password–protected computer. Any information presented to outside parties will be presented in summary form. The informed consent forms for the focus groups and key informant discussions conducted in-person and the informed consent scripts for the key informant discussions conducted over the telephone are provided in Appendix D.

B.11 Questions of a Sensitive Nature

The beneficiary survey does not contain any questions concerning sexual behavior and attitudes, or religious beliefs. The beneficiary survey does include key demographic items such as income, race and ethnicity, and health status. Collecting these key demographic data will allow us to determine whether samples in the program and comparison areas are comparable.

Respondents to the beneficiary surveys will be explicitly informed that their participation is voluntary, and that they may choose to withdraw from the study or to omit specific items without penalty. No written consent is obtained from the beneficiaries; their consent is implied by the completion of their surveys.

Participants in the focus groups and key informant discussions will be explicitly informed that their participation is voluntary, and that they may choose to withdraw from the study or refuse to answer any question, without penalty. Written consent will be obtained from participants for the in-person discussions and focus groups, and verbal consent will be obtained from participants for the telephone discussions (see Appendix D).

All consent forms and procedures have been reviewed by the Abt Associates Institutional Review Board, which will conduct continuing reviews each year during the period of this study.

B.12 Estimates of Respondent Burden

Exhibit 3 presents estimates of the annual reporting burden for each wave of the data collection process. Time estimates are based on experience with similar instruments used with comparable respondents. This burden estimate includes additional qualitative data collection during the immediate transition period in early 2011.

Exhibit 3: Estimates of Annual Burden Hours and Cost

Data Collection Sources	Number of Respondents	Minutes Per Respondent	Response Burden in Hours	Estimated Cost Per Hour^a	Costs per Respondent	Total Burden (Costs)
STATISTICAL DATA – HOUSEHOLDS
Completed Beneficiary Surveys in Program and Comparison Areas
Oxygen Users	3,333	30	1,667	$0.00	$0.00	$0.00
Hospital Bed Users	1,150	30	575	$0.00	$0.00	$0.00
Walker Users	1,450	30	725	$0.00	$0.00	$0.00
Power Mobility Device Users	1.109	30	554	$0.00	$0.00	$0.00
CPAP Users	1,294	30	647	$0.00	$0.00	$0.00
TOTAL	8,336		4,168			$0.00
NON-STATISTICAL DATA – PRIVATE SECTOR
Focus Groups
Referral Agent	40	90	60	$31.31	$46.96	$1,878.60
Suppliers	40	90	60	$46.88	$70.32	$2,812.80
TOTAL	80		120			$4,691.40
NON-STATISTICAL DATA – PRIVATE SECTOR
Key Informant Discussions
Beneficiary Group/ Advocates	10	60	10	$16.83	$16.83	$168.30
Referral Agent	8	60	8	$31.31	$31.31	$250.48
Suppliers	8	60	8	$46.88	$46.88	$375.04
Comparison Area Key Informant Discussions
Beneficiary Group/ Advocates	6	60	6	$16.83	$16.83	$100.98
Referral Agent	6	60	6	$31.31	$31.31	$187.86
Suppliers	6	60	6	$46.88	$46.88	$281.28
TOTAL	44		44			$1,363.94
NON-STATISTICAL DATA – FEDERAL GOVERNMENT
Key Informant Discussions
CMS/Contractor Officials	10	60	10	$0.00	$0.00	$0.00
TOTAL	10		10			$0.00

a Costs for beneficiaries assume that virtually all will be retired, and thus would incur no loss of wages.

Suppliers’ hourly wage estimate based on the targeted population of supply administrators (management level), and supervisors of claims-processing and sales staff. These annual salaries can range from $75,000-$120,00 or an hourly range of $36.06-57.69; estimate is based on the average of these two hourly wages.

Costs for referral agents are based on an average Registered Nurse hourly wage of $31.31 (Bureau of Labor Statistics, 2008).

Interviews with beneficiary advocates and CMS officials are especially important, both to measure impact and to identify issues specific to the transition. These interviews will be distributed as follows:

Wave 1 (mid-2010): 8 interviews with beneficiary advocates; 5-6 in the local areas where case studies will be conducted, and 2-3 with national organizations. 10 interviews with CMS officials, including Regional Office staff in regions containing the first competitive bidding areas.

Wave 2 (early 2011): 16 interviews with beneficiary advocates in the midst of the transition, especially national advocacy groups; and 10 interviews with CMS officials including the DMEPOS Ombudsman.

Wave 3 (mid-2011): 8 interviews with beneficiary advocates; 5-6 in the local areas where case studies will be conducted, and 2-3 with national organizations. 10 interviews with CMS officials, including Regional Office staff in regions containing the first competitive bidding areas.

Wave 4 (mid-2012): 8 interviews with beneficiary advocates; 5-6 in the local areas where case studies will be conducted, and 2-3 with national organizations. 10 interviews with CMS officials, including Regional Office staff in regions containing the first competitive bidding areas.

B.13 Estimates of the Cost Burden to Respondents

Other than their time to participate in the study, shown in Exhibit 3, there are no direct monetary costs to survey respondents and other participants.

B.14 Estimates of Annualized Government Costs

Below are the annual costs for the evaluation of the restarted national DMEPOS competitive bidding program, from 2010 through 2013. These costs include labor, other direct costs (printing, postage, travel, telecommunications, etc.) and indirect costs.

Exhibit 4: Annual Costs for the Evaluation of the National DMEPOS Competitive Bidding Program

Task	FY2010	FY2011	FY2012	FY2013	Total	Annual Average
Prepare for Survey	$33,262	$0	$13,335	$0	$46,597	$11,649
Qualitative Data Collection	$258,248	$118,104	$225,195	$0	$589,547	$159,387
Conduct Surveys	$270,964	$0	$264,796	$0	$535,760	$133,940
Assemble and Analyze Secondary Data	$0	$86,938	$99,033	$0	$185,971	$46,493
Reports	$26,922	$113,437	$57,145	$95,151	$292,655	$73,164
Data Tapes and Documentation	$1,055	$0	$0	$9,000	$10,055	$2,514
Analysis	$32,764	$80,083	$190,099	$0	$302,946	$75,736
TOTAL	$623,215	$398,562	$849,603	$104,151	$1,963,531	$502,883*

*Average Annual Cost = ($623,215 + $398,562 + $849,603 + $104,151)/4 = $502,883

B.15 Changes in Hour Burden

The initial OMB submission for this study included a survey of suppliers, which was conducted once (unsuccessfully) and will not be repeated. The burden for suppliers is thus diminished.

The initial OMB submission for this study included two waves of beneficiary surveys, one of which was conducted in 2007 in three CAAs and two comparison areas but must be repeated in 2010 due to substantial program delays. To provide for improved sample sizes, we will add two more survey areas (one CAA and one comparison area) for the remainder of the survey waves. Thus burden is increased for beneficiaries by the addition of two more survey areas and decreased due to lower response rate; combined, these cause a net decrease in burden.

The initial OMB submission for this study included three waves of qualitative data collection, two involving focus groups and all three involving key informant interviews. The first wave of qualitative data collection was conducted in 2007 but must be repeated in 2010 due to substantial program delays. In addition, we will conduct interviews with patient advocates and CMS officials during the transition in early 2011. Burden is increased by adding two more sites for nonstatistical data and additional interviews during the transition.

B.16 Time Schedule, Publication, and Analysis Plan

Described below are the analysis plans and timelines for both the statistical and non-statistical data collection efforts.

Statistical Data Collection

Analysis Plan: Beneficiary Surveys

The basic framework for the analysis of the beneficiary survey data will be “differences-in-differences,” in which we compare changes in key outcome variables in four (4) study CAAs with changes in the same variables in three (3) selected comparison areas.

The report will contain five (5) sets of analyses:

Analysis of response rates.
Descriptive tables presenting the survey data.
Univariate analyses of key independent variables (such as beneficiary demographic characteristics) by program area to determine whether samples in selected program and comparison areas are comparable.
Univariate analyses of outcome variables and univariate tests of demonstration effects.
Multivariate analyses of outcome variables and multivariate test of outcome effects. These analyses include the independent variables as control variables and may therefore offer more precise estimates of program effects.

Note that these two types of areas (CAA and comparison) should be comparable in terms of independent variables because they were selected in a similar way; if they are not comparable in terms of measured variables, then it becomes more important to use multivariate methods to adjust for the underlying differences among samples. In addition, the analysis of survey data will incorporate other variables from Medicare’s enrollment database, which will be linked to the survey data. These variables will also be used to establish comparability between CAA and comparison areas, and will also serve as control variables.

Timelines

Exhibits 5 illustrates the timelines for the beneficiary survey, as well as for the data analysis.

Exhibit 5: Data Collection Overview by Week – Beneficiary Survey

Date 2010	Date 2012	Week of Field Period	Task	Est. Qty. Beneficiary Survey
Feb-April 2010	Jan-March 2012		Survey sampling
4/30/2010	N/A		Finalize instrument
5/15/2010	4/30/2012		Program and test CATI system
5/16/2010	5/10/2012		Train interviewers
5/20/2010	5/15/2012	1	Survey packet I / Inbound calls (800-line) begin	20,251
5/27/2010	5/21/2012	2	Postcard Reminder	20,251
6/10/2010	5/282012	3	Survey Packet II	16,914
6/17/2012	6/05/2012	4	Postcard Reminder II	16,081
6/30/2010	6/12/2012	5	Survey Packet III	15,248
7/7/2010	6/20/2011	6	Postcard Reminder III	14,415
7/20/2010	6/27/2012	7	Survey Packet IV	13,582
7/27/2010–8/30/2010	7/15/2012–8/05/2012	9-12	Outbound calls to eligible non-responders	12,749
9/25/2010	8/20/2010	14	Deliver final data file	8,336
	Q4 2012		Analysis of changes from 2010 to 2012
	7/1/2001		Interim Report
	3/15/2013		Final Report

Non-Statistical Data Collection

Analysis Plan: Non-Statistical Data

The purpose of the non-statistical data collection is to understand the impact of the competitive bidding program from the perspective of the key stakeholders: beneficiary groups/advocates, CMS officials and bidding program managers, referral agents, and suppliers. The non-statistical data collected will contribute to the formative and summative evaluation of the competitive bidding program outreach, information dissemination, implementation and impact. In the discussion below, Wave 1 will take place in mid- 2010, Wave 2 will take place in early 2011 during the transition, Wave 3 will take place in mid-2011, and wave 4 will take place in mid- 2012. Wave 1 and Wave 4 will include both focus groups and key informant interviews; Wave 2 and Wave 3 will involve only key informant interviews.

Summaries of key themes or issues that arise in each of the focus groups and key informant discussions will be drafted. From Wave 1 the data will provide a baseline understanding of the DMEPOS environment in that study CAA and will be used as a point of reference from which to compare findings obtained in Waves 3 and 4; while Wave 2 will focus specifically on transition issues. The findings from Wave 4 will contribute to a formative evaluation of the program and provide recommendations for CMS for improving implementation of competitive bidding in the next 70 CAAs.

The summaries as well as the primary data will be analyzed by the project team with the aim of identifying common themes across the groups and for each of the research domains and questions. A content analysis will be performed for every research question and responses will be coded. The coded responses will be classified and analyzed for patterns and trends.

The selected comparison area findings will provide a baseline understanding of the environment for DMEPOS, quality and access for Medicare beneficiaries, and the new quality standards and accreditation requirements. Findings from the comparison areas may be used to identify market or other changes across the three (3) waves that could account for a change identified in the study CAAs. For example, the introduction of a new DMEPOS item that changed the quality of a specific durable medical equipment product may account for the identified change in the CAA.

The analysis plan for the non-statistical data draws its rigor from 1) the experience of the project team with non-statistical data collection and analysis and 2) data triangulation. The data are acquired from diverse data sources (four different informant types and documents) and data collection methods (focus groups and key informant discussions). Non-statistical data triangulation provides a test for consistency (Patton, 2002). This does not necessarily imply demonstrating similar findings across the different data sources; rather, it allows for the recognition of inconsistencies or differences that arise from the different data sources. These inconsistencies can be illuminative, particularly when evaluating a new program from the perspective of different stakeholders. This is the strongest design available for use with the observational data that can be generated for the evaluation of the competitive bidding program.

Timeline

Exhibit 6 illustrates the timeline for the non-statistical data collection and analysis.

Exhibit 6: Non-statistical Data Collection and Analysis Timeline

Subtask	Date (Wave 1, 2, 3,4)
Preparation for focus groups and key informant discussions
Collect and create recruitment lists	Feb.-March 2010, 2011, 2012
Contact and schedule CAA focus groups and key informant discussions	April 2010, 2011, 2012
Make travel arrangements for CAA site visits	April 2010, 2011, 2012
Contact and schedule comparison area key informant discussions	May-June 2010, Jan-Feb 2011, Sept-Oct 2011, May-June 2012
Conduct focus groups (waves 1&2) and key informant discussions (waves 1, 2,3,4)
Write summaries of focus group and key informant discussions	August 2010, Feb 2011, Nov 2011, August 2012
Analysis of data
Identify common themes from data collected in that wave	September 2010, March 2011, December 2011, September 2012
Conduct content analysis for research domains and questions in that wave	October 2010, April 2011, December 2011, October 2012
Code data in that wave	October 2010, April 2011, December 2011, October 2012
Synthesize data across the three waves, code data, and conduct analyses	October 2010, April 2011, December 2011, October 2012

B.17 Display of Expiration Date for OMB Approval

CMS will display the expiration date for OMB approval on all of the data collection instruments.

B.18 Exceptions to Certification Statement

This submission requires no exceptions to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).

Part C: Collection of Information Employing Statistical Methods

C.1 Respondent Universe and Sampling Methods

Beneficiary Survey

CMS requires a survey of a sample of Medicare beneficiaries in four (4) study CAAs and three (3) comparison areas. The four (4) study CAAs were selected based on size, presence of an adequate comparison area in the same state, and geographic diversity. The three (3) comparison areas were selected to come from the same state as each of the first three study CAAs, and to resemble the study CAAs with respect to the proportion of the population over 65, number of hospital beds per 1000, DMEPOS user rates, rates of new users and number of suppliers. The beneficiary survey is to be conducted in two (2) waves, the first wave in 2010 prior to the revised program launch on January 1, 2011, and the second in 2012 about 12–15 months after program launch.

The target population for the survey is all Medicare beneficiaries in each of the study CAAs and matched comparison areas, who are users of one of five (5) product categories: oxygen, hospital beds, power mobility devices, walkers, and CPAP machines. The focus will be on beneficiaries who began using their DME products during the nine months prior to each survey wave, in order to compare experiences just before the new program begins, and during the full year after it commences.

CMS is especially concerned about oxygen users, who are numerous in the Medicare population and who require uninterrupted, ongoing support from their DMEPOS suppliers. The potential for patient safety issues is arguably greatest for oxygen users, and the study design is therefore optimized to address this concern. The survey sample will support comparisons of oxygen users within each of the paired intervention/comparison areas. For other DME products, data pooled across the four (4) CAAs will be compared with data pooled across the three (3) comparison areas.

A total sample of 8,336 completed interviews is required per wave. Exhibit 7 shows the number of beneficiaries required in the sample in each stratum.

Exhibit 7: Number of Beneficiaries in Sample by Strata, per Wave

Product	Area	Required Number of Completes	Number to be Sampled	Total
Comparisons Pooled Across Sites
Bed	4CAAs	578	1,852	3,686
Bed	3 Comparison areas	572	1,834	3,686
Walkers	4CAAs	725	2,626	5,252
Walkers	3 Comparison areas	725	2,626	5,252
Wheel Chair	4CAAs	561	1,139	2,252
Wheel Chair	3 Comparison areas	548	1,113	2,252
CPAP	4CAAs	659	1,114	2,188
CPAP	3 Comparison areas	635	1,074	2,188
Within-Site Comparisons
Oxygen	Cleveland (CAA)	174	357	357
	Dallas (CAA)	597	1,231	2,308
	Houston (Comparison)	522	1,077	2,308
	Orlando (CAA)	487	1,005	2,104
	Tampa (Comparison)	533	1,099	2,104
	Riverside (CAA)	487	1,005	2,104
	San Francisco (Comparison)	533	1,099	2,104
Total		8,336	20,251	20,251

The sample size in each stratum is in terms of number of completed interviews. In the 2007 survey, response rate varied considerably for different types of DME products. The sample sizes for 2010 and 2012 are based on the response rates in the 2007 survey, and also on the cases (estimated) that exist in each site. In order to finish with 8,336 completed interviews, we will select a starting sample of 20,251 beneficiaries per wave.

C.2 Information Collection Procedures

Each type of DMEPOS user will be defined based on the presence of a claim within nine months with a pre-defined set of billed HCPCs codes. For each of the five (5) types of DME we will select beneficiaries residing in a zip code in any of our CAA or comparison areas, and who had a first DMEPOS claim within the previous nine months for a product in one of the pre-defined HCPCs. A first claim will be defined as a claim for one of the HCPCs in the previous nine months, but not in the six months prior to that.

Each selected beneficiary will receive the survey appropriate for the type of DMEPOS s/he uses. Almost all of the questions on the surveys are the same, regardless of product type, but a few questions are specific to the type of DME. For example, oxygen users will be asked whether they currently use a fixed oxygen tank, a portable tank or concentrator, or both.

Some beneficiaries use more than one of the five types of DMEPOS at the same time (e.g., oxygen and a walker). Those who use both oxygen and another DMEPOS will be classified as oxygen users, because we intend to focus special analytic attention on oxygen users. Beneficiaries who use more than one of the other types of DME (e.g., hospital bed and walker) will be randomly assigned to receive one survey or the other, but not both.

The two waves of the survey will be conducted in 2010 and 2012. Because the sample will focus on beneficiaries with recent DMEPOS claims (starting within the past nine months), no single beneficiary will be eligible for both waves of the survey. That is, the two survey samples will be completely independent.

C.3 Methods to Maximize Response Rate

The beneficiary surveys (written in English and Spanish) are mailed surveys with phone-follow-up. Based on the survey conducted in 2007, we expect response rates to range from 42% for hospital beds to 70% for CPAP machines. Response rates were lowest for products used by the most ill and frail patients, and this will likely be true in 2010 as well. We further expect that, as in 2007, nearly 90% of completed surveys will be returned by mail, and 10% by telephone.

In our experience with surveying elderly respondents, we have found a high degree of willingness to participate with Medicare surveys. However, this DMEPOS population is much more likely than the general Medicare population to have physical and cognitive limitations that affect their ability to complete a survey, making it difficult to achieve high response rates. In addition, many new DMEPOS users may be in a period of personal frailty and may relocate to live with relatives, or move into a nursing home. Those who are not living in their homes at the time of our survey will be particularly difficult to reach, by either mail or telephone. While our anticipated response rate is below the target of 80% specified by the Office of Management and Budget (OMB) guidelines, and varies depending on the type of DME equipment patient-respondents use, we know these response rates to be realistic based on the survey conducted in 2007. We will seek to minimize the impact of non-response bias by the techniques described below and through statistical measures such as adding weights to our final data.

Instrument Design

These surveys, intended for elderly and disabled people who use durable medical equipment, are clearly written, use simple vocabulary, and are printed in large font for respondents who might have visual problems.

The items in each instrument were designed not only to meet the analytical objectives necessary to answer the research questions, but also to be unambiguous, non-threatening, unbiased, non-repetitive and properly sequenced. Response categories are written to be mutually exclusive and collectively exhaustive, where applicable. Each survey has both a self-administered mail questionnaire and an interviewer-administered telephone survey script, in order to offer respondents an option to complete the survey by telephone.

Proxy respondents will be accepted for this survey, in part because many of the sampled beneficiaries are likely to be frail or disabled and to have difficulty responding on their own. We will collect information to identify which surveys are completed by proxy respondents.

Mail Survey Procedures

Once the sample for the beneficiary survey is drawn, we will send the names and addresses through the National Change of Address (NCOA) system to ensure the address file completeness and accuracy.

The outer encasement of the mailed materials will convey the importance of the contents. We will use a well-designed, high-quality outer envelope with graphics and colored print to convey a clear message about credibility of the source and the importance of a reply. We will mail the beneficiary survey packets using first-class mail, and the outer envelope will use the CMS Medicare logo.

The survey packets mailed to beneficiaries will include a cover letter, study fact sheet, the survey (Appendices B and C) and a postage-paid reply envelope. The cover letter will explain the purpose of the study and the risks and benefits of participation. In addition, it will offer a toll-free number to call the evaluation contractor, Abt Associates, with questions, request a Spanish survey, or complete the interview over the telephone. The materials will be personalized, and will clearly identify Medicare as the study sponsor.

We will mail a postcard reminder one week after each survey packet to each presumed eligible non-respondent. The follow-up postcard will stress the importance of participation in the study and offer the toll-free number with a request that the recipients call the evaluation contractor to complete the interview by phone or let us know why they did not to respond to the mailings.^¹ This presents an opportunity for the interviewing staff to conduct refusal conversion efforts, as needed.

Telephone Follow Up

We will minimize non-response by using highly effective sample management techniques that adhere to strict quality control standards with respect to interviewer monitoring, on-going feedback and training. Highly trained and skilled interviewing staff, including Spanish-speaking interviewers, will conduct phone surveys upon respondent request, and with beneficiaries who have not responded to the mailed surveys.

Phone follow-up to presumed eligible non-respondents will begin at week nine of the field period and will last through week twelve. First, we will send the list of presumed eligible non-respondents for tele-matching. Dialing for the beneficiary survey will take place from 9:00 a.m. to 9:00 p.m., respondent time. The telephone data collection will be supported by a full-featured CATI system with automated case delivery and built-in quality control.

C.4 Tests of Procedures

The beneficiary survey was successfully conducted in 2007. It is being repeated in 2010 because the competitive bidding program was delayed by more than two years and the data from 2007 are now too old to serve as credible baseline data for the evaluation. We have used the results of the 2007 survey to revise our sampling assumptions, and also to revise the questionnaire. For example, we made minor wording changes in a few questions for simplicity and revised the order of certain questions to improve logical flow. The revised questionnaires appear in Appendix B.

C.5 Individuals Consulted on Statistical Aspects of Design

Exhibit 8 contains the names, affiliations, and contact information for the individuals who consulted on statistical aspects of the design. These individuals also assisted in developing the project design and data collection protocols in 2007 and/or 2009. Abt Associates Inc., Cambridge, Massachusetts, is the contractor who will conduct the data collection and analysis for CMS.

Exhibit 8: Statistical Design Consultants

Kent Christopher, PhD, Denver, CO, 303-337-8080

Doran Edwards, PhD, Advanced Healthcare Consulting, LLC, 803-865-9225

Laraine Forry, LMF Consulting Services, LLC, 717, 579-2638

Bruce Friedman, PhD, University of Rochester, Rochester, NY, 585-273-2618

Joseph Lewarski, Inogen, Eastlake, OH, 440-269-8046

Jean Minkel, Minkel Consulting, New Windsor, NY, 845-496-5022

Jon Christianson, PhD, University of Minnesota, 612-624-3849

Roger Feldman, PhD, University of Minnesota, 612-624-5669

Marian Wrobel, PhD, (previously at) Abt Associates, Cambridge, MA, 617-349-2454

K.P. Srinath, PhD, Abt Associates, Bethesda, MD, 301-634-1836

Andrea Hassol, MSPH, Abt Associates, Cambridge, MA, 617-349-2488

1 Staff will also fulfill re-mail requests, as needed, during the outbound phone follow-up efforts.

Abt Associates Inc. Table of Contents 2

File Type	application/msword
File Title	Abt Single-Sided Body Template
Author	Abt Associates Inc.
File Modified	2010-03-24
File Created	2010-03-24