Supporting Statement for the
Paperwork Reduction Act of 1995
Part B. Collections of Information Employing Statistical Methods
QIC Demonstration Evaluation Contractor
(QDEC): Analyze Medicare Appeals to Conduct
Formal Discussions and Reopenings with
Suppliers
March 1, 2017
Centers for Medicare & Medicaid Services
Center for Medicare (CM)
Medicare Enrollment and Appeals Group (MEAG) Division of Appeals Operations (DAO)
7500 Security Boulevard
Baltimore, MD 21244
Table of Contents
1. Respondent Universe and Sampling Methods 3
2. Procedures for the Collection of Information 4
3. Methods to Maximize Response Rates and Deal with No Response 5
4. Test of Procedures or Methods to Be Undertaken 5
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 5
Supporting Statement – Part B
QIC Demonstration Evaluation Contractor (QDEC): Analyze Medicare Appeals to Conduct
Formal Discussions and Reopenings with Suppliers
The following is a description of data collection procedures.
Proposed data collection activities include a monthly web-based survey to assess experiences and satisfaction with the Demonstration (formal telephone discussions), and annual telephone key informant interviews to gather in-depth information about successes, challenges, and suggestions related to the Demonstration.
The respondent universe for the proposed web-based survey and key informant interview data collection includes all individuals affiliated with DME suppliers that participate in a formal telephone discussion with the Demonstration Qualified Independent Contractor (QIC) in a given month. The QIC shall provide approximately 45,000 formal telephone discussions per year, from January 2017 through December 2020, with the approximately 5,400 supplier organizations in MAC Jurisdictions C and D. Upon receipt of a valid reconsideration request in DME MAC Jurisdictions C or D, the QIC reviews the appeals received and selects appeals the QIC has determined may yield a favorable outcome through the discussion process. Upon identification of an appeal that could be resolved via the discussion process, the DME QIC issues a form letter notifying the appellant that the claim has been selected to participate in the discussion demonstration and includes a scheduled time for the telephone discussion. Participation in the telephone discussion process is a voluntary process.
From January 1, 2016 to June 30, 2017, 44% of suppliers who received invitations participated in at least one telephone discussion. During this same time period, he DME QIC held telephone discussions with suppliers from 965 DME supplier organizations, representing approximately 18 percent of DME supplier organizations in DME MAC Jurisdictions C and D that appealed claims to the DME QIC from January 1, 2016 to June 30, 2017.
The unit of analysis for the data collection activities is considered to be a specific, unique formal telephone discussion. The web-based survey will employ a random sampling technique to obtain an understanding of supplier experience and satisfaction with a particular, specific formal telephone discussion, the date and time of which will be noted in the invitation letter. A minimum of 381 complete responses are needed to achieve statistical power to determine overall satisfaction, at the 95% confidence interval level, with a 5% margin of error, assuming a 50% response distribution. We will invite a total of 2,540 suppliers annually to participate in the web-based survey, assuming a 15% response rate, to ensure we collect the minimum number of completes needed. Based on our experience, web-based survey response rates can range from 10% to 50% or higher. This response rate is a conservative estimate, since it is unknown how many suppliers will be willing to respond to the survey.
Sampling will occur on a monthly basis. A random sample of 212 suppliers will be drawn each month from the pool of suppliers who participated in a formal telephone discussion in the prior month (totaling 2,540 suppliers per year). Invitees will be sent up to two reminder notices before their unique survey link will be deactivated. We expect some suppliers will participate in more than one formal telephone discussion, and on average, most suppliers will be randomly selected twice per year. Because respondents will be instructed to answer the survey based on their experiences and satisfaction with one specific formal telephone discussion (specified on the invitation email), each set of responses obtained will be considered unique cases, even if an individual responds to the survey more than once. However, once the data collection begins, we will analyze responses after 12 months to determine if there is variation in repeat-responder data. If there is no variation, and individuals responding to the survey more than once are providing identical responses to questions, we may report two sets of results: one treating repeat-respondents as unique cases, and one removing their responses from analyses.
Key informant interviews will employ a simple stratified random sampling plan to select a total of 100 respondents each year: 50 randomly selected from the universe of formal discussion participants, and 50 selected from a pool of “dissatisfied” suppliers.
Fifty suppliers will be randomly selected from the pool of web-based survey respondents who indicated dissatisfaction with the formal telephone discussion; because the web-based survey has not yet been fielded, it is unclear what the size of this pool will be.
Then, fifty different suppliers will be randomly selected from the entire remaining universe of DME suppliers.
Exhibit B-1 shows these stratifications. If sampled suppliers refuse to participate, another random selection will be done to refresh the sample. The sample will be refreshed until 100 interviews are conducted. Although it is not standard or methodologically appropriate to calculate a minimum sample size required for qualitative data collection, we believe including 100 suppliers in our data collecting activities selected through a random process will allow us to reach saturation in supplier views and perspectives about the Demonstration.
Exhibit B-1: Simple Random Sampling Strata for Annual Key Informant Interviews
|
No. of Unique Suppliers |
Dissatisfied Supplier Group |
50 |
General Supplier Group |
50 |
Total |
100 |
Instruments and respondent recruitment materials are found under Attachments A (web-based survey) and B (key informant interviews).
Web-Based Survey:
Each month, the list of suppliers who participated in a formal telephone discussion in the prior month will be obtained from internal Agency records. This list will include the name and contact information of the person who participated. A simple random sample of 212 suppliers per month (2,540 per year) will be selected from that list using the Random Sample procedure in Excel. Each selected supplier will be sent an invitation to complete the short web-based survey (Attachment A). The web-based survey will be programmed with a unique date of the formal telephone discussion in question, for the respondent’s reference as they complete the survey. The web-based survey will be managed through the Voxco software platform. Voxco tracks responses and generates reminder e-mails, which will be sent to non-responders one week after the initial survey invite is sent, and again two weeks after the initial survey invite is sent. The survey links will remain active for 90 days from the initial invitation. No statistical methodology, estimations, or specialized procedures are necessary.
Key Informant Interviews:
On an annual basis, 100 suppliers will be randomly selected to be invited to participate in a telephone-based key informant interview. As described in Section 1, 50 suppliers will be randomly selected from the pool of web-based survey respondents who indicated dissatisfaction with the formal telephone discussion; and 50 different suppliers (who may or may not have responded to a web survey) will be randomly selected from the entire remaining universe of suppliers. Selected suppliers will receive an invitation letter via U.S. mail (Attachment B) which will explain the goals of the interview and that a member of the research team will be in touch to schedule the interview. One week after the letter is sent, the research team will reach out by phone and/or e-mail to confirm participation and identify a mutually-agreeable time.
At the agreed-upon date and time of the interview, the research staff will call the key informant and conduct a 30-minute discussion of the suppliers’ experiences, satisfaction, and recommendations for improving demonstration formal telephone discussions (Attachment B). At the beginning of the call, the interviewer will obtain verbal consent to audio record the interview. If the respondent refuses, only typed notes will be taken. The interview will begin with a brief description of the research, followed by the in-depth discussion. Audio-recorded interviews will be reviewed by the research team during the note-cleaning phase. No statistical methodology, estimations, or specialized procedures are necessary.
The following procedures will be used to maximize participation and to achieve the desired high response rate:
Web-Based Survey: We will receive contact information for the entire sampling frame on a monthly basis from the QIC, as described earlier. We will use two strategies to maximize our chances of obtaining a response rate of at least 15%. First, we will monitor and attempt to correct bounced back email addresses, using prior months’ contact information to try to identify a valid email address that might have been incorrectly entered in the current month’s roster. Second, because reminders have been shown to increase response rates, we will also monitor which sample members respond and do not respond to the initial email invite, and send up to two follow-up reminder each week to those who haven’t yet responded to a given month’s survey. No other contact with non-respondents will be made.
Key Informant Interviews: Selected interviewees will be informed about the benefit of their participation to the demonstration, and to future programs like the formal telephone discussions which are designed to improve the accuracy of suppliers’ payments. The goal of the key informant interviews is not to generalize or be representative of all supplier perspectives; rather, the key informant interviews are intended to gather in-depth information from participants in the demonstration to understand what worked well, and what could be improved. Collecting this level of information from a representative sample is not feasible for this project.
The web-based survey was pilot tested with nine randomly-selected suppliers who had participated in the Demonstration, the maximum number of participants allowed without PRA clearance, in September 2016. Results showed that the majority of suppliers rated the formal telephone discussion and reopenings processes favorably. Most indicated that they would recommend participating in the formal telephone discussion process to other suppliers and were likely to participate again in the future. While most felt that the QIC clearly explained reasons for denial and applicable Medicare requirements and policies, responses were mixed on gaining a better overall understanding of how the MAC’s decision on their case was reached after the formal telephone discussion. Similarly, although six suppliers found the opportunity to learn more about applicable Medicare policies and requirements to be a beneficial aspect of the formal telephone discussion, three did not. We made minor grammatical revisions to the instruments based on the pilot testing, but substantive changes were not needed.
Key informant interview guides have been reviewed internally by the Agency demonstration staff to ensure validity and relevance to the project.
Web-based survey and key informant interview data collection require a straightforward simple random selection procedure. IMPAQ International, LLC, will collect and analyze web-based survey and key information interview data on behalf of CMS.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |