CMS-10569 Supporting Statement B EQRS CY 2023 (07-05-2022) cs

Supporting Statement – Part B

Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The End-Stage Renal Disease (ESRD) Quality Incentive Program (QIP) Data Validation Contractor will randomly sample 300 facilities (roughly 5% of all dialysis facilities), per contract and ESRD Quality Incentive Program (QIP) rule guidelines, for participation in the validation project. As a random sample, this should be a nationally representative sample of all included renal dialysis facilities. The sample pool will consist of Medicare-certified dialysis facilities that are required to submit administrative and clinical data into the ESRD Quality Reporting System (EQRS) (formerly CROWNWeb) in order to meet Section 494.180(h) of the 2008 updated Conditions for Coverage for ESRD Dialysis Facilities.

The 300 facilities will be asked to submit records that will be validated for CMS-designated Critical Performance Measures (CPMs). The patient sample size is limited to 10 patients per facility, as per contract and QIP rule guidelines. The Data Validation Contractor will sample 10 patients (or the maximum patients possible) from each selected facility for CPM reviews. Historically, facility response rates have been sufficient for the purpose of validation of the data. Although the participation rate in the 2021 study was 67%, this was due to the impact of the COVID-19 public health emergency (PHE) and CMS allowing facilities to opt out in order to meet direct patient care demands. We anticipate that the participation rate will return to the normal rates during the 2022 study.

Sample Size Estimates

Facilities will be stratified by size before selecting random samples, based on the number of dialysis stations. Facilities with less than 20 dialysis stations will be categorized as Small; facilities with between 20 and 39 stations will be categorized as Medium; and facilities with at least 40 stations will be categorized as Large.

The estimated number of facilities and patients expected in the sampling pool, by facility size, is shown in Table 1, totaling 100 facilities each from small, medium, and large facilities, will be randomly selected, totaling 300 facilities from the total population of eligible facilities, and randomly selecting up to 10 records per facility, for the months of April, May, and June 2023.

Some smaller facilities had less than 10 patients treated for the period; in these cases, we selected all the patients treated at the facility during the study period for validation.

Table 1 Sample Pool Distribution of Facilities and Patients by Facility Size, April 1, 2021 – June 30, 2021
Facility Size	Number of Stations	Number of Facilities	Number of Patients
Small	<20	4,088	182,326
Medium	20-39	1,748	139,080
Large	=/> 40	99	14,908
Total		5,935	336,314

Table 2 provides the Distribution of Patient Records by Network and Table 3 provides the Distribution of Patient Records by Affiliation for the CY 2021 study.

Table 2: Distribution of Patients within Network Number, 2021 Q2

Network Number	Number of Patient Records	% Total Patients
1	133	1.6
2	561	6.8
3	336	4.1
4	252	3.1
5	477	5.8
6	1,450	17.6
7	605	7.4
8	636	7.7
9	523	6.4
10	317	3.9
11	334	4.1
12	492	6.0
13	244	3.0
14	773	9.4
15	251	3.0
16	138	1.7
17	423	5.1
18	285	3.5
Total	8,230	100

Table 3: Distribution of Patients within Affiliation, 2021 Q2

Affiliation	Number of Patient Records	% Total Patients
FRESENIUS	3,519	42.8
DAVITA	2,638	32.1
US RENAL CARE	224	2.7
AMERICAN RENAL ASSOCIATES	112	1.4
DCI	399	4.8
OTHER	1,338	16.3
TOTAL	8,230	100

Sampling Time Frame

The 300 facilities to be sampled for validation will be selected from a sample pool created by combining multiple data extracts for EQRS from March through August 2023. Data extracts will be provided by CMS, and all extracts are expected to be received by early November 2023. A second request for EQRS data extract of depression screening and ICH CAHPS will occur in January 2024. Once received, the data extracts will be combined at the record level to create the sampling pool.

This timeframe was selected after considering several factors. To ensure that the validation can be completed during the period of performance, the Validation Contractor considered the data reporting periods allowed to facilities to submit clinical data into EQRS. Facilities are given 60 days from the end of any month to enter EQRS clinical data. The mandated reporting period limits the time frame we can validate expeditiously, as we will not be able to obtain an extract until after the close of the data-reporting period.

Another important consideration is that it is mandatory for us to give facilities up to 60 days to submit records. Taking into consideration these factors as well as the need to ensure that there is adequate time to perform analysis and prepare reports, we decided on the second quarter of 2023 validation time frame. A breakdown of the mandated reported deadlines that were taken into consideration is displayed in Table 4.

Table 4: Mandated Reporting Deadlines

Submission Type	Mandated Reporting Deadlines
EQRS Data Submission	60 days after month close (Q2 – August 31, 2023)
Facility Record Submission Deadline	60 days after request receipt per QIP rule

Assuming the EQRS data team will need at least one week to export and send the data, the Validation Contractor has estimated preliminary dates for data availability. Table 5 provides the Validation Contractor’s estimates for when the EQRS data will be received for each corresponding data set.

Table 5: Estimated Timeline for Receiving EQRS Data

Type of Data	Data Reporting Period	Estimated Receive Date
CPM	April, May, June 2023	Starting mid-November through End of December 2023

Due to the tight timeframe for data abstraction, effective coordination and management as well as adherence to established schedules will be crucial to the project’s success.

2. Describe the procedures for the collection of information including:

- Statistical methodology for stratification and sample selection,

- Estimation procedure,

- Degree of accuracy needed for the purpose described in the justification,

- Unusual problems requiring specialized sampling procedures, and

- Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Please see response to question 1 for statistical methodology for stratification and sample selection, including estimation procedure. As noted below in response to question 4, there are no unusual problems requiring specialized sampling procedures as our previous experience on past CMS CROWNWeb (now EQRS) CPM validation efforts have shown near universal compliance with medical record requests. The period for data collection cycles is expected to be no more frequently than annually.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield 'reliable' data that can be generalized to the universe studied.

Facilities will be contacted in early winter via QualityNet using the Secure File Transfer option and will be asked to participate in the validation effort. The letter will provide instructions on the types of records to be submitted, methods to submit records to the Validation Contractor, and identify patients selected for validation. Facilities that do not respond to the initial request for records are contacted via phone by the Validation Contractor and receive a final request letter in early March 2023. To maximize facility response rates, we hold a townhall around January of each year to increase facility exposure to our validation study.

We also communicate/coordinate extensively with all facilities, using web conferences to facilitate on-time and accurate medical records submission by participating clinics. Facilities that do not respond to the request for records are subject to a 10-point reduction to their Total Performance Score (TPS). For future validations, we plan to follow the same records request methodology, follow-up, and ESRD community outreach approach we have used in the past since it has been effective in producing desired response rates.

Data Validation

The main objective of this analysis is to perform a single comparison of the EQRS system data against CPM element data obtained from the facilities’ records, leading to an evaluation of the reliability (i.e. the data are reasonably complete and accurate) and validity (i.e. the data represent what is being measured) of EQRS data.

• Reliability: Reliability means data are reasonably complete and accurate, meet intended purposes, and are not subject to inappropriate alteration. Where:

• Completeness refers to the extent that relevant records are present and the fields in each record are populated appropriately, and,

• Accuracy refers to the extent recorded data reflect the actual underlying information.

A more formal definition of reliability is the extent to which results are consistent over time and an accurate representation of the population under study:

• The degree to which a measurement, taken repeatedly, remains the same,

• The stability of the measurement over time, and

• The similarity of measurements within a given time period.

• Validity: Validity (as used here) refers to whether the data actually represent what one believes is being measured. Several measures are commonly used to assess validity of any measure.

To ensure the reliability of data collected by reviewers, we use two reviewers for each patient record. We systematically measure differences between reviewers for all patient records and provide ongoing training as needed to correct reviewer error tendencies. All discrepancies are reconciled by the second reviewer.

Implementing this element of the study design enables us to focus on reviewer accuracy rather than reviewer agreement. We use a system named EQRS Abstraction Processing System (EAPS) that presents the second reviewer a split screen page review, displaying the first and second reviewer results. This page provides the second reviewer the capability to identify any differences and make needed updates to the second reviewer findings. Consequently, we always use second reviewer results in our analysis.

Additionally, reviewers make full use of the Consult feature of EAPS. Whenever either reviewer needs to reach out to a more experienced reviewer, the person moves the patient record to the Consult phase. There the two of them resolve the issue and then move the record back to the point where regular review processing was interrupted.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

As noted above, the sample pool will consist of Medicare-certified dialysis facilities that are required to submit administrative and clinical data into EQRS to meet Section 494.180(h) of the 2008 updated Conditions for Coverage for ESRD Dialysis Facilities. The previous experience on past CMS CROWNWeb (now EQRS) validation efforts have shown near universal compliance with medical record requests. No additional tests of procedures or methods to be undertaken are expected.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Delia Houseal, Centers for Medicare and Medicaid Services, (410) 786-2724

7