CMS-10569 Supporting Statement B EQRS CY 2024 (07-12-2023)_clean

Supporting Statement – Part B

Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The End-Stage Renal Disease (ESRD) Quality Incentive Program (QIP) Data Validation Contractor will randomly sample 300 facilities (roughly 5% of all dialysis facilities), per contract and ESRD Quality Incentive Program (QIP) rule guidelines, for participation in data validation. As a random sample, this should be a nationally representative sample of all included renal dialysis facilities. The sample pool will consist of Medicare-certified dialysis facilities that are required to submit administrative and clinical data into the ESRD Quality Reporting System (EQRS) (formerly CROWNWeb) in order to meet 42 CFR 494.180(h) of the 2008 updated Conditions for Coverage for ESRD Dialysis Facilities.

The 300 facilities will be asked to submit records that will be validated for CMS-designated Critical Performance Measures (CPMs). The patient sample size is limited to 10 patients per facility, as per contract and ESRD QIP rule guidelines. The Data Validation Contractor will sample 10 patients (or the maximum patients possible) from each selected facility for CPM reviews. Historically, facility response rates have been sufficient for the purpose of validation of the data. Although the participation rate in the 2021 validation was 67%, this was due to the impact of the COVID-19 public health emergency (PHE) and CMS allowing facilities to opt out in order to meet direct patient care demands. We note that the participation rate returned to the normal rates during the 2022 validation.

Sample Size Estimates

Facilities will be stratified by size before selecting random samples, based on the number of dialysis stations. Facilities with less than 20 dialysis stations will be categorized as Small; facilities with between 20 and 39 stations will be categorized as Medium; and facilities with at least 40 stations will be categorized as Large.

The estimated number of facilities and patients expected in the sampling pool, by facility size, is shown in Table 1, totaling 100 facilities each from small, medium, and large facilities, will be randomly selected, totaling 300 facilities from the total population of eligible facilities, and randomly selecting up to 10 records per facility, for the months of April, May, and June 2024.

Some smaller facilities had less than 10 patients treated for the period; in these cases, we selected all the patients treated at the facility during the data period for validation.

Table 1 Sample Pool Distribution of Facilities and Patients by Facility Size, April 1, 2022 – June 30, 2022
Facility Size	Number of Stations	Number of Facilities	Number of Patients
Small	<20	4,088	182,326
Medium	20-39	1,748	139,080
Large	=/> 40	99	14,908
Total		5,935	336,314

Table 2 provides the Distribution of Patient Records by Network, which is taken into account as part of the sampling methodology, and Table 3 provides the Distribution of Patient Records by Affiliation for CY 2022 validation.

Table 2: Distribution of Patients within Network Number, 2022 Q2

Network Number	Number of Patient Records	% Total Patients
1	133	1.6
2	561	6.8
3	336	4.1
4	252	3.1
5	477	5.8
6	1,450	17.6
7	605	7.4
8	636	7.7
9	523	6.4
10	317	3.9
11	334	4.1
12	492	6.0
13	244	3.0
14	773	9.4
15	251	3.0
16	138	1.7
17	423	5.1
18	285	3.5
Total	8,230	100

Table 3: Distribution of Patients within Affiliation, 2022 Q2

Affiliation	Number of Patient Records	% Total Patients
FRESENIUS	3,519	42.8
DAVITA	2,638	32.1
US RENAL CARE	224	2.7
AMERICAN RENAL ASSOCIATES	112	1.4
DCI	399	4.8
OTHER	1,338	16.3
TOTAL	8,230	100

Sampling Time Frame

The 300 facilities to be sampled for validation will be selected from a sample pool created by combining multiple data extracts for EQRS from March through August 2024. Data extracts will be provided by CMS, and all extracts are expected to be received by early November 2024. A second request for EQRS data extract of depression screening data for the Clinical Depression Screening and Follow-Up measure and In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) will occur in January 2025. Once received, the data extracts will be combined at the record level to create the sampling pool.

This timeframe was selected after considering several factors. To ensure that the validation can be completed during the period of performance, the Validation Contractor considered the data reporting periods allowed to facilities to submit clinical data into EQRS. Facilities are given 60 days from the end of each month to enter EQRS clinical data. The mandated reporting period limits the time frame we can validate expeditiously, as we will not be able to obtain an extract until after the close of the data submission period.

Another important consideration is that facilities have up to 60 days to submit records. Taking into consideration these factors as well as the need to ensure that there is adequate time to perform analysis and prepare reports, we decided on the second quarter of 2024 validation time frame. A breakdown of the reporting deadlines that were taken into consideration is displayed in Table 4.

Table 4: Reporting Deadlines

Submission Type	Reporting Deadlines
EQRS Data Submission	60 days after month close
Facility Record Submission Deadline	60 days after request receipt per ESRD QIP rule

Assuming the EQRS data team will need at least one week to export and send the data, the Validation Contractor has estimated preliminary dates for data availability. Table 5 provides the Validation Contractor’s estimates for when the EQRS data will be received for each corresponding data set.

Table 5: Estimated Timeline for Receiving EQRS Data

Type of Data	Data Reporting Period	Estimated Receive Date
CPM	April, May, June 2024	Starting mid-November through early March 2025

Due to the tight timeframe for data abstraction, effective coordination and management as well as adherence to established schedules will be crucial to meet all operational program requirements.

2. Describe the procedures for the collection of information including:

- Statistical methodology for stratification and sample selection,

- Estimation procedure,

- Degree of accuracy needed for the purpose described in the justification,

- Unusual problems requiring specialized sampling procedures, and

- Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Please see response to question 1 for statistical methodology for stratification and sample selection, including estimation procedure. As noted below in response to question 4, there are no unusual problems requiring specialized sampling procedures as our previous experience on past CMS CROWNWeb (now EQRS) CPM validation efforts have shown near universal compliance with medical record requests. The period for data collection cycles is expected to be no more frequently than annually.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield 'reliable' data that can be generalized to the universe studied.

Facilities will be contacted in early spring via QualityNet using the HARP MFT option and will be asked to participate in the validation effort. As some facilities are not on HARP MFT, we also use FedEx to send the requests. The letter will provide instructions on the types of records to be submitted, methods to submit records to the Validation Contractor, and identify patients selected for validation. Facilities that do not respond to the initial request for records are contacted via phone by the Validation Contractor and receive a final request letter in early March 2024. To maximize facility response rates, we hold a townhall around January of each year to increase facility knowledge about the validation process and purpose.

We also communicate/coordinate extensively with all facilities, using web conferences to facilitate on-time and accurate medical records submission by participating facilities. Facilities that do not respond to the request for records are subject to a 10-point reduction to their Total Performance Score (TPS). For future validations, we plan to follow the same records request methodology, follow-up, and ESRD community outreach approach we have used in the past since it has been effective in producing desired response rates.

Data Validation

The main objective of this analysis is to perform a single comparison of the data reported to the EQRS system against CPM element data obtained from the facilities’ records, leading to an evaluation of the reliability (i.e. the data are reasonably complete and accurate) and validity (i.e. the data represent what is being measured) of EQRS data.

• Reliability: Reliability means data are reasonably complete and accurate, meet intended purposes, and are not subject to inappropriate alteration. Where:

• Completeness refers to the extent that relevant records are present and the fields in each record are populated appropriately, and,

• Accuracy refers to the extent recorded data reflect the actual underlying information.

A more formal definition of reliability is the extent to which results are consistent over time and an accurate representation of the population under study:

• The degree to which a measurement, taken repeatedly, remains the same,

• The stability of the measurement over time, and

• The similarity of measurements within a given time period.

• Validity: Validity (as used here) refers to whether the data actually represent what one believes is being measured. Several measures are commonly used to assess validity of any measure.

To ensure the reliability of data collected by reviewers, we have one trained ESRD medical reviewer per record who will consult the ESRD experts if there is a question or issue with the medical record.

Implementing this element of the validation procedures enables us to focus on reviewer accuracy rather than reviewer agreement. Additionally, reviewers make full use of a system called EMS. If the medical reviewer is unsure of whether the data matches the EQRS data, they request a second review by a senior reviewer. The senior reviewers also reach out to the provider for additional records or clarification. The statistic department performs quality checks to ensure the accuracy of the data.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

As noted above, the sample pool will consist of Medicare-certified dialysis facilities that are required to submit administrative and clinical data into EQRS to meet § 494.180(h) of the 2008 updated Conditions for Coverage for ESRD Dialysis Facilities. The previous experience on past CMS CROWNWeb (now EQRS) validation efforts have shown near universal compliance with medical record requests. No additional tests of procedures or methods to be undertaken are expected.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Delia Houseal, Centers for Medicare and Medicaid Services, (410) 786-2724

7