CMS-10639 Supporting Statement B NHSN CY 2019 (10-25-18)

Supporting Statement – Part B

Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The Data Validation Contractor will randomly sample the number of facilities proposed in the CY 2019 ESRD PPS proposed rule (150 in PY 2021), per contract and Quality Incentive Program (QIP) rule guidelines, for participation in the validation project. As a random sample, this should be a representative sample of all included facilities nationally. The sample pool will consist of Medicare-certified dialysis facilities that are required to submit administrative and clinical data into CROWNWeb to meet Section 4943108(h) of the 2008 updated Conditions for Coverage for ESRD Dialysis Facilities.

The selected facilities will be asked to submit records that will be validated for NHSN dialysis event elements. As finalized in the CY 2019 ESRD PPS final rule, the patient sample size is limited to 20 patients (or the maximum patients possible) per facility in each of the first two quarters of the calendar year, as per contract and QIP rule guidelines. The medical records will be for patients with “candidate events” as well as randomly selected patients.

Sample Size Estimates

The Data Validation Contractor will segment selected facilities by CMS Network Number and by affiliation with major dialysis organizations (DaVita, DCI, Fresenius, and all others as Independent). Sample size estimates for PY 2021 are shown in Table 1 and 2, respectively.

Table 1: Estimated Sample Distribution of Patients within Network Number

Network Number	Patient Population Percentage
1	2
2	4
3	3
4	4
5	6
6	9
7	6
8	7
9	8
10	4
11	9
12	4
13	7
14	8
15	5
16	5
17	4
18	5
Total	100

Table 2: Estimated Sample Distribution of Patients within Affiliation

Affiliation	Patient Population Percentage
DaVita	43
DCI	8
Fresenius	32
Independent	17
Total	100.00

Table 3 depicts the methodology that will be used when sampling for patients for NHSN reviews.

Table 3: Sampling Methodology for NHSN Reviews

Sampling Source	Sample to be Taken
Facility	Maximum number of patients with candidate events available, up to 20
General Pool of Patients at Facility	Remaining number of patients, randomly selected by the facility, needed to equal 20

Sampling Time Frame

Facilities will be randomly selected from the national collection of ESRD facilities that will not include those facilities already selected for participation in the CPM component of the study. Patient records for the first and second quarter of PY 2021 will be used according to the guidelines of the ESRD QIP rule.

A breakdown of the mandated reported deadlines that were taken into consideration is displayed in Table 4.

Table 4: Mandated Reporting Deadlines

Submission Type	Mandated Reporting Deadlines
PY 2021 NHSN Data Submission	90 days after quarter close (Q2 – September 30, 2019)
Facility Record Submission Deadline - NHSN	60 days after request receipt per QIP rule

Due to the tight timeframe for data abstraction, effective coordination and management as well as adherence to established schedules will be crucial to the project’s success.

2. Describe the procedures for the collection of information including:

- Statistical methodology for stratification and sample selection,

- Estimation procedure,

- Degree of accuracy needed for the purpose described in the justification,

- Unusual problems requiring specialized sampling procedures, and

- Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Please see response to question 1 for statistical methodology for stratification and sample selection, including estimation procedure and degree of accuracy needed for the purpose of this work. As noted below in response to question 4, there are no unusual problems requiring specialized sampling procedures as the previous experience on the previous CMS NHSN validation effort has shown near universal compliance by the hospitals with medical record requests. The period for data collection cycles is expected to be no more frequently than annually.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield 'reliable' data that can be generalized to the universe studied.

The Validation Contractor is expected to communicate directly with each facility using QualityNet. Facilities will be sent request letters that include instructions on the types of records to be submitted, methods to submit records to the contractor, and identified patients selected for validation. Facilities that do not respond to the initial request for records will be contacted via email sent directly to the designated contact person for each facility. If necessary, follow up phone calls will be made.

Data Validation

The main objective of this analysis is to perform a single comparison of the NHSN system against NHSN “candidate event” data obtained from the facilities’ records, leading to an evaluation of the reliability (i.e., the data are reasonably complete and accurate) and validity (i.e., the data actually represent what is being measured) of NHSN data. Candidate events include positive blood cultures, intravenous antimicrobials, or vascular site infection (e.g. pus, redness, or increased swelling).

Reliability: Reliability means data are reasonably complete and accurate, meet intended purposes, and are not subject to inappropriate alteration, where:

• Completeness refers to the extent that relevant records are present and the fields in each record are populated appropriately; and

• Accuracy refers to the extent recorded data reflect the actual underlying information. In this instance, that the data in the NHSN system accurately reflects the data contained within the source documents, i.e. the facilities’ medical records.

Validity: (as used here) refers to whether the data represent what one believes is being measured. A few measures are commonly used to assess validity of any measure.

In our interpretation of these measures, we identify the key sources of overall disagreement between the NHSN data and the patients’ medical records, which would serve as the “gold standard.” Typical sources of disagreement include missing information about events, inaccurate dates and inaccurate dialysis catheter information.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

As noted above, the sample pool will consist of Medicare-certified dialysis facilities that are required to submit administrative and clinical data into CROWNWeb in order to meet Section 494.108(h) of the 2008 updated Conditions for Coverage for ESRD Dialysis Facilities. The previous experience on CMS NHSN validation efforts has shown near universal compliance with medical record requests. No additional tests of procedures or methods to be undertaken are expected.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Khalil Abdul-Rahman, RELI Group, (410) 533-2384

Siva Bala, RELI Group, (440) 382-7415

Gladys Happi (410) 504-4394.