CMS-10639 NHSN Data Validation Supporting Statement A.CY2018X

Supporting Statement – Part A

National Healthcare Safety Network (NHSN) Data Validation Study for the End-Stage Renal Disease (ESRD) Quality Incentive Program (QIP)

1. Background

Pursuant to section 1881(h) of the Social Security Act (the Act) as amended by section 153(h) of the Medicare Improvements for Patients and Providers Act (MIPPA), the Centers for Medicare and Medicaid Services (CMS) established the End-Stage Renal Disease Quality Incentive Program (ESRD QIP) starting in 2011. The ESRD QIP is the first value-based purchasing program established by CMS, and it is aimed at promoting patient health by providing a financial incentive for renal dialysis facilities to deliver high-quality care.

In implementing the End-Stage Renal Disease Quality Incentive Program (ESRD QIP), CMS believes that a successful quality incentive program will promote the delivery of high quality health care services in the renal dialysis facility setting. Under section 1881(h)(2) of the Act, the Secretary is required to specify quality measures for evaluating the quality of care ESRD patients receive at renal dialysis facilities. While the Act outlines a few mandatory measure topics, the Secretary is authorized to adopt measures on specified areas or medical topics determined appropriate by the Secretary (§1881(h)(2)). The ESRD QIP began in calendar year (CY) 2011 with an initial set of three quality measures, and has dramatically increased its measure set over the intervening years through notice and comment rulemaking.

To score facility performance on quality measures, CMS must be able to collect data on these measures. CMS collects this data from multiple sources, including Medicare claims (OMB control number 0938-1197) and other tools such as the Centers for Disease Control and Prevention’s National Healthcare Safety Network (NHSN) Dialysis Event Protocol. To further expand the measures used to evaluate the quality of care provided to ESRD patients in renal dialysis facilities, CMS also collects data using the Consolidated Renal Operations in a Web-Enabled Network (CROWNWeb) system. The information collection requirements associated with the ESRD QIP measures that use data collected in CROWNWeb are covered under OMB Control Number 0938-1289.

One of the critical elements of the ESRD QIP’s success is ensuring that the data submitted to calculate measure scores and Total Performance Scores (TPS) are accurate. In the CY 2015 ESRD PPS final rule, CMS finalized a feasibility study for Payment years (PY) 2017 and 2018 to validate data reported to the Centers for Disease Control and Prevention (CDC’s) NHSN Dialysis Event Module for the NHSN Bloodstream Infection clinical measure. The purpose of this study was to compare the accuracy and frequency of dialysis event data reported to the NHSN system, to what was recorded in the patients’ medical record for those years respectively. Healthcare-Acquired Infections (HAI) are relatively rare, and CMS finalized that the feasibility study would target records with a higher probability of including a dialysis event, because this would enrich the validation sample while reducing the burden on facilities. For CY 2015, the feasibility study looked at records from only 9 ESRD facilities.

The Feasibility Study in PY 2017 was conducted as follows: The Validation contractor mailed CMS-approved formal request letters and packets to the nine selected facilities and asked for lists of all CY 2015 positive blood cultures, associated medical records, completed surveys, and patient cover sheets. The medical records were considered the “gold standard” for comparison purposes. Trained medical reviewers abstracted the data from the patient medical records. All nine selected facilities participated in the survey and submitted their positive blood culture templates. A total of 16 positive blood cultures were identified from those facilities. After abstracting the 16 patient charts, reviewers identified 21 reportable events. Due to the relatively small sample size, there were no instances where the chart reviewed was without an event and there were no events reported in NHSN for that patient during the time period. That would constitute a true negative. The facilities selected were also asked to complete a survey to provide information related to their participation with the feasibility study. They provided information regarding the processes used to track Dialysis Events as well has how their denominators were calculated. This information helped generalize findings and identify potential deficiencies and improvement opportunities. Based upon the results of the feasibility study and the survey responses, the validation contractor made several recommendations for improvement opportunities. Specifically, they recommended additional training for Dialysis Event abstraction covering complex multiple dialysis events reporting as well as commonly found reporting errors. They also recommended a revision of the definition of a “match,” considering a match of the date if the report is within +/- 3 days of the event. Finally, they recommended using a more robust study methodology, possibly expanding analysis from only one Dialysis Event to include the other two types of events and increasing the sample size to identify areas of required adjustment.

CMS took these recommendations and developed a more robust study to be implemented in the PY 2019 program. Due to the changes in sample size and the proposed methodology, described below, we developed a PRA package specific to the NHSN Data Validation work being done for the ESRD QIP beginning with the PY 2019 program. We are now updating this PRA Package for the PY 2020 program, as discussed more fully below.

1. NHSN Data Validation for the ESRD QIP

For payment year (PY) 2019, CMS finalized a new methodology and for PY 2020 CMS is proposing to continue use of this methodology with slight modifications to the sampling methodology. Additionally, for PY 2019, CMS finalized that the validation study would consist of 35 facilities, and for PY 2020 CMS is proposing to continue validating records from the same number of facilities. The purpose of this validation study is to compare the data entered by facilities into CDC’s NHSN system against what is reported in medical records. Specifically, for the PY 2020 ESRD QIP, CMS is proposing to select 35 facilities to participate in an NHSN Dialysis Event validation study by submitting 10 patient records covering two quarters of data reported in CY 2018. A CMS contractor will send these facilities requests for medical records for all patients with “candidate events” during the evaluation period; i.e., patients who had any positive blood cultures; received any intravenous antimicrobials; had any pus, redness, or increased swelling at a vascular access site; and/or were admitted to a hospital during the evaluation period. Facilities will have 60 calendar days to respond to the request for medical records based on candidate events either electronically or on paper. If the contractor determines that additional medical records are needed to reach the 10-record threshold from a facility to validate whether the facility accurately reported the dialysis events, then the contractor will send a request for additional, randomly selected patient records from the facility. The facility will have 60 calendar days from the date of the letter to respond to the request. Through collaboration with the CDC for system and data access, the CMS contractor will utilize the methodology described above for reviewing and validating records from candidate events and randomly selected patients, in order to determine whether the facility reported dialysis events for those patients in accordance with the NHSN Dialysis Event Protocol. We finalized that if a facility is selected to participate in the validation study but does not provide CMS with the requisite lists of positive blood cultures within 60 calendar days of receiving a request, then we would deduct 10 points from the facility’s TPS (81 FR 77895). Information from the validation study may be used in future years of the program to inform our consideration of future policies that would incorporate NHSN data accuracy into the scoring process.

2. Justification

1. Need and Legal Basis

Continued expansion of the End-Stage Renal Disease Quality Incentive Program (ESRD QIP) measure set is consistent with the letter and spirit of MIPPA. Section 1881(h)(2) of the Act requires that the Secretary specify measures for each year of the program and with each successive year of the ESRD QIP, CMS has increased the sophistication and scope of the program’s measure set. While Medicare claims can be an appropriate data source for some measures, claims do not represent the entirety of the ESRD population, and are also limited in the depth of information available. For these reasons, in furtherance of its obligations under section 1881(h)(2) of the Act, the ESRD QIP first introduced the NHSN BSI Clinical Measure into the ERSD QIP for PY 2016 with the stipulation that a facility must submit 12 months of accurately reported dialysis event data to NHSN on a quarterly basis. Dialysis Facilities, researchers, and patient advocacy groups as well as other stakeholders who have previously submitted public comments on the ESRD PPS Proposed Rule have expressed significant concerns about facilities not reporting dialysis events when they should be reported. These public comments, as well as a thorough review of data reported for the PY 2016 NHSN BSI clinical measure, and results from the NHSN data validation feasibility study, suggest that 23 percent of dialysis events are under-reported, and have clarified the delicate tradeoffs associated with incentivizing facilities to report and prevent dialysis events. To gain a more accurate understanding of the patient population and the data being submitted to NHSN, it is imperative that the data validation study be expanded to include a greater number of facilities.

2. Information Users

Section 1881(h) of the Act requires the Secretary, generally, to adopt a set of quality measures and assess the quality of care provided by renal dialysis facilities using those measures. The measures adopted by the Secretary in satisfaction of these requirements utilize a number of different data sources including the NHSN system, which collects data not otherwise available to CMS. In order to ensure the validity of the data coming into the NHSN system, performance of a validation study is necessary for assessing renal dialysis facility performance on this important measure finalized for the ESRD QIP. Conducting this validation study will help the ESRD QIP and CMS develop targeted strategies for quality improvement efforts at renal dialysis facilities. The information that comes out of the validation study will be used by CMS and others to monitor and assess the quality and type of care provided to ESRD patients, and will be made available to renal dialysis facilities for their use in internal quality improvement initiatives. The information will also be used by CMS to direct its contractors to focus on particular areas of improvement and develop quality improvement initiatives. Most importantly, this information is available to beneficiaries, as well as to the public, to provide renal dialysis information to assist them in making decisions about their health care.

3. Use of Information Technology

As the CDC has stated in previous submissions to OMB, 100% of the data for the NHSN are collected via a secure Internet application. Only the minimum amount of information necessary for the data collection is being requested. Institutions that participate in NHSN are required to have a computer and Internet Service Provider (ISP), and they must provide the salaries of the data collectors and data entry personnel. The NHSN data validation study allows facilities to submit their quarterly data either electronically or on paper. These expenses would not exceed what is normally expended for a typical healthcare facility infection surveillance program. Paper forms are provided to facilities for data collection but they are not required to use them for entry of data into NHSN or for submission to CMS for the data validation study.

4. Duplication of Efforts/Similar Information

The information to be collected is not duplicative of similar information collected.

5. Small Businesses

The CMS ESRD QIP NHSN bloodstream infection clinical measure only applies to facilities that treat hemodialysis patients in-center. Facilities treating 10 or fewer patients are excluded from NHSN Dialysis Event reporting. Individuals on peritoneal dialysis are also not included. Information collection requirements were designed to impose minimal burdens on small renal dialysis facilities subject to the ERSD QIP. Specifically, the NHSN system was created to allow small renal dialysis facilities to enter data via their web-based application rather than using paper-based data submission or employing a full electronic health record, which can be prohibitively expensive for these facilities. As a result, this effort facilitates small renal dialysis facilities’ collection and reporting of required data.

6. Less Frequent Collection

Each month, facilities report the number of maintenance hemodialysis outpatients who were dialyzed in the facility on the first two working days of the month, using the Denominators for Outpatient Dialysis form. This count is used to estimate the number of patients at the facility who are at risk of HAIs. Each month, facilities use a Dialysis Event form to report the details of each of three infection related dialysis events (IV antimicrobial starts, positive blood cultures, and evidence of local access site infection) that occurred among their patients. Due to the seasonal variability of bloodstream infections it is absolutely essential for facilities to report the full 12 months of data to reflect performance over the course of the entire performance period. The data validation study will require quarterly lists of dialysis events. Without this frequency of information collection, CMS would be unable to complete a validation study of a full year’s worth of data.

7. Special Circumstances

The data validation study methodology is designed to collect quarterly data from dialysis facilities. Without this quarterly data, the CMS contractor will not be able to complete the data validation on a full year’s worth of data. We therefore believe that quarterly collection is most appropriate in order to appropriately complete the NHSN data validation study for the ESRD QIP.

8. Federal Register Notice/Outside Consultation

The CY 2018 ESRD PPS proposed rule, serving as the 60-day Federal Register notice published on July 5, 2017 (82 FR 31190).

9. Payment or Gift to Respondent

Dialysis facilities are required to submit measure data to CMS as part of the Conditions for Coverage of End-Stage Renal Disease Facilities (see 42 CFR 494.180(h)). No additional payments or gifts will be given to respondents for compliance with the requirements of the NHSN Data Validation Study for the ESRD QIP.

10. Confidentiality

All information collected under the ESRD QIP will be maintained in strict accordance with the statutes and regulations governing confidentiality requirements for CMS data, including the Privacy Act of 1974 (5 U.S.C. 553a), the Health Insurance Portability and Accountability Act (HIPAA), and the Quality Improvement Organizations confidentiality requirements, which can be found at 42 CFR Part 480. CMS maintains this information in the CMS data warehouse, which contains all information collected under this and other quality reporting and value-based purchasing programs. In addition, the tools used for transmission and storage of data are considered confidential forms of communication and are HIPAA compliant.

11. Sensitive Questions

There are no questions of a sensitive nature being collected as part of this data validation study.

12. Burden Estimates

We have used the following equation to estimate the burden associated with the ongoing NHSN Data Validation Study:

Table A. NHSN Data Validation Burden Estimate Elements

Burden Estimate Element	CY 2018
Number of facilities participating in the NHSN Data Validation study, annually	35
Estimated number of medical records per facility per year	10
Time spent for record collection and submission per facility	2.5 hours (approx.. 0.25 hours per record)
Hourly wage per hour engaged in data collection and submission plus overhead and benefits	$37.36

Under the NHSN data validation study proposed for CY 2018, we propose to sample records from 35 facilities. A CMS contractor will send these facilities quarterly requests.

To derive wage estimates, we used data from the U.S. Bureau of Labor Statistics’ (BLS) May 2015 National Occupational Employment and Wage Estimates. We anticipate that the labor required to collect and submit this data will be completed by either Medical Records and Health Information Technicians or similar administrative staff. The mean hourly wage of a Medical Records and Health Information Technician is $37.36 per hour. Fringe benefit is calculated at 100%. Therefore, using these assumptions, we estimate an hourly labor cost of $37.36 as the basis of the wage estimates for all collection of information calculations in the ESRD QIP. We have adjusted these employee hourly wage estimates by a factor of 100% to reflect current HHS department-wide guidance on estimating the cost of fringe benefits and overhead. These are necessarily rough adjustment both because fringe benefits and overhead costs vary significantly from employer to employer and because methods of estimating these costs vary widely from study to study. Nonetheless, there is no practical alternative and we believe that these are reasonable estimation methods. Assuming full fringe benefits of $37.36, we estimate the total annual burden for the ongoing NHSN Data Validation study for CY 2018 to be $2,226.70.

Table B. NHSN Data Validation Burden Per Facility

NHSN Data Validation Facilities CY 2017	Number of Facilities	Number of Records Per Year	Estimated Time Per Records	Estimated Wage Plus Benefits Per Hour for Record Collection	Annual Hour Burden Per Facility	Annual Burden Per Facility
NHSN Data Validation	35	10	0.25	$37.36	2.5	$93.40

Table C. NHSN Total Data Validation Burden

Basis	Annual Hour Burden	Annual Burden
Each Facility	2.5	$93.40
National	88	$3,269

13. Capital Cost

There are no capital costs.

14. Cost to Federal Government

The cost to the Federal Government includes costs associated with the collection and validation of the data. The validation costs are an estimated $165,000 (FY) annually for the validation contract. The NHSN system is maintained by the CDC. The estimated cost to operate the validation contract includes 1/5 CMS staff at the GS-13 Level (approximate annual salary is $100,000). This results in a total estimated cost of $185,000 annually.

15. Changes to Burden

There are no changes in burden associated with this application.

16. Publication/Tabulation Date

NHSN is an ongoing data collection system and as such, does not have an annual timeline. The data are reported on a continuous basis by participating institutions and aggregated by CDC into a national database that is analyzed for two main purposes: to describe the epidemiology of healthcare-associated adverse events, and to provide comparative data for populations with similar risks. Comparative data can be used by participating and also by non-participating healthcare institutions that collect their data using NHSN methodology.

The reporting institutions will be able to access their own data at any time and analyze them through the internet interface. Reports containing aggregated data will be produced annually and posted on the NHSN website, which is http://www.cdc.gov/nhsn. The report is also published annually in a scientific journal to make NHSN data widely available. Other in-depth analysis of data from the NHSN will be published in peer-reviewed journals, and presented at scientific and professional meetings. The proposed modifications to NHSN will not alter the plans for tabulation, publication, nor the time schedule.

17. Expiration Date

CMS will display the expiration date and the new OMB control number on the collection instruments.

18. Explain any exceptions to the certification statement “Certification for Paperwork Reduction Act Submissions” of OMB form 83-I.

There are no exceptions to the certification statement “Certification for Paperwork Reduction Act Submissions” of OMB form 83-I.