60 Day Public Comment Response

Centers for Medicare and Medicaid Services Response to Public Comments Received for CMS-10630

The Centers for Medicare and Medicaid Services (CMS) received comments from Programs of All Inclusive Care for the Elderly (PACE) organizations, Part D organizations and an advocacy organization related to CMS-10630. This is the reconciliation of the comments.

General Comments:

Comment: Some commenters wanted assurance that organizations could access the PACE Audit Consistency Team (PACT) as needed during the audit. Multiple commenters specifically requested access to the PACT prior to conditions classified as Immediate Corrective Action Required (ICARs) being issued in order to discuss the potential conditions and the basis for those conditions being cited. These commenters also requested access to the PACT when they disagreed with a request for an Impact Analysis (IA). These organizations have felt that this opportunity has not always been available to them, and they want this communication documented in the PACE audit process and data request. One commenter requested clarification on the PACT and asked if that group would continue to exist as the audits become more consistent with the use of core audit teams. This commenter noted that the PACT was to serve as the subject matter experts on PACE and audit policy and ensure consistency in classification of audit conditions across all audits. This commenter also requested that, if the PACT continues to exist, that CMS clarify its composition, role relative to the Core Audit Teams and whether PACE organizations would have an opportunity to participate in the process of classification of audit conditions.

Response: Comments about access to the PACE audit consistency team or PACT are outside the scope of this data collection. However, we note that potential findings are discussed during daily debriefs and at the exit conference with the audit team during which organizations are given an opportunity to discuss the issues identified or ask questions. PACE organizations also have an opportunity to comment on (and dispute) classifications through their participation in the draft report comment process. PACE organizations are encouraged discuss the request for an IA with their audit team who will be able to explain why the request is being made. Additionally, organizations have access to CMS through the [email protected] mailbox to address questions that are not handled by the team directly.

Comment: One commenter noted that CMS audit team was responsive to their questions and always followed up quickly.

Response: We appreciate this commenter’s observation.

Comment: Multiple commenters indicated that they thought some PACE organizations might be disadvantaged by pre-audit disclosures based on how involved their regional account managers are. These commenters indicated that some account managers require monthly meetings and other regions do not have this level of involvement. These commenters asked for clarification on whether disclosures from a PACE organization to the AM in preparation for this type of meeting would count.

Response: We do not believe that PACE organizations will be disadvantaged by how often, or how in depth, a regional offices account manager interacts with them. The ability to disclose a potential issue of non-compliance is open to all organizations equally, and it is up to the organization to implement a system for identifying these issues and disclosing them to CMS. Issues identified by CMS (regional or central office staff) or the State administering agency (SAA) do not count as disclosures for audit purposes. However, if a PACE organization identifies an issue of non-compliance when preparing for a

meeting with the account manager (AM) and promptly discloses the issue prior to the AM identifying it, that action would constitute a disclosure.

Comment: Multiple commenters requested that CMS auditors allow the PACE organization to show auditors where items are located in the medical record before making an official request on the document request log (DRL).

Response: We agree that, when feasible, auditors and the PACE organization should have open communication regarding where items are located in the medical record, and an auditor may ask a PACE organization to point to documentation that is difficult for the auditor to locate. However, there may be situations where the audit team may need to request documentation through the DRL, including when documentation is not readily accessible/available in the record.

Comment: Multiple commenters noted that some organizations are required to upload numerous sometimes duplicative documents in response to the audit team which can increase burden. Commenters also indicated that the PACE protocol (and the process to request IAs under PACE) was modeled after the protocol applied in the MA and Part D programs is not appropriate for measuring PACE organizations that directly provide care to participants. The commenters stated that they are not aware of a provider-based audit in which this level of activity would be required and question whether comparable document requests are made of Medicare Advantage Organizations (MAOs) or Prescription Drug Plan (PDP) Sponsors considering the types of documentation they have available to them. The commenters suggested that CMS either modify the PACE protocol or implement it differently, and PACE organizations should not be required to provide the level of detailed information being requested.

Response:

We appreciate these commenters concerns and understand that PACE organizations are both an insurer for purposes of implementing the Medicare program, and a direct care provider. The PACE protocol was developed to assess compliance with statutory and regulatory requirements that are applicable to PACE organizations and because it is based on the Medicare Advantage and Prescription Drug program audit protocol, it shares a similar structure and method for ensuring that enrollees receive the benefits they are entitled to under the PACE program. The current protocol has helped CMS improve the PACE program by identifying non-compliance and ensuring organizations understand the actions that are needed to correct deficiencies. However, we understand the commenters’ concerns and, in this revised protocol have sought to strike a better balance between ensuring participants are receiving appropriate and timely care and services, and not overwhelming organizations with intensive data requests over the course of the audit. CMS has therefore taken a number of steps, which are discussed throughout the comments and responses, to eliminate or streamline our requests for data where the burden associated with its collection outweighs the value of the data from an oversight perspective. We will continue to evaluate whether we can obtain information from other sources and train auditors to request only what they need to evaluate an organization’s compliance with CMS requirements. We will also continue to identify opportunities to reduce burden when it comes to documentation requests while still ensuring CMS’ ability to effectively monitor organizations for compliance with regulatory requirements.

Comment: Multiple commenters indicated that HPMS functionality has been greatly improved, and also indicated that they appreciated CMS adding an audit survey to the collection package.

Response: Thank you for these comments.

Comment: One commenter requested clarification on the role of the State administering agency in CMS audits. The commenter noted that the 2020 protocol is silent as to the expected role of the State Agency within the CMS audit process and requested that CMS clarify the role. The commenter also noted that a state could issue a separate audit report and request additional corrective actions outside of the CMS report, which could duplicate efforts and create confusion.

Response: The audit protocol under this package only applies to CMS and would not govern state auditors. Although CMS works in cooperation with States to conduct reviews of a PACE organization, we do not have the authority to define the scope of their review or the timing of their audits. However, in an effort to help PACE organizations track and manage CMS and state findings, CMS has enabled States to use our Health Plan Management System (HPMS) to upload State reports and accept corrective action plans from PACE organizations. We appreciate this comment and will continue working cooperatively with State administering agencies to oversee the PACE program.

Comment: One commenter requested that CMS require draft reports to be issued within a specific timeframe and to not use an approximate timeframe. This commenter also asked if the approximate timeframe was still 60 days from the date of the exit conference.

Response: While we are committed to ensuring we issue draft reports in a timely manner, we cannot always anticipate circumstances that delay that issuance. Therefore, we cannot commit to an exact timeframe for issuing a draft report.

Attachment I- PACE Audit Process and Data Request:

Comment: Multiple commenters requested clarification on some of the references in the protocol and other documents. This included asking for clarification around whether "manual" meant "PACE manual", and asking for CMS to clarify that Level I and Level II was really PACE quality data.

Response: We appreciate this opportunity for clarification. We are confirming that we did intend "manual" to mean the "PACE manual", but have removed those references throughout since our audit focuses on regulatory requirements. Additionally, we will change all references of Level I and Level II to PACE Quality Data.

Comment: Multiple commenters requested clarification on what CMS wanted included within the PACE organizational chart, and whether this would just be senior leadership and not all staff at the organization.

Response: Upon further review, we believe we can obtain the information we need through other data

collected before or during the fieldwork stage of the audit. Therefore, we will not request an organizational chart as part of the pre-audit documentation.

Comment: Multiple commenters requested clarification on the timing of samples, specifically whether

CMS would provide two business days for the Service Delivery Requests, Appeals, and Grievances

(SDAG) and personnel samples to be pulled regardless of whether the elements were done remotely or onsite. Multiple commenters also noted that CMS had adjusted the timeframe for providing the Clinical Appropriateness and Care Planning (CACP) medical record samples to the PACE organization from one business day to one hour. These commenters requested CMS revert back to allowing for one business day to prepare samples for CMS review. Some of the commenters noted that PACE organizations may need time to gather medical record documentation from other centers in order to provide the full record for review. A few commenters requested confirmation from CMS that we would not expect all paper records and documentation to be gathered and uploaded within that timeframe.

Response: We appreciate the opportunity to provide clarification. We intend to provide the samples for

SDAG and personnel two business days ahead when we are conducting those elements via desk review (or remotely). If we review those elements onsite, we will discuss with the PACE organization how records are kept to determine if two business days are necessary to pull case files. Additionally, the medical record samples are provided as a courtesy to the organization and we do not expect that organizations scan, print, or upload medical record documentation related to those samples within an hour from our request. Rather, organizations are expected to provide CMS immediate access (whether through remote access, webinars, or computers onsite) to the medical records for each sample within that time, in whatever manner was agreed upon by the audit team and organization in advance. Therefore, we are keeping the timeframe of one hour for samples to be provided. Organizations will have a longer time to compile information and upload it as needed. We added clarifying language to the protocol in order to ensure organizations understand our expectation.

Comment: One commenter requested that CMS consider extending the timeframe from two business days to five days for organizations to prepare and submit sample case files. This commenter noted that it is sometimes difficult to find all pieces of the case file within the medical record within that timeframe.

Response: We do not believe five business days is a reasonable timeframe based on our experience auditing organizations from 2017 through 2019. The case files requested in this timeframe are generally small (i.e., not many documents) and do not require organizations to spend long amounts of time pulling different parts of the record in order to compile case files. We are therefore retaining the two business day requirement.

Universes - General:

Comment: Multiple commenters concurred with CMS' decision to shorten the universe timeframe from one year to six months of data. They noted that this reduction would decrease burden in pulling the requested universes.

Response: We appreciate the commenters support for decreasing the timeframe for universes from one year to six months. We agree that it should reduce burden and assist organizations with better populating that data.

Comment: These same commenters, however, noted that the increase in the amount of information collected would negate the benefit of shortening the timeframe. These commenters noted that there would be a large burden in compiling these data since it would likely require a full medical record review for all participants. Some commenters estimated that it would take approximately 750 hours to pull just the List of Participant Medical Records (LOPMR) universe. Some commenters noted that other types of audited organizations are not required to provide this level of data.

Response: We collect similar universes from all audited organizations, but the collection of participantspecific data is unique to the PACE universes because, unlike Medicare Advantage organizations, PACE organizations provide direct participant care to the frail elderly and we believe the collection of participant-specific data in PACE is necessary in order to ensure CMS can select clinically significant samples. However, in response to the commenters’ concerns, we carefully reviewed this universe and have eliminated some of the fields and made other changes which we discuss in more detail below.

Comment: Several commenters requested CMS provide PACE organizations with excel templates that include the required record layout. These commenters indicated these templates would be helpful for organizations and requested them by July 2019, or if not by July, by October 2019.

Response: We appreciate commenters request for templates and are committed to providing templates that align with the requested record layouts. We will provide these templates as soon as possible once the protocol has been approved through the PRA process.

Comment: One commenter requested clarification on whether the 2020 record layouts would have field limits like the 2016 record layouts.

Response: Thank you for this question. In the 2020 record layouts, we eliminated the field restrictions that limited the number of characters an organization can put into a row.

Comment: One commenter noted that an organization with multiple centers would have a harder time compiling data and universes for all participants than an organization with only one center. One commenter requested that, for POs that operate multiple centers, CMS limit the universe request to only one of the centers. As an alternative to one center, commenters suggested CMS could sample a proportion of PACE centers and/or interdisciplinary teams rather than request information from the entire contract.

Response: While we appreciate this commenter's suggestion, we must ensure that we have information on all participants in order to ensure compliance across all the centers. However, we are cognizant of the burden audits place on PACE organizations and, as noted in the section of this document related to universes, we are eliminating some of the fields in the universes to create additional efficiencies. We also note in the section of this document related to Impact Analyses (IAs), that we are eliminating some of the IA requests, however, eliminating the IAs increases CMS’ need to collect as much information in advance of the audit as possible in order to ensure we select an appropriate sample set.

Comment: One commenter requested CMS to continue utilizing a 30 calendar day timeframe for universe and data submissions instead of adopting a 20 day timeframe, arguing that 20 days was too short for the amount of information requested.

Response: We appreciate the opportunity to provide clarification. We determined in 2020 to no longer utilize a 30 calendar day timeframe because it allowed for situations where the timeframe would end on a weekend or holiday. Instead, we converted the timeframe to a 20 business day timeframe. We believe this change will not have a substantial impact on PACE organizations, because the 20 business days will not count weekends or holidays, and the due date will be within a day or two of the 30 calendar timeframe, but will always fall on a workday. We are therefore continuing to use a business day timeframe instead of calendar day timeframe for universe and data submissions.

Comment: One commenter requested clarification on whether CMS intended to hold PACE organizations accountable for submitting accurate universes within 3 attempts given the number of significant changes in the 2020 protocol.

Response: Yes. As mentioned in the second section of the protocol, under "Responding to Universes and Documentation Requests", we intend to limit the number of submissions a PACE organization makes when we collect information for audit. Doing so is consistent with the program audits, and we believe three attempts strikes the right balance between giving PACE organizations sufficient opportunities to provide accurate and complete data with CMS’ need to conduct audits in an effective and timely manner. We also want to reiterate that we encourage organizations to be audit ready by tracking this information and being able to efficiently create universes when an organization receives an audit engagement letter. As discussed below, we carefully reviewed the universes in light of the comments about burden and are now reducing the number of fields in the LOPMR universe from 49 to

40.

Universes - Service Delivery Requests:

Comment: Multiple commenters requested that CMS add an option for "withdrawals" into the service delivery request universe to account for situations where a participant or representative requests a service but then withdraws the request prior to the organization rendering a decision. One commenter also asked if dismissed or cancelled were valid options. If any of these fields were valid, the commenter recommended CMS allow for an entry of “NA” for the fields “Reason for Denial”, “Extension”, and “Date Service Provided”, and asked for clarification on whether organizations should populate the fields “Date of Oral Notification” and “Date of Written Notification” for these dispositions.

Response: We agree with commenters that we should add a disposition of withdrawn since a participant and/or designated representative always has the ability to withdraw a request. We are not adding options for cancel or dismissed since organizations would be expected to process all requests unless they are withdrawn. We have also added the option to put "NA" in column L (Reason for Denial) when requests are approved or withdrawn. We do not agree with entering “NA” into column M (Extension) because, even when the request is withdrawn, an organization should be able to answer whether it took an extension or not. We are modifying column P (Date Service Provided) to include "withdrawn" as a reason for entering “NA” in that field. As for the notification fields (oral and written), we are not making any adjustments to those fields because we currently allow the option of “NA” if notification was not provided or documented, and organizations already have the ability to enter “NA” in those fields for those reasons. We have added a reminder to Appendix A of the protocol that all fields must be completed.

Comment: Multiple commenters asked CMS to amend column J so a PACE organization can report that multiple assessments have been completed for a participant, and asked for clarification on how organizations should populate that field in those circumstances.

Response: We appreciate this opportunity for clarification and we have updated this field. Organizations would indicate “Yes” if any assessment was completed in-person, “No” if assessments were completed but none of them were in-person, and “NA” if no assessments were conducted at all or no assessments were conducted in response to a service delivery request.

Comment: Multiple commenters requested that CMS allow for “NA” to be added to the field description in Column L for when requests are approved.

Response: We agree with these commenters and have added the option to put "NA" in column L when requests are approved or withdrawn.

Comment: One commenter requested clarification on how service requests and appeals in the PACE protocol relate to Part D coverage determinations and appeals under 42 CFR 423 Subpart M. The commenter asked if CMS wanted Part D coverage determinations and reimbursement requests included in Table 1 (Service Delivery Requests), and if Part D appeals should be included in Table 2 (Appeals). The commenter also asked for additional clarification on the difference between a service delivery request and a coverage determination in Part D, and on the compliance standards that CMS would apply for purposes of assessing a PACE organization’s compliance with Part D requests and appeals.

Response: Service requests related to medication requests and Part D coverage determinations are two distinct determinations. For purposes of PACE audits, we are only collecting service delivery requests processed under 42 CFR 460.104(d)(2) and the subsequent appeals processed under 42 CFR 460.122 (collected in Tables 1 and 2 respectively). Any request processed as something different (for example a Part D coverage determination or redetermination) would not be included in the audit universes under this protocol. Similarly, we would not assess compliance with the Part D coverage determination or redetermination requirements under this protocol and would only assess the requirements found under the PACE regulations relating to service delivery requests and appeals.

Universes - Appeals:

Comment: Multiple commenters noted that some of the information requested in audit universes is also collected for quality reporting. These commenters requested CMS to consider if the requests could be aligned to make them more consistent and reduce burden. Specifically, commenters requested that we consider aligning the grievance universe and the appeals universe with the quality reporting measures.

Response: We reviewed both the grievance and appeal record layouts to determine where we could align our requests with the quality reporting requirements in order to eliminate unnecessary burden. For grievances, we modified the grievance source field (i.e., the person who initiated the grievance) in the record layout to include the options of family, caregiver, or participant. We also are excluding the following reporting requirement fields from the record layout because CMS either already has the data or does not need the PACE organization to report it again: contract number, site name, location, grievance type-other, specific issue-other, resolution, alternative solution, and actions taken. Lastly, we believe the quality data fields “grievance type” and “specific issue” are already accounted for in our audit universe through a similar field. For “grievance type”, the reporting requirements allow for limited options, however the field in the record layout is a free text field because CMS does not want to limit the types or grievances a PACE organization can report. For the “specific issue” field, a PACE organization can currently enter the description of the grievance, but we are now modifying the names of these columns (columns G and H) to more closely align with the column names for reporting requirements.

We also reviewed the appeals universe to determine where we could align collection tools. We note that, in order to have an accurate universe for audits, CMS needs to collect information at a more granular level than quality reporting data which is aggregated at the contract level. In addition, we do not believe we need to add fields for “Site Name”, “Appeal Type-Other”. Based on commenters’ suggestions, we did modify the free text field, “Category of the Appeal” to include the options identified in the reporting requirements. We also modified the name of this field to include the reporting requirement name “Appeal Type” and we modified the description of the field “Person who Submitted the Appeal” to allow PACE organizations to report participant, caregiver, or family under “Appeal Source”.

Comment: Multiple commenters requested that CMS add an option for "withdrawals" into the appeals universe to account for situations where a participant or representative appeals a denial but then withdraws the request prior to the appeal being resolved. One commenter also asked if “dismissed” or “cancelled” were valid options. If any of these options were valid, the commenter recommended CMS allow for an entry of “NA” for the fields “Reason for Denial” and “Date Service Provided”, and asked for clarification on whether organizations should populate the fields and “Date of Written Notification” and "Time of Written Notification" for these dispositions, and whether they required a response.

Response: We agree with commenters’ suggestion to add a disposition for withdrawn since a participant and/or designated representative always has the ability to withdraw an appeal. We are not adding an option for cancelled or dismissed since organizations would be expected to process all appeals unless withdrawn. We agree with these commenters and added the option for PACE organizations to enter "NA" in column L (Reason for Denial) when appeals are approved or withdrawn. We are not modifying column O (Date Service Provided) because the field already allows for “NA” to be entered if the request is not approved (e.g., denied or withdrawn). However, we already allow for an entry of “NA” for the notification fields (Date and Time of Written Notification) and therefore we are not making any adjustments to those fields. As noted previously, we have added a reminder to Appendix A of the protocol that all fields must be completed.

Comment: One commenter requested confirmation that the field "quality analysis" was removed from both the appeals and grievances universe.

Response: Yes, we intentionally removed that field from both of those universes.

Comment: Three commenters expressed concern at the addition of two fields in the appeals universe, specifically "Time Appeal Received" and "Time of Written Notification". These commenters indicated that they knew the requirement for expedited appeals was in hours, but expressed concern that the request for a timestamp would impact both expedited and standard appeals as changes would have to be made to their systems and it would increase burden.

Response: We thank the commenters for raising these concerns, however, timeliness of expedited appeals is measured by hours, and therefore these fields are necessary in order to determine the timeliness of expedited appeals which is done at the universe level in PACE. We also note that the two fields referenced are not new and are included in the current approved PRA package.

Universes - Grievances:

Comment: Multiple commenters requested that CMS provide a way for organizations to indicate when a participant or caregiver does not want to be included in the grievance process, or does not wish to receive notification of the resolution. These commenters indicated that doing so would allow for realworld scenarios where participants or caregivers don't want to be involved in a formal dispute resolution process.

Response: We agree with commenters that while a participant may initiate the grievance process by making a complaint, they may not want to receive notification of a formal resolution. We have therefore allowed for a new response option in the "Date of Resolution Notification" field. The new option will be for organizations to enter “NNR” when the participant and/or caregiver specifically requested not to receive notification of the resolution.

Comment: One commenter requested that CMS retain both the "Date of Oral Notification" and "Date of Written Notification" fields in the grievance universe instead of the proposed combined field.

Response: CMS allows organizations to determine how they will notify a participant of the grievance resolution (oral and/or written). Because CMS only needs to know the date the first resolution notification was made (regardless of whether it was oral or written), two fields are not necessary.

Comment: One commenter requested that CMS align the grievance universe with the service delivery request and appeal universe by using the terms "participant and/or designated representative" instead of "participant and/or caregiver".

Response: We appreciate this suggestion. Based on other comments received, we modified the appeal and grievance universes to align with the already collected quality data, which allows for "participant, family, or caregiver" as valid options. Therefore, we do not intend to add "designated representative" at this time.

Universes- List of Participant Medical Records (LOPMR):

Comment: Multiple commenters requested we modify Column AH "Number of Falls" to reference PACE Quality Data and not Level II events.

Response: We agree and have made this change.

Comment: One commenter requested that CMS consider adding "NA" as an option for fields, "currently in SNF/NF" and "Direct SNF Admission".

Response: We do not believe “NA” needs to be added as an option because the fields specifically say that the commenter may enter “N” if the field does not apply.

Comment: One commenter requested that CMS add another field, “Type of Transplant (i.e., kidney, heart, etc.)” and suggested CMS move the “Dialysis” field directly after this new field.

Response: We do not believe we need the type of transplant to be included in the universe submission as this information would be apparent during the review of the medical record. Because we are not adding this new field, we are also not moving the dialysis field from its original location.

Comment: One commenter requested we change the column regarding medication administration to replace "dispensed" with "administered".

Response: We agree and have made this change.

Comment: One commenter requested that CMS add “NA” as an allowable option for the field "Limitation on Opioid Usage". Another commenter requested that CMS define “limit” for purposes of populating that field.

Response: We are removing this field from the record layout while we investigate whether we can obtain this information from other data sources.

Comment: One commenter requested confirmation that the two fields "Psychoactive Medications" and "Impaired Hearing" were deliberately removed.

Response: Yes, we removed those fields from the 2020 PRA package.

Comment: Multiple commenters were concerned with 10 new columns that CMS proposed to add to the participant medical record universe. Of those 10 columns, four were related to specialist medications (Column X- Specialist ordered medications, Column Y- Delivery of Specialist Ordered Medications, Column Z- Specialist recommended medications, and Column AA- Delivery of Specialist recommended medications). A few commenters indicated that they were concerned by the two fields in the participant medical record universe related to specialist ordered medications. These commenters indicated that specialists often recommend medications but leave it to the PACE provider to order the medications. They recommended we remove the fields related to specialist ordered medications and only retain the fields related to specialist recommended medications.

Response: We appreciate the comments on the potential burden of collecting this information. While we believe that PACE organizations must be able to keep track of participants’ visits to specialists and what medications are recommended and ordered as a result of those visits, we agree that populating this into an audit universe would potentially be burdensome and do not believe the value of the data for audit purposes outweighs the burden associated with reporting it. Therefore we removed all four fields related to specialist ordered/recommended medications (Columns X, Y, Z, and AA) from the universe. In total, we eliminated nine fields from this universe.

Comment: A few commenters requested that we eliminate column AB related to participant pain. These commenters noted that this column was overly broad and would be intensive to pull this information. One commenter requested that if CMS keep this column, that “NA” would be an allowable option.

Response: We agree with commenters and do not believe the value of the data for audit purposes outweighs the burden associated with collecting it. We therefore removed this column from the record layout.

Comment: Multiple commenters had concerns regarding the proposed change to column AN “Significant Weight Loss” and the newly added column (AO) related to significant weight gain. The commenters noted that these fields’ utilized weekly or daily weights, which implied a participant needed to be weighed daily, which organizations indicated was unreasonable. These commenters requested that the fields be removed or CMS change the description to meet the minimum data set used by

nursing homes (i.e., loss equal to or greater than 5% within a 30-day period or 10% within a 180-day period.)

Response: We thank these commenters for their suggestion on these metrics. We originally added weight gain as a measure to assess congestive heart failure (CHF) exacerbation, which is not measured by a weight gain over a month or 180 days, but rather by a sudden weight gain in a short amount of time (i.e., 24 hours to a week). We are not implying that PACE organizations must weigh participants daily, rather, we are asking if the organization had noted during the data collection period a sudden weight gain that required action or attention. Since the intent of this field was to understand whether participants may have experienced a CHF exacerbation during the data collection period, we removed the weight gain field and replaced it with a “Yes/No” field that asks if participants with CHF had an exacerbation during the data collection period. For the field related to weight loss, we agree with commenters that we could use the nursing home guidelines and we amended the field description to align with that definition of weight loss.

Comment: Multiple commenters raised concerns regarding the burden of populating columns AP through AR (low blood glucose, high blood glucose, and oxygen saturation). These commenters stated that a full medical record review would have to be done to find this information and suggested CMS delete these fields. A few commenters also suggested that these data elements were not the most appropriate measures of whether diabetes is being appropriately controlled and suggested CMS use Hemoglobin A1C instead. One commenter also indicated that since CMS already asks if the participant has diabetes, that the glucose information is not necessary. Lastly, one commenter noted that Oxygen Saturation was duplicative of another field and suggested it be eliminated.

Response: We agree with most of the commenters’ suggestions and removed the three fields related to blood glucose and low oxygen saturation. We do, however, think that it is important that PACE organizations are aware of when a participant may experience acute events related to these issues, so we added a field related to emergency room (ER) visits and hospitalizations to determine if a participant visited an ER or was seen at the hospital for hypoglycemia, hyperglycemia, and/or oxygen saturation. Because this would only be relevant to participant hospital visits, we believe this “Yes/No” field will be much less burdensome to populate.

Universes-Personnel:

Comment: One commenter requested confirmation that only personnel from the PACE organization would be included in the universe.

Response: No, we do not want only PACE organization staff in the personnel universe. The PACE regulations require that all staff, employees, and contractors adhere to the personnel requirements. Since there may be a large number of contracted individuals, we are only asking for contracted staff that provide care/services at the PACE center or in the home, or transport the participant (e.g., van drivers) in the universe.

Comment: Three commenters asked for clarification on how to fill out the field relating to whether the person is an interdisciplinary team (IDT) member in the personnel universe. Specifically, how PACE organizations should treat a scenario in which the IDT member has designated someone who is appropriate to fill in for him or her. For example, if the supervising Registered Nurse (RN) steps in for the usual IDT RN, would the PACE organization indicate that the person is a member of the IDT?

Response: We would encourage the PACE organization to enter that an individual is a member of the IDT if they have any role on the IDT, even if it as a second RN or a person designated to represent or cover for a primary discipline. In the commenters’ example, the PACE organization would indicate that both the supervising RN and the “usual IDT” RN are members of the IDT.

Service Delivery Requests, Appeals, and Grievances (SDAG) Element:

Comment: Multiple commenters requested policy clarification on the definition of a service delivery request. Specifically, commenters requested clarification on service delivery requests compared to treatment discussions, and asked when it might be appropriate for an organization to process requests outside of the service request process. These commenters indicated policy clarification regarding the definitions would increase transparency and clarity on audit, and indicated they were concerned that auditors were being overly broad when reviewing records related to requests for items and/or services.

Response: The majority of the commenters’ requests are policy questions that are outside the scope of this package and can only be addressed through the rulemaking process. As for audits, auditors are trained to review documentation consistent with the regulatory requirements, and CMS will continue to monitor and work with the auditors to ensure they do so.

Comment: One commenter requested that CMS allow PACE organizations to accept assessments conducted prior to the service request to alleviate the burden of conducting two face to face assessments.

Response: We appreciate the comment. The part of the comment about the policy of service delivery requests and the assessments conducted in response to those requests is outside the scope of this package and can only be addressed through the rulemaking process. However, we want to reiterate that, for purposes of audit, CMS accepts assessments conducted prior to bringing the request to the full IDT, so long as the assessment is conducted in response to the request being made by the participant or designated representation, and the IDT determines that the assessment was conducted by the appropriate member of the IDT.

Comment: Most commenters requested clarification on the types of complaints that are considered grievances. Some commenters requested that PACE organizations be granted more latitude in determining what constitutes a grievance and how those complaints are processed. These commenters mentioned that, if organizations had more latitude in making that determination, the number of times CMS required a root cause analysis (RCA) or IA during audit would decrease.

Response: We thank these commenters for their recommendation, however the classification or definition of a grievance is outside the scope of this package and can only be addressed through the rulemaking process. For purposes of audit, auditors review participant information for grievances as defined by the PACE regulations.

Comment: One commenter noted that the notification timeframe for appeals was missing "representative" in the compliance standard language.

Response: Thank you for this comment. We added "representative" as recommended.

Comment: One commenter noted that there was an extra "s" in the appeal notification timeframe guidance in the audit protocol.

Response: We appreciate the commenter drawing our attention to that mistake, and removed the extra

"s".

Comment: One commenter requested confirmation that CMS will no longer assess timeliness for notification of grievances. This same commenter requested clarification on CMS’ requirement regarding notification of a grievance resolution, and whether oral or written notification is required.

Response: Thank you for this comment. In the 60-day PRA package for the 2020 protocol, we did not include the timeliness test in the grievance universe. PACE organizations are required to develop their own internal policies relating to the timeliness of grievance resolution notifications and without a standard notification requirement, we have seen vastly different time frames among PACE organizations which presents challenges in assessing organizations consistently across audits. In response to the question about what the policy requires for grievance notification, that question is outside the scope of this PRA package and can only be addressed through the rulemaking process.

Comment: A few commenters asked for CMS to clarify what auditors would do if an organization did not have five approved appeals for the audit team to sample.

Response: We appreciate the request for clarification. We noted in our protocol that CMS reserves the right to adjust the number of samples if a given universe does not contain enough cases to sample. For example, if a PACE organization does not have five approved appeals in the universe period, CMS would be unable to select five approved appeals for sampling, and would instead select additional denied appeals or approved service delivery requests to make up the difference. We added additional information into the SDAG element section of the protocol to clarify when CMS may expand sample review beyond what was initially noted as being required.

Provision of Services (Formerly CACP) Element:

Comment: Multiple commenters requested clarification on the compliance standard regarding Medicare and/or Medicaid benefits without limitation to amount, duration, or scope. These commenters requested that CMS confirm that this compliance standard was not intended to imply that the IDT does not have a critical role in assessing participant needs and developing care plans in collaboration with the participant and/or his/her caregiver that are inclusive of/limited to services intended to address these needs.

Response: Nothing in our protocol is intended to negate the role of the IDT in determining appropriate care for participants.

Comment: One commenter requested that CMS clarify how we intend to assess whether the IDT remained alert to pertinent information.

Response: We appreciate this commenter’s request for clarification, but we do not share the methods of evaluation that we use to assess compliance with specific requirements. However, CMS will ensure that we are transparent in regards to the reasons for a condition or why we are concerned with a specific situation, and we encourage organizations to have open discussions with their audit teams to ensure transparency and understanding.

Comment: One commenter requested clarification on whether CMS would be assessing a PACE organization’s emergency response plan or disaster plan as a part of assessing its emergency equipment.

Response: We appreciate this commenter’s request for clarification. CMS does not intend to review emergency preparedness or disaster preparedness plans as a part of the compliance standard for emergency equipment.

Comment: Multiple commenters requested CMS to allow a PACE employee to sit with CMS during our review of medical records in order to assist with navigating the record and finding documentation. For instances when CMS conducts remote medical record review, these commenters requested that CMS set up a webinar to allow employees to view the review. These commenters indicated they believed this practice would reduce the amount of documentation requests made by auditors while onsite. Additionally, these commenters raised concerns regarding HIPAA and commented that this practice would ensure that the organization would be able to tell participants when their medical records were accessed.

Response: We thank these commenters for their suggestions and we continue to be committed to ensuring transparency during our audit with sample review. While we do not require PACE staff to sit with us during our review in order to ensure that we are not placing undue burden on the organization, we do allow PACE staff to be as involved as they want. If a PACE organization wants to have a staff member shadow the review that would be acceptable so long as the staff member does not hinder or delay the review in any way. CMS would need to be able to independently navigate the record, but it would be beneficial to have a staff member available for questions on how to find particular records. We do not anticipate that this practice would reduce any of the documentation requests that auditors make as we would still need to request supporting documentation and responses to questions through the normal process of the audit. Additionally, organizations are notified in advance of any records that are accessed during audit and CMS audits do not violate HIPAA privacy standards.

Onsite:

Comment: Some commenters noted that the new Attachment IV Onsite_Obs_ParticipantList was overly burdensome. The commenters stated it seemed like CMS wanted to know all participants receiving a particular type of care during the onsite audit week, including medication administration, wound care, home care, and specialty diets. The commenters stated that providing that amount of information in the excel document CMS created (Attachment IV) would be very time consuming, and asked CMS to consider other ways to request the information. One suggestion was that CMS select only one center and limit requests to participants in that center. Another suggestion was that CMS ask for a percentage of participants across all centers instead of all participants. Lastly, these commenters indicated that the "type" of care received (i.e., type of medication administration, type of wound care, etc.) field would be the hardest to populate.

Response: We thank the commenters for raising this concern and providing suggested alternatives. We want to confirm that we are requesting this information for all participants receiving these types of care/services during the onsite week, regardless of whether the care/services are performed at the center or in the home. We cannot limit the request to only one center or to only a sample of participants as we believe we must have a full listing in order to appropriately assess what services are being provided and determine who we need to review during the onsite observations. We do not believe this will be overly burdensome because PACE organizations should already have methods in place to identify when services/care should be provided in order to ensure that they are actually providing the necessary care to participants. We understand, however, that this information may be in a format that would not easily translate to an excel document. Therefore, we are making Attachment IV an optional template. While we will still request all the information identified in Attachment IV to be provided to the audit team, we will accept the information in any format the PACE organization can provide. For example, if a PACE organization can print out a listing of all participants receiving wound care during the onsite week (and when care is scheduled), we would accept it. Additionally, while the type of service should be readily accessible for most services (meals, medications, etc.), if a PACE organization does not have that information immediately available, it can provide all other data to the team and the team will ask for specifics as needed.

Personnel:

Comment: One commenter requested clarification on what trainings are considered part of the competencies that CMS will be checking. This commenter noted that CMS noted specific trainings in compliance standards in the 2016 protocol.

Response: We appreciate the chance to provide clarification on this requirement. CMS made the determination that we will no longer be routinely auditing specific trainings for personnel other than the competency evaluations that PACE organizations perform prior to an individual performing direct participant care. We will no longer be routinely checking other trainings, such as OSHA or discipline specific training, as a part of the personnel review. Instead, we may request evidence of trainings on an ad hoc basis during the course of investigating other issues. For example, if there is an infection control issue discovered during the review of medical records, auditors may request to see what OSHA or infection control trainings staff may have received prior to that issues occurring.

Root Cause Analyses (RCAs) and Impact Analyses (IAs) - General

Comment: Multiple commenters indicated that the process for collecting RCAs and IAs for all conditions of non-compliance is overly burdensome and requires organizations to pull staff from providing direct participant care in order to populate the information. These commenters indicated IAs often require a full medical record review of all participants, and can take staff, including clinical team members and IDT members, away from providing direct care in order to fill out the documents. These commenters also indicated that CMS’ burden estimate significantly underestimated the burden associated with these IAs.

Response: We appreciate the comments regarding the potential burden of having to complete IAs following the audit. The IAs are important tools that allow an organization to investigate noncompliance in order to fully correct an issue, which we believe cannot be properly resolved without the PACE organization understanding the full scope of the problem. These analyses are only requested when non-compliance is discovered during audit fieldwork and the audit team determines that it needs to understand the scope of non-compliance. We acknowledge that there is burden associated with completing these analyses, as they often require personnel to review medical records to determine all of the participants who were impacted. When considering the issue raised by the commenters, CMS weighed the value of the requested information from a compliance and participant protection perspective against the potential burden associated with requesting it. We reviewed all of the proposed IA templates with that standard in mind and determined that CMS can remove some fields from some of the IA templates, and can eliminate some of the IAs where the data in the Root Cause Analysis (RCA) would be sufficient to ensure the PACE organization can begin correcting the non-compliance. We discuss the changes in more detail in the comments and responses that follow.

In addition to those changes, we believe we can limit the scope of the IAs that we intend to retain as proposed in the 60-day PRA package. We initially proposed for organizations to review all participants in order to determine the full impact of an identified compliance issue. For those IAs where the review is already limited in nature (for example, if an IA review only requires the PACE organization to review denied service delivery requests to determine the total impact), we are keeping the scope of the IA the same as before (i.e., an organization will be asked to review all denied service delivery requests within the IA review period). However, for IAs that require the organization to conduct a full medical record review in order to determine the impact, we are reducing the scope of IA reviews to 50% of the remaining enrolled participants that were not already reviewed by CMS. For example, for a PACE organization that has 80 participants, and CMS reviews 30 medical records and finds an issue of noncompliance that requires the review of participant medical records to determine the impact, we would request that the organization review 25 participant medical records. CMS auditors would select the participants that should be reviewed, and require the organization to assess this subset of participants for the requested IA. Additionally, for the personnel IA, we will implement a similar process and will ask the organization to review 50% of the remaining personnel that may have been impacted by the condition. For example, if the condition is only assessed for new-hire employees or contractors, we will ask the organization to review 50% of their remaining new-hire personnel records (that were not already reviewed by CMS). We will clarify the scope of review for each IA on the instruction page.

Comment: One commenter commended CMS on making the IAs more specific and targeted.

Response: We appreciate this commenters input.

Comment: Multiple commenters requested that individual columns be removed from the IA templates CarePlanContent1P84, CenterSrvcs1P01, CarePlanPartCGInvolvement1P20, EmergencyCare1P79, PACEIDT1P101P131P15, CDC1P25, MedRecs1P22, PracticeScope1P33, and SDRExtensions1P58.

Response: We removed these IAs from our collection and will only request a Root Cause Analysis for the related conditions. We believe that with the increased number of samples reviewed during audit and the supporting information provided for each case file, there will be sufficient information to determine when corrective action is necessary and the underlying cause of the non-compliance for the conditions associated with the IA templates noted in the comment.

Comment: Multiple commenters asked CMS to remove columns related to participants suffering negative outcomes as a result of non-compliance from some of the IAs. The commenters indicated that requesting that information would require clinicians to review all medical records to determine if the participant suffered harm as a result of the violation.

Response: We appreciate the chance to address these concerns. We believe that it is vital that organizations are monitoring their performance and determining when non-compliance may result in a negative outcome to a participant. We have reviewed the IAs in light of the commenters’ concerns and are keeping the negative outcome field for IAs where there is a greater potential for harm and are eliminating this field from the following IAs were a negative outcome may be less likely to occur: Assessments1P491P501P82, AppealExt1P71, Appeals1P651P661P681P73, Personnel, and SDRs1P601P611P85.

Comment: Multiple commenters requested that CMS consider an alternative approach to requesting IAs, as the impact on a direct care provider was much greater than an MAO. The commenters requested that instead of PACE organizations conducting a review of all participants, that CMS employ a sampling methodology for IAs in order for PACE organizations to determine the scope of non-compliance with 100% of samples being required only in much more limited situations. These commenters recommended that the percentage of the population included in the IA should decline as the number of records increases, e.g., for an IA focused on participants enrolled during the audit review period, for a program of less than 50 participants, 25% of participants; for a program of between 51 and 100 participants, 20% of participants; for a program with more than 100 participants, 15% of participants. If the IA suggests that non-compliance is widespread, a more comprehensive review could be included as part of the PACE organization’s correction action plan (CAP), but the PACE organization would not need to extend the IA to additional participants. These commenters also indicated that the burden on reviewing records rest on the PACE organization, and indicated that CMS should take on additional burden if more information is needed.

Response: IAs are an effective mechanism to determine the cause and scope of an issue. Through the use of these tools, we have identified issues that were unknown to PACE organizations or the scope of the issues were not known. CMS will continue to use IAs for PACE audits for the reasons previously mentioned, but as discussed, we reduced the number of IAs that will be requested, made modifications to streamline the remaining IAs, and reduced the breadth of review required for some IAs to 50% of remaining enrolled participants (not already sampled) instead of 100%. However, we may require that the organization conduct a more thorough review in order to fully remediate the issue. In order to allow for those changes, CMS expanded the number of medical record samples reviewed during the audit fieldwork stage from 15 to 30. We also included language in the protocol that explains the need to collect and/or review additional medical records and/or case files in order for CMS to validate information provided by the organization and/or fully investigate a potential issue. By taking these steps, CMS is shifting some of the burden for determining the potential scope of non-compliance from PACE organizations to CMS’ auditors.

Comment: These commenters noted that there was a large increase in the number of IAs that CMS requested during the 2019 audit year as a result of CMS utilizing thresholds that were too low to trigger the need for an IA. These commenters also stated that an IA should not be required when the sample number was low and an RCA clearly indicates how the correction should be implemented.

Response: We appreciate the comment, but are not modifying the thresholds because we continue to believe they are appropriate for the PACE program. We have seen on numerous occasions where an IA was requested based on one non-compliant sample, and reveals non-compliance that is actually large in scope. In these instances the PACE organization was unaware of how wide-spread the issue was or that a problem even existed. However, we recognize the burden associated with IAs and, as previously mentioned, we made a number of changes that will better balance CMS’ need to collect the information in the IAs with the burden of providing it. We also believe that by expanding the number of samples reviewed we will better be able to target when an IA should be requested, and we may be able to better identify when non-compliance is truly isolated. Lastly, as a reminder, IAs are only requested when auditors determine that the organization is non-compliance with a regulatory requirement and are not routinely requested as a part of all audits.

Comment: Multiple commenters stated that the timeframe of 10 business days that CMS provides for organizations to complete these analyses is not sufficient.

Response: As we mentioned above, we will be reducing the number of IAs we request for this package, and we will be reducing the review scope of the IAs that require a full medical record review by approximately 50%. Based on those changes, we will not be altering the timeframe for IAs to be submitted to CMS, but we will continue to grant reasonable extensions of the timeframe as necessary.

Comment: Multiple commenters requested clarification on how to complete columns that don't apply in an IA. These commenters requested that CMS review all IAs for columns that would allow a PACE organization to enter "NA" and clearly indicate in the IA when “NA” may be entered. The commenters also asked CMS to clarify if a PACE organization would only need to complete the columns applicable to the non-compliance (for example, if the non-compliance relates to the initial competency evaluations, the organization would only need to complete the columns related to the initial competency evaluations).

Response: We have updated each IA to clarify when “NA” may be entered in a column. However, we want to note that an organization may always enter “NA” anytime a field does not apply to a participant or situation, even if the field description does not indicate that as an option. As for the determination of which columns would be completed by the organization, it is not our intention to request columns that are unnecessary. CMS is required to include in the PRA package all of columns we could request in an IA. However, CMS would only collect the columns that relate to the specific non-compliance discovered during an audit. For example, if initial competencies is the only non-compliance found during the personnel review, CMS would remove all of the columns that are not related to initial competencies prior to requesting the IA.

Root Cause Analyses (RCAs) and Impact Analyses (IAs) - Specific

Comment: Multiple commenters asked CMS to modify the Assessment1P491P501P82 IA by allowing “NA” to be an option for any column that doesn’t apply and eliminating columns I, O, V, AC, AF because the commenters believe that information is unnecessary or duplicative.

Response: We appreciate the suggestions and made a number of changes in response. First, we eliminated columns I, O, V, and AF which we agreed where unnecessary or duplicative. We retained column AC because the data in that column helps CMS determine if a participant who is not residing at home (e.g., the participant is living in a long-term care facility) received a required assessment. We also added “NA” as an allowed option whenever it was necessary, made clarifications to columns H, Q, S, U, and W to better explain the information we are collecting, and removed columns R and T since these columns related to annual assessments, and as a result of the provisions in the 2019 PACE regulation (84 FR 25610, June 3, 2019), PACE organizations are no longer required to conduct annual assessments.

Comment: Multiple commenters requested changes to the EmergencyCare1P07 IA. These commenters noted the IA, while important, was too lengthy and confusing, and requested CMS to simplify it. These commenters also requested clarification on the purpose of the IA in order to better understand the columns. Commenters also asked for clarification on why there would be billing questions included in the ER IA. Lastly, commenters noted that this IA, as currently designed, would require an intensive review of records to complete.

Response: The purpose of this IA is to determine if a PACE organization requires participants to meet certain requirements or follow a specific process before calling or going to an emergency room, and to determine if participants, who are required to be held harmless, incurred costs following an emergency room visit. Because both of these situations have been seen on audit, both situations are accounted for in this IA in order to allow PACE organizations to determine the scope of non-compliance. However, as previously mentioned, CMS will not request information that is not relevant to the non-compliance noted on audit, and would remove columns that do not apply to a particular PACE organization. After reviewing the IA in light of the comments, we removed columns H, I, M, N, O, Z, AC, and AD that we considered confusing and revised language in the remaining columns to ensure it is more clear. We also added column headers to help organizations understand which fields are required and how to populate them.

Comment: Multiple commenters requested modifications be made to the grievance IA,

Grievances1P311P751P77. First, commenters indicated the scope of the review that must be conducted is extensive given that part of the IA is reviewing medical records and other documentation to determine if the PACE organization did not appropriately recognize and process a grievance. Second, the commenters indicated that organizations continue to be confused by the types of issues that are considered grievances, which makes addressing the questions in the IA difficult. Third, the commenters requested that, if CMS retains the IA, the scope of the review should be limited to a sample of participants and CMS should remove some fields, including those related to negative outcomes.

Response: The commenters’ requests for additional clarification on the issues that are considered grievances are outside the scope of this package and can only be addressed through the rulemaking process. A PACE organization’s ability to process grievances is a vital part of ensuring participants are receiving the services they require and resolving complaints quickly and appropriately. When noncompliance in this area is noted on audit, we still intend to request an IA, but we reduced the scope of the participants being reviewed from 100% of participants to 50% of the remaining participant population. We retained the negative outcome columns in this IA because we have seen that complaints that are not addressed timely or at all can lead to adverse health consequences.

Comment: Multiple commenters requested clarification and revisions on the SDRIdentification1P76 IA. These commenters noted that when something was processed as a service delivery request (SDR), they were still required to input information that could be found in the SDR universe, and when something was not processed as an SDR, that analysis required a comprehensive review of the participant's medical record and other documentation. Additionally, the commenters indicated that asking if there were negative outcomes associated with the non-compliance was overly burdensome. Lastly, these commenters indicated the policy was unclear and overly broad and solicited policy guidance from CMS on what constitutes a service delivery request.

Response: The commenter’s requests for CMS to provide guidance on the types of requests should be treated as service delivery requests are outside the scope of this package and can only be addressed through the rulemaking process. CMS did, however, make changes to clarify our expectations related to this IA. We agree with commenters that, if a PACE organization processes a request as a service delivery request and that information is included in the universe CMS obtained before audit fieldwork began, the organization should not have to re-enter the information. We therefore shifted the IA columns in order to first ask whether a service delivery request was processed. If the PACE organization enters “Yes” in that column and the case was included in the universe, the organization is instructed to enter “NA” in all remaining columns. We did not eliminate columns in the SDRIdentification1P76 IA because CMS needs that data to determine the impact to participants when requests are not processed as service delivery requests.

Comment: Multiple commenters requested that we remove column S as it was duplicative.

Response: We agree and removed this column.

Comment: Multiple commenters noted that the effectuation or provision of service IAs

(Effectuation1P021P111P30, HomeCare1P02, ProvisionofServices1P021P81 and SrvcRestric1P90) would require an intensive review of medical records, especially for fields related to negative outcomes. These commenters indicated that it would be overly burdensome and would take clinicians and staff away from providing care for participants.

Response: We appreciate the opportunity to address these concerns. These four IAs are vital to ensuring participants are receiving care and services that are necessary and appropriate to meet participants’ needs. Each one serves a different purpose, and CMS tailored them to limit the scope of the review and minimize burden on PACE organizations as much as possible. Effectuation1P021P111P30 would be requested when approved service delivery requests or appeals were not effectuated. A PACE organization would only review that subset of information (services that resulted from an approved request) in order to compile the IA. HomeCare1P02 only relates to home care services being provided, and the review, while requiring a medical record review, would be limited to participants receiving home care services. The last two IAs are broader and are meant to capture the provision of other IDT approved services, PACE provider ordered services, and when a PACE organization is inappropriately restricting or limiting covered services. We also believe it is important for CMS and a PACE organization to be able to determine when the lack of services led to a negative outcome for a participant.

Comment: Multiple commenters requested revisions to Effectuation1P021P111P30 IA. Specifically, commenters requested CMS to remove the columns related to negative outcomes, eliminate columns H and L because they are duplicative, and clarify columns AA, AF, and AH.

Response: We appreciate the opportunity to clarify this IA and address these concerns. In response, we eliminated column H but retained column L. Column L is not asking if the request was approved, but rather, if the request was provided as approved. We also clarified when "NA" would be an appropriate response in any given column. We modified the language for columns I, R, V, and W to clarify CMS’ expectations. Additionally, upon reviewing columns AA, AF and AH, we realized that the IA was overly broad and therefore added two new columns (now AA and AC) and eliminated four columns (previously AB through AE). We believe these changes will better define and limit the IA to the provision of approved services. Lastly, we retained the collection of negative outcomes in this IA because not providing a service may directly lead to participants experiencing negative health outcomes such as falls, hospitalizations, etc.

Comment: Multiple commenters requested revisions to the HomeCare1P02 IA. Specifically, they requested "IDT recommended" be removed from columns F and G, and "ordered by a physician or NP" be removed from column G. They also requested that the columns related to negative outcomes be removed from the IA altogether.

Response: We acknowledge that the terms "IDT recommended" and "IDT approved" are confusing. We therefore eliminated these terms and instead asked whether the IDT determined home care was necessary in columns F and G. We also made this edit to all remaining columns where the term "recommended" was used. We did not delete the option of "ordered by a primary care provider or nurse practitioner" because we recognize that a primary care provider or nurse practitioner may order home care outside of a formal IDT meeting and we believe CMS needs to account for those situations. Lastly, we did not remove the columns related to negative outcomes because this IA relates to the provision of approved services and it important for CMS and a PACE organization to be able to determine when the delay of an approved service led to a negative outcome.

Comment: Multiple commenters requested clarification on IAs ProvisionofServices1P021P81 and SrvcRestric1P90. For both of these IAs, they asked CMS to clarify what "recommended by the IDT" means.

Response: As we indicated for the IA related to home care, we added clarifying language to these two IAs, and changed "recommended" to "determined necessary by the IDT".

Comment: Multiple commenters requested clarification on the Appeals1P651P661P681P73 IA. Specifically, they asked if the organization would be expected to undertake a full medical record review on all participants in order to determine if an appeal was not appropriately categorized.

Response: We appreciate these commenters’ question, and want to clarify that we do not expect that organizations undertake a full medical record review for all participants in this IA. Rather, we would expect that organizations focus on participants that had a service delivery request denied during the IA review period, and determine if any of the identified participants made a request for appeal that was not immediately recognized and processed.

Attachment II:

Comment: Multiple commenters requested clarification on Attachment II, Question 6 regarding the policies that place limits on the amount, duration, or frequency of items or services. These commenters requested that CMS clarify that this question refers to either policies that are fixed or predetermined limits, and it is not intended to limit the IDT’s ability to develop care plans that are inclusive of/limited to services intended to address the needs of participants as assessed by the IDT in collaboration with the participant and/or his/her caregiver.

Response: We appreciate the opportunity for clarification of this question. We are asking about a PACE organization’s policies that may limit the amount/duration/scope of services, not the IDT's individual assessment of a participant's needs. If a PACE organization has policies that restrict services, it should respond accordingly to this question. The PACE organization will also be provided an opportunity to discuss during the audit how it implemented the policy and take into consideration the participant's needs.

Comment: Multiple commenters requested clarification on Question 9, "Can participants obtain prescriptions written from any prescriber including specialists? If no, explain the process of reviewing the order and rewriting the prescription." Specifically, whether CMS was only requesting this information for prescription medications.

Response: Thank you for this comment. We originally intended to request this information for prescription medications. However, we now realize that other prescribers or specialists could write prescriptions or orders for other types of care (e.g. eye glasses) and we believe this question is applicable to those items too. We therefore revised the question to clarify that we are requesting information for any/all types of prescriptions or orders, and also revised this question to determine how a PACE organization reviews recommendations from other prescribers if the organization does not accept prescriptions or orders from other prescribers .

Comment: Many commenters requested clarification on Question 16, "Are there any participants who have opioid restrictions? If yes, please explain what the restrictions entail?" These commenters asked CMS to remove the question because providing participant-level explanations of any limitations on access to opioid medications in the context of Attachment II: PACE Supplemental Questions is burdensome and it is also duplicative of the information available to auditors in the LOPMR.

Response: We appreciate the opportunity to clarify this question. We are not asking for participant specific restrictions. We are asking generally if the PACE organization has any opioid restrictions in place for participants. If it does, we expect the PACE organization to report them. A PACE organization can report this information at a high level and does not need to include specific restrictions for individual participants. We added clarifying language in Attachment II.

Comment: One commenter requested clarification on Question 4, "Does your organization have the ability to provide remote access to medical records?" This commenter asked what CMS meant by "ability," indicated that their system was linked with the parent company’s system and restricting access to only PACE participants was difficult, and asked if a PACE organization could deny remote access for CMS based on an internal policy.

Response: When we ask about whether the PACE organization has the ability to grant access to remote records, we are not asking if the system is capable, but rather will the organization allow remote access. We have found that remote access helps to improve the effectiveness of the review and streamlines the process so that the audit team can more efficiently interact with electronic medical record (EMR) support staff to help locate documentation and spend less time onsite at the organization. However, if an organization cannot or will not provide remote access, auditors will either access records when they are onsite, or will have the organization navigate records in a webinar forum for the duration of the auditors’ review.

Attachment V - Audit Survey

Comment: Multiple commenters requested that we add some general questions that address the burden and timeframes associated with the audit. These commenters wanted CMS to solicit the time and number/type of staff that are needed to complete audit-related requests from CMS, including the universe submission, IA submissions, and other documentation requests.

Response: We are added questions to the audit survey that account for the average amount of time it has taken PACE organizations to complete certain audit-related requests and the staff that were involved in completing them. We are also soliciting information on the reasonableness of the timeframes that apply to documentation requests outside of the IAs, for example the timeframe for submitting pre-audit universes and documentation.

Comment: Multiple commenters requested that CMS add questions to the survey that generally address the audit process. More specifically commenters asked CMS to add questions about whether the DRL was accurately maintained, whether audit reports and ICAR notifications were sent in a timely manner, and whether any changes were made from the issuance of the draft report to the issuance of the final report based on organization comment.

Response: We thank the commenters for these suggestions, but we disagree with adding questions related to these issues in the audit survey. CMS relies on the audit survey to collect information that CMS cannot otherwise obtain from the organization or its own systems. We already track when audit notifications such as engagement letters, ICAR notifications, and reports are issued, so we do not need to solicit this information from organizations. Additionally, we currently track when we make changes from the draft report to the final report, so we do not need to collect this during the survey process. As for the accuracy of the DRL, we expect that audit teams will maintain this information appropriately, and that in the isolated instance when it is not accurate, we would hope an organization would use the free text field of the audit survey to let CMS know about this issue.

Comment: Multiple commenters asked CMS to add survey questions about the organizations communication with the PACT; including asking about whether the organization had a chance to bring a dispute of ICARs to the PACT, and a general question about whether the PACE organization was given a

chance to respond to the audit team regarding audit conditions and bring those concerns to the PACT.

Response: We appreciate the commenters’ suggestions but we do not believe these questions are appropriate for the survey. All organizations are given a chance to comment and dispute each condition cited in the audit during the draft report comment stage of the audit process. This is true regardless of whether the conditions are classified as an ICAR, Corrective Action Required (CAR), or observation. Therefore, CMS doesn’t need to solicit this information in a survey. Additionally, while we always strive for transparency and ensure CMS program staff are available as needed, we do not believe it is appropriate to give organizations the impression that they should routinely bypass audit teams to solicit information from the PACT. Organizations should ensure they are discussing any concerns with their audit team as they arise.

Comment: Multiple commenters requested that CMS modify Questions 3 and 4 in the general questions to allow organizations to compare their audit experiences to any other organization and not just a related organization.

Response: We thank the commenters for this suggestion but do not believe it is appropriate to modify Questions 3 and 4 as requested. We believe that the best comparison will be done by related organizations that have the same parent corporation, and we would like to be able to distinguish comparisons done at that level from other comparisons that could be less reliable. However, to address these commenters concerns, we are adding a new question that will allow an organization to comment on general comparisons to other organizations that are not related to its contract.

Comment: One commenter requested that we make the survey mandatory instead of voluntary.

Response: We appreciate this suggestion but do not believe the survey should be mandatory. CMS encourages all organizations to take part in the survey process as it allows for organizations to provide vital information to CMS concerning their audit experiences, however an organization may choose not to complete the survey which we believe should be allowed.

Comment: One commenter requested that CMS add questions that focus more on the substance of the audit and less on the audit teams. This commenter requested that CMS add questions about how to reduce audit burden or redundancies, suggestions for audit efficiencies or improvements, and additional audit training suggestions.

Response: We agree with this commenter and added these questions to the general questions section of the survey.

Burden Estimate:

Comment: The majority of commenters who commented on the burden estimates indicated CMS underestimated almost every aspect of the PACE audit, including the amount of time and resources that were involved in the pre-audit work, and the amount of time and resources that were involved in the post-audit work. Multiple commenters estimated that, in total for the four components of the 2020 audit, the burden would be closer to 1,210 hours, more than double CMS’ estimate of 600 hours. As a result, the estimated cost for a PACE organization to undergo an audit would be about $83,000.

Response: We appreciate the comments. As noted throughout the comments in this document, we have made a number of changes to the collection activities in this package that would greatly reduce the burden organizations would experience when undergoing a CMS PACE audit. Based on the revisions, we believe the original estimate for PACE organizations now more accurately reflects the cost of the audit. However, because of the increased burden for CMS related to sample size and validations we modified the estimate for CMS to undertake a PACE audit.

Comment: Multiple commenters noted that CMS underestimated the staff that were involved in PACE audits and that 600 hours did not accurately reflect the time organizations spent on audit. Several commenters also indicated that CMS burden estimates did not account for the time that senior leadership or the medical director were involved in the audit. One commenter indicated their organization spent approximately 750 hours in the course of an audit. Additionally, one commenter indicated that CMS did not account for the involvement of IT staff and other clinical or IDT members’ involvement. These commenters all expressed concerns that the amount of time staff spent on audit work was time away from providing care to participants.

Response: When determining our burden estimates, we tried to account for the individuals at an organization that spent the most time involved in an audit. In our experience, the participation of senior leadership and IT staff are is usually limited. For purposes of the burden estimate, we estimate the time for the staff who are involved in the audit from the beginning of the process until the end. However, we accepted the commenters’ suggestions to add questions to the audit survey about the burden of audit in the post-audit survey. Once we have that information, we will use it to update our burden estimates in the future.

Comment: Multiple commenters indicated that the burden estimates in the supporting statement underestimate what a PACE organization actually experiences, especially for medium to large size organizations. These commenters indicated that the time and effort that organizations take to complete an audit can vary greatly depending on the size of the organization and stated that some organizations have over 2,000 participants which requires a significantly higher amount of time and effort to compile data for.

Response: CMS developed the burden estimates based on the average size of all PACE organizations. We have added questions into the post-audit survey to solicit information regarding burden from organizations in order to better account for the burden of an audit in future PRA packages.

Comment: Multiple commenters indicated that CMS underestimated the burden associated with the pre-audit activities. While the commenters appreciated CMS proposing to shorten the universe timeframe to six months and remove the quality universe, they indicated that the amount and detail of data collected still requires a significant amount of staff and resources to compile. Some commenters noted that populating the universes is a big burden, and suggested that CMS consider reducing the number of universes collected. Commenters also suggested that the changes to the LOPMR universe were overly burdensome and thought an estimate of 210 hours is much more realistic for the pre-audit portion.

Response: In addition to reducing the universe time frame by half, we also made changes to specific universes, including LOPMR, as discussed throughout our responses to comments in this package. These changes would eliminate a significant amount of burden for PACE organizations.

Comment: Multiple commenters noted that the burden estimate for audit fieldwork may be accurate if CMS would account for reductions in the requirement of submitting the information for the onsite participants via the proposed attachment. If CMS did not accept that suggestion, commenters thought 200 hours was a more reasonable estimate.

Response: As discussed above, we are no longer requiring the onsite participant list to be submitted in any required format, and organizations may provide this information in any format(s) they already have available. Because organizations should have that information readily available in order to know which participants will be receiving services in any given week, we do not think it is necessary to reduce the burden estimate to account for providing that information to CMS.

Comment: Multiple commenters thought CMS did not accurately assess the burden of the post-audit work, including populating and submitting IAs. These commenters noted that CMS has significantly increased the number of IAs being requested and the amount of information requested in them, but did not increase the burden estimate to account for the amount of time and resources undertaken to complete IAs. One commenter noted that, for PACE organizations with less than 300 participants, the IA would require at least 21 staff to complete and would require reviewing over 6,500 orders, and for larger organizations, sometimes up to 20,000 assessments would need to be reviewed. Multiple commenters thought that, based on 2019 audit experiences, larger organizations could spend up to 1,800 hours completing this information. Commenters suggested we double our burden estimate and others suggested increasing estimated hours from 160 to 600 hours to account for the work of conducting full medical record reviews and compiling the detailed information for audit.

Response: We appreciate the comments regarding the potential burden of having to complete IAs following the audit. As discussed throughout our responses to comments in this package, we made many changes to the IA templates and process that we believe would significantly reduce burden on audited organizations. The changes include removing a number of IAs, deleting IA specific columns including negative outcome when possible, and reducing the scope of IA reviews to 50% of the remaining enrolled participants that were not already reviewed by CMS relevant to the issue identified. We have also increased the number of medical record samples reviewed by CMS in order to eliminate the need for auditors to request an IA or reduce the number of records organizations need to review for an IA. Based on these changes, we believe the original estimates for post-audit activities now more accurately reflect the burden associated with a PACE audit.

Comment: Multiple commenters agreed that the burden estimate of 200 hours was accurate for the corrective action and closure process of the audit.

Response: We appreciate concurrence on this estimate and will not make changes to this portion of the burden estimate.

Comment: Multiple commenters recommended specific ways CMS could reduce the burden of audit on an organization. These include: 1) reducing the number of requests for information that can be met only by undertaking exhaustive medical record reviews; 2) clarifying expectations of PACE organizations with respect to service delivery requests and grievances; 3) raising thresholds for requiring PACE organizations to undertake RCAs and IAs; and, very importantly, 4) using a sampling methodology that allows PACE organizations to initially undertake IAs for a sample, e.g., of participants, with 100% samples being required only in much more limited situations.

Response: We appreciate these commenters’ suggestions and believe the changes we made in this revised package address many of these suggestions and more generally address the overarching concerns about the burden of PACE audits on organizations. As noted throughout this document, we reduced the number of IAs we request and removed the fields related to negative outcomes from some of the remaining templates. Some of the IAs templates that we removed required a significant amount of resources to complete. For example, we removed the IA related to complete and accurate medical records, which will mean an organization will no longer be asked to go through all medical records to look for missing or inaccurate documentation. We would, however, still require the organization to submit an RCA in order to determine the cause of non-compliance. Additionally, we have removed some of the columns in the LOPMR universe in order to eliminate more of the burden on the front end of the audit. While we cannot address the second suggestion because it is outside the scope of this package and could only be addressed through the rulemaking process, we did consider and address the remaining two suggestions. As for thresholds, as we discussed in a previous response, by reviewing more medical records and samples during the audit fieldwork, auditors will have a better sense of whether an IA is necessary. Additionally, we reduced the scope of some IAs from requiring a review of all medical records, to requiring organizations review 50% of the remaining participant population (for IAs that require a full review of participant medical records).

Comment: Some commenters indicated that the burden estimates assumed that PACE organizations had more advanced electronic systems than what they actually have. These commenters noted that much of the data requested has to be pulled and compiled manually, and usually by clinicians which would pull them away from providing direct care. Of the most concern to these commenters was the participant medical record universe, the onsite participant list, and the IAs.

Response: We understand that many PACE organizations do not have systems that are capable of efficiently tracking the provision of care and services for participants or compile audit relevant information, which may hinder an organization’s ability to respond to oversight requests. However, the ability of a PACE organization to maintain information on requested and approved services is critical to ensuring services are being provides to participants. Therefore, we strongly encourage PACE organizations to develop and maintain an appropriate infrastructure to ensure the needs of participants are met in accordance with CMS’ requirements. In the meantime, we have tried to reduce the burden where possible when data collection requires a more manual review, including eliminating fields from the LOPMR universe, eliminating IAs in some situations, and allowing for an onsite participant list to be submitted in any format the organization has available.

Comment: One commenter thought that the increase in data collection tools from 18 to 33 could lead to clinical staff spending 1500 hours performing an audit.

Response: While we have increased the number of data collection tools in this package primarily by modifying existing IAs to be more condition specific, we do not believe that this increase directly leads to a significant increase in burden. However, based on commenters’ concerns regarding the intensity of these reviews and compiling this information, we have, as mentioned in our responses to comments throughout this package, made changes that would reduce burden on audited organizations compared to the proposals in the initial 60-day package. We have eliminated a number of IAs entirely, which will have a significant impact on burden since we would have requested an organization undertake these reviews in 2017-2019, and we will no longer ask organizations to submit these IAs for certain conditions going forward. Additionally, we are limiting the scope of the IAs that require a full medical record review to instruct the organization to only review 50% of their remaining participant population.

Comment: One commenter indicated that the changes between the 2017 and 2020 protocol do not streamline or reduce burden, but rather they increase burden and demand on organizations. This commenter indicated that this increase appeared to go against the efforts HHS has made to reduce burden on the industry. This commenter noted that the addition of numerous audit documents and fields within the documents significantly increases provider burden.

Response: We appreciate the commenters concerns. As discussed in our responses to comments throughout this document, we have taken these concerns into consideration as we reviewed our data and documentation requests, and made a number of changes that we expect will reduce burden for audited organizations while ensuring we continue to be able to ensure participants are receiving the level and type of care they are entitled to receive and in accordance with the statutory and regulatory requirements that PACE organizations are bound to and have agreed to comply with. We want to remind organizations that part of the burden of audit is related to IAs, which are only requested when auditors discover non-compliance during an audit and organizations are asked to determine which participants were affected by the non-compliance.

Comment: One commenter noted that our burden estimate doesn't account for staff spent with CMS auditors during the audit fieldwork stage to assist with navigating the EMR.

Response: We appreciate the opportunity to respond to this comment. Our estimates are based on our experience across all of the organizations we have audited since 2017, and the time and participation on audit from PACE organization staff varies largely from organization to organization. We encourage auditors to try and minimize the burden on staff as much as possible, and we do not require staff to navigate the medical records for us, or to sit with us during our review in order to ensure they are not taken away from other duties they may be responsible for. We do understand that some staff may provide support when auditors are unable to find documentation on their own, and other staff may need to upload documents requested by auditors, however we believe our estimates account for this burden.

Comment: One commenter noted that section 5 of the supporting statement states “The collection of information will have a minimal impact on small businesses since applicants must possess an insurance license and be able to accept substantial financial risk. Generally, state statutory licensure requirements effectively preclude small business from being licensed to bear risk needed to serve Medicare enrollees.” This commenter noted that PACE organizations are not required to possess an insurance license, but this commenter believed that PACE organizations would be considered small businesses and, as such, additional regulatory burdens should be weighted accordingly.

Response: We believe this is an accurate statement given our experience in the PACE program and understanding of what constitutes a small business, however in order to simplify this section and better align with other PACE PRA packages, we are changing this section to say, “This collection will not impact small businesses or other small entities.”