30 Day Public Comment Response

Centers for Medicare and Medicaid Services Response to 30-day Public Comments Received
for CMS-10630
The Centers for Medicare and Medicaid Services (CMS) received comments from Programs of All
Inclusive Care for the Elderly (PACE) organizations and an advocacy organization related to CMS-10630.
This is the reconciliation of the comments.

General Comments:
Comment: One commenter requested that CMS continue utilizing a 30 calendar day timeframe for
universe and data submissions instead of adopting a 20 day timeframe, arguing that 20 days was too
short for the amount of information requested.
Response: As we stated in the response to the 60-day comments, we determined in 2020 to no longer
utilize a 30 calendar day timeframe because it allowed for situations where the timeframe would end on
a weekend or holiday. Instead, we converted the timeframe to a 20 business day timeframe. We
believe this change will not have a substantial impact on PACE organizations, because the 20 business
days will not count weekends or holidays, and the due date will be within a day or two of the 30
calendar timeframe, but will always fall on a workday. We are therefore continuing to use a business
day timeframe instead of calendar day timeframe for universe and data submissions.
Comment: Multiple commenters requested assurance that organizations could access the PACE Audit
Consistency Team (PACT) as needed during the audit. Specifically, commenters requested access to the
PACT in order to express concerns regarding requests for impact analyses (IAs) if the PACE organization
believes the requests are not warranted.
Response: As we indicated in the 60-day response, comments about access to the PACT are outside the
scope of this data collection. PACE organizations are encouraged to discuss requests for IAs with their
audit team who will be able to explain why the request is being made. If PACE organizations have
engaged in discussions with the audit team and still believe that a request for an IA is not warranted
they may contact CMS through the [email protected] mailbox.
Comment: One commenter noted that PACE organizations are required to provide education to all
contracted staff and facilities on an annual basis and encouraged CMS to provide additional education to
PACE organizations in order to clarify CMS expectations and improve PACE organizations’ understanding
and performance.
Response: We appreciate this commenter’s suggestion, but policy education questions are outside the
scope of this package.
Comment: One commenter recommended the protocol include a description of the risk-based
assessment utilized by CMS to determine which PACE organizations are audited in order to clarify the
frequency with which PACE organizations can expect to be audited.
Response: We appreciate this commenter’s suggestion and we will strive to be transparent in the
factors or performance standards we use for our internal risk assessment tool. For existing information

about utilizing a risk based approach for audit selection beyond the trial period, please refer to the
Medicare and Medicaid Programs; Programs of All-Inclusive Care for the Elderly
(PACE) Final Rule published in the Federal Register on June 3, 2019 (84 FR 25610). The reference to this
final rule has also been included in Supporting Statement A.

Technical Comments:
Comment: One commenter recommended that the protocol be designed in a manner similar to the
program audit protocol so that PACE organizations could utilize claims data rather than having to review
medical records in order to submit participant data.
Response: Thank you for your comment. The PACE audit protocol introduced in 2017 was based on the
program audit protocols, but with modifications to account for the differences in the PACE program.
We encourage PACE organizations to use any and all data to provide the requested information
including claims data if it is applicable. We have added this clarification at the beginning of the List of
Participant Medical Records (LOPMR) universe to ensure organizations understand that they may use
any available data resources to populate the requested information.
Comment: Multiple commenters requested that CMS clarify the requirements for responding to column
L, Hospitalization/Emergency Room Reason in the LOPMR universe. Specifically, the commenters asked
if organizations could base their responses on hospital and ER claims data.
Response: Yes, PACE organizations may use claims data to complete column L and should respond "Y"
only when the participant has a primary or secondary diagnosis of hypoglycemia, hyperglycemia, or
decreased oxygen saturation.
Comment: Two commenters requested clarification regarding how PACE organizations should complete
the LOPMR columns N, Currently in SNF/NF, and AB, Current Center Attendance, for participants who
disenrolled during the audit review period.
Response: PACE organizations should enter "N" in column N, Currently in SNF/NF, for all participants
who are not in a Skilled Nursing Facility/Nursing Facility (SNF/NF) at the time the universe is completed,
including participants who disenrolled during the audit review period. PACE organizations should enter
"0" in column AB, Current Center Attendance, for all participants who do not attend the PACE center at
the time the universe is completed, including participants who disenrolled during the audit review
period. We have updated these two columns in the protocol in order to provide this clarification.
Comment: One commenter requested that CMS identify specific International Classification of Diseases,
10th revision (ICD-10) codes associated with the LOPMR columns Q, R, S, T, and V, in order to reduce
burden for PACE organizations.
Response: We appreciate this commenter’s suggestion, but we believe this approach may be more
burdensome to organizations or hinder their ability to complete these fields. We understand that PACE
organizations may use a variety of methods to identify diagnoses in the medical record and we do not
want to limit how organizations collect and populate these data by restricting to specific ICD-10 codes.
Comment: One commenter requested that CMS define functional decline (as referenced in column AE,
Functional Decline of the LOPMR universe).

Response: CMS believes that PACE organizations are in the best position to identify which participants
experienced a functional decline during a given audit period and therefore, it would not be beneficial to
impose more restrictive criteria. An example of functional decline could include a decline in a
participant's ability to perform two or more activities of daily living (ADLs) or independent activities of
daily living (IADLs) during the audit review period.
Comment: Two commenters requested clarification regarding how PACE organizations should complete
column K, IDT Role, for the List of Personnel (LOP) universe if one individual fills two interdisciplinary
team (IDT) roles.
Response: If one individual fills two IDT roles, PACE organizations should enter both roles in column K.
For example, if one individual fulfilled the role of registered nurse (RN) and home care coordinator (HCC)
the PACE organization would enter "RN and HCC." We made a minor change to this column in order to
clarify that two roles may apply to one individual.
Comment: One commenter noted that the record layouts do not include field lengths and requested
clarification regarding the maximum field lengths for each column.
Response: CMS intentionally removed the field length indicator from the record layouts. There are no
character limitations for any of the fields in the audit universes.
Comment: One commenter requested clarification regarding whether CMS intends to conduct
participant observations at Alternative Care Setting (ACS) sites. The same commenter recommended
that if CMS intends to conduct observations at ACS sites, CMS modify the language in the protocol to
include ACS sites.
Response: CMS does not intend to regularly conduct participant observations at ACS sites; however,
CMS reserves the option to conduct participant observations at any PACE operated center as well as ACS
sites utilized by PACE organizations. We appreciate the commenter’s feedback and have updated the
protocol to include ACS sites.
Comment: One commenter requested that CMS add to the protocol, the definitions or criteria for
condition classifications that are described in the audit reports. This commenter stated that they believe
the classification of conditions have been inconsistently applied and that defining these classifications
would increase consistency and transparency. Another commenter indicated that audit timeframes are
very well defined for PACE organizations, but there are no timelines outlined for CMS, other than
engagement letter issuance.
Response: As the commenter noted, definitions for classifications are provided to organizations in the
draft and final audit report following the audit. CMS has also included this information in industry-wide
communications about the audit process. We appreciate the comment related to timeframes. Current
PACE audit experience demonstrates the need for flexibilities in finalizing audit results due to audit
related factors such as the number of findings and the significance of the non-compliance as well as the
unique structure and operations of each PACE organization. However, we remain committed to
continuing to ensure transparency, while improving audit processes and communication based on
continued PACE audit experience.

Comment: One commenter recommended modifying question 7a of Attachment V, PACE Audit Survey,
to state, "How many staff members do you estimate it took to complete the requested Impact
Analyses?"
Response: Thank you for your comments. The purpose of question 7a in the PACE Audit Survey is to
gather information that will assist CMS in determining the number of hours needed for PACE
organizations to complete requested IAs. Question 7b of the PACE Audit Survey addresses the number
of staff members needed to complete requested IAs. We believe that it is necessary to determine the
number of hours and the number of staff members needed to complete IAs in order to fully understand
the burden associated with the completion of IAs. Therefore, no changes will be made to question 7a.
Comment: Multiple commenters recommended minor typographical and/or technical modifications to
the following documents: Attachment I, Attachment III, Attachment IV, and Attachment V, Supporting
Statement A and the Impact Analyses (IAs): Effectuation1PO21P111P30, EmergencyCare1P07,
Grievances1P311P751P77, HomeCare1P02, Personnel, SDRIdentification1P76, SrvcRestrict1P90, and
WoundCare1P02.
Response: Thank you for your comments. CMS has reviewed and agreed with the commenters'
recommendations. Changes will be reflected in the finalized audit protocol documents issued by CMS
and will also be included in the crosswalk.

Burden Comments:
Comment: One commenter expressed concerns that the audit universes, as proposed, would increase
the need for manual data collection resulting in increased burden for PACE organizations. Another
commenter expressed concern that the information requested in column P, Specialist/Consultation
Visits in the LOPMR universe, would be difficult to gather and would require a manual review of
participant medical records.
Response: We appreciate the comments on the potential burden of collecting this information. As
noted in the response to the 60-day comments, we have sought to strike a better balance between
ensuring participants are receiving appropriate and timely care and services, and not overwhelming
organizations with intensive data requests over the course of the audit. CMS therefore took a number of
steps to eliminate or streamline our data requests following the 60-day comment period, where the
burden associated with its collection outweighs the value of the data from an oversight perspective,
including removing a number of proposed columns in the LOPMR universe related to specialists. We
believe that the remaining data requested in the revised protocol is necessary to effectively monitor
organizations for compliance with regulatory requirements.
Comment: One commenter recommended that CMS increase the sample submission timeframe for
samples reviewed as part of a desk review. The commenter recommended that PACE organizations be
provided sample cases 5 days prior to the start of the review rather than the 2 days as proposed in the
protocol. The same commenter recommended that CMS increase the sample submission timeframe for
participant medical record samples from 1 hour to 1 business day. The commenter expressed concern
that PACE organizations may not have adequate time to gather records not included in electronic record
systems.

Response: As stated in the 60-day response, we do not believe five business days is a reasonable
timeframe based on our experience auditing organizations from 2017 through 2019. The case files
requested in this timeframe are generally small (i.e., not many documents) and do not require
organizations to spend long amounts of time pulling different parts of the record in order to compile
case files. We are therefore retaining the two business day requirement. Additionally, as noted in the
60-day response and revised protocol, when organizations are notified via the samples about the
medical records CMS will review, we do not expect that organizations scan, print, or upload medical
record documentation related to those samples within an hour from our request. Rather, organizations
are expected to provide CMS immediate access (whether through remote access, webinars, or
computers onsite) to the medical records for each sample within that time, in whatever manner was
agreed upon by the audit team and organization in advance. Therefore, we are keeping the timeframe
of one hour for samples to be provided. Organizations will have a longer time to compile information
and upload it as needed.
Comment: One commenter requested clarification on CMS’ intent behind increasing sample sizes for
elements. Specifically, the decision to add 5 additional service delivery request approvals and 5 appeal
approvals, and increasing the Provision of Service sample size (i.e., medical records). This commenter
expressed concern regarding the effect an increased sample size would have on documentation requests
under the revised protocol.
Response: We appreciate the opportunity to provide clarification. We increased the sample size in the
Service Delivery Request and Appeals element in order to better evaluate approved requests and
whether those approvals are being appropriately implemented and effectuated. As we explained in
response to the 60-day comments, we increased the number of samples for the Provision of Services
element to balance the decreased scope of the IAs. By increasing the number of samples, and adding
more validation reviews after audit fieldwork, CMS is shifting some of the burden for determining the
potential scope of non-compliance from PACE organizations to CMS’ auditors.
Comment: One commenter recommended that CMS select overlapping service delivery request and
appeal samples in order to streamline the audit process and reduce the amount of sample
documentation provided by PACE organizations. Multiple commenters recommended that CMS select
overlapping participant or personnel samples when multiple IAs were requested.
Response: We appreciate these suggestions. While there may be times when auditors can select
overlapping service delivery request and appeal samples, we cannot guarantee this will occur. Based on
our universe review, we may identify issues that warrant investigation and require sampling of unique
participants. When feasible, CMS will attempt to ensure that the same participants or personnel are
selected across multiple IAs within the same element in order to ensure the burden is not increased
inadvertently.
Comment: The majority of commenters expressed concern that CMS did not significantly alter or
eliminate Attachment IV, the On-Site Observation Participant List, and stated that preparation of the list
would be overly burdensome. Several of the commenters noted that although CMS proposed to provide
latitude in how the information was provided, the decrease in burden would not be significantly
reduced. Several commenters also noted that in order to provide the requested information staff would
need to perform a manual review of participant medical records. One commenter expressed concern
that the request for information would encompass all participants scheduled to receive home care

during the week of the on-site audit. One commenter suggested that CMS provide additional time to
collect the information such as extending the pre-audit timeframe, but the majority of commenters
encouraged CMS to limit the scope of data collected for Attachment IV, the On-Site Observation
Participant List. Several commenters recommended that CMS limit the scope to participants at the PACE
center where the onsite audit is being conducted.
Response: We appreciate the commenter’s concerns. The information reported in the On-Site
Participant Observation List is expected to be current as of the date it is submitted for the care and
services that will be provided during the onsite portion of the audit. As we stated in response to the 60day comments, PACE organizations should already have methods in place to identify when services/care
should be furnished in order to ensure that they are actually providing the necessary care to
participants; therefore, we do not believe that extending the pre-audit timeframe is necessary or will
improve the PACE organizations ability to collect these prospective data. However, based on
commenters continued concerns, we will limit the initial observation data requests to those participants
assigned to an IDT at the center where CMS auditors are conducting the onsite portion of the audit.
CMS reserves the right to request data for participants from additional PACE centers, as needed, to
ensure all of the observations can be completed. For example, if medication administration or wound
care is not being provided at the center where the onsite audit is conducted, CMS auditors may request
data from other PACE centers in order to determine whether they can conduct these observations at an
alternative site. We have updated the protocol to reflect this change.
Comment: Multiple commenters requested that CMS auditors allow the PACE organization to show
auditors where items are located in the medical record before making an official request on the
document request log (DRL).
Response: As noted in the response to the 60-day comments, we agree that when feasible, auditors and
the PACE organization should have open communication regarding where items are located in the
medical record, and an auditor may ask a PACE organization to point to documentation that is difficult
for the auditor to locate. However, there may be situations where the audit team may need to request
documentation through the DRL, including when documentation is not readily accessible/available in
the record.
Comment: Multiple commenters requested that CMS limit the number of documentation requests in
HPMS, when possible. Additionally, one commenter stated that there appears to have been a shift in
CMS’ documentation request practices between 2017 and 2019 resulting in an increase in burden.
Response: Although the process for requesting documentation did not change for audits conducted
between 2017 and 2019, we have continued to improve our audit process based on lessons learned
from previous years, including what documentation may be necessary in order to adequately investigate
a potential issue of non-compliance and support an accurate and complete audit record. We understand
commenters concerns on potential burden and we will continue to identify opportunities to reduce
burden when it comes to documentation requests while still ensuring CMS’ ability to effectively monitor
organizations for compliance with regulatory requirements.
Comment: The majority of commenters requested that CMS limit the number of Root Cause Analyses
(RCAs) and IAs requested during audits. Commenters noted that RCAs are routinely requested for all
instances of non-compliance regardless of the number of issues of non-compliance noted. Many

commenters requested that CMS establish thresholds for requesting RCAs and IAs and one commenter
requested that CMS document the thresholds as part of the revised audit protocol. Lastly, several
commenters recommended that IAs should not be requested when the number and seriousness of
instances of non-compliance are low and the RCA suggests that the means by which the issue should be
addressed are well understood.
Response: We appreciate the comments, but we are not further modifying the process for requesting
RCAs and IAs because we continue to believe it is appropriate for the PACE program. The RCAs are
routinely requested anytime potential non-compliance is suspected because it allows auditors and the
organization to better understand why the issue occurred and helps the auditors determine whether or
not an IA is actually necessary. As stated in the response to the 60-day comments, we have seen on
numerous occasions where an IA was requested based on one non-compliant sample, and reveals noncompliance that is actually large in scope. In these instances the PACE organization was unaware of how
wide-spread the issue was or that a problem even existed. However, we recognize the burden
associated with IAs and, as previously mentioned, we made a number of changes that will better
balance CMS’ need to collect the information in the IAs with the burden of providing it. We also believe
that by expanding the number of samples reviewed we will better be able to target when an IA should
be requested, and we may be able to better identify when non-compliance is truly isolated. Lastly, as a
reminder, IAs are only requested when auditors determine that the organization is non-compliant with a
regulatory requirement and are not routinely requested as a part of all audits.
Comment: Multiple commenters expressed appreciation for CMS' efforts to reduce the burden of IAs.
The commenters expressed support for CMS' proposal to eliminate certain IAs and reduce the scope of
the review for the remaining IAs. Although the commenters were supportive of the changes proposed
by CMS, they strongly encouraged CMS to consider further reductions in the scope of IAs, particularly
for larger PACE organizations. Some commenters requested that CMS utilize a statistical sampling
methodology when determining the sample size for IAs. Two commenters recommended providing
additional time to complete IAs and one expressed that the timeframe afforded to PACE organizations
for completing IAs should be equal to the time provided to CMS auditors.
Response: We appreciate the comments on the scope of the IAs and the support for the changes made
in the revised protocol. The IAs are important tools that allow an organization to investigate noncompliance in order to fully correct an issue, which we believe cannot be properly resolved without the
PACE organization understanding the full scope of the problem. These analyses are only requested when
non-compliance is discovered during audit fieldwork and the audit team determines that it needs to
understand the scope of non-compliance. As we explained in the response to the 60-day comments, we
weighed the value of the requested information in the IAs from a compliance and participant protection
perspective against the potential burden associated with requesting that information. We took into
consideration the sampling methodologies proposed in the 60-day comments and believe that we have
arrived at a fair balance in the IA scope by requesting that PACE organizations review up to 50% of the
remaining participant population. We intend to continue allowing 10 business days to complete IAs,
with the understanding that CMS will grant reasonable extensions to the timeframe when feasible and
as needed. Some IAs will be narrower in scope and will require less time, and some may be more
difficult. We believe we will better be able to address organizations concerns about completing IAs on
an individual basis taking into account factors such as the size of the organization and number of IAs
requested.

Comment: One commenter expressed concerns that there were no opportunities for PACE
organizations to provide input on the audit until the Draft Audit Report response, which may result in
increased burden for PACE organizations.
Response: We appreciate the commenter’s concern. PACE organizations and auditors are encouraged
to have open and transparent communications throughout the audit. Potential findings are discussed
during daily debriefs and at the exit conference with the audit team during which organizations are
given an opportunity to discuss the issues identified or ask questions. PACE organizations also have an
opportunity to comment on (and dispute) classifications and conditions through their participation in
the draft report comment process. PACE organizations are encouraged to discuss the request for an IA
with their audit team who will be able to explain why the request is being made. Additionally,
organizations have access to CMS through the [email protected] mailbox to address questions
that are not handled by the team directly.

Burden Estimate Comments:
Comment: Several commenters expressed concerns that the increased audit costs for CMS may not be
sustainable if the number and size of PACE organizations expands in the future. One commenter
disagreed with the burden estimate for CMS auditors, stating that it was over-estimated.
Response: We appreciate commenters’ concerns that the burden on conducting audits has increased
substantially for CMS. We increased our burden in response to the 60-day comments indicating that
responding to audit requests imposed significant burden on organizations. In an effort to shift some of
that burden, we increased the number of samples collected during audit, thereby increasing the number
of auditors necessary to complete the review. Our burden estimates are based on experience
conducting PACE audits over the last 3 years and we do not believe these estimates are overstated.
Comment: Several commenters requested that CMS monitor the PACE organization's audit experience
under the proposed audit protocol in order to assess the accuracy of CMS' burden estimate.
Response: We agree that we should collect more data on the burden of audits for PACE organizations
which is why we added questions regarding the burden of audits into the PACE audit survey
(Attachment V). We will review the information provided by organizations in order to better inform our
estimates in future years.
Comment: Several commenters acknowledged that changes proposed by CMS in response to the 60-day
comments will alleviate some burden for PACE organizations; however, many of the commenters
continued to express concern that the burden estimates developed by CMS under-estimate the costs for
PACE organizations. Several of the commenters stated that the burden related to post-audit activities
have been underestimated. One commenter stated that the changes proposed by CMS in response to
the 60-day comments will alleviate some of the post-audit burden; however, the changes will not

reduce pre-audit burden. One commenter suggested that CMS' estimate of 600 hours to complete audit
activities was under-estimated and that the actual number of hours required to complete the audit is
approximately 1,000 hours. Two commenters disagreed with CMS' burden estimates related to the
number of PACE organization staff needed to respond to CMS requests during the audit, provide
documentation, and submit IAs following the completion of the onsite audit. Despite the acknowledged
reduction in the scope of the IAs to 50% of the un-audited participants, these commenters estimated
that it will require more than 4 PACE staff at 40 hours per staff member to complete the required
activities and indicated as many as 8-10 PO staff may work more than 8 hours per day to respond to
questions and requests for documentation.
Response: We appreciate the feedback from these commenters. We understand that every
organization and every audit will have a different burden based on the non-compliance noted and the
number of requested documents from CMS. As noted in the response to the 60-day comments, CMS
has taken a number of steps to streamline the data requests and reduce burden on organizations not
just by decreasing the scope of IAs, but also by reducing the audit review period by 6 months,
eliminating some IAs, and increasing the number of samples reviewed by CMS. Based on these changes,
we believe the original estimates now more accurately reflect the burden associated with a PACE audit.
In addition, we added multiple questions into the audit survey to allow us to better understand the
burden for PACE organizations going forward. Based on the information gathered through the audit
survey, we hope to be able to better quantify the audit estimates in future audit data collection
packages.