Response to Comments

Responses to Comments Received
Federal Register Notice on (CMS-10630)
PACE Audit Protocol
CMS received 14 public submissions, which included 265 distinct comments on the August 5,
2016 (CMS 10630) The PACE Organization Monitoring and Audit Process proposed information
collection. We then combined the 265 comments into 103 unique comments and provided
responses in the document below. Comments are separated by element or record layout.
Additionally, some general comments received are addressed in the first part of this document.

GENERAL COMMENTS
Comment 1: Several commenters supported our distinction between Medicare Advantage (MA)
and PACE as two distinct programs in our supporting statement. These commenters offered
further justification as to why the programs are different, including the fact that PACE is a direct
care model as well as a payer. One commenter requested clarification on whether any audit
protocols other than the PACE audit protocol would be used to assess a PACE Organization’s
(PO’s) compliance with regulations. This commenter specifically referenced the Part D
Coverage Determinations, Appeals and Grievances (CDAG) protocol used in Part D audits.
Response 1: We appreciate the feedback from these commenters. Due to the different
requirements in PACE, PACE programs have always undergone different audits than MA and
Part D organizations. However, we had previously included the PACE audit protocols in the MA
and Part D audit PRA package. For 2017 we decided that we should pull those PACE protocols
out of the MA and Part D audit package, and create a standalone PACE audit PRA package
that only included the documents relevant to PACE audits and included a burden estimate
specific to PACE audits. At this time, POs (POs) will only undergo audits using the protocol in
this PRA package.
CMS Action 1: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 2: Multiple commenters supported the idea of focusing on outcomes during the
PACE audit rather than administrative requirements. Some of these commenters supported
CMS's use of a data driven approach as a good way to measure outcomes, and supported our
revised protocols as achieving that goal. A few commenters specifically supported CMS no
longer requesting numerous documents and policies and procedures prior to conducting the
audit. However, a few commenters thought that the proposed audit protocol was still too
administrative and should instead focus on quality metrics to determine if a PO was providing
high level care to participants. These commenters suggested our audits focus on review of
medical records and observations of participants. One commenter requested assurance that
the audit process would be person-centered. Another commenter urged CMS to focus more on
the provider based nature of the PACE program, rather than the operational effectiveness of the
insurer side. This commenter suggested that the PACE audit protocol was more focused on
compliance with federal requirements than whether the PO delivered high quality care.
Response 2: We appreciate both the support and outstanding concerns of these commenters
relating to the structure of our audits. This protocol significantly modified our audit structure to
be an outcomes based model of auditing. We have significantly reduced the administrative

burden on POs by eliminating over 80 distinct policies and procedures that POs used to be
required to produce. Instead, our audit is now focused on the participant experience within the
PO and whether those participants are receiving the access to care required by federal
regulations. We determined the best way to conduct these person centered audits would be to
request real data on participants, in areas relating to access to services and care such as
service requests, appeals and participant medical records. We then take that participant level
data and examine whether the participant received medically necessary services and care.
Additionally, our audit continues to place a heavy emphasis on both participant medical records
and participant observations. We believe that by using real participant data and by conducting a
review of medical records and observations we will be able to appropriately assess whether a
PO is both meeting federal requirements, as well as providing appropriate, high quality care.
CMS Action 2: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 3: One commenter requested clarification on whether we will audit at the Parent
Organization level or the contract level.
Response 3: For PACE we audit at the contract level.
CMS Action 3: No change was made to the protocol in response to this comment. No change
was made to the burden estimate as a result of this comment.
Comment 4: One commenter recommended we consider the PACE Innovations Act and how it
may impact audits.
Response 4: We appreciate this commenter’s request. We continually review any new
guidance or regulations issued by the agency and modify our audit protocols as necessary once
the new guidance is implemented. In addition to modifying our external protocols, we also
modify our internal methods of evaluation. At this time no modifications are needed in response
to the PACE Innovations Act.
CMS Action 4: No change was made to the protocol in response to this comment. No change
was made to the burden estimate as a result of this comment.
Comment 5: One commenter supported our recommendation to limit the audit review period to
one year of time.
Response 5: We appreciate this comment and agree that one year provides enough data for
auditors to do a comprehensive and thorough review of a PO’s compliance with regulations.
CMS Action 5: No change was made to the protocol in response to this comment. No change
was made to the burden estimate as a result of this comment.
Comment 6: One commenter requested clarification on whether CMS would be visiting only
one PACE center during an onsite audit, or possibly more than one center or Alternative Care
Setting (ACS).
Response 6: CMS may visit more than one center or ACS during an onsite audit. How many
sites are visited may depend on multiple factors including geographical spacing, participant
complaints and any other factors deemed relevant by the auditors. The same audit protocol will
be utilized by auditors no matter how many physical sites are visited.

CMS Action 6: No change was made to the protocol in response to this comment. No change
was made to the burden estimate as a result of this comment.
Comment 7: Several commenters, while appreciative of the detailed compliance standards that
were included in the PACE protocol, expressed concerns that a lack of interpretive guidance
within the compliance standards would lead to inconsistent audits.
Response 7: While CMS appreciates the concerns raised by these commenters, we want to
assure commenters that CMS takes consistency in audits very seriously. As such, we have
revised every single audit document that is currently used in audits. This includes not only a
new external audit protocol, but also new internal methods of evaluation, auditor tools, and
finally a new PACE Audit Consistency Team (PACT). All of these tools will be used by all
auditors to ensure consistent application of CMS regulations and manual guidance.
Additionally, we encourage POs to have open communications with auditors during the audit.
We strive to be as transparent as possible in our audits, and consider the audit an opportunity to
provide clarification or guidance on regulatory requirements. If POs have concerns with the
interpretation of a requirement, they should address this with their audit team. Lastly, POs will
have a formal opportunity to dispute any audit findings if they disagree with CMS interpretation.
CMS Action 7: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 8: Multiple commenters raised concerns regarding the universes of data that we
proposed collecting in the PACE audit protocol. Some commenters said that we requested too
much data, and questioned whether the data requested was really needed and whether it would
actually drive improvements. A few commenters suggested that different POs, depending on
their size, would have a difficult time pulling the information.
Response 8: We appreciate the concerns raised by these commenters. We consider the
amount of data we are requesting reasonable compared to what has been previously requested
during audit. We greatly reduced the amount of documents requested during audits, eliminating
the need for POs to send in numerous policies and procedures prior to auditors going onsite.
We believe most of the data requested in these universes is information that is already collected
by the PO. Additionally, these universes will not change throughout the year, allowing POs to
maintain this data in this format prior to an audit being conducted. However, after receiving
these comments, we reviewed all of the universes again and eliminated some fields that we
thought were either too burdensome or that were unnecessary. Those specific fields will be
discussed below when each element or specific record layout is discussed.
CMS Action 8: We reviewed and revised the universes of data that are collected for audit.
Comment 9: A few commenters suggested that due to the changes in the universes we are
requesting that we delay this new audit process until 2018. These commenters suggested that
if we don't delay the new process until 2018, we should at least ensure that we will not penalize
a PO for not providing accurate universes in 2017. One commenter suggested that if we
collected this much data the PO would need additional time to compile the data.
Response 9: We thank these commenters for their suggestion, but we feel that it is important to
make improvements to the audit process immediately. Therefore we will continue implementing
this new outcomes based auditing strategy with an implementation date of January 2017. In

order to do this, we will need to collect these universes of data from a PO. PACE organizations
are expected to provide data to CMS for audits as requested, however, we recognize that it may
take some time to build systems and track this data, so CMS will be reasonable when accepting
the data. We also believe that because much of the information we are requesting is already
collected by the PO, 30 calendar days is enough time for POs to pull and submit the requested
information. We are deleting some of the text in this section that discusses the three attempts
to submit a universe. Instead we are simplifying the language to only say that POs must submit
data as requested.
CMS Action 9: We modified the language surrounding submitting universes to clarify that POs
must submit the data as requested, but it no longer references a maximum of three attempts.
Comment 10: A few commenters suggested that our collection of universes was duplicative of
other reporting requirements or quarterly reporting that POs already have to comply with. They
encouraged us to not request similar data more than once as it would be burdensome to the
PO.
Response 10: We agree with commenters that we should avoid requesting duplicative
information whenever possible. We believe the information collected during audits is not
duplicative of other collections. Information submitted quarterly or during Level I and Level II
reporting is aggregate data. The information we are requesting on audit is participant specific.
This level of personal information is necessary for an audit to be conducted, while not
necessarily needed for quarterly reporting or calls.
CMS Action 10: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 11: One commenter mentioned that the universes had some questions that
contained drop down options. This commenter stated that the drop downs were similar but not
exact to already used HPMS terms. The commenter suggested we modify the terms to be
consistent.
Response 11: We have reviewed the fields to determine if changes were possible, however,
most of the fields that aren’t a specific response field (e.g., yes/no, dates, etc.) are free text
fields which allow the PO to populate answers as they wish. If a PO wishes to use the terms
currently used in HPMS reporting in these free text fields, they are welcome to do so.
CMS Action 11: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 12: One commenter suggested that instead of collecting full universes, we collect
only data for a sample of participants.
Response 12: While we appreciate this commenter’s suggestion, we cannot only ask for data
on a sample of participants. In order to do a comprehensive audit, we need all data requested.
We will be targeting our samples in 2017, which means we need a full universe of information in
order to appropriately select participants to review. Additionally, we will be running some tests
at a universe level, such as the timeliness of service delivery requests, appeals and grievances.
In order to appropriately run this test, we must have a full universe of all requests.

CMS Action 12: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 13: Several commenters requested clarification on our sampling methodology. A few
of these commenters asked for us to release our sampling methodology, including what we
mean by “significant or clinically significant” samples. Another commenter suggested that we
increase the number of samples for larger organizations, or if we would increase our sample
size if we discovered issues at the PO. One commenter stated that using a targeted sampling
methodology is unfair to organizations, and suggested we use random sampling to assess the
true compliance of the organization. Lastly, one commenter requested information on when
samples would be provided to the PO prior to the review.
Response 13: We have tried to be transparent regarding our sampling methodology in the
PACE audit protocol, by specifically informing POs of the number and types of samples we will
pull (i.e., how many denials, approvals, etc) as well as some of the clinical or operational
triggers that we may use when sampling (i.e., hospitalizations in the medical record, or sampling
a variety of personnel). We cannot release more information as auditors need some flexibility in
determining which samples are relevant for a particular audit. We also do not believe that
targeting samples is unfair to POs. It is important to pull samples that will be as clinically
significant or useful as possible, especially since we have a relatively small sample set. For
purposes of audit, the term “clinically significant” or “significant” means that we are looking for
samples that seem to warrant review based on the triggers identified in the protocol (e.g., a
hospitalization). The purpose of our audit is to determine a POs compliance with our
regulations as well as to ensure the safety of PACE participants. This does not necessarily
mean that we will be targeting for non-compliance. Additionally, if a sample involves an issue of
non-compliance, the audit team will still need to review additional information to determine how
to classify that non-compliance, which could be either a CAR, ICAR or an observation (worth
zero points) depending on what the auditors discover. Auditors will not be expanding the
sample size when issues are noted, but may request the PO to conduct an analysis to
determine the scope or impact of the issue. As for when we will be releasing the samples to the
PO, all samples will be sent 1 business day prior to the review of that element starting. This
means that for the onsite portion of the audit, the samples will be sent 1 business day before
auditors begin their onsite review, or it may mean that samples are given 1 business day prior to
conducting a remote review of medical records or other samples. We added a new section into
the protocol which discusses when samples will be given to the PO.
CMS Action 13: Added a new section titled “Selecting Samples” into the PACE audit protocol.
Comment 14: A few commenters requested clarification on the scoring methodology. One
commenter requested clarification on how the scores would be used by CMS and whether they
would be released to the industry. Another commenter requested clarification on how we intend
to score POs, and specifically the use of ICARs, CARs, and Observations. Lastly, one
commenter stated that they appreciated the revised scoring by combining the enhanced Level 1
and Level 2 reporting.
Response 14: We appreciate this opportunity to clarify our scoring methodology for PACE
audits. In our 2017 audit protocol, we proposed scoring POs during audits. In order to achieve
these scores we would classify conditions of non-compliance as either an observation,
Corrective Action Required (CAR) or Immediate Corrective Action Required (ICAR). Generally,

a condition will be classified as an observation when it is not systemic (i.e., it is a one-off
instance of non-compliance). Observations would not be worth any points and would not count
against the PO in scoring. CARs are conditions that are systemic in nature but do not impact
access to services or care. ICARs are those conditions that are systemic, impact access to
care and need to be immediately corrected. CARs would receive 1 point and ICARs would
receive 2 points. This classification would be done at the cross-regional PACT and applied
consistently throughout audits. Once the conditions are classified, all conditions would be
added up to achieve an initial score. For example, if the PO received 5 CARs and 3 ICARs in an
audit, the initial score would be 11 points (5 CARs at 1 point each and 3 ICARs at 2 points
each). Then that initial score would be divided by the number of elements reviewed (i.e., 5
elements) to achieve the PO’s overall score. In the example above, 11 would be divided by 5
and the PO’s overall score would be 2.2. We do intend to release these scores to the industry
in our annual audit report which will discuss individual scores so that POs, advocates and
States can assess how different organizations compare. As for the comment regarding Level I
and II reporting, we are unsure what this commenter meant by this comment, but will clarify that
audit scores are not tied to Level I or II reporting and should not be compared.
CMS Action 14: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 15: A few commenters requested clarification on the draft audit report process. One
commenter asked when a PO could expect to receive the draft audit report following the audit.
Several commenters requested clarification on the dispute process following the draft audit
report. These commenters encouraged CMS to have any disputes heard by an impartial third
party.
Response 15: We appreciate these commenters’ questions regarding the draft audit report
process. While we intend to release the draft audit reports within 30 calendar days following the
onsite portion of the audit, we cannot guarantee that this will always be the case. Because of
the feedback we have received regarding consistency in audits, we are implementing multiple
new processes designed to ensure that every PACE audit is handled consistently. Included in
these new processes is the PACE Audit Consistency Team (PACT) which will weigh in on every
condition from every audit prior to issuing the draft audit report. This process, while beneficial to
ensuring consistency, may extend the time it takes for auditors to issue reports following the
audit. Additionally, auditors will be requesting Impact Analysis templates to be completed when
noting conditions of non-compliance, and POs will have 10 business days to submit those
templates following the onsite portion of the review. These templates must be received before
the PACT can appropriately classify conditions. Once the draft report has been issued, the PO
will have 10 business days to respond to the report and dispute any findings. A PO may use
this process to provide written disputes or evidence in response to conditions, or offer
suggestions on rewording statements concerning the findings, such as the cause and effect
report language. As for the request that disputes to the draft report be handled by an impartial
third party, that comment is beyond the scope of this collection, but we will take the suggestion
under consideration. If disputes do occur, the audit team that was onsite must be involved in
the dispute process as they have the most personal knowledge of the conditions cited. While
these auditors will be the responders to any comments received, they will have the PACT
available to them for any assistance with how to re-word, re-classify or discard any conditions
cited in the draft report.

CMS Action 15: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 16: A few commenters provided overall support for the process improvements made
by CMS in this protocol. Specifically, commenters supported not only the revised audit process,
but also the other attempts at being transparent, such as releasing an annual audit report, using
improved tools, and utilizing a PACE Audit Consistency Team (PACT).
Response 16: We appreciate the support from these commenters. We believe the process
improvements we are proposing will help ensure that PACE participants are appropriately
protected, while also ensuring that POs are treated consistently and fairly during CMS audits.
We are committed to conducting our audits as transparently as possible, including encouraging
POs to engage auditors during the audit and during the dispute process, as well as releasing an
annual audit report. We also believe by instituting a PACT for PACE we will be taking an
important step to making sure that we consistently apply and classify audit conditions across the
nation.
CMS Action 16: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 17: Several commenters requested specific clarification regarding the role of State
Administering Agencies (SAAs). One commenter encouraged us to issue guidance to SAAs
since state practices greatly differ. Another commenter requested that CMS clarify that the SAA
would still have their own authority to audit or monitor a PO as the State determines necessary.
Specifically, the commenter requested confirmation that the SAA could audit elements not
covered by CMS, including requesting additional documentation, policies and procedures, or
expanding the universes if needed. This commenter mentioned that a State needs the ability to
monitor state specific requirements and initiatives as needed.
Response 17: We thank these commenters for their questions regarding SAAs. As part of the
three-way agreement, CMS and the SAAs share oversight responsibility for ensuring POs
compliance with pertinent regulations. This oversight can differ from State to State, with some
States choosing to join CMS onsite for audits, and other States conducting their own audits at
different times of the year. While CMS intends to share our audit strategy and tools with the
SAAs, we recognize a state’s need for their ability to conduct oversight as their state sees fit.
This oversight may include conducting an audit other than the CMS audit, or it may include
auditing elements or requirements that are outside the scope of the CMS audit. While CMS
intends to be consistent with the elements we audit, the State has the authority to expand the
scope of our audit as they determine necessary. For example, if the SAA joins CMS onsite
during our audit, and determines that they need to review additional elements, such as
enrollment or disenrollment, the SAA has the authority to do so. Additionally the SAA will have
the authority to audit or monitor any State specific requirements that the PO is required to
adhere to. We are not making changes to the CMS protocol, as our protocol does not dictate or
limit the SAA’s authority in any way.
CMS Action 17: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 18: One commenter also requested that CMS and the SAA coordinate findings so
that the stricter standard would apply. Specifically, this commenter requested that we ensure

one agency doesn't "pass" a requirement that the other agency would fail. Similarly, this
commenter requested that we clarify that just because an element passed in one audit, does not
mean that it will pass future audits if regulations or guidance is changed.
Response 18: We appreciate this commenters concerns on maintaining consistency with
findings and ensuring the stricter standard applies during audit. CMS, in an effort to audit POs
consistently, will be applying federal regulations when onsite to determine compliance. That
being said, if a State has a stricter standard that the PO must adhere too, the State can and
should monitor that PO for compliance with that standard. If a PO meets the federal standard,
but does not meet the stricter State standard, that finding can be reported by the SAA as an
issue of non-compliance, even if it does not get cited as a finding by CMS auditors. Also, when
regulations or guidance change, POs are expected to implement the new guidance in
accordance with any implementation timeframes.
CMS Action 18: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 19: One commenter asked how SAAs would report audit findings to POs. This
commenter suggested that SAAs be allowed to post their own findings into HPMS through
uploading a word or excel document, but stressed that reporting should be optional and should
not determine whether a SAA takes an enforcement action on their own.
Response 19: We appreciate these suggestions and agree with this commenter. We intend to
allow any SAAs that are onsite during our audit to upload a separate word document to the
CMS audit report that will contain any additional elements reviewed by the SAA or any State
specific requirements that the PO failed to meet. This reporting is entirely optional, and a SAA
can choose not to upload any findings, or not to conduct an additional review outside of the
CMS elements. If the SAA goes onsite at a different time than CMS, or conducts a review
outside of the timeframe of our audit, the SAA would continue to report on those findings in the
same way they do now. Additionally, an SAA being onsite with CMS, or reporting findings
through HPMS will in no way determine whether or not an SAA may take an enforcement action
on their own.
CMS Action 19: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 20: One commenter asked how the Star Ratings and PACE audits correlate. This
same commenter asked that we make sure that results from audits and Star Ratings would be
similar, so that a PO with a low audit score wouldn't have a high star rating.
Response 20: POs are not part of the Star Ratings program, so there is no correlation between
our audits and Star Ratings.
CMS Action 20: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 21: One commenter requested that we clarify what the PACE Questions document
was that the PACE Audit Process and Data Request referred to. This commenter did not see
that document in the package.

Response 21: We apologize for the confusion. We have changed the reference in the PACE
protocol to refer to the correct attachment name in the package. The name of the document is
PACE_SupplementalQuestions.
CMS Action 21: Changed the protocol to refer to the correct attachment name.
Comment 22: Multiple commenters requested clarification on the reporting of disclosed and
self-identified issues. A few commenters requested clarification on what should be included in
this reporting. Some commenters stated that this reporting could discourage POs from
reporting self-identified issues since if they are not fully corrected they could be cited a finding.
Another commenter wanted clarification on what "corrected" meant and what period of time
would impact the reporting of these issues.
Response 22: CMS appreciates the points raised by these commenters and agrees that
changes should be made to the request for disclosed and self-identified issues. CMS has
always recognized the importance of a robust internal monitoring system in order for an
organization to ensure ongoing compliance with CMS regulations. Even though POs are not
currently required to have a formal robust monitoring program, POs are required to stay in
compliance with CMS regulations, and as such, they often identify areas of non-compliance on
their own. It is important that a PO can quickly identify and correct issues of non-compliance
before a PACE audit is conducted. It is also important for POs to maintain communication with
CMS about these issues, and disclose them promptly to their Account Manager or other CMS
personnel as appropriate. Therefore, for 2017, we are eliminating the reporting of self-identified
issues and only asking POs to include issues that have been previously disclosed to CMS that
may impact their audit universes. For those disclosed issues that were promptly identified and
corrected, CMS may consider that disclosure as a reason to downgrade the classification of that
condition from an ICAR to a CAR when on audit. In other words, if the condition was severe
enough to warrant classification as an ICAR, but the PO promptly identified, disclosed and
corrected the issue, CMS would downgrade the ICAR to a CAR. This modified approach will
also ensure CMS appropriately recognizes organizations that are transparent with CMS when
discovering issues of non-compliance. Additionally, while POs will be required to discuss what
corrective action was taken in Attachment III, we have removed the language from the protocol
that discusses how to classify issues as either corrected or uncorrected. For purposes of
Attachment III, a PO will be required to identify their remediation (correction) efforts, which could
entail a variety of activities depending on the issue identified. Full correction would mean that if
there was a system issue, the system was fully fixed so that the issue no longer occurred. If the
issue identified impacted participants, correction would be considered complete once all
impacted participants were made whole. However, for purposes of Attachment III, POs will be
asked to provide the status of correction, including if the corrective action is ongoing. Lastly,
we want all disclosed issues that were disclosed to CMS prior to the PO receiving the audit
engagement letter that might affect the one year review period for the universes. We are
changing the instructions for submitting disclosed issues in the audit protocol to reflect this new
guidance. We are also updating Attachment III to reflect these changes.
CMS Action 22: We modified the disclosed and self-identified issues section of the protocol to
reflect this new guidance. We also updated Attachment III to reflect these changes.
Comment 23: Several commenters had questions relating to the attachment for reporting of
disclosed and self-identified issues (Attachment III_Pre-AuditIssueSummary). Two commenters

questioned why CMS would need the name of the individual to whom an issue was reported
(i.e., CMS or PO). Two other commenters questioned why CMS would need to know all actions
taken to address individual participants, as that could be onerous on the PO to submit.
Response 23: We appreciate the opportunity to provide clarification on how to populate
Attachment III. CMS has asked for the name of the individual at CMS that the PO reported the
issue of non-compliance to. This would not be the name of an individual at the PO, but only the
CMS employee that the disclosed issue was reported to. This would likely be the PO’s account
manager. We need that name in order to verify that the disclosure actually occurred. As for
reporting the corrective action take to remedy individual participants, Attachment III is looking for
an explanation of that corrective action, not an individual accounting. Therefore, if 10
participants were adversely impacted by the issue, and the remedy was to mail corrected
materials to those individuals, an overall explanation as well as the date the remedy was
completed (i.e., when mailings were finished) would be all that was needed.
CMS Action 23: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 24: One commenter requested clarification on whether CMS would collect issues
disclosed to the SAA or only issues disclosed to CMS. This commenter also asked if the SAA
would be able to review the disclosed issues.
Response 24: While we certainly encourage POs to disclose issues to both CMS and the SAA,
for purposes of the CMS audit the only disclosures we are requesting would be issues disclosed
to CMS. As we partner with SAA in the monitoring and oversight of POs, any disclosures
received by CMS will be shared with the SAA.
CMS Action 24: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 25: One commenter requested clarification on when CMS may cite a condition of
non-compliance based on an issue that was disclosed. This commenter pointed out that if an
issue had not been corrected they would recommend that CMS change the "may" to "will" and
cite a condition of non-compliance.
Response 25: The language referenced by the commenter has been deleted in the protocol
revisions issued by CMS. Auditors will collect and analyze disclosed issues that impact the
universe and determine how to cite conditions as appropriate, and what classification should be
used.
CMS Action 25: In response to other comments received, this language was deleted from the
protocol.
Comment 26: One commenter requested that CMS validate all self-identified issues that were
corrected. This commenter stated that ensuring appropriate correction is important and should
be done onsite.
Response 26: We are no longer requesting self-identified issues to be reported to CMS prior to
us going onsite. We are only requesting issues that were previously disclosed to CMS to be
reported to us. However, for those issues that were corrected, we will attempt to verify
correction when onsite whenever possible.

CMS Action 26: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 27: One commenter requested that the timeframe to submit this report be extended
from 5 business days to 10 business days in order to allow the PO appropriate time to QA the
report.
Response 27: We appreciate this suggestion, however we believe that 5 business days is
sufficient when considering the new changes to this section. The only issues we are requesting
from PO’s would be issues that were previously disclosed to CMS (prior to the PO receiving the
engagement letter or audit start notice). This list should be readily available and it should not
require much effort for the PO to submit to CMS.
CMS Action 27: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.

Burden Estimate:
Comment 28: Multiple commenters stated that we underestimated the burden for POs that
undergo a CMS audit. Several commenters suggested that because of the increased amount of
data we are requesting, that we should allow for more upfront costs that will be required for a
PO to build the logs and systems to capture this data. One of these commenters requested that
we add a one-time burden of $42,500 per PO. Other commenters suggested that the hours
CMS projected for the entire audit were underestimated. Some commenters thought that the
burden should, at a minimum, be the same as what CMS projected the government burden was
(i.e., 220 hours). One commenter offered specific feedback on increasing the pre-audit time
allotment from 40 hours to 80 hours.
Response 28: We appreciate the feedback from commenters on our burden estimate. We
agree that we underestimated the burden on POs and are making adjustments to reflect specific
comments received. We agree that although the information we requested from POs is
information most organizations already collect, there will probably be a one-time burden for POs
to create systems to capture, track and submit the data in the format we are requesting. We
considered each universe, the information we were requesting, and determined what we believe
is an appropriate hourly burden on tracking and capturing the information in that universe. The
specific hours per universe have been added into the revised burden estimate. We believe that
it will take approximately 630 hours for a PO to build new systems, at $61 dollars an hour, which
will equal a one-time burden of $38,430 for each of the 119 POs. We also believe that the
hours involved in both the pre-audit and post-audit estimates are underestimated. We therefore
modified the pre-audit hours from 40 hours to 80 hours in response to comments. We also
made adjustments to the overall hours a PO spends for a CMS audit. The breakdown of those
hours are discussed more specifically in comments below, but the overall hours for a PO to
undergo a CMS audit was adjusted from 180 to 240 hours. When taking into account the onetime burden of $38,430 for each PO and the increased hours for POs during the actual audit,
the total cost of the collection went from $790,560 to $5,627,250. This total is for the industry as
a whole and not per PO.

CMS Action 28: We adjusted the burden estimate to include a one-time burden for all POs, and
to reflect more pre-audit hours spent by the PO in preparation for audit and adjusted the overall
hours for a PO to undergo an audit. We also adjusted the total collection estimate for POs to
account for this new hourly estimate.
Comment 29: One commenter requested that we also account for more hours post audit. This
commenter suggested that a PO would need more than 20 hours to respond to the draft report,
and suggested we increase this estimate to 40 hours. Additionally, this commenter thought that
80 hours for corrective action and audit close out activities was underestimated and suggested
we increase that time to account for implementing corrective action plans following the CAP
process.
Response 29: We agree with this commenter that we should increase the time allowed for the
draft audit report response to more accurately reflect a POs time in drafting that response.
Therefore we are changing our estimate from 20 hours to 40 hours. However, we disagree with
the commenter that we should allow for implementing the CAPs as a part of our burden
estimate. Although we allow for some close out activities as a part of the audit, such as building
and submitting a CAP. The actual process improvements or implementing the CAP are not a
part of our burden estimate.
CMS Action 29: We changed the draft response burden to account for more hours to prepare
disputes and responses to audit conditions or deficiencies.
Comment 30: One commenter requested that our burden estimates be adjusted based on size
of the PO.
Response 30: While we appreciate this suggestion, we cannot modify our burden to account for
individual POs. Instead, we have attempted to create an average burden that would account for
both small and large organizations. While this burden may not reflect the true costs of every
PO, we feel it would accurately reflect the majority of POs.
CMS Action 30: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 31: One commenter requested that we consider implementation costs to be $1,200,
annual maintenance costs of the proposal to be $72,775 and audit preparation of logs to be
$10,800.
Response 31: We appreciate this commenter’s feedback. We took into account the time
required to gather and submit this information when creating our burden. We added a one-time
burden of $38,430 for collecting and tracking this data which we believe accounts for the costs
raised by this commenter. However, we also recognize that all POs are different and that this
burden may not accurately reflect every POs experience. We tried to adjust the burden to
represent a fair average for what expenses and time a PO may incur.
CMS Action 31: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 32: One commenter stated that the hours a PO spent during the actual audit used to
be 28 hours and asked why CMS had increased the amount of hours to 40. This commenter
also asked for confirmation that the audit would happen all onsite.

Response 32: As mentioned previously, we had to derive an average amount of hours spent by
a PO during an audit, even though that may not be accurate for every PO. Some POs choose
to have staff involved in all stages of the audit, which could mean a PO has staff involved in the
full 40 hour week. We therefore estimated this burden at 40 hours to account for those
organizations. Some organizations may spend less staff time on audit during this onsite week.
Additionally, some of the CMS review may be conducted prior to arriving onsite (such as remote
review of medical records) but we believe that any PO staff utilized for that effort would have a
minimal burden in time or money.
CMS Action 32: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 33: One commenter requested clarification on how we arrived at the estimation that
there would be 72 audits in 2017. This commenter asked if that number was based on the
current regulations at 42 CFR 460.190 and 460.192, and wanted to confirm that should the new
regulation be finalized the number would decrease.
Response 33: We calculated this number based on current regulation requirements in 42 CFR
460.190 and 460.192 which stipulates how often a PO must be audited by CMS. The estimated
72 audits is a combination of ongoing and trial period audits. The proposed regulation that
would potentially limit the number of audits each year is not yet final.
CMS Action 33: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.

Onsite Element:
Comment 34: A few commenters raised concerns regarding the onsite element in the new
proposed protocol. These commenters were concerned that CMS was severely limiting the
scope of the onsite review. These commenters noted several areas or elements that were in
the CMS audit prior to this revision, including dietary services (e.g., checking food temperatures,
food storage, kitchen inspections, emergency food supplies), physical environment/ infection
control standards (e.g., water temperatures, Life Safety Code, disposal of waste, etc.),
transportation (e.g., checking maintenance logs, verifying emergency equipment is on board,
verifying no smoking signs, etc.). Some of these commenters pointed out that not all states
have licensures, and without CMS to do this onsite review, these commenters weren't sure
anyone would review these items. These commenters wanted to know how CMS would ensure
these onsite requirements were met without doing a thorough onsite review.
Response 34: We appreciate the concerns raised by these commenters regarding the scope of
our proposed onsite element. When we began making revisions to the CMS audit protocols, we
focused first on creating an outcomes based auditing process that we believe, once
implemented, will ensure that participants are safe as well as receiving medically necessary
care and medications. We are also making efforts to be as transparent as possible in our
auditing approach, and in doing so, we have eliminated any items that we feel are either not
outcomes based or are not clearly defined in regulations or manual guidance. We determined
that some of the onsite requirements that we had formally included in our audits fell into one of
the two categories mentioned above. Additionally, in the three way agreement CMS holds with

the SAA, it clearly states that auditing of the Life Safety Code is the responsibility of the State.
That being said, although CMS does not intend to include some of these former elements into
our proposed revisions for the onsite element, that does not mean that POs are not required to
adhere to any regulations or manual guidance that we are not specifically auditing. It also
doesn’t prohibit SAAs, even ones without a specific licensure, from auditing any areas they
deem relevant.
CMS Action 34: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of these comments.
Comment 35: One commenter requested clarification on what sort of review would be done on
emergency equipment at the PO. This commenter specifically asked whether CMS would just
check whether or not the emergency equipment was there (Yes or No) or whether CMS would
also check whether equipment and drugs were not expired and functioning.
Response 35: We intend to verify that the emergency equipment specified in CMS regulations
or manual guidance is available onsite. This will include ensuring the physical presence of the
equipment as well as some basic functionality (i.e., emergency drugs are not expired).
CMS Action 35: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of these comments.
Comment 36: One commenter was concerned with CMS’s removal of specific elements or
parts of previous elements, specifically contracts, governing body, observations of a
participant’s shower/bath, observations of IDT meetings, and interviews with participants and/or
staff.
Response 36: When redesigning the audit protocol, we tried to focus on those elements that
were most specific to the participant experience and access to care/services. We determined
that some of the elements previously reviewed through audits (such as governing body or
contracts) were not outcomes based and mainly focused on written policies and procedures.
These elements can easily be reviewed through ongoing monitoring of a PO, either by CMS or
the SAA. As for observations, we want to focus our efforts on clinical observations whenever
possible. However, we still intend to observe an IDT meeting as a part of the clinical
appropriateness and care planning/onsite elements. Additionally, we still intend to conduct
interviews of staff and participants as necessary in order to verify any audit documentation
discovered during medical records or service/appeal requests. While we added an IDT
observation into the list of observations that may be conducted as part of the onsite element, we
did not specify conducting specific interviews. We believe that auditors need some flexibility in
determining which interviews need to be conducted based on what they observe at the PO.
CMS Action 36: We modified the onsite element in order to specify that CMS may conduct an
observation of the IDT.
Comment 37: One commenter raised concerns that CMS was proposing to reduce the
frequency of audits, while also proposing to reduce the scope of the onsite element. This
commenter was concerned that if these two proposals went forward it would potentially
negatively impact participants.
Response 37: While we appreciate the concerns raised by this commenter, the frequency of
audits is outside the scope of this PRA package. That is a part of a proposed rule that is not

finalized. While we are reducing the scope of the onsite element, we have increased the focus
on participant experience and ensuring that they are receiving appropriate services and
medications, and that the PO is adhering to our requirements for providing that care. We
believe the new protocol will drive improvements in the care provided to PACE participants.
CMS Action 37: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimates as a result of this comment.

On-Call Logs:
Comment 38: Two commenters requested clarification on what CMS would use the on-call logs
for as the protocol was unclear. These commenters also stated that this information is currently
captured through different mechanisms by different POs.
Response 38: We will be using the on-call logs to help us appropriately target participants for
review. Although the information specific to the call is located in the medical record, we will
need a universe of calls in order for us to know what participant to select. We will then verify the
call details and responses when reviewing the medical record or service delivery request
records. This on-call log is also an important universe for ensuring that participants have
access to care at all hours.
CMS Action 38: We clarified in the protocol that the on-call log would be utilized for targeting
participants in clinical appropriateness and the SDAG element.
Comment 39: Two commenters stated they felt requesting the on-call Log is duplicative and
unnecessary as call information is maintained in the medical record and can therefore be
reviewed during the medical record review.
Response 39: While we agree that verifying the PO’s actions through the medical record is
important, we do not request this call data as a part of the participant medical record universe.
In order to know which medical records to review, we need a universe of calls prior to going
onsite. Therefore we are keeping this as a separate universe.
CMS Action 39: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 40: One commenter asked if we intended to match participants with the Medical
Records and pointed out that we did not include participant ID in the on-call log table.
Response 40: We appreciate this commenter’s point. We do intend to match participants with
the on-call log and we have added a column into the on-call record layout for participant ID.
CMS Action 40: We added a participant ID column into the record layout for on-call logs.
Comment 41: One commenter questioned why we needed to capture seconds when requesting
the time a call was received.
Response 41: We have changed all time requests within the record layouts to no longer
request seconds in the fields. The record layout now asks for time to be enter with hours and
minutes but not seconds.

CMS Action 41: We have made revisions to all record layouts to no longer include seconds in
the time fields.
Comment 42: One commenter stated that the date field would need to be specifically
programmed by the PO in order to capture that information.
Response 42: We agree that some fields, including the date field in the on-call universe, will
need to be customized or specifically programmed in order to capture the data the way we
requested it. However, we feel we have appropriately captured the customization of these fields
in the burden estimate we constructed in the supporting statement.
CMS Action 42: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 43: Two commenters requested clarification on why we needed to ask for the
"response to call" in the record layout as this information is contained in the medical record.
Response 43: As we stated above, this information is useful to know for each call in order to
appropriately select participants for review. This information will also be verified through the
medical record during the sample selection and audit.
CMS Action 43: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 44: Two commenters requested clarification on why we were requesting "PO Follow
Up" and "Date of PO Follow Up" as this information is also included in the medical record.
Response 44: We agree with commenters that we do not need these two fields in order to
sample participants. This information can be ascertained during the medical record review
while CMS reviews participant samples. We have therefore removed these two fields from the
universe.
CMS Action 44: Removed “PO Follow up” and “Date of PO Follow up” from the on-call record
layout.

Quality Assessment:
Comment 45: Two commenters wanted clarification on what a quality initiative was, and how
CMS defined it.
Response 45: We appreciate these commenter’s request for clarification. A quality initiative is
a set of data used to measure and identify areas of good or problematic performance within a
PACE organization. We have added this definition into the PACE audit protocol for POs to use.
CMS Action 45: We defined quality initiatives in the PACE audit protocol.
Comment 46: One commenter mentioned that several of these fields would need to be
customized by the PO in order to populate them.

Response 46: We agree that POs will have to customize some fields to meet CMS
expectations. We feel we have appropriately captured the cost of populating these universes in
our revised burden estimate.
CMS Action 46: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimates as a result of this comment.
Comment 47: One commenter requested that we change our quality assessment language to
quality improvement to match the proposed regulation.
Response 47: The use of “quality improvement” is a change that has been proposed in the new
regulation put forward by CMS. This regulation is not finalized and therefore no changes can be
made to this audit protocol based on these proposals at this time.
CMS Action 47: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimates as a result of this comment.
Comment 48: One commenter asked if we needed separate fields for "corrective action
required" and "corrective action implemented". This commenter suggested having these two
fields was redundant.
Response 48: A PO may identify that corrective action is needed, but not have the time or
resources to implement it immediately. Therefore we believe that keeping both of these two
columns is necessary.
CMS Action 48: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimates as a result of this comment.
Comment 49: Two commenters requested clarification on the date corrective action was
implemented. These commenters asked if multiple dates would be needed if multiple actions
were taken within one quality initiative.
Response 49: We appreciate the opportunity to provide clarification on this field. We have
modified this field from Date of Corrective Action Implementation to Start Date of Corrective
Action Implementation. POs should populated this field with the date corrective action began,
regardless of whether there were multiple actions taken in performing correction.
CMS Action 49: We modified the Date of Corrective Action Implementation to the Start Date of
Corrective Action Implementation.

Personnel:
Comment 50: Two commenters requested clarification on what criteria CMS expects to include
in the personnel background checks. One commenter mentioned that some SAAs require police
background checks to be conducted.
Response 50: While we publish our external protocol, we do not share our internal methods of
evaluation. For information on what CMS expects for background checks, please refer to 42
CFR 460.68 and the PACE Manual, Chapter 2, Section 50.1. For purposes of the audit, CMS
will verify that the PO has satisfied federal requirements. SAAs will be responsible for ensuring
that State specific requirements are met.

CMS Action 50: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 51: Two commenters requested clarification on what criteria CMS expects to be
included in OSHA trainings.
Response 51: While we publish our external protocol, we do not share our internal methods of
evaluation. For information on what CMS expects for OSHA training, please refer to the PACE
Manual Chapter 9, Section 20.4. For purposes of the audit, CMS will verify that the PO has
satisfied these federal requirements for training.
CMS Action 51: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 52: One commenter requested clarification on how often a PO was expected to
conduct competency evaluations.
Response 52: While we publish our external protocol, we do not share our internal methods of
evaluation. For information on what CMS expects for competency evaluations, including how
often they should be conducted, please refer to 42 CFR 460.66, 460.71 and the PACE Manual
Chapter 9, Section 10.3. For purposes of the audit, CMS will verify that the PO has satisfied
these federal requirements for competency evaluations.
CMS Action 52: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimates as a result of this comment.
Comment 53: One commenter stated that they felt most of the fields requested in the List of
Personnel record layouts were reasonable, even though some would require some
customization in order to implement them.
Response 53: We recognize that some of the fields we are requesting will require a PO to
customize their systems, but we feel we have appropriately captured the burden of that
customization in our new burden estimate.
CMS Action 53: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimates as a result of this comment.
Comment 54: One commenter requested that we add "volunteer" as an option for “Type of
Employment”.
Response 54: While we appreciate this suggestion, “volunteer” is already a type of employment
in this category, therefore no revisions will be made.
CMS Action 54: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimates as a result of this comment.
Comment 55: One commenter requested that we change the name of one field from "Direct
Participant Contact" to "Immunizations/ Medical Clearance". This commenter mentioned that
some states require all staff to be medically cleared, not just those with direct participant
contact.

Response 55: We based the name of this column off of federal requirements therefore we are
keeping the column name “Direct Participant Contact”. We cannot modify our record layouts to
account for different state interpretations.
CMS Action 55: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimates as a result of this comment.
Comment 56: One commenter requested we add a column to inquire about emergency training.
Response 56: We appreciate this commenter’s suggestion and we agree that testing to ensure
a PO provides appropriate emergency training is an important part of the personnel record
review. However, we will ascertain this during the review of the personnel records and will not
ask for this information upfront.
CMS Action 56: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimates as a result of this comment.
Comment 57: One commenter argued that the burden of populating this personnel universe
was enormous as it would require the PO to review every personnel record individually when
populating this universe.
Response 57: We have reviewed the personnel record layout again and we agree that some of
the fields we asked for do not need to be requested upfront. We are therefore eliminating four
fields from the personnel universe. These fields include: “Background Check”, “Excluded
Provider List”, “Competency Evaluations”, and “OSHA training”. Instead of requesting this
information up front, we will sample personnel records and review this information in the record.
However, if we see an issue in one of these areas we will ask for an Impact Analysis (IA) to be
done to see the scope of the issue. We are therefore creating an IA template for personnel.
CMS Action 57: We removed the following fields from the personnel record layout:
“Background Check”, “Excluded Provider List”, “Competency Evaluations”, and “OSHA training”.
We also created an Impact Analysis template for personnel related conditions.
Comment 58: A few commenters requested that we add a compliance standard that specifically
assesses whether the PO is conducting training related to infection control standards.
Response 58: We are limiting our review in personnel to trainings that are specifically
referenced in either regulations or manual guidance. Therefore we are not going to add in an
infection control training. However, we believe that through other elements, including clinical
appropriateness and quality assessment, we will be able to adequately assess whether a PO is
appropriately implementing an infection control plan.
CMS Action 58: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimates as a result of this comment.

Clinical Appropriateness and Care Planning:
Comment 59: A few commenters asked for clarification on when an assessment should not be
conducted "in person" as the regulation requires all assessments to be in person. This
commenter suggested we remove the "when applicable" from the bullet in clinical
appropriateness under Review Sample Case Documentation.

Response 59: We agree with the commenter that this phrase is not necessary and have
removed the phrase “when applicable” from the bullet.
CMS Action 59: We modified the bullet under Review Sample Case Documentation to remove
“when applicable” from the phrase “documentation that assessments were done in-person.”
Comment 60: A few commenters requested we clarify two of our care plan compliance
standards by removing "if appropriate" from compliance standards 3.5.4 and 3.5.5. These care
planning compliance standards referred to the explanation of the care plan being given to a
participant and also the participant having a role in care planning decisions, which these
commenters argued was always appropriate. These commenters also requested clarification on
how CMS would define "having a role" and requested confirmation that having a signature is not
enough.
Response 60: We agree with these commenters and have removed the phrase “if appropriate”
from both of these compliance standards. As for how CMS would define “having a role”, while
we publish our external protocol, we do not share our internal methods of evaluation. For
purposes of audit, CMS will ask the PO to provide evidence that shows that the participant had
a role in care planning decisions.
CMS Action 60: We removed “if appropriate” from compliance standards 3.5.4 and 3.5.5.
Comment 61: A few commenters stated that the amount of information we are requesting in the
List of Participants was too burdensome. These commenters argued that POs are often small
without advanced systems, and that asking for this much information upfront would be difficult
for a PO to provide. One commenter suggested that in addition to requesting more data, we
were also requesting a larger roster. This commenter stated that we had previously only
requested information on a portion of participants, and not the full PACE roster. One
commenter wanted clarification on whether we were requesting the entire census of participants
or only those newly enrolled.
Response 61: We appreciate the concerns raised by these commenters. We considered the
burden on POs when putting forward audit protocols, and believe that the amount of information
requested was reasonable considering the reduction in documentation that we requested in
prior years. Most of the information contained in the record layouts is either information that has
been previously collected in audit, or information that should be readily assessable to the PO
when populating a universe. However, we have reviewed the record layouts again for fields that
are not imperative to conducting our audit. We have removed some fields from the record
layout when we thought the burden outweighed the benefit. Additionally, we want to clarify that
we are requesting the full roster of PACE participants. We do not need the medical records for
all of these participants, only a listing of the information described in Table 5. This approach,
requesting a full census of participants, is consistent with how we have conducted audits in the
past. We also believe that by publishing our record layouts in advance of an audit, it will help a
PO to prepare and be ready to submit this data, as they could utilize our universes as a way to
track data on participants if they chose to. The deleted fields are discussed in more detail below.
CMS Action 61: We eliminated some of the fields in the record layout where we deemed the
burden outweighed the benefit. The fields deleted include: “Most recent date of hospitalization”,
“Most recent date of emergency room visit”, “Number of falls reported as a Level I event”, “List
of Infections”, “Ambulation” and “Quality Analysis”.

Comment 62: A few commenters noted that specific fields in the List of Participants record
layout were reasonable for CMS to ask for information on. One commenter noted that some of
these fields would need to be customized in order to meet CMS intent but did not raise a
concern about including them.
Response 62: We appreciate the support on these fields and agree that most of the fields in
the record layout are reasonable to request of the PO. We are not going to specifically address
fields deemed “reasonable” and are focusing the remainder of comments on fields that
commenters either addressed concerns or where commenters requested clarification.
CMS Action 62: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 63: One commenter requested clarification that how a PO populates the field
"Reason for Disenrollment" is left to the PO's discretion.
Response 63: This field is a free text field and can be populated with any reason the PO feels
is appropriate so long as that entry is less than 100 characters.
CMS Action 63: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimate in response to this comment.
Comment 64: Two commenters requested clarification on a few fields, specifically: "Number of
Hospital Admissions/Observations" and the "Number of Emergency Room Visits" fields.
Specifically that this information used to be requested as a "yes/no" field in prior audits, and that
this change would require PO's to customize how they track this data.
Response 64: Although in past audits we requested this field to be entered with a “Yes/No”
answer, we are now requesting a number to be entered. We feel knowing how many
hospitalizations is important for understanding a participant’s medical history. We recognize
that this change will require the PO to customize how they track and record this information.
CMS Action 64: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 65: Two commenters requested confirmation that "Most Recent Date of
Hospitalization" and "Most Recent Date of Emergency Room Visit" is a new field and would
require customization from POs.
Response 65: In reviewing the record layout, we determined that these fields were not
necessary and have deleted them from the revised record layout.
CMS Action 65: We deleted these two fields from the revised record layout.
Comment 66: Two commenters requested clarification on the "Number of SNF/NF admissions"
and that this information used to be requested as a "Yes/No" field. These commenters also
requested clarification on how to populate the field. Specifically, whether POs should separate
medical admissions from admissions that were based on a service request.
Response 66: We appreciate the opportunity to provide clarification on this field. We do want a
number of admissions as we feel this information is important for understanding a participant’s
medical history. We are also clarifying the field to indicate that this field should capture all

admissions, for any cause, regardless of whether the admission was based on medical need or
as a result of a request for services.
CMS Action 66: We clarified this field to include language that all admissions should be
counted, even those that happened as a result of a service delivery request.
Comment 67: Two commenters requested clarification on the field "Current Center
Attendance", specifically how CMS would define "partial" when requesting whether the
participant attended the center for a full or partial day.
Response 67: In an effort to limit confusion for this field, we are modifying our request to the
number of days a participant attends the center, and we are no longer asking for whether those
days are full or partial.
CMS Action 67: We modified this field to ask for how many days a week the participant attends
the center and not whether those days are partial or full.
Comment 68: Two commenters remarked that CMS should not request the fields "Number of
Falls Reported as a Level I Event" and "Number of Falls Reported as a Level II Event." These
commenters argued that these two fields are duplicative of information already requested by
CMS. These commenters mentioned that POs have to report these events and go over them
on quarterly calls with CMS.
Response 68: While we appreciate the concerns raised by these commenters, we do not
believe that requesting this information is duplicative. While POs are required to report Level I
and Level II events to CMS, that reporting is done through aggregate data and not at a
participant specific level. For audits, we need participant specific information in order to
understand the participant’s medical history. However, when reviewing the record layouts, we
determined that requesting falls reported as a Level II were sufficient, and we are no longer
requesting falls reported as a Level I event.
CMS Action 68: We deleted the column requesting falls reported as Level I events.
Comment 69: Two commenters requested clarification on the field "Currently Recovering from a
Fall Reported as either a Level I or Level II event". Specifically, these commenters wanted to
know CMS's definition of "recovering".
Response 69: As mentioned above, we are no longer requesting information on falls related to
Level I events, only those falls related to Level II events. Generally, CMS would consider a
participant to be “recovering” if they are continuing to receive any treatment related to the fall
(e.g., physically therapy, pain medication, or the participant still resides in a SNF due to a fall).
We do not believe we should define “recovering” in the record layout as the definition may
change slightly depending on the PO.
CMS Action 69: We removed the reference to Level I falls.
Comment 70: Two commenters requested clarification on the "Number of Infections" field.
Specifically, that this field would require customization from POs in order to populate it. Also
commenters requested a definition of "infections".
Response 70: We agree with commenters that this field will require the PO to customize how
they track and record data, however, we believe that knowing how many infections a participant

has faced in a one year period is important to understanding the participant’s medical history
and how POs handle infection control. We are not defining infection control in the record layout
as the definition could vary slightly among organizations. For purposes of audit, a PO should
populate this field based on how they define infections within their infection control plan. We are
including this guidance in the record layout field.
CMS Action 70: We modified the record layout field to instruct POs to populate this field based
on how they define infections in their infection control plan.
Comment 71: Two commenters stated that the field "List of Infections" was burdensome for the
PO to populate.
Response 71: We are removing this field from the record layouts in response to these
comments.
CMS Action 71: We have removed the column “List of Infections” from the record layouts.
Comment 72: Two commenters stated that the field "Number of Pressure Ulcers" was
duplicative of reporting requirements and therefore CMS did not need to request this information
again.
Response 72: As we stated in our response to the Level I and Level II reporting requirements,
we do not believe this request is duplicative as this record layout is focused on participant
specific information and Level I and Level II reporting is done at an aggregate level.
Additionally, Level II reporting only requests pressure ulcers staged III or IV. We are requesting
pressure ulcers staged II or higher. However, we are modifying this field to simplify our request,
and changing the field from requesting the number of pressure ulcers, to a “Yes/No” field of
whether the participant had a pressure ulcer during the audit review period.
CMS Action 72: We modified the field name from “Number of Pressure Ulcers” to “Pressure
Ulcers” and changed the field description of this field to make it a “yes/no” field.
Comment 73: Two commenters requested clarification on the field "Ambulation", specifically
how CMS would define this field.
Response 73: We have removed this field in an effort to reduce the burden on POs in
populating this record layout.
CMS Action 73: The field “Ambulation” has been removed from the record layout.
Comment 74: Two commenters requested clarification on the field "Significant Weight Loss"
and how CMS would define that term. These commenters asked if the following examples were
considered significant (5% in 30 days; 7.5% in 90 days, or 10% in 180 days), and also asked
CMS to distinguish unplanned/ unanticipated versus planned/anticipated weight loss.
Response 74: We do not believe that we should define “significant weight loss” in the record
layout as that definition could change slightly in different POs. We believe the examples
presented by the commenters could meet what we would consider significant weight loss,
depending on the participant and their health status. We do want to clarify that we are only
requesting unplanned or unanticipated weight loss, and we have clarified that in the record
layout.

CMS Action 74: We have modified this field in the record layout to reflect that we are only
asking for significant unanticipated weight loss.
Comment 75: Two commenters argued that the "Skilled Therapy" field was cumbersome the
way it was written with a description of the type of therapy involved.
Response 75: We agree with commenters that this field was cumbersome and requested more
information than we needed. We have therefore changed this field from requesting a
description of the therapy to only requiring the PO to input “Yes or No” as to whether the
participant ever received skilled therapy during the audit review period.
CMS Action 75: We modified the “skilled therapy” column to be a “Yes/No” answer.
Comment 76: Two commenters requested clarification on what CMS considers a "Functional
Decline" for purposes of populating that field.
Response 76: We believe that the PO is in the best position to define whether the participant
has had a functional decline. This information has been requested on past audits and should be
defined by the PO as it would have done so in the past. We are therefore not making any
adjustments to the record layout for this field.
CMS Action 76: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 77: Two commenters requested clarification on how CMS would define "Oxygen
Use" for purposes of populating this field.
Response 77: We appreciate the opportunity to clarify this field. We are clarifying in the record
layout that we only expect a “Yes” to be entered when the participant regularly uses oxygen as
a part of their care plan (not as a result of an acute event).
CMS Action 77: We modified this field to clarify our expectations for what constitutes “oxygen
use”.
Comment 78: Two commenters requested clarification on how CMS defines "Impaired Vision"
for purposes of populating this field.
Response 78: We are clarifying this field to define “impaired vision” as when a participant is
blind or he/she has visions that is severely impaired without corrective lenses.
CMS Action 78: We modified this field to include a description of what we considered “impaired
vision”.
Comment 79: Two commenters requested clarification on how CMS defines "Impaired Hearing"
for purposes of populating this field.
Response 79: We are clarifying this field to define “impaired hearing” as when a participant is
deaf or his/her hearing is severely impaired without an assistive hearing device.
CMS Action 79: We modified this field to include a description of what we considered “impaired
hearing”.
Comment 80: Two commenters requested clarification on the intent of the field "Quality
Analysis" as the field seemed vague and not clearly defined.

Response 80: We are removing this field from the record layout to prevent confusion.
CMS Action 80: We removed “Quality Analysis” from the record layout.

Service Delivery Requests, Appeals and Grievances (SDAG):
Comment 81: A few commenters requested clarification on why CMS was requesting
information on both approved and denied service delivery requests. These commenters thought
this was an administrative burden without offering a benefit.
Response 81: We disagree with these commenters. While we understand the importance of
testing denied service requests, there is also a benefit to testing approved requests. First,
regardless of whether the request is approved or denied, the PO must process that request
within a specific timeframe, so those requests will also be run through the timeliness test.
Second, once a request is approved, the PACE organization should provide the service as
expeditiously as the participant’s health condition requires, and we may choose to review some
samples to ensure services are being provided in a manner that meets that requirement.
CMS Action 81: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 82: One commenter requested that instead of using the templates (record layouts)
we created for audit we use information already gathered from CMS through Level I or Level II
reporting.
Response 82: While we appreciate this commenter’s suggestion, we cannot use the
information in Level I and Level II reporting because it is not participant specific. We need to
have participant specific information in order to conduct a comprehensive audit that focuses on
participant care and the participant experience.
CMS Action 82: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 83: A few commenters noted that specific fields in the Service Delivery Requests,
Appeals and Grievances record layouts (Tables 1-3) were reasonable for CMS to ask for during
an audit. One commenter noted that some of these fields would need to be customized in order
to meet CMS intent.
Response 83: We appreciate these commenters and agree that most of the requested
information is reasonable, even though some will require customization from the PO. We will
only be discussing those specific fields where commenters raised concerns or questions
regarding the data we were requesting.
CMS Action 83: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 84: A few commenters raised concerns regarding CMS's request for "time" as it
relates to service delivery requests and appeals. Specifically fields requesting the time of
decision, time of notification, and time request was received. These commenters stated that
time should not be requested as that level of specificity is not required in order for CMS to do

their review. Additionally these commenters said that if CMS requested time, they should not
request this time to include seconds, only hours and minutes.
Response 84: We appreciate the concerns raised by these commenters and have closely
reviewed our requests for time within the different record layouts. We agree that we do not need
to request time for service delivery requests in order to assess timeliness of the decisions.
Therefore we have removed all of the “time” fields from the service delivery request universe.
However, in order to appropriately assess timeliness of expedited appeal requests we do need
to ascertain the time of receipt and notification, so we are keeping the “time” fields in the
appeals universe, although we have noted that POs do not need to populate the time for nonexpedited appeals. Lastly, we agree with commenters that we do not need to request time in
seconds for any of the time fields.
CMS Action 84: We have removed the “time” fields from the service delivery requests universe
and modified the time fields in the other universes to remove the seconds from them.
Comment 85: Two commenters noted that there was one use of "beneficiary" instead of
"participant" under the Review Sample Case Documentation. These commenters also
requested "decision letter" be changed to "resolution letter" in this section. Lastly, these
commenters requested that "time" be removed from the grievance documentation list.
Response 85: We appreciate these commenters pointing these items out, and have made the
changes mentioned above in the protocol.
CMS Action 85: We changed the term “beneficiary” to “participant”, changed “decision letter” to
“resolution letter” and removed “time stamp” from the documentation list.
Service Delivery Requests (Table 1):
Comment 86: A few commenters requested clarification on when a request is deemed
"received" by the IDT for purposes of starting the service request.
Response 86: For purposes of audit, POs should input the date they deemed the request
received in the “Date Service Delivery Request Received” field. Additionally, we are adding a
new question to the PACE Supplemental Questions document requesting information on when
the PO deems the request “received” and the portion of internal policy or procedure that
addresses the PO’s receipt of the service request.
CMS Action 86: We have added a question to the PACE Supplemental Questions document
relating to when a service delivery request is deemed received.
Comment 87: One commenter requested clarification on the field "Category of Requests" and
wanted to verify that this field was free text and could be populated as the PO sees fits.
Response 87: This field is free text and can be populated as the PO wants as long as the
category is within the 50 character field limit.
CMS Action 87: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimate in response to this comment.
Comment 88: Two commenters requested clarification on the fields "Date(s) Assessment(s)
Performed", “Discipline(s) Performing Assessment(s)”, and “Assessment(s) In Person”.

Specifically these commenters argued that the population of these fields were cumbersome,
and requested clarification on what would happen if a PO entered NA in these fields, and
whether they would automatically be cited for non-compliance.
Response 88: Although we understand that providing this information may require the PO to
track and record data differently, we feel this information is important to have for every
participant. When populating these fields, POs should enter NA if there were no assessments
performed for that particular service delivery request. CMS encourages PO to begin
programming their system or creating records that house this information prior to an audit, so it
will be less of a burden when an audit occurs. CMS will be reasonable and work with POs
during the initial transition to the new protocol and record layouts.
CMS Action 88: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 89: Two commenters stated that the field "Request Disposition" only had approved
and denied as options, even though the compliance standard in SDAG referred to partial denials
as well.
Response 89: We agree with commenters and modified the “Request Disposition” field to
include “partially denied” as a third option.
CMS Action 89: We added “Partially Denied” as an option for request disposition.
Comment 90: Two commenters requested clarification on why CMS was requesting the date
and time of the decision for the service delivery request as there is no requirement in the
regulation regarding decision making.
Response 90: We agree with commenters that we do not need to request this information as
we are not performing a timeliness assessment on the decision date, only on the notification
date. We are therefore removing these fields.
CMS Action 90: We have removed the “Date of Decision” and “Time of Decision” fields.
Comment 91: One commenter requested clarification on whether the PO could populate the
field "Date of written notification" with the date the written notification was printed.
Response 91: When requesting the date of the written notification we are requesting when the
notification was provided to the participant or caregiver. A PO may populate this field with the
print date if that date is also when the PO mails or delivers the letter.
CMS Action 91: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 92: Two commenters requested clarification on the “Quality Analysis” field within the
service delivery request layout.
Response 92: We have reviewed this field and do not believe it is necessary to collect for each
participant. We are therefore removing this field from the record layout.
CMS Action 92: We removed the field “Quality Analysis” from the record layout.
Appeal Requests (Table 2):

Comment 93: Two commenters raised concerns regarding CMS's request for an appeals
universe. These commenters argued that POs already report appeals data through Level I
reporting and therefore this universe is duplicative. Additionally, these commenters pointed out
that most POs already have appeals logs and therefore would need to make modifications to
capture the data as requested by CMS.
Response 93: While we appreciate the concerns raised by these commenters, the information
reported to CMS through Level I and Level II reporting is aggregate data and not participant
specific. We need participant specific universes for purposes of audit. We are also
standardizing universes, and we understand in doing so that some POs will need to modify or
change the way they currently collect appeals information, however, we feel that we have
appropriately captured those changes in our burden estimates.
CMS Action 93: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 94: Two commenters stated that the field "Reviewer" is not necessary as the
reviewer must be a credentialed impartial third party.
Response 94: We agree with commenters that this field is unnecessary. We will verify the
appropriate reviewer during the sample review.
CMS Action 94: We removed the field “Reviewer” from the record layout.
Comment 95: Two commenters stated that the field "Request Disposition" only had approved
and denied as options, even though the compliance standard in SDAG referred to partial denials
as well.
Response 95: We agree with commenters and modified the “Request Disposition” field to
include “partially denied” as a third option.
CMS Action 95: We added “Partially Denied” as an option for request disposition.
Comment 96: Two commenters requested clarification on why CMS was requesting the date
and time of the decision for the appeal request.
Response 96: We agree with commenters that we do not need to request this information as
we are not performing a timeliness assessment on the decision date and time, only on the
notification date and time. We are therefore removing these fields.
CMS Action 96: We have removed the “Date of Decision” and “Time of Decision” fields.
Comment 97: Two commenters requested that "Time of Oral Notification" include an option of
NA for non-expedited appeals.
Response 97: We agree with commenters and have added the option for NA to be populated
when the appeal is not an expedited appeal.
CMS Action 97: We added an option to provide NA if the appeal was not an expedited appeal.
Comment 98: Two commenters argued that the field "Time of Written Notification" is an
unreasonable request, and that it is nearly impossible for a PO to provide that information.

Response 98: While we understand these commenters’ concerns, POs are responsible for
ensuring that they follow federal regulations, and that they can show federal auditors that they
adhere to these regulations. POs are required to provide written notification within 72 hours of
receiving the expedited appeal request, and they are also responsible for assuring federal
auditors that they have satisfied this timeframe.
CMS Action 98: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 99: Two commenters requested clarification on the field "Quality Analysis". These
commenters suggested the field was vague. They also asked what this information might be
used for.
Response 99: We appreciate the opportunity to provide clarification on this field. POs are
required to analyze appeal information. This field is asking for whether the particular appeal in
question was used or analyzed in the PO’s quality program. We have clarified the field to add
the word “particular” to clarify this intent.
CMS Action 99: We clarified the field “Quality Analysis” to indicate that we are referring to the
particular participant’s appeal.
Grievances (Table 3):
Comment 100: A few commenters stated that POs already keep logs of this grievance
information and that CMS universes would mean changing the way they capture or track data.
Additionally commenters raised concerns that this universe is duplicative to the reporting POs
already do in Level I reporting.
Response 100: While we appreciate the concerns raised by these commenters, we do not
believe this information is duplicative of information already requested by CMS. Level I and
Level II reporting is aggregate data and not participant specific. We need participant specific
information in order to efficiently audit. We understand that asking for the universes in a
standardized format may require PO’s to change how they track and record data, but we believe
we have covered the costs of these changes in our estimated burden.
CMS Action 100: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 101: One commenter requested clarification that the "category of request" field could
be customized by the PO.
Response 101: This field is a free text field and can be populated as needed by the PO so long
as it stays within the 50 character field limit.
CMS Action 101: No changes were made to the protocol in response to this comment. No
changes were made to the burden estimate in response to this comment.
Comment 102: Two commenters stated that for the field "Date of Decision" that this field would
require POs to create resolution timeframes and update policies and procedures.

Response 102: POs are already required to develop grievance timeframes and policies and
procedures. In order to assess timeliness of grievances, we will request the policies and
procedures that POs already have in place.
CMS Action 102: No changes were made to the protocol in response to these comments. No
changes were made to the burden estimate in response to these comments.
Comment 103: Two commenters requested clarification on the field "Quality Analysis" as they
saw this field as vague.
Response 103: We appreciate the opportunity to provide clarification on this field. We are
requesting a Yes or No on whether that particular grievance was analyzed or used in the quality
assessment program by the PO. We have modified the field to include the word “particular” into
the description.
CMS Action 103: We added the word “particular” into the field description for “Quality
Assessment”.