CMS-R-305 Monitoring Medicaid MCOs and PIHPs final BBA

OMB Approval No.
0938-0786

Monitoring Medicaid Managed Care
Organizations (MCOs) and Prepaid Inpatient
Health Plans (PIHPs):
A protocol for determining compliance with Medicaid Managed
Care Proposed Regulations at 42 CFR Parts 400, 430, et al.

Department of Health and Human Services
Centers for Medicare & Medicaid Services

Final Protocol
Version 1.0
February 11, 2003

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid
OMB control number. The valid OMB control number for this information collection is 0938-0786. The time required to complete this
information collection is estimated to average 1,591 hours per response for all activities, including the time to review instructions, search existing
data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the
time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer,
Baltimore, Maryland 21244-1850.
Form CMS-R-305

Table of Contents
I.

Purpose and Origin of the Protocol......................................................................................1

II.

Overview of the Protocol .....................................................................................................3

III.

Protocol Activities ...............................................................................................................5
ACTIVITY 1: Planning compliance monitoring activities .....................................5
ACTIVITY 2: Obtaining background information from the State .........................7
ACTIVITY 3: Document review ..........................................................................17
ACTIVITY 4: Conducting interviews ..................................................................25
ACTIVITY 5: Collecting accessory information .................................................63
ACTIVITY 6: Analyzing and compiling findings ................................................63
ACTIVITY 7: Reporting evaluation results to the State Medicaid Agency..........68

Attachment A: Summary of Compliance Determination Activities of Public and Private Quality
Oversight Organizations .......................................................................................70
Attachment B: Compliance Determination Activities for Individual Regulatory Provisions .......74
Subpart C--Enrollee Rights and Protections
438.100: Enrollee rights ....................................................................................................75
438.10: Information requirements .........................................................................75
438.100: Enrollee rights (continued) .................................................................................93
438.102: Provider-enrollee communications ........................................................95
438.100: Enrollee rights (continued) .................................................................................98
Subpart D--Quality Assessment and Performance Improvement
Access Standards
438.206: Availability of services .....................................................................................109
438.208: Coordination and continuity of care .................................................................119
438.210: Coverage and authorization of services............................................................127
Structure and Operation Standards
438.214: Provider selection .............................................................................................134
438.12: Provider discrimination prohibited .......................................................135
438.218: Enrollee Information.........................................................................................139
438.224: Confidentiality ..................................................................................................139
438.226: Enrollment and disenrollment...........................................................................139
438.56: Disenrollment: Requirements and limitations .....................................139
438.228: Grievance systems ............................................................................................143
438.230: Subcontractual relationships and delegation ....................................................144

Measurement and Improvement Standards
438.236: Practice guidelines ............................................................................................147
438.240: Quality assessment and performance improvement program...........................151
438.242: Health information systems ..............................................................................158
Subpart F--Grievance System
438.400: Statutory basis and definitions..........................................................................162
438.402: General requirements........................................................................................163
438.404: Notice of action.................................................................................................166
438.406: Handling of grievances and appeals .................................................................173
438.408: Resolution and notification: Grievances and appeals.......................................176
438.410: Expedited resolution of appeals........................................................................180
438.414: Information about the grievance system to providers and subcontractors .......182
438.416: Recordkeeping and reporting requirements......................................................184
438.420: Continuation of benefits while the MCO or PIHP appeal and the State fair
hearing are pending...........................................................................................185
438.424: Effectuation of reversed grievance resolutions.................................................187

Attachment C: Sample Documentation and Reporting Tool .......................................................189

I. Purpose and Origin of the Protocol
PURPOSE OF THE PROTOCOL
The Balanced Budget Act of 1997 (BBA) requires State Medicaid agencies that contract with
Medicaid managed care organizations (MCOs) to develop a State quality assessment and
improvement strategy that is consistent with standards established by the Department of Health
and Human Services (DHHS). 1 A final regulation issued in the Federal Register on June 14,
2002 specifies these standards. 2 The BBA also required DHHS to develop protocols to be used
in independent, external reviews of the quality and timeliness of, and access to, care and service
provided to Medicaid beneficiaries by Medicaid MCOs. In a separate NPRM published on
December 1, 1999, DHHS identified monitoring of MCO and Prepaid Health Plan (PHP)
compliance with the BBA-required quality standards as one of the activities to be addressed by
the protocols.
1
2

The requirement for these standards is located in section 1932(c)(1)(B) of the Social Security Act.

DHHS first proposed regulations to specify these standards in a Notice of Proposed Rulemaking (NPRM)
published in the Federal Register on September 29, 1998; and in a final regulation issued in the Federal Register on
January 19, 2001. The final regulations published in the Federal Register on June 14, 2002 amended the Medicaid
managed care regulations published on January 19, 2001.

Private accreditation organizations, State licensing and Medicaid agencies, and the Federal
Medicare program all recognize that having standards for quality health care is only the first step
in promoting safe and effective health care. Making sure that the standards are followed is the
second step. This protocol describes that second step: How State Medicaid agencies or their
contractors can determine, in a manner consistent with standard industry practices, the extent to
which MCOs and Prepaid Inpatient Health Plans (PIHPs) comply with the Federal quality
standards for MCOs and PHPs.
ORIGIN OF THE PROTOCOL
To develop this protocol, approaches used by the following organizations during the period
January 1999 to January 2000 to determine compliance with their MCO standards were
reviewed.
-

American Accreditation Health Care Commission/URAC - National Network
Accreditation Program

-

Health Care Financing Administration - Medicare Contractor Performance
Monitoring System

-

Joint Commission on Accreditation of Healthcare Organizations (JCAHO) Accreditation Program for Health Care Networks

-

National Committee for Quality Assurance (NCQA) - Accreditation Program
for Managed Care Organizations

Summaries of their compliance determination procedures are found in Attachment A. An
analysis of their procedures for evaluating compliance with standards similar to those in the
Medicaid Managed Care Final Rule provided the basis for this protocol. This comparative
analysis identified two common approaches that each of the noted reviewing bodies utilize to
determine compliance with regulations and standards: 1) document review and 2) interviews
with organization staff.

II. Overview of the Protocol
The protocol uses two main sources of information to determine compliance with the proposed
BBA requirements: 1) document review and 2) interviews with MCO/PIHP personnel.
Individually, document review and interviews do not always give a complete picture of an
organization’s compliance with regulatory provisions. However, when combined, they can lead
to a better understanding of organization performance. This protocol describes how to efficiently

combine and conduct document review and interview activities in order to determine the extent
to which an MCO/PIHP complies with the BBA regulatory provisions.
When monitoring for compliance with specific regulatory requirements, it generally is inefficient
to monitor compliance by focusing on one regulatory provision at a time. Instead, grouping
document review and interview activities together in some logical fashion produces a more
efficient evaluation process, and can lead to evaluation results that better reflect an
organization’s overall performance. Because of this, the protocol activities are not organized by
regulatory provisions, but by the groups of documents to be reviewed and interviews to be
conducted. All regulatory provisions are addressed in this approach. Parties interested in a
“crosswalk” that shows how each regulatory provision is addressed in the interviews and
document review should refer to Attachment B. This attachment presents each individual
regulatory provision and its corresponding document review and interview activities used to
assess MCO and PIHP compliance.
Document review. The protocol groups documents to be reviewed according to subject matter.
This will assist MCOs/PIHPs to identify what is to be evaluated. This should also assist
reviewers to identify all issues to which a particular document or group of documents pertain,
allowing one evaluation of the document to address multiple requirements. The protocol
references documents by their generic name or title. Organizations may refer to documents by
various names. Therefore, the name and title of a document are not necessarily what is
important, rather the presence or absence of evidence within a document is what is important
when determining compliance. The content for which the document is being reviewed is
identified for individual regulatory provisions or groups of related regulations. Attachment B
lists documents to be reviewed under each regulatory provision for which they may provide
evidence of compliance.
Interviews. While document review is an important part of determining compliance,
understanding the document content and performance of procedures outlined in the documents
typically can only be determined by talking with MCO/PIHP personnel. Therefore, interaction
with MCO/PIHP staff is required in order to obtain a complete picture of the degree of
compliance with requirements. Interviews are an effective way of gathering data from people
and are the second component of this protocol
Interviews provide clarification. They can reveal the extent to which what is documented is
actually implemented. Interviews also provide an opportunity to explore any issues that were not
fully addressed in documents, and also provide a better understanding of MCO/PIHP
performance. However, the interview process can provide many logistical challenges -- both for
reviewers and the MCOs/PIHPs being evaluated. Because of this, it is important that the process
be coordinated in order to provide the best results. Individuals typically are grouped together for
interviews, based on the functions that they perform or to which they contribute in some
capacity. Individuals who are anticipated as being able to enhance understanding of the

MCO’s/PIHP’s performance are identified generically in this protocol by their title or function.
As MCOs/PIHPs may use various titles and function identifiers, interviews can be conducted
with individuals or groups serving in similar capacities as those typically associated with the
generic definitions.
Issues to be addressed during interviews are specified in suggested or potential questions or
probes. The questions or probes are not designed to be read, but rather to serve as potential
guides for exploration of compliance issues. These questions or probes can and should be used
in more than one interview to determine consistency of the responses. The questions or probes
also should be posed in such a way that a reply by more than one participant can be expected.
Open-ended questions that encourage conversation among participants in an interview are more
useful than those that call for a “yes” or “no” answer.
The questions or probes for each regulatory provision or group of provisions appear in
Attachment B.
Although document review and interviews are the core activities that comprise this protocol,
additional processes are necessary to prepare for, and thereby effectively support and conclude
the compliance determination activities. All these protocol activities are presented in the order
in which they are intended to be performed. Each activity includes a description,
recommendations for timing, structure and process, and when applicable, other supplemental
information, such as lists or forms. The seven activities that comprise this protocol are:
1.
2.
3.
4.
5.
6.
7.

Planning for compliance monitoring activities
Obtaining background information from the State Medicaid agency
Document review
Conducting interviews
Collecting any other accessory information; e.g., from site visits
Analyzing and compiling findings
Reporting results to the State Medicaid Agency

III. Protocol Activities
ACTIVITY 1: Planning compliance monitoring activities
Onsite visits to MCOs/PIHPs are commonly used by organizations such as private accreditation
organizations to perform quality oversight and compliance determination assessments. Onsite
visits are an effective method for performing monitoring activities such as document review and
interviews. Arranging an onsite visit requires careful planning in order for the effort to provide
useful results. It is recommended that these activities begin anywhere from two to six months
prior to the actual visit date. Steps include:

-

Establishing contact with the MCO/PIHP
The success of an onsite evaluation hinges on the coordination between the evaluator and the
entity being evaluated. Early contact and communication with the MCO/PIHP is necessary
to plan an efficient and effective survey and therefore is a crucial step in arranging and
conducting an onsite evaluation. Efforts should be made to establish and maintain consistent
communication links for both the evaluator and the MCO/PIHP. When feasible, these links
should be with more than a single individual. A communication plan and expectations
should be outlined and followed to the extent possible.

-

Gathering information on the MCO’s/PIHP’s characteristics
Planning and conducting an efficient and effective onsite evaluation of an MCO/PIHP
requires knowledge about the entity’s structure, enrolled population, providers, services,
operations (e.g., locations where activities take place, contractors), resources and delegated
functions. This information may be obtainable from the State Medicaid agency, or may need
to be requested directly from the MCO/PIHP. The information collected should focus on
what is needed to plan for an onsite evaluation; it is not intended to be the collection of
compliance information. A consistent set of information with defined uses should be
identified, and methods for collection need to be established. A sample set of information
might include:
-

-

Organization name and mailing address
Contact person’s name, title, phone number, E-mail address
Location for the onsite visit (e.g., headquarters address)
Organizational charts or other structural descriptions of the MCO/PIHP
Product lines offered
Total individuals enrolled (for current and previous year) with a breakdown by
product line
Total individuals served (for current and previous year)
Total number of network practitioners (for current and previous year) with a
breakdown by type, for example, primary care, OB/GYN, and other specialties
Total number of network organizational providers (hospitals, ambulatory care,
home care, laboratories, etc.)
Service descriptions and benefit designs available to purchasers
Delegated activities

Determining the length of visit and the visit dates
The length of an onsite visit will vary based on the scope of the evaluation, the number of
reviewers available to conduct the visit, and characteristics of the MCO/PIHP collected
in the previous step. Based on the experience of accrediting bodies who survey for

compliance with standards similar to those requirements contained in the Final Rule, the
estimated length for an onsite visit is generally from three to five days.
Scheduling an onsite evaluation can be accomplished by identifying a potential range of
dates from which the MCO/PIHP can select specific days for their onsite visit. Since
reviewer interaction with staff is a primary component of the evaluation process, it is
important to provide the MCO/PIHP an opportunity to select dates that ensure the
greatest availability of interview participants.
- Identifying the number and types of reviewers needed
Reviewer skills and experience will be instrumental to successful implementation of the
evaluation. Effective reviewers need to have interviewing skills, and the ability to read
and process a variety of data in order to make determinations of whether an MCO/PIHP
is demonstrating conformity with the regulatory provisions. Knowledge or experience in
State Medicaid programs and managed care in general, are highly desirable qualifications
for reviewers. Reviewer orientation and training should be considered to ensure
familiarity with the regulatory provisions, the evaluation process and performance
expectations.
A defined number of reviewers should be assigned to conduct the onsite evaluation of an
MCO/PIHP. The number should be based on the characteristics of the MCO/PIHP that is
being evaluated. Conditions such as the type of MCO/PIHP, size of provider network,
number of Medicaid enrollees and scope of programs being contracted for by the State
Medicaid agency will be some of the determinants used in identifying the number of
reviewers needed.
- Establishing an agenda for the visit
Performing an efficient and effective onsite evaluation requires a plan, or agenda. An
agenda will assist the MCO/PIHP in planning for staff participation, gathering
documentation and addressing logistical issues, such as arranging locations for reviewers
to conduct document review and interviews. An agenda sets the tone, as well as
expectations for the onsite visit so that both the MCO/PIHP and the reviewers understand
the objectives and time frames for the review. The MCO/PIHP should be consulted
throughout the agenda setting process as it is in the best position to know which staff
need to be involved.
- Providing preparation instructions and guidance to the MCO/PIHP
An onsite evaluation process requires the cooperation of the MCO/PIHP being evaluated.
If the MCO/PIHP is prepared for the onsite visit, reviewers can remain focused on
conducting and completing the evaluation in the allotted time frame. In preparation for
the onsite visit, MCOs/PIHPs should be provided with information such as: the scope of
the evaluation to be performed, how the evaluation will be conducted, lists of documents

that need to be available, instructions for the organization and presentation of documents,
completion of any forms or other data gathering instruments (e.g., an Information
Systems Capability Assessment - see Appendix Z), expected interview participants,
administrative arrangements, and other expectations or responsibilities.

ACTIVITY 2: Obtaining background information from the State
Some provisions of the final regulations published on June 14, 2002 contain language that either
required or allowed the State Medicaid agency to make certain decisions and specify certain
standards or requirements for MCOs/PIHPs. This typically was done in situations in which there
was felt to be a need to respond to known or potential regional, state or local practices and
geographic conditions, or where the state of the art has not yet evolved to produce a single
standard of care. Because of this, when assessing compliance with certain regulatory provisions,
the entity conducting the compliance review will need to obtain from the State Medicaid agency
any standards, requirements, or decisions pertaining to MCOs/PIHPs that the State has specified
in response to the BBA regulatory requirements on the State. The following list represents a
compilation of these State standards, requirements, or decisions consistent with the June 14,
2002 regulations. These standards, requirements, and decisions are also noted in Attachment B.

Subpart C--Enrollee Rights and Protections
Regulatory Provision
438.100(b)(2)(i): Enrollee right to
receive information

Information to Obtain from the State
1) the language(s) that the State Medicaid agency
has determined are prevalent in the MCO’s/PIHP’s
geographic service area.
2) any requirements the State has issued to the
MCO/PIHP specifying a standard for the reading
level of written materials prepared for Medicaid
enrollees.
3) the State’s decision about whether or not the
MCO is to notify all enrollees at least once a year
of their right to request and obtain the information
listed in paragraphs (f)(6) and (g) of §438.10.
4) the State’s decision about whether the MCO is
to furnish to each of its Medicaid enrollees the
information listed in paragraphs (f)(6) and (g)
within a reasonable time after the MCO/PIHP
receives, from the State or its contracted
representative, notice of the recipient’s enrollment.

Subpart C--Enrollee Rights and Protections
Regulatory Provision

Information to Obtain from the State
5) information on how the State has defined a
“significant change” in the information MCOs/
PIHPs are required to give enrollees pursuant to
§438.10(f) and (g).
6) whether or not the MCO/PIHP is part of a State
managed care initiative that employs mandatory
enrollment of beneficiaries in the MCO/ PIHP
under section 1932(a)(1)(A) of the Act. If the
MCO/PIHP is part of such an initiative, obtain
information from the State on the State’s decision
about whether the State or the MCO is to provide
potential enrollees with the information contained
in §438.10(h).
7) IF the MCO/PIHP is part of a mandatory
managed care initiative AND IF the State has
directed the MCO to provide comparative
information on disenrollment as part of a chart-like
comparison of MCOs and PIHPs, obtain the State
agency’s definition of “disenrollment rate.”
8) whether or not the State agency has chosen to
give providers the right to challenge the failure of
an MCO/ PIHP to cover a contracted service.
9) any applicable State laws on enrollee rights.

438.100(b)(2)(iii): Enrollee right to
receive information on available
treatment options and alternatives . . .
including requirements of §438.102:
Provider-enrollee communications

Information on whether or not the MCO/PIHP has
documented to the State any moral or religious
objection to providing, reimbursing for, or
providing coverage of, a counseling or referral
service for a particular Medicaid service or
services.

438.100(b)(2)(iv) and (v): Enrollee
right to:
- participate in decisions regarding
his or her care, including the right to
refuse treatment;
- Be free from any form of restraint . .
. as specified in other Federal

1) a written description of any State law(s)
concerning advance directives. The written
description may include information from State
statutes on advance directives, regulations that
implement the statutory provisions, opinions
rendered by State courts and other States
administrative directives. [Note to reviewers:
Each State Medicaid agency is required under

Subpart C--Enrollee Rights and Protections
Regulatory Provision
regulations.
And related:
438.6(i) Advance directives

438.100(d): Compliance with other
Federal and State laws

Information to Obtain from the State
Federal regulations at 42 CFR 431.20 to develop
such a description of State laws and to distribute
it to all MCOs. Revisions to this description as a
result of changes in State law are to be sent to
MCOs no later than 60 days from the effective
date of the change in State law.]
2) information on whether or not the MCO/PIHP
has documented to the State any moral or religious
objection to fulfilling the regulatory provisions
pertaining to advance directives.
Obtain from the State Medicaid agency the
identification of all State laws that pertain to
enrollee rights and with which the State Medicaid
agency requires its MCOs/PIHPs to comply.

Subpart D: Quality Assessment and Performance Improvement
Regulatory Provision
438.206: Availability of services

Information to Obtain from the State
Information on whether or not:
1) the State agency has required the MCO or PIHP
to adhere to any explicit standards for provider
network adequacy, such as prescribed primary
physician/enrollee ratios or specialist/enrollee
ratios;
2) the State agency has in place any time or
distance standards for beneficiary travel to access
covered services in Medicaid fee-for-service; and
3) there are any State laws requiring MCOs and
PIHPs to make specific types of providers available
for the provision of certain services.

438.206(c)(1): Furnishing of services
and timely access

Obtain a copy of the State Medicaid agency’s
standards for timely enrollee access to care and
services required of Medicaid MCOs and PIHPs.

438.206(c)(2): Furnishing of services
and cultural considerations.

1) descriptive information on the State’s efforts to
promote the delivery of services in a culturally
competent manner to all enrollees, including those
with limited English proficiency and diverse
cultural and ethnic backgrounds.
2) the requirements the State has communicated to
the MCO/PIHP with respect to how the
MCO/PIHP is expected to participate in the State’s
efforts to promote the delivery of services in a
culturally competent manner.

438.208: Coordination and continuity
of care

1) definition/specifications used by State to
identify individuals with special health care needs
(SHCNs).
2) methods used by the State to identify to the
MCO/PIHP new enrollees with SHCNs.
3) whether the MCO/PIHP is required to screen to
identify and/or assess persons with SHCNs using
the State’s definition of SHCNs.
4) State requirements for MCO/PIHP care
coordination programs.
5) if the organization to be reviewed is a PIHP,
whether the PIHP is required to ensure each
enrollee has: A) an ongoing source of primary care
appropriate to his/her needs, and B) a person/entity
formally and primarily responsible for coordinating
the health care services furnished to the enrollee.
6) if the organization is an MCO serving enrollees
also enrolled in a Medicare+Choice plan and
receiving Medicare benefits, information about the
extent to which the MCO is required to implement:
- for enrollees determined to have ongoing special
conditions that require a course of treatment or
regular care monitoring, a mechanism to ensure
that: (1) the enrollee may directly access a
specialist (e.g., through a standing referral or
approved number of visits) as appropriate for the
enrollee’s condition and identified needs, and (2) a
treatment plan that, if required by the MCO/PIHP

is developed by the specialist in consultation with
the enrollee’s primary care provider, and is
(i) developed with enrollee participation;
(ii) approved by the MCO/PIHP in a timely
manner, if this approval is required; and
iii) in accord with the State’s quality assurance
and utilization review standards.
- a primary care and coordination program that
meets State requirements and ensures each enrollee
has 1) an ongoing source of primary care
appropriate to his/her needs and 2) a person or
entity formally and primarily responsible for
coordinating health care services furnished to the
enrollee.
7) The State’s quality assurance and utilization
review standards.
438.210(b-e): Coverage and
authorization of services, including
438.114, emergency and poststabilization services

Obtain from the State Medicaid agency the Stateestablished standards for MCO/PIHP processing of
standard authorization decisions.

438.214: Provider selection

Obtain from the State information on any
credentialing, recredentialing or other provider
selection and retention requirements established by
the State.

438.226: Enrollment and
disenrollment, including section
438.56: Enrollment and
disenrollment: Requirements and
limitations

Information on:
1) whether or not the State Medicaid agency allows
the MCO or PIHP to process enrollee requests for
disenrollment for cause and, if so, whether or not
the State requires enrollees to seek redress through
the MCO’s or PIHP’s grievance system before the
State makes a determination on the enrollee’s
request.
2) a copy of the State-MCO contract provisions
which specify the methods by which the MCO or
PIHP assures the State Medicaid agency that it
does not request disenrollment for reasons other
than those permitted under the contract.

438.228: Grievance systems

Obtain information on whether or not the State
delegates to the MCO or PIHP responsibility for
providing each Medicaid enrollee (who has
received an adverse decision with respect to a
request for a covered service) notice that informs
him or her that he or she has the right to a State
Fair hearing to reconsider their request for the
covered service.

438.230: Subcontractual relationships
and delegation

Obtain from the State the “periodic schedule”
established by the State according to which the
MCO/PIHP is to monitor and formally review on
an ongoing basis all subcontractor’s performance
of any delegated activities.

438.240: Quality assessment and
performance improvement program
(a) General rules
(b) Basic elements of MCO and PIHP
quality assessment and performance
improvement programs
(d) Performance improvement
projects

Obtain from the State Medicaid agency:
1) information on whether or not the State
Medicaid agency has required the MCO’s/PIHP’s
performance improvement projects to address a
specific topic or topics, or address a specific
topic(s) and also use specific quality indicators
identified by the State Medicaid agency;
2) the State’s requirements with respect to
MCO/PIHP reporting of the status and results of
each performance improvement project to the State
Medicaid agency; and
3) any reports on the status and results of the
performance improvement projects submitted by
the MCO/PIHP in response to State requirements
for reporting the status and results of each
performance improvement project to the State
Medicaid agency.

438.240(c): Performance
measurement and improvement

Obtain from the State Medicaid agency:
1) a list of all performance measures required of
the MCO/PIHP by the State for the year or years
for which the review is being conducted;
2) the actual performance measures submitted by
the MCO/PIHP to the State for the year or years for
which the review is being conducted; and
3) instructions from the State on whether or not the
State wishes the EQRO to validate the
MCO’s/PIHP’s submitted performance measures.

438.240(e): Program review by the
State.

Determine from the State Medicaid agency whether
or not the State has required the MCO/PIHP to
have in effect a process for its own evaluation of
the impact and effectiveness of its quality
assessment and performance improvement (QAPI)
program and, if so, how frequently the MCO/PIHP
is to make such an evaluation.

438.242: Health information systems

1) Information on whether or not the State has
required the MCO/PIHP to undergo, or has
otherwise received, a recent assessment of the
MCO’s/PIHP’s health information system (IS). If
the State has required or received such an
assessment, obtain a copy of the IS Assessment
from the State or the MCO/PIHP. Also obtain
contact information about the person or entity that
conducted the assessment and to whom follow-up
questions may be addressed.
2) State specifications for data on enrollee and
provider characteristics that must be collected by
the MCO/PIHP.
3) State specifications for how MCOs/PIHPs are to
collect data on services furnished to enrollees; i.e.,
whether or not the MCO/PIHP must collect
encounter data or may use other methods. If the
State allows the MCO/PIHP to use other methods,
what are the State’s requirements with respect to
these “other methods?” If the State requires
MCOs/PIHPs to collect encounter data and report it
to the State, does the State validate this data or
require it to be validated? If the data is validated,
obtain a copy of the most recent validation report.

Subpart F--Grievance System
438.402: General requirements

Obtain from the State information on:
1) the time frame during which enrollees and
providers are allowed to file an appeal;
2) whether or not the State requires enrollees to
exhaust MCO/PIHP level appeals prior to
requesting a State fair hearing; and
3) whether enrollees are required or permitted to
file a grievance with either the State or the
MCO/PIHP or both.

438.404: Notice of Action

Obtain from the State Medicaid Agency
information on the time frames within which it
requires MCOs/PIHPs to make standard (initial)
coverage and authorization decisions and provide
written notice to requesting enrollees. These time
frames will be the required time-frames within
which MCOs/PIHPs must provide Medicaid
enrollees with written notice of any intent to deny
or limit a service (for which previous
authorization has not been given by the
MCO/PIHP) and the enrollee’s right to file a MCO
or PIHP appeal (or request a State fair hearing if
the State does not require the enrollee to exhaust
MCO/PIHP level appeals prior to requesting a
State fair hearing).

438.408: Resolution and notification:
Grievances and appeals

Obtain from the State Medicaid Agency:
1) the State-established standard time frames
during which the State requires MCOs/PIHPs to:
- dispose of a grievance and notify the affected
parties of the result;
- resolve appeals and notify affected parties of the
decision; and
- expedite and resolve appeals and notify affected
parties of the decision.
2) the methods prescribed by the State that the
MCO/PIHP must follow to notify an enrollee of the
disposition of a grievance.
3) information on whether or not the State requires
Medicaid enrollees to exhaust MCO or PIHP level
appeals before receiving a State fair hearing.

438.414: Information about the
grievance system to providers and
subcontractors

Obtain from the State Medicaid Agency
information on:
1) whether the State develops or approves the
MCO’s/PIHP’s description of its grievance system
that the MCO/PIHP is required to provide to all
Medicaid enrollees. [Note that under regulations at
§438.10(g)(1) the State must either develop a
description for use by the MCO/PIHP or approve a
description developed by the MCO/PIHP.]
2) if the States approves, rather than develops, the
description of the MCO’s/PIHP’s grievance
system, information on whether or not the State has
already approved the MCO’s/PIHP’s description.
3) the State-specified time frames for disposition of
grievances.

438.420: Continuation of benefits
while the MCO or PIHP appeal and
the State Fair Hearing are pending

Obtain from the State Medicaid Agency
information on any time limits specified by the
State that must be met by Medicaid enrollees who
wish to file an appeal, request for expedited appeal
or State fair hearing.

438.424: Effectuation of reversed
appeal resolutions

Obtain from the State Medicaid Agency
information on whether the State or the MCO/PIHP
is required to pay for services in situation in which
the MCO or PIHP, or the State fair hearing officer
reversed a decision to deny authorization of
services, and the enrollee received the disputed
services while the appeal was pending.

ACTIVITY 3: Document review
Typically, an onsite visit begins with document review. Prior to the onsite visit, the MCO/PIHP
should receive from the entity conducting the compliance monitoring a list of documents needed
for review. This should be accompanied by instructions on how to organize and prepare the
documents for the reviewers. These instructions should request that documents remain available
to reviewers for the duration of the onsite visit. Reviewers should request the MCO/PIHP to
provide an orientation to the organization of their documents. Also prior to the onsite visit,
reviewers should request reports on previous reviews and subsequent MCO/PIHP corrective

actions in order to identify areas on which the reviewers might need to focus the current
monitoring.
Prior to conducting document review, reviewers should re-familiarize themselves with the
standards for which they will be evaluating compliance and the document list to determine the
items of information that they should be seeking in the MCO/PIHP documentation. In instances
where multiple reviewers are conducting the onsite visit, the team may want to identify in
advance what issues each reviewer will take responsibility for evaluating in documents and
pursuing during interviews. Pre-planning for this activity will be key to accomplishing the task.
During document review, reviewers begin the assessment of compliance with regulatory
provisions, and identify issues that will be pursued during interviews. MCO/PIHP staff do not
need to be present during this onsite activity, but should be available if reviewers have questions
or difficulty locating a particular document or item of information.
During the review of documentation, reviewers should:
- Take notes that will assist in making determinations about compliance with the
regulatory provisions and support the completion of a Documentation and
Reporting Tool such as that provided in Attachment C;
- Identify topics or issues that need clarification or follow-up during interviews;
- Identify items of information that were not available or located in documents to
provide the MCO/PIHP an opportunity to respond; and
- Identify specific document content for discussion at an interview to provide the
MCO/PIHP an opportunity to prepare participants with copies or to identify
additional participants that may be necessary for the discussion.
A list of the documents needed to evaluate compliance with the Medicaid regulatory provisions
follows. Documents are identified using generic names, except in instances where the regulatory
provisions refer to and require a specific document be present and reviewed for content. The
documents in the list are grouped by subject matter, with the regulatory provision numbers noted
to aid in identifying where knowledge gained by reviewing the documents can be applied. This
does not imply that the document cannot be used as a data source for addressing other provision
issues, or that it should be the sole source of data in evaluating compliance with the provisions
noted. Reviewers should use Attachment B in conjunction with this list to identify specific
information that needs to be located in these documents.

Document

Applicable Regulatory Provision
(Full regulatory citations and provisions are
located in Attachment B)

Administration/Managerial
Service planning documents and provider
network planning documents (e.g.,
geographic assessments, provider network
assessments, enrollee demographic studies,
population needs assessments)

438.206(b)(1) through (5)
438.206(c)
438.214

Service availability and accessibility
expectations and standards

438.206(b)(1) and (2)
438.206 (c)

Other performance standards and quality
indicators established by the MCO/PIHP

438.206(b) and (c)

Any measurement or analysis reports on
service availability and accessibility

438.206(b) and (c)

List of all care and service providers in the
MCO’s/PIHP’s network (may be the same as
the provider directory)

438.206(b)(1) and (2)

Organization strategic plans

438.206(b)(1)
438.206(c)

Administrative policies and procedures

438.206(b)

Procedures and methodology for oversight,
monitoring, and review of delegated activities

438.214
438.230

Contracts or written agreements with
organizational subcontractors

438.210
438.214
438.230
438.414

Completed evaluations of entities conducted
before delegation is granted

438.210
438.214
438.230
438.414

Document
Ongoing evaluations of entities performing
delegated activities

Applicable Regulatory Provision
(Full regulatory citations and provisions are
located in Attachment B)
438.230

QAPI program description

438.240(a) through (d)

QAPI project descriptions, including data
sources and data audit results

438.236(c)
438.240(b)(2) and (3)
438.242

QAPI project quality indicators, the selection
or development criteria, and processes for
selection or development

438.240(d)

QAPI program evaluation

438.240(e)

QAPI data analysis and reports

438.240

Performance measures produced by the
MCO/PIHP

438.240

Policies and procedures related to data
collection and data quality checks for QAPI
projects

438.240

Policies and procedures for researching,
selecting, adopting, reviewing, updating and
disseminating practice guidelines

438.236(a) through (c)

Practice guidelines adopted by the
MCO/PIHP

438.208
438.236(a) and (b)

Medicaid and other enrollee grievance and
appeals data

438.210
438.242

Medicaid and other enrollee survey results

438.10
438.100
438.206(c)

Utilization Management
Utilization management policies and
procedures

438.206(b)(2) through (5)
438.210
438.236
438.240

Coverage rules and payment policies

438.114
438.210

Data on claims denials

438.210
438.404

Service authorization policies and procedures

438.206(b)
438.210

Policies and procedures for notifying
providers and enrollees of denials of service

438.210

Analytic reports of service utilization

438.242

Policies and procedures related to practice
guidelines, including selection criteria,
involvement of practitioners in review and
adoption process, schedule for updating, etc.

438.236

Processes used to review practice guidelines
for consistency with other criteria and
services of the MCO/PIHP

438.236(c)

Information Systems
Information systems capability assessment
reports

438.242(a) and (b)

Policies and procedures for auditing data or
descriptions of other mechanisms used to
check the accuracy and completeness of data
(internally generated and externally generated
data)

438.242

Completed audits of data or other evidence of
data monitoring for accuracy and

438.240
438.242

completeness both for MCO data and
contractor (delegate) data
Provider/Contractor Services
Provider/Contractor Services policies and
procedures manuals

438.6
438.100(a)(2)
438.208
438.242

Provider contracts

438.6
438.100(a) and (b)
438.102
438.206(b) and (c)
438.208
438.242
438.414

Provider/Contractor procedure manuals

438.6
438.100(a)(2)
438.206(b)(2)
438.206(c)
438.208
438.236(c)
438.414

All other written forms of standard provider
communication

438.236(c)

Provider/Contractor oversight and evaluation
policies and procedures, audit tools

438.100(a)(2)
438.206(c)

Provider/Contractor files, 15-20 individual
health care professional files, and 15-20
institutional provider files

438.6
438.214

Provider/contractor orientation and education
curricula

438.100(a)(2)

Credentialing committee or other provider
review mechanism meeting minutes

438.214

Sample of files of practitioners who have not
been appointed or reappointed

438.214

Enrollee Services
Medicaid enrollee services policies and
procedures

431.51(a)(4) through (a)(6)
438.6
438.10(a) through (g)
438.100(b)(2)
438.106
438.108
438.114
438.206(b)(2) through (b)(5)
438.206(c)
438.208

Statement of enrollee rights

431.51(a)(4) through (a)(6)
438.6
438.10 (a) through (h)
438.100
438.114
438.206(b)(3) through (b)(5)

Medicaid enrollee marketing materials

431.51(a)(4) through (a)(6)
438.10(a) through (h)
438.100(b)(2)(i)
438.104

Medicaid marketing plans, policies and
procedures

438.10(a) through (h)
438.100(b)(2)(i)
438.104

Medicaid enrollment and disenrollment
policies and procedures

438.10(a) through (f)
438.56(b) through (e)i
438.100
438.208
438.226
434.242

Medicaid enrollment data
Medicaid enrollee handbooks

438.6
438.10(a) through (d)
438.10(f) through (h)
438.100
438.206(b)(2) through (5)
438.206(c)
438.208

Medicaid provider directory

438.10 (b) through (e)
438.10(f)(6)
438.206(b)

Medicaid enrollee orientation curriculum

438.10(b) through (d)
438.10(f)
438.10(g)
438.100
438.206(b)(2) through (5)

Enrollee health education planning and
program content

438.236(c)

Medicaid enrollee grievance and appeals
policies and procedures

438.10(f) and (g)
438.56
438.100
438.206(c)
438.228
438.402(a) and (b)
438.404
438.406(a) and (b)
438.408(a) through (e)
438.410(a) through (c)
438.420
438.424(a) and (b)

Medicaid enrollee grievance and appeal files
or records

438.100
438.402(a) and (b)
438.404
438.406(a) and (b)
438.408(a) through (e)
438.410(a) through (c)

Medicaid enrollee grievance and appeal
tracking reports

438.228
438.408(a) through (e)
438.410(a) through (c)

Care coordination policies and procedures,
and enrollee records

438.208

Sample of Medicaid enrollee records

438.208

Staff Planning, Education, Development and Evaluation
Staff handbooks

438.10
438.100

Staff orientation and training curriculum

438.10
438.100

ACTIVITY 4: Conducting interviews
Purpose of Interviews
Data gathered from interviews should supplement and verify what is learned during document
review. Interview participants provide clarification and confirm that what is documented is what
is carried out in practice, thus supplying further evidence that the MCO/PIHP understands the
regulatory provisions and is able to demonstrate compliance in its performance.
Interview Planning
The following descriptions of interviews are intended to serve as interview templates,
recognizing that, in practice, interviews should be customized to the MCO/PIHP being
evaluated. This customization should be based on information that is known about the
MCO/PIHP from pre-onsite data gathering activities (characteristics, population demographics,
etc.) and from what reviewers learned from the document review. As a result, effective
interviewing requires advance planning. This planning is an informal, but important, process to
achieve the MCO/PIHP evaluation objectives. Planning should include:
-

-

-

Preparing a list of issues that need to be addressed in each interview, based on the
regulatory provisions (Attachment B), organization characteristics, and
information gathered during document review.
Reviewing the MCO’s/PIHP’s anticipated participants for the interview and
identifying topics that will engage as many individuals in the discussion as
possible.
When multiple reviewers are assigned to an MCO/PIHP, reviewers should
determine what role each will play during the interviews, for example,
interviewer or notetaker. Although pre-established as primary roles, this does not
preclude designated note-takers from sharing in the interviewing of participants or
preclude interviewers from taking notes.

It is recommended that document review not be performed during scheduled interview time. If
interview participants bring documents with them, they should be reviewed following the
interview or at another time. The exception would be in those instances in which the documents
are key to the discussion and where the review can be interactive and involve all participants.
Interview Participants
Interviews should be scheduled in coordination with the MCO/PIHP (See Activity 1: Evaluation
Planning) in order to gain the greatest participation of staff and other individuals. Interviews
should be conducted with groups, rather than with single individuals. Rarely is one individual
within the MCO/PIHP solely responsible for a particular function. Therefore, it is recommended
that at the interview several people represent the differing functions, services or departments of
the MCO/PIHP. Cross-functional/service/department interviews are also beneficial to the data

gathering process. What one area does not know, another may, and the interview then becomes a
learning opportunity for the MCO’s/PIHP’s staff as well.

The following group interviews should be conducted:
- MCO/PIHP leadership
- MCO/PIHP information system personnel
- Quality assessment and performance improvement program personnel
- Provider and contractor services staff
- Enrollee services staff
- Utilization management staff
- Medical Director
- Case managers and care coordinators
Additionally, MCO/PIHP providers could be interviewed if time and resources permit.
Conducting the Interview
During interviews, reviewers should:
- Begin with brief introductions of themselves and MCO/PIHP participants,
- Share with the participants the interview goals based on the planning outline,
- Maintain control of the interview discussion, politely redirecting participants
when necessary,
- Remain sensitive to the time (e.g., begin and end as close to the times noted on
the agenda),
- Be attentive to participants and listen carefully; summarize or restate participant
responses to ensure understanding,
- Take notes as needed for use in completing a Documentation and Reporting Tool
(such as that in Attachment C) or to be reviewed for other interviews or to
confirm document findings,
- Review documents that participants bring to the interview (at the end, or
interactively during),
- Conclude the interview with a review of the outlined goals so that all present may
determine if they have been met.
Interview content
Descriptions of each group interviews follows. Each description includes:
- the purpose for the interview;
- suggested interview participants (as described in Attachment B); and
- potential questions/probes related to the subject matter of the regulatory
provisions (as described in Attachment B) for use in prompting discussion.

Organization Leaders Interview
A. Description/Purpose/Recommended Participants
The leadership interview is an opportunity to talk with the senior representatives of the
MCO/PIHP about the following Medicaid managed care requirements.
-

Enrollee rights
Enrollee information
Availability of services (e.g., delivery network, provider credentialing, hours of
operation and emergency services)
Timely access to care
Enrollees with special health care needs
Cultural considerations
Continuity and coordination of care
Delegation
Practice guidelines
Quality assessment and performance improvement
Health information systems
Complaints and grievances

At a minimum, the following representatives of the MCO’s/PIHP’s leadership should be
present:
-

Chief executive officer (CEO);
Chief operating officer (COO), when applicable;
Chairman of the governing body, or a representative, when applicable;
Medical director;
Chief elected or appointed officer of the MCO/PIHP's licensed independent
practitioners (where found);
Quality improvement committee chairperson;
Quality improvement program director or coordinator; and
Human resources leader.

As determined by the CEO, other senior staff of the MCO/PIHP may also be in attendance.
However, attendance at this interview should be carefully limited in order to foster candor
and the free exchange of information.

B. Potential Questions to Ask
438.100 and 438.10--Enrollee right to information
1. What information is your MCO/PIHP required to disseminate to Medicaid enrollees?
How often is your MCO/PIHP required to make this information available?
2. How does your MCO/PIHP give each enrollee written notice of any change (that the
State defines as “significant”) in the information specified above, at least 30 days before
the intended effective date of the change? How does the State define “significant?” Have
you made any such “significant” changes in the last year?
3. How does your MCO/PIHP give written notice of termination of a contracted provider,
within 15 days after receipt or issuance of the termination notice, to each enrollee who
receives his or her primary care from, or is seen on a regular basis by, the terminated
provider? Have you terminated the contract of any providers in the last year?
4. How do you ensure that your staff and affiliated providers comply with Federal and State
laws that pertain to enrollee rights?
438.100--Enrollee right to respect, dignity, privacy
1. How does the MCO/PIHP ensure that its own facilities and those of its affiliated
providers comply with enrollee rights such as: treatment with respect, dignity and
consideration for privacy, confidentiality of information? Provide examples.
2. What processes are in place to ensure that staff observe the MCO’s/PIHP’s policies and
procedures on privacy and confidentiality of enrollee information?
3. What does the MCO/PIHP do to raise staff awareness of its policies on nondiscriminatory
behavior towards enrollees? How are staff monitored to determine that they comply with
these policies?
438.100 and 438.102--Enrollee right to receive information on available treatment options
1. Does your MCO/PIHP have any moral or religious objection to providing, reimbursing
for, or providing coverage of, a counseling or referral service for a particular Medicaid
service or services? If so, how do you inform potential enrollees and current enrollees of
this?
2. How does the MCO/PIHP ensure that providers share information on available treatment
options and alternatives with enrollees? Does this include alternatives and options that
are outside, as well as within, the Medicaid contract’s scope of benefits?

3. What steps does the MCO/PIHP take to ensure that enrollees receive information on
available treatment options and alternatives in a manner appropriate to their condition
and ability to understand?
438.100 and 438.6--Enrollee right to participate in decisions regarding his/her health care
and advance directives.
1. To what extent does the MCO/PIHP allow enrollees to participate in care and treatment
decisions? Describe.
2. Does the MCO have any limitations in implementing State and Federal laws pertaining to
advance directives? If so, what are these?
438.100--Compliance with other Federal and State laws.
1. What steps do MCO/PIHP leaders take to ensure compliance with Federal and State laws
on enrollee rights?
2. Has the MCO/PIHP ever been found non-compliant with any Federal and State laws on
enrollee rights? If yes, in what area, and what steps were taken to clear the violation?
3. If a provider/contractor is found to be in violation of any Federal and State laws on
enrollee rights, how does the MCO/PIHP respond?
4. To what extent does the MCO/PIHP orient new staff to Federal and State laws on
enrollee rights that must be observed during day-to-day operations? How does the
MCO/PIHP remind staff of the importance of observing these laws during interactions
with other employees and with enrollees?
5. Describe the steps taken by the MCO/PIHP when staff report, or are involved in a
violation of Federal or State laws on enrollee rights.
438.206--Availability of services.
1. Describe the MCO’s/PIHP’s process for assessing the needs for providers to deliver each
type of covered service and need for major specialties within each type. What issues
were considered in the assessment process?
2. How does the MCO/PIHP determine the adequacy of its network to serve its Medicaid
enrollees?
3. What assumptions and methodologies are used to project the number, type (in terms of
training, experience and specialization) and location of primary care providers and
specialists necessary to serve its anticipated Medicaid enrollees?
438.206(b)(3) through (5)--out-of-network providers.

1. Approximately what proportion of Medicaid enrollee provider encounters are made to
out-of-network providers? If this is a significant percent, what are the reasons for this?
2. How do you pay out-of-network providers? Do you receive claim or encounter data from
out-of- network providers similar to the claim or encounter data that you receive from
your network providers?
3. How does your MCO/PIHP ensure that any costs to the Medicaid enrollee for out-ofnetwork services is no greater than the costs the Medicaid enrollee would incur if they
used a network provider for the same service?
438.206(c)(1)--Furnishing of services and timely access.
1. Describe how the MCO/PIHP monitors for compliance with its Medicaid standards for
timely access to care and services.
2. How does the MCO/PIHP ensure the 24 hours per day, 7 day per week availability of
Medicaid services included in its contract with the State when medically necessary?
3. How does the MCO/PIHP determine that the individual and institutional providers it
contracts with have sufficient capacity to make services available when medically
appropriate 24 hour per day, 7 days per week to Medicaid enrollees?
4. How does the MCO/PIHP ensure that its provider network’s hours of operation do not
discriminate against Medicaid enrollees; i.e., are not different for Medicaid enrollees than
for commercial enrollees?
5. How big of a problem is inappropriate use of emergency rooms by your Medicaid
enrollees?
6. What was the volume of denied claims for emergency and post-stabilization services in
the most recent year?
438.206(c)(2)--Cultural considerations.
1. What have been the State Medicaid agency’s efforts to promote the delivery of services
in a culturally competent manner to all enrollees, including those with limited English
proficiency and diverse cultural and ethnic backgrounds? How has your MCO/PIHP
participated in these efforts? What documentation exists describing your efforts and the
results of your efforts?

438.208--Coordination and continuity of care.

1. How are “individuals with special health care needs” defined by the State Medicaid
agency? Has your MCO/PIHP developed any other operational definition or definitions
of individuals with special health care needs? If yes, what is/are these and how were they
developed? How do they differ from the State definition?
2. Does the State Medicaid agency require your MCO/PIHP to screen Medicaid enrollees to
identify those with special health care needs?
3. How are individuals with special health care needs - including both individuals with
special health care needs identified by this MCO/PIHP and those identified by the State
Medicaid agency or its agent - identified and tracked within your MCO/PIHP?
4. Does the State Medicaid agency require your MCO/PIHP to assess Medicaid enrollees
with special health care needs? If yes, how are these activities conducted?
5. Who may serve as enrollee primary care providers?
6. What steps does the MCO/PIHP take to promote Medicaid enrollees’ ongoing
relationship with a usual source of primary care?
7. What process(es) is (are) used to coordinate services for enrollees? Are there different
types of care coordination mechanisms for different types of enrollees? If so, what are
these?
8. If your MCO/PIHP establishes separate coordination of care for medical services and
mental health and substance abuse services, how does it ensure exchange of necessary
information between providers?
9. How does the MCO/PIHP ensure coordination of its services with services enrollees may
receive from other MCOs or PIHPs?
10. Under what circumstances may Medicaid enrollees have direct access to specialists?
11. How does your MCO/PIHP manage the provision or any specialty care services currently
not provided in-network?
12. Does your MCO/PIHP require written treatment plans to be developed for enrollees? If
yes, under what circumstances are written treatment plans required?
438.210 and 438.114--Coverage and authorization of services, including emergency and
post-stabilization services.

1. How big of a problem is inappropriate use of emergency rooms by your Medicaid
enrollees?
2. Has your MCO/PIHP investigated a potential relationship between inappropriate
emergency room use and enrollee access to routine and urgent care?
3. What was the volume of denied claims for emergency and post-stabilization services in
the most recent year?
438.214 and 438.12--Provider selection and non-discrimination.
1. What is the basis or criteria used to determine individual provider participation in the
MCO’s/PIHP’s network?
2. What is the basis or criteria used to determine institutional or other non-individual
practitioner participation in the MCO’s/PIHP’s network?
3. What types of providers are subject to the MCO’s/PIHP’s credentialing process?
4. Describe the provider credentialing process used by the MCO/PIHP.
5. What steps does the MCO/PIHP take to ensure that it does not employ or contract with
providers who have been excluded from participation in Federal health care programs?
6. What steps does the MCO/PIHP take to ensure that providers who serve high-risk or
costly populations are not discriminated against in the selection process, and when
considering reimbursement and indemnification?
7. What criteria are the basis for denial of provider participation in the MCO’s/PIHP’s
network?
438.228--Grievance systems.
1. How does your MCO/PIHP track requests for covered services that your MCO/PHP or its
providers has denied?
2. What was the volume of denied claims for services in the most recent year?
3. How do you ensure that Medicaid enrollees who were denied services were notified of
their right to a State fair hearing?
438.230--Subcontractual relationships and delegation.
1. Which of the following OR OTHER activities are performed by (and thereby delegated
to) contractors?

-

establishment or maintenance of your provider network,
provider credentialing,
initial assessments of enrollees,
care coordination,
service authorization,
providing information to enrollees,
grievance systems,
quality assessment and performance improvement,
performance measurement,
information systems management,
others

2. What steps does your MCO/PIHP take to determine that an entity to which functions will
be delegated is capable of performing the function? Describe any such evaluation
process that your MCO/PIHP has in place.
3. Describe the MCO’s/PIHP’s process for identifying and selecting contractors? How is it
determined that a contractor has the ability to perform the activities that are to be
delegated?
4. How does your MCO/PIHP determine who will monitor and review delegate activities?
How often does delegate evaluation occur? Who in your MCO/PIHP is presented with
evaluation results? Who in your MCO/PIHP takes action on evaluation results?
5. For each of the activities that have been delegated, describe the review methodology
adopted by your MCO/PIHP to oversee entities that have been delegated this activity
through contracts or agreements. Has the methodology been effective in determining
delegate compliance with your MCO’s/PIHP’s performance expectations?
438.236(b)--Practice guidelines: adoption
1. What organizational component is responsible for the adoption of practice guidelines
used by your MCO/PIHP?
2. How does your MCO/PIHP establish priorities for adoption of practice guidelines?
3. What guidelines has your MCO/PIHP adopted?
4. By what process were they adopted?
5. To what extent are your MCO’s/PIHP’s guidelines “evidence-based”?

6. How does your MCO/PIHP consider the enrolled Medicaid population’s health needs in
the adoption of practice guidelines?
7. How are affiliated providers consulted as guidelines are adopted and re-evaluated?
8. What mechanism(s) does your MCO/PIHP have for periodically evaluating and updating
the guidelines it has adopted?
438.236(c)--Practice guidelines: dissemination and application
1. How are practice guidelines disseminated to providers?
2. When and how are guidelines disseminated to enrollees and potential enrollees?
3. To what extent are the practice guidelines adopted by your MCO/PIHP a component of
your MCO’s/PIHP’s QAPI program?
4. Is there an interface between the QAPI program and practice guidelines adoption
process?
5. What steps are taken to ensure that decision-making in the areas of utilization
management or coverage determinations and other functional areas are consistent with
the adopted practice guidelines?
438.240--Quality assessment and performance improvement program.
1. How does your MCO/PIHP define enrollees with “special health care needs.?” How are
these enrollees identified/ tracked within your MCO/PIHP?
2. How does your MCO/PIHP assess the quality and appropriateness of care furnished to
enrollees with special health care needs?
438.240(e)--Quality assessment and performance improvement program: Program review
by the State.
1. Does the MCO/PIHP evaluate the effectiveness of its quality assessment and
performance improvement program? How often?
2. What were the findings of the MCO’s/PIHP’s most recent self-evaluation of its QAPI?
What action did the MCO/PIHP take as a result of these findings?
438.242--Health information systems.
1. Describe the types of data collection systems that are in place to support the clinical and
administrative operations of your MCO/PIHP. Specifically, what data is routinely
collected to support utilization management, grievance systems, and enrollment services?

2. What processes are in place to obtain data from all components of your network (e.g.,
health care facilities, physician, laboratories)? To what extent does your MCO/PIHP
require and receive data in standardized formats? Are there any components of your
network from which you do not receive standardized (or any) information on services to
enrollees?
3. How are enrollee and provider data collected and integrated across all components of
your MCO’s/PIHP’s network? How is this used to produce comprehensive information
on enrollee needs and utilization and to otherwise support management?
438.402--Grievance system: general requirements.
1. Who in the MCO/PIHP is responsible for the development and oversight of the appeals
and grievance resolution process and access to State fair hearings?
2. What has been the volume of appeals/grievances/requests for State fair hearings in the
past year and the most common areas of concern expressed by Medicaid enrollees? How
has the MCO/PIHP addressed these concerns?
3. Describe the notice and appeals process for adverse actions on enrollee requests for
services or payment. Elaborate on the particular steps, including time frames.
438.406--Handling of grievances and appeals.
1. To what extent does your MCO/PIHP provide Medicaid enrollees with assistance in
completing forms and taking other procedural steps in the grievance and appeal process?
How does it do this?
2. How does your MCO/PIHP treat oral requests by Medicaid enrollees to appeal actions?
3. As part of an appeal, to what extent do enrollees and their representatives have an
opportunity to
- present evidence, and
- examine the enrollee’s case file, including medical records, and any other documents
and records considered during the appeals process.
4. What are the qualifications and credentials of individuals who make decisions on
grievances and appeals? How does the MCO/PIHP ensure that these individuals-- were not involved in any previous level of review or decision-making; and
- if deciding any of the following, have the appropriate clinical expertise in treating
the enrollee’s condition or disease.
(A) An appeal of a denial that is based on lack of medical necessity.
(B) A grievance regarding denial of expedited resolution of an appeal.

(C) A grievance or appeal that involves clinical issues?
5. Is there a process in place to monitor either the appeal and grievance process or the areas
of concern identified by enrollee appeals and grievances?
438.408--Resolution and notification: Grievances and appeals.
1. Approximately how many grievances did the MCO/PIHP receive in the most recent
reporting year?
2. Approximately how many appeals did the MCO/PIHP receive in the most recent
reporting year?
3. Approximately what percent of notices of action on requests for service authorization or
payment by Medicaid enrollees are appealed to the MCO/PIHP?
4. Approximately what percent of notices of action on requests for service authorization or
payment by Medicaid enrollees are appealed to the State fair hearing process?
Approximately what percent of these are overturned by the State?
438.410--Expedited resolution of appeals.
1. Is there a process in place for those instances when an enrollee’s health condition
requires expedited resolution of an appeal? Describe this process. What are the time
frames for this process?
2. Are physicians allowed to request expedited appeals on behalf of an enrollee? How does
the MCO/PIHP protect physicians who make such requests?
438.414--Information about the grievance system to providers and subcontractors.
1. Who in your MCO/PIHP has responsibility for the proper functioning of the grievance
process and the authority to require corrective action?
2. Did your State Medicaid agency develop or approve the description of your
MCO’s/PIHP’s grievance system that your MCO/PIHP provides to Medicaid providers?
Which, develop or approve? If it approved your description, how is the State’s approval
documented?
438.416--Recordkeeping and reporting requirements: Grievances and appeals.
1. Where in your MCO/PIHP are records on Medicaid enrollee grievances and appeals
kept?

Health Information Systems Staff Interview

A. Description and Purpose
This interview will assess the MCO’s/PIHP’s information management function and how it
relates to and supports the other functions of the organization, such as planning and
operations, quality assessment and improvement program activities, care coordination, etc.
The interview should include MCO/PIHP staff responsible for health information systems
issues at the MCO/PIHP. It should not be limited to just those responsible for technology
implementation, but should include staff who are responsible for the quality of information,
movement of information, sharing of information, and information policy and procedure
development and implementation.
Interviews with the Chief Information Officer (CIO) and selected health information systems
staff of the MCO/PIHP will address the following:
1. validation and clarification of an Information Systems Capability Assessment (see
Appendix Z) either completed within the past two years or newly completed as a part
of this compliance review;
2. the MCO’s/PIHP's assessment of its information needs;
3. the extent to which the health information systems process is planned and designed to
meet those needs;
4. how data are defined and captured across the MCO/PIHP;
5. how data transmission and integration take place across the MCO/PIHP;
6. how enrollee-specific data and information are made available when and where
needed by the MCO’s/PIHP’s provider network;
7. how the MCO/PIHP makes available and uses needed aggregate data and
information;
8. how the MCO/PIHP meets external reporting needs;
This is also an opportunity to explore the extent to which the health information needs of
the entire MCO/PIHP and provider network are measured, assessed, and, as indicated,
improved.

B. Potential Questions to Ask
A copy of an Information Systems Capability Assessment (ISCA) (such as that found in
Appendix Z) that either has been completed in the last two years by an independent organization
reviewing the MCO/PIHP or has been completed by the organization conducting this compliance
review, should be obtained. During this interview, validate the information provided about the
MCO/PIHP on the ISCA, explore any areas of concern, and gather missing or additional
information for use in evaluating standards compliance, paying particular attention to how data
are defined and captured across the MCO/PIHP and how data transmission and integration takes
place across the MCO/PIHP. The findings of the ISCA will serve as a guide to conducting this
interview, whether or not the entire Appendix Z protocol is being followed. Questions to be
asked are dependent upon the findings of the ISCA. However, some common question to be
asked include:
1. What are the findings of the most recent assessment of the MCO’s/PIHP’s information
systems capacity? What are your IS’ strengths and weaknesses?
2. What has the MCO/PIHP done to address IS problem areas?
3. What information needs does your MCO/PIHP have that are not currently met by your
present IS? What has the MCO/PIHP done to address these needs?
4. Is the data collected from network providers on services to enrollees subject to accuracy
and timeliness checks?
5. Describe procedures used to screen all data, both internal and external, for completeness,
logic and consistency.
6. How is enrollee-specific data and information made available when and where needed by
the MCO’s/PIHP’s provider network?
Additional questions should be asked to validate and gain greater understanding of the
MCO’s/PIHP’s information system and how it is used in the support of regulatory requirement.
Potential questions to be asked of the Chief Information officer and Information Systems staff
include:
438.206 Delivery network
1. How does your IS track services provided by and/or reimbursed to out-of-network
providers?

2. Is there any routinely collected and available data on use of out-of-network providers
(excluding Point of Service-related use)? Is data on use of out-of-network providers
separately available for Medicaid enrollees?
438.242 Health information Systems
1. Is the data collected from network providers on services to enrollees subjected to
accuracy and timeliness checks?
2. Describe procedures used to screen all data, both internal and external, for completeness,
logic and consistency.

Quality Assessment and Performance Improvement Program Interview
A. Description and Purpose
This interview with quality improvement program leaders and staff provides an opportunity
to gain a more thorough understanding of the approaches and processes used by the
MCO/PIHP to assess and improve quality.
B. Potential Questions to Ask
438.206(c) Furnishing of services-timely access
1. Have any recent QAPI activities been implemented to monitor the MCO’s/PIHP’s
compliance with its established standards for timeliness of access to care and member
services?
2. What are the results of these QAPI activities?
438.214 Provider selection
1. What type of information is generated through the quality improvement program to
support recredentialing of individual practitioner providers?
2. What types of information does the quality improvement program provide to support the
recredentialing of institutional and other non-practitioner providers?
438.236 Practice guidelines
1. How does the QAPI program interface with the administrative function responsible for
adopting practice guidelines?

438.240 Quality assessment and performance improvement program
1. Does the State require your MCO/PIHP to address a specific topic or topics and/or
indicators in your performance improvement projects? If yes, what types of projects are
required?
2. How does your MCO/PIHP detect over- and under-utilization? Provide examples of how
your quality assessment and improvement program has monitored to detect under- and
over-utilization
3. How does your MCO/PIHP define enrollees with “special health care needs”? How are
these enrollees identified/ tracked within your MCO/PIHP?
4. How does your MCO/PIHP assess the quality and appropriateness of care furnished to
enrollees with special health care needs? Provide examples.
5. Does the MCO/PIHP evaluate the effectiveness of its quality assessment and
performance improvement program? How often?
6. Describe the evaluation process. What aspects of the program are encompassed in the
evaluation?
7. What were the findings of the MCO’s/PIHP’s most recent self-evaluation? What action
did the MCO/PIHP take as a result of these findings?
8. Is the evaluation conducted to meet the State’s requirements, and if so, what is reported
to the State and how often?
438.242 Health information systems
1. How are enrollee and provider data from all components of your MCO’s/PIHP’s network
used in your MCO’s/PIHP’s quality assessment and performance improvement program?
Are there any components in your network for which you do not have adequate enrollee
utilization and provider data?
438.406 Handling of grievances and appeals
1. Is there a process in place to monitor either the appeal and grievance process or the areas
of concern identified by enrollee appeals and grievances?
438.416 Recordkeeping and reporting requirements on grievances and appeals
1. To what extent is information on Medicaid enrollee grievances and appeals analyzed and
included as part of your MCO’s/PIHP’s Quality Assessment and Performance
Improvement Program?

Provider/Contractor Services Staff Interview
A. Description and Purpose
This is an interview of MCO/PIHP staff who are responsible for establishing and maintaining
communications with the MCO’s/PIHP’s individual practitioners and other types of health
care providers (e.g., organizations). This includes staff responsible for management of the
credentialing process and oversight of delegated activities. Through these interviews, assess
the following:
1. The extent to which providers adequately address enrollee rights;
2. The manner in which the credentialing and appointment process is implemented;
3. Oversight of activities delegated to providers; and
4. Processes for providing information to providers.
B. Potential Questions to Ask
438.100 Enrollee rights
1. How does the MCO/PIHP inform its individual and institutional providers about enrollee
rights and responsibilities? How does the MCO/PIHP monitor for compliance with these
rights by its providers?
2. To what extent, if any, does the MCO/PIHP supply providers with information on where
to refer enrollees who are having difficulty understanding the materials that have been
provided to them by the MCO/PIHP?
3. Does the MCO/PIHP require providers to have access to oral interpreter services? Does
the MCO/PIHP supply providers with guidance or assistance in accessing oral interpreter
services if necessary?
4. How does the MCO/PIHP ensure that its own facilities and those of its affiliated
providers comply with enrollee rights to treatment with respect, dignity and consideration
for privacy? Provide examples.
5. How does the MCO/PIHP ensure that enrollees are not discriminated against in its own
facilities and those of its affiliated providers when seeking health care services consistent
with their covered benefits?

6. Describe the MCO’s/PIHP’s credentialing and oversight process for primary care
providers, other health care professionals and institutional providers. What is
encompassed by reviews and evaluations of these providers? Do these processes involve
visits to the providers’ care delivery sites?
7. Are providers encouraged to share information on available treatment options and
alternatives with enrollees? If so, what methods are used?
8. What processes are in place for monitoring providers to determine that they are providing
information on available treatment options and alternatives?
9. What requirements does the MCO/PIHP have for providers/contractors relative to
enrollee advance directives? How is it determined that providers/contractors are meeting
the MCO’s/PIHP’s requirements?
10. How does the MCO/PIHP inform its individual and institutional providers about enrollee
rights to service availability, coordination and continuity of care, coverage and
authorization of service, and to obtain a second opinion from an appropriately qualified
health professional? How does the MCO/PIHP monitor for compliance with these rights
by its providers?
11. How are the MCO’s/PIHP’s network providers informed of enrollees’ right to request
and receive a copy of their medical records, and to request that they be amended or
corrected?
12. What steps does the MCO/PIHP take to ensure that providers/contractors are aware of
and in compliance with applicable Federal and State laws on enrollee rights?
13. If a provider/contractor is found in violation of any Federal and State laws on enrollee
rights, what action is taken by the MCO/PIHP?
438.206 Availability of services
1. Describe the MCO/PIHP credentialing and recredentialing process. Is this different for
Medicaid providers?
2. How is it determined that providers are geographically accessible to Medicaid enrollees
and physically accessible to enrollees with disabilities?
3. Describe the processes for monitoring the provider network to determine that Medicaid
requirements pertaining to timeliness, availability and accessibility are being met. What
are the most recent findings from this process?

4. How often in the last year has your MCO/PIHP had to arrange for services or
reimbursements to out-of -network providers?

438.206(c) Timely access to service
1. Are MCO/ PIHP and provider services available 24 hours a day, 7 days a week, when
medically necessary?
2. Are the hours of operation of the provider network serving Medicaid enrollees different
from the hours of operation of the provider network serving other enrollees? If so, why?
3. Does the MCO/PIHP continuously monitor its provider network for compliance with
established standards on timeliness of access to all care and member services? If yes,
how, and what are the most recent findings?
4. Is corrective action initiated when providers are not in compliance with established
standards for timeliness of access to care and member services? Are the corrective
actions assessed for effectiveness?
438.208 Coordination and continuity of care
1. How are primary care providers serving enrollees with special health care needs made
aware of and involved in procedures for:
- assessing these individuals?
- ensuring that their treatment plans address the needs identified by the assessment?
- treatment planning and determining and assuring appropriate use of specialists?
- coordinating their care with care provided by other MCOs and PIHPs serving the
enrollee?
2. How are specialty providers serving enrollees with special health care needs made aware
of and involved in procedures for:
- assessing individuals with special health care needs?
- ensuring that treatment plans address the needs identified by the assessment?
- treatment planning?
- coordinating the care of individuals with special health care needs with the care
provided by other MCOs and PIHPs serving the enrollee?
438.210 Coverage and authorization of services
1. Do contracts/agreements with individuals or organizations performing utilization review
provide for any performance incentives? If yes, please describe the incentives. [Note to
reviewers: Look for any incentives for denying, limiting or discontinuing
authorization of services.]

2. Are network providers notified of the information ordinarily required to process an
authorization request?
3. Describe the process for notifying the requesting provider of any decision to deny, limit,
or discontinue authorization of services. What are the MCO’s/PIHP’s time frames for
notification?
438.214 Provider selection
1. What types of individual practitioners are subject to the MCO’s/PIHP’s credentialing
process?
2. Describe the MCO’s/PIHP’s credentialing processes for individual practitioners. How
often does this process take place? What items of credentials information are updated
during the process? Are site visits made to providers? When and how often? How is it
determined that a site visit will be made? Who is involved in the MCO’s/PIHP’s
credentialing activities?
3. Describe the MCO’s/PIHP’s recredentialing processes for individual practitioners. What
types of information are monitored and reviewed during the recredentialing process?
What other operations of the MCO/PIHP contribute information to be used in the
recredentialing process?
4. Describe the MCO’s /PIHP’s processes for selecting and monitoring institutional and
other non- practitioner network providers. What information is reviewed as a part of this
process? Are site visits made? When and how often?
5. Describe the MCO’s/PIHP’s credentialing and recredentialing processes for institutional
providers. How frequently is re-credentialing performed? What items of information are
typically reviewed during the evaluation and reevaluation process?
6. Are site visits a part of the process to credential and recredential institutional providers?
7. What other MCO/PIHP operations contribute to the evaluation of a network institutional
provider?
8. What criteria are the basis for denial of provider participation in the MCO’s/PIHP’s
network?

438.228 Grievance systems
1. Describe the process for notifying the requesting provider of any decision to deny, limit,
or discontinue authorization of services. What are the MCO’s/PIHP’s time frames for
notification?
438.230 Subcontractual relationships and delegation
1. What types of activities are performed by (and thereby delegated to) contractors?
2. Describe your MCO’s/PIHP’s process for identifying and selecting contractors? How is
it determined that a contractor has the ability to perform the activities that are being
delegated by the organization?
3. What steps does your MCO/PIHP take to determine that an entity to which functions will
be delegated is capable of performing the function? Describe any evaluation process that
your MCO/PIHP has in place.
4. For each of the activities that have been delegated:
- Is there any ongoing monitoring and review of entities performing delegated
activities? How this is accomplished? Is the process the same for all delegates at
all times? Are there any instances when your MCO/PIHP varies the monitoring
process or the timing of evaluation?
-

Does your MCO/PIHP perform an annual evaluation of the delegate’s
performance? Describe the process undertaken to conduct this evaluation. What
is included in the evaluation?

-

What is done with the results of delegate evaluations? Do the results of the most
recent delegate evaluations specify any necessary corrective action for problems
or deficiencies identified? Describe some of the recommendations made to
delegates in an effort to improve performance.

-

What steps does your MCO/PIHP take to assure that the delegate implements
corrective actions?

-

Who in the MCO/PIHP is assigned responsibility for monitoring the delegate’s
performance?

438.236 Practice guidelines
1. What mechanism is in place to consult affiliated providers as practice guidelines are
adopted and re-evaluated?
2. How are practice guidelines disseminated to providers?

438.242 Health information systems
1. Does the MCO/PIHP have data collection requirements for health care facilities and
physicians? How are the requirements relayed to these organizations and individuals?
2. If issues arise in the timeliness and accuracy of the data that is being collected and
submitted, who notifies the health care facility or physician?
438.414 Information about the grievance system to providers and subcontractors
1. When are providers given information about the MCO’s/PIHP’s Medicaid complaint and
grievance system? What is typically included in the information given to providers
relative to Medicaid grievances?

Enrollee Services Staff Interview
A. Description and Purpose
The enrollee services staff interview provides an opportunity to talk with MCO/PIHP staff
who are responsible for communicating with enrollees. This includes those individuals
responsible for: written communication, phone responses to inquiries and problems, the
complaint and grievance system and other services designed to assist Medicaid enrollees in
their use of MCO/PIHP services. Through this interview, assess the following:
1. The manner in which the MCO/PIHP and its provider network address issues relating to
the rights of enrollees;
2. The MCO’s/PIHP’s efforts regarding enrollee education and communication;
3. The mechanisms in place to insure that information needed to provide services to
enrollees is available throughout the MCO/PIHP; and
4. What aspects of enrollee services are measured, how collected data is assessed, and what
efforts have been made to improve enrollee services.
B. Potential Questions to Ask
438.100 and 438.10 Enrollee right to information
1. What information is routinely provided to Medicaid enrollees? What is the process for
disseminating information to new and existing enrollees? How often is information
distributed to existing enrollees? In what format is this information presented?

2. Describe or provide copies of the formats in which information is presented to enrollees.
3. In what languages or alternative formats are enrollee materials and information
presented? How was it determined that materials were needed in different languages?
4. Does the MCO/PIHP need written materials in alternative formats for the visually
impaired? How did the MCO/PIHP determine this?
5. Describe the procedure for handling calls to the MCO/PIHP from non-English speaking
enrollees. What instruction or guidance is available for providers that may need
interpretation assistance to provide care and services to assigned enrollees?
6. To what extent is the MCO/PIHP responsible for responding to requests for information
for potential Medicaid enrollees?
7. How does the MCO/PIHP inform enrollees (and potential enrollees, if applicable) about
how to obtain oral interpreter services if they have limited proficiency in English?
9. Are there any benefits that an enrollee is entitled to under the Medicaid program, but that
are not made available through the MCO contract? What are those benefits? How are
enrollees made aware of the Medicaid program benefits that are outside the scope of
services available through the MCO?
10. How does the MCO/PIHP ascertain the primary language spoken by individual Medicaid
enrollees?
11. Are enrollees provided with a listing of primary care providers? Does the listing include
providers’ non-English language capabilities?
12. Does your MCO/PIHP give written notice of termination of a contracted provider to
enrollees who receive primary care from, or are seen on a regular basis by, the terminated
providers? How is this accomplished? Have you had to make any such notifications in
the last year?
13. Does your MCO/PIHP give enrollees any notice of significant changes change in the
information specified above? When and how does this occur? Have you had to make any
such notifications in the last year?

438.100 Enrollee right to respect, . . . dignity, and . . . privacy

1. How does the MCO/PIHP ensure that its own facilities and those of its affiliated
providers comply with enrollee rights to treatment with respect, dignity and consideration
for privacy? Provide examples.
438.100 Enrollee right to participate in decisions regarding his or her health care and
438.10(g) regarding advance directives
1. To what extent does the MCO/PIHP allow enrollees to participate in care and treatment
decisions? Describe some of the ways in which this is accomplished.
2. To what extent are Medicaid enrollees informed at the time of enrollment of their right to
accept or refuse treatment and to execute an advance directive and the MCO’s/PIHP’s
policies on implementation of that right?
438.100 Enrollee right to service availability, coordination and continuity of care, coverage
and authorization of service, and to obtain a second opinion from an appropriately
qualified health professional
1. How does the MCO/PIHP monitor for compliance with enrollee rights to service
availability, coordination and continuity of care, coverage and authorization of service,
and to obtain a second opinion from an appropriately qualified health professional?
What are the most recent results of this monitoring?
438.100 Enrollee right to request and receive medical records
1. How do enrollees obtain access to their medical records maintained by the MCO/PIHP,
including records maintained by providers/contractors from whom the enrollee has
received services?
2. How are enrollees informed of their right to request and receive a copy of their medical
records, and to request that they be amended or corrected?
3. Has the MCO/PIHP received any complaints about an enrollee’s inability to timely
access their medical records? If yes, what was the volume and nature of the complaints?
How were they resolved?
438.100 Compliance with other Federal and State laws
1. Does the MCO/PIHP orient staff to the Federal and State laws on enrollee rights that
must be observed during day-to-day operations? Does the MCO/PIHP remind staff of the
importance of observing these laws during interactions with other employees and with
enrollees?
2. Describe the procedure for handling an enrollee complaint involving a perceived
violation of their rights.

438.206 Availability of services
1. What processes does the MCO/PIHP take to monitor availability and accessibility of
services to Medicaid enrollees? What are the most recent findings from this process?
2. Is there any information that is routinely collected and monitored to determine that care
and services are being rendered to Medicaid enrollees in a timely manner? What are the
most recent findings of this monitoring?
438.206(b) Availability of services-Delivery network
1. Are Medicaid enrollee requests for out-of-network providers tracked? How often do
Medicaid enrollees request services from out-of-network providers? What are their
reasons for requesting out-of-network providers?
2. How often do Medicaid enrollees receive services from out-of-network providers?
438.206(c) Availability of services-Furnishing of services
1. Are MCO/ PIHP and provider services available 24 hours a day, 7 days a week, when
medically appropriate?
2. How frequently does enrollee services staff receive complaints about provider hours of
operations not being available to enrollees when medically necessary?
3. Does the MCO/PIHP conduct surveys, focus groups or other activities to receive the
feedback of Medicaid enrollees? If so, what are the most recent findings about Medicaid
enrollee perceptions about availability of MCO/PIHP and provider services?
438.208 Coordination and continuity of care
1. Does this MCO/PIHP screen Medicaid enrollees to identify those with special health care
needs? If yes, how is this implemented?
2. How are Medicaid enrollees with special health care needs - including both any
individuals with special health care needs identified by your MCO/PIHP and any
identified by the State Medicaid agency or its agent - identified and tracked within your
MCO/PIHP?
3. Does this MCO/PIHP assess Medicaid enrollees with special health care needs? If yes,
how are these activities conducted?
4. What proportion of Medicaid enrollees has a ongoing source of primary care?
5. What proportion of Medicaid enrollees has a person or entity formally designated as
primarily responsible for coordinating their health care services?

6. What proportion of Medicaid enrollees with special health care needs has a person or
entity formally designated as primarily responsible for coordinating their health care
services?
438.210 Coverage and authorization of services
1. How frequently does enrollee services staff receive complaints about difficulty obtaining
emergency or post-stabilization services?
2. Describe the procedure for handling member calls regarding need for emergency
services.
438.226 Enrollment and disenrollment
1. Describe the procedures that are followed when a request for disenrollment is received
from an enrollee.
2. Is disenrollment information tracked through or by other MCO/PIHP operations (e.g.,
grievance process, quality improvement, administration)? How many requests by
Medicaid enrollees were received last year for disenrollment? What were the cited
causes?
438.228 Grievance systems
1. Describe the process for notifying Medicaid enrollees of any decision to deny, limit, or
discontinue a request for service. What are the MCO’s/PIHP’s time frames for
notification?
438.236 Practice guidelines
1. How often does your MCO/PIHP receive requests from enrollees and potential enrollees
for practice guidelines? How does your MCO/PIHP respond to these requests?
2. When and how does your MCO/PIHP disseminate practice guidelines to enrollees?
438.402 Grievance system - general requirements
1. What enrollee materials contain information about the complaint and grievance
processes? When are enrollees presented with this information?
2. Describe the process for handling authorization decisions that are adverse to the enrollee.

438.406 Handling of grievances and appeals
1. What MCO/PIHP department or staff are responsible for assisting enrollees to use the
organization’s complaint or grievance system, including completing forms, or taking

other steps to resolve an appeal or grievance? What kind of assistance is made available
to Medicaid enrollees?
2. What are the qualifications and credentials of individuals who make decisions on
grievances and appeals? How does the MCO/PIHP ensure that these individuals-- were not involved in any previous level of review or decision-making; and
- if deciding any of the following, have the appropriate clinical expertise in treating
the enrollee’s condition or disease.
(A) An appeal of a denial that is based on lack of medical necessity.
(B) A grievance regarding denial of expedited resolution of an appeal.
(C) A grievance or appeal that involves clinical issues?
3. How does your MCO/PIHP treat oral requests by Medicaid enrollees to appeal actions?
4. As part of an appeal, to what extent do enrollees and their representatives have an
opportunity to:
- present evidence and
- examine the enrollee’s case file, including medical records, and any other documents
and records considered during the appeals process.
438.408 Resolution and notification: Grievances and appeals
1. Describe the MCO’s/PIHP’s grievance resolution process.
2. Describe the MCO’s/PIHP’s appeal resolution process. Are Medicaid enrollees required
to exhaust the MCO’s/PIHP’s internal appeals process before seeking and receiving a
State fair hearing?
3. How is it determined that an enrollee’s appeal requires expedited resolution?
4. What percent of appeal resolutions that are completely or partially adverse to Medicaid
enrollees are appealed to the State fair hearing process? Of these, what percent are
overturned by the State Medicaid agency?
438.410 Expedited resolution of appeals
1. Is there a process in place for those instances when an enrollee’s health condition
requires expedited resolution of an appeal? Describe this process. What are the time
frames defined for this process?
2. How does the MCO/PIHP notify enrollees of any denials of a request for expedited
resolution?

3. Have there been any complaints by Medicaid enrollees that their requests for expedited
appeals have not been acted upon timely; e.g., within three working days. If so, how
many such complaints were received in the year under review?
438.416 Recordkeeping and reporting requirements
1. How are Medicaid grievances and appeals registered and tracked for resolution? Is each
grievance and appeal tracked through to resolution?
2. How often is Medicaid grievance and appeal information analyzed for trends? Who
receives this analysis? Does the MCO/PIHP provide any information to the State relative
to its grievances and appeals?
3. How long are Medicaid grievance and appeal records retained?
4. To what extent is information on Medicaid enrollee grievances and appeals analyzed and
included as part of your MCO’s/PIHP’s Quality Assessment and Performance
Improvement Program?
438.420 Continuation of benefits while the MCO or PIHP appeal and the State Fair
hearing are pending
1. What happens to enrollee benefits once continuation of benefits has been denied by the
MCO/PIHP, and an appeal has been filed by the enrollee or the treating physician? Are
there any mechanisms in place to continue the benefits pending the outcome of the
appeal? If so, under what circumstances?

Utilization Management Personnel Interview
A. Description and Purpose
The utilization management interview provides an opportunity to assess the processes used to
ensure the following:
-

service authorization;
use of practice guidelines;
responsiveness to grievances and appeals; and
the management of resources across all MCO/PIHP network provider sites where
enrollees receive health care.

MCO/PIHP interview participants should include: the Medical Director, utilization
management directors or managers, utilization management review staff, case managers or

care coordinators, and any other individuals who have information pertinent to these
regulatory provisions.
[Note: This interview can be combined with the Medical Director interview or the Care
Coordinators and Case Managers interview.]
B. Potential Questions to Ask
438.206(b) Delivery network
1. What procedures must a Medicaid enrollee follow if he/she wishes to receive a second
opinion? For what types of services are second opinions available?
438.210 Coverage and authorization of services
1. What types of services require pre-authorization?
2. What are the MCO’s/PIHP’s time frames for processing standard and expedited requests
for service authorization?
3. How does the MCO/PIHP monitor its compliance with these time frames? What sources
of documentation exist to provide evidence of the monitoring by the MCO/PIHP?
4. How often and under what circumstances are requesting providers consulted when the
MCO/PIHP makes service authorization decisions?
5. To what extent does the MCO/PIHP assess the consistency of authorization decisions?
How does the MCO/PIHP do this?
6. What is the process when a decision is being made to deny authorization for a service?
Who makes the decision to deny a request to authorize a service?
7. Describe the process for notifying the requesting provider and the enrollee of any
decision
to deny, limit, or discontinue authorization of services. What information is typically
included in enrollee and provider notification? What are the MCO’s/PIHP’s time frames
for notification?
8. How big of a problem is inappropriate use of emergency rooms by your Medicaid
enrollees?
9. Has your MCO/PIHP investigated a potential relationship between inappropriate
emergency room use and enrollee access to routine and urgent care?

10. What was the volume of denied claims for emergency and post-stabilization services in
the most recent year?
438.228 Grievance systems
1. What types of services require pre-authorization?
2. Describe the process for notifying the requesting provider and the enrollee of any
decision
to deny, limit, or discontinue authorization of services. What information is typically
included in enrollee and provider notification? What are the MCO’s/PIHP’s time frames
for notification?
3. How does your MCO/PIHP track requests for covered services that your MCO/PHP or its
providers has denied?
4. What was the volume of denied request for services in the most recent year?
438.236(c) Application of [practice] guidelines
1. What practice guidelines has your MCO/PIHP adopted?
2. To what extent are your utilization management review guidelines (criteria) consistent
with these practice guidelines? How do you promote or ensure consistency?
3. Describe how utilization management review guidelines (criteria) are modified to reflect
the adoption or revision of practice guidelines. Are both sets of guidelines updated
through the same process, at the same time?
438.240 Quality assessment and performance improvement program
1. What information is analyzed to detect over- and under-utilization of services? Who is
involved in the analysis and review of this information? Have any trends been identified?
What are the typical follow-up actions taken when either condition is discovered?
438.402 Grievance system - General requirements
1. Describe the appeals process and the role of utilization management staff in the
resolution process. Elaborate on the particular steps, including time frames, in which
utilization management staff are involved.
2. Is there a process in place for those instances when an enrollee’s health condition
requires expedited resolution of an appeal? Describe this process and its time frames.
438.406 Handling of grievances and appeals

1. What MCO/PIHP department or staff are responsible for assisting enrollees in using the
MCO’s/PIHP’s appeal or grievance system, including completing forms, or taking other
steps to resolve an appeal or grievance?
2. What are the qualifications and credentials of individuals who make decisions on
grievances and appeals? How does the MCO/PIHP ensure that these individuals-- were not involved in any previous level of review or decision-making; and
- if deciding any of the following, have the appropriate clinical expertise in treating
the enrollee’s condition or disease.
(A) An appeal of a denial that is based on lack of medical necessity.
(B) A grievance regarding denial of expedited resolution of an appeal.
(C) A grievance or appeal that involves clinical issues?
438.410 Expedited resolution of appeals
1. Is there a process in place for those instances when an enrollee’s health condition
requires expedited resolution of a grievance? Describe this process. What are the time
frames defined for this process?
2. How does the MCO/PIHP notify enrollees of any denials of a request for expedited
resolution?
438.420 Continuation of benefits while the MCO or PIHP appeal and the State Fair
Hearing are pending
1. What happens to enrollee benefits once continuation of benefits has been denied by the
MCO/PIHP, and an appeal has been filed by the enrollee or the treating physician? Are
there any mechanisms in place to continue the benefits pending the outcome of the
appeal? If so, under what circumstances?

Medical Director Interview
A. Description and Purpose
The interview with the Medical Director provides an opportunity to assess MCO/PIHP
processes for authorizing services and coverage for those services. The interview will
address such topics as provider involvement in the review of criteria used in the utilization
management process, consistency between utilization management criteria and practice
guidelines, and QAPI efforts.
[Note: This interview can be combined with the Utilization Management interview or the
Care Coordinators and Case Managers interview.]

B. Potential Questions to Ask
438.210 Coverage and authorization of services
1. How does the MCO/PIHP monitor its compliance with the State’s time frames for
processing standard requests for service authorization?
2. What are the MCO’s/PIHP’s standards for processing expedited requests for service
authorization? How does the MCO/PIHP monitor its compliance with these time frames?
3. Under what circumstances is there consultation with requesting providers when
responding to service authorization requests?
4. How does the MCO/PIHP ensure consistent application of criteria used in making service
authorization decisions?
5. What mechanism does the MCO/PIHP use to assure that any decision to deny a service
authorization request or to authorize a service in an amount, duration or scope that is less
than requested, be made by a health care professional who has appropriate clinical
expertise in treating the enrollees’s condition or disease ?
6. How are employees and any contractors used by the MCO/PIHP to perform service
authorization and utilization management financially compensated? Are they paid in any
way other than on a straight salary or per case review basis? Do their financial
compensation arrangements involve the use of any financial incentives?
438.240 Quality assessment and performance improvement program
1. Does the MCO/PIHP have any processes for reviewing claims, payment systems,
encounter data and medical records to assess utilization of services? What reports on
service utilization are regularly produced by these processes? What are the most recent
findings with respect to over- and under-utilization?
2. How does your MCO/PIHP define enrollees with “special health care needs?” How are
these enrollees identified within your MCO/PIHP?
3. How does your MCO/PIHP assess the quality and appropriateness of care furnished to
enrollees with special health care needs? Provide examples.

Case Managers and Care Coordinators Interview
A. Description and Purpose

Case managers and care coordinators typically are among the few MCO/PIHP staff with
opportunity to interact closely and directly with Medicaid enrollees. These individuals are
often responsible for guiding enrollees to the care and services available through their
benefits and the provider network. These individuals play a key role in assisting enrollees in
managing and maintaining their health and managing complex conditions. Interviewing
these individuals will provide reviewers the opportunity to discuss topics surrounding
MCO/PIHP processes related to:
- Availability of services,
- Assessment of enrollees needs,
- Services available to women,
- Services offered to enrollees with special health care needs, and
- Continuity and coordination of enrollee health care.
[Note: This interview can be combined with the Medical Director interview or the
Utilization Management interview.]
B. Potential Questions to Ask
438.100(b)(iv) Enrollee right to participate in decisions regarding his or her health care
1. To what extent does the MCO/PIHP allow enrollees to participate in care and treatment
decisions? Describe some of the ways in which this is accomplished.
438.206(c) Furnishing of services and timely access
1. To what extent are services offered through the MCO/PIHP available to Medicaid
enrollees and others coordinating care 24 hours per day, 7 days per week when medically
necessary?
2. What types of services require pre-authorization?
438.208 Coordination and continuity of care
1. Does this MCO/PIHP screen Medicaid enrollees to identify those with special health care
needs? If yes, how is this implemented?
2. How are Medicaid enrollees with special health care needs - including both any
individuals with special health care needs identified by your MCO/PIHP and any
identified by the State Medicaid agency or its agent - identified and tracked within your
MCO/PIHP?
3. Does this MCO/PIHP assess Medicaid enrollees with special health care needs? If yes,
how are these activities conducted?

4. Does this MCO/PIHP require written treatment plans to be developed for enrollees with
ongoing special conditions that require a course of treatment or regular care monitoring?
If yes, how is it decided which Medicaid enrollees will receive a written treatment plan?
5. If treatment plans are required by this MCO/PIHP, how does the MCO/PIHP ensure that
treatment plans for individuals with special health care needs address the needs identified
by the assessment?
6. Describe the treatment planning process for individuals with special health care needs
and the process for determining and assuring appropriate use of specialists.
7. Within the last year, how many treatment plans have been developed? How many
requests for treatment plans have been denied? What were the reasons for these denials?
How many treatment plans have been denied?
8. What process(es) is/are used to coordinate services for enrollees? Are their different
types of care coordination mechanisms for different types of enrollees? If so, how are
these different and how do they work?
9. Who is responsible for coordinating the care of individuals with special health care
needs?
10. What are the procedures for coordinating the services that the MCO/PIHP furnishes to
the enrollee with services the Medicaid enrollee receives from any other MCOs and
PIHPs?
11. If the MCO/PIHP establishes separate coordination of care for medical services and
mental health and substance abuse services, how does the MCO/PIHP ensure exchange of
necessary information between providers?
438.210 Coverage and authorization
1. What types of services require pre-authorization?

Providers and Contractors Interview (Optional)
A. Description and Purpose
Interviewing providers and contractors requires additional time and resources. However, it is
an opportunity to obtain further information about MCO/PIHP performance from those
health care professionals and institutions that often serve as the first point of contact for
Medicaid members and health care providers. Because of this, provider and contractor

interviews should be considered as an optional component of this protocol -- to be considered
whenever there is a strong need for additional information and when time and resources
permit.
The provider network is the source of interview participants. Participants should be selected
to give, as much as possible, a representative view of the breadth of the MCO’s/PIHP’s
primary care, specialist, and institutional providers. Discussions with providers and
contractors can often clarify issues pertaining to:
- Communication,
- Movement of enrollees through the MCO/PIHP system,
- Observance of enrollee rights, and
- Information required to provide care and services to enrollees.
The interview can be arranged with a group of individual health care practitioners and with a
group of institution representatives. It can be coordinated as one interview for each group or
as a combined group. Geographic location of providers should be considered, and
conference calls are a viable option for conducting an interview of this type, and often
preferred by providers as only a brief interruption in their daily activities. In order for this
interview to be effective, reviewers are reminded to emphasize with providers that this is not
an evaluation of the care and services they offer to Medicaid enrollees, but rather their
opportunity to provide insight on the MCO’s/PIHP’s performance.
B. Potential Questions to Ask
438.100 Enrollee rights and 438.10 Enrollee information
1. When the MCO’s/PIHP’s enrollees present for services, do they appear to have a clear
understanding of their rights and responsibilities? Their benefits? How to obtain
services?
2. Does the MCO/PIHP provide you with information on where to refer enrollees who are
having difficulty understanding the materials that have been provided to them by the
MCO/PIHP?
3. How often do you and your staff have to assist enrollees with understanding the materials
provided by the MCO/PIHP?
4. Does the MCO/PIHP require providers to have access to oral interpreter services? Does
the MCO/PIHP provide your office with guidance or assistance is accessing interpreter
services if necessary?
438.100 Enrollee rights to receive information on available treatment options and 438.102
Provider-enrollee communications

1. Does the MCO/PIHP place any limits on your ability to counsel or advise a Medicaid
enrollee on treatment options that may be appropriate for the enrollee’s condition or
disease?
2. Does the MCO encourage providers to share with enrollees information on available
treatment options and alternatives? Does this include options and alternatives that are
within as well as those outside the scope of the enrollees benefits? If so, how does the
MCO/PIHP do this?
438.206(c) Availability of Services: Furnishing of services
1. Are your hours of operation for Medicaid enrollees different from the hours of operation
for other MCO/PIHP enrollees? If so, why?
438.236 Practice guidelines
1. Are affiliated providers/contractors consulted as practice guidelines are adopted and reevaluated?
2. How does the MCO/PIHP make providers/contractors aware of practice guidelines
currently in use and those under consideration for adoption?
438.410 Expedited resolution of appeals
1. Have there been any instances in the most recent year under review when the MCO/PIHP
took any punitive action against you for requesting an expedited resolution of an appeal
on behalf of Medicaid enrollees or for supporting an enrollee’s appeal?

ACTIVITY 5: Collecting accessory information
Sometimes information is needed that cannot be collected from interviews and review of
MCO/PIHP internal documents. In these situations, it is necessary to pursue information from
other sources. Such sources might include:
-

results of Medicaid beneficiary surveys
results of independent assessments of the MCO’s/PIHP’s information systems
results of independent assessments of MCO/PIHP encounter data, or
results of independent validations of MCO/PIHP performance measures or
improvement projects

Reviewers will take notes when they review any such information sources and identify any
information pertaining to MCO/PIHP compliance with the regulatory provisions. These findings
will be transferred to a documentation and reporting tool such as that found in Attachment C.

ACTIVITY 6: Analyzing and compiling findings
Completing the Documentation and Reporting Tool
Reviewers will be taking notes during document review, interview, and accessory activities.
These notes will need to be translated into compliance determinations for each regulatory
provision. This protocol includes a sample documentation and reporting tool in Attachment C
that can be used to convert reviewer notes into findings. Reviewers should use this or a similar
tool to document their findings. Once completed, the tool serves as a comprehensive record of
review activity.
The documentation and reporting tool in Attachment C is presented in numerical order by
Proposed Rule requirement. Regulatory provisions appear in the left column of the tool. The
provisions are divided into individual components, when possible, so that reviewers may identify
specific aspects of the regulation that an MCO/PIHP may only partially meet or not meet at all.
There are three columns on the right side of the tool for reviewers to check a designation of Met,
Partially Met or Not Met (however, an EQRO can use as many compliance categories as the
State determines appropriate-see below). These determinations will be based on reviewer notes
taken throughout the onsite evaluation.
When reviewers select a designation of Partially Met or Not Met for any of the regulatory
provisions, documentation should be supplied to explain the MCO’s/PIHP’s deficiency. The
documentation should be written in such a way that any audience viewing the reporting tool can
identify the specific deficiency. Documentation fields are not provided for each individual
component of the regulatory provision, but rather for a group of related components or the
provision in its entirety.
Reviewers can use the reporting tool as both a guide or a form for directly recording information
when performing the onsite evaluation. However, this may be difficult as the regulatory
provisions are not grouped by topic, documents, or type of MCO/PIHP operations.
Summarizing Findings
Analysis and presentation of findings from the evaluation of an MCO/PIHP will be highly
dependent on the audience. While non-summarized findings might be of interest to some
individuals, other parties might prefer a summary of the findings or information about
compliance with specific regulatory provisions.
One commonly used approach to summarizing an evaluation is to assign a numerical value to the
findings. A numerical scale can be used to indicate the degree of compliance with a given
regulatory provision. The documentation and reporting tool in Attachment C has been designed

to allow reporting of findings for (1) one, (2) multiple, or (3) larger combinations of regulatory
requirements. Each regulatory provision can be assigned a rating, which can be combined with
ratings of related regulatory provisions, which in turn can be combined with ratings of other
groups of regulations, and eventually into a single compliance rating for the MCO/PIHP. The
degree of “roll-up” or aggregation will be determined by the detail of reporting that the State
Medicaid agency expects relative to the regulatory provisions.
Rating scales can be established for all four levels described above. Each scale can be designed
to build upon the last level, or can be developed independently of the others. Some common
rating or scoring systems include:
Two point rating or scoring - Either the requirement is met or is not met in the evaluation.
1 - Met
2 - Not Met
Three point rating or scoring - This scale allows for credit when a requirement is partially
met, if this level of performance is acceptable.
1 - Fully Met
2 - Partially Met
3 - Not Met
Five point rating or scoring - One of five levels of compliance could be noted, allowing for
greater sensitivity to the potential levels of performance, especially in complex performance
areas.
1 - Fully Met
2 - Substantially Met
3 - Partially Met
4 - Minimally Met
5 - Not Met
While one rating or scoring scale may serve as the primary system, it does not necessarily
preclude the combined use of all the noted alternatives. While a five point scale may be
appropriate to use for most of the provisions, there may be instances when a provision or
component of a provision cannot be partially met.
The State Medicaid agency will need to identify how much compliance detail it requires, and
what rating or scoring system is to be used at each point of summary to designate a level of
compliance. The significance of the rating or scoring system must be clearly defined to
avoid any confusion about the meaning of summarized data.

Defining Compliance with Regulatory Provisions
Thresholds for compliance or non-compliance determinations with each regulatory provision
or component of a provision should be specified by the State Medicaid agency, based on its
expectations of MCO/PIHP performance. For example, the agency may decide that full
compliance is defined as any of the following:
-

all documentation listed under a regulatory provision, or component thereof, must
be present (that is, if only one item is not available, the MCO/PIHP is considered
“Not compliant”); and

-

MCO/PIHP staff are able to provide responses to reviewers that are consistent
with each other, and with the documentation; or

-

a percentage of data sources (i.e., documents and individuals interviewed able to
provide evidence of compliance with regulatory provisions) can be established
and used as the threshold for determining what constitutes “full compliance.”

Partial compliance could be determined as:
-

there is compliance with all documentation requirements, but staff are unable to
consistently articulate processes during interviews; or

-

Staff can describe and verify the existence of processes during interview, but
documentation is found incomplete or inconsistent with practice; or

-

Any combination of “Met,” “Partially Met” and “Not Met” findings at the
components of a regulatory provision would result in a “Partially Met”
designation for the provision as a whole.

No compliance could be defined as:
-

No documentation is present and staff have little to no knowledge of processes or
issues addressed by the regulatory provisions; or

-

For those provisions with multiple components: Key components of the provision
could be identified, and any findings of Not Met or Partially Met would result in
an overall provision finding of no compliance, regardless of the findings noted for
remaining components.

The State Medicaid agency can establish thresholds for performance at different levels of
regulatory provisions; e.g., individual regulations, a group of related regulatory provisions,
and for all regulatory provisions combined, depending on the intended use of the evaluation
results. Both Attachment B and the documentation and reporting tool (Attachment C)
supplied with this protocol currently separate the regulatory provisions into four major
sections: Enrollee Rights, Access, Measurement and Improvement and Grievance System.
State Medicaid agencies may want to break these major sections down even further to focus
in on specific aspects or components of the regulatory provisions to make performance
improvement more manageable and targeted. For example, an agency may determine that
MCOs/PIHPs must demonstrate the following levels of compliance, which look at two levels
of summary, the major section and one level below at each regulatory provision:
1. Under the Enrollee Rights heading: 5 of the 7 Regulatory Provisions (in their
entirety) must have findings of “Met” and the remainder must have findings of
“Partially Met.”
2. Under the Access heading: 10 of the 16 Regulatory Provisions (in their entirety)
must have findings of “Met” and the remainder must have findings of “Partially
Met.”
3. Under the Measurement and Improvement heading: All three (3) regulatory
provisions (in their entirety) must have findings of “Met.”
4. Under the Grievance System heading: 8 of the 10 regulatory provisions (in their
entirety) must have findings of “Met” and the remainder must have findings of
“Partially Met.”
The State Medicaid agency will need to consider all the uses for the MCO/PIHP evaluation
in establishing rules to guide the compliance scoring and determination. Pre-planning for the
use of results will be vital to implementation of an effective evaluation process.

Reporting Evaluation Results to the MCO/PIHP
Upon completion of a performance evaluation it is common practice to share the results with the
subject, in this case the MCO/PIHP. There are a number of alternative strategies for reporting
evaluation results to the MCO/PIHP. The State Medicaid agency will need to determine the
method for reporting results to the MCO/PIHP that best serves its purpose. Following are four
possible alternatives for reporting.
-

Alternative One - Reviewers do not share findings with the MCO/PIHP,
deferring to the report the MCO/PIHP will receive from the State following
review of the evaluation outcome. This may take the form of reporting
performance deficiencies with requests for MCO/PIHP corrective action.

-

Alternative Two - Reviewers report compliance issues (but not individual
regulatory provision compliance determinations) they have identified during the
course of the evaluation. There is an opportunity for the MCO/PIHP to offer
additional information, if such evidence of compliance is available. There is no
attempt to aggregate findings or indicate any MCO/PIHP overall “level of
performance” on the regulatory provisions. This verbal report would usually be
presented at the conclusion of the onsite evaluation during a closing session with
MCO/PIHP leadership.

-

Alternative Three - Reviewers verbally discuss the findings that resulted in
determinations of less than full compliance (This alternative would require that
the reviewers have knowledge of the thresholds determined to arrive at these
conclusions, as determined by the State Medicaid agency.) The MCO/PIHP
would be able to provide further evidence of compliance, assuming such is
available. No overall judgement regarding performance of the MCO/PIHP would
be offered. This discussion would occur in the same format as described in
alternative two.

-

Alternative Four - In addition to discussing specific compliance issues (as
above), the reviewers would indicate an overall compliance finding or rating,
based on aggregation of all the regulatory provisions. Reviewers would highlight
those areas of deficiency that will be presented to the State Medicaid agency. The
aggregation process would be based on the State’s established thresholds.
Presentation of findings would occur in the same format as described in
alternative two, but could also involve leaving a written report with the
MCO/PIHP.

ACTIVITY 7: Reporting evaluation results to the State Medicaid
Agency

Outlining State Medicaid Agency reporting needs is an important planning step and should be
factored into all decisions that are made relative to Activity 6, Analyzing and Compiling
Findings. State Medicaid agencies must carefully consider the types of information requests
they must be responsive to relative to the Medicaid managed care program. These requests can
be made by federal or state government agencies, the state legislature, local advocacy groups as
well as other interested parties. If the regulatory provision evaluation process is a source for this
information, it is important that a system of analyzing and compiling findings support these
requests.
Following are some of the possible alternatives for reporting compliance evaluation findings to
the State Medicaid agency. Reporting alternatives are heavily dependent on the decisions made
at Activity 6. That is, if findings are not summarized in some manner according to compliance
definitions, groupings of regulatory provisions, etc., reporting to the State Medicaid Agency will
be limited.
-

Alternative One - The documentation and reporting tool, reflecting reviewer
findings, is submitted to the State Medicaid agency without further analysis by
the entity responsible for evaluation activity. Decisions relative to compliance
would be made by the State Medicaid agency.

-

Alternative Two - The entity responsible for the evaluation would complete a
narrative summary of reviewer findings as documented on the documentation and
reporting tool for each MCO/PIHP, (summary of the information contained in the
documentation and reporting tool) with simple analysis, such as the total number
of regulatory provisions with a status of Met, Partially Met and Not Met.
Regulatory provision compliance determinations and decisions would rest with
the State Medicaid agency.

-

Alternative Three - Based on State Medicaid agency established thresholds and
compliance guidelines, the entity responsible for the evaluation would aggregate
or summarize reviewer findings noted on the documentation and reporting tool for
each MCO/PIHP. These would be submitted to the State Medicaid agency for
making regulatory provision compliance determinations.

-

Alternative Four - The entity responsible for the evaluation would present all the
reporting noted above and additionally an overall compliance summary and
determination for the MCO/PIHP. The compliance summary would be based on
pre-established State established thresholds and guidelines.

The State Medicaid Agency may desire a report that demonstrates how all plans performed
when evaluated for compliance with the regulatory provisions. The above noted alternatives
will accommodate this type of report, with some being more succinct and easier to produce
than others. The State Medicaid Agency may choose alternatives other than those described
above.

END OF PROTOCOL

ATTACHMENT A
Summary Of Compliance Determination Activities of Public and Private
Quality Oversight Organizations
American Accreditation Health Care Commission/URAC - National Network
Accreditation Program
The Commission/URAC is a private, non-profit accrediting organization that accredits managed
care organizations (MCOs). Commission/URAC accreditation products assess either all or
selected (e.g. utilization or case management) components of a MCO. The Commission/URAC
Accreditation attests to the extent to which an agency complies with structural and operational
standards established by the Commission/URAC. The process begins with a review of
documents submitted by the application organization. This is designed to make an initial
determination of whether or not the applicant meets each of the standards. Additional
information may be requested from the organization as a part of this process. Topics addressed in
the review include network participation and management, quality management, utilization
management, provider credentialing, and member participation and protection.
This “paper review” is followed by an onsite review that is carried out according to an agenda
agreed upon between the Commission/URAC and the managed care organization to be reviewed.
Interviews are conducted with key staff of the MCO/PIHP responsible for implementing the
MCO’s/PIHP’s managed care functions, including the medical director, quality assurance
manager, provider contracting manager, credentialing manager, and others as indicated.
Additional documents are also reviewed. This may include credentialing files, minutes, appeal
process documentation, etc.
Based on the findings, the Accreditation Committee makes a recommendation to the
Commission/URAC Executive Committee regarding accreditation. Accreditation is conferred
for a two-year period.
Health Care Financing Administration - Medicare Contractor Performance Monitoring
System
Reviews of MCOs that contract with Medicare are conducted every two years using the
Contractor Performance Monitoring System inspection instrument specified by HCFA. In
addition, during the two-year period, sections of the review instrument may be used to conduct
focus reviews or enforcement reviews when issues are identified. Sections of the instrument
may also be used to determine if corrective action plans, resulting from prior reviews, have been
implemented. Issues that may trigger intra-cycle reviews include provider or beneficiary
complaints, such as nonpayment of claims.

The length of the review depends on the membership size and overall scope of the MCO.
Typically, a standard review using the entire survey instrument takes two to four reviewers three
to four days to complete. Depending on the need to conduct an in-depth investigation, an
enforcement review may take additional days.
A pre-site visit letter and suggested agenda for the review is made available to the MCO prior to
the on-site visit. Revisions in this agenda may be made by the reviewers onsite, depending upon
their findings.
In some instances there may be a desk review of documents supplied by the MCO conducted
before the onsite review.
Providers may be interviewed during the review of the MCO. These interviews are usually
conducted by telephone. The principal focus of the review of delegated services is to assess the
MCO’s oversight process.
At the conclusion of a survey the reviewers share preliminary findings with the MCO. A final
report is prepared and returned to the MCO shortly after the onsite review. It includes the
deficiencies identified and requires submission of a corrective action plan by the organization.
Joint Commission on Accreditation of Healthcare Organizations (JCAHO) - Accreditation
Program for Health Care Networks
The Joint Commission network accreditation process includes an assessment of the central
operations of an MCO/PIHP and visits to settings (components) where members of the
MCO/PIHP actually receive care (e.g., hospitals, ambulatory surgery centers, long term care
facilities, home care agencies, and physician offices).
The network central offices are surveyed against the standards in the Comprehensive
Accreditation Manual for Health Care Networks. Issues addressed include rights,
responsibilities, and ethics; continuum of care; education and communication; health promotion
and disease prevention; leadership; management of human resources; management of
information; and improving network performance.
Surveys are three to five days in duration. They are conducted by surveyors with extensive
experience in managed care settings. They include physicians, nurses, administrators, and other
health care professionals.
Pre survey information is limited to the application, a pre survey planning questionnaire, and
information on the structure of the network. Lists of care settings (components and practitioner

offices) are submitted. From these, using a sampling methodology, sites to be visited as part of
the survey are selected.
The components are surveyed by individuals with both managed care and setting-specific survey
and professional experience. The standards are drawn from the network manual, as well as a
subset of the field-specific accreditation manuals. These visits are intended to ascertain the
extent to which the care setting is carrying out its responsibilities to provide care and service to
the members of the network. As well, a determination is made regarding the effectiveness of the
network in supporting the care setting in its role.
Surveyors use a defined agenda for all onsite activity. It is negotiated with the network prior to
the survey by staff in the Central Office.
Surveyors use computer technology to record their findings. They supply the network with a
preliminary report at the conclusion of the survey. This includes a tentative accreditation
decision.
There is post-survey review of the report, under rules determined by the Accreditation
Committee of the Board of Commissioners. Appeals of adverse decision are also possible to the
Board Accreditation Review Committee.
Networks are accredited for three years.

National Committee for Quality Assurance (NCQA) - Accreditation Program for Managed
Care Organizations
The accreditation survey of a MCO by NCQA begins with the submission of an application to
NCQA for accreditation review. The application contains detailed information about the MCO.
This information is intended to determine the accreditable entity of the MCO, its product lines
and products. The MCO can select which product lines (e.g., HMO, PPO) it wishes to have
accredited. Assuming that these activities meet the eligibility requirements, NCQA establishes a
survey process to address the selected activities. There is a desktop review of information
submitted by the MCO, which occurs six weeks before the onsite survey. This desktop review
includes such things as program descriptions, policies and procedures, and member materials.
Other documentation review takes place as a part of the onsite survey process.
Onsite surveys are from two to four days in duration. At a minimum, a survey team consists of
two members -- a physician and an administrator. A full scope survey will usually have a team
of four members -- two physicians and two administrators. Surveyors typically have experience
in, and responsibility for, quality management in their own organizations. Surveyors carry out

their activities through review of documents and records, onsite observation, and interviews with
staff.
On-site review includes file review - UM denials, credentialing and recredentialing files,
complaint and appeals files, and medical records. Surveyors are data collectors and evaluators.
They assign the preliminary compliance designations. File review results are reviewed with the
MCO's staff. One of five levels of compliance can be assigned to each standard. Topics
addressed in the review include quality management, utilization management, credentialing,
members’ rights and responsibilities, medical records, and preventive health services programs.
During the summation conference, the survey team identifies key strengths and opportunities.
Surveyors will not come to conclusions regarding standards compliance or the overall possible
accreditation status of the organization at the summation conference. The team submits its
findings to a Review Oversight Committee (ROC) composed of 14 physicians, who all have
managed care experience, who make the accreditation decision. This can be appealed to a
Reconsideration Committee by the MCO. The decision of the Reconsideration Committee is
final.
In addition to conducting an onsite survey to determine the MCO’s compliance with the
standards (worth 75 points), since 1999, NCQA also determines how well the MCO performs on
selected audited measures from the Health Plan Employer Data and Information Set (HEDIS),
and Consumer Assessment of Health Plans Study (CAHPS). The total score allocated to HEDIS
and CAHPS is 25 points - 12.5 points for each. This data is required of accredited organizations
on an annual basis. Performance on these measures will influence the accreditation decision
annually if changes occur. An MCO’s accreditation status is thus reevaluated annually. A
standards score is valid up to three years.

END OF ATTACHMENT A