External Quality Review of Medicaid MCOs and Supporting Regulations in 42 CFR 438.360, 438.362, and 438.364

2010 EXTERNAL QUALITY REVIEW (EQR) PROTOCOLS
APPENDIX V: INFORMATION SYSTEM CAPABILITIES ASSESSMENT – ACTIVITY
REQUIRED FOR MULTIPLE PROTOCOLS

TABLE OF CONTENTS
PURPOSE AND OVERVIEW OF THE APPENDIX .........................................................................1
DATA COLLECTION, PROCESSING, AND REPORTING PROCESSES .......................................2
ACTIVITY 1: MCO COMPLETES INFORMATION SYSTEM CAPABILITY ASSESSMENT (ISCA) .3
ACTIVITY 2: REVIEW ISCA AND ACCOMPANYING DOCUMENTS ..............................................3
ACTIVITY 3: INTERVIEW MCO STAFF ..........................................................................................4
ACTIVITY 4: ANALYZE ISCA FINDINGS ........................................................................................4
THE FUTURE OF INFORMATION SYSTEMS ASSESSMENT .......................................................4

Attachments
Attachment A: Information System Capability Assessment (ISCA) Template
Attachment B: Information System Review Worksheet and Interview Guide

PURPOSE AND OVERVIEW OF THE APPENDIX
The purpose of this appendix is to define the desired capabilities of the MCO’s information system
and assess the strength of the MCO’s information system capabilities. Please note that although
this appendix is not mandatory, States are mandated to perform information system capability
assessments as part of the mandatory protocols. Furthermore, 42 C.F.R. § 438.242 requires the
State to ensure that each MCO maintains a health information system that collects, analyzes,
integrates, and reports data for areas including, but not limited to, utilization, grievances and
appeals, and disenrollment for other than loss of Medicaid eligibility. Per 42 C.F.R. § 438.242, the
system must be able to achieve the following:
1. Collect data on enrollee and provider characteristics as specified by the State and on
services furnished to enrollees through an encounter data system or other methods as may
be specified by the State;
2. Ensure that data received from providers are accurate and complete by:
a. Verifying the accuracy and timeliness of reported data;
b. Screening the data for completeness, logic, and consistency; and
c. Collecting service information in standardized formats to the extent feasible and
appropriate; and
3. Make all collected data available to the State and upon request to CMS, as required in this
subpart.

This appendix provides a basic overview of the processes of collecting, processing, and reporting
data, four activities for performing the assessment, and information about the future of information
system assessments.
Assessment of MCO’s information systems is a process of 4 consecutive activities. The MCO will
complete Activity 1, which involves the collection of standard information about its information
system using an Information Systems Capability Assessment (ISCA) (found in Attachment A) or
similar tool. Activities 2 and 3 are reviews of the ISCA performed by the EQRO and based on
interactive sessions with MCO staff to validate the completed ISCA and gather additional
information to assess the integrity of the information in the ISCA. The EQRO can use the Managed
Care Organization Information System Review: Worksheet & Interview Guide (Worksheet) in
Attachment B to conduct interviews with MCO staff who completed the ISCA, as well as other
necessary MCO staff. Activity 4 involves analyzing the findings from both the completed ISCA and
the follow-up discussions with MCO staff.

DATA COLLECTION, PROCESSING, AND REPORTING
PROCESSES
A basic overview of the data process flow from healthcare encounter to EQRO is provided here to
provide a context for the information system assessment for which the State and EQRO are
responsible. The process is described in more detail below Diagram 1.
Diagram 1: Data Process Flow

1. Provider Data Activities
1.1 Enter data about services in IS

1.2 Process encounter data

1.3 Submit requested data and/or
elements to MCO

2. MCO Data Activities
2.1 Abstract data elements from data
submitted by provider (1.3)

2.2 Validate provider data processing
and submission (in 1.2 and 1.3)

2.3 Submit data processed by MCO IS
to EQRO

3. EQRO Data Activities
3.1 Validate MCO data (in 2.3)

1.

3.2 Validate MCO processing (in 2.1)

3.3 Validate MCO's validation of
provider data processing (in 2.2)

Provider data activities
1.1.
Provider records their encounters in a hybrid of paper/ electronic format
(increasingly electronic)

EQR Protocol Appendix V
Information System Capabilities Assessment
December 2011

2

1.2.

2.

3.

Provider uses its information system to process and clean the data, compile it,
scrub it, etc.
1.3.
Provider submits processed encounter data to the MCO for payment, for specific
measures, for audit, etc.
MCO data activities
2.1.
MCO receives provider data (claims, encounters, certified EHR technology), uses it
for payment (claims) or other purposes (received in a transaction file), and enters it
into a data repository
2.2.
MCO validates the provider’s accurate capture, processing, and submission of their
data to the MCO by performing edit checks on claims, random medical record
reviews, and/or other methods
2.3.
MCO processes data in its data repository and submits data to the EQRO (or State)
EQRO data activities
3.1.
EQRO validates the data in its repository or that the MCO or other organization
(e.g., claims processing vendor, State repository) submitted for EQR purposes
3.2.
EQRO validates the accuracy of the provider’s data and the accuracy with which it
was processed by the MCO (instructions for this validation are provided in Protocols
2, 4, 6, and 9)
3.3.
EQRO validates that the MCO is ensuring the providers are accurately processing
and submitting data to the MCO

ACTIVITY 1: MCO COMPLETES INFORMATION SYSTEM
CAPABILITY ASSESSMENT (ISCA)
Some States assess the capabilities of the MCO’s information system as part of pre-contracting,
contract compliance, or contract renewal activities. If an assessment has been completed by the
State through these or other means (e.g., private sector accreditation or performance measures
validation), and the information gathered is the same as or consistent with what is described in this
assessment, it may not be necessary to repeat this assessment process. However, information
from previously conducted assessments must be accessible to EQRO reviewers.
Completing the ISCA provided in Attachment A requires that the MCO provide all requested
documentation identified on a checklist at the end of the assessment tool and return it to the
EQRO within a State-defined time frame.

ACTIVITY 2: REVIEW ISCA AND ACCOMPANYING DOCUMENTS
The EQRO assesses the adequacy of MCO policies and procedures as portrayed by the
information submitted by the MCO on the ISCA. MCO answers should be evaluated against the
information system standards established by the State to calculate and report specific plan-level
performance measures, and collect and submit encounter data to the State. The EQRO should
note incomplete or insufficient sections of the ISCA to identify specific policies, procedures, and
documentation most likely to affect the integrity of information collected by the MCO. The EQRO
should discuss and further review such issues during the MCO site visit.
EQR Protocol Appendix V
Information System Capabilities Assessment
December 2011

3

ACTIVITY 3: INTERVIEW MCO STAFF
The EQRO reviewer(s) conduct(s) interviews with MCO staff responsible for completing the ISCA,
as well as additional staff responsible for aspects of the MCO’s information system function. The
interviews focus on the topics outlined in the structured ISCA interview guide, with additional topics
covered as necessary based on the pre-onsite analysis of the ISCA in Activity 2. Attachment B
provides a worksheet and interview guide for reviewing the ISCA. The information system interview
with the MCO should be closely coordinated with the MCO site visit performed in Protocol 1. Refer
to Protocol 1, Activity 3 for steps in conducting a successful MCO site visit.

ACTIVITY 4: ANALYZE ISCA FINDINGS
The EQRO will write a statement of findings at the conclusion of the review of the ISCA. This
statement of findings about the MCO’s information system should include implications of the
finding for the following:
1. Completeness and accuracy of encounter data collected and submitted to the State;
2. Validation and/or calculation of performance measures;
3. Ability of the information system to conduct MCO quality assessment and improvement
initiatives; and
4. Ability of the information system to oversee and manage the delivery of health care to the
MCO’s enrollees.

THE FUTURE OF INFORMATION SYSTEMS ASSESSMENT
With the increasing uptake of more sophisticated and comprehensive information systems, it will
be important to adapt the way information systems’ capabilities are assessed. As information
systems evolve, so will the tools and rules with which States and EQROs can use to assess them.
Currently, the industry has a clear understanding of physical security and integrity of information
systems, but is much less clear on how to assess the technical security and integrity of those same
information systems. As noted elsewhere (see Methodology), existing statutory requirements
regarding privacy and security supports, as well as guidance and tools, might be considered
suitable topics for voluntary special projects in performance improvement, as in Protocols 7 and 8.
Given the ongoing and accelerating accumulation of health information technology (HIT)
standards, HIT certification requirements, and HIT qualifications proposed and imposed by Federal
payers, organizations must anticipate near-term changes in assessments of new information
system requirements. Particularly pertinent to the future of information systems assessment is
Section 4201 of the American Recovery and Reinvestment Act of 2009 (ARRA) which allows for
the payment of incentives to eligible professionals and hospitals to promote the adoption and
meaningful use of certified electronic health records (EHRs). Under the Medicaid Electronic Health
Record Incentive Program, eligible providers can receive 100 percent federal financial participation
EQR Protocol Appendix V
Information System Capabilities Assessment
December 2011

4

(FFP) to adopt, implement, upgrade, and meaningfully use certified EHR technology. Additionally,
States can benefit from 90 percent FFP for any State administrative expenses related to the
implementation of the program. Therefore, it is important for both States and MCOs to work
cooperatively in the planning and use of certified EHRs and health information exchange systems.
The design and utilization of secure EHRs will become an increasingly important element in the
EQR process and is reflected in Appendix B of this Appendix V.
Finally, with the enactment and implementation of the Affordable Care Act, States and MCOs must
coordinate their HIT planning efforts to ensure interoperability between systems that effectively
provide for future data needs to meet eligibility, enrollment, Health Insurance Exchange, and
Accountable Care Organization statutory and regulatory requirements. EQROs should continually
assess MCO planning activities to ensure alignment with and responsiveness to these initiatives.
In order to learn from and share State experiences with emerging HIT and EHR initiatives that can
impact reporting of performance measure and performance improvement project outcomes, CMS
strongly encourages States to contract with EQROs to include results of State HIT and EHR
initiatives in annual EQR reports. This may include successful implementation of health information
exchange with other State agencies to improve data source collection efforts for performance
measures or performance improvement projects. Similarly, including lessons learned from
challenging or unsuccessful HIT initiatives are just as informative to Federal and other State
partners, and may be a valuable source of information to be included in the Annual Secretary’s
Report on Quality published each September.

END OF DOCUMENT

EQR Protocol Appendix V
Information System Capabilities Assessment
December 2011

5