Form CMS-10589 QECP Annual Report Workbook

---

Overview

Intro
1-Background & Claims
2-Measures
3-Level of Analysis
4-Public Use of Reports
5-Provider Requests
6-Response to Requests
7-Security Breaches
8-Security Practices
9-Non-Public Analyses
10-Data
11-Feedback

---

Sheet 1: Intro

QECP ANNUAL REPORT WORKBOOK
Overview
As part of the Qualified Entity Certification Program (QECP), all QEs are required to submit a QECP annual report each year.
Instructions
This workbook consists of eleven tables. Detailed instructions for completing each table can be found on the worksheet for that table. Once the workbook is complete, please upload it to the annual report module of the QECP application portal. The primary QE application point of contact for your organization must enter his/her electronic signature into the annual report module of the portal to finalize submission of the QECP annual report.
Tab 1	Background and Volume of Claims Data (Table 1)
Tab 2	Number of Performance Measures (Table 2)
Tab 3	Level of Analysis (Providers and Populations) (Table 3)
Tab 4	Public Use of Performance Reports (Table 4)
Tab 5	Provider Requests for Corrections or Appeals (Table 5)
Tab 6	Response to Requests (Table 6)
Tab 7	Data Security Breaches (Table 7)
Tab 8	Changes in Data Security Practices (Table 8)
Tab 9	Non-Public Analyses (Table 9)
Tab 10	Data (Table 10)
Tab 11	Opportunity for Feedback (Table 11)
*See 42 CFR Subpart G.
**See Section 2.13.4 of the 2015 QECP Operations Manual for a detailed description of the required annual report elements.
PRA Disclosure Statement According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number.  The valid OMB control number for this information collection is 0938-XXXX.  The time required to complete this information collection is estimated to average 80 hours per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.  If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.  Please do not send applications, claims, payments, medical records or any documents containing sensitive information to the PRA Reports Clearance Office. Please note that any correspondence not pertaining to the information collection burden approved under the associated OMB control number listed on this form will not be reviewed, forwarded, or retained. If you have questions or concerns regarding where to submit your documents, please contact Kari Gaare.

---

Sheet 2: 1-Background & Claims

Table 1: Background and Volume of Claims Data
Instructions: All QEs are required to complete this table regardless of whether there have been any changes since QE certification. Any changes since QE certification in the geographic region for QE performance reporting (Element 1B), or volume of claims data from other payer sources (Element 2A) must be reported following the “Reporting Changes” procedures outlined in Section 2.13.3 of the 2015 QECP Operations Manual, and not solely as part of this annual report.
Geographic Region		Response
1. What is the geographic area for which your organization is reporting (or plans to report)?	1a. List the state(s) in which your organization's data and reporting cover the entire state.
	1b. For states in which your organization's data and reporting cover only part of the state, list the state and include (in parentheses) the counties, MSAs, or other boundaries of the coverage area.
Volume of Claims Data		Response
2. Provide the total number of covered lives included in the claims data sources you have obtained/will obtain.	For QEs that have not yet received or integrated QE Medicare data, estimate based on Element 2A Data Source Attestation.
	To obtain the estimate of covered lives in the FFS data, visit:	CMS Medicare Enrollment Reports
	and adjust for Medicare Advantage penetration rate:	Medicare Advantage Penetration Rate
	Detailed Instructions for calculating covered lives can be found in Appendix A of the QECP Data Source Attestation
	2.a.i. Other Payer:
	2.a.ii. Medicare FFS:
	For QEs that have received and integrated QE Medicare data with other sources of data, provide the number of covered lives below.
	2.b.i. Other Payer:
	2.b.ii. Medicare FFS:
3. Provide the number of covered lives residing in the geographic coverage area on which your organization intends to report provider performance. Please use the U.S. Census Bureau website (Table S2701) to determine the number of covered lives.	Total number of covered lives:
US Census Bureau
Percentages Below are Calculated Automatically. No Response Required.
Percentage of market share that other payer and Medicare FFS claims data represent:	3.a. For QEs that have not yet received and integrated QE Medicare data with other sources of data:	#DIV/0!
	3.b. For QEs that have received and integrated QE Medicare data with other sources of data:	#DIV/0!

---

Sheet 3: 2-Measures

	Table 2: Number of Performance Measures
	Instructions: Only QEs that have released a QE public performance report by ____are required to complete this table. If not applicable, enter “N/A” here:
	Information Request	Number of Measures*	*Although measures may be reported in different ways (e.g., in different public report formats, by product line), please only count unique measures. Do not include the different ways that your organization reports each measure. A composite measure is an individual/unique measure.
	1. Enter the number of standard measures including QE Medicare data that your organization publicly reported in ___.
	2. Enter the number of alternative measures including QE Medicare data that your organization publicly reported in ____.
	3. List measures not included in the performance reports but where the QE Medicare data were accessed. Please explain briefly why the measure was not included in the performance reports.	Measure Name	Reason for Not Including

---

Sheet 4: 3-Level of Analysis

	Table 3: Level of Analysis (Providers and Populations)
	Based on National Quality Forum (NQF)'s "Level of Analysis:"
		NQF Measures
	Instructions: Only QEs that have released a QE public performance report by _____are required to complete this table. If not applicable, enter “N/A” here:
	Providers
	1. Enter the number of each type of provider included in your organization's QE public performance report(s) in 2015. Respond based on the level of analysis included in the QE public report(s). For example, if your organization reports at the clinic level, fill in item "c" below. If your organization reports at the facility level, fill in item "e" below.		For each level of analysis, list the type and number (in parentheses) of specialists, subspecialists, or subtypes included (e.g. cardiologist, primary care physician, dialysis facility) in the peer groups being compared. Please list all that apply. A list of provider types and specialty definitions can be found at:
			Medicare Specialty Definitions
	Provider Level of Analysis ( NQF Definitions)	Number of Providers	Specialists/Subtypes (Number)
	a. Individual Clinician
	Various types of healthcare practitioners/providers, which may include but is not limited to, physicians, nurses, and allied health professionals.
	b. Clinic
	Setting in which outpatient healthcare services are provided by physicians or other healthcare providers, including but not limited to, primary care, family practice, general internal medicine, and faculty practice plans.
	c. Group/Practice
	Two or more healthcare clinicians/providers who practice together, either at a single geographic location or at multiple locations.
	d. Team
	Two or more healthcare clinicians/providers, at one location or across different settings, who collaborate together for the care of a single patient or multiple patients.
	e. Facility
	A single entity that provides healthcare, which may include but is not limited to, a hospital, nursing home, dialysis center, and home health agency.
	f. Health Plan
	An organization that acts as an insurer for an enrolled population.
	g. Integrated Delivery System
	A healthcare entity that may include a variety of facilities and/or services including, but not limited to, hospitals, medical groups, skilled nursing facilities, home health, and/or insurance vehicles. This includes delivery systems that assume responsibility across settings for the complete patient-focused episode of care, such as accountable care organizations.
	Populations
	2. Enter the number of measures (including benchmarks) for each population analyzed in your organization's QE public performance reports in ____. For example, if a measure is reported at the provider level, and a state and national benchmark are also reported for the measure, the measure must be included in the measure count for the provider, state, and national categories.
	Population Measurement	Number of Measures
	National
	State
	Regional
	Community
	County or City
	Providers

---

Sheet 5: 4-Public Use of Reports

Table 4: Public Use of Performance Reports
Instructions: Only QEs that have released a QE public performance report by ______ are required to complete this table. If not applicable, enter “N/A” here:
Information Request	Response
1. Provide a hyperlink to (or an electronic copy of) your organization’s most recently released QE public performance report. (If providing an electronic copy, please upload to the annual report module of the QECP application portal.)
2. How does your organization measure the public use of its QE public performance reports? Please provide quantifiable data (e.g., number of website visits per month, number of pages viewed per visit). Provide examples of how consumers are using your measure results. (If providing supporting documentation, please upload to the annual report module of the QECP application portal.)

---

Sheet 6: 5-Provider Requests

Table 5: Provider Requests for Corrections or Appeals
Instructions: Only QEs that have released QE confidential provider reports by ________ are required to complete this table. If not applicable, enter "N/A" here:
Information Request*	Number			5. For each measure in your organization’s QE public performance reports for which corrections were requested, enter the number of providers that requested a correction.
1. Number of requests for additional information—Medicare FFS claims data or beneficiary names—through the corrections and appeals process in ____.				Measure Name		Number of Providers Requesting Correction
2. Number of providers that requested additional information— Medicare FFS claims data or beneficiary names—through the corrections and appeals process in _____.
3. Number of requests for error correction through the corrections and appeals process in ____.
4. Number of providers that requested a correction through the corrections and appeals process in _____.
*These questions capture the total number of requests as well as the total number of providers making requests. For example, if one provider makes two requests, count one provider and two requests. These questions also distinguish between requests for additional information and requests for error correction.

---

Sheet 7: 6-Response to Requests

Table 6: Response to Requests
Instructions: Only QEs that have released QE confidential provider reports by _____ are required to complete this table. If not applicable, enter "N/A" here:
Information Request and Response
1. Number of requests for additional information fulfilled in ____—Medicare FFS claims data or beneficiary names.				Number
For error correction requests as a direct result of the provider corrections and appeals process:				Min	Mean	Max
2. Number of business days (minimum, mean, and maximum) to acknowledge receipt of error correction requests in _____.
3. Number of business days (minimum, mean, and maximum) to resolve error correction requests in _____.
4. Number of error correction requests in _____ that were not resolved.				Number
5. Number of error corrections made to the provider reports in _____.
6. Describe the types of problems leading to requests for error correction as a result of the provider corrections and appeals process in ____, and indicate which types of problems required correction and which did not.				Response:
For error corrections outside the corrections and appeals process:				Number
7. Number of error corrections detected by or reported to your organization outside the corrections and appeals process in _____.
8. Explanation of the types of errors (outside the corrections and appeals process) that required correction in _____.				Response:

---

Sheet 8: 7-Security Breaches

Table 7: Data Security Breaches
Instructions: All QEs (regardless of public report status) are required to complete this table. If no new or unresolved breaches occurred, enter "None" here:
Information Request and Response
1. Provide information on all new data security breaches that occurred in _____ and any unresolved breaches from previous years, including date(s), type(s), and resolution. (Note: This summary is in addition to the required immediate notification that your organization must provide to CMS and the QECP team each time a data security breach occurs.)
Data Security Breach	Date of Breach	Description/Type	Resolution	Date of Resolution
1
2
3
4
5
6
7
8
9
10

---

Sheet 9: 8-Security Practices

Table 8: Changes in Data Security Practices
Instructions: Only QEs that have obtained Phase 2 Data Security approval and received QE Medicare data by ____________ are required to complete this table. If not applicable, enter "N/A" here:
The purpose of this table is to confirm and capture any changes that have taken place in your organization's data security practices since the date of Phase 2 approval, the submission of your organization's last QECP annual report, or the submission of your organization's last QECP reapplication. An individual listed as Data Custodian on your organization's QE DUA must complete and sign this table. If your QE consists of more than one organization with a Phase 2-approved QECP Data Security Workbook, a separate worksheet for Table 8 must be completed by a Data Custodian at each organization.
PHYSICAL LOCATIONS
List physical location(s) in which CMS data were stored or accessed in ____:
Event	Event Occurred Since Date of QECP Data Security (Phase 2) Approval, Submission of Last QECP Annual Report, or Submission of Last QECP Reapplication		If “Yes,” Date Reported to QECP
	YES	NO
New physical location(s) to store or access CMS data.
Physical location(s) discontinued accessing CMS data.
QE relocated within existing physical location(s).
QE conducted a major remodel of existing physical location(s) (e.g., changed existing floor plan).
Provide additional details for all physical location events that have occurred since Phase 2 approval, submission of your organization's last QECP annual report, or submission of your organization's last QECP reapplication.
INFORMATION SYSTEMS
Event	Event Occurred Since Date of QECP Data Security (Phase 2) Approval, Submission of Last QECP Annual Report, or Submission of Last QECP Reapplication		If “Yes,” Date Reported to QECP
	YES	NO
QE engaged new information technology contractor (onsite support, remote support, hosting, Internet service provider).
QE changed alternate storage sites (e.g., offsite backups, archive storage).
QE changed alternate processing sites (e.g., disaster recovery).
QE disposed of IT equipment that stored, processed, or accessed CMS data.
QE implemented significant change to information system (e.g., XP to Windows 7 or 8, implemented virtualization).			N/A
QE maintains a current inventory of all IT hardware and software that stores, processes, accesses, or transmits CMS data.			N/A
QE maintains a current inventory of all removable media that store CMS data (e.g., flash drives, CDs/DVDs, backup tapes).			N/A
QE made changes in the Configuration Management.
QE made System and Services Acquisition
Provide additional details for all information systems events that have occurred since Phase 2 approval, submission of your organization's last QECP annual report, or submission of your organization's last QECP reapplication.
SECURITY ACTIVITY
Event	Event Occurred Since Date of QECP Data Security (Phase 2) Approval, Submission of Last QECP Annual Report, or Submission of Last QECP Reapplication		If “Yes,” Date Reported to QECP
	YES	NO
QE has reviewed and updated information security policies and procedures.			N/A
QE has reviewed and updated system security plan.			N/A
QE has reviewed and updated risk assessments.			N/A
QE has conducted required security and awareness training.			N/A
QE has reviewed, tested, and updated incident response plans.			N/A
QE has reviewed, tested, and updated contingency plans.			N/A
For all “No” answers, describe why the activity was not conducted.
SECURITY RESPONSIBILITY
Event	Event Occurred Since Date of QECP Data Security (Phase 2) Approval, Submission of Last QECP Annual Report, or Submission of Last QECP Reapplication		If “Yes,” Date Reported to QECP
	YES	NO
QE has assigned primary security responsibility to a new individual.
QE has undergone a change in ownership or management structure.
There has been a change in the contractors that make up the QE.
There has been an internal unauthorized disclosure of beneficiary information.
There has been an external unauthorized disclosure of beneficiary information.
Provide additional details for all security responsibility events that have occurred since Phase 2 approval, submission of your organization's last QECP annual report, or submission of your organization's last QECP reapplication.
REGULATORY COMPLIANCE
Event	Event Occurred Since Date of QECP Data Security (Phase 2) Approval, Submission of Last QECP Annual Report, or Submission of Last QECP Reapplication		If “Yes,” Date Reported to QECP
	YES	NO
There has been a change in state privacy and security laws.
Changes to policies and procedures were made to comply with the new state privacy and security laws (if any).			N/A
Provide additional details for any regulatory compliance events that occurred since Phase 2 approval, submission of your organization's last QECP annual report, or submission of your organization's last QECP reapplication.
DATA SECURITY ATTESTATION AND DECLARATION
I attest that this table reflects an accurate picture of events that have occurred since the date of QECP Data Security Approval (Phase 2), the date of submission of my organization’s last QECP annual report data security attestation, or the date of submission of my organization's last QECP reapplication. Declaration I, (Insert QE DUA Data Custodian’s Name), am familiar with the controls implemented to become a Qualified Entity and attest that (Insert Name of Organization) has remained in compliance with the Qualified Entity Certification Program and will continue to meet the data security requirements of the program. Additionally, I declare that my organization is currently in compliance with the most recent version of the CMS Acceptable Risk Safeguards (ARS) (Appendix B: CMSR Moderate Impact Level Data).
(QE DUA Data Custodian’s Electronic Signature)		(QE DUA Data Custodian’s Name and Title)

---

Sheet 10: 9-Non-Public Analyses

Table 9: Non-Public Analyses
Instructions: Only QEs that have provided or sold non-public analyses to authorized users by ________ are required to complete this table. If not applicable, enter "N/A" here:
Summary of Analyses Provided or Sold	Number			Description of Non-Public Analyses
1. Number of analyses provided or sold in ____.				4. For each non-public analysis that your organization provided or sold, please provide a description of the topic and purpose.
2. Number of purchasers of non-public analyses in ____.				Topic		Purpose
3. Total fees received for non-public analyses in ____.

---

Sheet 11: 10-Data

Table 10: Data
Instructions: Only QEs that have provided or sold data to authorized users by _________ are required to complete this table. If not applicable, enter "N/A" here:
Information Request and Response
1. For each occurrence of data bought or sold, provide information on entities who received data, the basis under which each entity received such data, and the total amount of fees received for providing, selling, or sharing data.
Entity that Received Data	Basis Under Which Entity Received Data	Fees Received

---

Sheet 12: 11-Feedback

Table 9: Opportunity for Feedback
The QECP team invites your organization to provide feedback on the QE program—to explain what has worked well, to identify challenges encountered, and to suggest opportunities for improvement. The QECP team will use this feedback to improve the quality of the program. In particular, we ask your organization to identify what has worked well and opportunities for improvement in the areas of certification, training, and technical support.
Instructions (Q1–Q5): Please share you organization’s experience with each phase of the QECP. We value your feedback on what has worked well and what could be improved. To help us continue to improve the process, please be as specific as possible and only provide feedback based on your experiences over the past year.
Q1. Application and Certification (Phase 1):	Worked Well
	Opportunities for Improvement
Q2. Data Security (Phase 2):	Worked Well
	Opportunities for Improvement
Q3. Data Integration and Measures Calculation (Phase 3):	Worked Well
	Opportunities for Improvement
Q4. Reporting (Phase 4):	Worked Well
	Opportunities for Improvement
Q5. What advice would you give to existing or potential QEs based on your experience?
Instructions (Q6–Q8): Please share your organization’s experience with using the various tools and support provided by the QECP. To help us continue to improve the process, please be as specific as possible and only provide feedback based on your experiences over the past year.
Q6. Communications with QECP Team Members	How satisfied are you with communication with the QECP team (e.g., program manager, other QECP staff)? Has the quality of communication changed over the past year?
	How satisfied are you with the timeliness and consistency of responses that you receive to questions posed to the QECP team? Has this changed over the past year?
	Does your organization have any suggestions for how communication can be improved to better meet your organization’s needs?
Q7. Webinars	Which webinars, if any, have you participated in over the past year? Which were the most helpful? Least helpful? Please explain. For example, was sufficient detail provided on the topics covered in the webinar?
	Are there days or times of the week that would be more convenient for your organization to participate in live webinars?
	Does your organization have suggestions for additional webinar topics or format?
Q8. QECP Operations Manual	How helpful has the QECP Operations Manual been when submitting evidence throughout the application process?
	Does your organization have specific recommendations for improving the QECP Operations Manual?
If you have any additional comments, please provide them. For example, is there additional training or technical assistance that would have been helpful to you earlier or would be helpful in the future? Any other comments or suggestions?
Q9. Additional Comments