Supporting Statement – Part B
Collection of Information Employing Statistical Methods
The HEDIS® collection uses the same statistical methods described in the original request.
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used.
CMS requires reporting of HEDIS® at the contract level from all Medicare Advantage Organizations. A Medicare Advantage managed care organization (MAO) is defined based on the legal and management structure and delivery system that support the product-line contracting with Medicare, and offering services to Medicare beneficiaries. This same definition is used for HEDIS® reporting and accreditation. A MAO is usually a single legal entity that offers one provider network and is marketed under one name. All MAO contracts that have been in place for one full calendar year are expected to report HEDIS® on their Medicare product as part of their contractual obligation to CMS.
HEDIS® measures are collected using two methods: administrative and hybrid. Some measures just require administrative data collection, while for other measures the MAO has the option to use a hybrid methodology which supplements the collection of data with medical record review of a sample (411 records) of records.
In the administrative method, transaction data or other administrative databases are used by NCQA auditors and the MA organizations providing HEDIS® data to NCQA, to identify the eligible population and numerator. The reported rate is based on all members who meet the eligible population criteria (after optional exclusions, if applicable) and who are found through administrative data to have received the service required for the numerator.
In the hybrid method, organizations look for numerator compliance in both administrative and medical record data. The denominator consists of a systematic sample of members drawn from the measure’s eligible population. Organizations should review administrative data to determine if members in the systematic sample received the service and review medical record data for members who do not meet the numerator criteria through administrative data. The reported rate is based on members in the sample who are found to have received the service required for the numerator. There are seven Medicare measures that may be collected using what is referred to as the “hybrid” method for reporting. These seven measures are: Adult BMI Assessment, Colorectal Cancer Screening, Care for Older Adults, Cholesterol Management for Patients With Cardiovascular Conditions, Controlling High Blood Pressure, Comprehensive Diabetes Care, and Medication Reconciliation Post-Discharge. For these seven measures, the sample size for Medicare patients is no larger than 411.
All HEDIS® clinical measures based on medical record review have a minimum sample size of 411. The sample size of 411 specified by NCQA is based on a statistical estimation of providing an 85 percent chance of identifying a five percentage point difference between plans.
The calculation of the required sample size is dependent on three factors.
The percentage point difference between plans desired to be identifiable
NCQA has chosen their sample size to allow the ability to identify a five percentage point difference between plans. If one wishes to have the ability to identify smaller differences between plans, then the sample size must increase. If one is satisfied only identifying differences greater than five percentage points, then the sample size can decrease.
The likelihood of identifying the difference
Likelihood of identifying the differences between plans is the likelihood that the minimum percentage point difference identifiable will be detected. To increase this likelihood, one must increase the sample size.
The average score of the measure
In setting the sample size at 411, NCQA has made the conservative assumption that scores will average 50 percent. As a score moves closer to 0 percent or 100 percent, a smaller sample size is required to detect the same percentage point difference. The formula used to identify differences between proportions takes into account the standard error of each proportion. The formula for the standard error of a proportion is (p*(1-p)/n)^.5, where p is the proportion and n is the sample size. As a result, the standard error is at its greatest when a proportion of .5 and it decreases as proportions approach 0 or 1. The implication of this formula is that the confidence interval around the proportion of 50 percent will be greater than the confidence interval around any other proportion given that the sample sizes are the same, thus making differences between proportions that average 50 percent most difficult to identify.
Complete information regarding the guidelines for calculations and sampling is available in NCQA’s publication HEDIS® 2017 Volume 2: Technical Specifications. The volume is available from https://downloads.ncqa.org/customer/Login.aspx.
2. Describe the procedures for the collection of information.
Detailed sample size calculation, instructions for systematic sampling and complex probability sampling, oversampling rates, and confidence interval calculations for the hybrid collection methodology are located in the HEDIS® 2017 Volume 2 Technical Specifications.
In general, the sample size is calculated assuming a two-tailed test of significance between two proportions (α = .05, 80 percent power, two tailed test of significance). A normal approximation to the binomial with a continuity correction was employed in the sample size calculation. The worst case assumption of a 50 percent expected value was assumed. The detectable difference for most measures is 10 percentage points. This was chosen because it is a big enough difference to be actionable, it is not unduly burdensome for data collection and it is not so small as to be “swamped” by non-sampling error.
3. Describe methods to maximize response rates and to deal with issues of non-response.
This is not a survey involving beneficiaries or other survey respondents. Therefore, discussions of “Response rates” do not apply to this measurement set. All MA organizations are required to provide 100% reporting so response rates are not applicable.
4. Describe any tests of procedures or methods to be undertaken.
The HEDIS® Summary Data Collection procedures concern either Administrative or Hybrid methods. For the administrative method, transaction data or other administrative databases are used to identify the eligible population and numerator. The reported rate is based on all members who meet the eligible population criteria (after optional exclusions, if applicable) and who are found through administrative data to have received the service required for the numerator. For the hybrid method, organizations look for numerator compliance in both administrative and medical record data. The denominator consists of a systematic sample of members drawn from the measure’s eligible population. Organizations should review administrative data to determine if members in the systematic sample received the service and review medical record data for members who do not meet the numerator criteria through administrative data. The reported data is based on members in the sample who are found to have received the service required for the numerator.
Each HEDIS® measure has clearly defined calculation instructions, but the instructions are complex. Medical record reviewers and database administrators can and do make mistakes. The only protection against errors is to have HEDIS® production systems audited by an independent third party. NCQA has developed a precise, standardized methodology for verifying the integrity of HEDIS data collection and analyses, called the HEDIS® Compliance Audit. The initial focus of the audit is on actions that an organization can take to correct its results. The final audit report indicates which measures are reportable and which are not. All MA contracts use NCQA certified auditors.
NCQA and the auditors have an intensive process for improvements in data collection instructions to the plans. NCQA makes this a very high priority. They have year-round meetings; NCQA conducts site visits at the plans when the 3rd party auditors are at the plans; they have written documents that contain every problem, issue, lesson learned, and process for improvement, the entire audit process is documented in Volume 5 annually. NCQA compiles a detailed “Lessons Learned” document on an annual basis. NCQA and the auditors meet many times during the year to work on the suggested improvements in the next data collection period.
5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
The National Committee for Quality Assurance (NCQA) is the federal contractor for HEDIS® summary level data.
Individuals consulted on statistical aspects of the hybrid collection methodology include:
Mary
Braman , MPH
Vice-President,
Performance Measurement
NCQA
2000
L Street, NW, Suite 500
Washington,
DC 20036
Phone:
(202) 955-5139
Fax:
(202) 955-3599
Email:
[email protected]
Alan
Hoffman ,
MPH
Vice
President, Research & Analysis
NCQA
2000
L Street, NW, Suite 500
Washington,
DC 20036
Phone:
(202) 955-1726
Fax:
(202) 955-3599
Email:
[email protected]
Lori Teichman, Ph.D.
Centers for Medicare & Medicaid Services
CM/CPC/MDBG/DCAPP
7500 Security Boulevard
Mail Stop: C1-25-05
Baltimore, MD 21244
Phone: (410) 786-6684
Fax: (410) 786-6303
Email: [email protected]
| File Type | application/msword |
| File Title | Supporting Statement – Part B |
| Author | CMS |
| Last Modified By | SYSTEM |
| File Modified | 2017-11-27 |
| File Created | 2017-11-27 |