Supporting Statement Part B -- Collection of Information for CAHPS Health Plan Survey Comparative Database 11-21-2013

Supporting Statement Part B -- Collection of Information for CAHPS Health Plan Survey Comparative Database 11-21-2013.docx

Collection of Information for Agency for Healthcare Research and Qualitys (AHRQ) Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey Comparative Database

OMB: 0935-0165

Document [docx]
Download: docx | pdf



SUPPORTING STATEMENT


Part B







Collection of Information for

Agency for Healthcare Research and Quality’s (AHRQ)

Consumer Assessment of Healthcare Providers and Systems (CAHPS)

Health Plan Survey Comparative Database





Version 11-21-2013









Agency of Healthcare Research and Quality (AHRQ)



Table of contents


B. Collections of Information Employing Statistical Methods 1

1. Respondent Universe and Sampling Methods 1

2. Information Collection Procedures 10

3. Methods to Maximize Response Rates 11

4. Tests of Procedures 15

5. Statistical Consultants 15

































B. Collections of Information Employing Statistical Methods


  1. Respondent Universe and Sampling Methods


Universe of health plans and representativeness of the data. While there are many survey vendors that collect Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey data and may maintain a database of their own clients’ data, the Agency for Healthcare Research and Quality (AHRQ) is the only entity that serves as a comprehensive repository of CAHPS survey data. However, the CAHPS Health Plan Database is comprised of data that are voluntarily submitted by health plans that have administered the CAHPS Health Plan Survey and is not a statistically selected sample, nor is it a representative sample of all health plans in the U.S. Voluntary participants include public and private employers, State Medicaid agencies, State Children's Health Insurance Programs (SCHIP), the Centers for Medicare & Medicaid Services (CMS), individual health plans, and the Department of Defense. AHRQ collaborates with CMS and the National Committee for Quality Assurance (NCQA) to combine their CAHPS Health Plan Survey data with the CAHPS Health Plan Database.

The CAHPS Health Plan Survey Database is a voluntary, comprehensive database of CAHPS Health Plan Survey responses gathered directly from Medicaid plans, NCQA and CMS. NCQA is the managed care plan accreditor and requires yearly CAHPS results for accreditation. Medicare conducts a yearly survey of all Medicare plans and provides the CAHPS Health Plan Database these results. Many Medicaid program health plans do not seek NCQA accreditation, but submit instead directly to the CAHPS Health Plan Database.


The CAHPS Health Plan Survey Chartbook contains a section entitled “Data Sources and Comparison Limitations” which outlines the limitations of the data. Because of differences in the source of data submissions to the CAHPS Health Plan Database from year to year, it is not possible to directly compare results for the 2 years. Comparison of results across sectors also should be made with caution, since significant variations may exist in benefit design and other factors that might affect survey responses across sector.


The number of health plans in the U.S. is estimated to be as follows:

  • 689 HMO/POS

  • 451 PPO

  • 2,861 Medicare Advantage and

  • 545 Medicaid plans.

(HMO Source: http://www.statehealthfacts.org/comparemaptable.jsp?ind=348&cat=7. Medicaid Source: http://www.cms.hhs.gov/MedicaidDataSourcesGenInfo/Downloads/mmcer06.pdf, Medicare Source: http://healthplantracker.kff.org/topicresults.jsp?i=64&rt=1).

The AHRQ 2008 CAHPS Health Plan Survey results for 2011 consists of 1,140 health plans covering 517,478 respondents which represents only 25% of the total estimated population of U.S. health plans.

Table B-1 presents the number of commercial, Medicaid, SCHIP, and Medicare survey respondents and health plan submissions included in the CAHPS Health Plan Database for 2010 and 2011. The number of health plan submissions is indicated in parentheses.








Table B-1. Number of Survey Respondents and Health Plan Submissions: 2010 and 2011

Year

Commercial

Medicaid

SCHIP

Medicare

(Version)

Adult

Child

Adult

Child

Child

Adult

2011 (4.0)

168,341 (376)

900 (1)

73,820 (148)

85,003 (129)

26,232 (41)

163,182 (445)

2010 (4.0)

139,156 (288)

1,474 (2)

97,626 (132)

88,694 (132)

N/A

221,120 (431)









Commercial Data: The 4.0 Health Plan Survey results for the commercial sector were obtained from the NCQA, under an agreement between the CAHPS Database and NCQA. Health plans submitting data to NCQA in 2011 were given the option to allow their results to also be included in the CAHPS Health Plan Database. Virtually all health plans submitting to NCQA provided their authorization to have their results included in a data file compiled by NCQA for use by the CAHPS Health Plan Database. The final data file obtained from NCQA included a total of 376 health plan submissions for the adult commercial population. Only 1 health plan submission was included for the 4.0 child commercial population, a number too small for reporting in the chartbook. Commercial results are based on survey data collected from September through June.


Medicaid Data: The 4.0 CAHPS Health Plan Survey results for the Medicaid sector were obtained from data submitted directly to the CAHPS Health Plan Database by State Medicaid agencies and individual health plans. A total of 17 States submitted data in 2011. The total number of plan submissions and respondents for the 4.0 version in 2011 remained consistent with 2007 levels. Medicaid results are based on survey data collected from October through June.


Medicare Data: Each year, the CAHPS Health Plan Database receives the CAHPS Medicare Managed Care survey data collected by the Centers for Medicare & Medicaid Services (CMS). CMS collected survey information from beneficiaries enrolled in managed care health plans that provide a prescription drug benefit. For 2011, 445 managed care plans participated in the survey representing a sizable increase from 2008. Survey participants included both enrollees receiving prescription drug coverage through their health plan and those that don’t receive prescription drug coverage through their health plan. Beneficiaries enrolled in traditional Medicare (Medicare fee-for-service) as well as fee-for-service enrollees who selected a prescription drug plan were also surveyed in 2011, but these data are not represented in the Chartbook. The results in the Chartbook include only beneficiaries who were enrolled in a managed care health plan. The Medicare Survey data are collected from February through June. The Medicare results presented in the CAHPS online reporting system may differ from other reports due to the inclusion or exclusion of certain beneficiary groups and/or the use of case-mix adjustment variables.


Tables B-2 and B-3 present the number of survey respondents and health plan submissions obtained for the 4.0 results within each major sector by State, including U.S. territories and the District of Columbia.

Table B-2. 2011 Survey Respondents and Health Plan Submissions by State (4.0 Results)

State

Commercial Adult

Commercial Child

Medicaid Adult

Medicaid Child

SCHIP

Medicare Adult

Alabama

317 (1)

-

-

-

997 (1)

1,849 (5)

Arizona

997 (2)

-

532 (1)

585 (1)

-

6,268 (18)

Arkansas

917 (2)

-

482 (1)

452 (1)

516 (1)

1,948 (5)

California

14,930 (23)

-

3,030 (5)

3,376 (3)

12,222 (24)

10,526 (32)

Colorado

2,965 (6)

-

1,963 (4)

2,730 (4)

2,833 (5)

3,646 (9)

Connecticut

45,539 (118)

-

982 (2)

966 (1)

-

2,418 (7)

Delaware

1,506 (3)

-

422 (1)

399 (1)

325 (1)

427 (1)

District of Columbia

-

-

439 (1)

1,212 (2)

-

-

Florida

3,472 (8)

-

4,193 (14)

5,036 (15)

-

10,200 (32)

Georgia

7,491 (20)

-

-

-

-

5,030 (14)

Hawaii

4,313 (4)

-

-

3,018 (5)

-

2,157 (6)

Idaho

-

-

-

-

-

1,691 (4)

Illinois

9,825 (24)

-

-

-

-

3,806 (10)

Indiana

1,661 (4)

-

1,568 (3)

1,453 (2)

-

2,639 (6)

Iowa

1,062 (2)

-

-

-

898 (1)

1,881 (4)

Kansas

759 (2)

-

368 (1)

1,312 (1)

1,098 (1)

346 (1)

Kentucky

4,268 (11)

-

-

-

-

1,555 (4)

Louisiana

1,942 (5)

-

-

-

-

2,217 (6)

Maine

1,385 (2)

-

-

-

-

1,997 (4)

Maryland

1,395 (3)

-

3,766 (7)

7,207 (7)

-

2,061 (5)

Massachusetts

4,657 (10)

-

2,475 (5)

-

-

2,977 (9)

Michigan

4,242 (8)

-

6,891 (14)

6,701 (14)

-

4,198 (10)

Minnesota

2,070 (4)

-

10,862 (8)

-

-

5,300 (13)

Mississippi

-

-

-

-

950 (1)

706 (2)

Missouri

2,994 (7)

-

1,345 (3)

4,097 (4)

1,612 (1)

5,164 (13)

Montana

-

-

-

-

-

487 (1)

Nebraska

480 (1)

-

506 (1)

-

-

1,225 (3)

Nevada

832 (2)

-

-

-

-

2,720 (7)

New Hampshire

435 (1)

-

-

-

-

-

New Jersey

1,492 (3)

-

3,804 (14)

5,845 (14)

-

2,665 (9)

New Mexico

2,131 (5)

-

1,796 (4)

2,730 (3)

-

2,408 (7)

New York

8,904 (21)

-

1,437 (3)

-

-

12,881 (37)

North Carolina

881 (2)

-

-

-

-

2,662 (7)

Ohio

5,892 (12)

-

8,025 (11)

9,781 (7)

-

7,331 (19)

Oklahoma

382 (1)

-

-

613 (1)

-

1,913 (5)

Oregon

2,872 (6)

-

6,525 (17)

6,607 (17)

-

6,398 (16)

Pennsylvania

4,471 (10)

-

3,044 (6)

3,426 (6)

904 (1)

7,149 (20)

Puerto Rico

-

-

359 (1)

-

-

4,055 (13)

Rhode Island

-

-

1,091 (2)

594 (1)

-

1,122 (3)

South Carolina

487 (1)

-

972 (2)

1,190 (2)

-

1,452 (4)

South Dakota

416 (1)

-

-

-

-

377 (1)

Tennessee

730 (2)

-

2,694 (6)

6,626 (6)

1,748 (1)

3,535 (10)

Texas

3,122 (9)

-

521 (2)

1,492 (2)

1,543 (2)

7,912 (24)

Utah

791 (2)

900 (1)

429 (1)

-

586 (1)

2,847 (7)

Vermont

1,901 (3)

-

-

-

-

-

Virginia

2,467 (5)

-

2,155 (5)

4,891 (5)

-

1,397 (4)

Washington

3,022 (5)

-

436 (1)

850 (1)

-

4,767 (12)

West Virginia

427 (1)

-

708 (2)

1,074 (2)

-

1,119 (3)

Wisconsin

7,499 (14)

-

-

740 (1)

-

5,753 (13)

Totals

168,341 (376)

900 (1)

73,820 (148)

85,003 (129)

26,232 (41)

163,182 (445)

Table B-3. 2010 Survey Respondents and Health Plan Submissions by State (4.0 Results)

State

Commercial Adult

Commercial Child

Medicaid Adult

Medicaid Child

SCHIP

Medicare Adult

Alabama

380 (1)

-

-

-

-

4,206 (5)

Arizona

1,134 (3)

-

483 (1)

659 (1)

-

7,976 (17)

Arkansas

424 (1)

-

-

-

-

1,944 (4)

California

7,380 (7)

-

18,799 (40)

23,585 (37)

-

21,533 (32)

Colorado

2,983 (6)

-

2,221 (4)

3,411 (5)

-

4,388 (8)

Connecticut

36,716 (85)

-

308 (1)

861 (1)

-

3,732 (8)

Delaware

1,584 (3)

-

425 (1)

869 (2)

-

519 (1)

District of Columbia

-

-

355 (1)

1,261 (2)

-

-

Florida

4,762 (11)

-

4,520 (15)

4,200 (14)

-

19,729 (34)

Georgia

2,066 (5)

-

-

-

-

4,054 (10)

Hawaii

3,299 (4)

-

3,339 (5)

-

-

2,878 (6)

Idaho

-

-

-

-

-

1,180 (3)

Illinois

5,586 (12)

-

-

-

-

4,475 (11)

Indiana

776 (2)

-

314 (1)

527 (1)

-

3,703 (9)

Iowa

1,022 (2)

-

-

1,023 (1)

-

1,592 (4)

Kansas

907 (2)

-

969 (2)

3,105 (3)

-

514 (1)

Kentucky

3,389 (8)

-

-

-

-

1,966 (5)

Louisiana

2,221 (5)

-

-

-

-

3,091 (5)

Maine

869 (1)

-

-

-

-

1,840 (4)

Maryland

2,892 (6)

-

3,973 (7)

7,334 (7)

-

3,207 (7)

Massachusetts

5,658 (10)

1,041 (1)

2,853 (5)

-

-

4,383 (10)

Michigan

4,520 (8)

-

6,705 (14)

-

-

2,588 (6)

Minnesota

1,472 (3)

-

11,565 (8)

-

-

6,288 (15)

Mississippi

-

-

-

-

-

928 (2)

Missouri

1,618 (4)

-

1,527 (4)

5,622 (7)

-

5,684 (13)

Montana

-

-

-

-

-

408 (1)

Nebraska

449 (1)

-

463 (1)

-

-

1,107 (3)

Nevada

-

-

-

-

-

3,839 (8)

New Hampshire

1,000 (3)

-

-

-

-

-

New Jersey

1,131 (3)

-

1,781 (13)

2,530 (13)

-

3,422 (8)

New Mexico

1,863 (4)

-

986 (2)

2,216 (2)

-

2,836 (7)

New York

8,774 (19)

-

12,753 (21)

-

-

16,891 (32)

North Carolina

1,111 (3)

-

-

-

-

3,975 (7)

Ohio

5,088 (10)

-

8,695 (11)

10,702 (7)

-

7,136 (14)

Oklahoma

442 (1)

-

629 (1)

646 (1)

-

2,259 (6)

Oregon

2,876 (6)

-

491 (1)

620 (1)

-

7,236 (16)

Pennsylvania

4,619 (11)

-

4,756 (8)

5,760 (9)

-

13,391 (21)

Puerto Rico

-

-

513 (1)

-

-

8,086 (13)

Rhode Island

-

-

1,121 (2)

-

-

1,246 (3)

South Carolina

529 (1)

-

976 (2)

1,581 (3)

-

2,191 (4)

South Dakota

469 (1)

-

-

-

-

-

Tennessee

1,192 (3)

-

1,143 (3)

3,262 (3)

-

6,333 (10)

Texas

4,708 (10)

-

68 (1)

1,271 (2)

-

13,820 (23)

Utah

418 (1)

433 (1)

-

2,359 (4)

-

1,940 (5)

Virginia

909 (2)

-

1,309 (3)

3,063 (3)

-

1,708 (4)

Washington

5,354 (8)

-

2,353 (4)

930 (1)

-

4,526 (11)

West Virginia

-

-

749 (2)

1,297 (2)

-

1,286 (3)

Wisconsin

6,566 (12)

-

484 (1)

-

-

5,086 (12)

Totals

139,156 (288)

1,474 (2)

97,626 (186)

88,694 (132)

-

221,120 (431)

The CAHPS Health Plan Database currently contains 14 years of data from the CAHPS Health Plan Survey. Table B-4 shows data submissions to the CAHPS Database from 1998 to 2011. The total number of respondents is presented by population sector, with the number of health plan submissions given in parentheses.

Table B-4. Data Submissions to the CAHPS Health Plan Database From 1998-2008

Year
(CAHPS Version)

Commercial

Medicaid

SCHIP

Medicare

Adult

Child

Adult

Child

Child

Adult

2011 (4.0)

168,341 (376)

900 (1)

73,820 (148)

85,003 (129)

26,232 (41)

163,182 (445)

2010 (4.0)

139,156 (288)

1,474 (2)

97,626 (186)

88,694 (132)

N/A

221,120 (431)

2009 (4.0)

179,528 (405)

751 (2)

63,391 (126)

68,697 (107)

N/A

206,647 (405)

2008 (4.0)

174,307

(410)

0

(0)

59,840

(120)

9,755

(29)

0

(0)

207,366

(343)

2008 (3.0)

0

(0)

1,882

(4)

0

(0)

37,347

(64)

0

(0)

0

(0)

2007 (4.0)

106,811

(239)

0

(0)

45,979

(109)

4,647

(16)

0

(0)

115,910

(296)

2007 (3.0)

0

(0)

1,659

(4)

0

(0)

64,039

(103)

0

(0)

0

(0)

2006 (3.0)

124,585

(271)

2,400

(7)

43,174

(119)

50,204

(95)

9,303

(30)

97,955

(273)

2005 (3.0)

123,272

(254)

2,661

(4)

32,115

(76)

40,204

(65)

1,252

(3)

127,930

(276)

2004 (3.0)

111,680
(223)

7,024

(12)

59,515

(149)

86,159

(128)

16,657

(29)

132,420

(288)

2003 (3.0)

114,063
(216)

1,866
(4)

39,275
(112)

31,081
(69)

19,061
(49)

141,421
(295)

2002 (2.0)

94,546
(219)

5,600
(10)

48,109
(136)

60,534
(122)

18,910
(43)

153,172
(321)

2001 (2.0)

165,500
(266)

9,913
(24)

45,127
(142)

36,940
(124)

0
(0)

179,451
(381)

2000 (2.0)

135,479
(270)

2,760
(8)

49,327
(156)

41,400
(140)

0
(0)

166,072
(367)

1999 (2.0)

168,234

(307)

42,879

(149)

28,420

(77)

14,106

(66)

0

(0)

0

(0)

1998 (1.0)

34,965

(54)

0

(0)

23,519

(31)

9,871

(33)

0

(0)

0

(0)

TOTALS

1,840,467

81,769


709,237

728,681

91,415

1,912,646


Most of the CAHPS Health Plan Survey questions ask respondents to report on their experiences with different aspects of their care. These reporting questions are combined into groups that address the same aspect of care or service to arrive at a broader assessment. The 4.0 version of the CAHPS Adult and Child Health Plan Surveys reporting questions fall into four major “composites” that summarize consumer experiences in the following areas: 1) Getting needed care, 2) Getting care quickly, 3) How well doctors communicate, and 4) Health plan information & customer service.


The CAHPS Health Plan Survey collects four separate global ratings to distinguish between important aspects of care. The four questions ask plan enrollees to rate their experiences in the past 6 months with: 1) their personal doctor, 2) the specialist they saw most often, 3) health care received from all doctors and, 4) other health providers. Ratings are scored on a 0 to 10 scale, where 0 is the “worst possible” and 10 is the “best possible.” The ratings are analyzed and presented in the three-part bar chart display used in the CAHPS Health Plan Survey reports: the percentage of respondents who gave a rating of either 0-6, 7-8, or 9-10. This three-part scale is used because testing by the CAHPS team determined that these cut-points improve the ability to discriminate among plans while simplifying the presentation of results.


Weighting. Each item is given equal weight in calculating the composite results for CAHPS. Computationally, this implies calculating the mean of each item within the plan and then taking an unweighted distribution of the item means to obtain the composite mean. Equal weighting follows from the fact that there is no evidence to suggest that any item is more important than another. For example, the number of members who have a personal doctor is likely to be larger than the number of members who receive care from a specialist. Therefore, survey results will likely include more responses for a question related to a personal doctor than for one about a specialist. Despite this difference, the item about specialty care is included in the report or composite with equal weighting because it is regarded as potentially important to every member. Another advantage of equal weighting is that the weights are consistent from year to year as well as across plans within the same year.


Sampling Methodology. The CAHPS Health Plan sampling recommendation is to achieve a minimum of 300 completed responses per plan, with at least a 50-percent response rate. If there are multiple plans in a health plan participant’s portfolio, the recommendation is to draw equal sample sizes from each of the plans, regardless of the size of the plan membership, so as to achieve 300 completed responses. And the plan samples are not adjusted for unequal probabilities of selection. This logic stems from the principle that the precision of the estimates depends primarily on the size of the sample and not on the size of the population from which it is drawn. Therefore, the given sample size will give the same precision for means or rates regardless of the overall size of the population.


Response Rate Calculation. In its simplest form, the response rate is the total number of completed questionnaires divided by the total number of respondents selected. Following CAHPS guidelines, the CAHPS Health Plan Database adjusts response rates according to the following formula:

Number of completed returned questionnaires

Total number of respondents selected – (deceased + ineligible)


In calculating the response rate, the CAHPS Health Plan Database does not exclude respondents who refused, had bad addresses or phone numbers, or were institutionalized or incompetent.


Case-Mix Adjustment. Several methodological problems complicate the measurement and reporting of health care data, particularly when reports draw comparisons among health plans, as in the CAHPS Health Plan Chartbook and the individual participant’s comparative reports. Among these challenges is the need to adjust appropriately for case-mix differences. Case-mix refers to the proportion of enrollees with serious health conditions and other demographic characteristics that have been demonstrated to affect respondents’ reports and ratings of the quality of care received. Case-mix takes into account enrollee characteristics that are not under the control of the plan but may affect measures of outcomes or processes, such as demographic and social characteristics or health status. Many of the CAHPS Health Plan Survey questions ask about aspects of access or processes of care that should not vary by enrollee characteristics. Therefore, case-mix adjustment may be less important for CAHPS Health Plan Survey data than for outcomes of care, which are known to be influenced by enrollee characteristics in a way that is independent of plan performance.


Nonetheless, there are at least two reasons why case-mix adjustment might still be necessary. First, there are certain processes that one would expect to vary according to the characteristics of enrollees. For example, one CAHPS question is "How often did your health plan’s customer service give you the information or help you needed?" Although it is desirable to communicate clearly with all enrollees, it probably is harder to do so with enrollees who have less education than with other enrollees.

Second, enrollee characteristics might influence the response to questions, even if the process of care is the same for different enrollees. For example, individuals' expectations might strongly influence their response to questions asking for evaluations, such as "How often did you get an appointment for your health care at a doctor's office or clinic as soon as you thought you needed?" If an enrollee has very low expectations for the quality of care, he or she might be very satisfied with poor quality. Also, certain types of enrollees may have a general tendency to give positive ratings or have biases that are not associated with the quality of care. For example, some groups of enrollees may generally have more trust and confidence in authority figures and institutions, even if there are no differences in their care.


Testing for Statistical Differences. The individual participant’s comparative reports test for statistically significant differences between mean scores and ratings of individual health plans and the mean of all plan means in the CAHPS Health Plan Database using the t-test. A significance level of 0.05 or less is considered statistically significant. As described in the previous sections, the mean scores are adjusted for case-mix differences before the statistical tests are applied. To compute the means, reports and rating responses are grouped into three categories and assigned a score of 1, 2, or 3. Then, significance tests for both the reports and ratings are conducted on the mean scores. Individual plan results that differ significantly from the overall mean are denoted by arrows, either pointing up (significantly higher than the overall mean) or down (significantly lower than the overall mean).


2. Information Collection Procedures

Information collection for the AHRQ CAHPS Health Plan Survey Database occurs in a regular data collection cycle each year in June/July. The information collection procedure for submitting and processing data for the database is shown in Figure B-1. Each of the steps is described below. Screen shots of each step are provided in Attachment F.

Figure B-1. CAHPS Health Plan Database Data Submission Process

Shape1







Step 1: Call for Data Submission. State Medicaid agencies and health plans that have administered the CAHPS Health Plan survey are recruited through multiple outlets and asked to submit to the database. The call for data submission is done through various publicity sources such as AHRQ’s electronic newsletters, the CAHPS Connection, and communication with prior year’s participants. Organizations that have administered the Health Plan Survey and are interested in participating in the CAHPS Health Plan Database communicate with Westat through a dedicated email address ([email protected]) that routes directly to Westat and a toll-free phone number (1-888-808-7108) to indicate their interest in participation.


Step 2: Complete Registration Form. A database submission extranet has been set up so that interested parties such as, state Medicaid agencies, coalitions, vendors, and health plans register for the data submission process. Information about eligibility requirements, benefits of participation, data use agreement, and data file specifications regarding how to prepare their data file for inclusion in the CAHPS Health Plan Database are posted on the extranet. The data file specifications ensure that data files received from users are standardized and consistent in the way variables are coded and formatted. Potential participants’ online registration information will be reviewed by Westat staff. Upon approval of the registrant, two automated emails that contain separately a username and password to the data submission extranet are sent to the registered user. The automatic email informs registrants of the information needed in the next steps of the data submission process.


Step 3: Complete Health Plan Information Form. This step requires each health plan that administered the CAHPS Health Plan survey, submit the requested characteristics including such details as the name of the plan, product type (e.g., HMO, PPO), the population surveyed (e.g., adult Medicaid, child Medicaid), plan State, total enrollment at time sample frame was generated, mode of survey administration and how the sample was selected.


Step 4: Submit Data Use Agreement. To protect the confidentiality of all respondents and entities that are included in any CAHPS Database, all participating institutions must sign a data use agreement (DUA) that has been reviewed and approved by AHRQ. The data use agreement specifies how the submitted data will be used, provides assurance that the identity of the participating institution will be protected and ensures the confidentiality of the data. Data are not included in the database without this signed data use agreement. Users can fax and/or mail a copy of the signed agreement. Data collection vendors may not sign and submit the DUA on behalf of an institution (even if they have been given permission by the entity to handle the actual submission of data). Only a duly appointed representative may sign the DUA.


Step 5: Upload Questionnaire. Each health plan must upload a copy of the questionnaire used. The CAHPS Database reviews the questionnaire to ensure that it meets CAHPS Health Plan Survey standards (the survey instrument must include all core questions, not alter the wording of any core questions, and must not omit any of the survey items related to respondent characteristics that are used for case mix adjustment.) Once the questionnaire is reviewed by CAHPS Database staff, an email notification is sent to the registrant within three business days with an approval or rejection. Only health plans that receive questionnaire approval may submit data files.


Step 6: Secure Online Data Submission. To enable participants to transmit their CAHPS survey data to Westat in a secure manner, an online data submission extranet has been developed. The online system will be expanded and adapted to include data submission for all CAHPS surveys. Data are accepted in ascii/flat format. Data files must conform to the Data File Layout Specifications provide by the CAHPS Database. Since the unit of analysis is at the health plan level, users must upload one data file per health plan.


Data File Approval. Once a data file is successfully uploaded, a separate load program developed in Visual Basic (VB) reads the submitted files and loads them into the SQL database that stores the data. Upon submission, a data file status report is produced and made available to the participant. This report displays item frequencies and flags out-of-range values. If there are any out of range values or problems with the data file the submitter may review the Data File Status Report for further detail. Participants are expected to fix any errors and resubmit their data file(s) for processing. If the data have been properly received, a CAHPS Database staff member then reviews the report to conduct data quality checks. If any data problems are discovered, users will be notified immediately along with a description of the problem. If there are no problems with the data file the CAHPS Database staff review all aspects of the submission for a account final approval status and an email will be sent to the participant contact via the database submission extranet indicating their data will be included in the CAHPS Health Plan Database.


Step 7: Account Status. Participants have the opportunity to check the status of their account at any time during the submission process. Only accounts that receive the CAHPS Database Final Approval status will be included in the CAHPS Health Plan Database.


3. Methods to Maximize Response Rates


AHRQ promotes the voluntary participation in the CAHPS Health Plan Survey Database using several methods to different target audiences. We continually conduct general marketing through existing CAHPS channels and targeted outreach to existing and previous health plan participants. The CAHPS Database staff also contact national quality initiatives to promote the Database and have sought data partners that result in the yearly data contributions from NCQA and CMS.


Ongoing general marketing includes:


  • CAHPS Database Web Site: Announcements regarding data submission and reporting timetables;

  • CAHPS Database Annual Report and Related Press Release(s). Announcements for release of Annual Report that includes CAHPS Database contact information for plans and purchasers interested in participating;

  • CAHPS User Group Meeting. Plenary session at CAHPS User Group Meetings on use of CAHPS Database data;

  • CAHPS Survey Users Network (SUN). Build the Database into general information products (including CAHPS Web site and CAHPS Implementation Kit) that are disseminated to current and prospective CAHPS survey users; and

  • AHRQ Web Site. Brief summary of CAHPS Database products and benefits of participation, with link to the CAHPS Database website.

On a yearly basis, current and prior participants are contacted by phone and e-mail to determine their plans to participate in the upcoming cycle. These steps include:

  1. Send email memo with data submission and reporting timetables and requirements.

  2. Send email announcement of release of CAHPS Health Plan Survey results with message thanking participants for helping making the CAHPS Health Plan Database possible.

  3. Special Federal participants (DoD, Medicare): Make individual telephone calls to key contacts, to request feedback on individual participants comparative reports (if applicable), identify any special problems or concerns, confirm continued participation, and reinforce and coordinate data submission schedules for upcoming phase.


Attachment G contains the e-mails that are used to solicit participation.


In addition to the direct contact of health plans themselves, the CAHPS Database staff contact many national leaders and programs and direct them to the annual chartbook and references to the AHRQ National Healthcare Quality and National Healthcare Disparities Reports. These organizations and programs often cite and use CAHPS Database information. These include:


National Quality Initiatives
National Forum on Health Care Quality Measurement and Reporting (board members)
Quality Interagency Coordinating Committee (e.g., federal agencies such as HHS, Labor, Defense, Veterans Affairs, Federal Trade Commission, etc.)
Institute of Medicine Quality of Health Care in America Project

Federal and State Health Policy Leaders
Appropriate Federal and State Agency Administrators (including public health)
Federal and State Congressional Staffs
National Governors Association
National Conference of State Legislatures
National Association of Health Data Organizations
National Association for State Health Policy
State Medicaid Directors Association

Consumer Advocacy Groups
American Association of Retired Persons
Consumer Coalition for Quality Health Care
Families USA
Family Voices

Business Leaders on Health
National Business Coalition on Health
Managed Health Care Association
Leapfrog Group
Washington Business Group on Health
Midwest Business Group on Health
National Health Care Purchasers Institute

Health Care Industry Leaders
American Association of Health Plans
Health Insurance Association of America
American Health Quality Association
National Association of Insurance Commissioners

CAHPS Users and Researchers
CAHPS Database Advisory Group and Participants
CAHPS Survey Users Network (SUN)
CAHPS Consortium
CAHPS Advisory Committee
Medicare Managed Care CAHPS Technical Expert Panel
Medicare Health Outcomes Study Technical Expert Panel

Health Policy and Health Services Researchers
Grant Makers in Health
Academy for Health Services Research and Health Policy


4. Tests of Procedures

Every year the CAHPS Database staff provides a training video on the CAHPS Database website for health plans and their survey vendors to view that provides an overview of the submission process with an emphasis on any changes since the previous year.


In addition, each year the CAHPS Database staff talks with submitters about their experience and use their feedback to improve the collection process. A summary of the feedback from 2008 is provided in Attachment H.


5. Statistical Consultants

Ron Hays, PhD

Professor of Medicine

UCLA School of Medicine/RAND

[email protected]


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleSUPPORTING STATEMENT
Authorwcarroll
File Modified0000-00-00
File Created2021-01-28
© 2024 OMB.report | Privacy Policy