Supporting Statement
Part A
Field Test of Low Literacy Version of CAHPS Clinician & Group Survey - 0935-0124
December 20, 2018
Agency for Healthcare Research and Quality (AHRQ)
TABLE OF ContentS
SUPPORTING STATEMENT: FIELD TEST OF LOW LITERACY VERSION OF CAHPS CLINICIAN & GROUP SURVEY 1
A1. Necessity of Information Collection 1
A2. Purpose and Use of Information 2
A3. Technological Collection Techniques 2
A5. Impact on Small Businesses 3
A6. Consequences of Less Frequent Data Collection 3
A8. CMS Federal Register Notice 3
A9. Respondent Payments or Gifts 3
A10. Assurance of Confidentiality 3
A12. Burden of Information Collection 4
A14. Estimates of Annualized Cost to the Government 5
A15. Program Changes or Adjustments to Annual Burden 5
The Agency for Healthcare Research and Quality (AHRQ) requests clearance from the Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995 to implement the field test of the survey entitled “Field Test of Low Literacy Version of CAHPS Clinician & Group Survey.”
Producing evidence to make health care safer, higher quality, more accessible, equitable, and affordable is part of AHRQ’s mission. As a program sponsored by AHRQ, the Consumer Assessment of Healthcare Providers and Systems (CAHPS) is purposed with advancing the scientific understanding of the patient’s experience of care, including the development and testing of new surveys and/or approaches to data collection to promote or improve the collection of consumer reports and evaluations of their experiences with health care.
The proposed field test of a low literacy version of CAHPS Clinician & Group Survey (CG-CAHPS) fulfills AHRQ’s mission by testing a modified version of CG-CAHPS designed to be more accessible to individuals with limited education and/or reading skills. The field test furthers the purpose of CAHPS by conducting a scientifically designed test to determine if the proposed low literacy CG-CAHPS Survey (CG-CAHPS EZ) increases survey response rates, and to assess comparability of the resulting CG-CAHPS EZ composite scores to composite scores generated using the standard CG-CAHPS survey (version 3.0).
The CAHPS program develops standardized surveys to measure what happened, or how often something happened, in a health care encounter. The surveys ask questions for which patients are the best or sometimes the only source of the information. The CAHPS surveys reflect the patient’s voice and include questions that reflect what they’ve stated is important to measure when seeking high quality health care. The CAHPS surveys are the gold standard for patient experience of care measurement.
The CAHPS suite of surveys is extensive and includes surveys covering health care facilities (hospital, nursing home, in-center hemodialysis); providers (clinician and group practices; hospices); health care plans (commercial, Medicaid, Medicare); and condition-specific care (mental health, cancer). CAHPS survey data are used as a component of provider quality payments, including pay-for-reporting by ACOs under the Medicare Shared Savings Program, hospital value-based purchasing payments, quality bonus payments for Medicare Advantage Plans, and dialysis center value-based purchasing payments. Given the widespread use of CAHPS surveys such as the Clinician & Group Survey in payment and recognition programs, it is important that they are accessible to the broadest range of patients including those with limited education and/or reading skills.
The current CG-CAHPS Survey is at a 10th grade reading level. To promote survey participation from individuals with limited literacy, we propose to test CG-CAHPS EZ, a modified CG-CAHPS survey, which is at a 5th to 6th grade reading level.
CG-CAHPS EZ was designed using plain language principles and two new methods to simplify the CG-CAHPS survey: grammatic parsing and stanzaic versification. Grammatic parsing identifies conjunctives (connector words within sentences) and parses compound and complex sentences into their grammatical components (phrases and clauses); stanzaic versification converts simple survey sentences into 2 or 3 shorter lines to create a survey akin to a stanza in poetry. Each line of the survey stanza, usually a phrase or clause, represents a single idea. Unlike the unsystematic truncation produced by automatic text wrapping from word processing programs, stanzaic versification breaks are designed to represent a single idea.
This data collection effort is a one-time field test to be completed in 2019. The field test will randomize consumers receiving care from a large community health network operating in southern California (AltaMed) to receive the standard CG-CAHPS or CG-CAHPS EZ for the following purposes:
Assess overall response rate. We will assess and compare the overall response rate for each survey version to determine if the response rates are comparable, whether either survey version achieves a response rate that is lower or higher than the other, and to test any difference in overall response rates for statistical significance. In addition, we will compare the characteristics of the consumers who respond to each survey version about self-reported race/ethnicity, education level, and main language spoken at home.
Assess item-level response rates. We will review item-level response rates to assess the extent of missing data for each survey item and evaluate whether either survey version results in lower or higher rates of item-level missing data than the other.
Psychometric analysis. We will compare correlations among items in the same multi-item composite, internal consistency and physician-level reliability of composites, assess correlations between reports of experience and overall ratings of care, and conduct other analyses to provide information to assess the psychometric performance of CG-CAHPS EZ in comparison to CG-CAHPS.
Comparison of CAHPS scores. We will compare the composite scores generated from CG-CAHPS EZ and CG-CAHPS and correlations among the scores for the two instruments at the physician-level.
The result of these analyses will inform making about revision to and further use of CG-CAHPS EZ, and whether CG-CAHPS EZ should be included in the suite of CAHPS surveys.
The field test will be conducted using mail survey mode, a standard mode for the conduct of CAHPS surveys.
No survey instrument comparable to CG-CAHPS EZ is currently in use and the proposed data collection is designed to test performance of the CG-CAHPS EZ survey instrument compared to CG-CAHPS. The proposed information collection does not duplicate any other effort and the information cannot be obtained from any other source.
Survey respondents are consumers of health care delivered by practices and clinics within the AltaMed community health network, and the field test will employ a standard CAHPS mail survey protocol which is designed to minimize burden on survey respondents. Small businesses or other small entities will be not significantly impacted by the field test.
This is a one-time data field test.
There are no special circumstances associated with this information collection request.
This proposed information collection is being submitted under AHRQ’s generic clearance (OMB No. 0935-0124). Therefore, publication in the Federal Register is not required.
This data collection does not include any respondent payments or gifts.
The surveys will be printed with the following confidentiality statement:
Your Privacy is Protected. All information that would let someone identify you will be kept private. RAND will not share your personal information with anyone without your permission.
Your Participation is Voluntary. You may choose to answer this survey or not. If you choose not to answer the survey, it will not affect the health care you get.
Your Doctors Won’t See Your Survey. Your answers to this survey are confidential. RAND will not show your answers to your doctors.
Informed consent from survey respondents will be implied by their completion of the mail survey. Consumers in the survey sample will receive additional informed consent and confidentiality information via the letters that accompany the mail survey (See Attachment A for English language survey materials, and Attachment B for Spanish language survey materials)
The study will have a Data Safeguarding Plan to further ensure the privacy of the information that is collected. In addition, RAND will:
Assign a data identifier (ID) to each sampled consumer.
Store all electronic files directly related to the administration of the survey on a restricted drive of a secure local area network. Access to data is limited to those employees working on the project.
Store files containing survey response data separate from files containing sampled consumers’ individual identities. No single file will contain both an individual consumer’s survey responses and his or her contact information.
Destroy consumer contact information once all survey data are collected and the associated data files are reviewed and finalized by the project team.
Transmit and store data files and files containing contact information for data collection following procedures reviewed and approved by RAND’s Institutional Review Board.
RAND will execute a data use agreement with AltaMed to receive consumer contact and provider information as part of the sample. (See Part B for more details of survey procedures.)
The survey does not include any questions of a sensitive nature.
Table 1 shows the estimated annualized burden and cost for survey respondents' time to participate in this data collection.
The length of the CG-CAHPS EZ Survey is estimated at .11 hours (6.89 minutes) and the length of the standard CG-CAHPS Survey is estimated at .12 hours (7 minutes). The estimates are based on tests of the survey conducted on nine or fewer persons and on prior work that established guidelines for estimating survey completion time (Berry, 2009).
The May 2017 National Employment and Wage Estimates reported by the Bureau of Labor statistics indicate an average hourly wage of $24.34 across the 50 U.S. states and the District of Columbia. We have used the national average to estimate the wages of survey respondents. The population served by AltaMed is primarily blue collar and/or publicly insured thus a national average may be a conservative estimate of the wages of survey respondents.
Table 1. Estimated annualized burden hours and cost
Collection Task |
Number of Respondents |
Number of Responses per Respondent |
Hours per Response |
Total Burden hours |
Average Hourly Wage Rate* |
Total Cost Burden |
CG-CAHPS EZ |
400 |
1 |
.11 |
44 |
$24.34 |
$1,071 |
CG-CAHPS |
400 |
1 |
.12 |
48 |
$24.34 |
$1,168 |
Totals |
800 |
1 |
.11 |
92 |
|
$2,239 |
*Based upon mean hourly wage, “May 2017 National Occupational Employment and Wage Estimates United States,” U.S. Department of Labor, Bureau of Labor Statistics, retrieved at https://www.bls.gov/oes/current/oes_nat.htm
As indicated in Table 1 above the annual burden hours are estimated to be 92 hours in total for 400 consumers responding to CG-CAHPS EZ Survey and 400 patients responding to the standard CG-CAHPS Survey.
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
Information collections conducted under this generic clearance will in some cases be carried out under contract. Assuming one data collection per year at an average cost of $106,484 total contract costs could be $106,484 per year. An additional annual cost of about $25,392 for agency staff would be associated with this assuming one per fiscal year with one GS15/5 project officer and one GS13/5 program analyst ($152,760 and $109,900 annual salary*) and 5% of their time each fiscal year. The total annual cost to the government is estimated to be $705,392.
*Based on 2018 OPM Pay Schedule for Washington/DC area: https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2018/DCB.pdf
This is a new information collection request.
We will not generate or publish population estimates from this field test. We will publish methodological findings further the purpose of CAHPS to advance the scientific understanding of the patient experience. Our methodological findings will be generated from the analysis of the data collected using CG-CAHPS EZ in comparison to CG-CAHPS. We will test the significance of difference in survey response rates, item missing data rates, internal consistency reliability and between physician reliability between CG-CAHPS EZ and CG-CAHPS using between group t-tests and chi-square as appropriate. We will also estimate product-moment and intraclass correlations between physician-level scores estimated using the CG-CAHPS EZ and CG-CAHPS surveys.
For planning purposes, we anticipate data collection will begin as early as March 2019 and no later than May 2019. Table 2 provides an approximate timeline for survey tasks including tabulation of data following review and approval of this field test.
Table 2: Time Schedule of Survey Tasks
Activity |
Proposed Timing of Activity |
Prepare field materials |
Months 1 and 2 |
Identify target respondent |
Months 2 and 3 |
Field surveys |
Months 4 through 6 |
Analyze data |
Months 7 and 8 |
Revise Survey |
Month 9 |
The expiration date for OMB approval of this information collection will be displayed on the survey.
Berry, S. (2009) How To Estimate Questionnaire Administration Time Before Pretesting: An Interactive Spreadsheet Approach. Survey Practice, Vol 2(3).
Attachment A – English Language Survey and Survey Cover Letters
Attachment B – Spanish Language Survey and Survey Cover Letters
| File Type | application/msword |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |