SUPPORTING STATEMENT
Part A
Cognitive Testing of the MEPS Cancer SAQ
April 19, 2011
Agency for Healthcare Research and Quality (AHRQ)
2. How, by Whom, and for What Purpose Information Will Be Used 4
3. Use of Improved Information Technology 5
4. Efforts to Identify Duplication 5
6. Consequences if Information Collected Less Frequently 5
8. Federal Register Notice and Outside Consultations 5
9. Payments/Gifts to Respondents 6
10. Assurance of Confidentiality 6
11. Questions of a Sensitive Nature 6
12. Estimates of Annualized Burden Hours and Costs 6
13. Estimates of Annualized Respondent Capital and Maintenance Costs 7
14. Estimates of Annualized Cost to the Government 7
16. Time Schedule, Publication and Analysis Plans 7
17. Exemption for Display of Expiration Date 8
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
Furthermore, AHRQ shall conduct and support research to provide data to improve the quality of health care through the healthcare utilization and expenditure estimates. [Section 912, (b) (2) (A) (ii) (II) and (iii) (II) and (c) (1) (2) and (3) (http://www.ahrq.gov/hrqa99b.htm)].
The Medical Expenditure Panel Survey (MEPS) is a nationally representative survey of the civilian noninstitutionalized population of all ages in the US that collects comprehensive data on health care and health care expenditures from all payors (including private payors, Medicaid, the VA, and out-of-pocket) over a two year period. The MEPS has been conducted annually since 1996. Its current clearance is OMB# 0935-0118 with an expiration date of 01/31/2013.
The purpose of this request is to cognitively test a self administered questionnaire (SAQ) on “Experiences with cancer”. Once the SAQ is finalized it will be added to the MEPS and administered to MEPS participants identified as having had cancer. The target for the administration of this SAQ nationally is February, 2012. A request for modification to the MEPS clearance will be submitted shortly.
There are several benefits to administering this SAQ nationally as a supplement to the MEPS. First, the accompanying oversample of persons with cancer will improve our cost estimates for patients with this disease and will allow us to conduct analysis on the long term costs of cancer for survivors. It will also allow research directed at long term consequences of cancer and overall medical expenses. Finally, this activity will allow us to examine the feasibility of using MEPS as a vehicle for in depth analysis of other specific conditions.
This research has the following goals:
1) Cognitively test and modify as necessary the Survey of Cancer Experiences SAQ;
2) Make the final SAQ available for use in the MEPS.
To achieve these goals the following activities will be implemented:
1) Screener – A short screening questionnaire will be used to recruit eligible persons for the cognitive interviews (Attachment B).
2) Cognitive interviews – Two rounds of in-person cognitive interviews will be conducted, with the second round conducted in both English and Spanish. We will conduct 24 English-language interviews in Round 1 and 20 English-language interviews and 16 Spanish-language interviews in Round 2 using a Spanish-language translation of the finalized English-language questionnaire. See Attachments C, D and E for the cognitive interview guide, draft questionnaire and recruitment advertisement, respectively;
3) Implement the Survey of Cancer Experiences SAQ nationally in the MEPS in February 2012.
The cognitive testing is being conducted by AHRQ through its contractor, Westat, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).
Most, if not all, cognitive testing will be done in a focus group room at Westat’s Rockville, MD location. The focus group area has an observation room that can be used for viewing the testing. The interviews, with respondent written consent, will be observed and audio recorded for later observation, and will take about one hour to complete.
The information collected will be used to test and to support development of the questionnaire. Cognitive testing will be conducted and the questionnaire will be modified to reflect results from the testing. The purpose of these cognitive interviews is to refine the questionnaire’s items and composites. To achieve that purpose, we will ask respondents how they interpret specific words, phrases or items in the questionnaire; what kinds of information they use to help answer the items; how easy or difficult it is for them to answer the items; and how easy or difficult it is for them to navigate through the paper questionnaire. For the Spanish-language interviews, we will also explore respondents’ interpretations of and reactions to the translation. The open-ended and general probes used to get at these issues will be the same for each round, although the questionnaire items they are targeting may be revised between Rounds 1 and 2.
With the respondent’s permission the interviews will be recorded; other than that cognitive interviews cannot benefit from the use of improved information technology.
Some of the questions included in the Survey of Cancer Experiences SAQ have been part of other surveys, such as National Health Interview Survey, NHIS Sample adult - Cancer Control supplement, and Medical Expenditures Panel Survey (Priority Enumeration). Since all cancer survivors will have responses to the NHIS and the MEPS household surveys, information about general health, employment, health insurance, access to care, conditions, and expenditures will already be available. Additionally, for the subset of cancer survivors in the NHIS who complete the sample adult cancer control supplement (only one adult per household), detailed information about cancer screening, family history of cancer, and genetic testing will be available. By combining the data available from these sources, we will have a larger nationally representative sample which should allow us to maximize the research possibilities related to cancer survivorship. Once this SAQ is incorporated into the main MEPS study, an analysis has been planned to use the available data to compare and contrast the two points in time (NHIS cancer control and MEPS SAQ).
No
other cognitive interviewing of the Survey of Cancer Experiences SAQ
has been performed.
5. Involvement of Small Entities
This data collection will not impact small entities.
This effort is a one-time test.
The data collection efforts will be consistent with the guidelines at 5 CFR 1320.5(d)(2).
8. Federal Register Notice and Outside Consultations
8.a. Federal Register Notice
This information collection request is being submitted under AHRQ’s generic pretesting clearance “Questionnaire and Data Collection Testing, Evaluation, and Research for the AHRQ” OMB Control Number 0935-0124 and therefore does not require publication in the Federal Register.
8.b. Outside Consultations
Following individuals outside of AHRQ have been consulted on the design of the pretest questionnaire:
Kathy Virgo, PhD MBA (American Cancer Society)
Donatus Ekwueme, PhD (Centers for Disease Control and Prevention)
Juan Rodriguez MPH (Centers for Disease Control and Prevention)
Wendy Hicks, MS Westat, Inc.
Gordon Willis, PhD (National Cancer Institute)
Robin Yabroff, PhD, MBA (National Cancer Institute)
To successfully recruit 60 cognitive interview participants, it is appropriate to offer a cash incentive. In keeping with the current standard amount provided to general population respondents for one-hour interviews in the metropolitan D.C. area, we propose a $50 cash remuneration.
Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose. Participants will be asked to sign an informed consent form (Attachment F).
We do not believe there are questions of a particularly sensitive nature included in the survey, but if during cognitive testing sensitivities are discovered, such questions will be modified to ensure they are not of a sensitive nature.
Exhibit 1 shows the estimated annualized burden hours for respondents’ time to participate in this research. The screener questionnaire will be administered to about 100 persons and takes 8 minutes to complete. Two rounds of in-person cognitive interviews, lasting one hour and 15 minutes in length, will be conducted with a total of 60 respondents. This estimate includes about 20 minutes to complete the SAQ. Round one will include 24 persons and round 2 will include 36 persons. It is possible that the round 2 interview instrument will be slightly different from the round one instrument based on results from round one. What those changes may be cannot be known at this time; therefore only the round one instrument can be included here. We do not expect changes to the round 2 instrument, if any, to significantly change the burden estimates in Exhibits 1 and 2 below. The total annualized burden is estimated to be 88 hours.
Exhibit 2 shows the estimated annualized cost burden associated with respondents’ time to participate in this research. The total cost burden is estimated to be $1,879 annually.
Exhibit 1: Estimated annualized burden hours
Activity |
Number of respondents |
Number of responses per respondent |
Hours per response |
Total burden hours |
Screener Questionnaire |
100 |
1 |
8/60 |
13 |
Cognitive test of the Survey of Cancer Experiences SAQ |
60 |
1 |
1.25 |
75 |
Total |
160 |
n/a |
n/a |
88 |
Exhibit 2: Estimated annualized cost burden
Activity |
Number of respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Screener Questionnaire |
100 |
13 |
$21.35 |
$278 |
Cognitive test of the Survey of Cancer Experiences SAQ |
60 |
75 |
$21.35 |
$1,601 |
Total |
160 |
88 |
n/a |
$1,879 |
*Based on the mean average hourly rate for all occupations (00-0000), National Compensation Survey: Occupational Wages in the United States May 2010, “U.S. Department of labor, Bureau of Labor Statistics”.
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. The only cost to the respondent will be that associated with their time to respond to the information collection, as shown in Exhibits 1 and 2.
Exhibit 3 shows the estimated total cost for the pretest, which will last for less than one year. The total cost for this project is approximately $42, 500.
Exhibit 3. Estimated Total and Annualized Cost
Cost Component |
Total Cost |
Annualized Cost |
Project Development |
$25,000 |
$25,000 |
Data Collection Activities |
$7,500 |
$7,500 |
Analysis |
$5,000 |
$5,000 |
Reformatting of SAQ |
$5,000 |
$5,000 |
Total |
$42,500 |
$42,500 |
This is a new activity.
As soon as OMB approval is received, cognitive interviewing activities will begin. The estimated time schedule to conduct the interviews is shown below:
May 23-27, 2011 – Round 1 of cognitive interviewing
July 1, 2011 – Report on Round 1 cognitive interviews and recommendations for questionnaire changes
July 18-22, 2011 – Round 2 of cognitive interviewing in English and Spanish
September 1 – Report on Round 2 of cognitive interviews and recommendations for final version of questionnaire
Data from the cognitive testing will consist of notes interviewers will take during each interview, audio recordings of each interview, respondent answers to the survey questions, and the detailed summaries interviewers will produce after each interview. Interviewers will use their notes and the recording of the interview to write the detailed summaries. Following each round of testing, the lead analyst will review the detailed summaries and use qualitative data reduction methods to develop item-by-item descriptions of key results, within and across the respondent characteristics of interest. Based on evidence from those results, the lead analyst will identify problems and the survey development team will identify revisions to address each problem. A report will be produced at the end of each round of cognitive testing that consists of a brief description of methods, including a table of respondent characteristics and any noteworthy issues related to protocol development; a summary of key findings and recommendations; and, if appropriate, a more detailed presentation of section-by-section or issue-by-issue results. Recommendations will include those for improving wording and presentation of the key survey concepts, as well as for layout, formatting and design.
No exemption is being requested.
List of Attachments
Attachment B – Screener Questionnaire
Attachment C – Cognitive Interview Guide
Attachment D – Draft Cancer SAQ
Attachment E – Recruitment Advertisement
Attachment F – Informed Consent Form
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | SUPPORTING STATEMENT |
| Author | AHCPR |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |