Supporting Statement Part A

The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:

1. research that develops and presents scientific evidence regarding all aspects of health care; and

2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and

3. initiatives to advance private and public efforts to improve health care quality.

Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.

Furthermore, AHRQ shall conduct and support research to provide data to improve the quality of health care through the healthcare utilization and expenditure estimates. [Section 912, (b) (2) (A) (ii) (II) and (iii) (II) and (c) (1) (2) and (3) (http://www.ahrq.gov/hrqa99b.htm)].

In support of this mission AHRQ sponsor the Medical Expenditure Panel Survey (MEPS). MEPS is a nationally representative survey of the civilian non-institutionalized population of all ages in the U.S.A. that collects comprehensive data on healthcare coverage and expenditures from all payors (including private payors, Medicaid, the VA, and out-of-pocket) over a two year period. The MEPS Household Component collects data on health insurance use, cost, and coverage directly from individuals and supplements this with information from medical providers. Participants in the MEPS Household Component are drawn from a subsample of households that participated in the National Health Interview Survey (NHIS), conducted by the National Center for Health Statistics. The MEPS has been conducted annually since 1996. Its current clearance is OMB# 0935-0118 with an expiration date of 12/31/2018.

Following a preliminary contact interview, MEPS respondents complete five rounds of interviews over two years. One adult in the household aged 18 or over is responsible for answering about all others in a Respondent Unit (RU) who have a direct or indirect family relation, including children. After the Round 1, coverage is established by asking if that reported in the earlier round(s) is still applicable or when it ended, and identifying any new coverage. The combined data provide information on health insurance coverage over the two-year time period.

Comparison with administrative data and similar nationally-representative surveys suggests that MEPS undercounts certain categories of health insurance coverage. As part of the 2018 redesign, MEPS developers attempted to address question design elements that may be leading to an undercount by including a health insurance verification module, among other changes. The goal of the new health insurance verification module is to capture those who may have been misreported as uninsured in the first pass at health insurance questions within a round, without eliciting duplication or over-reporting.

The purpose of this request is to cognitively pre-test the interviewer-administered 2018 health insurance verification module of questions, with a view to making any changes or adaptions to the questioning for administration in 2019 MEPS.

All data and estimates produced by surveys derive from questions. Pretesting of the survey questions is a credible way to ensure that questions accurately collect data consistently across respondent groups, in accordance with the objectives of the research. Pretesting ensures that a survey respondent is able to complete the survey questions easily and productively, and that the design of the questions, as well as the amount of time taken to complete them, does not cause unnecessary burden. Cognitive testing is a qualitative method routinely used in survey question pretesting. In cognitive testing, one-on-one interviews are conducted with a small-scale purposive sample of respondents chosen to reflect, as far as possible, the population under investigation. Cognitive interviewing techniques are designed to be used to determine if respondents understood specific terms and concepts; whether response categories were appropriate or complete, and whether questions were unambiguous and easily answered by respondents. This approach enables researchers to identify potential problems with questions that could be prone to measurement error when administered in a quantitative survey.

The cognitive test described below complies with OMB guidance on cognitive testing as described in Statistical Policy Directive No. 2: Standards and Guidelines for Statistical Surveys; Addendum: Standards and Guidelines for Cognitive Interviews, dated October 12, 2016. An iterative approach to the cognitive testing will be taken as recommended in OMB cognitive testing guideline A.3.1., following a test – modification – retest approach over a possible three rounds.

This research has the following goal:

1) Cognitively test and modify as necessary the MEPS health insurance verification module;

To achieve this goal the following activities will be implemented:

1) Pre-screener – A short screening questionnaire will be used to recruit eligible persons for the cognitive interviews (Attachment B).

2) Cognitive interviews – Three rounds of in-person cognitive interviews will be conducted with a total of 44 respondents to test the 2018 health insurance verification module. During rounds one and two we will conduct 16 cognitive interviews. Round 3 will be optional dependent on findings from the previous rounds. As such, we will conduct 12 cognitive interviews in round three. See Attachments C1 – C3 for the cognitive interview guide and questionnaire, and Attachments D1 – D5 for the recruitment advertisements.

The cognitive testing is being conducted by AHRQ through its contractor, Westat, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).

The cognitive interviewing will be conducted at Westat’s cognitive testing facility located at its Rockville, MD location. The facility area has an observation room that can be used for viewing the testing. Respondents will be selected according to specific sampling criteria which will include respondents with multiple attributes. The interviews, with respondent written consent, will be observed and audio recorded, and will take about one hour to complete. The consent form is provided in Attachment E.

2. Purpose and Use of Information

The information collected will be used to test and to support development of the health insurance verification module. Cognitive testing will be conducted and the questionnaire will be modified to reflect results from the testing. The purpose of these cognitive interviews is to refine the questionnaire’s items and composites. To achieve that purpose, we will ask respondents how they interpret specific words, phrases or items in the questionnaire; what kinds of information they use to help answer the items; and how easy or difficult it is for them to answer the items for themselves and on behalf of others in their RU. The open-ended and general probes used to get at these issues will be the same or similar for each round, although the questionnaire items they are targeting may be revised somewhat between rounds.

3. Use of Improved Information Technology

With the respondent’s permission the interviews will be audio recorded using a small digital recording device.

4. Efforts to Identify Duplication

Indeed, data on health insurance coverage in the United States are collected by several major federal government-sponsored surveys, in addition to MEPS and NHIS. These include the Survey of Income and Program Participation (SIPP), the Current Population Survey Annual Social and Economic Supplement (referred to as CPS), and the American Community Survey (ACS), all sponsored by the U.S. Census Bureau. While differences in the overall methods for implementing each of these surveys can lead to variations in the estimates they produce, there is general agreement in the literature that the approach to the administration of the questioning about health insurance can contribute to differences in the proportion of the population in each survey that is reported to be without health insurance. Each of the surveys employs different questionnaire design strategies for collecting information on rates of the insured and uninsured.

Although the NHIS and the CPS have introduced a single health insurance verification question, which has been cognitively tested, no one agency has attempted to test a health insurance verification module of questions. As such, this MEPS pretesting task will not only help to ensure that the questioning provides good data but the testing will add to the current body of literature on ways to ensure the accuracy of health insurance questioning and ability to reduce measurement error in this regard. No other cognitive testing of the MEPS health insurance verification module has been performed.

5. Involvement of Small Entities

This data collection will not impact small entities.

6. Consequences if Information Collected Less Frequently

This effort is a one-time test.

7. Special Circumstances

The data collection efforts will be consistent with the guidelines at 5 CFR 1320.5(d)(2).

8. Federal Register Notice and Outside Consultations

8.a. Federal Register Notice

This information collection request is being submitted under AHRQ’s generic pretesting clearance “Questionnaire and Data Collection Testing, Evaluation, and Research for the AHRQ” OMB Control Number 0935-0124 and therefore does not require publication in the Federal Register.

8.b. Outside Consultations

Following individuals outside of AHRQ have been consulted on the design of the cognitive interviewing protocol:

Wendy Hicks, Westat, Inc.

Angie Kistler, Westat, Inc.

Martha Stapleton, Westat, Inc.

Amanda Wilmot, Westat, Inc.

Karen Stein, Westat, Inc.

Heather Brotsos, Summit

Oswaldo Urdapilleta,Summit

9. Payments/Gifts to Respondents

Cognitive interview respondents are identified based on their relationship or experiences with key study characteristics. In order to successfully fulfill the required numbers in the required subgroups and to allow for meaningful analysis on sub-groups of the population, sufficient numbers must be recruited in those subgroups (OMB cognitive testing guideline A.2.1). To successfully recruit 44 cognitive interview participants from the various sub-groups required for the study within a reasonable timeframe, it is appropriate to offer a cash incentive. Keeping in mind the recruitment effort required to fill effectively hard-to-reach small sub-groups for this test and effectively recruit respondents for one-hour interviews in the metropolitan D.C. area, we propose a $40 cash remuneration. This will be paid as a thank you for taking part, out of respect for people’s time, and more importantly to cover any out of pocket expenses involved such as childcare, travel costs involved in getting to Westat, and other incidentals. The amount offered is less that that generally offered by local market research establishments for a one-to-one interview. Although this can make recruitment for such studies more difficult, we hope that the fact that this is a federally funded study on a salient topic will be a motivation for respondents to take part. Westat have found that lower incentive amounts can lead to an increase in the number of respondents who cancel at short notice or do not show for interview. This in turn adds cost to the project in finding replacements and means that replacement respondents are provided with very short notice to attend. The incentive amount proposed offsets potential increased recruitment costs to the project and added burden on members of the public.

10. Assurance of Confidentiality

IRB approval for the cognitive testing study will be obtained prior to starting any recruitment. Respondents will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). All those taking part will be aged 18 years or older. They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose. Participants will be asked for their written consent before participating in any interview and for audio recording the interview. The cognitive interviewer will review the consent form with respondents and offer to answer any questions they may have about participation. All participants will be asked to sign the form prior to starting the cognitive interview.

Information that can directly identify respondents is collected as part of the screening process so that they can be contacted in order to schedule the cognitive interviews. This information is stored separately from any other information provided either at the screening interview or the cognitive interview, for which respondents are allocated an ID number only.

11. Questions of a Sensitive Nature

We do not believe there are questions of a particularly sensitive nature included in the study.

12. Estimates of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated annualized burden hours for respondents’ time to participate in this research. We anticipate that the screener questionnaire will need to be administered to a high number of respondents due to the select sampling criteria being used for this study. The screener questionnaire will be administered to about 400 persons and takes 5 minutes to complete. Three rounds of in-person cognitive interviews, lasting one hour in length, will be conducted with a total of 44 respondents. The total annualized burden is estimated to be 77 hours. Exhibit 2 shows the estimated annualized cost burden associated with respondents’ time to participate in this research. The total cost burden is estimated to be $1,837. The estimates for the hourly wage of adult participants are based on the national median hourly estimate for all occupations reported in the Bureau of Labor Statistics’ Occupational Employment Statistics, May 2016 National Occupational Employment and Wage Estimates United States. (See http://www.bls.gov/oes/current/oes_nat.htm.)

Exhibit 1: Estimated annualized burden hours

Activity	Number of respondents	Number of responses per respondent	Hours per response*	Total burden hours
Screener Questionnaire	400	1	5/60	33
Cognitive test of the MEPS Health Verification Module	44	1	1	44
Total	444	n/a	n/a	77

* Time is an average response per respondent

Exhibit 2: Estimated annualized cost burden

Activity	Number of respondents	Total burden hours	Average hourly wage rate*	Total cost burden
Screener Questionnaire	400	33	$23.86	$787
Cognitive test of the MEPS Health Verification Module	44	44	$23.86	$1,050
Total	444	77	n/a	$1,837

*Based upon the mean hourly wage for Compensation, Benefits, and Job Analysis Specialists occupation code 00-0000, at https://www.bls.gov/oes/current/oes_nat.htm (U.S. Department of Labor, Bureau of Labor Statistics.)

13. Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. The only cost to the respondent will be that associated with their time to respond to the information collection as shown in Exhibits 1 and 2, and any travel or other incidental expenses incurred in order to attend the cognitive interview, which will vary for each respondent according to personal circumstances.

14. Estimates of Annualized Cost to the Government

Exhibit 3 shows the estimated total cost for the pretest, which will last for 15 months. The total cost for this project is approximately $220,337

Exhibit 3. Estimated Total and Annualized Cost

Cost Component	Total Cost	Annualized Cost
Project Development	$94,409	$75,528
Data Collection Activities	$68,154	$54,516
Analysis and reporting	$57,774	$46,224
Total	$220,337	$176,268

Exhibit 4: Annual Cost to AHRQ for MEPS-HC Oversight

Personnel	Staff Count	Hourly Rate	% of Time	Cost
RS	1	$62.38	13.5%	$21,958
RS	1	$53.17	20.0%	$27,702
Grade 14	1	$36.66	15.0%	$14,297
Grade 12	1	$26.07	14.0%	$9,385
SPS Hourly	3	$12.00		$14,400
Total				$73,342

https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2017/DCB.pdf

15. Change in Burden

This is a new activity.

16. Time Schedule, Publication and Analysis Plans

As soon as OMB approval is received, cognitive interviewing activities will begin. The estimated time schedule to conduct the interviews is shown below:

May 1, 2017 – Round one recruitment begins
May 12 – 26, 2017 – Round one of cognitive interviewing
June 30, 2017 – Report on round one cognitive interviews and recommendations for questionnaire changes finalized
July 10, 2017 – Round two recruitment begins
July 24 – August 4, 2017 – Round two of cognitive interviewing
September 8, 2017 – Report on round two of cognitive interviews and recommendations for questionnaire changes finalized
September 18, 2017 – Round three recruitment begins
September 25 – October 3, 2017 – Round three of cognitive interviewing
November 16, 2017 – Full report on all three rounds of cognitive interviews and recommendations for questionnaire changes finalized

In accordance with OMB cognitive testing standard A.4., we will take a systematic approach to the analysis. The cognitive interview data will be analyzed using an approach based on the Framework Method. Framework is a matrix-based approach for managing qualitative data that allows for both case and theme based analysis of the data set. The tested questionnaire items and associated probes will provide the framework for written interview summaries. Interviewers will prepare summaries of each interview based on the completed questions and associated audio recordings. Transcriptions of the interviews will not be made. Summaries will be entered into a series of grids; each row representing a single participant, and each column an area of investigation. This analysis will guide any recommendations.

After each round of testing recommendations for revising and improving any problematic questioning will be made. Findings to support each recommendation will be briefly summarized. A final report will be written consolidating the findings from each round of cognitive testing. The report will detail the methodology used and contain question-specific testing results. An executive summary of the final report will also be provided.

17. Exemption for Display of Expiration Date

No exemption is being requested.

List of Attachments

Attachment A – Agency for Healthcare Research and Quality (AHRQ) Mission

Attachment B – Screening Questionnaire

Attachment C1 - Cognitive Interview Guide and Questionnaire

Attachment C2 – Cognitive Interview Show Cards

Attachment C3 – Cognitive Interview Enumeration Template

Attachment D1 – Thank You (General Population)

Attachment D2 – Thank You (U.S. Armed Services)

Attachment D3 – Thank You (Medical Assistance)

Attachment D4 – Thank You (Affordable Care Act)

Attachment D5 – Invitation/Recruitment Letter

Attachment E – Respondent consent form

Attachment F – OMB Cover Letter

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	SUPPORTING STATEMENT
Author	AHCPR
File Modified	0000-00-00
File Created	2021-01-22

Supporting Statement Part A