SS Part B - MEPS-HC MPC - Economic Burden SAQ - 06122024

SUPPORTING STATEMENT

Part B

Medical Expenditure Panel Survey (MEPS) Household Component and the MEPS Medical Provider Component

Version: March 14^th, 2024

Revision of Previously Approved Information Collection

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent Universe and Sampling Methods 3

2. Data Collection Procedures 8

3. Methods to Maximize Response Rates 10

4. Weighting and Nonresponse Adjustment 14

5. Tests of Procedures 15

6. Statistical Consultants 16

B. Collections of Information Employing Statistical Methods

To fill in major data gaps identified by the Department of Health and Human Services, the Medical Expenditure Panel Survey (MEPS) is specified as a continuous survey. Each year, a new nationally representative MEPS sample will be selected from a subset of households that participated in the prior year’s National Health Interview Survey (NHIS). A preliminary contact with the NHIS responding households selected for the MEPS study will take place to announce the MEPS survey and introduce records keeping activities.

The MEPS was initiated in 1996. Each year a new panel of sample households is selected. Recent annual MEPS-HC sample sizes average about 13,500 households. Data can be analyzed at either the person, family, or event level. The panel design of the survey, which includes 5 Rounds of interviews covering 2 full calendar years, provides data for examining person level changes in selected variables such as expenditures, health insurance coverage, and health status Using a combination of computer assisted personal interviewing (CAPI), computer assisted video interviewing (CAVI), and self-administered paper and web questionnaires, information about each household member is collected, and the survey builds on this information from interview to interview. CAVI is a new data collection technology and offers the best of both telephone and in-person interviewing, while offering opportunities for cost savings and more accurate reporting.

1. Respondent Universe and Sampling Methods

Household Component

The initial MEPS Household Component (HC) sample consists of households that responded to the prior year’s NHIS. The basic analysis unit in the MEPS is defined as the person.

A Reporting Unit (RU) is a person or group of persons in the sampled dwelling unit that are related by blood, marriage, adoption or other family associations and for whom data are to be collected during the MEPS-HC interview. Typically, one adult family member provides information for the entire family although all adult family members are encouraged to participate. Each year’s MEPS-HC sample will be surveyed to collect annual data for two consecutive years. Each new MEPS-HC sample will be a randomly selected subsample of households that responded to the prior year’s NHIS. The NHIS is based on a stratified cluster sample design (see http://www.cdc.gov/nchs/nhis/about_nhis.htm for more information about the NHIS).

Beginning in 2016 the NCHS implemented a new sample design for the NHIS. From a broad perspective, the new design is similar to the previous design because clusters of households are still selected within PSUs which are still essentially formed at the county level. However, within sampled PSUs, the clusters of addresses (households) that are sampled under the new design are not in the form of segments as in the previous design due to utilization of an address-based list of households. Also, the new design used each of the 50 states as well as the District of Columbia as explicit strata with oversampling some of the smaller states with the intent of providing the capability of state-level NHIS estimates. However, the precision of estimates at the national level is expected to be very similar to the previous design. Although the PSUs in the new design were selected independently from the previous design, all large PSUs are selected in the sample with certainty in both designs which should contribute to the efficiency for trend analysis across designs. Moreover, the MEPS sample will continue to have the overlapping panels that will also be a major contributor to the efficiency of year to year estimates of change.

Another notable difference is that the current design does not involve oversampling of any minority group (although it may in the future) that will reduce the number of minorities in the MEPS. However, NHIS responding households with members who are Hispanics, black, or Asian will continue to be oversampled to maximize minority sample sizes in the MEPS-HC. See https://meps.ahrq.gov/data_stats/download_data/pufs/h233/h233doc.pdf for more detailed information about the MEPS-HC.

Beginning in 2019, NCHS implemented a new questionnaire for the NHIS. Under the redesigned questionnaire, only a few basic items will be collected about all household members (e.g., age, sex, race/ethnicity and education). Detailed demographic, family and health information will be collected only from a sample adult and a sample child (if available) in a household. This change in the NHIS questionnaire will have a minimal impact on the MEPS sample selection because race/ethnicity, the main variable used by MEPS for oversampling, will still be collected in NHIS for all members in the household. This change means that MEPS can no longer oversample cancer patients for the planned Cancer Self-Administered Questionnaire as has been done previously. However, this supplement can still be fielded without oversampling.

Table 1 shows the expected eligible sample sizes, response rates and number of respondents associated with producing calendar year estimates for the 2025 to 2027 MEPS-HC data collection components. 

1. MEPS-HC Core Interview - AHRQ will annually sample approximately 21,139 responding households from the pervious years NHIS. With a cumulative response rate of 60% over an average of 2.5 interviews per year, AHRQ expects 12,683 RUs to complete the Core MEPS-HC interview per year.

2. Adult Self-Administered Questionnaire (Adult SAQ) - Each RUs has an average of 1.5 adults making 19,024 adults eligible for the Adults SAQ. The Adult SAQ is administered only once, in the even years. The expected response rate is 82%, resulting in 15,600 completed SAQs.

3. The Preventive Care Self-Administered Questionnaire (PSAQ) – The PSAQ will be administered to all adults in the RU once, in the odd years. The expected response rate is 82%, resulting in 15,600 completed SAQs.

4. Diabetes Care SAQ - Administered to adults identified in the Core interview as having diagnosed diabetes. The CDC estimates that 11.3% of adults have diagnosed diabetes (National Diabetes Statistics Report | Diabetes | CDC); based on that AHRQ estimates that 2,150 adults will be eligible for the Diabetes Care SAQ. An expected response rate of 65% should yield 1,398 completed Diabetes Care SAQs annually.

5. Burdens and Economic Impacts of Medical Care SAQ - Administered to all adults in 2025 only and will be removed from the MEPS-HC after the 2025 data collection. AHRQ is offering a $20 incentive to increase the response rate to 85%, resulting in an estimated 16,170 completed SAQs in 2025.

6. Authorization forms for the MEPS-MPC Provider and Pharmacy Survey - An estimated 9,233 RUs will have a medical or pharmacy event that will make them eligible to sign at least one permission form. The expected response rate is 80% resulting in an estimated 7,386 RUs that will sigh at least one form annually.

7. MEPS-HC Validation Interview - Each year 15% of completed Core MEPS-HC interviews are selected for a validation interview, resulting in an average of 1,902 RUs selected annually. The response rate for the validation interview is 96% giving an average of 1,826 completed interviews.

Table 1.  MEPS-HC expected annual sample for 2025 to 2027

Data collection component	Sample size	Response ratea	Number of respondents
1. MEPS-HC Core Interview	21,139b	60.0c	12,683
2 & 3. Adult SAQ (even years); The Preventive Care SAQ (PSAQ) (odd years)	19,024d	82.0	15,600
4. Diabetes Care SAQ	2,150e	65.0	1,398
5. Burdens and Economic Impacts of Medical Care SAQ	19,024d	85.0f	16,170
6. Authorization forms for the MEPS-MPC Provider and Pharmacy Survey	9,233g	80	7,386
7. MEPS-HC Validation Interview	1,902h	96.0	1,826

^a Response rates based on pre-covid rates (2019) and not conditioned on response to the NHIS. Final response rates are conditioned on response to the NHIS. Response rates declined during the covid pandemic. AHRQ expects to return to pre-covid rates by 2025.

^b Expected RU sample size from the NHIS. Average number of persons per RU is about 2.4

^c Average expected response rate, cumulative over an average of 2.5 interviews per year.

^d 12,683 * 1.5 = 19,024; 12,683 responding RUs, 1.5 adults per RU.

^e 12,683 * 1.5 * 0.113 = 2,150; 12,683 responding RUs, 1.5 adults per RU, 11.3% with

diagnosed diabetes.

^f Expected response rate with $20 incentive.

^g 12,683 * 0.728 = 9,233; 12,683 responding RUs, 72.8% eligible to sign at least one form.

^h 12,683 * 0.15 = 1,902; 12,683 responding RUs, 15% selected for validation interview.

The overall MEPS-HC response rate is a product of the response rate for each round of data collection in the MEPS and the response rate for the previous year NHIS survey from which the MEPS-HC sample was drawn.

The sample size specifications for the MEPS-HC have been set to meet specific precision requirements. For each estimation year, the relative standard error for a person level population estimate of 20 percent was specified to average about 2 percent. For example, the national population estimate of the percent of persons with no usual source of care was about 20 percent in most years from 2010-2015, with an average relative standard error of 2.3 percent.

Medical Provider Component

The sample for the MEPS Medical Provider Component (MPC) is designed to provide data on events for which household respondents are unlikely to know, such as charges and payments, to enrich the sample of events available as donors for imputation, and to provide a basis for methodological analysis of household reported charges and payments for all types of events.

Table 3 below shows the expected sample sizes, response rates and number of respondents for the MEPS-MPC, by provider type. The overall response rate for the MEPS-MPC is expected to be 76.1 percent, based on expected response rates for the 2021 data collection.

Table 3. MEPS-MPC expected annual sample by provider type for 2025 to 2027

Provider type	Eligible Sample size	Response rate*	Number of respondents
Home care -- health care providers	752	71.9	540
Office‑based providers	14,486	64.2	9,300
Separately billing doctors	9,524	49.1	4,676
Hospitals	6,419	61.3	3,935
Institutions (non-hospital)	102	84.7	86
Pharmacies	7,400	82.6	6,112
Total	38,683		24,649

* Based on results from the 2021 MEPS-MPC data collection

All hospitals and home health care agency providers are "in-scope" for the MEPS-MPC. Other providers and sites of care are in-scope if the provider is either a doctor of medicine or osteopathy, or if the provider practices under the direction or supervision of a MD or DO. For example, physician assistants and nurse practitioners working in clinics are medical providers considered in-scope for the MEPS-MPC. Chiropractors and dentists are out of scope (unless practicing in a hospital).

All office based physicians reported as providing care to persons in the MEPS-HC sample are eligible for inclusion in the MEPS-MPC sample (if permission provided).  Unique person/provider pair combinations are sampled in a manner designed to achieve a general budgeted sample size while representing different sampling subgroups.   In recent years the overall sampling rate has ranged from approximately 50 to 60%, with varying rates for different sample subgroups.  The MEPS-MPC sample also includes 100 percent of hospitals identified as providers of care by household respondents (if permission provided), including all inpatient stays, emergency room, and outpatient department visits.  All physicians identified by hospitals and/or households as providing care to sampled persons during the course of inpatient, outpatient department or emergency room care, but who bill separately from the hospital are included in the MPC sample. The physicians who bill separately from the hospital are sampled at different rates based on response propensity for predefined prioritization groups. This sampling rate for separately billing doctors has ranged from 95 to 97% in recent years depending on budget; however, we expect that the sampling rate for future years of data collection may be less than recent year rates, given planned budget and schedule. All home health agencies that provided care to household sampled persons are also included in the MPC sample. Finally, all pharmacies that have dispensed prescribed medicines to sampled persons are included in the MPC.

Over the last three years a number of lessons have been learned and incorporated into the MPC data collection strategy. Specific areas of focus are as follows:

1. As more hospitals have been acquired by large healthcare systems, the patient account records have become more specialized.   For example, for many hospitals there are now multiple points of contact to gather data for facility expenditures (including the in-patient room, operating or other treatment room, supplies, prescriptions) and professional fees (all healthcare providers included in a hospital bill).  As a result data collection strategies that address obtaining records from multiple sources in order to compile a complete record of inpatient expenditures for identified events have been developed. 

2. For office-based doctors, a larger proportion of providers now contract with billing services rather than staffing that administrative function within their office.  Similarly, hospitals not housed within a larger healthcare system more frequently use third-party record management vendors. In such cases a key to efficient data collection is the development of procedures that identify the billing services and/or record vendor initially so that data collection resources can be used more effectively and are not expended on contacts directly to the provider.  

2. Data Collection Procedures

Household Component

Preliminary Contact. Households responding to the NHIS and subsampled as part of a MEPS-HC panel will be contacted by mail prior to their first interview. The mailing contains an advance letter and brochure. Senior interviewing staff contact the households by telephone as a follow up to the mailing. The purpose of the advance telephone call is to verify the arrival of the materials, answer questions about MEPS, and obtain the best times for conducting the round 1 interview. Additional informational and record keeping materials are available to the respondent to assist them in preparing for their first interview. See Attachments 2 to 17 and Attachments 27 and 28 for all of the core instrument respondent materials. After the advance contact, households will be contacted in-person for the first of five interviews. The first two interviews are conducted primarily in person with the option to conduct interviews by video for respondents who are reluctant to be interviewed in person; telephone interviews are available in very limited circumstances. Later interviews may be conducted more often by video once the respondent is familiar with the study and trained to maintain and use medical records to answer questions about their health care events. All interviews use the same computer assisted personal interview (CAPI) instrument. The instrument is organized as a core instrument that will repeat unchanged in each of the rounds. Additional sections are asked only once a year and provide greater depth. Dependent interviewing methods in which respondents are asked to confirm or revise data provided in earlier interviews will be used to update information such as employment and health insurance data after the round in which such data are initially collected. The MEPS brochure will introduce the study. The Assurance of Confidentiality is covered in both the letter and the brochure and the Reporting Burden statement appears in the brochure. Five interviews (rounds 1-5) will be conducted with each sampled household at 4-6 month intervals over a 30 month time period. Round 1 will ask about the period since January 1 to the date of the interview. Round 2 will ask about the time since the Round 1 interview through the date of the Round 2 interview. Round 3 and Round 4 interviews cover the interview to interview interval. The Round 5 interview covers the period from the Round 4 interview to December 31, the end of a household’s second calendar year of MEPS participation.

Questionnaires for these field periods are largely parallel to those used in prior MEPS interviews. The instruments contain items that are asked once in the life of the study, items that are asked in each round, and items that are updated from round to round. Items only asked once include basic sociodemographics. Core questions asked include health status, health insurance coverage, employment status, medical utilization, hospital admissions, and purchase of medicines. For each health encounter identified, data will be obtained on the nature of health conditions, the services provided, the associated charges and sources (and amounts) of payments. Authorization forms for contacting medical providers and pharmacies will be collected in the field. Self-Administered-Questionnaires (SAQs) including topics such as health, preventive health, diabetes, and cancer will be collected in the field as well.

Medical Provider Component

The MEPS-MPC survey begins with the selection of the sample during the household interview. For those medical events and prescribed medicines reported in a household interview that meet the targeting criteria described above, a permission form is generated for each provider of the sampled person/provider pairs involved. This form describes the purpose of the survey and the information that is being collected, and authorizes the provider to release that information. The form is signed by the patient (or patient or guardian if person is under 18 years of age, or witness or proxy if patient is disabled or deceased). To expedite the identification of providers and assist with the preparation of an unduplicated provider list for the fielding of the MEPS-MPC, interviewers use a computerized database of medical providers, the National Provider Identifier (NPI) Provider Directory, which has been loaded onto the laptop. The NPI database is directly from the National Plan and Provider Enumeration System (NPPES) and Centers for Medicare & Medicaid Services. The NPI is a unique identification number for covered health care providers and uniquely identifies a health care provider. If a match is found with a provider identified by the household respondent, the matched directory record will be associated with the household event. The NPI directory records include, for each provider: a unique provider ID, the provider's name, and the provider’s practicing address and phone number(s). The MPC is conducted by telephone and record abstraction. The data collection process contains three basic steps:

1) an initial telephone screening to confirm provider eligibility and determine the appropriate person to whom the survey materials should be sent.

2) the mailing, faxing, or secure electronic submission of an advance package to the provider which describes the survey and the types of information that will be collected and also includes the permission forms for each patient; and

3) a phone call to actually collect the data. However, many providers prefer to send in records rather than provide information over the telephone.  The information is abstracted from the records, when records are sent in; when necessary, follow-up phone calls are made to the providers to clarify items in the records or to retrieve critical data items not contained in the records.  The majority (90 percent) of hospital providers choose to mail, fax or electronically submit records and approximately 50 percent of office based providers’ mail or fax records. The rest are obtained by telephone. Very few, other than some pharmacy chain providers, submit electronically.

For office‑based physicians, home health agencies, clinics, and separately billing doctors the data collection call is directed to the person who handles the billing for the provider. Often this is not someone in the provider's office, but an outside billing organization.

In the case of hospitals, data are collected not only from the billing department but from the medical records department and administrative office. Previous experience has shown that the names of the separately billing doctors usually cannot be obtained from the hospital's billing department. Consequently, there is an additional call to the medical records department to determine the names of all the doctors who treated the patient during a stay or visit. Moreover, in some cases the hospital's administrative office must be contacted to determine whether or not the doctors identified by medical records bill separately from the hospital itself.

Although experience has shown that telephone interviewing tends to be a very efficient method of collecting MEPS-MPC data and imposes minimal burden on providers, the MEPS-MPC data collection process has been designed to be as flexible as possible to accommodate the needs of respondents. Procedures for self‑administration are available, should respondents prefer that mode of data collection, and in‑person interviewing, for a small number of hospitals which may be identified by multiple persons in the household sample. Most recently, Secure File Transfer (FTP) and submission to a secure electronic portal have also been provided as an option to large MPC providers.

The pharmacy data collection process -- for individual, non-chain pharmacies -- consists of:  (1) an initial phone call to the pharmacy to solicit cooperation and determine how to send the survey materials; (2) materials are faxed, mailed, or electronically submitted to the pharmacy; (3) pharmacies respond by sending in, by fax, mail, or electronic submission, patient profiles.  Sometimes the pharmacist is willing to give the information over the phone and the data is collected into an Integrated Data Collection System (IDCS) on a secure web portal; (4) pharmacies are followed-up to prompt for response or if data items in submitted profiles are not clear. The process for the larger chains that have requested centralized corporate contacts can vary, depending on the preferences of the chain.  All begin with a telephone contact and include a step in which the authorization forms are sent to the company, but then data collection proceeds as desired by the chain: some respond in electronic format (approximately 1 to 2 percent); many send in (hard copy) profiles (approximately 94 to 96 percent reply by mail or fax with the split between the two modes fairly evenly divided).

3. Methods to Maximize Response Rates

Household Component

Households in the MEPS-HC sample are interviewed in person or by video by trained interviewers using a CAPI application to record the respondent’s answers to the survey questions. In addition to providing information on family composition, health status, employment, and health insurance, household respondents are asked to report details on health events for all members of the family. The interviews vary in length depending upon the number of persons in the family and the number of health care events the family has to report. Round 1 interviews typically last between one and a half and two hours. Subsequent round interviews are somewhat shorter.

Over time, the MEPS-HC has refined a series of activities and procedures designed to build and maintain response rates. These activities begin with a sequence of advance mailings that provide a first introduction to the study and continue through concerted follow up efforts to gain the participation of the households that are difficult to contact or reluctant to participate. These efforts are particularly concentrated in the first round of a new panel’s participation, but continue with efforts to maintain cooperation through the full five rounds of interviewing. The standard practices include:

• Pre-interview contacts. Before an interviewer makes the first attempt to contact a sampled household in person, the household receives a series of two mailings and one advance telephone contact.  The first mailing notifies the family of its selection for the survey, and includes a brochure explaining the study and the nature of participation. The second mailing is a brief reminder of the coming interview, timed to arrive shortly before the interviewer’s first attempt to contact the family in person.  Shortly following the first mailing, respondents are contacted by telephone to verify their receipt of the package and to answer their questions about the study.  These calls serve to provide an early indication of the households that have moved since the NHIS and require tracking and an early assessment of the likelihood of the household’s participation when contacted.

• Careful attention to the selection and training of data collection staff. Training sessions are designed to prepare interviewers to be knowledgeable about the study, comfortable in using study materials, and prepared with answers to common respondent questions. In recent years, as the level of effort required to obtain cooperation has increased, more attention has been given to training interviewers in techniques for avoiding refusals. For some segments of the training, bilingual interviewers meet separately to practice introducing and administering the survey in Spanish.

• Attention to the appropriate assignment of cases to interviewers. As the MEPS-HC is a subset of households that participated in the prior year’s NHIS, information available from the NHIS interview and from the advance contact calls is taken into account by field supervisors when making assignments and by individual interviewers when planning their first contact attempts. When the NHIS information indicates that a case was only “partially completed” it usually indicates that the NHIS household was reluctant to participate and only willing to complete part of the NHIS interview. These cases are assigned to interviewers who have demonstrated skill with refusal aversion techniques. Similarly, if the interviewer conducting the advance contact call indicates that the household seems hesitant to participate, the case is also assigned to an interviewer skilled in refusal aversion.

• Close monitoring of the field data collection effort by field supervisors and project managers. Paradata documenting every interviewer attempt to contact a household is made available to supervisors to guide interviewers’ timing of contact attempts. In weekly calls, supervisors and interviewers discuss work plans and alternative approaches for contacting and gaining cooperation of individual cases. Weekly calls among the managers of the field operation allow discussion of solutions to common response problems, planning and coordination of efforts to follow-up non-responding households, and efficient allocation of field resources.

• Determining where to place resources to build the response rate requires reliable data on production and response rates, contact efforts, interviewer availability, location of pending work, and dispositions of remaining cases. All of this information is contained within the MEPS-HC management database and available in reports. A number of ‘real time’ reports using paradata are available to field management staff for daily use. In addition, weekly reports are generated throughout the field period to monitor production and response rates by domain, primary sampling unit (PSU), and region to ensure the work is progressing toward schedule and response rate goals. The key to the approach is early identification of response rate issues that allows sufficient time to formulate and implement plans for conversion. Recent history regarding MEPS-HC unweighted response rates for MEPS only (i.e., rates that are conditional on response to NHIS) are as follows:

MEPS 2013
Panel 17	66.0%
Panel 18	65.1%
MEPS 2014
Panel 18	62.3%
Panel 19	63.6%
MEPS 2015
Panel 19	60.6%
Panel 20	65.3%
MEPS 2016
Panel 20	60.9%
Panel 21	65.0%
MEPS 2017
Panel 21	61.5%
Panel 22	64.3%
MEPS 2018
Panel 22	61.1%
Panel 23	64.1%
MEPS 2019
Panel 23	60.2%
Panel 24	60.0%
MEPS 2020
Panel 23*	41.7%
Panel 24	44.8%
Panel 25 MEPS 2021 Panel 23* Panel 24* Panel 25 Panel 26	39.4%

^*Panels 23 and 24 were invited to continue participating in MEPS during the COVID-19 pandemic

• Interviewers are provided with a variety of materials to support their efforts to gain cooperation: handouts printed in Spanish and English that explain different aspects of the study and research highlights and news items reporting findings from MEPS data are provided for the interviewers to use as needed to address concerns expressed by respondents. These are also available online at MEPSdocs.org and meps.ahrq.com.

• In return for the time respondents spend preparing for the MEPS-HC interview, households receive a gift of $50 per interview. The $50 gift has been in place since the start of Panel 16 in 2011 (OMB approval obtained January 26, 2010 version 1).

• The project has developed a number of letters that address areas of concern commonly raised by respondents who do not respond when initially contacted by an interviewer. Supervisors can request mailing of the specific letter (available in English and Spanish) that is most appropriate for a given household.

• For households that are difficult to contact, interviewers make multiple contact attempts, at different times of day and days of the week, using information from the NHIS and their own prior contact attempts to determine the best time for each successive attempt.

• For households that refuse an initial request to participate, the interviewer and supervisor decide on an approach for attempting to convert the refusal, taking into account all information available from the NHIS and prior contact efforts. Depending on the specifics of each case, one of the refusal conversion letters may be sent before another attempt is made in person, points to be made to address the reasons for the refusal are discussed, and frequently, a different interviewer will be assigned to make the next attempt.

• For households that require tracking, the interviewer who determines that the household has moved makes initial, local attempts to obtain new locating information. When those local sources are not successful, the case is referred to senior field management staff for additional searching through approved internet resources.

Since resources—time, budget, and staff—are not limitless, selection of the areas and specific cases on which to concentrate effort is critical. To guide these decisions, the project draws on multiple sources of information: information from prior panels on the characteristics of responders and nonresponders, information from the NHIS on the characteristics of the sampled households, paradata from the project management system, and information on the location, experience level, and skill sets of the interviewing field force.

While planned upcoming NHIS sample design changes and the ongoing impacts of the COVID-19 pandemic have the potential to affect MEPS response rates and data quality, it is not completely feasible at this juncture to assess any impact of these changes. The NHIS sample redesign will be fully launched in January 2019.

Medical Provider Component

MEPS-MPC staff plans to maintain the high response rates for the MEPS-MPC by bringing forward to the current data collection effort the methods that have been successful in maintaining provider cooperation in the past. An initial telephone screening to confirm provider eligibility and determine the appropriate person to whom the survey materials should be sent and the mailing, faxing, or electronic submission of an advance package to the provider which describes the survey and the types of information that will be collected (and includes the permission forms) helps to maintain the high response rates.

Data collection staff who appreciate the difficulty and importance of the task, and are capable of establishing good rapport with providers and placing as little burden on them as possible to accurately collect the data, will be recruited and retained. All data collection staff participate in an in-depth initial training as well as on-going performance improvement activities. MEPS-MPC identity and logos will be maintained so that providers who have participated in the past will recognize the study, but data collection materials will be customized to the current year’s data collection so providers understand what is currently being requested of them. Data collection protocols and instruments are also customized to the different types of providers to make it as easy as possible for providers to provide data in the manner in which it appears in their records. Providers with a previous history of being reluctant to participate will be assigned to data collection staff specializing in working with such respondents to maximize the possibility that they will participate. Providers with particularly large numbers of study patients will be assigned to staff capable of working out means of obtaining the large number of records with minimal burden to the provider. Finally, the use of an electronic data capture system, which allows real-time checking for the entry of complete and accurate information into the data collection forms while they are being filled out, helps minimize return calls to providers to resolve missing or confusing items and make it more likely that their cooperation will be maintained in future data collection efforts.

4. Weighting and Nonresponse Adjustment

The general weighting approach that has been used for earlier MEPS-HC panels will continue to be employed in future years. This includes nonresponse adjustments to sample weights to reduce the potential for bias in sample estimates arising from nonresponse at the household, family, and person levels. After adjustments for nonresponse, for those MEPS weights that pertain to population counts for which stable estimates can be obtained from independent sources (e.g., the civilian, noninstitutionalized population or adult members of this population), MEPS weights are calibrated to such estimates. Specifically, MEPS nonresponse adjusted weights are poststratified or raked to demographic and geographic subgroup population estimates based on corresponding data from the Current Population Survey (CPS). Such calibration serves to reduce the potential for bias arising from under coverage and can be expected to help reduce sample variability as well.

Under the current weighting scheme, a household-level poststratification and a nonresponse adjustment are applied utilizing the household characteristics collected in NHIS for both responding and nonresponding households to MEPS. Some of these household characteristics are based on characteristics of the household reference person (i.e., the person identified as owning or renting the home) such as age, sex, race/ethnicity, education, marital status, employment status and income of the reference person. With the NHIS instrument change, neither a reference person is identified nor is detailed information about all persons in the household collected. Consequently, the weighting procedure has been modified slightly where characteristics of the sample adult are used instead of the reference person. This modification has not had a major impact on the poststratification or nonresponse adjustments.

There have been concerns of potential increased bias with data collected since the start of the COVID-19 pandemic, arising from differential response propensity among households of different socioeconomic statuses. To better reduce the potential for bias, poverty status from the previous year has been added to the nonresponse adjustment for those panels that are in their second or more years of data collection. The inclusion of poverty status in nonresponse adjustment will be the standard weighting process going forward.

Nonresponse Bias Studies

Nonresponse bias concerns arise to the extent that nonrespondents differ from respondents, particularly on key analytic variables, and how well the responders represent the target populations of interest. Since the MEPS-HC sample is drawn from NHIS participant households, the NHIS provides the best source for identifying characteristics of responders and nonresponders. The analyses also include across panel comparisons in MEPS.

Using weighted response rates, nonresponse bias analyses periodically conducted in MEPS examine the following:

• How well do responders represent the target population on key characteristics such as race and ethnicity, urban/rural status, age, household size, income level, etc.

• Whether responders and nonresponders differ on key analytic variables such as health insurance status, chronic disease status, and health care utilization – all of these items are collected in both the NHIS and MEPS-HC. Linkage to the NHIS provides substantial information for this assessment.

• What are the contact patterns identified by paradata for responders and whether these data can predict propensity to respond. These paradata are obtained from both the NHIS and MEPS-HC such as length of the interview, number of contacts, mode of contact, etc. Appropriate multivariate analytic methods are employed to determine if contact data correlates with propensity to respond.

Previous analysis conducted by AHRQ showed that weighted estimates based on the MEPS final responding sample were mostly consistent with estimates from the full NHIS which are based on a much larger sample size (https://meps.ahrq.gov/data_files/publications/workingpapers/wp_13002.pdf). This provided evidence of minimal bias in MEPS estimates. Moreover, as mentioned above, the variables from the NHIS that are used to adjust for nonresponse are often evaluated and updated to help reduce nonresponse bias. This process of nonresponse adjustment is found to be effective since the characteristics of nonrespondents in MEPS are available from the NHIS. There have been concerns with data collected since the start of the COVID-19 pandemic related to impacts on the NHIS, on MEPS, and on the CPS data used as control totals for calibration. Ongoing analyses to compare trends in MEPS healthcare utilization estimates to other data sources suggests that inpatient hospital stays and emergency room utilization in 2021 were lower than expected in MEPS. Exact comparisons are difficult to make because of differences in populations covered, definitional differences, and other differences between MEPS and administrative data sources.

5. Tests of Procedures

Whenever major changes are made to the MEPS they are pretested to ensure that data quality is not negatively impacted. In the Household Component, all updates to the CAPI instrument and web SAQs include continuous, iterative cycles of specification, testing, and evaluation by Westat and testing of alpha and beta versions of each instrument by AHRQ. Testing of the developing instrument is guided by increasingly sophisticated and complex testing scenarios. Test data preparation uses an iterative process that first yields data for round 1 interviews for the most common scenarios, and then increases the breadth of scenarios within round 1 before moving into the preparation of data for later rounds. Test scripts cover all aspects of the design layout for each question within a section (e.g. adherence to conventions for display of question text, interviewer instructions, response categories, etc.) and cover the full range of routing instructions through each questionnaire section. Multiple rounds of usability testing are conducted with field staff with varying levels of technical skills when introducing new features. Before each field period, interviewers receive training including online training modules on changes to the CAPI instrument and field procedures.

Recent changes to the MEPS Household Component since 2020 include the introduction of video interviewing (CAVI), web SAQs, and electronic authorization forms. Web SAQs and electronic authorization forms (added to the traditional paper options) were tested similar to the procedures described in the paragraph above. Development of CAVI in fall 2021 included testing three different video platforms, first with Westat staff, then with friends and family of employees, and finally with a small number of live respondents. Throughout spring 2022 interviewers were trained in batches to set up and use video interviews, and the use of video has been closely monitored for quality and assessed for improvements to the procedures and equipment setup.

There are currently no firm plans for implementation of significant additional changes to MEPS that would require pretesting. When adding new items to the study, MEPS uses well-established items from other national surveys if pretesting is not possible.

6. Statistical Consultants

The following are responsible for statistical aspects of the MEPS Study:

Sadeq Chowdhury, Ph.D.

Division of Statistical Research and Methods

Center for Financing, Access and Cost Trends

AHRQ

(301) 427-1666