Supporting Statement Part B -- MEPS HC 09072018v4.mns (002)

SUPPORTING STATEMENT

Part B

Medical Expenditure Panel Survey (MEPS) Household Component and the MEPS Medical Provider Component

Version: 2018

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent Universe and Sampling Methods 3

2. Data Collection Procedures 8

3. Methods to Maximize Response Rates 10

4. Weighting and Nonresponse Adjustment 14

5. Tests of Procedures 15

6. Statistical Consultants 16

B. Collections of Information Employing Statistical Methods

To fill in major data gaps identified by the Department of Health and Human Services, the Medical Expenditure Panel Survey (MEPS) is specified as a continuous survey. Each year, a new nationally representative MEPS sample will be selected from a subset of households that participated in the prior year’s National Health Interview Survey (NHIS). A preliminary contact with the NHIS responding households selected for the MEPS study will take place to announce the MEPS survey and introduce records keeping activities.

1. Respondent Universe and Sampling Methods

Household Component

The initial MEPS Household Component (HC) sample consists of households that responded to the prior year’s NHIS, in the panels reserved for the MEPS. The basic analysis unit in the MEPS is defined as the person.

A Reporting Unit (RU) is a person or group of persons in the sampled dwelling unit that are related by blood, marriage, adoption or other family associations and for whom data are to be collected during the MEPS-HC interview. Typically, one adult family member provides information for the entire family although all adult family members are encouraged to participate. Each year’s MEPS-HC sample will be surveyed to collect annual data for two consecutive years. Each new MEPS-HC sample will be a randomly selected subsample of households that responded to the prior year’s NHIS. The NHIS is based on a stratified cluster sample design (see http://www.cdc.gov/nchs/nhis/about_nhis.htm for more information about the NHIS).

Beginning in 2016 NCHS implemented another new sample design for the NHIS. From a broad perspective, the new design is similar to the previous design because clusters of households are still selected within PSUs which are still essentially formed at the county level. However, within sampled PSUs, the clusters of addresses (households) that are sampled under the new design are not in the form of segments as in the previous design due to utilization of an address-based list of households. Also, the new design used each of the 50 states as well as the District of Columbia as explicit strata with oversampling some of the smaller states with the intent of providing the capability of state-level NHIS estimates. However, the precision of estimates at the national level is expected to be very similar to the previous design. Although the PSUs in the new design were selected independently from the previous design, all large PSUs are selected in the sample with certainty in both designs which should contribute to the efficiency for trend analysis across designs. Moreover, the MEPS sample will continue to have the overlapping panels that will also be a major contributor to the efficiency of year to year estimates of change.

Another notable difference is that the current design does not involve oversampling of any minority group (although it may in the future) that will reduce the number of minorities in the MEPS. However, NHIS responding households with members who are Hispanics, black, or Asian will continue to be oversampled to maximize minority sample sizes in the MEPS-HC. See https://meps.ahrq.gov/data_stats/download_data/pufs/h181/h181doc.pdf for more detailed information about the MEPS-HC.

Beginning in 2019, NCHS will implement a new NHIS questionnaire for the NHIS. Under the redesigned questionnaire, only a few basic items will be collected about all household members (e.g., age, sex, race/ethnicity and education). Detailed demographic, family and health information will be collected only from a sample adult and a sample child (if available) in a household. This change in the NHIS questionnaire will have a minimal impact on the MEPS sample selection because race/ethnicity, the main variable used by MEPS for oversampling, will still be collected in NHIS for all members in the household. It may reduce the ability to oversample some population subgroups for occasional MEPS supplements such as households with veterans or with a cancer patient. However, such supplements are included only occasionally in MEPS and can still be done without oversampling, or if necessary, using alternative approaches to identify cases for oversampling such as relying on sample adult/child information or using a field screening procedure.

Table 1 shows the expected eligible sample sizes, response rates and number of respondents associated with producing calendar year estimates for the 2019 to 2021 MEPS-HC data collection components.  The target number of completed respondents to produce calendar year estimates for the MEPS-HC is approximately 12,200 RUs (families), containing about 32,000 persons.

Table 1. MEPS-HC expected annual sample for 2019 to 2021

Data collection component	Sample size	Response rate	Number of respondents/ completed forms
MEPS-HC Core Interview (RUs)	16,750a	64.4b	10,788
Adult Male SAQ (PSAQ) - Years 2019 and 2021; Adult SAQ (SAQ)-Year 2020	10,705c	87.6d	9,377
Adult Female SAQ (PSAQ) - Years 2019 and 2021; Adult SAQ (SAQ)- Year 2020	12,670c	87.6d	11,099
Diabetes care SAQ	2,404c	85.9d	2,065
Veterans SAQ	1,500	90.0d	1,350
Authorization form for the MEPS-MPC Provider Survey	81,881e	91.0d	74,511
Authorization form for the MEPS-MPC Pharmacy Survey	32,558e	78.1d	25,428
MEPS-HC Validation Interview	3,596	96.0d	3,452

^a Expected RU sample size for first interview in year; average number of persons per RU is about 2.4

^b Average expected response rate at rounds 2/4; conditioned on response to the NHIS

^c Based on adjusted yields for 2015.

^d Conditioned on response to the MEPS-HC core interview.

^e Based on adjusted yields for 2016.

The overall MEPS-HC response rate is a product of the response rate for each round of data collection in the MEPS and the response rate for the previous year NHIS survey from which the MEPS-HC sample was drawn. Table 2 shows the sample data by panel and round for the 2015 MEPS-HC. In order to produce annual health care estimates for calendar year 2015 based on the full MEPS sample data from the MEPS Panel 19 and Panel 20, the two panels are combined. More specifically, full calendar year 2015 data collected in Rounds 3 through 5 for the MEPS Panel 19 sample are combined with data from the first three rounds of data collection for the MEPS Panel 20 sample. Conditional on response to the NHIS, the overall MEPS-HC response rate in 2015 was 63.1. When accounting for nonresponse to the NHIS, the final response rate was 47.7 percent. This compares to an overall response rate for 2012 of 47.7 percent.

Table 2. Sample Size and Unweighted Response Rates for 2015 Full Year File

(Panel 20 Rounds 1-3/Panel 19 Rounds 3-5)

	Panel 19	Panel 20	2015 Combined
A. Percentage of NHIS households designated for use in MEPS (those initially characterized as responding) †	76.2%	75.1%	—
B. Number of households sampled from the NHIS	9,700	10,610	—
C. Number of Households sampled from the NHIS and fielded for MEPS	9,667	10,571	—
D. Round 1 – Number of RUs eligible for interviewing	10,346	11,283	—
E. Round 1 – Number of RUs with completed interviews	7,430	8,287	—
F. Round 2 – Number of RUs eligible for interviewing	7,669	8,554	—
G. Round 2 – Number of RUs with completed interviews	7,176	7,991	—
H. Round 3 – Number of RUs eligible for interviewing	7,335	8,136	—
I. Round 3 – Number of RUs with completed interviews	6,949	7,743	—
J. Round 4 – Number of RUs eligible for interviewing	7,083	—	—
K. Round 4 – Number of RUs with completed interviews	6,855	—	—
L. Round 5 – Number of RUs eligible for interviewing	6,910	—	—
M. Round 5 – Number of RUs with completed interviews	6,792	—	—
Overall annual unweighted response rates P21: A x (E/D) x (G/F) x (I/H) P20: A x (E/D) x (G/F) x (I/H) x (K/J) x (M/L) Combined: 0. 510 x P20 + 0. 490 x P21	46.1% (Panel 19 through Round 5)	49.0% (Panel 20 through Round 3)	47.7%

†Among the panels and quarters of the NHIS allocated to MEPS, the percentage of households that were considered to be NHIS respondents at the time the MEPS sample was selected.

The sample size specifications for the MEPS-HC have been set to meet specific precision requirements. For each estimation year, the relative standard error for a person level population estimate of 20 percent was specified to average about 2 percent. For example, the national population estimate of the percent of persons with no usual source of care was about 20 percent in most years from 2010-2015, with an average relative standard error of 2.3 percent.

Medical Provider Component

The sample for the MEPS Medical Provider Component (MPC) is designed to provide data on events for which household respondents are unlikely to know charges and payments, to enrich the sample of events available as donors for imputation, and to provide a basis for methodological analysis of household reported charges and payments for all types of events.

Table 3 below shows the expected sample sizes, response rates and number of respondents for the MEPS-MPC, by provider type. The overall response rate for the MEPS-MPC is expected to be 78.4 percent, based on expected response rates for the 2017 data collection.

Table 3. MEPS-MPC expected annual sample by provider type for 2019 to 2021

Provider type	Eligible Sample size	Response rate	Number of respondents
Home care -- health care providers	539	90.0*	485
Home care – non-health care providers	11	90.0*	10
Office‑based providers	16,000	80.0*	12,800
Separately billing doctors	13,793	60.0*	8,276
Hospitals	10,000	88.0*	8,800
Institutions (non-hospital)	110	90.0*	99
Pharmacies	19,200	85.0*	16,320
Total	59,653		46,790

* Based on expected results from the 2017 MEPS-MPC data collection

All hospitals and home health care agency providers are "in-scope" for the MEPS-MPC. Other providers and sites of care are in-scope if the provider is either a doctor of medicine or osteopathy, or if the provider practices under the direction or supervision of a MD or DO. For example, physician assistants and nurse practitioners working in clinics are medical providers considered in-scope for the MEPS-MPC. Chiropractors and dentists are out of scope (unless practicing in a hospital).

All office based physicians reported as providing care to persons in the MEPS-HC sample are eligible for inclusion in the MEPS-MPC sample (if permission provided).  Unique person/provider pair combinations are sampled in a manner designed to achieve a general budgeted sample size while representing different sampling subgroups.   In recent years the overall sampling rate has ranged from approximately 50 to 60%, with varying rates for different sample subgroups.  The MEPS-MPC sample also includes 100 percent of hospitals identified as providers of care by household respondents (if permission provided), including all inpatient stays, emergency room, and outpatient department visits.  All physicians identified by hospitals and/or households as providing care to sampled persons during the course of inpatient, outpatient department or emergency room care, but who bill separately from the hospital are included in the MPC sample. The physicians who bill separately from the hospital are sampled at different rates based on response propensity for predefined prioritization groups. This sampling rate for separately billing doctors has ranged from 95 to 97% in recent years depending on budget; however, we expect that the sampling rate for future years of data collection may be less than recent year rates, given planned budget and schedule. All home health agencies that provided care to household sampled persons are also included in the MPC sample. Finally, all pharmacies that have dispensed prescribed medicines to sampled persons are included in the MPC.

Over the last three years a number of lessons have been learned and incorporated into the MPC data collection strategy. Specific areas of focus are as follows:

1. As more hospitals have been acquired by large healthcare systems, the patient account records have become more specialized.   For example, for many hospitals there are now multiple points of contact to gather data for facility expenditures (including the in-patient room, operating or other treatment room, supplies, prescriptions) and professional fees (all healthcare providers included in a hospital bill).  As a result data collection strategies that address obtaining records from multiple sources in order to compile a complete record of inpatient expenditures for identified events have been developed. 

2. For office-based doctors, a larger proportion of providers now contract with billing services rather than staffing that administrative function within their office.  Similarly, hospitals not housed within a larger healthcare system more frequently use third-party record management vendors. In such cases a key to efficient data collection is the development of procedures that identify the billing services and/or record vendor initially so that data collection resources can be used more effectively and are not expended on contacts directly to the provider.  

2. Data Collection Procedures

Household Component

Preliminary Contact. Households responding to the NHIS and subsampled as part of a MEPS-HC panel will be contacted by mail prior to their first interview. The mailing contains an advance letter and brochure. Senior interviewing staff contact the households by telephone as a follow up to the mailing. The purpose of the advance telephone call is to verify the arrival of the materials, answer questions about MEPS, and obtain the best times for conducting the round 1 interview. Additional informational and record keeping materials are available to the respondent to assist them in preparing for their first interview. See Attachments 2 to 17 and Attachments 27 and 28 for all of the core instrument respondent materials. After the advance contact, households will be contacted in-person for the first of five in-person interviews. The interviews are conducted as a computer assisted personal interview (CAPI). The CAPI instrument is organized as a core instrument that will repeat unchanged in each of the rounds. Additional sections are asked only once a year and provide greater depth. Dependent interviewing methods in which respondents are asked to confirm or revise data provided in earlier interviews will be used to update information such as employment and health insurance data after the round in which such data are initially collected. The MEPS brochure will introduce the study. The Assurance of Confidentiality is covered in both the letter and the brochure and the Reporting Burden statement appears in the brochure. Five interviews (rounds 1-5) will be conducted with each sampled household at 4-6 month intervals over a 30 month time period. All interviews will be conducted in person with CAPI as the principal data collection mode. Round 1 will ask about the period since January 1 to the date of the interview. Round 2 will ask about the time since the Round 1 interview through the date of the Round 2 interview. Round 3 and Round 4 interviews cover the interview to interview interval. The Round 5 interview covers the period from the Round 4 interview to December 31, the end of a household’s second calendar year of MEPS participation.

Questionnaires for these field periods are largely parallel to those used in prior MEPS interviews. The instruments contain items that are asked once in the life of the study, items that are asked in each round, and items that are updated from round to round. Items only asked once include basic sociodemographics. Core questions asked include health status, health insurance coverage, employment status, medical utilization, hospital admissions, and purchase of medicines. For each health encounter identified, data will be obtained on the nature of health conditions, the services provided, the associated charges and sources (and amounts) of payments. Authorization forms for contacting medical providers and pharmacies will be collected in the field.

Medical Provider Component

The MEPS-MPC survey begins with the selection of the sample during the household interview. For those medical events and prescribed medicines reported in a household interview that meet the targeting criteria described above, a permission form is generated for each provider of the sampled person/provider pairs involved. This form describes the purpose of the survey and the information that is being collected, and authorizes the provider to release that information. The form is signed by the patient (or patient or guardian if person is under 18 years of age, or witness or proxy if patient is disabled or deceased). To expedite the identification of providers and assist with the preparation of an unduplicated provider list for the fielding of the MEPS-MPC, interviewers use a computerized database of medical providers, the National Provider Identifier (NPI) Provider Directory, which has been loaded onto the laptop. The NPI database is directly from the National Plan and Provider Enumeration System (NPPES) and Centers for Medicare & Medicaid Services. The NPI is a unique identification number for covered health care providers and uniquely identifies a health care provider. If a match is found with a provider identified by the household respondent, the matched directory record will be associated with the household event. The NPI directory records include, for each provider: a unique provider ID, the provider's name, and the provider’s practicing address and phone number(s). The MPC is conducted by telephone and record abstraction. The data collection process contains three basic steps:

1) an initial telephone screening to confirm provider eligibility and determine the appropriate person to whom the survey materials should be sent.

2) the mailing, faxing, or secure electronic submission of an advance package to the provider which describes the survey and the types of information that will be collected and also includes the permission forms for each patient; and

3) a phone call to actually collect the data. However, many providers prefer to send in records rather than provide information over the telephone.  The information is abstracted from the records, when records are sent in; when necessary, follow-up phone calls are made to the providers to clarify items in the records or to retrieve critical data items not contained in the records.  The majority (90 percent) of hospital providers choose to mail, fax or electronically submit records and approximately 50 percent of office based providers’ mail or fax records. The rest are obtained by telephone. Very few, other than some pharmacy chain providers, submit electronically.

For office‑based physicians, home health agencies, clinics, and separately billing doctors the data collection call is directed to the person who handles the billing for the provider. Often this is not someone in the provider's office, but an outside billing organization.

In the case of hospitals, data are collected not only from the billing department but from the medical records department and administrative office. Previous experience has shown that the names of the separately billing doctors usually cannot be obtained from the hospital's billing department. Consequently, there is an additional call to the medical records department to determine the names of all the doctors who treated the patient during a stay or visit. Moreover, in some cases the hospital's administrative office must be contacted to determine whether or not the doctors identified by medical records bill separately from the hospital itself.

Although experience has shown that telephone interviewing tends to be a very efficient method of collecting MEPS-MPC data and imposes minimal burden on providers, the MEPS-MPC data collection process has been designed to be as flexible as possible to accommodate the needs of respondents. Procedures for self‑administration are available, should respondents prefer that mode of data collection, and in‑person interviewing, for a small number of hospitals which may be identified by multiple persons in the household sample. Most recently, Secure File Transfer (FTP) and submission to a secure electronic portal have also been provided as an option to large MPC providers.

The pharmacy data collection process -- for individual, non-chain pharmacies -- consists of:  (1) an initial phone call to the pharmacy to solicit cooperation and determine how to send the survey materials; (2) materials are faxed, mailed, or electronically submitted to the pharmacy; (3) pharmacies respond by sending in, by fax, mail, or electronic submission, patient profiles.  Sometimes the pharmacist is willing to give the information over the phone and the data is collected into an Integrated Data Collection System (IDCS) on a secure web portal; (4) pharmacies are followed-up to prompt for response or if data items in submitted profiles are not clear. The process for the larger chains that have requested centralized corporate contacts can vary, depending on the preferences of the chain.  All begin with a telephone contact and include a step in which the authorization forms are sent to the company, but then data collection proceeds as desired by the chain: some respond in electronic format (approximately 1 to 2 percent); many send in (hard copy) profiles (approximately 94 to 96 percent reply by mail or fax with the split between the two modes fairly evenly divided).

3. Methods to Maximize Response Rates

Household Component

Households in the MEPS-HC sample are interviewed in person by trained interviewers using a CAPI application to record the respondent’s answers to the survey questions. In addition to providing information on family composition, health status, employment, and health insurance, household respondents are asked to report details on health events for all members of the family. The interviews vary in length depending upon the number of persons in the family and the number of health care events the family has to report. Round 1 interviews typically last between one and a half and two hours. Subsequent round interviews are somewhat shorter.

Over time, the MEPS-HC has refined a series of activities and procedures designed to build and maintain response rates. These activities begin with a sequence of advance mailings that provide a first introduction to the study and continue through concerted follow up efforts to gain the participation of the households that are difficult to contact or reluctant to participate. These efforts are particularly concentrated in the first round of a new panel’s participation, but continue with efforts to maintain cooperation through the full five rounds of interviewing. The standard practices include:

• Pre-interview contacts. Before an interviewer makes the first attempt to contact a sampled household in person, the household receives a series of two mailings and one advance telephone contact.  The first mailing notifies the family of its selection for the survey, and includes a brochure explaining the study and the nature of participation. The second mailing is a brief reminder of the coming interview, timed to arrive shortly before the interviewer’s first attempt to contact the family in person.  Shortly following the first mailing, respondents are contacted by telephone to verify their receipt of the package and to answer their questions about the study.  These calls serve to provide an early indication of the households that have moved since the NHIS and require tracking and an early assessment of the likelihood of the household’s participation when contacted.

• Careful attention to the selection and training of data collection staff. Training sessions are designed to prepare interviewers to be knowledgeable about the study, comfortable in using study materials, and prepared with answers to common respondent questions. In recent years, as the level of effort required to obtain cooperation has increased, more attention has been given to training interviewers in techniques for avoiding refusals. For some segments of the training, bilingual interviewers meet separately to practice introducing and administering the survey in Spanish.

• Attention to the appropriate assignment of cases to interviewers. As the MEPS-HC is a subset of households that participated in the prior year’s NHIS, information available from the NHIS interview and from the advance contact calls is taken into account by field supervisors when making assignments and by individual interviewers when planning their first contact attempts. When the NHIS information indicates that a case was only “partially completed” it usually indicates that the NHIS household was reluctant to participate and only willing to complete part of the NHIS interview. These cases are assigned to interviewers who have demonstrated skill with refusal aversion techniques. Similarly, if the interviewer conducting the advance contact call indicates that the household seems hesitant to participate, the case is also assigned to an interviewer skilled in refusal aversion.

• Close monitoring of the field data collection effort by field supervisors and project managers. Paradata documenting every interviewer attempt to contact a household is made available to supervisors to guide interviewers’ timing of contact attempts. In weekly calls, supervisors and interviewers discuss work plans and alternative approaches for contacting and gaining cooperation of individual cases. Weekly calls among the managers of the field operation allow discussion of solutions to common response problems, planning and coordination of efforts to follow-up non-responding households, and efficient allocation of field resources.

• Determining where to place resources to build the response rate requires reliable data on production and response rates, contact efforts, interviewer availability, location of pending work, and dispositions of remaining cases. All of this information is contained within the MEPS-HC management database and available in reports. A number of ‘real time’ reports using paradata are available to field management staff for daily use. In addition, weekly reports are generated throughout the field period to monitor production and response rates by domain, primary sampling unit (PSU), and region to ensure the work is progressing toward schedule and response rate goals. The key to the approach is early identification of response rate issues that allows sufficient time to formulate and implement plans for conversion. Recent history regarding MEPS-HC unweighted response rates for MEPS only (i.e., rates that are conditional on response to NHIS) are as follows:

MEPS 2009
Panel 13		67.2%
Panel 14		65.1%
MEPS 2010
Panel 14		62.0%
Panel 15		64.6%
MEPS 2011
Panel 15		61.6%
Panel 16		71.1%
MEPS 2012
Panel 16		67.6%
Panel 17		70.1%
MEPS 2013
Panel 17		66.0%
Panel 18		65.1%
MEPS 2014
Panel 18		62.3%
Panel 19		63.6%
MEPS 2015
Panel 19		60.6%
Panel 20		65.3%
MEPS 2016
Panel 20		60.9%
Panel 21		65.0%

• Interviewers are provided with a variety of materials to support their efforts to gain cooperation: handouts printed in Spanish and English that explain different aspects of the study and research highlights and news items reporting findings from MEPS data are provided for the interviewers to use as needed to address concerns expressed by respondents.

• In return for the time respondents spend preparing for the MEPS-HC interview, households receive a gift of $50 per interview. The $50 gift has been in place since the start of Panel 16 in 2011 (OMB approval obtained January 26, 2010 version 1).

• The project has developed a number of letters that address areas of concern commonly raised by respondents who do not respond when initially contacted by an interviewer. Supervisors can request mailing of the specific letter (available in English and Spanish) that is most appropriate for a given household.

• For households that are difficult to contact, interviewers make multiple contact attempts, at different times of day and days of the week, using information from the NHIS and their own prior contact attempts to determine the best time for each successive attempt.

• For households that refuse an initial request to participate, the interviewer and supervisor decide on an approach for attempting to convert the refusal, taking into account all information available from the NHIS and prior contact efforts. Depending on the specifics of each case, one of the refusal conversion letters may be sent before another attempt is made in person, points to be made to address the reasons for the refusal are discussed, and frequently, a different interviewer will be assigned to make the next attempt.

• For households that require tracking, the interviewer who determines that the household has moved makes initial, local attempts to obtain new locating information. When those local sources are not successful, the case is referred to senior field management staff for additional searching through approved internet resources.

Since resources—time, budget, and staff—are not limitless, selection of the areas and specific cases on which to concentrate effort is critical. To guide these decisions, the project draws on multiple sources of information: information from prior panels on the characteristics of responders and nonresponders, information from the NHIS on the characteristics of the sampled households, paradata from the project management system, and information on the location, experience level, and skill sets of the interviewing field force.

While recent NHIS survey and sample design changes and planned upcoming instrument changes have the potential to effect MEPS response rates and data quality, it is not completely feasible at this juncture to assess any impact of these changes. The NHIS redesign will be fully launched in January 2019.

Medical Provider Component

MEPS-MPC staff plans to maintain the high response rates for the MEPS-MPC by bringing forward to the current data collection effort the methods that have been successful in maintaining provider cooperation in the past. An initial telephone screening to confirm provider eligibility and determine the appropriate person to whom the survey materials should be sent and the mailing, faxing, or electronic submission of an advance package to the provider which describes the survey and the types of information that will be collected (and includes the permission forms) helps to maintain the high response rates.

Data collection staff who appreciate the difficulty and importance of the task, and are capable of establishing good rapport with providers and placing as little burden on them as possible to accurately collect the data, will be recruited and retained. All data collection staff participate in an in-depth initial training as well as on-going performance improvement activities. MEPS-MPC identity and logos will be maintained so that providers who have participated in the past will recognize the study, but data collection materials will be customized to the current year’s data collection so providers understand what is currently being requested of them. Data collection protocols and instruments are also customized to the different types of providers to make it as easy as possible for providers to provide data in the manner in which it appears in their records. Providers with a previous history of being reluctant to participate will be assigned to data collection staff specializing in working with such respondents to maximize the possibility that they will participate. Providers with particularly large numbers of study patients will be assigned to staff capable of working out means of obtaining the large number of records with minimal burden to the provider. Finally, the use of an electronic data capture system, which allows real-time checking for the entry of complete and accurate information into the data collection forms while they are being filled out, helps minimize return calls to providers to resolve missing or confusing items and make it more likely that their cooperation will be maintained in future data collection efforts.

4. Weighting and Nonresponse Adjustment

The general weighting approach that has been used for earlier MEPS-HC panels will continue to be employed in future years. This includes nonresponse adjustments to sample weights to reduce the potential for bias in sample estimates arising from nonresponse at the household, family, and person levels. After adjustments for nonresponse, for those MEPS weights that pertain to population counts for which stable estimates can be obtained from independent sources (e.g., the civilian, noninstitutionalized population or adult members of this population), MEPS weights are calibrated to such estimates. Specifically, MEPS nonresponse adjusted weights are poststratified or raked to demographic and geographic subgroup population estimates based on corresponding data from the Current Population Survey (CPS). Such calibration serves to reduce the potential for bias arising from under coverage and can be expected to help reduce sample variability as well.

Under the current weighting scheme, a household-level poststratification and a nonresponse adjustment are applied utilizing the household characteristics collected in NHIS for both responding and nonresponding households to MEPS. Some of these household characteristics are based on characteristics of the household reference person (i.e., the person identified as owning or renting the home) such as age, sex, race/ethnicity, education, marital status, employment status and income of the reference person. With the planned NHIS instrument change, neither a reference person will be identified nor will detailed information about all persons in the household be collected. Consequently, the weighting procedure will be modified slightly where characteristics of the sample adult can be used instead of the reference person. This modification is not likely to have a major impact on the poststratification or nonresponse adjustments.

Nonresponse Bias Studies

Nonresponse bias concerns arise to the extent that nonrespondents differ from respondents, particularly on key analytic variables, and how well the responders represent the target populations of interest. Since the MEPS-HC sample is drawn from NHIS participant households, the NHIS provides the best source for identifying characteristics of responders and nonresponders. The analyses also include across panel comparisons in MEPS.

Using weighted response rates, nonresponse bias analyses periodically conducted in MEPS examine the following:

• How well do responders represent the target population on key characteristics such as race and ethnicity, urban/rural status, age, household size, income level, etc.

• Whether responders and nonresponders differ on key analytic variables such as health insurance status, chronic disease status, and health care utilization – all of these items are collected in both the NHIS and MEPS-HC. Linkage to the NHIS provides substantial information for this assessment.

• What are the contact patterns identified by paradata for responders and whether these data can predict propensity to respond. These paradata are obtained from both the NHIS and MEPS-HC such as length of the interview, number of contacts, mode of contact, etc. Appropriate multivariate analytic methods are employed to determine if contact data correlates with propensity to respond.

An analysis conducted by AHRQ showed that weighted estimates based on the MEPS final responding sample are mostly consistent with estimates from the full NHIS which are based on a much larger sample size (https://meps.ahrq.gov/data_files/publications/workingpapers/wp_13002.pdf). This provided evidence of minimal bias in MEPS estimates. Moreover, as mentioned above, the variables from the NHIS that are used to adjust for nonresponse are often evaluated and updated to help reduce nonresponse bias. This process of nonresponse adjustment is found to be effective since the characteristics of nonrespondents in MEPS are available from the NHIS.

5. Tests of Procedures

Whenever major changes are made to the MEPS they are pretested to ensure that data quality is not negatively impacted. In 2015, the MEPS contract was modified to fund a technology upgrade and modernization for the data collection instrument and supporting systems. The period of performance was April 1, 2015 through November 30, 2017. The new modernized CAPI instrument was fielded in January 2018 with the beginning of the spring rounds of data collection (Panel 21 Round 5, Panel 22 Round 3 and Panel 23 Round 1). The new instrument is designed to streamline administration of the CAPI interview, simplify the response task for both the respondent and interviewer, and increase respondent reporting while maintaining or reducing costs. Development of the new CAPI instrument in a Blaise commercial off the shelf (COTS) environment began with the development of Blaise COTS User Interface conventions and on-going research and evaluation activities guided by AHRQ oversight and Westat design staff, methodologists, MEPS programmers, and corporate Blaise programmers. Throughout the 2 ½ year period of performance, development of the new CAPI instrument included continuous, iterative cycles of specification, testing, and evaluation. Westat design staff conducted detailed walk-throughs of programming specifications for each section of the instrument for both AHRQ and Westat programming staff followed by demonstration of the programmed modules on a flow basis. A testing lab was set up at Westat so that AHRQ and Westat staff had full access to each of the multiple, interim instrument releases that preceded the more formal alpha and beta versions of the instrument. Testing of the developing instrument was guided by increasingly sophisticated and complex testing scenarios. Test data preparation used an iterative process that first yielded data for round 1 interviews for the most common scenarios, and then increased the breadth of scenarios within round 1 before moving into the preparation of data for later rounds. Actual test scripts were also developed to cover all aspects of the design layout for each question within a section (e.g. adherence to conventions for display of question text, interviewer instructions, response categories, etc.) and to cover the full range of routing instructions through each questionnaire section. Multiple rounds of usability testing were conducted with field staff with varying levels of technical skills. Online training modules on questionnaire flow and navigation of the new CAPI instrument were also piloted with field staff to inform later design proposals and define training points to establish field interviewer training goals. No additional, significant changes to the 2018 Blaise COTS instrument that would require pretesting are being implemented at this time.

6. Statistical Consultants

The following are responsible for statistical aspects of the MEPS Study:

Steve Machlin, M.S.

Director, Div. of Statistical Research and Methods

Center for Financing, Access and Cost Trends

AHRQ

(301) 427-1480

Sadeq Chowdhury, Ph.D.

Division of Statistical Research and Methods

Center for Financing, Access and Cost Trends

AHRQ

(301) 427-1666