SS-B NEHRS 14July2017

The National Electronic Health Records Survey (NEHRS), formerly known as the Electronic Medical Records Supplement (EMRS) from 2008-2011 when it was a part of the National Ambulatory Medical Care Survey (NAMCS) (OMB No. 0920-0234, Exp. Date 03/31/2019), has been conducted annually since 2008. The target universe of NEHRS, is non-federally employed physicians (excluding those in the specialties of anesthesiology, radiology, and pathology) practicing in the United States and classified as “office-based patient care.” The sampling frame used for the survey consists of physicians listed as being in the target universe by the American Medical Association (AMA) and/or the American Osteopathic Association (AOA). To enable state-based estimates from this survey, a sample of 202 physicians is selected from each state and DC (a national total of 10,302) annually for the NEHRS. Within each state, the physicians are selected using systematic random sampling from lists in which physicians are arrayed by specialty groups and metropolitan statistical area (MSA) status.

Data collection for the 2017 NEHRS (Attachment E) is expected to be conducted in the third quarter of 2017 and the supplemental survey will be conducted in the first quarter of 2018. The supplemental survey focusing on content related to physician attitudes on using EHRs (Attachments F1, F2, and F3) will be administered to the same sample as used for the 2017 NEHRS. At this time, the supplemental survey is only going to be administered on the 2017 NEHRS sampled physicians. Most of the content of the NEHRS supplement come from previously approved NEHRS survey questionnaires with modifications that are responsive to decisions made in the context of the constantly evolving program goals of ONC. Furthermore, some of the content come from the Physician Workflow Supplement (2011 – 2013) (OMB No. 0920-0234, Exp. Date 03/31/2019), which was a study that expanded on the 2011 NEHRS to help measure progress towards the HITECH program goals and to provide insight into where scarce resources need to be devoted to help physicians achieve Stage I and Stage II meaningful use of certified EHR technology.

Following the design model used in the Physician Workflow Survey, the eligible respondents to the 2017 NEHRS and non-respondents to the 2017 NEHRS will each be sent a 4-page supplemental survey (20-minute survey) (Attachments F1, F2, and F3). There will be common content across all 3 types of respondents, but also tailored content that will allow delving into their experience (or lack thereof) with electronic exchange of information as described below.

Respondents to the 2017 NEHRS who reported they either electronically send or receive data from outside sources. They will be asked to answer questions (Attachment F1) that relate to their experiences with electronically exchanging information.

Respondents to the 2017 NEHRS who reported they neither electronically send nor receive data from outside sources. They will be asked tailored questions (Attachment F2) regarding why they are not engaging in electronic exchange of information (e.g., specific types of barriers).

Non-respondents to the 2017 NEHRS are physicians who did not respond to the initial 2017 NEHRS survey and will receive a brief 4 page survey (Attachment F3) consisting of some content from the 2017 NEHRS and the 2017 NEHRS supplement. This will allow for more robust national estimates related to EHR interoperability (i.e., measured by electronically sending, receiving, integrating and searching for patient health information) by offering non-respondents another opportunity to answer key questions related to interoperability.

The 2017 NEHRS and NEHRS supplement data will be weighted to produce state and national estimates using the inverses of selection probabilities with non‑response adjustments done within state and specialty group. Both data sources will undergo calibration adjustment factors to adjust estimated total physicians to known totals within specialty strata. Sampling errors are computed using the linearized Taylor series method of approximation as applied in the SUDAAN software package, which take into account the complex sample design of NEHRS. A description of the software and its approach has been published.^¹

2. Procedures for the Collection of Information

Data Collection

As mentioned previously, the survey formerly known as the NAMCS Electronic Medical Records Supplement, NEHRS will continue to be fielded by NCHS with a sample of 10,302 physicians.

In order to keep costs as low as possible, the sample mentioned above will continue to be conducted using self-administered web questionnaire, self-administered paper questionnaire or computer-assisted telephone interviewing (CATI).

NEHRS data collection for the 2017 NEHRS and NEHRS supplement follows the Dillman^² survey method with some modifications. Sampled physicians will be invited to participate on NEHRS through the Internet by mail or if available through email as email addresses are obtained from a database of physicians; Attachment G represents letters for the 2017 NEHRS while Attachments H1 and H2 represent letters for the 2017 NEHRS respondents and non-respondents in the 2017 NEHRS supplement data collection, respectively. Mail and email will provide physicians with login instructions for the web version of the survey along with elements of informed consent. A follow-up email will be sent about 7-10 days later, and 7-10 days prior to each milestone for those we have email addresses.

About 4 weeks after inviting physicians to take the survey through the Internet, the contractor will mail the questionnaire along with an introductory letter (Attachment K for 2017 NEHRS , Attachments L1 and L2 for 2017 NEHRS supplement), a 2017 NEHRS questionnaire (Attachment E for 2017 NEHRS, Attachments F1, F2 and F3 for NEHRS supplement), a pen, and a previous data brief (Attachment M1) and a self-addressed return envelope for the paper questionnaire. For the NEHRS supplement, a previous QuickStats (Attachment M2) will be used in place of a data brief. One week after sending the questionnaire by mail, all sampled physicians will receive a postcard thanking them for their participation or reminding them that their participation is still needed (Attachment K for 2017 NEHRS, Attachments L1 and L2 for 2017 NEHRS supplement). This postcard also allows sampled physicians to request additional information or another copy of the survey instrument. About 4 weeks after the initial questionnaire mailing, nonresponding physicians will be sent a 2nd mailing, which will include a modified introductory letter (Attachment K for 2017 NEHRS, Attachments L1 and L2 for 2017 NEHRS supplement), a paper questionnaire and self-addressed return envelope for the paper questionnaire. Three to four weeks after the 2nd mailing, non-respondents will receive a third mailing that will include a modified introductory letter (Attachment K for 2017 NEHRS, Attachments L1 and L2 for 2017 NEHRS supplement), the paper questionnaire and a self-addressed return envelope. All letters inform respondents of the voluntary nature of the survey.

Roughly three weeks after the 3rd mailing, telephone calls using computerized-assisted telephone interviewing will be made over the next 9 to 10 weeks to all remaining non-responding physicians in a final attempt to obtain survey data. If the physician is contacted and agrees to participate, the survey will be administered via telephone. If the physician declines participation this will be documented by the interviewer.

The contractor will track, document and collect response to the web-based questionnaire and the paper questionnaires. No individual patient data will be collected. See Attachment E for a list of questions that will be used for data collection and Attachment N for the phone script in 2017 NEHRS. See Attachments O1, O2, and O3 for the phone scripts for the supplemental survey (Attachments F1, F2 and F3), respectively.

Monitoring Data Collection and Quality Control

Throughout the data collection period, conference calls are held among Ambulatory and Hospital Care Statistics Branch (AHCSB) staff and the contractor who handles the mailing and follow-up phone calls to discuss issues relevant to the data collection. There are project management principles that guide our quality control, including weekly data status reports to monitor the flow and completeness of data collection. NCHS also includes internal staff to receive mailings and phone calls to allow NCHS to monitor the quality of the products to ensure materials, survey mailings and phone scripts are followed to our specifications.

As in any survey, results are subject to both sampling and nonsampling errors. Nonsampling errors include reporting and processing errors, as well as biases due to nonresponse and incomplete response. To eliminate ambiguities and encourage uniform reporting, attention has been given to the phrasing of items, terms, and definitions.

Quality control procedures and edit checks reduce errors in data coding and processing. During processing, our contractor performs a 100 percent independent rekey of the data. The contractor performs adjudication if discrepancies occur in the two sets of keyed data Additionally, NCHS staff go on a site visit to evaluate, monitor, and improve on the contractor’s survey operations.

Estimation Procedures

NEHRS data can be used to produce national and state estimates of nonfederal office-based physicians and their practices. The estimation procedure has four basic components: (1) inflation by reciprocals of the sampling selection probabilities, (2) calibration to known totals, (3) adjustments for nonresponse, and (4) weight smoothing. Starting in 2010, physicians who cannot be located are deemed as out-of-scope under the assumption that if survey personnel cannot find them in the survey telephone follow-up, patients would also not be able to find the physicians as required if the physicians were, indeed, seeing patients in an office-based practice.

Physician weights are used to estimate national numbers and characteristics of office-based physicians (e.g., sex, age, and specialty) and their practices (e.g., numbers of physicians in the practice and single-specialty compared with multispecialty practices). The physician sampling weight can also be modified to produce a national medical practice estimator (e.g., practice size, breadth of specialization, and selected EHR functionalities available onsite).

The relative standard error (= standard error/estimate) is one criterion that NCHS uses to determine reliability. For an estimate to be considered reliable, its relative standard error must be 30 percent or less. The sample of 202 physicians per state is sufficient to produce relative standard errors of 30 percent or less for the key statistics targeted in the survey (namely, for each state, the percent of physicians adopting and the percent using specific features of electronic health record systems). Such precision is adequate for planned analysis of the survey data.

Sampling Errors

Standard errors are calculated using a first‑order Taylor series approximation method as applied in SUDAAN software.

3. Methods to Maximize Response Rates and Deal with Nonresponse

The 2015 NEHRS had an unweighted response rate of 52 percent, and a weighted response rate of 49 percent. We expect the 2017-2019 NEHRS response rates to be similar to those in 2015, because the procedures and materials are the same or similar to those used in 2015. Efforts to raise the response rate of future surveys are currently ongoing.

NEHRS uses multiple methods for maximizing physician response. The survey form is designed to minimize the time required of physicians to participate. Along with the questionnaire, we provide a NCHS report that uses NEHRS data to show the importance of the survey.

Extensive web searches and follow-up phone calls will be performed to locate the status of non-responding physicians. Techniques for converting refusals have been developed that are quite effective, each flexible and responsive to individual concerns. Conversion is successful by emphasizing the following ideas: professional responsibility to enhance knowledge of the adoption of EHR in the United States, and data are only reported as descriptive statistics.

NCHS will investigate the specific causes of nonresponse, so as to devise additional corrective measures, funding permitting. This may include further understanding about survey methods in order to inform the reason for non-response (e.g., burden, brand, time, content). Each year in an annual statistical report, we describe weighted characteristics of NAMCS physician respondents and nonrespondents on numerous variables including age, gender, geographic region, metropolitan statistical area (MSA) status, type of doctor, specialty, specialty type, type of practice, and annual visit volume.

4. Test of Procedures or Methods to be undertaken

No tests of procedures are anticipated because the survey questions and procedures have been used in prior surveys.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The statistician responsible for the survey sample design is:

Iris Shimizu, Ph.D.

Mathematical Statistician

Statistical Research and Survey Design Staff

Office of Research and Methodology

National Center for Health Statistics

(301) 458‑4497

[email protected]

The data will be analyzed under the direction of:

Carol DeFrances, Ph.D.

Chief, Ambulatory and Hospital Care Statistics Branch

Acting Ambulatory Care Team Lead

Division of Health Care Statistics

National Center for Health Statistics

(301) 458 4440

[email protected]

1 Research Triangle Institute. SUDAAN User’s Manual, Release 11.0.1. Research Triangle Park, NC: Research Triangle Institute, 2013.

2 The Dillman survey method, also known today as the Tailored Design Method (TDM) is often regarded as the standard for mail surveys. The Dillman survey method includes steps such as sending a personalized letter, the questionnaire with return postage prepared, a follow-up postcard, and duplicate packets to non-respondents.

File Type	application/msword
File Modified	2017-07-14
File Created	2017-07-14

SS-B NEHRS 14July2017

SS-B NEHRS 14July2017.doc

National Electronic Health Records Survey (NEHRS)

OMB: 0920-1015

Table of Contents