Leter to Dr. Schwab

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

February 29, 2015

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Questionnaire Design Research Laboratory (QDRL) (OMB No. 0920-0222, exp. 06/30/2015) plans to evaluate questions for the 2015 National Electronic Health Records Survey (NEHRS), which is an annual supplement to the ongoing National Ambulatory Medical Care Survey (NAMCS, OMB# 0920-0234).

We propose to start recruiting for volunteer participants as soon as we receive clearance and to start testing as soon as possible after that.

Background Information about Cognitive Testing of Questionnaires

The methodological design of this proposed study is consistent with the design of typical cognitive testing research. As you know, the purpose of cognitive testing is to obtain information about the processes people use to answer survey questions as well as to identify any potential problems in the questions. The analysis will be qualitative.

Proposed project: 2015 NEHRS Supplement

The 2015 NEHRS questions have been drafted to address the following objectives:

• Evaluate whether and how doctors, and their businesses, have implemented the use of Electronic Health Records (EHR).

• Examine the ways non-institutional doctors use EHR for medical purposes

• Examine the ways non-institutional doctors use EHR in conjunction with other entities, such as other medical practices and with public health agencies.

The 2015 NEHRS has already been approved by both the NCHS ERB and by OMB. Given the recent statutory changes pushing the adoption and meaningful use of EHR, DHCS decided that a thorough evaluation of the questionnaire was necessary. This cognitive interviewing project will evaluate the current form, and any suggested modifications that emerge from the analysis of the cognitive interviews may be added to the 2016 NEHRS. This evaluation project has been scheduled to meet DHCS’ production deadlines for 2016.

The 2015 NEHRS questions that we are evaluating appear as Attachment 1.

The target population of the NEHRS is non-institutional doctors (i.e. doctors who do not primarily practice out of a hospital or clinic). In order to iteratively evaluate the questions, we propose to recruit up to 50 respondents. We will actively recruit these

doctors, using a frame created by the QDRL for this and other NAMCS-related projects (as previously approved for the “DCHS NAMCS Feasibility Study, which was approved under OMB #0920-0222 on 07/25/14). We plan on recruiting doctors across three areas of practice: generalists, specialists, and surgeons (as defined by the American Medical Association’s SPECCAT typology¹).

The sample frame was constructed from the Centers for Medicare and Medicaid Services’ “Physician Compare” web tool². Potential respondents will be purposively sampled from the CMS-based frame on a number of attributes, including geographic location within the DC Metropolitan area, type of practice (solo, group, clinic, or hospital), and area of practice/specialty. Doctors sampled from the frame will be sent an advanced letter signed by Charles J. Rothwell, Director of NCHS (see Attachment 2), which explains the purpose of both the NEHRS and the proposed cognitive interviewing project, and alerts them to the fact that the QDRL will be reaching out to schedule an interview. The QDRL will then contact these doctors a week after the letter was sent and attempt to recruit them into the project. Potential respondents will be screened for their eligibility using the questionnaire found below in Attachment 3. It is anticipated that as many as 84 doctors may need to be screened in order to recruit 50 respondents. Please note that the wording of our eligibility screener has been approved and is contained within the umbrella generic package: Only project specific information has been added to the document. The testing procedure conforms to the cognitive interviewing techniques that have been described in QDRL’s generic OMB clearance package (No. 0920-0222, exp. 06/30/2015).

Interviews will be conducted by QDRL staff members with English speaking respondents. All interviews conducted in the Questionnaire Design Research Laboratory will be video and audio recorded to allow researchers to review the behaviors and body language of the respondents. Interviews conducted offsite will only be audio recorded. These recordings will allow researchers to insure the quality of their interview notes.

After respondents have been briefed on the purpose of the study and the procedures that QDRL routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent document (Attachment 4). Only project specific information has been added to the document. Respondents will also be asked to fill in their self-identified demographic characteristics on the Respondent Data Collection Sheet (Attachment 5), which has been modified for this particular establishment-based project. This document is contained in our umbrella package. The burden for completion of this form is captured in the interview.

The interviewer will then ask the respondent to confirm that he/she understands the information in the Informed Consent, and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon. The interviewer will then orient the respondent to the cognitive interview with the following introduction:

Lauren may have told you that we will be working on some questions that will eventually be added to national surveys. Before that happens, we like to test them out on potential respondents. The questions we are testing today focus on electronic health records. We are interested in your answers, but also in how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you hear certain words, and so on.

I want you to fill out the survey while I observe. Occasionally, I will stop you to explain some of your answers or how you thought about a question. Please try to tell me what you are thinking as you figure out how to answer. Also, please tell me if:

there are words you don’t understand,

the question doesn’t make sense to you,

you could interpret it more than one way,

it seems out of order,

or if the answer you are looking for is not provided.

The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start? If yes, answer questions. If not, let’s get started.

As the production NEHRS’ primary collection mode is self-report mail-back, the cognitive interviews will be primarily conducted using retrospective probing. The respondents will be presented with the questionnaire and will be observed by the cognitive interviewer during the administration in order to note any clear usability issues with the instrument. Following the completion of the instrument, the cognitive interviewer will retrospectively probe the respondent following the normal form outlined in our generic clearance.

After the interview, respondents will be given the thank-you letter (document contained in umbrella package) signed by Charles J. Rothwell, Director of NCHS, a copy of the informed consent document, and $100. This amount has been increased over and above the normal cognitive interview incentive level for a number of reasons. First, the recruitment of physicians is necessary for the success of the cognitive study. Second, given funding and time constraints cognitive interviews will be limited to the Washington DC/Baltimore metropolitan area. Lastly, we will be asking the respondents to participate in the cognitive interview during their working hours, placing an extra burden on these respondents. Our proposed incentive level will be invaluable to obtaining a high response rate and reducing the number of cancelations from this busy, specialized population.

After the interview, respondents will be asked to read the Special Consent for Expanded Use of Video and Audio Recordings (Attachment 5). There will be no coercion and the respondents will be told that they can call and reverse the decision at any time if they change their minds. If respondents do sign the special consent form they will be given a copy of that as well.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets.

In total, for this project, the maximum respondent burden will be 57 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden in hours
Screener	84	1	5/60	7
Questionnaire	50	1	60/60	50

Attachments (6)

cc:

V. Buie

T. Richardson

DHHS RCO

1 http://www.mmslists.com/definitionspdf/AMA%20Specialty%20Codes.pdf

2 http://www.medicare.gov/physiciancompare

2 | Page