Supporting Statement

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

July 24, 2014

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the National Center for Health Statistics (NCHS) Questionnaire Design Research Laboratory (QDRL) (OMB No. 0920-0222, exp. 06/30/2015) plans to conduct a feasibility study designed to identify options and provide recommendations for collecting additional data on physician practice characteristics and medical organization characteristics through the National Ambulatory Medical Care Survey (NAMCS) (OMB No. 0920-0234, exp. 12/31/2014) mail survey. The feasibility study is a collaborative effort between NCHS and the Agency for Healthcare Research and Quality (AHRQ). The study will begin as soon as OMB approval is obtained.

This research has the goal of improving methods for collecting data on physicians and their practices, particularly organizational information. This goal will be met by conducting a pretest in which three alternative questionnaires will be fielded using an experimental design: a physician questionnaire, a medical organization questionnaire, and a combined physician and medical organizational questionnaire. This experiment is expected to generate recommendations for future, larger data collection through NAMCS.

During or after the feasibility study, a combination of cognitive interviews and focus groups will be conducted to investigate how questions are interpreted and to identify items requiring further refinement.

In order to continue providing relevant, objective data to inform health care policy and serve a variety of research needs, NCHS must adapt surveys in response to changes in the health care system. Hospital networks are acquiring private practices, physicians are becoming employees of larger health care systems, and systems are evolving into accountable care organizations. In this environment, it is important for NCHS to collect data not only about physician practices and their patients, but also on how health care delivery is organized and how the medical organizations and health care systems are coordinating that care.

The feasibility study will identify possibilities for expanding the role of NAMCS to collect these data. It will also explore the feasibility of widening the scope of NAMCS to include all physicians engaged in clinical work and the different settings in which physicians work. Questions will provide information about their medical organizations, the patients they care for, daily activities of physicians (including clinical and nonclinical time), and emerging issues affecting physician practices. Results from this study may lead to further proposals to expand NAMCS to collect data along these lines.

NAMCS Feasibility Study

Physicians play a vital role in the American health care system. Their provision of direct medical care and their authorization for the majority of medical services make their decisions directly or indirectly important for access to care, quality of care, and a significant proportion of U.S. health care spending (Martin et al., 2011). Given their key role, accurate and timely information about physicians is essential for understanding the functioning of the health care system, identifying best practices and potential efficiencies. Many of the changes taking place in the health care market are occurring at the organizational level. Physicians may or may not be the most reliable respondents for questions about organizational changes that do not directly affect the physician-patient interaction. However, organizational information is vitally important for assessing the changing health care landscape. This project aims to fill that gap by providing data on optimal methods for collecting organizational information about the settings in which all physicians practice.

This feasibility study is being conducted by NCHS, through its contractor SRA International, as part of the National Ambulatory Medical Care Survey (NAMCS). NCHS is building on the work of AHRQ which had previously convened a technical expert panel to provide input on a universal survey of physicians. NCHS has further refined and developed the instruments through its experience with NAMCS and other physician surveys. NCHS is doing this work to explore augmentation of NAMCS scope, content, and methods, which is part of NCHS’ long term vision for capturing health care delivery across the many settings in which physicians work. NCHS will continue this work to be responsive to the needs for data on the changing health care delivery system.

Background on the National Ambulatory Medical Care Survey (NAMCS)

The National Ambulatory Medical Care Survey (NAMCS) was conducted from 1973 through 1981, in 1985, and since 1989 has been an annual survey.

The term “core NAMCS” refers to the traditional sample of nonfederal office-based physicians, and physicians and mid-level providers sampled in community health centers (CHCs). The specific purpose of the core NAMCS is to meet the needs and demands for statistical information about the provision of ambulatory medical care services in the United States, and as such, fulfills one mission of NCHS: to monitor the nation’s health. Ambulatory services are rendered in a wide variety of settings, including physician offices and hospital outpatient and emergency departments. Since more than 80 percent of all direct ambulatory medical care visits occur in physician offices, NAMCS provides data on the majority of ambulatory medical care services. To complement these data, NCHS also conducts mail surveys of physicians. The Department of Health and Human Services (DHHS) Office of the National Coordinator for Health Information Technology (ONC) has sponsored the NAMCS National Electronic Health Record Survey (NEHRS) and the NAMCS Physician Workflow Survey; both mail surveys are supplements of NAMCS.

Methodological design of the proposed feasibility study

This feasibility study expands the existing core NAMCS and the NAMCS mail surveys in several important ways. First, unlike NAMCS, all physicians regardless of specialty are included in this survey. Second, the NAMCS feasibility survey contains an experiment to determine the quality of information about the medical organization provided by different potential respondents (the physician, practice administrator or the physician’s choice of best respondent, see medical organization experiment below). Third, the survey includes content on practice characteristics and physician activities that extend beyond what is currently covered in NAMCS. Lastly, the feasibility study questionnaires will be evaluated through cognitive testing and focus groups which will identify items requiring further refinement in order to assure they meet NCHS survey methodological and statistical standards. Some prior cognitive testing informed the design of the current, yet more information about how these questions compare to current NAMCS questions requires further cognitive interviews and focus groups.

New questions were developed and guided by a thorough environmental scan, an expert panel, a broad stakeholder meeting, and extensive cognitive testing. For example, to obtain information on the time spent by physicians on various clinical activities, NAMCS includes questions on patient volume and, through chart review, gathers information on types of services provided. This information can be used to estimate the proportion of time physicians practicing in ambulatory care settings for more than 20 hours per week spend providing services noted in patient medical records. However, this does not provide information on how physicians in various specialties allocate their time across various clinical settings throughout the typical work week. Since the NAMCS feasibility study sample includes all physicians in all settings, the questions designed to gather information about how they spend their time at work include categories relevant for physicians in other settings such as hospitals or imaging centers. These include procedures, chart review, emailing patients, interpreting x-rays, and other activities not captured through NAMCS.

Medical organization experiment:

Three questionnaires were developed using data collected from NAMCS and other surveys over the past decade, feedback from the technical expert panel and expertise at NCHS -- The Physician Component of NAMCS Feasibility study, the Medical Organization Component of the NAMCS Feasibility study, and the combined Physician and Medical Organization Component to the NAMCS Physician Feasibility study. The experiment will assess the optimal respondent and method for reaching that respondent for the Medical Organization component. To test the feasibility of obtaining practice-level information from a respondent other than the sampled physician, sample members in the medical organization experiment will be randomly assigned to one of three approximately equally-sized experimental groups (Group 1-3). A fourth group of sampled physicians is not in the experimental group for the Medical Organization component and will just receive the Physician Component (control).

The data collection methods outlined in the experiment will examine three methods for collecting information about the organizations in which physicians practice:

• Group 1: Physician is asked to complete the NAMCS Physician and Medical Organization Survey, which contains the same content from both physician and medical organization surveys.

• Group 2: Physician is asked to complete the NAMCS Physician Survey but allowed to choose who the best respondent is for the NAMCS Medical Organization Survey, which may include the sampled physician.

• Group 3: Physician is asked to complete the NAMCS Physician Survey while their practice administrator, identified separately, is asked to complete the NAMCS Medical Organization Survey.

• Control: Physicians in the control group will only receive the NAMCS Physician Survey.

Test objectives.

The data collection effort will result in several methodological and substantive outcomes. The analysis of study data, including the embedded experiments, will answer the following research questions:

• Practice Organization Experiment

• To what extent do unit and item response rates for the practice organization module vary by experimental group (Group 1-4)? Do they differ by specialty and organizational characteristics?

• Do physicians or other practice administrators provide more complete organization-level information, with completeness measured across relevant item responses? (Groups 2, 3)

• To what extent can practice administrators be located successfully without the help of a physician? (Group 3)

• Modular component – Primary care module

• Evaluate the use of the primary care module as a ‘modular’ strategy to provide survey-to-survey flexibility in emphasizing different or emerging aspects affecting physician practices (PS-Q20a-k)

• Targeted Item analysis – identifying problematic items

• What items have higher than average non-response across all sections of the physician survey, organization survey, and the physician and organization survey?

• Are there any specific specialties that have difficulty answering the screener questions (Physician Survey (PS) Q2,Q7,Q38)?

• Are physicians across a variety of settings able to answer items about the health care system/organizational level characteristics? (PS - Q21-28; Organization Survey (OS) Q1-Q18)?

• Are there specific types of organizations (PS- Q28) that have characteristics (PS- Q21, Q23, Q25, Q26) requiring further cognitive testing of relevant items?

• To what extent do items with high item nonresponse on the feasibility study questionnaires compare to similar content items on the NAMCS surveys?

• Are there items with write-ins, unknowns, N/A’s or other specify that require further refinement/restructure to improve the item? (PS- Q7, Q8, Q11, Q15,Q16, Q17, Q23. Q24, Q25, Q32, Q33, Q36; OS- Q1,Q5-Q8, Q10-Q13;Q15;Q16f; Q18)

• Are there any notes or indications of problematic questions either by mail or phone-follow-up?

• What is the physician’s preferred mode to answer the survey (i.e., web vs. mail survey)? (PS-38 OS-Q19) This will help us evaluate the preferred mode to collect data from physicians.

• Are the physicians skipping to the end of the survey, not eligible for the survey?(PS - Q1,2,7) How good are the NAMCS screener items?(PS - Q2,7) What are their reasons for skipping out of the survey?(PS - Q40) This will help us understand whether we are missing people otherwise in scope and evaluate the items used for eligibility in the NAMCS feasibility study (PS- Q1,2,7,40 ).

• Evaluate the use of the primary care module/‘modular’ strategy to provide survey-to-survey flexibility in emphasizing different or emerging aspects affecting physician practices (PS-Q20a-k).

Cognitive testing

In addition to identifying the respondents best suited to provide organizational data, the feasibility study will identify items across instruments that have high item nonresponse and topic areas that may differ from current NAMCS data collection. After identifying these challenging items, we will evaluate these items through cognitive testing and focus groups and targeted follow-up of a diverse group of physician respondents. Because we are expanding the scope of NAMCS to non-traditional NAMCS respondents involved in clinical care yet in settings where they may not see ambulatory patients (e.g., radiologists, pathologists, anesthesiologist) we will want to ensure that items are vetted through cognitive testing and through focus groups. We will evaluate items on the feasibility surveys (e.g., primary care, health information technology, payment participation arrangements, physician work time, workforce) to determine whether these survey items or comparable items currently on the NAMCS are best suited for collecting information about the physician, their organization, and their primary practice location across many diverse physicians with the objective of expanding the role of NAMCS to all physicians. Thus, a second goal of the feasibility study is to determine which questions most effectively yield high quality data on this universal group of physicians.

In particular, the tests are designed to increase the ability to understand the limitations of specific items and whether the NAMCS screener question for clinical care and hours of clinical work is an effective approach to ensuring that the instruments meet data collection goals. Additionally, understanding the potential limitations of data collection about the organization and item improvements would allow NAMCS to collect better information about the organization.

To further refine the data collection instruments, cognitive testing and targeted focus groups will be used. The following items will be targeted for cognitive testing and focus groups: PS- Screener items (Q2,Q7,Q38); PS work time (Q4-Q10); PS-primary practice location (Q11-Q19); PS-Primary care module (Q20, 20a-k); PS-organization (Q21-28); PS- physician (Q29-37). Health care system/organizational level characteristics (Q21-28)?

Protocol for cognitive testing and focus group. As many as fifty 90-minute cognitive interviews and five 90-minute focus groups of ten individuals (50 respondents) may be conducted. The testing procedure will conform to the cognitive interviewing and focus group techniques that have been described in our generic OMB clearance package. Cognitive interviews and focus groups will be conducted by QDRL staff members in the laboratory or a private room at a mutually agreeable location. With the consent of the respondents, cognitive interviews and focus groups will be audio recorded. Respondents will be informed of recording procedures in the process of reviewing the consent forms, and the equipment will be turned on once it is clear that the procedures are understood and agreed upon.

We plan on providing the individuals participating in the 90-minute cognitive interview a $100 incentive for their participation, and we plan on providing individuals participating in the 90-minute focus group a $100 incentive. This amount has been increased over and above the normal cognitive interview and focus group incentive level for a number of reasons. First, the recruitment of physicians is necessary for the success of the cognitive study and focus groups. Second, given funding and time constraints cognitive interviews and focus groups will be limited to the Washington DC/Baltimore metropolitan area. Lastly, we will be asking the respondents to participate in the cognitive interview during their working hours, placing an extra burden on these respondents. Our proposed incentive level will be invaluable to obtaining a high response rate and reducing the number of cancelations from this busy, specialized population.

At the end of the cognitive interviews and focus groups, respondents will be paid and provided with copies of all papers they signed.

Extreme care will be taken with all recordings and paperwork from the cognitive interviews and focus groups conducted off-site (outside of the laboratory). Recordings and identifying paperwork will be stored in a secured computer and travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets.

Roles and responsibilities of the parties involved

This feasibility study is being conducted by NCHS through its contractor SRA International (SRA) in collaboration with AHRQ as part of the National Ambulatory Medical Care Survey (NAMCS). AHRQ is the sponsor of the feasibility study.

NCHS will secure the American Medical Association (AMA) and American Osteopathic Association (AOA) statistical file and secure a random sample of physicians for the feasibility study, and provide SRA project materials (sample files, advanced letters, processing instructions, instruments, and a list of project milestones) in order to implement project data collection. NCHS and SRA will monitor data quality during the field period. After the field period, NCHS will edit and process data for quality control and assure data meet our confidentiality requirements. Additionally, NCHS will ensure AHRQ receives a quality check file and a final file to ensure quality expectations.

The NAMCS feasibility study developed new items to assess a number of key domains that are not assessed in the NAMCS- these include the provision of various aspects of primary medical care, characteristics of the diverse types of medical organizations employing physicians, and physician views of those organizational arrangements.

Sample design

Although NAMCS excludes anesthesiology, radiology, and pathology, and only focuses on office-based physicians, the target universe for the NAMCS Physician Feasibility study is expanded to include all active doctors of medicine (M.D.s) and doctors of osteopathy (D.O.s) in the 50 States and the District of Columbia. The NAMCS Physician Feasibility study universe will exclude physicians who are retired, have left the field of medicine, medical trainees (residents/fellows), are federally employed (e.g., Bureau of Prisons, Veteran Administration, Indian Health Service, etc.), are in the Military, or provide fewer than four hours of clinical work per week. The sampling frame will be constructed from the American Osteopathic Association (AOA) and the American Medical Association (AMA) Physician Master file, a list of 1.4 million physicians in the United States and outlying territories. To avoid the cost of purchasing the entire list from the AMA (Medical Marketing Services Inc. (MMS) the company that maintains the Master file for marketing and research purposes), MMS will construct the sampling frame from criteria provided by NCHS. These will include instructions to remove physicians from the sampling frame if they are part of any of the excluded categories cited above, making the total number of physicians in the sample frame slightly smaller than 1.4 million. MMS will provide NCHS the limited AMA statistical file containing the criteria for NCHS to select the sample from the sampling frame. Thereafter, NCHS will select a final sample and receive a sample twice the size of our final sample from MMS, as more detail about each physician is needed from MMS (e.g., physician name, contact information, other provider and organization characteristics). Since we are getting more detail about those physicians in the sample, MMS will be asked to sign nondisclosure agreements to protect the confidentiality of the sample.

Many NAMCS respondents are physicians in solo practices. In order to reduce respondent burden for these and all respondents, the sampling procedures are designed to limit participation to once every three years for the combined samples of the traditional NAMCS, NAMCS mail surveys, and the NAMCS Feasibility study participants.

Screening questions will exclude physicians who are ineligible because they do not do any clinical work (defined as providing less than four hours of clinical work per week). Because the AMA file does not provide this level of detail about each physician’s clinical work hours per week, NCHS will screen sampled physicians for eligibility using screener questions items 2, 7 (see Attachment 1 and 3). These screening questions instruct the respondent to end of the survey if the eligibility criteria are not met.

NCHS anticipates a 60 percent response rate and 60 percent eligibility rate for sampled cases from the AMA-created sampling frame. This rate is consistent with NCHS’ previous experience with physician mail surveys that include telephone follow-up. To obtain 1,764 completed physician interviews, and 504 organization interviews, NCHS will contact 4,900 physicians and up to 700 practice administrators. Of the 4,900 sampled physicians, 2,100 will be randomly assigned to one of three approximately equally-sized experimental groups (700 physicians in each group). (Groups were described above). The remaining physicians (n=2,800) will receive the physician survey only.

Recruitment methods and advanced letters

The mail survey is a self-administered paper questionnaire that is sent from NCHS and returned in the mail by the sampled physician, practice administrators, and select medical organizational staff. Each mail survey will include an introductory letter and a survey questionnaire. Two subsequent mailings which include modified introductory letters and the survey questionnaire will be sent to non-respondents. After all three mailings, telephone calls will be made to all non-responding physicians in a final attempt to obtain survey data. If the physician is contacted and agrees to participate, the information will be obtained via telephone.

Advance letters are modified for each experimental group and scenario and have been approved by the NCHS Director (see Attachment 4). For example, Group 2 allows the physician the option of choosing the best respondent to complete the organizational questionnaire, which may be the physician. For non-responding physicians, different letters are required to take into account if the physician survey or medical organization survey or both were not completed. Therefore there are 3 different 2^nd and 3^rd mailing letters (see Attachment 4).

Instruments

To achieve the goals of this project the following data collection instruments will be administered:

1. NAMCS Physician Survey. The questionnaire used for the physician component and will be completed by a random sample of physicians and includes questions to assess physician demographics, career satisfaction, details of a typical work day, compensation, practice location characteristics, and patient characteristics. Questions included will also assess the following five dimensions of primary care at the physicians’ practice: (1) access, (2) continuity of care, (3) coordination of care, (4) comprehensiveness of care, and (5) whole-person care (see Attachment 1). NOTE: In the context of the feasibility study, a subset of these primary care questions (PS: Q20a-k) provides an example of a potential long term ‘modular’ strategy to provide survey-to-survey flexibility in emphasizing different and/or changing aspects of physician practices. Primary care plays the role of a special current emphasis within the prototype survey of the feasibility study. For example, over a 2 year cycle NCHS could replace a subset of questions (35a through 35j) with equivalent-burden questions, depending on changing information needs and results of the feasibility study.

2. NAMCS Medical Organization Survey. The questionnaire used for the medical organization survey will be completed by either the physician, the practice administrator, or the person the physician determines is the best person to answer questions about the medical organization. The questionnaire will assess characteristics of the medical group, including its relationship to networks, care management infrastructure, and payment and incentive arrangements (see Attachment 2).

3. NAMCS Physician and Medical Organization Survey. This questionnaire combines the content of the two surveys to be administered in one instrument (Attachment 3).

Assurance of Confidentiality and safeguarding of materials

All staff (NCHS and contractual staff) who have access to confidential information are given instruction on the requirement to protect confidentiality, and are required to sign a pledge to maintain confidentiality; only such authorized personnel are allowed access to confidential records, and only when their work requires it; when confidential materials are moved between locations, records are maintained to ensure that there is no loss in transit; and when confidential information is not in use, it is stored in secure conditions. Hard copies of the survey forms will be stored in a locked file cabinet at NCHS for 2 years after data collection. Thereafter, the records will be sent for storage at the National Archives.

The NAMCS Feasibility study will offer monetary incentives to respondents for participants in the cognitive testing and focus groups. However no monetary incentives will be offered to respondents for participation on the mail or phone surveys.

Burden statement:

In total, for this project, the maximum respondent burden will be 927 hours (75 hours for cognitive interviewing, 75 hours for focus groups, and 777 hours for feasibility testing (assuming 60% eligibility from the AMA Master file and 60% response rate)). In terms of the feasibility study, the Physician Survey will be completed by 1,512 physicians (Group 2 Group 3, and Control), which takes 20 minutes to complete. The NAMCS: Medical Organization Survey will be completed by 252 practice administrators (Group 3) and 252 physicians (Group 2) (504 total), which takes 15 minutes to complete. The 252 physicians (Group 1) will be completing the NAMCS: Physician and Medical Organization Survey, which takes 35 minutes to complete. Exhibit 1, noted below shows the estimated annual burden hours for each respondent’s participation time in this research project.

Exhibit 1.  Estimated annualized burden hours

Projects	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
QDRL Cognitive Interviews	50	1	90/60	75
QDRL Focus Groups (5 groups of 10)	50	1	90/60	75
Feasibility Testing
NAMCS: Physician Survey	1512	1	20/60	504
NAMCS: Medical Organization Survey	504	1	15/60	126
NAMCS: Physician and Medical Organization Survey	252	1	35/60	147
Total	2268	na	na	927

The estimated annual cost burden associated with the respondents’ time to participate in this research is estimated to be $82,560.00. These estimates are based on information obtained from the Bureau of Labor Statistics web site (http://www.bls.gov) , using the “May 2013 National Occupational Employment and Wage Estimates”. The average hourly wages presented in the table below for physicians was averaged across different specialties. For example, to better approximate costs, the estimate of $93.56 (office-based physicians) was an average based on the hourly salary of family and general practitioners, general internists, obstetricians and gynecologists, general pediatricians, psychiatrists, surgeons, and a catch-all category “Physicians and Surgeons, All Other.” The estimate used for practice administrators ($48.72) captures the average hourly wage for medical and health services managers.

Table of annualized respondent cost

Type of Respondent	Response Burden (in hours)	Average Hourly Wage	Total Cost
Office-based physicians	834	93.56	78,029.04
Office staff	93	48.72	4,530.96

References:

Martin A, Lassman D, Whittle L, et al. Recession contributes to slowest annual rate of increase in health spending in five decades. Health Affairs 2011 Jan;30(1):11-22. Available at: http://content.healthaffairs.org/content/30/1/11.full?sid=7c47031f-90bd-4c64-ac4e-706b06f4e892

Attachment (4)

cc:

V. Buie

T. Richardson

DHHS RCO

8