Supporting Statement B

Quantitative Survey of Physician Practices in Laboratory Test Ordering and Interpretation

Request for Approval of New Data Collection

Supporting Statement B

July 21, 2011

Contact:

Julie Taylor

Division of Laboratory Science and Standards

Laboratory Science, Policy, & Practice Program Office

Centers for Disease Control and Prevention

1600 Clifton Rd. USPS

MS G25

Atlanta, Georgia 30333

Phone: 404.498.2291

Fax: 404.498.2219

Email: [email protected]

Quantitative Survey of Physician Practices in Laboratory Test Ordering and Interpretation

Request for Approval of New Data Collection

This is a request for OMB approval of a new data collection, Quantitative Survey of Physician Practices in Laboratory Test Ordering and Interpretation. CDC is requesting a twelve month approval to collect the data.

B. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The primary research focus of this program is to understand the utilization of laboratory medicine for diagnostic purposes among primary care physicians. We are not investigating the practices of specialist physicians in this area, since they utilize a narrower range of laboratory test procedures specific to their area of specialization and would be expected to typically possess a deeper awareness of these test characteristics. Our survey population frame is primary care physicians categorized as Family Practice or General Internal Medicine physicians. These physicians encounter a broader range of diagnostic situations and must effectively choose among several thousand extant laboratory tests. The research goal of this proposed survey is develop a better understanding of the challenges and facilitators encountered by primary care physicians in ordering and interpreting laboratory tests in the course of their typical practice environments.

The initial qualitative phase of our research indicated that there may be differences between Family Practice and Internal Medicine physicians in their utilization of laboratory medicine, and our stratification plan is designed specifically to support a comparative analysis of these two groups. There are several other categorization of interest, including region, practice setting (group, hospital, and academic medical center), years in practice, and related demographic and practice characteristics.

Because this is a preliminary investigation, we are not incorporating a further stratification of the sample by these other categories of potential interest. We want to achieve a conventional 80% power in the resultant data, which would require a sample size of roughly 400 per cell. We intend to collect a total of 1600 cases in order to adequately power a comparison among groupings that may not be as evenly proportioned as our basic Family Practice/General Internal Medicine stratifier.

We have arranged to obtain access to the American Medical Association (AMA) Master File, which contains contact information and demographic and practice characteristics for approximately 850,000 active US physicians. Of this total, there are entries for approximately 132,000 Family Practice and 140,000 General Internal Medicine physicians. This list originates with the AMA but is made available for research purposes by Redi-Data, a commercial market research company. We intend to conduct the survey via a Web-based survey system and to invite the participation of sampled physicians through an initial postal mailing followed by one or more emailed invitations and reminder messages. Thus we require both postal and email addresses. Approximately 40% to 50% of Family Practice and General Internal Medicine physicians in the Master File have both postal and email addresses available. The email addresses are added to the file from a variety of external sources by the vendor. It is possible that older physicians or physicians from smaller practices do not have email addresses, but we suspect that the pattern of missing email addresses is random and not a source of bias. The AMA Master File is the largest compilation of all physicians available. Since we are focusing on Family Practice and General Internal Medicine physicians, alternate sources of sample would be lists obtained from their specific professional organizations. However, the AMA, through its commercial partner, is more flexible in working with researchers and providing alternate contact information including alternate email and postal addresses. Moreover, additional data within the record will support a non-response analysis. There is a potential source of bias in sampling only physicians with available email addresses. These physicians may tend to be younger and may be more likely to be in larger practice groups. We could explicitly oversample based on age and practice characteristics, but we believe that any imbalances in a random sample will be slight and can be adjusted as part of applying post-stratification weights. In general, we believe that we can detect and compensate for this source of bias.

The Physician Masterfile includes current and historical data for more than one million residents and physicians and approximately 82,000 students in the United States. This figure includes approximately 353,737 graduates of foreign medical schools who reside in the United States and who have met the educational and credentialing requirements necessary for recognition and approximately 66,000 doctors of osteopathy.

A record is established when individuals enter medical schools accredited by the Liaison Committee on Medical Education (LCME), or in the case of international medical graduates, upon entry into a post-graduate residency training program accredited by the Accreditation Council for Graduate Medical Education (ACGME). IMG's are also identified when they obtain a license from one of the 68 US licensing jurisdictions. As a physician's training and career develop, additional professional certification information is added to their Masterfile record. Physicians' records are subject to change and are continuously updated through the extensive data collection and verification efforts by AMA.

We will do an initial sampling bias analysis by comparing basic demographics of the email versus no-email groups to confirm this. Based on previous experience with physician surveys, we are anticipating a 30% response rate. Thus to generate 800 completed cases for each of our two physician categories, we should require 800/0.30 = 2667 initial sample records, or 5334 in total. However, response rates are difficult to predict in advance, and so we intend to select a total of 10,000 records initially.

The sample generation process will proceed in several steps. First we will select only physicians in active practice and coded as Family Practice or General Internal Medicine. Second, we will select only those records with a valid email address. Some email addresses may appear valid but be out-of-date or otherwise invalid. We intend to ask for confirmation of the email address within our initial postal mailing. In the absence of replies, we will not be able to determine validity until we actually generate an email to the address. Bounced emails will become will become part of non-response. Once we have selected our sample frame, we will draw the actual sample using a randomizing process. Typically, we employ the SAS procedure SURVEYSELECT, which selects records randomly based on total required sample. This procedure has the advantage of calculating the basic sampling weight which measures the probability of selection for each sampled physician while constructing the sample. SURVEYSELECT uses a true randomizing process, so the original order of frame dataset does not influence the constructed sample.

We will organize these into replicates and begin the survey fielding with an initial small release. We will calibrate the response rate based on actual field experience and release additional replicates in order to converge to our targeted completed case count as efficiently as possible. Released sample records that do not result in a completed case will be retained, and a non-response bias analysis will be conducted after the field period to compare the demographic and practice characteristics between the response and non-response groups.

2. Procedures for the Collection of Information

The research group conducted three focus groups each concerning different subjects with nine or fewer participants with Family Practice and General Internal Medicine physicians during 2010. This qualitative research identified the primary themes in current physician practice with regard to laboratory test ordering and test interpretation. The group utilized these findings to create a survey questionnaire that will be administered to the sampled physicians during this survey research phase. The questionnaire is composed of primarily closed-end items. The finalized questionnaire was tested for comprehension and face validity by administering it individually to three physicians. This test was conducted in a cognitive interviewing format where the physicians were asked to comment on each question as they encountered it. We estimate that the typical respondent will require approximately 14 minutes to complete the questionnaire.

The questionnaire will be programmed for delivery using the Illume Web survey system. This is a commercial software system that is fully developed and supports compliance with Section 508 of the American Rehabilitation Act. Illume is used by numerous healthcare research organizations for similar applications. Extensive internal testing of questionnaires and associated field protocols is performed prior to any use in the field.

Each sampled physician will receive a postal mailing on CDC letterhead. The mailing will contain a letter explaining the research purpose of survey, a toll-free number and email address of the researchers to allow the physician to confirm legitimacy, obtain more information or request to be removed from the sample. The letter will be signed by one or more credible officials from the CDC. The letter will also notify the physician that the survey will be conducted via a Web-based survey system and that the physician should receive an emailed invitation containing an embedded link that will take them to the Web site automatically. We will include the email address and request corrections as needed, via email or a toll-free number.

Approximately seven days after the postal mailing, we will email a survey invitation to each of the physicians. The email message will reiterate the information contained in the postal mailing and contain an embedded link. Most email client software allows the user to click on a link within an email message; this invokes a browser and transfers the user to the web site. We will also include a brief instruction to copy and paste the link in the event that the email client does not support embedded links. Each email invitation is personalized to the physician and contains an anonymous and unique identifier.

When the survey field period closes, we will export the collected data for cleaning and analysis. The final dataset will include one record for each originally sampled physician, whether or not a response was recorded. Each record will contain a disposition code indicating a completed case or one of several categories of non-response, such as bad email address, respondent refusal, and so forth. We will retain demographic and practice characteristics from the sample record but all identifying information will be removed before an analysis dataset is released.

3. Methods to Maximize Response Rates and Deal with No response

We will encourage high response through a combination of an initial postal mailing to establish credibility, repeated email reminders to non-respondents, and the inclusion of a modest contribution to a charitable organization of the respondent’s choice.

The Illume system tracks respondents using the unique ID and flags completed cases. Physicians that have not responded after approximately five days will receive a reminder email. We intend to send between two and four reminders over a three week period in order to encourage response. We will cease reminders for physicians who complete the interview or who request to be dropped from the sample. The unique identifier is maintained by the Illume software in a separate database and is not retained within the survey response database, preserving respondent confidentiality.

In order to encourage response, physicians completing the survey will be offered the option of having Altarum Institute, the survey vendor, contribute $10.00 on their behalf to their choice of one of several charitable organizations. The charitable organization choices will all be United States-based 501(c) non-profit organizations. This amount was selected to be sufficiently small as to not represent an undue compensation or possibly coercive influence on respondents.

We will monitor survey response daily to gauge if the current active sample is likely to generate the required number of completed cases. If response is lower than estimated, we will release additional small replicates and repeat the field protocol for subsequent releases. We will maintain web survey access for approximately one week after the final reminder. Minimizing the use of sample will maximize the response rate while limiting the total number of physicians contacted. In recognition of the reality that response rates for surveys of active physicians are typically lower than desirable, and we will perform a non-response bias analysis that will examine the demographic and practice characteristic variations between the response group and the nonresponse group. Our non-response bias analysis will compare the response versus the nonresponse group to assess any systematic bias attributable to non-response.

4. Tests of Procedures of Methods to be Undertaken

The first phase of data analysis will be to complete the survey weights. We will adjust for survey non-response and also include a post-stratification adjustment based on physician demographic and practice characteristics, using data available in the initial sample plus selected survey items such as rural/urban splits. Final weights will be calculated using the SUDAAN survey statistical package. The constructed sample, drawn from the AMA Master File, will be drawn to be representative of the entire Master File, and the frame contains additional physician characteristics.

With the collection of 800 completed cases for each of two physician types, our analysis will be adequately powered to detect differences between these groups. Additionally, we anticipate that we will have sufficient data to analyze differences attributable to such relevant variables as years in practice, gender, practice type, and related demographic and practice characteristics. To conserve resources, we are not explicitly stratifying on these other potential analytical categories because this information is either not available in the sample frame or we are unable to predict differential response rates across these categories. Additionally, this investigation is among the first to explore variations among physicians in laboratory testing practices, and there is as yet insufficient understanding how practices vary by physician characteristics to support any further a priori sample stratification.

Our primary analytical goal is to build on the anecdotal findings from the previously conducted focus groups involving fewer than ten participants. Within the focus groups, we identified a number of factors surrounding test ordering and interpretation, but these findings are inherently non-quantitative and anecdotal. For example, physicians are increasing utilization of electronic decision tools to guide test ordering, but we could not determine from a focus group format the prevalence of this utilization and the diffusion of different technologies across varying practice environments. The proposed survey will enable us to develop quantitative estimates of the themes that emerged from the focus groups. These analyses will guide us to designing effective education, interventions, and related aids to increasing the effectiveness of laboratory medicine in primary care settings, leading to improved patient health outcomes and more cost-effective medical care.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Statistical aspects of the study have been reviewed by the individuals listed below:

Tom Wilkinson

Lead Survey Methodologist

Altarum Institute

3520 Green Court, Suite 300

Ann Arbor, MI 48105

[email protected]

Joe Dorris

Senior Statistician

Altarum Institute

3520 Green Court, Suite 300

Ann Arbor, MI 48105

[email protected]

13