B

Supporting Statement B for

National Physician Survey of Practices on

Diet, Physical Activity, and Weight Control

Ashley Wilder Smith, Ph.D., M.P.H.,

Task Order Monitor

National Institutes of Health

National Cancer Institute

Division of Cancer Control and Population Sciences

Applied Research Program

Outcomes Research Branch
6130 Executive Boulevard
Bethesda, MD 20892-7344
Phone: 301-451-1843
Fax: 301-435-3710
E-mail: [email protected]

B. Collections of Information Employing Statistical Methods

B.1. Respondent Universe and Sampling Methods

The population of interest for this study is the set of physicians in four physician specialties who are currently practicing and office-based (i.e., seeing patients for at least 20 hours a week): Family Practitioners (FP), Internal Medicine (IM), Obstetrician/Gynecologists (OB/GYN) along with gynecologists (GYN), and Pediatricians (Ped). Gynecologists represent a much smaller group than OB/GYNs and will be pooled with OB/GYNs for analysis purposes. The two groups will be referred to as OB/GYNs here. Family Practitioners will be considered from two perspectives for the analyses: all FPs who meet the eligibility criteria and the subpopulation of eligible FPs who see both children and adults. A call will be made to identify FPs who report that they see only adult patients. Osteopaths will not be included in this study.

The contractor for the Energy Balance Survey is currently carrying out the Cancer Screening Study (OMB No. 0925-0562) for NCI, involving all the specialties of interest to the Energy Balance Survey except pediatricians. However, the survey is not yet completed, so the development of population estimates based on the study data has not yet been undertaken. As a result population distribution data from two surveys previously completed are displayed in Table B.1 – 1. The table provides information on the expected population distribution as well as response rates, from NCI’s 1999-2000 National Survey of Colorectal Cancer Screening Practices study (OMB No. 0925-0468) for the three specialties associated with the adult questionnaire for the Energy Balance Study. The sample frame for the study was the AMA master file.

Table B.1 - 1 Sample and Population Distributions and Response Rates from the 1999-2000 National Survey of Colorectal Cancer Screening Practices

Physician Type	Number of Eligible Physicians Responding	Reported Response Rate Among Eligibles1	Estimated Number of Known Eligibles Sampled	Estimated Population Distribution (Percentage)
Family Practitioner	423	68	622.1	38.2%
Internal Medicine	488	69	707.2	43.4%
Obstetrics/Gynecology	224	75	298.7	18.3%
TOTAL (Adult)	1,235	72	1,628.00	100.0%

Table B.1 - 2 provides similar information from the 2003 NAMCS (OMB No. 0920-0234) which was conducted by the National Center for Health Statistics (NCHS), again with the AMA master file as the sample frame. Note that FPs and GPs are combined for the NAMCS surveys. The NAMCS estimates can be expected to have more variability due to the NAMCS design and smaller sample sizes. Nevertheless, the estimated population distribution across the three NAMCS strata associated with the adult questionnaires is close to that estimated from the Colorectal Cancer Screening Practices data, the design of which resembles the design developed for the Energy Balance study. Pediatricians are also included in the table. The estimated number of family practitioners who see children as well as adults, of interest in assessing the sample distribution of family practitioners across the two questionnaire types, is not known.

Table B.1 - 2 Sample and Population Distributions and Response Rates from the 2003 NAMCS

Physician Type	Number in Frame	Number Sampled	Number In Scope	Eligibility Rate	Number In Scope Who Responded	In Scope Response Rate	NCHS Population Estimate	Population Percentages for those receiving the adult questionnaires
Family Practitioner/ General Practitioner	71,559	317	207	65.3	145	70.0	56,287	44.0
Internal Medicine	73,480	171	112	65.5	71	63.4	46,847	36.6
OB/GYN	33,174	157	113	72.0	113	56.6	24,772	19.4
TOTAL (Adult)	178,213	645	432		329	---	127,906	100.0
Pediatrics	49,465	197	110	55.8	87	79.1	28,100	---

In developing the sample design for the Energy Balance study, we have focused on three analytic objectives:

1. Establishing estimates for the three specialties focused on adults (FPs, IMs, and OB/GYNs) as well as the three considered as a whole

2. Establishing estimates for the two specialties focused on children (FPs who see children and pediatricians) as well as the two considered as a whole; and

3. Making comparisons between the different specialty types.

Three distinct questionnaires will be fielded: one to be completed by physicians and focused on adult patients; one to be completed by physicians and focused on child patients; and one that can be completed by office staff instead of the physician, to help reduce burden. The total targeted number of responding physicians for the study is 2,000 with a completed “office administrator” questionnaire targeted for each completed responding physician. The sample of physicians asked about adult patients is to be about 1,200 while the sample size for the sample of physicians asked about child patients is to be about 800. Doctors in the OB/GYN stratum will be oversampled. This will ensure greater power to detect existing differences between OB/GYNs, FPs, and internists for the questionnaire covering adult patients as well as increase the precision of estimates for OB/GYNs. (See Section B.2 below for a discussion of expected power and precision levels). A design effect will also be incurred for FPs receiving the adult questionnaire. A discussion of this is found in the Sample Sizes portion of Section B.2.

We expect a response rate of at least 70 percent, consistent with the contractor’s previous experience with the CSS, where the out-of-date contact information on the AMA file served to limit the size of the response rates despite extensive tracing efforts.

B.2. Procedures for the Collection of Information

Target Population and Sample Frame. As indicated earlier, the full target population for this study consists of non-federal physicians who are FPs, IMs, OB/GYNs, and Peds with patient care as their major activity.

The AMA master file will be used to construct the sample frame. Current data suggest that the AMA file provides the most complete coverage available of physicians of all available sample frames, and it has been used in numerous studies, including the NAMCS, the 1999-2000 National Survey of Colorectal Cancer Screening Practices (OMB No. 0925-0468), and the 2006-2007 Cancer Screening Study or CSS (OMB No. 0925-0562).

In constructing the sample frame from the AMA file, we will exclude physicians with addresses outside the continental U.S. as well as physicians who do not meet survey eligibility requirements based on data appearing on the listings (not in active practice; deceased; not among the targeted physician specialties; federal; classified as research, administration, or teaching as the primary professional activity).

Sampling Procedures. Once the frame is formed, we will select systematic samples within strata represented by the four specialties. Within each sample stratum, we will sort by variables that will help achieve an implicit stratification prior to sample selection. Candidates for sort variables include region of country, metropolitan statistical area (MSA) status, and age group and sex of physician. We will select a systematic sample of the total number of physicians required for fielding for each specialty. For FPs this will entail sampling enough to cover both questionnaire types assuming all FPs see both adult and child patients. Then, after confirmation calls to this sample to determine which FPs see only adult patients, a subsample will be selected from among those who see both children and adults with a targeted sample size to obtain an expected 400 participating FPs who see children. The remainder will be sent the questionnaire focused on adults. If all, or virtually all, FPs see both children and adults, no meaningful design effect will be incurred. Otherwise, a design effect will result for physicians answering the adult questionnaire associated with unequal sampling rates for the two types of FPs. If the proportion of FPs seeing children as well as adults is 90 percent, the corresponding estimated design effect for FPs completing the adult questionnaire is 1.06; if the percentage is 75 percent, the corresponding design effect is estimated to be 1.25. We anticipate that the design effect will be closer to 1.06 than 1.25, but we will not be able to obtain an estimated design effect until the confirmation calls are completed and we can estimate the percentage of FPs who sees both children and adults.

The sample sizes selected will take into account expected levels of nonparticipation and ineligibility, so that the number of survey participants will be sufficient to accomplish the analytic objectives. A reserve sample will be selected to be used in the event that some of the assumptions employed in determining initial sample sizes depart from what was expected.

When this Study enters the field, the CSS, involving three of the four specialties involved in the Energy Balance Study, will have just recently been completed. The names of the sampled physicians for CSS will be removed from sampling consideration for Energy Balance. This will help eliminate the possibility of burden being imposed on physicians who may have been selected for both. This will thus eliminate a potential source of nonresponse. It is possible to do this in a straightforward fashion because we have the IDs from the AMA file associated with all CSS sampled physicians. Since the CSS sample was selected with equal probability within specialty and the two samples will have been selected within roughly a year of each other, there is no reason to expect that this approach would raise concerns of bias.

Power Analysis. There are a number of analytic objectives for this survey effort. A margin of error of plus or minus three percent at the 95 percent confidence interval has been targeted for national estimates based on pooling all completed adult questionnaires from the three specialties. The planned sample allocation of 1,200 completed questionnaires across the three specialties is expected to permit this, though there is some degree of oversampling of OB/GYNs and some design effect is expected to be incurred to the degree to which some family practitioners see only adults.

Tables B.2 – 1a and B.21b show the equal sample allocation planned for this study (for adult and child questionnaires separately), compared with the expected sample distribution with a proportional allocation. They also show the estimated power to detect a difference of 10 percent (comparing estimates of 60 percent to 50 percent) with an alpha level of .05, the estimated effective sample sizes, and the corresponding expected half-widths of a 95 percent confidence interval for a full sample estimate of 50 percent. For the adult questionnaires the expected power was computed between FPs and each of the other two physician types of interest for illustrative purposes.

Table B.2 – 1a Estimated Power and Precision Associated with Proportional and Study (Equal) Allocations: Questions Related to Adult Patients

Allocation Strategy	FP	IM	OB/ GYN	Expected Power to detect the specified difference between:		Estimated effective sample size with projected 1,200 respondents	Expected half-width of confidence interval for full sample estimate of 50 percent
				FP and OB/ GYN	FP and Internist
Proportional	444	538	219	0.776	0.925	1,200	2.83%
Study	400	400	400	0.887	0.886	1,079	2.98%

Table B.2 – 1b Estimated Power and Precision Associated with Proportional and Study Allocations: Questions Related to Child Patients

Allocation Strategy	FPs	Pediatricians	Expected Power to detect the specified difference between the two:	Estimated effective sample size with projected 800 respondents	Expected half-width of confidence interval for full sample estimate of 50 percent
Proportional	478	322	.875	800	3.46%
Study	400	400	.886	771	3.53%

Sample Size. Based on previous experience with collecting data from physicians, we believe that we will obtain a response rate of at least 70 percent. Since the eligibility criteria for this study are similar to those used for the CSS (OMB No. 0925-0562), we expect that the eligibility rates will be around 75 to 80 percent, depending on the specialty. If we assume a 77 percent eligibility rate for illustration purposes, we would sample a little over 3,700 physicians in all for fielding purposes (2000/(.77 x .7) = 3,711). We will use the final eligibility rates from the CSS to guide the final sample size determinations. Again, if we assume 77 percent eligible, Table B.2-2 shows the targeted sample yields, based on precision and power considerations, and the corresponding sample sizes required to achieve them. A reserve sample of physicians will also be selected to permit the supplementation of the sample, should assumptions about eligibility or response rates depart substantially from initial expectations.

Table B.2 - 2 Targeted Sample Yields and Corresponding Sample Sizes Need to Achieve Them

	Family Practitioners	Pediatrics	Internists	OB/GYNs
Targeted Sample Yield	800	400	400	400
Number to be Sampled	1,484	782	782	782

Weights. Sample weights will be developed for use in the analyses. An initial or base weight will be assigned to each sample physician, reflecting his or her probability of selection. Base weights will vary by physician type, due to the sample allocation employed to oversample OB/GYNs for the questionnaire associated with adults and pediatricians for the questionnaire associated with children as well as to meet the various analytic objectives of the study. After the base weight assignment, weights will be adjusted for nonresponse using information from the AMA file available for both respondents and nonrespondents. These will include variables such age, sex, and geographic location as well as any relevant practice related data. An evaluation of these variables will be undertaken to identify those that appear most effective in characterizing the propensity to respond. Cells will be formed from the variables so identified, and the weights of participating physicians associated with the cell will be adjusted to compensate for those in the same cell who do not participate. Since there are no independent counts of the population of physicians who are survey eligible, no poststratification of the nonresponse adjusted weights is planned.

For purposes of variance estimation and analyses, either replicate weights will be created using a jackknife replication methodology or “stratum” and “PSU” variables will be created for use with a Taylor Series approach to variance estimation.

Survey Procedures. The contractor plans to draw a sample of roughly 3,700 to 3,800 physicians from the frame as well as a reserve sample. To help minimize the proportion of ineligible physicians who are mailed and respond to the survey, confirmation calls will be made to the physicians’ offices prior to mailout. The telephone call will verify the information from the survey frame indicating that the potential respondent is within the scope of the survey, as well as the mailing address. For those physicians who have moved, a new address will be obtained from the old office or through tracing procedures using a variety of Internet sources using an established tracing protocol.

The confirmation call will also provide estimates of the number of physicians eligible for the study across the various physician specialties. Based on the results of this effort, the need to supplement the sample to achieve targeted sample yields can be evaluated. If necessary, supplemental samples can be selected from the reserve sample of physicians for each physician type sample that appears to require supplementation.

Each sampled physician will be sent a package containing the survey instrument, a cover letter from the NCI describing the study and the importance of their participation, a document describing the study in more detail, a letter of support from a specialty organization, a $30 honorarium check, and a postage-paid return envelope. The package will be sent by Federal Express.

A second physician survey package will be sent by overnight Federal Express 14 days after the initial mailing to those who have not responded. The second package will contain a more urgently worded cover letter from NCI, a questionnaire, and a prepaid return envelope. Follow up calling for this group will begin the day the second package arrives in the physician’s office. At this time, physicians will also be offered the option of completing the survey over the phone. An additional survey and a replacement honorarium check will be sent if necessary. An additional follow up call will be made to physicians who have agreed to fill out the survey, but from whom no survey has been received by two weeks after the physician has agreed.

A third physician survey package will be sent by Federal Express to those physicians who say they have lost the second package and to physicians who have not responded approximately 10 weeks after the initial mailing. This package will contain a cover letter from the contractor, urging the nonrespondents to participate, along with a survey. Follow up calls will be made after the third mailing only if the overall response rate is below 70 percent. Experienced refusal conversion interviewers will attempt to obtain data on the most critical questionnaire items over the telephone as needed.

The name of the administrator in each office is obtained from the physician survey. About one month after the first physician survey mailing, the administrator survey will be fielded to all offices returning the physician survey. Offices returning a physician survey after this date will be sent an administrator survey on a flow basis, within a few days of receipt of the physician survey. Calls to offices for the names of administrators may be necessary if the physician does not report the administrator’s name, although pretests indicated that responding physicians willingly provided this information.

Mailings to administrators will follow procedures similar to the mailing procedures for the physician survey. The subsample will be sent a survey package via FedEx containing the administrator survey, a cover letter from NCI, background information about the study, a return envelope, and an honorarium of $30. Followup procedures for the administrator survey will be parallel to those of the physician survey, with a second letter and survey sent 14 days after the first one. In the days immediately following the receipt of the second mailing, follow up calls will be conducted. Additional surveys will be sent as needed, and the administrator will be given the option of completing the survey by telephone.

If an administrator survey has not been received 10 weeks after the first mailing, a final mailing will be sent with a cover letter, urging the nonrespondents to participate, along with a survey. Follow up calls will be made after the third mailing only if the overall response rate is below 70 percent. Experienced refusal conversion interviewers will attempt to obtain data on the most critical questionnaire items over the telephone as needed.

Attachment 6 contains all supporting documentation for the procedures described in this section, including the confirmation call script, transmittal and reminder letters, background information for the study, and follow up telephone call scripts.

B.3. Methods to Maximize Response Rates and Deal with Nonresponse

Several procedures will be implemented to maximize the response rate. Physician mail survey response rates are more robust when the research topic is salient to their practice, when the questionnaire has been well designed for maximum ease of administration, and when the data collection protocol is tailored through a variety of incentives and accommodations to acknowledge physician cooperation and contribution. The presentation of the survey is also important to differentiate it from the multitude of research studies for which physicians are targeted. For this reason, the survey will be carefully designed with a graphically simple but pleasing layout, and sent via Federal Express, which has proven to be a more effective mechanism for gaining the attention of physicians than the U.S. mail.

The initial telephone call helps to improve the response rate by identifying physicians who are not locatable before the surveys are mailed, because the tracing procedures instituted before the mailing of the survey ensure there is plenty of time to find the missing physicians. Because respondents that are not locatable are counted as nonresponders, this initial call and comprehensive tracing procedure helps to minimize nonresponse rates. Ineligible physicians, including those that are no longer in practice will be replaced by physicians in the reserve sample, maintaining the sample size.

The introductory letter accompanying the questionnaire will indicate that it is sponsored by NCI, recognized to be a premier public health researcher in the U.S. The letter will succinctly inform the reader of the importance of the survey, as well as procedures for maintaining the confidentiality of respondents (i.e., identities of individual physicians will not be released, identifying information will be stored separately from the survey responses, and all information collected will be analyzed in the aggregate).

In addition to the one-page letter, a two page document describing the study in greater detail will be enclosed. Since many physicians are reluctant to read a letter greater than one page in length, the initial letter will not exceed one page. However, some potential respondents with questions about the study will want more detailed information, which will be included in this second document.

The mailing will also include a letter of recommendation from the professional specialty society to which the respondent is likely to belong. A letter from the American Academy of Family Physicians will be sent to FPs, one from the Society of General Internal Medicine will be sent to general internists, one from American College of Obstetricians and Gynecologists will be sent to OB/GYNs, and one from the American Academy of Pediatrics will be sent to Pediatricians. It is expected that these letters will emphasize the importance of the study and the need for physicians’ cooperation in completing the survey.

A monetary incentive or honorarium that sufficiently acknowledges the time and cooperation of the physician is increasingly common and expected. An incentive of $30 will be included in the survey mailing. A full discussion of how the incentive amount was determined may be found in Section A.9.

The strategy for telephone follow up has been carefully designed and will be staffed with callers with experience contacting physicians’ offices. The callers will be trained and supervised by an individual with extensive expertise in interfacing with physicians’ “gate-keepers,” whose job it is to minimize contact with physicians in the office. Callers will work with receptionists to verify that the package has been given to the physician, and that the physician is aware of the package. By placing follow up calls on the day that the second mailing is received and the days immediately following, the package is highly likely to be near-at-hand when the physician receives a reminder message. By following up consistently and persistently, NCI will demonstrate that it is committing time and energy to obtain the most valid data possible by obtaining the opinions of as many physicians as possible.

Survey staff will work with the physician to obtain data in whatever manner is convenient to the physician. Additional copies of the survey package will be sent to the physician if so required, and the physician will have the option of calling The contractor to respond to the survey by telephone.

Consistent with the response rate calculations approved by the American Association for Public Opinion Research (AAPOR), response rates for this study will be calculated as follows:

Number of Completed Surveys

Number of Completed Surveys + Number of Nonrespondents

Note that sampled physicians who, based on the responses provided, can be definitively characterized as “ineligible” for the study will be included as a “completed survey”. Sampled physicians, who do not provide a completed survey, including those classified as “unlocateable”, will be included among the nonrespondents.

The contractor’s method for survey data collection from physicians has been consistently successful in obtaining response rates of over 70 percent.² NCI expects to obtain a high response rate because of the initial contact by telephone to present the survey, the flexibility in the methods of response, and the extensive follow up calls to interim nonresponders, accompanied by remailing of the survey. The contractor will use refusal avoidance methods during all communications to lessen the need for refusal conversion. For physicians who do refuse, an experienced refusal conversion interviewer will attempt to collect responses on questions deemed most critical.

B.4. Tests of Procedures or Methods to be Undertaken

Each survey instrument was tested with cognitive interviews with nine physicians. In response to their comments, questions were revised, dropped or combined, response categories added to several items, and several small wording changes made.

Tests of confirmation interviews were conducted with nine physicians to confirm that the confirmation procedures would yield information about specialty for each physician respondent, and to refine these procedures.

A pre-test was conducted with nine physicians for each survey instrument. Care was taken to ensure that solo, small, medium, and large physician practices were included in the pre-test. Of the 18 instruments sent, 6 surveys were received within the initial two-week window, and an administrator survey was sent to these offices. A second mailing and reminder phone calls were sent to the remaining physicians, with four additional surveys received after this effort. Minor revisions to procedures were made based on pretest experience.

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Individuals who are providing statistical consultation on this project include:

Steve Clauser, PhD Chief, Outcomes Research Branch National Cancer Institute (301) 451-4402 William W. Davis, Survey Statistician Statistical Research and Applications Branch National Cancer Institute (301) 594-3582 Ralph DiGaetano, M.A., Senior Statistician Westat (301) 294-2062 Mary Horlick, MD, Director Pediatric Clinical Obesity Program National Institute of Diabetes and Digestive and Kidney Diseases (301) 594-4726 Terry T-K Huang, PhD, MPH, Behavioral Scientist Endocrinology, Nutrition and Growth Branch National Institute of Child Health and Human Development (301) 594-1846

Carrie Klabunde, PhD, Epidemiologist Health Services and Economics Branch National Cancer Institute (301) 402-4366 Caroline McLeod, Ph.D., Senior Study Director Westat (240) 453-2786 Ashley Wilder Smith, PhD, MPH Behavioral Scientist, Outcomes Research Branch National Cancer Institute (301) 451-1843 Gordon Willis, PhD Cognitive Psychologist, Applied Research Program National Cancer Institute (301) 594-6652

1 Note that the response rates presented here were based only on those identified as eligible during the survey. Including the identification of ineligibles as part of the response computation to estimate the number of ineligibles among the “nonrespondents with unknown eligibility”, consistent with the guidelines provided by the American Association of Public Opinion Researchers (AAPOR) would be expected to increase the response rates somewhat, making them consistent with the 72 percent response rate reported in publications.

2 Evaluation of the Buprenorphine Waiver Program: Addiction Physician Survey, OMB No. 0930-0246

Evaluation of the Buprenorphine Waiver Program: Waivered Physician Survey, OMB No. 0930-0262

Cancer Screening Survey OMB No. 0925-0562