[Survey of Health Care Practitioners for Device Labeling Format and Content
0910-NEW
SUPPORTING STATEMENT PART B
B. Statistical Methods
Respondent Universe and Sampling Methods
As previously mentioned, the first phase of the study under OMB #0910-0497, used focus groups to gather information about current device labeling, including how it is currently used and how it is understood by the HCPs. The second phase of the project will obtain HCPs’ feedback on new formats for medical device labeling, developed with providers’ feedback gathered through the Phase I focus groups. Three formats will be the basis of the survey (Attachments E, F, and G; see Attachment D for a version explanation table). However, each HCP will only see and provide feedback on just one particular format. The FDA wants to learn which format is most useful, easy to understand, and readable. In order to obtain more thorough data related to the three formats, RTI will conduct interviews with approximately 600 HCPs. These interviews will assess what the respondents like and dislike about the formats, gather recommendations for suggested revisions, and collect information as to how FDA can make device labeling more useful.
Due to the fact that response rates for web surveys among physicians are lower than those found in a typical web panel of the general public, a total of 8000 records will be purchased from Direct Medical Data. This ensures a sufficient amount of records will be available to us, should we encounter response rates as low as 8-10%. However, we intend to take a conservative, multi-wave approach, contacting only 1500 sample members during the first wave. FDA’s contractor RTI will gauge the response rate for this population based on the first release of sample. If a minimum of 40% response rate is achieved in this first wave, no further release of sample records will be required to meet our goal of 600 completed cases. Garnering anything less than 40% will result in the release of a second wave of sample records, congruent with the expected response rate as based upon the first wave of data collection efforts. We do not expect to have to utilize all 8000 records, but thought this to be the most sound approach due to the high variability in the literature regarding response rates among physicians, paired with the shortage of literature specifically detailing expected response rates for physicians on web surveys, and taking into consideration that the incremental cost of more contact information is not as great as the incremental costs of additional purchases (i.e., it would cost more to purchase the same total amount in two smaller purchases than one large purchase).
Our goal is to complete 20% of the interviews with physicians or other medical prescribers (e.g., nurse practitioners), 40% with nurses, and 40% with therapists and technicians, our sample will be composed of the same percentages. A total of 1800 physicians and prescribers, 3100 nurses, and 3100 therapists and technicians will be included in this file. RTI will randomly select, in proportionate percentages from each group, the records that will be included in the first release of 1500. Should a second wave be required, RTI will again randomly select the appropriate number of records from the remaining sample members.
The survey should take approximately 30 minutes to complete. The Contractor will provide the survey on-line. After an initial lead letter with the survey URL enclosed, RTI will send a follow-up letter approximately two weeks later. Approximately two weeks after that time, trained callers will make up to seven reminder calls offering to provide the survey website and login information as needed.
In addition to the survey, the Contractor will conduct up to 24 in-depth, face-to-face interviews using the same basic questions as the survey, but probing, as appropriate, to find out why the participant answered in a given way and how, if at all, they would change the medical device format to make it more useful. The interview will last no longer than one hour, including the time to provide consent. To obtain this sample, a small number of the overall sample purchased who reside in the Raleigh/Durham/Chapel Hill region of North Carolina, as well as the greater Washington, DC, metro area will be reserved for conducting the required 12-24 face-to-face interviews. These sample members will be contacted separately by RTI staff trained in in-person and cognitive interviewing, and will not be included in the mail and telephone outreach efforts for completing the web survey.
Respondent Universe:
The universe of respondents includes physicians, physician assistants, nurse practitioners and therapists involved in wound care, respiratory therapy, and infusion therapy. Physicians will be drawn from the AMA Masterfile, or a comparable list of currently practicing physicians. The other HCPs will be drawn from lists of professional associations’ commercially available lists of HCPs, or other means to assure a diversity of sample from which to select participants.
It is expected that the Contractor will perform several activities intended to ensure the final data set contains a statistically valid number of respondents to be able to infer the survey result to the sample universe, e.g., 600 respondents. These procedures are described below. A balance of approximately 20% of the completed surveys coming from physicians and other prescribers, and the balance evenly distributed among nurses and nurse practitioners (40%) and other allied health care technicians (40%). The in-depth interviews will consist of a similar group of participants. Access to the online survey shall be controlled through a process of enrollment and ID/password distribution. The survey will contain primarily closed-ended questions. Survey respondents shall be health care professionals who work in place such as the hospital, clinic, rehabilitation facility, or the home. Respondent experience and area of medical expertise will be documented and their type of degree(s) such as physician, nurse, or therapist.
The sample size proposed should provide sufficient power to detect significant differences between the three device label formats to be tested and the types of participants to be recruited. More specifically, given the proposed sample size and distribution
there is an 80% probability that a difference of approximately 0.4 points in the mean score between any two of the three device label formats would be statistically significant.
there is an 80% probability that a difference of approximately 0.6 points in the mean score between technicians and nurses for a particular question would be statistically significant.
there is an 80% probability that a difference of approximately 0.7 points in the mean score between physicians and other providers for a particular question would be statistically significant.
Procedures for the Collection of Information
Participation in this data collection is voluntary and the respondents will remain private to the fullest extent allowed by law. Respondents will be recruited through up to 12 rounds of mail and telephone contacting attempts. Responses to the survey or through the in-depth interviews cannot be linked to the respondent list. During the data collection, each respondent will view one labeling format and answer questions about the format, the information contained in it, the organization and presentation of material, and about themselves (e.g. number of years in practice, type of licensure, etc.). They will respond to the order of the information, determine if it is easy or difficult to use, determine if diagrams of the device are needed, and if they would use the labeling as designed. They should recommend if they find the labeling format usable, understandable, and readable.
All survey results will be collated and analyzed by RTI and the respondents will be de-identified by RTI. The results from the in-depth interviews will be analyzed separately. The results of these two data collection activities will provide vital information to the FDA experts who convened to determine medical device labeling needs. The survey data collection field period will be approximately 16weeks
Methods to Maximize Response Rates and Deal with Non-response
Given the target population and the general downward trend of response rates in current survey research, it is difficult to predict a realistic response rate. We will strive to maximize the response rate but realistically, given current trends in general population survey response rates, we will be doing very well if we reach 50%. In many cases, a 20% or lower response rate is good for a well designed and executed survey. Unfortunately, there is not an abundance of literature available regarding response rates for conducting web surveys with physicians and other members of the healthcare community. However, response rates appear to be varied, and rates of less than 20% are not uncommon (Golnik et al 2009; Rodriguez et al 2006; Yusuf & Baron 2006). Response rates of 50% or greater are rare, and tend to be more common among physicians who are known to be internet savvy (Potts &Wyatt 2002). A recent study (Dykerna, et al 2011) conducted an 80-item web survey with physicians and experimented with several incentives. The highest incentive ($100) resulted in a response rate of 25.4%. An online survey of prescribers in Australia (Aitken et al 2008) resulted in extremely low response rates (9% overall). A multi-mode approach tested recently among Alabama physicians (Nicholls et al 2011) found that, when given the option of mode, only 2% of surveys were completed online, while 88% were completed by mail, and 10% by fax. In one other multi-mode study (Scott et al 2011); the online response rate was 13%, lower than the simultaneous mixed mode (20%) or the sequential mixed mode (21%).
A literature review of survey research among physicians and healthcare professionals published several years ago (Flanigan et al 2008) indicated that many studies in the medical community have response rates ranging on average from 30-60%; however, all but one of the studies included in the review focused on telephone and mail administered studies. Since healthcare professionals are a difficult population to engage, and due to the lower response rates seen with web surveys, the data collection contractor, RTI, feels that the combination of our sampling approach along with the proposed incentives will help us complete data collection in a timely, cost-effective manner. Moreover, following the lead of many others in the field, we have expanded our focus to: 1. maximizing the response rate and, 2. studying nonresponse bias. For maximizing the response rate, we will use two different means of contacting the participants (2 US Mail and 7 telephone calls) plus an incentive to convince sampled members to participate. Repeated attempts will be made to try to turn around refusals.
FDA recognizes that there may be different characteristics between respondents and non-respondents and other important factors not included in the survey (e.g., hospital policy related to use of devices which is not included in the survey as it is outside of the current scope which examines labeling content and format preferences, not hospital policy). We do not know of any large scale surveys specifically looking at medical device labeling or healthcare provider use of labeling, nor differences in who might or might not use labeling, save that providers use labeling less than non-physician groups. As such, FDA does not have a theoretical reason to believe that respondents would substantively differ from non-respondents in terms of medical device labeling and has chosen not to include extensive efforts to assess nonresponse bias in this research.
The goal of our nonresponse analyses will be to identify dimensions along which the non-respondents differ from respondents. This will include, where available, comparisons between the respondents and a number of variables that are available on the sample frame and that can be used to identify major sources of non-response bias. Weighting adjustment will be implemented to compensate for these known sources of bias. Furthermore, in order to get a better handle on nonresponse bias, we will use respondent data to study the extent to which variables available for non-respondents are correlated with other variables of interest.
Test of Procedures or Methods to be Undertaken
Pretests were done in August with 7 CDRH health care professionals providing input on the survey and templates. The survey was minimally revised based on this feedback. The templates were not revised because the HCPs did not have feedback on them.
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The contractor, RTI, provided the statistical expertise to the development of the survey and template. RTI will also collect and analyze the data. RTI International, the Contractor for the data collection, will collaborate with Stephen Woloshin, MD, of Dartmouth University, Lisa Schwartz, MD, also of Dartmouth University, and Maureen Garner, MS, Founder and President of New World Regulatory Solutions, Inc. Drs. Woloshin and Schwartz will contribute their knowledge related to issues surrounding drug labeling. Ms. Garner’s experience with the medical device industry and issues related to the development and clearance of device labels brings greater depth to the project related to the concerns of device manufacturers
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Gittleson, Daniel |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |