natl consumer sruveys ss PART B 6-29-11

National Consumer Surveys on Understanding the Risks and Benefits of
FDA-regulated Medical Products

SUPPORTING STATEMENT

PART B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Methods

The data collection will consist of parallel surveys of 1,500 noninstitutionalized U.S. adults (3,000 total respondents) that will take no more than 15 minutes to administer. FDA contracted with Research Triangle Institute (RTI) to conduct these surveys.

The proposed sample size will result in a 95% confidence interval of (0.473, 0.526), for a proportion equal to 0.50, assuming a design effect of 1. That is, FDA will have 95% confidence that the corresponding true population value will lie between 0.473 and 0.526 if the sample estimate is 0.50. This calculation accounts only for sampling variance and does not incorporate bias and other sources of error.

For both the telephone and Internet panel surveys, the Contractor will use a sample selection procedure that results in a sample that is similar to the population within the United States with respect to age, gender, geographic distribution, and race/ethnicity. Participants will be recruited according to the screener criteria asked in the first items of the surveys; respondents must (a) have themselves used either over-the-counter (OTC) drugs, prescription drugs, medical devices, or biologics within the past three months, or (b) been the primary caregiver for a family member who has used such medical products within the past three months.

B.2. Procedures for the Collection of Information

The telephone survey will use an Address-Based Sampling (ABS) methodology. ABS improves the telephone survey's reach beyond that of a traditional Random Digit Dial (RDD) methodology by minimizing the risk of systematic exclusion of households from the survey sample. In addition to including households with a landline telephone number on record, ABS will be used to also include households that are cell phone-only. Results from the 2006 National Health Interview Survey suggest that one out of every eight American adults live in cell phone-only households. The percentage of cell phone-only households is highest among younger adults (25% of 18 to 24 year olds, and 29% of 25 to 29 year olds). Results also indicate lifestyle differences. For example, cell phone-only adults are more likely to be current smokers than adults living in households with a landline (30% vs. 19%).¹ Given the known differences in demographics and lifestyle preferences between cell phone-only and landline adults, ABS offers a way to reduce this potential problem of coverage bias. ABS will also include households that do not have a phone number on record.

For the telephone survey, the Contractor will send an initial mailing (Attachment 7) to the entire telephone survey sample (approximately 30,000 households), providing a toll-free telephone number to call to take the survey. The Contractor expects to answer calls from a few hundred potential call-in respondents. The majority of responses will come from records with attached telephone numbers where the Contractor will have trained interviewers call households to conduct the survey. The contractor will use computer-assisted telephone interviewing (CATI) to complete the interviews. The Contractor shall send a second mailing (Attachment 8) to nonresponding addresses not matched to a phone number to further solicit their participation.

The Internet panel will be constructed by filling quotas in a number of key subpopulations defined by various demographic and socioeconomic variables. The goal will be to make the sample resemble the U.S. population with respect to these variables. The online data collection vendor will use a recruitment methodology that invites pre-validated individuals to participate in its Consumer Panels. In order to supply a sample of both the general population and hard-to-reach audiences, the Internet panel members will be recruited from a diverse set of consumer and business-to-business sources using a "by-invitation-only" approach. This means that the online data collection vendor will be able to control and manage the demographic make-up of the panel "up front" and send an invitation email (Attachment 9) to only the types of individuals that fit the current normalization needs of the panel. The online data collection vendor will send one email reminder (Attachment 10) to Internet panel nonrespondents to further solicit their participation. These panel establishment methodologies are fully compliant with CASRO guidelines.

Both the telephone and Internet panel survey samples will be decomposed into random subsamples or replicates and these replicates will be released such that completed interview targets are achieved without unduly affecting the response rate. Once a replicate is released it will be completely worked in the field.

The Contractor will program the Internet panel questionnaire to be a self-administered online survey that is consistent with the telephone survey. Access to the online survey shall be controlled through a process of enrollment and ID/password distribution. The ID/password are built into the landing page URL to enter the survey. The respondent clicks the URL in the survey invitation e-mail to enter the Contractor landing page. From the landing page, the respondent clicks a link to enter the survey. The telephone and Internet panel surveys will include parallel content, with only minor differences to assist telephone respondents in their comprehension of the questions and recall of response options.

For both surveys, professionally recognized procedures will be followed in each information collection activity to ensure high quality data. Examples of these procedures include the following:

• The telephone survey will be monitored both visually and audibly by supervisory staff during all shifts;

• After a small amount of live completed data is gathered (usually no more than 25 completes), early data validation will be conducted and periodic checks will continue throughout the study;

• Data submitted through on-line surveys will be subjected to statistical validation techniques (e.g., disallowing out-of-range values, detecting persons who complete the survey outside the expected time range).

The final data for both samples will be weighted independently to adjust for unequal probabilities of selection, variable nonresponse, and overall poststratification that brings in line the weighted sample distributions with known corresponding population distributions.

B.3. Methods to Maximize Response Rates and Deal with Non-response

The Contractor for the online data collection will recruit approximately 14,000 individuals, Experience with online experimental studies suggests that about 15% of those who are sent survey invitations will complete a study. The Contractor for the telephone survey will send recruitment letters to approximately 30,000 possible participants, receive calls to a toll-free number provided in the survey, and call households to which a telephone number can be matched. It is estimated that 75% to 80% of the persons reached by phone will complete the survey.

Past experience has shown that the proposed amount of over-recruitment generally ensures a respectable response rate. FDA believes that the issue of medical product safety will be of high interest to the public and that they will consequently be very interested in participating in the survey. Nonetheless, RTI will conduct a non-response analysis to determine whether there are significant differences between those who agreed to participate and those who did not.

The Contractor has conducted cognitive interviews with nine members of the public to help improve understandability of the questionnaire, reduce participant burden, and enhance interview administration. The Contractor will also implement several procedures proven effective in previous studies to maximize response rates:

• Potential respondents will be informed about the importance of these studies and encouraged to participate.

• Lead letters will be sent to telephone survey sample members. A second mailing will be sent to nonresponding addresses not matched to a phone number.

• A dedicated toll-free number will be established by the Contractor to allow potential telephone respondents to confirm the study’s legitimacy and take the survey.

• Interview staff will be able to provide respondents with the name and telephone number of an official at FDA. This official will confirm with respondents the importance of their participation.

• Experienced, highly-trained staff will conduct all telephone interviews. Training topics will include study objectives, question-by-question reviews of data collection instruments, strategies for engaging respondents, and techniques for fostering respondent cooperation and survey completion.

• For telephone interviews, up to 20 call-backs will be made to encourage participation. Outgoing calls that result in a disposition of no answer, a busy signal, or an answering machine will be automatically rescheduled for subsequent attempts. Up to 20 outgoing calls to a given number with dispositions of the sort listed will be made before declaring it a non-response.

• Well-defined conversion procedures will be established. If a respondent for a telephone survey declines to be interviewed, a member of the contractor's conversion staff will contact the respondent to explain the importance of their participation. Conversion staff are highly experienced telephone interviewers whose style and persuasive abilities have demonstrated success in eliciting cooperation. They receive a pay differential to acknowledge these skills, which also serves as an incentive to the interviewer pool, whose completion rates are carefully monitored to assess their qualifications to serve as conversion staff.

• The surveys are designed to be completed in a reasonable amount of time (15 minutes) to minimize break-offs.

• Should a respondent interrupt a survey for any reason, such as needing to attend to a personal matter, the respondent will be able to continue at a later time. For the telephone survey, the interviewer will reschedule or have a predictive dialer automatically reschedule the interview for completion at a later time. For the Internet panel survey, the respondent will be allowed to continue by clicking the same link they utilized to open the survey. The study will be available for respondents to continue up until the time the survey is closed or reaches its quota.

• An invitation email will be sent to Internet panel survey sample members. Nonrespondents will receive one e-mail reminder from the online data collection vendor requesting their participation in the survey.

• Fielding of the surveys will occur over at least a 1 to 2-week period for the on-line version of the survey and data collection for the telephone survey will take place over a 5 to 7 week period to allow sufficient time for the lead letters to be delivered by the U.S. Postal System and increase the waiting times in between call attempts, which typically increases the response rate (i.e., the up to 20 call attempts are spread out over time). Based on past experience, this time frame will allow the contractor to reach individuals who are on vacation, out of the home during irregular periods, have a temporarily disconnected telephone, or who are not answering the phone for some other reason.

B.4. Test of Procedures or Methods to be Undertaken

The Contractor has conducted cognitive interviews with a total of nine members of the public to get feedback on the survey content. The face-to-face interviews were conducted with adults (18 years or older) that differed in age, gender, and education. This feedback from the public was used by FDA to evaluate and refine the draft questionnaire, helping to ensure survey questions and response options are understandable and appropriate.

The Contractor will also pretest the telephone and Web survey administration methods prior to implementing the main study. Pretesting will help evaluate the effectiveness of the programming of the interview protocol, online filters and skip patterns. Thirty pretest participants will be randomly drawn from the sample populations (15 randomly dialed and 15 from the Internet panel).

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Karol Krotki (RTI) will be responsible for the design of statistical and sampling procedures undertaken as part of these data collection activities. Brian Lappin (FDA) and Michael Burke (RTI) will be responsible for the tabulation and analysis of data collected.

1“Wireless Substitution: Early Release of Estimates Based on the National Health Interview Survey, July – December 2006. (PDF)” Blumberg, Stephen J., and Julian V. Luke. May 14, 2007a. Report by the U.S. Centers for Disease Control and Prevention.