TESTING COMMUNICATIONS ON NUTRITION AND FOOD PRODUCT SAFETY
SUPPORTING STATEMENT
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Respondent Universe and Sampling Methods
Testing includes various methods and approaches. The method(s) chosen for use depend on the nature of the message or materials tested, as well as their intended audience. Recommended methodologies and sample sizes are based on a review of the relevant literature, consultation with experts in the field, and previous studies, regardless of source.
In general, testing relies on qualitative methods and is not intended to yield projectable results. However, communication messages will be designed and marketed with specific audiences in mind. In qualitative studies, quota sampling is often used to select a convenience sample of individuals who meet certain qualifications that reflect characteristics typical of the target audience. Response rate is not applicable to quota sampling because this type of sampling results in a nonprobability sample which is not representative of the population. In qualitative studies, respondents are usually initially contacted by telephone or through the mail; over-recruiting is done to compensate for not following up with non-respondents.
Where quantitative methods are used, information collection activities will target the particular audiences with statistical sampling procedures employed to identify potential respondents. Mail, telephone and Internet surveys typically will seek a convenience sample that nonetheless has a reasonable diversity in key demographic characteristics such as age, gender, education, and race/ethnicity. Telephone samples may be selected with random digit dialing (RDD) techniques, lists, or with stratified sampling of telephone exchanges. For these samples, each sampling unit (e.g., telephone household or respondent within a household) has a known non-zero probability of selection.
2. Procedures for the Collection of Information
Questions in all testing methodologies include the following:
Standard measures of communications that are designed to assess to what degree the message was successful in communicating information. These questions include measures of main idea recall, comprehension, believability, personal relevance, and likes and dislikes.
Questions tailored for the communication message to address any special concerns the producer of the message may have (e.g., the effect and /or appropriateness of graphic depictions of negative health outcomes).
The methodologies planned for use in this submission will follow standard state-of-the-art approaches adapted from marketing and communications research. In this context, the term “testing” refers to testing messages, strategies, and communication materials, and should not be confused with "pretesting of questionnaires" prior to their full-scale use. The following methodologies will be used:
QUALITATIVE INTERVIEW METHODS
Individual In-depth Interviews. Individual in-depth interviews are surveys that use interview protocols designed to guide the interview without limiting the respondent’s ability to offer attitudes or beliefs on the topic. Most frequently, this method is used to determine mental models (i.e., belief structures) or for testing message concepts, drafting materials, and communication strategies. Individual in-depth interviews can either be conducted on-line at a designated Internet location, conducted in-person, or conducted over the telephone. In some cases, respondents can be sent material in advance, asked to read them, and told that someone will call to get their opinion.
The interviews are conducted by skilled interviewers who follow a prescribed interview protocol. Normally, in-depth interviews are limited to 30 or fewer respondents. In-depth interviews are generally 30 to 45 minutes in length.
Focus Groups. Focus groups are usually composed of 8 - 10 people who have characteristics similar to the target audience or to subgroups of the target audience. The groups are conducted by a professional moderator who keeps the session on track while allowing respondents to talk openly and spontaneously. The moderator uses a loosely structured discussion outline, which allows him/her to change direction as the discussion unfolds and new topics emerge.
QUANTITATIVE INTERVIEW METHODS
Self-Administered Surveys. Self-administered surveys either can be mailed to respondents, accessed on-line at a designated Internet location, or distributed to respondents gathered at a central location. As many as 2,400 surveys may be collected. No interviewer oversees data collection, and the survey is designed so that respondents anonymously can deliver their surveys.
Web Surveys. Web surveys may be administered to online panels and may use experimental designs to test hypotheses about the relative efficacy of communication messages. Samples of 1,000 – 2,800 are common, depending upon the number of stratification factors and the rarity of the topic. FDA does not intend to generate policy or regulations from the results of these surveys.
Mail or Telephone Surveys. Mail and telephone surveys use address-based random sampling and a formatted questionnaire for all respondents. Due to high non-response, these survey methodologies costs have risen, and FDA is less likely to use these methods. Due to non-response, samples are likely to number near 3,000. FDA will not use the results from these surveys to generate policy or regulations.
For all methodologies, professionally recognized procedures will be followed in each information collection activity to ensure high quality data. Examples of these procedures include the following:
A minimum of 10 percent of telephone interviews will be monitored by supervisory staff;
Data from mail or paper-and-pencil surveys will be computerized through scannable forms or checked through double-key entry;
Observers will monitor focus groups, and focus group proceedings will be recorded; and
Data submitted through on-line surveys will be subjected to statistical validation techniques (such as disallowing out-of-range values).
All data collection and analysis will be performed in compliance with OMB, Privacy Act, and Protection of Human Subjects requirements.
3. Methods to Maximize Response Rates and Deal with Non-response
In the case of data collection activities that involve interviews or telephone, mail, and in-person surveys, several procedures proven effective in previous studies will be used to maximize response rates:
Potential respondents will be informed about the importance of these studies and encouraged to participate through a variety of methods, including letters of support from key individuals.
Experienced, highly-trained staff will moderate all focus groups and conduct all interviews and surveys.
Interviewers will participate in thorough training sessions. Training topics will include study objectives, question-by-question reviews of data collection instruments, strategies for engaging respondents, role playing, and techniques for fostering respondent cooperation and survey completion.
Well-defined conversion procedures will be established. If a respondent for a survey declines to be interviewed, a member of the contractor's conversion staff will contact the respondent to explain the importance of their participation. Conversion staff members are highly experienced telephone interviewers whose style and persuasive abilities have demonstrated success in eliciting cooperation. They receive a pay differential to acknowledge these skills, which also serves as an incentive to the interviewer pool, whose completion rates are carefully monitored to assess their qualifications to serve as conversion staff.
For telephone interviews, outgoing calls that result in a disposition of “no answer,” a busy signal, or an answering machine will be automatically rescheduled for subsequent attempts. Up to 20 outgoing calls to a given number with dispositions of the sort listed will be made before declaring it a non-response.
Should a respondent interrupt an interview for any reason, such as needing to attend to a personal matter, the interviewer will reschedule or, in the case of telephone surveys, a predictive dialer will automatically reschedule the interview for completion at a later time.
Fielding for telephone and mail surveys will occur over at least a six-week period. Based on past experience, this time frame will allow the contractor to reach individuals who are on vacation, out of the home during irregular periods, have a temporarily disconnected telephone, or who are not answering the phone for some other reason.
Interview staff will be able to provide respondents with the name and telephone number of an official at FDA. This official will confirm with respondents the importance of their participation.
A dedicated toll-free number will be established at FDA or a contractor’s office to allow potential respondents to hear a pre-recorded message to confirm a study’s legitimacy.
For mail surveys, a number of techniques will augment response rates:
A self-addressed, stamped return envelope will be enclosed with each survey.
Surveys will be mailed to respondents using stamps instead of metered postage labels.
Creative and attractive graphics will be used to attract the attention of respondents (e.g., different colored paper for successive survey iterations).
Hand-signed cover letters will be sent with each survey.
Follow-up mail (up to 7 mailings) or phone contacts (up to 20 call-backs) will be made to encourage participation; participant objectivity will be encouraged by reminding participants about the importance of providing both negative and positive feedback.
Respondents will be allowed the option of faxing back completed surveys (and possibly offered the option of completing the survey on-line).
4. Test of Procedures or Methods to be Undertaken
Before each information collection is implemented, a contractor will pilot test the instrument(s) and method of data collection. Lessons from the pilot test will be identified, and changes as necessary will be incorporated into the instrument and method. All pilot tests will involve no more than nine individuals unless OMB clearance is sought for more than nine.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
N/A
C. ATTACHMENTS
The Federal Food Drug and Cosmetic Act, Section 1003(d)(2)(D)
The Public Health Service Act, Section 301
60-day Federal Register Notice
Statement of Exemption from 45 CFR 46
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | miriam.campbell |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |