Generic
Information Collection Through Web-based Surveys for Evaluating Act Against AIDS (AAA) Social Marketing Campaign Phases Targeting Consumers
Supporting Statement B
Submitted by:
RTI International
Date: September 12, 2012
Contact Person:
Jo Ellen Stryker, PhD
1600
Clifton Rd. NE
Mailstop E-49
Atlanta, GA 30329
Telephone: (404) 639-2071
Fax: (404) 639-2007
E-mail: [email protected]
Table of Contents
B. Collections of Information Employing Statistical Methods 1
B.1 Respondent Universe and Sampling Methods 1
B.2 Procedures for the Collection of Information 3
B.3 Methods to Maximize Response Rates and Deal with Nonresponse 7
B.4 Test of Procedures or Methods to Be Undertaken 8
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 9
Exhibits
Number Page
Exhibit 8.
The study will consist of a tracking survey of AAA target audiences to measure exposure to each phase and messages of the campaign and interventions implemented under the AAA campaign. The study will include a sample of adults aged 18 or older recruited from online survey vendors (e.g., Harris Interactive, Knowledge Networks, e-Rewards) and/or the evaluation contractor (RTI). The online survey vendor and/or RTI sample will be supplemented with additional respondents generated by (1) partnership organizations (e.g., the National Urban League, the National Medical Association), (2) online advertising (e.g., banner ads, electronic bulletin boards), and (3) external partners (e.g., community-based organizations, health departments).
Obtaining a probability-based sample to reach the desired subpopulations of interest is cost prohibitive for this survey. The target audiences for the AAA campaign are varied and in most instances will be narrowly defined (e.g., African Americans at high risk of HIV infection; MSM). Although the sample is not meant to be generalized to the entire population of interest, this collection of information will enable CDC to gather evaluation information to more effectively address HIV testing and prevention.
The data provided from the proposed evaluation will be used to understand the appropriateness of continued or expanded funding and dissemination of the campaign. Note that these data will not be used for surveillance purposes, but rather to gather consumer responses to campaign phases and messages to inform and refine future campaign messages and media placement. Because the available sample pool from any one source will not provide the necessary sample size, the combination of sources described previously will be utilized. The advantage of this approach is that it provides for joint analysis of online survey vendor panels and additional sample sources and comparisons between them, provided sufficient sample sizes from all sources.
The online survey vendors and/or RTI will use a range of proprietary sources to obtain their samples. Because the samples are not randomly selected, they may be biased. Because the sample for this study is extremely targeted, our participants should represent some, but not all, of the campaign’s target populations. The campaign’s target audiences may have more limited Internet access and may have other differences when compared with the populations of interest. However, detailed information on respondent characteristics will be gathered. The screeners will gather basic demographic information from which we can describe the sample.
The frequency and timing of the online surveys will vary, but will not exceed 6,000 burden hours per year (as reported in Exhibit A.12.1 Annualized Burden Hours). The proportions of the sample will be determined by the target audience of the campaign phases and specific parameters of sample composition. To reduce the effects of nonsampling error, nonresponse and post-stratification weighting adjustments will be applied to the sample when feasible.
This study will allow us to: 1) to evaluate the potential effectiveness of the AAA campaign messages as they are developed; and 2) to examine the associations between those groups who report exposure to the various AAA messages and those reporting no exposure to the various AAA messages. Any differences in detected cannot be directly attributed to the campaign.
We conducted power analyses to determine the optimal sample size for detecting statistically significant differences among individuals exposed to the various AAA campaign messages and those not exposed. For this analysis, we tested the power to detect a difference between the percentage of exposed and non-exposed participants who talked to their partner about HIV status. We expect to find that 30% of participants have been exposed to the intervention. We assume, based on other studies, that 65% of nonexposed participants report talking to their partner about their HIV status. Given a type I error rate of 0.05 (alpha = 0.05), we will achieve 80% power to detect that 74% of respondents in the exposed group talked about HIV status with a sample size of 1,000.
Subject recruitment procedures will vary based on how they are identified: (1) online survey vendors, (2) partnership organizations, (3) internet, or (4) external partners. The procedures for collection of information will be the same for subjects identified through online survey vendors and partnership organization. The selected online survey vendor will identify individuals who fall into the targeted audience for the specific campaign being assessed using their market research panel. This list will be supplemented with sample lists received from partnership organizations. The online survey vendor and/or RTI will send e-mail invitations to the combined sample list. Each invitation will contain the survey title, a brief description of the survey, the length of the survey, incentive amount provided for successful completion of the survey, and instructions for accessing the secure Web site for the online screener (including the provision of a personal password they must use to enter the screener).
To recruit participants via the internet, we will purchase online advertisements (e.g., banner advertisements) on targeted websites. Websites will be chosen based on the campaign’s focus. For example, if the campaign is focused on African American men who have sex with men, we will place the advertisement on websites catering to this demographic group. The advertisement will describe the survey in brief, length of the survey, and incentive amount provided for successful completion of the survey. Individual will be asked if they are willing to answer a few questions to determine if they are eligible. Each participant must check either a box labeled “YES, I agree to be screened” or “NO, I do not wish to be screened.” Only respondents who consent will be provided with a personal password they must use to enter the screener.
We will also partner with external organizations, such as CBOs and health departments, to recruit potential participants for data collection. By working with external partners, we can extend our reach to individuals who are unlikely to be accessed through online recruitment methods. We will identify external partners who work with members of a particular campaign’s target audience and ask them to assist us with subject identification by placing flyers, posters, and other materials (e.g., palm cards, brochures) in client waiting areas and exam rooms. The materials will include a URL for more information about the survey. The URL will describe the survey in brief, length of the survey, and incentive amount provided for successful completion of the survey. Individual will be asked if they are willing to answer a few questions to determine if they are eligible. Each participant must check either a box labeled “YES, I agree to be screened” or “NO, I do not wish to be screened.” Only respondents who consent to be screened will be provided with a personal password they must use to enter the screener.
After participants enter their personal passwords, they will complete a brief online screener (see sample in Attachment 6) to determine eligibility. Eligible participants include adults who meet the criteria for the targeted audience of the AAA campaign phase being assessed.
Individuals who are eligible will be provided with general information about the survey, topics to be covered, potential risks from the survey, and the token of appreciation available for completing the survey and administered informed consent. Potential participants will be informed of the private and voluntary nature of the survey. Panelists and sample participants identified through other means will be given separate consent forms that differ only in the explanation of the token of appreciation available for completing the survey. Panelists will be given the token of appreciation via the vendor’s bonus points system, which includes points that are redeemable for cash, while sample participants identified through other sources will receive checks directly sent to them in the mail. After reading the informed consent, each participant must check through point-and-click acceptance either a box labeled “YES, I agree to participate” or “NO, I do not wish to participate.” Only respondents who consent will be routed to the survey.
To enter the survey, participants must enter their personal password which will be needed each time they access the survey. A respondent’s personal password will not change. Study participants will be given a designated period during which the survey will be available for them to complete, making it feasible for participants to complete the survey during their own time, in private. This mechanism therefore makes this study suitable for addressing sensitive topics, such as sexual behavior and HIV status, while also improving the accuracy and validity of the data obtained for these sensitive topics. Participants can complete each survey only once.
Data from completed surveys will be compiled into a SPSS data set by the online survey vendor and/or RTI. RTI will do all analyses.
Recruits from online survey vendors and/or RTI and the additional sample members identified through partnership organization who do not respond will receive one e-mail reminder from the online survey vendor and/or RTI requesting their participation in the survey. A sample e-mail notification for non respondents is provided in Attachment 5. The surveys will be self-administered and accessible any time of day for a designated period. All data collection materials are at an 8th grade reading level or below due to sample eligibility criteria and CDC requirements.
Individuals who complete the survey will receive up to $40 in cash or cash equivalent as a token of appreciation. Adults are difficult to engage in a survey about this sensitive topic without the use of a small token of appreciation. The token of appreciation is intended to recognize the time burden placed on participants, encourage their cooperation, and convey appreciation for contributing to this important study over three data collection periods. A toll-free telephone number for CDC INFO (1-800-CDC-INFO) will be provided to all participants during the informed consent process if they have additional questions about HIV/AIDS.
Our sample design is based on conservative assumptions about survey response. Thus, our estimates of cooperation rates should be viewed as “worst case” scenarios that, if they hold true, would still ensure sufficient sample sizes to reasonably detect small message effects. We estimate that at least 20% of respondents will complete the survey.
Exhibit 8. Total Number of Participants for the Survey over a Three Year Period
Numbers and Cooperation Rates |
Three Year Total |
Number of participants to be contacted |
60,000 |
Expected cooperation rate |
20% |
Number of completed surveys |
12,000 |
The following procedures will be used to maximize cooperation and to achieve the desired high response rates:
A token of appreciation in cash or cash equivalent (e.g., bonus points) will be provided to respondents upon completion of the survey.
Survey vendors and/or RTI will send one email reminder to panelists and individuals identified through partnership organizations who do not respond requesting their participation in the survey.
Online survey vendors (if used) will provide toll-free telephone numbers to all respondents so that they can call with any questions or concerns about any aspect of the study. RTI will provide a toll-free telephone number for the RTI project director and a toll-free telephone number for the RTI institutional review board (IRB) hotline should participants have any questions about the study or their rights as a study participant.
Online survey vendor data collection staff (if used) will work with RTI project staff to address concerns that may arise.
A study overview will be included in the introductory information for participants prior to each survey. The information will present an interesting and appealing image and alert participants to the upcoming surveys.
To estimate the survey burden for each respondent, two survey specialists will be consulted. The survey specialists will conduct mock interviews and provide affirmative responses to most or all questions that branch to follow-up questions. In this way, the burden estimate will most closely resemble a maximum average burden, since almost all survey questions were presented in the interview. In addition, the survey specialists will deliberately read each item slowly. RTI’s experience has shown that interviewer-administered questionnaires take longer to complete than self-administered questionnaires. The survey specialists tested the study screener and estimated the maximum average burden to be 2 minutes for the study screener. The survey specialists will test each survey separately. Any one survey will consist of items that take no more than an average of 30 minutes to complete. The sample survey Attachment 3a and 3b provides the sample items used to address the purpose of the study to; 1)evaluate the potential effectiveness of the AAA campaign messages during the campaign development phase; and 2) examine the associations between those groups who report exposure to the various AAA messages and those reporting no exposure to the various AAA messages. All survey items for each campaign will be included in the mini-ICR. The sample study screener is provided in Attachment 6.
Before implementing each study, RTI, the selected online survey vendor (if used), and CDC staff will test the entire process of self-administering each online survey. This will enable us to pilot test each survey programming and logic and correct any potential problems before the experiment is implemented with the actual sample of participants.
Burton Levine
Statistician
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
919-541-1252
Kevin Davis
Data Collection, Analysis, and Reporting Task Leader
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
919-541-5801
Bethany
D. Moffett
e-Rewards Market Research Contact
8401 N.
Central Expressway, Suite 900
Dallas, TX 75225
214-365-5024
Jennifer D. Uhrig
RTI Project Director
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
919-316-3311
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | snaauw |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |