Screener

Attachment A

Screener

Screener

Center for Substance Abuse Prevention (CSAP) Underage Drinking Prevention: “Talk. They Hear You.” Impact Evaluation Tool Development—Media Testing Survey

Composition of survey population:

Target population—parents and caregivers of children ages 9 to 15 who have had some exposure to the “Talk. They Hear You.” Campaign.

Sample frame:

To ensure that a critical proportion of respondents have been exposed to the campaign, we will build the sample frame using contact lists from organizations that have partnered with SAMHSA to advocate against underage drinking and related risky behaviors. We will request lists from at least three national organizations with a focus on parental engagement in preventing risky behaviors. We will request that the lists include the membership’s e-mail addresses, and if possible, city and state. If we are able to obtain the city and state, we will select our respondents in the metropolitan statistical areas (MSAs) of the six focus cities of the campaign:

• Atlanta (Georgia)

• Los Angeles (California)

• Manhasset (New York)

• Oklahoma City (Oklahoma)

• Phoenix (Arizona)

• Washington, DC

These areas were identified as having the greatest number of media exposures to the campaign materials, increasing the chances that individuals contacted will have seen the “Talk. They Hear You.” materials.

From these regional lists filtered from the master lists, we will select a simple random sample of respondents from an unsorted list containing contact information from all participating organizations. 

Our goal is to receive responses from 150 parents/caregivers from the target population. Assuming a typical 12-percent response rate for electronic surveys would dictate an initial mailing of 1,250 survey invitations. However, some percentage of respondents will be screened out because they no longer have children in the desired age range. We are therefore planning to oversample slightly at 1,500. We also anticipate that there will be some percentage of undeliverable e-mails. We will replace any undeliverable messages with additional individuals selected from the e-mail list to match the MSA of the rejected address.

Methodology:

The media testing survey will be administered electronically, via Qualtrics Survey Suite. This application allows the design of the survey to follow multiple paths so that response time is minimized—respondents do not see questions that do not apply to them. There is a single screening question, included in the screen shot below, to determine whether the respondent is a member of the target audience. Additional sample screen shots and a hardcopy of the full survey are included in Attachment C.

Electronic e-mail lists provided by our partner agencies can be loaded directly into Qualtrics, minimizing time spent recruiting participants; however, this list is maintained separately from the responses in order to maintain anonymity (for additional information, see http://www.qualtrics.com/). Individuals will be sent an e-mail invitation to participate in the survey, with an anonymous survey link embedded in the invitation. The system will provide reports on which e-mails were undeliverable, allowing us to replace these invitations with additional e-mail addresses.

Synergy Enterprises, Inc. (Synergy) is our contractor on this project. Synergy has developed the survey in consultation with CSAP and will program the survey in Qualtrics, upload the mailing lists, and distribute and track the survey responses. There is no additional expense as Synergy already maintains a license with Qualtrics.

Timeframe:

The start date is dependent on receiving Office of Management and Budget (OMB) clearance. Some ground work has been done with partners in preparing this study plan, and we anticipate that we can have the mailing lists ready to upload within 2 weeks of receiving approval. To avoid conflicts with holidays, we do not plan to send the e-mail invitation before January 4. The survey will remain open for 15 days.

Screen shot 1: Draft screening question