Part B SS (Quantitative Study of Tobacco Facts Designed to Inform Youth Tobacco Prevention Messaging)

Quantitative Study of Tobacco Facts Designed to Inform Youth Tobacco Prevention Messaging

0910-0810

Supporting Statement: Part B

B. Statistical Methods

1. Respondent Universe and Sampling Methods

The one-time actual burden figures are listed in Tables 2 & 3, Part A

The primary outcome of this study would be based on a probability sample provided by GfK’s KnowledgePanel. The goal is to receive 750 survey completes among youth ages 13-17 who are susceptible to smoking and 750 survey completes among youth ages 13-17 who have experimented with smoking. The survey is a cross-sectional design using a nationally representatives participant panel of adult parents with children between ages 13-17. Participants will be recruited through an existing GfK panel of adults with children ages 13-17 who give their permission for their youth to complete the survey. The screening criteria are based on age, smoking status, and intention to smoke in the future. As this study is considered part of formative research for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample drawn here is designed primarily to provide information on the perceived effectiveness of various tobacco-related facts that may be used in future tobacco prevention campaigns.

Sampling Methods

The GfK sample provided will be of a sample of adults who have children ages 13-17. The adult will determine if they are interested in their child participating in the survey. As this study is considered part of formative research for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample drawn here is designed primarily to provide information on the perceived effectiveness of tobacco-related facts under test. The sampling of adults is provided below.

KnowledgePanel’s probability-based recruitment was originally based exclusively on a national RDD frame. Currently, approximately 40% of panel members were recruited through RDD, while 60% were recruited using ABS. After initially accepting the invitation to join the panel, participants are asked to complete a short demographic survey (the initial profile survey); answers to these questions allow efficient panel sampling and weighting for surveys. Completion of the profile survey allows participants to become panel members. These procedures were established for the RDD-recruited panel members and continued with ABS recruited panel members. Respondents sampled from the RDD and ABS frames are provided the same privacy terms and confidentiality protections.

ABS Recruitment. ABS involves probability-based sampling of addresses from the U.S. Postal Service’s Delivery Sequence File. The key advantage of the ABS sample frame is that it allows sampling of almost all U.S. households. An estimated 97% of households are “covered” in sampling nomenclature. Regardless of household telephone status, those households can be reached and contacted through postal mail. For every survey sample, an appropriate base weight adjustment is applied to each relevant sample to correct for the stratified designs.

Randomly sampled addresses are invited to join KnowledgePanel through a series of mailings, including an initial invitation letter, a reminder postcard, and a subsequent follow-up letter.

RDD Recruitment. For panel members who were recruited using RDD-based sampling (pre-April 2009), list-assisted RDD sampling techniques were used on the sample frame consisting of the entire U.S. residential telephone population.

Sample Size

To obtain a final sample of 750 youth ages 13-17 who have experimented with smoking and 750 youth ages 13-17 who are susceptible to smoking in the future, we will need to screen approximately 6,251 potential respondents. This is because we anticipate approximately 24% of youth have experimented with smoking, and about 16% are susceptible to initiating smoking in the future (NYTS, 2014). In addition, we anticipate only about 75% of screened respondents will give both parental permission and child assent to participate in the study. The attrition is explained in Exhibit 5.

Exhibit 5: Sample Size for Tobacco Messages Survey

Exhibit 6. Addresses and the Associated Assumptions to Yield the Needed Number of Completes

Activity	Sample (Youth 13-17 who have experimented with smoking)	Sample (Youth 13-17 who are susceptible to smoking))
Households who review consent	6,251 (75%)	6,251 (75%)
Eligible youth	4,688 (24%)	4,688 (16%)
Survey completes	1,125 (however, capped at 750)	750

2. Procedures for the Collection of Information

This section describes the procedures for the survey data collection. The survey will be conducted via a web-based survey disseminated by GfK. To be eligible, the parent must give their permission, the youth must give their assent, and the youth (age 13-17) must be either a person who has experimented with cigarettes or a youth (ages 13-17) who is susceptible to smoking cigarettes in the future. The screener is outlined in Attachment C. The survey instrument (Attachment D) will include the survey questions and contact information for technical staff at RTI who will be available to respond to questions posed by participants. The survey will be hosted on GfK’s cloud-based servers. All surveys will be conducted using a self-administered, online questionnaire.

Summary of Protocol

The list of study procedures is as follows:

1. Adult panelists are recruited through GfK’s existing KnowledgePanel. Panel members are prescreened for their child’s age as well as their willingness for their child to participate in online surveys.

2. The selected adult panelists will receive an initial email invitation that indicates their child has been invited to participate in a new survey (Attachment A). The email invitations will also state the length of the survey, confidentiality standards, implementer and incentive he/she will receive if the child qualifies and completes the survey.

3. If the parent determines that they would like their child to participate in this survey, they will be asked to provide parental permission and an email address for the child.

4. An introductory email will be sent to the youth. The email will invite them to participate and request the child’s assent (Attachment B). The assent forms will provide a description of the purpose, implementer and confidentiality standards associated with the study.

5. If the youth gives their assent, then they will be redirected to the online screener questions (Attachment C).

6. If the respondent qualifies for the survey, he or she will begin the survey questions (Attachment D).

7. If the respondent does not qualify for the survey, he or she will receive text thanking them for their time and explaining that they do not qualify for the survey.

Due to GfK’s protected panel technology, it will not be possible for anyone to enter the survey who has not been recruited through their parent, or for a respondent to complete the survey more than once. In addition, the same-worded invitation will be sent at regular intervals after the original invitation is sent to those respondents who have not yet responded.

Unusual Problems Requiring Specialized Sampling Procedures

No specialized sampling procedures are involved.

Use of Periodic Data Collection Cycles to Reduce Burden

This is a one-time survey data collection effort.

3. Methods to Maximize Response Rates

The ability to obtain the cooperation of potential respondents in the survey will be important to the success of this study. RTI will minimize the non-response rate by employing the following measures:

1. Working with GfK, who uses an existing panel of parent participants, who have previously indicated their willingness for a child in their household ages 13-17 to participate in youth studies.

2. Sending a reminder email for initial non-response

3. Provide incentives

GfK maintains a panel of participants who have expressed interest in completing surveys. By opting in to the pool of potential survey respondents and being familiar with GfK, we will be increasing the likelihood of participant response.

In addition to the participant’s familiarity with GfK, reminder emails will also be sent to the parental panel member or youth respondent if they do not respond to the survey inquiry after 7-10 days (Attachments A & B).

Incentives will be received as non-monetary ‘points’ through the parent’s GfK account. Points can later be redeemed by the parent on behalf of their child. These points can be redeemed by the parent through GfK’s system for goods or gift cards. The approximate value of the points is $5 per survey. We estimate that the survey will take 20 minutes to complete. The incentives are intended to recognize the time burden placed on participants, encourage their cooperation, and convey appreciation for contributing to this important study and are similar to incentives that are offered for most surveys of this type. Numerous empirical studies have shown that incentives can significantly increase response rates in cross-sectional surveys and reduce attrition in longitudinal surveys (e.g., Abreu & Winters, 1999; Castiglioni, Pforr, & Krieger, 2008; Jäckle & Lynn, 2008; Shettle & Mooney, 1999; Singer, 2002).

4. Tests of Procedures or Methods

The evaluation contractor RTI will conduct rigorous internal testing of the online survey instrument prior to its fielding. Evaluators will review the online test version of the instrument that we will use to verify that instrument skip patterns are functioning properly, delivery of campaign media materials is working properly, and that all survey questions are worded correctly and are in accordance with the instrument approved by OMB.

5. Individuals Involved in Statistical Consultation and Information Collection

The following individuals inside the agency have been consulted on the design of the campaign evaluation plan, audience questionnaire development, or intra-agency coordination of information collection efforts:

Tesfa Alexander

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993-0002

Phone: 301-796-9335

E-mail: [email protected]

Atanaska (Nasi) Dineva

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993-0002

Phone: 301-796-4498

E-mail: [email protected]

Chaunetta Jones

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993-0002

Phone: 240-402-0427

E-mail: [email protected]

Xiaoquan Zhao

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993-0002

Phone: 240-402-0296

E-mail: [email protected]

The following individuals outside of the agency have been consulted on questionnaire development.

Jennifer Duke

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-485-2269

E-mail: [email protected]

James Nonnemaker

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-541-7064

E-mail: [email protected]

Annice Kim

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-316-3972

E-mail: [email protected]

Bonnie Hepler

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-541-5928

E-mail: [email protected]

References

Abreu, D. A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. In Proceedings of the Survey Research Methods Section (pp. 533-538).

Castiglioni, L., Pforr, K., & Krieger, U. (2008, December). The effect of incentives on response rates and panel attrition: Results of a controlled experiment. In Survey Research Methods (Vol. 2, No. 3, pp. 151-158).

Jäckle, A., & Lynn, P. (2008). Offre de primes d’encouragement aux répondants dans une enquête par panel multimodes: effets cumulatifs sur la non-réponse et le biais. Techniques d’enquête, 34(1), 115-130.

Singer, E. (2002). The use of incentives to reduce nonresponse in household surveys. Survey nonresponse, 51, 163-177.

Shettle, C., & Mooney, G. (1999). Monetary incentives in US government surveys. Journal of Official Statistics, 15(2), 231.