Rationale for Outcome Evaluation Measures

ATTACHMENT 2c: Rationale for Outcome Evaluation Measures

Measurable population-level behavior change—such as a change in adult smoking prevalence—is the product of a series of changes in interrelated, individual-level beliefs, attitudes and perceptions about norms, and environmental-level factors such as smoke-free laws (Ajzen & Fishbein, 1980; Bandura, 1985; Hornik, 2002; Rosenstock, Strecher, & Becker, 1988; Trickett et al., 2011). Behavior change theory guides our understanding of how campaigns function (Ajzen, 1991; Bandura, 1985; Rosenstock et al., 1988; Trickett et al., 2011) and defines our expectations about the order in which campaign effects should occur: belief change, attitude and social norm change, intention, and finally behavior change (Fishbein, 1967). The Centers for Disease Control and Prevention’s (CDC’s) Best Practices for Comprehensive Tobacco Control Programs quantifies the timeline for these expectations, indicating that campaigns that deliver a sufficient amount of media will produce campaign awareness at 6 months, attitude change at 12 to 18 months, and behavior change at 18 to 24 months (CDC, 2007). A National Cancer Institute (NCI) study similarly concludes that campaigns “influence attitudes toward tobacco within a short period, followed by longer-term effects on smoking behavior” (NCI, 2008, p. 534). In practice, changes in beliefs, attitudes, and intention are often the first indicators of campaign effectiveness and, as a result, are among the first outcomes examined in the course of campaign evaluation (Cowell et al., 2009; Farrelly et al., 2005; Murray, Prokhorov, & Harty, 1994; Murukutla et al., 2012; Vallone et al., 2011a, b).

Evaluation Logic Model

Based on this evidence base and previous experience with tobacco-focused public education campaigns, we have mapped the expected relationships between specific campaign activities and downstream outcome indicators (Figure 1-1). This model further outlines key variables and other contextual influences on tobacco-related outcomes that may moderate the effects of the campaigns and therefore must be accounted for in our assessment of the campaigns’ impacts on key outcomes. Based on this model, we hypothesize that greater exposure to the campaigns will lead to greater changes in all key outcomes at all stages of time (short-term, intermediate, and long-term).

Figure 1-1. Evaluation Logic Model

Evaluation Questions

In this section, we present initial evaluation questions following the logic model described above and our current understanding of the creative direction of the point of sale campaign. The key evaluation questions we seek to answer fall under several broad domains, as outlined in Table 1‑1: campaign awareness and receptivity, short-term cognitive outcomes, intermediate-term cognitive outcomes, and long-term behavioral outcomes. Although this is not an exhaustive list of all possible evaluation questions that this evaluation will address, the enumeration of questions in Table 1‑1 provides a detailed overview of the outcomes that are the focus of these campaigns and our evaluation. Table 1-2 provides the survey items enumerated by type and a description of how the various survey measures will be used in analyses.

Table ‎1-1. Campaign Key Evaluation Questions

Short-Term Cognitive Outcomes (illustrative)

Is there a difference in tobacco-related knowledge, attitudes, and beliefs, between the treatment and control groups, and do those difference change over time?

Intermediate Cognitive Outcomes

Do participants in the treatment group demonstrate greater intention to quit smoking cigarettes than those in the control group, and does this difference change over time?

Do participants in the treatment group demonstrate greater motivation to quit smoking cigarettes than those in the control group, and does this difference change over time?

Long-Term Behavioral Outcomes

Is the number of quit attempts in the treatment group greater than the control group, and does this difference change over time?

Among those smokers that continue smoking through the evaluation, are participants in the treatment group more likely to decrease the number of cigarettes they smoke per day compared to participants in the control group, and does this difference change over time?

Table 1-2. Survey Items by Type and Intended in Analysis, Outcome Evaluation Survey

Type of Item	Survey Item	Intended Use in Analysis of Outcome Evaluation Data
Wave 1 Questionnaire
Instructions	A	none
Demographics	B1 – B5	control variable
Tobacco-related Behaviors	C1-C18	control variables
Tobacco Use Intentions and Self-efficacy	D1-D4	outcome variable
Cessation	E1-E20	outcome variable
Tobacco-related Attitudes, Beliefs, Risk Perceptions, and Social Norms	F1-F6	outcome variable
Media Use and Awareness	G1-G12	control and independent variables
Environment	H1-H16	control variable
Locator Questions	AL_INT1- AL_A2PEM	none
Participation in App-based Portion of Evaluation	J1	none

References

Ajzen I. The theory of planned behavior. Organ Behav Hum Dec, 1991;50:179-211.

Ajzen I, Fishbein M. Understanding attitudes and predicting social behavior. London, England: Pearson; 1980.

Bandura A. Social foundations of thought and action: a social cognitive theory. London, England: Pearson; 1985.

Centers for Disease Control and Prevention. Best Practices for Comprehensive Tobacco Control Programs—2007. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2007.

Centers for Disease Control and Prevention (CDC). Framework for program evaluation in public health. (1999). MMWR Recomm Rep, 1999;48(RR-11), 1-40.

Cowell AJ, Farrelly MC, Chou R, Vallone DM. Assessing the impact of the national ‘truth’ antismoking campaign on beliefs, attitudes, and intent to smoke by race/ethnicity. Ethn Health. Feb 2009;14(1):75-91.

Farrelly MC, Davis KC, Haviland ML, Messeri P, Healton CG. Evidence of a dose-response relationship between “truth” antismoking ads and youth smoking prevalence. Am J Public Health. Mar 2005;95(3):425-431.

Fishbein M. A consideration of beliefs and their role in attitude measurement. In: Fishbein M, ed. Readings in attitude theory and measurement. New York, NY: Wiley; 1967:257–266.

Hornik R. Public health communication: Evidence for behavior change. London, NJ: Lawrence Erlbaum Associates; 2002.

Murray DM, Prokhorov AV, Harty KC. Effects of a statewide antismoking campaign on mass media messages and smoking beliefs. Prev Med. Jan 1994;23(1):54-60.

Murukutla N, Turk T, Prasad CV, et al. Results of a national mass media campaign in India to warn against the dangers of smokeless tobacco consumption. Tob Control. Jan 2012;21(1):12-17.

National Cancer Institute (NCI). The role of the media in promoting and reducing tobacco use. Tobacco Control Monograph No. 19. Bethesda, MD: U.S. Department of Health and Human Services, National Institutes of Health, National Cancer Institute 2008.

Rosenstock IM, Strecher VJ, Becker MH. Social learning theory and the Health Belief Model. Health Educ Q. Summer 1988;15(2):175-183.

Trickett EJ, Beehler S, Deutsch C, et al. Advancing the science of community-level interventions. Am J Public Health. Aug 2011;101(8):1410-1419.

Vallone DM, Duke JC, Cullen J, McCausland KL, Allen JA. Evaluation of EX: a national mass media smoking cessation campaign. Am J Public Health. Feb 2011;101(2):302-309.

Vallone DM, Niederdeppe J, Richardson AK, Patwardhan P, Niaura R, Cullen J. A national mass media smoking cessation campaign: effects by race/ethnicity and education. Am J Health Promot. May-Jun 2011;25(5 Suppl):S38-50.

4