SFW-Study_Mini-SSA_03102014

Mini Supporting Statement A For

“A Generic Submission for Formative Research, Pretesting, and Customer Satisfaction of NCI’s Communication and Education Resources”

OMB No. 0925-0046, Expiration Date 5/31/2016

Title of Sub-Study: Smokefree Women Customer Satisfaction Study

Heather Patrick, Tobacco Control Research Branch, Division of Cancer Control and Population Sciences, National Cancer Institute (NCI), National Institutes of Health (NIH)

Section A.

A1. Circumstances Making the Collection of Information Necessary

Section 410 of the Public Health Service Act (42 USC § 285) authorizes the collection of the information. The PHS Act authorizes National Cancer Institute (NCI) to establish and support programs for the detection, diagnosis, prevention and treatment of cancer; and to collect, identify, analyze and disseminate information on cancer research, diagnosis, prevention and treatment. The Tobacco Control Research Branch (TCRB) located in the Division of Cancer Control and Population Sciences (DCCPS), initiates, supports, and evaluates both basic behavioral research efforts and research that supports cancer interventions. TCRB’s mission is to lead and collaborate on research, and to disseminate evidence-based findings to prevent, treat, and control tobacco use.

This study is part of a broader study in which the Office of the Assistant Secretary for Health (OASH) at the Department of Health and Human Services (DHHS) has requested that the TCRB continue to provide smoking cessation resources to targeted populations, such as women, as part of a national comprehensive tobacco control effort. This formative research study will demonstrate the feasibility of conducting a rigorous evaluation of the broader initiative (for which a separate new OMB submission will be submitted). This study is simply assessing which website content is most effective and satisfying in support of women becoming smoke free.

TCRB developed the Smokefree program – a program that includes a variety of web- and mobile-based resources to help people initiate and sustain quit attempts. One component of TCRB’s Smokefree program includes the Smokefree Women (SFW) website (women.smokefree.gov) which in the past 18 months has had nearly 1 million visitors. In 2012, TCRB conducted a simple customer satisfaction survey to better understand the stakeholders that were frequenting the website and their smoking behaviors to ultimately tailor the website for the users’ needs (OMB No. 0925-0642-12, Approved 5/16/2012). This current survey is to assess which website the stakeholder is most satisfied with as well as the relevancy of that content for the intended stakeholder population. To conduct this formative study, a sample will be randomized to view 1 of 4 different variations of the Smokefee Women website. The websites include:

• Current Smokefree Women website (women.smokefree.gov)

• Smoking cessation (SC) content only,

• SC plus weight management content,

• SC plus healthier lifestyle content.

This study fits within scope of the full generic as stated in the original Supporting Statement A:

“The formative research process is used to determine whether or not a draft message or message concept is effective in reaching and communicating with its audience. Pretest­ing involves presentation of draft messages designed to convey specific information to a sample of the audience for whom the materi­als are intended. These respon­dents are asked to give their reaction to the messages…. Information collected to determine the level of customer satisfaction with products helps NCI identify strategies for improving the accessibility of materials/programs, their user-friendliness, and their relevance...” (SSA written 4/8/2013, p. 4)

A2. Purpose and Use of the Information Collection

The primary purpose of the SFW Customer Satisfaction Study is to better understand the attitudes, preferences, and experiences of the SFW target audience in an effort to improve future products and communication materials. The data obtained will be used to customize SFW’s content more specifically to users’ needs. The ultimate goal is to increase customer satisfaction and website content relevancy for the target population, ultimately increasing attempted and sustained quit attempts of our audience.

In 2012, TCRB conducted a simple customer satisfaction survey to better understand the stakeholders that were frequenting the website and their smoking behaviors to ultimately tailor the website for the users’ needs (OMB No. 0925-0642-12, Approved 5/16/2012). This current survey is to assess which website the stakeholder is most satisfied with as well as the relevancy of the website content for the intended stakeholder population. In order to understand and increase satisfaction of the target population, this study evaluates customer satisfaction through the lens of cessation outcomes. Ultimately, we will attempt to better understand cessation outcome measures in relation to participant’s attitudes, perceptions, and behaviors. This is based upon the primary purpose of this web-assisted smoking cessation intervention, to help people quit smoking. Ability to achieve this desired outcome may affect customer satisfaction. In order to evaluate this, a more sophisticated approach to customer satisfaction will be employed within this study.

Respondents will be asked to complete a web-based 1) screener, 2) initial survey, and 3) customer satisfaction survey after an extended opportunity to interact with specific site content over the course of four weeks. The purpose of the screener is to ensure that the intended population is being reached for this study and driven to the website. The purpose of the initial survey is to collect information on demographic, motivation, and health behavior patterns that will help to further segment the audience and ensure the website content is relevant to the target population (e.g. motivation to quit; physical activity level; weight status; current smoking status). These factors are known to influence customer satisfaction and engagement with health behavior change resources. The purpose of the customer satisfaction survey is to gather website satisfaction information.

Ultimately, this will allow TCRB to know whether or not it is delivering relevant website content that was intended upon its inception to its diverse audience base. At the conclusion of this study, one website will be identified based on users’ needs, satisfaction and relevancy.

A3. Use of Information Technology and Burden Reduction

All study contact (recruitment, consent, email reminders, and survey administration) will occur online. The recruitment will take place online via established Smokefree websites (e.g. Smokefree.gov) or ad placement by trusted vendors. These vendors will have established recruitment panels to target this specific population to better meet eligibility requirements and minimize fraud. Potential participants will be taken to the study site landing page. After potential participants give consent (Attachment A) and their email address, they are sent an email verification request (Attachment B), providing a link to the study Screener (Attachment C). If users are eligible they are opted into the study and will receive study reminders (Attachment B) and the Customer Satisfaction Survey (Attachment D) online.

All surveys are administered through a secure electronic platform utilized by NCI. Participant responses are exported to an excel file and will be tabulated and analyzed by the research team.

A4. Efforts to Identify Duplication and Use of Similar Information

To date NCI has not had an assessment of these four potential Smokefree websites. The Office of the Assistant Secretary for Planning and Evaluation (ASPE) in HHS has reviewed this proposed collection of information, and has determined that it does not duplicate other collections because this ICR is designed to test the specific content of four websites, all of which are targeted to women and all of which were specifically developed as part of the Smokefree program. As a result of the specific characteristics of the respondent population as well as the specific website content, this collection of information is not duplicative of other satisfaction surveys.

A5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this study.

A6. Consequences of Collecting the Information Less Frequently

This is a one-time collection that includes an initial (baseline) and a final survey.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This study will be implemented in a manner that fully complies with the Guidelines of 5 CFR 1320.5, except that it will request information from the stakeholder twice within a one month period. The reason for a one month separation between the two surveys is to allow the stakeholder to access and view the website resources several times. It has been shown that people are most likely to leave a web page in the first 30 seconds.¹ In order to increase the validity of this study, respondents are sent two emails a week for four weeks with the request that they visit the website to increase their familiarity with the content.

A8. Comments in Response to Federal Register Notice and Efforts to Consult Outside Agency

The Smokefree program represents a collaborative effort between The Office of the Assistant Secretary for Health (OASH) at the Department of Health and Human Services (DHHS) and NCI’s TCRB. HHS allocated additional funds to support the Smokefree program this fiscal year. OASH has been consulted regarding studies aimed at increasing audience usage, satisfaction, and sustained quit attempts, which includes this project.

A9. Explanation of Any Payment or Gift to Respondents

The full generic states that, “…in situations when the general public is completing an online survey, no remuneration will be involved unless influenced by other factors” (underline provided for emphasis). Nonresponse and high attrition rates are known challenges to performing research. Effective incentive amounts can reduce both item nonresponse and attrition (Castiglioni et al., 2008; Jackle & Lynn, 20⁰8²). The full generic states that:

“Research has shown the advantages of providing a small incentive for improving response rates and decreasing item nonresponse, especially in mail and telephone surveys.³ The National Survey of Family Growth conducted an experiment with remuneration of respondents and found that incentives increased response rates, reduced interviewer labor (broken appointments and callbacks), and improved data quality.⁴

For this project, respondents are asked to complete 60 minutes worth of surveys (screener, initial and final surveys) over the course of 4 weeks. To try and reduce the amount of nonresponse and high attrition rates, respondents will receive a chance to win a $20 Amazon gift card as a token of appreciation upon completion of the final survey (Attachment D). The context of this project is similar to a previously approved OMB study in which $100 was offered to respondents to complete 110 minutes of burden (Smokefree TXT (Text Message) Evaluation Program”, OMB No. 0925-0676, Expiration Date 5/31/2015).

A10. Assurance of Confidentiality Provided to Respondents

All participation will be voluntary. All participants will be given an opportunity to review the consent form (Attachment A) and they can choose to continue on with the screening survey or terminate at that point. Although we are not collecting names, the incentive will be sent electronically to the participants’ self-disclosed email addresses. Personally Identifiable Information will be collected in the form of participants’ email addresses. Rights of study participants are protected by The Privacy Act of 1974, as amended, 5 U.S.C. 552a. Given the study seeks to evaluate tasks aimed at increasing audience usage, satisfaction, and sustained quit attempts and better understand the attitudes, preferences, and experiences of the target audience in an effort to improve future products and communication materials, the Privacy Act Systems of Records Notice (SORN) #09-25-0156, “Records of Participants in Programs and Respondents in Surveys Used to Evaluate Programs of the Public Health Service, HHS/PHS/NIH/OD” (67 FR 60743, 9/26/2002), has been deemed appropriate by the NIH Privacy Act Officer.

Participant email addresses will be de-identified so they cannot be used to identify an individual by persons outside of the core project team. Should the participant choose to discontinue participating at any time during the survey, their information is erased and they will not receive compensation. Upon completion of the study and participant redemption of the e-gift card, their information will be erased.

A11. Justification for Sensitive Questions

Personally identifiable information (PII) is being collected in the form of the respondent’s email address. There are no sensitive questions being asked.

A12. Estimates of Hour Burden Including Annualized Hourly Costs

Respondents will be asked to complete a consent form, screener, initial survey, and customer satisfaction survey after an extended opportunity to interact with specific site content over the course of four weeks. These activities amount to a total of 60 minutes of burden per respondent.

Table A12-1. Estimates of Hour Burden

Type of Respondent	Instrument	Number of Respondents*	Number of Responses Per Respondent	Average Burden Per Response (in hours)	Total Burden Hours
Individuals	Consent	400	1	5/60	33
	Screener	400	1	5/60	33
	Initial (Baseline) Survey	280	1	10/60	47
	Final Survey*	224	1	40/60**	149
Totals					262

*This is an estimated 20% attrition rate over four weeks from the initial to the final survey.

**This burden calculation takes into account the time it will take to complete the final survey (20 minutes) plus additional website visits over the course of 4 weeks. Emails will be sent each week for 4 weeks, requesting the respondent visit the website. It is anticipated that the respondent will spend 10 to 20 minutes reviewing the website over the course of the study.

The average hourly mean wage rates are based on the May 2012 National Occupational Employment and Wage Estimates for the Unites States (http://www.bls.gov/oes/current/oes_nat.htm#00-0000).

Table A12-2. Cost to Respondents

Type of Respondent	Number of Respondents	Total Burden Hours	Wage Rate	Respondent Cost
Individuals	400	33	$22.01	$726.33
	400	33	$22.01	$726.33
	280	47	$22.01	$1,034.47
	224	149	$22.01	$3,279.49
Totals				$5,766.62

A13. Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no capital costs, operating costs, or maintenance costs to report.

A14. Annualized Cost to the Federal Government

The annualized cost to the Federal Government amounts to $48,980.

Table A14-1. Cost to the Federal Government

Staffing	Task	Annualized Cost
NCI	NCI GS-12 Program Officer (10% time for 6 months)	$4,500
Contractor	Study Management Support, Recruitment, and Incentive	$24,480
	R&D, Study Site Development, Development of Database, Collecting Data, Cleaning Data, Sending Email, Running Reports, and Managing Data Tables	$20,000
Totals		$48,980

A15. Explanation for Program Changes or Adjustments

This is a new collection of information.

A16. Plans for Tabulation and Publication and Study Time Schedule

Analysis of results will include the following:

• Tabulation of Initial Survey and Customer Satisfaction Survey

• Analysis of specific audience characteristics in relation to their satisfaction with resources on the site and how characteristics such as motivation to change health behavior may change following interaction with specific site features.

The results from this formative research and any publications or presentations are not generalizable or used to make broad, expansive conclusions from this sample size. The results of this study will be used to assess which website best meets the needs of the audience and may be presented and published as part of the larger, upcoming evaluation Smokefree initiative at HHS and NCI.

The study time schedule is outlined in Table A.16-1.

Table A16-1. Study Time Schedule

Activity	Months after OMB Approval
Recruit participants (collect information)	Month 0 – 3
Complete Study (final participants)	Month 4
Clean data and run analyses	Months 5 & 6
Summarize results	Month 7

A17. Reason(s) Display of OMB Expiration Date Is Inappropriate

The OMB Clearance Number, Expiration Date, and Burden Disclosure Statements will be displayed on the applications.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the Certification for Paperwork Reduction Act Submissions.

Attachments:

1. Consent Form

2. Email Verification and Study Reminders

3. Screener Survey

4. Initial Survey

5. Customer Satisfaction Survey

1 Liu C, White RW, Dumais S. Understanding web browsing behaviors through Weibull analysis of dwell time. Paper presented at: SIGIR '10 Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval; July 19-23, 2010; Geneva, Switzerland.

2 Castiglioni, L., Pforr, K., & Krieger, U. (2008). The effect of incentives on response rates and panel attrition: Results of a controlled experiment. Survey Research Methods, 2(3), 151-158.

Jackle, A., & Lynn, P. (2008). Respondent incentives in a multi-mode panel survey: Cumulative effects on nonresponse and bias. Survey Methodology, 34(1), 105-117.

3 E. Singer, J. Van Hoewyk, and M. P. Maher, “Experiments with Incentives in Telephone Surveys,” Public Opinion Quarterly, Vol. 64, No. 2, Summer 2000, pp. 171-188; A. H. Church, “Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta-Analysis,” Public Opinion Quarterly, Vol. 57, No. 1, Spring 1993, pp. 62-79.

4 W. D. Mosher, W. F. Pratt, and A. P. Duffer, “CAPI, Event Histories and Incentives in the NSFG Cycle 5 Pretest,” American Statistical Association, 1994 Proceedings of the Section on Survey Research Methods, Vol. 1, 1995, pp. 59-63.

SFW Customer Satisfaction Study – Page 7