0925-0046 Support Statement[1]

PRETESTING OF NCI’S COMMUNICATIONS MESSAGES

REQUEST FOR OMB REVIEW

AND SUPPORTING STATEMENT

Project Officer: Nina Goodman

National Cancer Institute

Office of Education and Special Initiatives

6116 Executive Blvd., Suite 400

Rockville, MD 20892-8351

Telephone: (30l) 435-7789

Fax: (30l) 480-3441

E-mail: [email protected]

DATE: August 31, 2006

TO: Reports Clearance Officer, DHHS

Through: Project Clearance Officer, NIH

Project Clearance Officer, NCI

FROM: Nina Goodman, Project Officer

National Cancer Institute (NCI)

SUBJ: Pretesting of NCI’s Communication Messages

This is a request for OMB to approve the attached extension request for “Prestesting of NCI’s Communications Messages” (current OMB #0925-0046; expiration date 10/31/06). As part of NCI’s mandate from Congress to disseminate information on cancer research, detection, prevention, and treatment, the NCI develops a wide variety of messages and materials. Testing these messages and materials assesses their potential effectiveness in reaching and communicating with their intended audience while they are still in the developmental stage and can be revised. The pretesting process thus contributes to maximizing NCI’s limited dollar resources for information dissemination and education.

Approval is requested for up to 30 pretests annually using such methods as individual in-depth interviews, focus groups, intercept interviews, self-administered questionnaires, gatekeeper reviews, and omnibus telephone surveys. The content, timing, and number of respondents to be included in each pretest will vary, depending on the nature of the message/material being tested, the methodology selected, and the target audiences. Total annual respondent burden is calculated at 2009.12 hours.

Please feel free to call me if there are any questions about this submission. Thank you in advance for your consideration.

Nina Goodman, MHS

Office of Education and Special Initiatives (OESI)

National Cancer Institute

Table of Contents

A. Justification 1

A.1. Circumstances Making the Collection of Information Necessary 1

A.2. Purpose and Use of the Information 3

A.3. Use of Information Technology and Burden Reduction 15

A.4. Efforts to Identify Duplication and Use of Similar Information 16

A.5. Impact on Small Business or Other Small Entities 17

A.6. Consequence of Collecting the Information Less Frequently 17

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 17

A.8. Comments in Response to the Federal Register Notice and Efforts to

Consult Outside Agency 18

A.9. Explanation of Any Payment or Gift to Respondents 19

A.10. Assurance of Confidentiality Provided to Respondents 20

A.11. Justification for Sensitive Questions 21

A.12. Estimates of Hour Burden Including Annualized Hourly Costs 22

A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers 24

A.14. Annualized Cost to the Federal Government 25

A.15. Explanation for Program Changes or Adjustments 25

A.16. Plans for Tabulation and Publication and Project Time Schedule 25

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate 27

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions 27

B. Collection of Information Employing Statistical Methods 28

C. Attachments 33

1. NCI Legislative Authority: 42 USC 285a-2

2. Listing of Data Collection Instruments Used During Clearance Period November 1, 2003 Through October 31, 2006

3. Online Survey to Help Inform Changes to the NCI Cancer Bulletin

4. Follow-Up Telephone Interview to Help Inform Changes to the NCI Cancer Bulletin

5. Formative Research to Guide Development of the Cancer Biomedical Informatics Grid (caBIG)

6. Online Survey to Assess Usefulness of and Inform Changes to NCI’s Cancer Progress Report

7. Survey to Determine User Preferences in Development of HINTS One-Page Summaries

8. Formative Research to Assess Intermediate Outcomes of NCI’s “Plan, Link, Act, and Network with Evidence-Based Tools” (PLANET) Resource

9. Formative Research to Assess Non-Government Funding Opportunities for Complementary and Alternative Medicine (CAM) Cancer Research

10. Telephone Questionnaire to Assess Promotional Efforts of the Body & Soul: A Celebration of Healthy Eating & Living Program Guide

11. Formative Research to Assess Public Attitudes and Behaviors Regarding Fruit and Vegetable Consumption

12. National Telephone Survey of African American Males on Their Knowledge, Attitudes, and Practices Regarding Fruit and Vegetable Consumption

13. Formative Research to Assess How Facing Forward: Life After Cancer Treatment Impacts Knowledge, Attitudes, and Practice of Cancer Patients’ Follow-Up Care

14. Mail Questionnaire to Members of the Vietnamese American Medical Association to Understand Barriers to Cervical Cancer Screening Among Vietnamese Women

15. Formative Research to Assess Factors Affecting the Translation of Research into Cancer Care Practice

16. Survey of Bureau of Primary Health Care (BPHC) Staff to Assess Factors that Contribute to Successful Dissemination of Planned Cancer Screening

17. Baseline and Follow-Up Survey to Determine Awareness Levels and Communications Preferences of Advocates Regarding Colorectal Cancer Clinical Trials

18. Formative Research to Further Develop and Improve NCI’s Consumer Advocate in Research and Related Activities (CARRA) Program

19. Formative Research to Understand the Needs, Attitudes, and Beliefs of NCI’s Consumer Advocates in Research and Related Activities (CARRA) Members

20. 60-Day Federal Register Notice

21. Statement from NIH Privacy Act Officer

22. Statement of Exemption from 45 CFR 46

SUPPORTING STATEMENT

A. JUSTIFICATION

A.1. Circumstances Making the Collection of Information Necessary

The National Cancer Institute (NCI) is the Federal Government's principal agency for research on cancer cause, prevention, detection, diagnosis, treatment, and rehabilitation, and for the dissemination of information for the control of cancer. Current authorization for NCI's education and information dissemination activities is contained in 42 USC 285a-2. (Attachment 1).

Several offices with the NCI¹ provide communications expertise within the Institute and between NCI and a variety of organizations and audiences, including Congress, other executive agencies, state and local governments, scientific and medical communities and institutions, voluntary groups, the press, the general public, and cancer patients.

Information programs within NCI create and use a variety of media including print (for example, brochures, posters, fact sheets, information kits), broadcast (for example, public service announcements and video news releases), and electronic formats (for example, Internet, listservs, CD-roms), as well as direct response (Cancer Information Service) to inform and educate the public and health profes­sionals about cancer. Production of these materials is the major way that the Insti­tute relays messages to the audienc­es it is mandated to reach.

To ensure that health messages have the potential to be received, under­stood, and accepted by those for whom they are intended, NCI communications offices employ formative evaluation. This type of evaluation involves 1) assessing audience knowledge, attitudes, behaviors and other characteristics for the plann­ing/development of health messages, education products, communication strategies, and public information programs; and 2) pretesting these health messages, products, strategies, and program components while they are in developmental form to assess audience comprehen­sion, reactions, and perceptions. The informa­tion obtained from audience research and pretesting can lead to improve­ments in materi­als and strategies while revisions are still affordable and possi­ble. By maximiz­ing the effective­ness of these messages and strategies for reaching targeted audienc­es, the frequen­cy with which publica­tions, products, and programs need to be modified is reduced.

Because NCI offices such as the Office of Education and Special Initiatives (OESI) and Office of Communications (OC) are NCI's chief resources for cancer education and outreach, they have a respon­sibility to both the Institute and its many constitu­encies to provide credible cancer information messages and services. The complexity of these messages will only continue to increase as scientific progress in cancer continues its exponential growth. Unless NCI is able to empiri­cal­ly show that its messages and materials have the potential to be understood­ and useful to their target audiences, it will be unable to carry out its mandate effectively.

Formative (pretest) evaluation is an activity that NCI communication offices ­perform on almost all its print pieces, broadcast products, and informa­tion­al services and messages in order to maximize their usefulness. They are conducted on a small scale and focus on potential effectiveness with specific target audiences. Attach­ment 2 contains a listing of data collection instruments that illustrate different research methodologies used during the clearance period starting November 1, 2003, and ending October 3l, 2006. This listing also displays the number of respondents in each pretest activity, the burden per respondent, and the total burden.

Approval is requested for 30 pretests using methods described in section B with respondents from audiences targeted by messages to be developed by NCI. The total number of respondent burden hours will not exceed 2,009.12 annually.

A.2. Purpose and Use of the Information

Evaluation is an integral part of NCI’s overall plan for its communications strategies. Pretesting of print and broad­cast products and information services assesses the potential effectiveness of these products while they are still in the developmental stage. Later on, analysis of existing data systems that NCI supplements or uses, such as the National Health Interview Survey (NHIS) Cancer Supple­ments conducted by the National Center for Health Statistics, enables NCI’s communication offices to evaluate the long-term effectiveness of its cancer education programs.

The formative evaluation process is used to determine whether a draft message or message concept is effective in reaching and communicating with its audience. Pretest­ing involves presentation of draft messages designed to convey specific information to a sample of the audience for whom the materi­als are intended. These respon­dents are asked to give their reaction to the messages through either individu­al or group interviews. Respon­dents assess the materials in the following areas:

• Attention - Do the messages attract and/or hold the audience's attention? For example, if they were to see a particu­lar bro­chure in their doctor's office, would they pick it up and look at it?

• Comprehension - Are the messages or main points clearly understood? Does the main theme of the mes­sage get across to the audience? Is the language clear? For example, a respondent may be asked whether a fact sheet clearly explains the limitations and risks of mammography, along with the benefit of finding breast cancer early.

• Personal Relevance and Self-efficacy - Do members of the target audience perceive the message as personally relevant? For example, do the respon­dents see that the message in the public service announce­ment (PSA) applies to them -- that the information is impor­tant to them? Do the respondents see themselves as capable of acting on the message?

• Believability - Is the message and/or its source perceived as credible? For example, does the respondent believe the message in a PSA that women continue to need regular mammograms and Pap tests even as they get older? Is the person delivering the message credible?

• Acceptability - Is there any­thing in the message that may be offensive or unacceptable to the target audience? For example, does the respondent react negatively to a print message that warns of the dangers of smoking and drink­ing? Is the piece culturally sensitive?

• Accessibility - Will members of the target audience be able to find the information or message with relative ease? For example, do information seekers know how and where to find the most accurate, up-to-date information on the risks of taking hormone therapy for menopausal transition and afterwards?

• Usability - How likely is the respondent to use the information in the format provided? For example, is the content provided on a web site presented in a well-organized, logical, and user-friendly way?

• Behavioral Intent - Do respondents think they will take action as a result of seeing/hearing the message? For example, does an educational piece containing easy hints for eating five or more fruits and vegetables a day motivate the target audience to think more about, or plan to eat more fruits and vegetables?

Respondents' input and reactions to each of these areas provide insight into how the audiences for these messages may react and how the messages should be formulated or revised to communicate most effectively­.

Other information gathered on respondents regarding gender, age, socioeconomic level, race/ethnicity, and family medical history provides a basis for evaluating whether the messages may be perceived differently by different segments of the audience. For example, selected age groups may find a particular brochure or message on cancer prevention more relevant than other age groups.

Systematic formative evaluation has been widely adopted by health educa­tion program planners as an integral step in the development and targeted dissemination of messages and materi­als. Through pretesting NCI is able to:

• Understand characteristics of the target audience--its attitudes, beliefs, and behaviors--and use these in the development of effective communications tools;

• Design messages and select formats that have the greatest potential to influence the target audience’s attitudes and behavior in a favorable way;

• Determine the best promotion and distribution channels to reach the target audience with appropriate messages; and

• Expend limited program resource dollars wisely and effectively.

Results of past pretesting efforts have been instrumental in helping NCI carry out its legislative mandate. A few examples of formative/pretesting efforts which have resulted in the development and refinement of NCI messages, materi­als, strate­gies, and formats are described below. Please note that all the formative/pretesting efforts described on the next several pages were conducted under OMB No. 0925-0046, expiring 10/31/06.

One of the first projects conducted under the current clearance was for the NCI’s Office of the Director (OD) and Office of Communications (OC), in order to pretest the NCI Cancer Bulletin. Since January 2004, the OC has produced a weekly electronic publication, the NCI Cancer Bulletin, to provide useful, timely information to the cancer community about NCI, its programs and policies, and the latest innovations in cancer research. Originally started as a pilot, the weekly distribution of this cancer communication vehicle achieved growing success. In May 2004, NCI wished to determine whether the NCI Cancer Bulletin was meeting subscribers’ needs in terms of the usefulness of content and frequency of delivery in order to establish it as a permanent resource rather than just a pilot. To this end, NCI conducted an online survey and in-depth telephone interviews (Attachments 3 and 4) with Bulletin subscribers in May 2004. The results of the survey helped the OC improve the Bulletin to best meet readers’ needs, as well as promoted it to permanent project status. Readership and positive feedback has continued to increase as the OC has considered new ways of promoting the resource to expanded audiences outside of the Institute.

Another key initiative that utilized formative research was the NCI’s cancer Biomedical Informatics Grid (caBIG), which is an informatics infrastructure designed to connect teams of cancer and biomedical researchers, thus enabling them to better develop and share tools and data in an open environment with common standards. The multimillion-dollar caBIG initiative is a priority project at the NCI and will have tremendous impact for the cancer research community. NCI’s Center for Bioinformatics (NCICB), who is heading the project, felt that formative research of this nature was crucial, since the success of the initiative hinges largely on the interest in and subsequent adoption of caBIG technology by cancer researchers, both at the NCI and in the extramural community. Therefore NCICB commissioned a needs assessment (Attachment 5) of potential users including researchers, administrators, and bioinformatics experts at the Cancer Centers and at the NCI to identify and assess the bioinformatics needs of the cancer research community. The information obtained from this research helped inform the continued development of the final version of caBIG to ensure it best met the needs of its end-users.

Another office that recognized the value of conducting formative/pretesting research for its programs and resources is the NCI’s Division of Cancer Control and Population Science (DCCPS). The DCCPS issues the Cancer Progress Report (CPR), a NCI resource that summarizes our nation’s progress against cancer (http://progressreport.cancer.gov/). A survey (Attachment 6) was sent electronically to potential readers of the report, including health policy makers, members of cancer advocacy organizations, and cancer researchers in order to assess what CPR readers would like to see in the revised edition and to determine the best ways to disseminate the CPR to reach these audiences. The findings assisted NCI in updating and revising the report to best meet the needs of its readers, as well as improve dissemination efforts to ensure that the report is widely distributed among target audiences.

In addition, the DCCPS administers the HINTS Survey (http://dccps.nci.nih.gov:0/hints/index.jsp) every three years to learn more about the public’s use of cancer-related information. DCCPS was interested in developing one-page summaries that provide brief snapshots of the data from HINTS so that those who are interested can get a quick idea of the findings and trends from the survey. In order to develop these “one-pagers”, DCCPS conducted a survey (Attachment 7) with its main users, namely health communicators, health educators, and public affairs professionals, to determine what topic areas of the HINTS survey are of greatest interest to its users and how users like the data to be presented. The results of this formative research allowed DCCPS to decide how to prioritize, streamline, and package the findings into easy-to-read one-page summaries.

Finally, DCCPS has developed Cancer Control PLANET, a web-based repository of evidence-based cancer-control tools that are designed to provide practitioners, researchers, and federal program staff the information they need to implement and evaluate effective cancer control intervention strategies. It was created through a partnership among the National Cancer Institute (NCI), Centers for Disease Control and Prevention (CDC), Agency for Healthcare Research and Quality (AHRQ), Substance Abuse and Mental Health Services Administration (SAMHSA), and American Cancer Society (ACS). It is the joint mission of these agency partners to encourage the translation of research into practice and to facilitate the adoption of evidence-based interventions in public health practice. In order to determine the extent to which the Cancer Control PLANET has achieved its program goals, NCI conducted a web-based survey (Attachment 8) with policy makers, practitioners, social workers, agency members, and other communications professionals who had attended at least one training on the background and potential uses of PLANET. This formative research focused on process and intermediate outcomes, and helped to lay the groundwork for a comprehensive long-term outcome evaluation for NCI to conduct in the next several years.

Formative research was also used to assist the Office of Cancer Complementary and Alternative Medicine (OCCAM), a relatively new office at NCI, better understand some of the health professional audiences they serve. The OCCAM’s goals are to increase the amount of high-quality information and research on the use of CAM and CAM-related approaches for the diagnosis, prevention, and treatment of cancer and cancer-related symptoms and treatment side effects, and to provide the NCI with an interface with the general public, health practitioner communities, and research communities regarding CAM cancer issues. A survey (Attachment 9) was conducted that allowed OCCAM to assess how organizations external to the Federal government offer funding for cancer CAM research, as well as develop a directory of these types of non-governmental organizations that offer funding for cancer CAM research. From previous research, OCCAM has learned that one of the hurdles that many cancer CAM researchers encounter is the difficulty of obtaining research funding for foundational or exploratory research. Often, researchers must obtain their initial funding through non-Federal sources, so that they can demonstrate proof of concept, which can be a pre-condition of obtaining Federal funds. The information collected from this formative research provided OCCAM with an understanding of how non-Federal funding sources operate, with hopes to share this information with cancer CAM researchers so that they can better target the funding sources that are most closely aligned with their research objectives.

Beyond working closely with various offices within the Institute, pretesting/formative research was conducted with some long-standing health promotion programs, including NCI’s Body & Soul program. Body & Soul is a faith-based initiative grounded on over ten years of evidence-based research that shows the unique influence of the church in helping its members adopt healthier lifestyles and behaviors. More specifically, it is a wellness program designed specifically for African-American churches to help members increase the number of servings of fruits and vegetables they eat to reduce their risk for diet related chronic diseases. NCI utilized a questionnaire (Attachment 10) to determine what types of organizations are ordering the Body & Soul: A Celebration of Healthy Eating & Living Program Guide. The instrument was administered to callers to the NCI’s Publication Ordering Service (POS), part of the Cancer Information Service (CIS), who were specifically calling to place an order for copies of the Body & Soul Program Guide. This information assisted the program planners by helping them understand exactly who was ordering the guide and how it is being disseminated nationwide, so they could modify their outreach and promotional efforts in order to reach the most people. Ultimately, this formative research helped NCI understand how to best promote and disseminate the guide in the future in order to most effectively reach it=s target audiences.

Another health promotion program, the “5-to-9 A Day for Better Health Program,” now in its 15th year, has consistently relied on pretesting to develop and refine its messages urging Americans to eat at least five servings of fruits and vegetables a day. During the current clearance period, oversight of this program transitioned from the NCI to the CDC. Though continuing collaboration with the CDC, several formative research activities helped ensure that the national campaign was transferred seamlessly and continued to achieve results. First, a mail survey (Attachment 11) was designed to inform program planning for the revised program under the CDC. More specifically, the research will allowed for exploration of the public’s understanding of the new “cup” portion sizes (consistent with the 2005 Dietary Guidelines) that are being used to describe optimal fruit and vegetable consumption measures. In addition, the research was designed to explore social and psychological determinants of fruit and vegetable consumption, both previously established ones such as self-efficacy, benefits, and barriers to fruit and vegetable consumption, as well as new predictors/constructs that have not yet been examined such as vegetarianism, taste preferences, and environmental level variables such as shopping patterns. The results of this survey assisted with the development of subsequent in-depth research, with the ultimate goal of helping to further the development of standardized measures of consumer knowledge, attitudes, and behaviors regarding the consumption of fruits and vegetables.

Following a thorough 2002 literature review of the health status and eating habits of various segments of the U.S. adult population, it was decided to shift the focus of the national campaign from the public at large to African American men. NCI conducted a national telephone survey of African American men (Attachment 12) to gain an understanding of their specific attitudes, knowledge, and practices regarding their intake of fruits and vegetables as part of their diet. Findings from this formative research will helped NCI’s “5 to 9-A-Day for Better Health Program” to refine an information campaign targeting African American men ages 21 and older to increase their fruit and vegetable consumption.

Several pretest efforts focused on better understanding public, patient, and survivor issues from the caregiver perspective as well as that of the patient. For the Office of Education and Special Initiatives (OESI), questionnaires (Attachment 13) were administered to help NCI ascertain whether the information in the Facing Forward: Life After Cancer Treatment booklet impacted a patient’s knowledge, attitudes, and practice with respect to cancer follow-up. The participants of the surveys were adults completing or having recently completed cancer treatment at four predetermined NCI-designated Cancer Centers. The primary purpose of this formative research was to help NCI determine what the immediate needs of cancer patients and survivors are following cancer treatment, and if the Facing Forward resource plays a role in their post-treatment decisions and care.

In addition, the OESI was interested in learning more about barriers to cervical cancer screening among Vietnamese women and to assess what types of educational materials might be useful to help inform women about the importance of screening and early detection (Attachment 14). NCI data shows that cervical cancer is the number one incident cancer in Vietnamese women, and incidence rates are five times higher among Vietnamese American women than White women. Screening and early detection is important as cervical cancer can be cured in almost 100% of cases if it is treated early. Findings from this formative research helped guide the development of NCI educational materials, written in Vietnamese, designed to help motivate women to receive routine Pap tests.

Besides focusing on caregivers, patients, and the public, OC directed a number of pretest efforts toward obtaining feedback from health professionals – an important intermediary group in the endeavor to provide information to NCI’s end users. In order to assess attitudes and beliefs of cancer practitioners about how clinical research findings are translated into clinical practice, NCI mailed a survey (Attachment 15) to oncologists, general practitioners, oncology nurses, nurse practitioners, oncology social workers, and therapeutic radiation oncologists who were members of various cancer-related professional societies which partner with NCI. The primary purpose of the survey was to help guide NCI program planning when developing resources that assist practitioners in bridging cancer research findings into clinical practice methods. More specifically, the survey helped NCI identify key supports and barriers in the implementation of current evidence-based practices, as well as sources of practice-related information that are used by practitioners. Ultimately, this information helped define NCI’s role in developing materials that support the dissemination of evidence-based cancer information into practice.

In another example, the NCI collaborated with the Center for Disease Control and Prevention (CDC) and the Bureau of Primary Health Care (BPHC), which is part of the Health Resources and Services Administration (HRSA) of DHHS, to test a method of teaching and disseminating planned cancer screening. The teaching method was instituted as a pilot project, on a small scale, with 20 community clinics in the U.S. Before disseminating these methods to community clinics across the U.S, NCI and its partners found it necessary to conduct formative research regarding best methods for disseminating/teaching this information. The purpose of this research (Attachment 16) was to collect information from a variety of health care staff at the pilot clinics, including health center directors, financial officers, health care providers, and information systems personnel, to learn how well this teaching method was working, how it could be improved before dissemination on a wide scale, and what factors affected its success. The findings were used to help NCI improve the method by which planned cancer screening will be disseminated to community health centers.

Finally, NCI sent a web-based survey (Attachment 17) to members of colorectal cancer advocacy groups in order to assess their level of knowledge about specific colorectal cancer trials and to determine the best methods to reach these audiences with clinical trials awareness messages. Through a partnership with two specific advocacy groups, the Colon Cancer Alliance (CCA) and the Colorectal Cancer Coalition (C3), NCI was working to raise awareness of colorectal cancer among the public and to support patients with and survivors of colorectal cancer. Part of the mission of these partner organizations is to increase awareness of the various NCI-sponsored colorectal clinical trials that are currently underway. The NCI developed a targeted promotional plan to help raise awareness of colorectal clinical trials among constituents of advocacy groups, and conducted this formative research allow NCI to better understand what types of communication channels should be utilized to reach advocacy constituents with clinical trial messages, with the ultimate goal of improving NCI’s promotional efforts to reach the advocacy community with information about clinical trials.

Working with advocates is a critical component of the NCI’s Office of Liaison Activities, which was established in 1996 in order to help strengthen the Institute's communications and relationships with national advocacy and voluntary organizations who work with consumer advocates, and scientific and professional societies concerned about cancer. The OLA administers a variety of formal and informal programs to ensure that the voices of those who are affected by cancer are included in NCI's priorities, programs, and policies.

One of these programs, the NCI’s Consumer Advocates in Research and Related Activities (CARRA) program provides a “ready and waiting” group of consumer advocates who are available to participate in a wide range of NCI activities, including peer review of cancer research grants and education materials, participating in meetings to provide opinions about NCI research plans and policies, and evaluating patient-oriented research at cancer research centers. There are approximately 200 CARRA members who represent many different cancer types, age groups, and ethnic groups from across the Nation. A survey (Attachment 18) was mailed to CARRA members after completing an activity (as described above), in order to obtain feedback that will be used to further develop and refine the CARRA program. In addition, a second survey (Attachment 19) was sent to CARRA members to determine how relevant the programs activities, communications, and structure are to the CARRA members’ advocacy needs, concerns, and interests. This formative research allowed NCI’s OLA to determine the CARRA members’ attitudes and experiences with the program, with the ultimate goal of improving the program to ensure it contributes to the overall NCI mission.

A.3. Use of Information Technology and Burden Reduction

The information will be collected through the use of one-on-one interviews, group interviews, or self-administered questionnaires, depending upon the target audience being questioned and the subject matter being addressed. Improved technol­ogy in the collection and processing of data will be used to reduce respondent burden and make processing maximally efficient. For example, telephone focus groups will be convened when geographic diversity is important and participants come from hard-to-recruit populations, such as physicians or Native Americans. When telephone interviews are used, computer-assisted telephone interviewing (CATI) will be employed whenever possible. For self-administered questionnaires, closed-ended questions (for example, multiple-choice items or Likert scales) and machine-readable answer sheets will be used when feasible. Transmission of data collection instruments and responses by electronic mail or facsimile will be utilized as appropriate (for example, with intermediary audiences such as NCI’s Cancer Information Service or Public Affairs Network).

As computer technology has continued to improve and become more widespread, opportunities to pretest messages on the Internet using either web site questionnaires or on-line focus groups with Internet users have increased. Using computer-assisted information technology to transmit data collection instruments and/or collect responses will continue to reduce the burden on respondents; for example, respondents can access and respond to data collection requests at a time and place that is convenient to them, eliminating the need to travel for in-person or group interviews. Wherever possible, NCI will make use of web- or computer-based data collection methods.

A.4. Efforts to Identify Duplication and Use of Similar Information

The general areas in which information needs to be gathered (as described in A.2. above - attention, comprehension, etc.) to pretest effective cancer messages (brochures, PSAs, media campaigns, etc.) are generally similar from pretest to pretest. However, the specific questions that are asked of respondents will differ with the message content, audience targeted, and medium of the message.

As each new message, strategy, or product is developed, NCI reviews existing litera­ture and data bases, including pretesting reports on existing messages and materials, and consults with outside experts to evaluate available informa­tion on similar messages with comparable audienc­es. However, since the cancer field is so diverse and complex, and each message is essentially different, new data collection instruments generally must be prepared for each pretest.

A.5. Impact on Small Businesses or Other Small Entities

Physicians and other health care providers may sometimes be the target audience for NCI information materials. When testing of these materials is required, NCI generally works through established medical and professional societies to gain access to the audience, and to obtain feedback on its instruments and data collection plans. As a result of this contact, NCI is able to minimize the placement of addition­al burden on health care providers.

A.6. Consequence of Collecting the Information Less Frequently

Information will be collected only one time for each print, broadcast, or electronic message, product or strategy tested. Respondents will not be recontacted.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

NCI communications offices recognize the need to collect information in a manner that places minimal burden on each respondent. Therefore, when NCI requires responses to a self-administered written questionnaire in less than 30 days, receipt of the questionnaire is generally preceded by advance notification to respondents explaining the purpose of the questionnaire, the approximate length of time that the questionnaire will take, and the voluntary nature of participation. All efforts are made to keep such questionnaires short and focused.

Because NCI’s pretesting activities are primarily qualitative in nature, the results are not generalizable to the population at large or to the particular target audience under study. However, the nature of pretesting is such that generalizability is not a critical feature; the emphasis is on obtaining timely, useful information that can be fed back into the development of new messages or materials or the revision of existing ones.

There are no other special circumstances.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside

Agency

Attached is a copy of the 60-day notice which appeared in the Federal Register on August 14, 2006 (Volume 71, Number 156, p. 46486), soliciting comments on the requested extension to NCI’s current data collection project (Attachment 20). No public comments were received in response to that notice.

NCI, along with other Public Health Service agencies, has been a leader in the development of methods for developing, testing, and dissemi­nating health information. The work of many advisers over many years has brought us to where we are today. A number of outside health communications experts were consulted to review the plans contained herein for program development research and evaluation of NCI communications programs and their comments and suggestions have been incorporated into these data collection plans.

Although all consumer materials, as well as materials produced for the media, must go through DHHS approval procedures, NCI is not required to coordinate with other Federal agencies in its education, information dissemination, and evalua­tion activities. Nevertheless, NCI has consulted frequently with other agencies in the past and will continue to do so in the future, as appropriate, to help ensure accuracy and consistency, and to avoid duplica­tion of effort. For example, in the past three years NCI has collaborated with the Center for Disease Control and Prevention (CDC) and the Bureau of Primary Health Care (BPHC), which is part of the Health Resources and Services Administration (HRSA) to test a method of teaching and disseminating planned cancer screening; again with the CDC to transfer the “5-to-9 A Day for Better Health Program,” which urges Americans to eat at least five servings of fruits and vegetables a day; with the CDC, Agency for Healthcare Research and Quality (AHRQ), Substance Abuse and Mental Health Services Administration (SAMHSA), and American Cancer Society (ACS) on Cancer Control PLANET, a web-based repository of evidence-based cancer-control tools; and with the Colon Cancer Alliance (CCA) and the Colorectal Cancer Coalition (C3) to raise awareness of colorectal cancer among the public and to support patients with and survivors of colorectal cancer. It should also be noted that many of the pretesting and formative research efforts conducted by NCI and referenced in this submission have been requested and used by outside agencies to inform their own communications activities.

A.9. Explanation of Any Payment or Gift to Respondents

It is standard practice in commercial market research to offer recruited respondents some form of remuneration for the time they spend engaged in a pretest activity. Small amounts of money (where appropriate, $17.00 per hour), a free meal or snack scheduled around the time of the pretest, and/or remuneration for parking and/or trans­portation are most often used, particularly when recruiting hard-to-reach and minority respondents.

Market research literature suggests that monetary incentives have a strong positive effect on the response rate and no known adverse effect on reliabil­ity. (Response rate and participant objectivity are further encouraged by reminding participants, either orally or in writing, about the importance of providing both negative and positive feedback.) Circumstances, however, do not always require that remunera­tion be given; many audiences including the public, patients, survivors, and health and other professionals often participate gratis because of their interest or involvement in the topic, or as a professional courtesy.

A.10. Assurance of Confidentiality Provided to Respondents

Information provided by respondents will be kept confidential and private, except as otherwise required by law. This will be communicated to respondents by means of introductory letters, explanatory texts on the cover pages of question­naires, scripts read prior to focus groups or telephone interviews, and consent forms. Respondents will also be advised of the following: the nature of the activity; the purpose and use of the data collected; NCI sponsorship; and the fact that participation is voluntary at all times. Because responses are voluntary, respondents will be assured that there will be no penalties if they decide not to respond, either to the information collection as a whole or to any particular questions.

As a further guarantee of confidentiality, all presentation of data in reports will be in aggregate form, with no links to individuals preserved. Reports will be used only for research purposes and for the development of communica­tion messages and educational materi­als.

The NIH Privacy Act Officer will be reviewing the work scope of this proposal to determine whether the Privacy Act is applicable to this data collection. Although some personal information will be collected, data will not be retrieved by personal identifiers (Attachment 21).

Pretesting efforts described in this proposal are also considered exempt from the “Regulations for the Protection of Human Subjects” in accordance with paragraph (b)(3) of 45 CFR Sec. 46.101 (Attachment 22).

A.11. Justification for Sensitive Questions

As mentioned in sections A.2. and A.10. above, some studies require the inclusion of people who match selected characteristics of the target audience that NCI is trying to reach. This sometimes requires asking a question about race/ethnicity, income, education and/or health status on the initial screening question­naire used for recruiting. Potential participants are informed that this is being done to make sure that NCI speaks with the kinds of people for whom its messages are intended. Again, respondents are assured that the informa­tion is voluntary and will be treated as confidential. All information on race/ethnicity will comply fully with the standards of OMB Statistical Policy Directive No. 15, October 1997 (http://www.whitehouse.gov/omb/fedreg/1997standards.html).

Since NCI communications are concerned with the detection, diagnosis, treatment, and prevention of cancer, some projects may involve asking questions about (or discussing) how one perceives his/her own personal risk for serious illness. Fears of cancer and experiences with cancer may also be covered. This information is needed to gain a better understanding of the target audience so that the messages, strategies, and materials designed will be appropriate and sensitive. Questions of this nature, while not as personal as those on sexual behavior or religious beliefs, still require some sensitivity in how they are worded and approached. In face-to-face data collections, questions of this kind are generally asked later in the interview or group discussion when respondents are more comfortable with the interview situation and are more at ease with the interview­er/moderator. As noted in section A.10.,­ participants are informed in advance about the nature of the activity and the voluntary nature of their partici­pation. The interviewer/moderator makes it clear that they do not have to respond to any question that makes them uncomfortable.

Raw data from data collections that include sensitive information (for example, screening questionnaires and audio tapes) are not retained once the data have been extracted and aggregated; nor does the information become part of a system of record containing permanent identifiers that can be used for retrieval.

A.12. Estimates of Hour Burden Including Annualized Hourly Costs

The number of respondents to be included in each new pretest will vary, depending on the nature of the material or message being tested and the target audience. However, for illustrative purposes, Table A.12-1 below provides an example of a distribution of respondents and hours by type of data collec­tion. Time to read, view, or listen to the message being tested is built into the "Hours Per Response" figures. Proposed data collection methodolo­gies are de­scribed in more detail in Section B.

A.12-1 -- Estimates of Hour Burden by Anticipated Data Collection Methods

Note: The burden table below reflects what NCI anticipates would be accomplished over the total 3-year life of the project. (Annual burden, therefore, is one-third of the total figures presented here.)

	Total Number of Respondents	Frequency of Response	Hours Per Response	Total Hours
Individual In-Depth Interviews	600	1	.75	450
Focus Group Interviews	540	1	1.5	810
Intercept Interviews: Central Location	1800	1	.25	450
Intercept Interviews: Telephone	30,000*	1	.08	2400
Self-Administered Questionnaires	1200	1	.25	300
Gatekeeper Reviews	1200	1	.50	600
Omnibus Surveys	6000	1	.17	1020
Totals	41,340			6030

* Brief interviews with CIS callers to test message concepts and strate­gies following their call-in request to the 800 number.

(Note: On an annual basis, the total number of respondents is 13,780; the annual number of responses is 13,780; and the total annual hours are 2009.12, or approximately 2010.)

Table A.12-2 presents the cost to respondents over the 3-year life of the project. Annual cost, therefore, is one-third of the total figures shown.

A.12-2 Cost to Respondents

Type of Respondents	Number of Respondents	Frequency of Response	Hourly Respondent Wage Rate	Cost
General Public	41,196	1	$17.00	$105,050
Physicians	144	1	$67.50	$14,580
			TOTAL	$119,630

­­ The cost to individual respondents who are members of the general public is approximate­ly $2.55 based on the estimate of $17.00/hour and an average respondent burden of .15 hours per respondent. Public respondents will be reim­bursed for their time at the rate of $17 an hour for a cost of $35,017 annually or $105,050 over the 3-year period. This rate of reimbursement is a general­ly accepted one in the market research industry.

While physicians (general practitioners) sometimes­ participate gratis in telephone or self-adminis­tered surveys (time permitting), it is customary to reimburse them at the average rate of $67.50 per hour for taking part in focus groups (http://www.bls.gov/oes/current/oes291062.htm). Assuming one focus group study with physicians per year (6 groups of 8 physicians each for 1 ½ hours), the annual cost would be $4,860.

A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers

There are no capital or start-up costs to the data collection efforts requested; nor are there any costs associated with operation, maintenance or purchase of services.

A.14. Annualized Cost to the Federal Government

The total annual cost to the Federal Government will be approximately $374,620 (or $1,123,860 over 3 years). This estimate is based on annual performance of up to: 4 in-depth inter­view studies at $13,860 each ($55,440); 3 focus group studies at $35,280 each ($105,840); 3 central location interview studies at $11,340 each ($34,020); 10 telephone inter­view studies at $3,780 each ($37,800); 4 self-adminis­tered ques­tion­naire stud­ies at $5670 each ($22,680); 4 gatekeeper reviews at $11,340 each ($45,360); and 2 telephone omnibus surveys at $15,120 each ($30,240). These figures include the costs of study design, facility rental (e.g., for focus groups), data collec­tion, analysis, and report/publication writing. This estimate also includes monitoring by the Government Project Officer and involvement by NCI's Senior Analyst, projected to be about 1000 hours of effort a year. Given an NCI personnel cost of $43.24 per hour, $43,240 would be spent annually on Government staff salaries (or $129,720 over the 3-year period).

A.15. Explanation for Program Changes or Adjustments

This information collection is an extension of a currently approved collection, OMB No. 0925-0046, which expires on October 3l, 2006. While the number of respondents and hours remain the same, there is a small increase in annualized cost due to increases in labor costs associated with carrying out data collections of this kind­­.

A.16. Plans for Tabulation and Publication and Project Time Schedule

The process for developing the analytical plan for the pretest is similar to that used in any formal evaluation. Evaluation staff review the material to be pretested, discuss the objectives with the individuals responsible for developing the materials, determine the analytic questions to be addressed in the pretest, and then prepare the pretest procedures, instruments, and data analysis plan. The analyses conducted for each pretest will be determined by the objectives of the pretest, the messages being pretested, and the audience for the messages. Specifics of the analyses cannot be determined until the messages to be pretested are prepared.

Techniques include primarily qualitative analyses (for example, content analysis for in-depth interviews), although some results such as those from central location intercept interviews are summarized quantitatively using descriptive statistics. No complex analytic techniques are used.

While the primary purpose of a pretest is to provide information to the developers of the messages for the purpose of improving them, NCI makes pretest results available to a variety of health program planners at Government agencies, voluntary organizations, health professional organiza­tions, and medical institutions.

In addition, NCI presents the findings of its pretest work at profes­sional associations, including the American Public Health Association. Society for Public Health Education, and the Association for Health Care Research, and publishes its findings in profes­sional journals such as the Journal of the American Medical Women’s Association, Journal of the National Cancer Institute (News Section), Journal of Women’s Health and Gender-based Medicine, and Women’s Health. Formative research conducted by OC is also sometimes summarized in news-related publications such as the NIH Record.

The specific messages that will be pretested and the timing of these messages are not known at this time. However, as indicated in section A.1. above, approximately 30 pretest studies are planned. While the pretesting period varies somewhat depending on the complexity of the testing and number of respondents required, the typical pretest will require approxi­mately 8 weeks from initial design to preparation of the report of pretest findings. A schedule for a typical pretest is shown below:

A.16-1 Project Time Schedule

Activity Time Schedule

Initial review of materials 1 week after OMB approval

Preparation of pretest design 2 weeks after OMB approval

Review of design 3 weeks after OMB approval

Collection of data 5 weeks after OMB approval

Analysis of data 7 weeks after OMB approval

Report on pretest 8 weeks after OMB approval

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

NCI will continue displaying the OMB control number and expiration date in the upper right-hand corner of all data collection instruments.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

NCI is in full compliance with the provisions contained within the Certification for Paperwork Reduction Act Submissions.

ATTACHMENTS

1. NCI Legislative Authority: 42 USC 285a-2

2. Listing of Data Collection Instruments Used During Clearance Period November 1, 2003 Through October 31, 2006

3. Online Survey to Help Inform Changes to the NCI Cancer Bulletin

4. Follow-Up Telephone Interview to Help Inform Changes to the NCI Cancer Bulletin

5. Formative Research to Guide Development of the Cancer Biomedical Informatics Grid (caBIG)

6. Online Survey to Assess Usefulness of and Inform Changes to NCI’s Cancer Progress Report

7. Survey to Determine User Preferences in Development of HINTS One-Page Summaries

8. Formative Research to Assess Intermediate Outcomes of NCI’s “Plan, Link, Act, and Network with Evidence-Based Tools” (PLANET) Resource

9. Formative Research to Assess Non-Government Funding Opportunities for Complementary and Alternative Medicine (CAM) Cancer Research

10. Telephone Questionnaire to Assess Promotional Efforts of the Body & Soul: A Celebration of Healthy Eating & Living Program Guide

11. Formative Research to Assess Public Attitudes and Behaviors Regarding Fruit and Vegetable Consumption

12. National Telephone Survey of African American Males on Their Knowledge, Attitudes, and Practices Regarding Fruit and Vegetable Consumption

13. Formative Research to Assess How Facing Forward: Life After Cancer Treatment Impacts Knowledge, Attitudes, and Practice of Cancer Patients’ Follow-Up Care

14. Mail Questionnaire to Members of the Vietnamese American Medical Association to Understand Barriers to Cervical Cancer Screening Among Vietnamese Women

15. Formative Research to Assess Factors Affecting the Translation of Research into Cancer Care Practice

16. Survey of Bureau of Primary Health Care (BPHC) Staff to Assess Factors that Contribute to Successful Dissemination of Planned Cancer Screening

17. Baseline and Follow-Up Survey to Determine Awareness Levels and Communications Preferences of Advocates Regarding Colorectal Cancer Clinical Trials

18. Formative Research to Further Develop and Improve NCI’s Consumer Advocate in Research and Related Activities (CARRA) Program

19. Formative Research to Understand the Needs, Attitudes, and Beliefs of NCI’s Consumer Advocates in Research and Related Activities (CARRA) Members

20. 60-Day Federal Register Notice

21. Statement from NIH Privacy Act Officer

22. Statement of Exemption from 45 CFR 46

1 ¹ Please note that oversight of this package formally resided within the NCI’s Office of Communications (OC), and is now residing within the Office of Education and Special Initiatives (OESI). The main users of this package are the OC and OESI, along with other NCI offices conducting communications research. Since this package is shared within the Institute, the previous name “Pretesting of NCI’s Office of Communications Messages” was modified to exclude that office (OC) so as to provide a more accurate description of how the package meets multiple communications research needs of the Institute.