#2 REVISED Justification memo Cancer evol

Date: August 6, 2010

(Originally submitted June 24, 2010)

To: Office of Management and Budget (OMB)

Through: Seleda Perryman, DHHS Report Clearance Officer

Mikia Currie, NIH Program Analyst, OPERA

Vivian Horovitch-Kelley, NCI OMB Project Clearance Liaison OMAA

From: Nina Goodman, Project Officer

Office of Communications and Education (OCE)

National Cancer Institute/NIH

Subject: Generic Sub-Study, “NCI Cancer.gov Evolution - User Focus Groups and Triads” (OMB No. 0925-0046-02, Expiration Date 2/28/13)

The National Cancer Institute (NCI) submits for OMB review the proposed NCI research project, “NCI Cancer.gov Evolution - User Focus Groups and Triads.” As a world leader in cancer research, NCI’s Web site is a primary means of communication and education to its major audiences. The Cancer.gov Evolution process will provide incremental improvement for all stakeholders, and foster conversation and collaboration across the cancer continuum.

The research objectives guiding this sub-study are to learn more about Cancer.gov user preferences related to concept and design in order to 1) inform the Evolution process and 2) ensure meeting the needs of a variety of audiences.

Background on Project

As part of its mission to use effective methods to reach its diverse audiences with the latest, evidence-based cancer information, the National Cancer Institute’s Office of Communications and Education (OCE) has embarked upon an effort to “re-imagine” Cancer.gov. Although this effort is being led by OCE, many across the Institute and throughout the government will be involved. Groups such as the NCI Web Council, which includes representatives from a cross-section of the Institute, NIH, and other government agencies, play an integral role in providing vision, direction, and expertise.

Rather than a one-time re-design, this effort is seen as an “evolution” that will take place in phased releases, be driven by dynamic user-centered design, and be continuously validated by subject matter experts. Through this effort, OCE will develop concepts and recommendations to deliver a multi-channel vision for Cancer.gov to better fulfill the needs of its users, focusing on several primary audience groups, including: patients, friends, and family; health professionals; and researchers. Through numerous feedback mechanisms, these audiences will help provide input for the next generation of Cancer.gov.

In tandem with the Cancer.gov Evolution project, OCE is leading an NCI web strategic planning effort. Cross-disciplinary teams have been created to help advise in the development of a 3-5 year Web Strategic Management Plan. The Web Strategic Management Plan will include an assessment and recommendations in ten key areas—ranging from guiding principles and formalization of authority, to web standards and strategic business metrics.

Explicitly built in to the Evolution process is user-research. As the Evolution process seeks to develop and implement improved Web content, it is critical to conduct research with key audiences, including: 1) healthcare professionals, 2) researchers, 3) advocates, and 4) current and former cancer patients, friends, and family (PFF), in order to assess their needs.

The proposed data collection will inform the development of the new content and design for targeted Web pages to meet the needs of healthcare providers, cancer researchers, advocates, and the general population with personal cancer experience. The proposed methodologies for this data collection are focus groups and triads.

Background Information on Focus Groups

Focus groups, or group interviews, are used to obtain insights into target audience perceptions, beliefs, and attitudes in the early stages of the communication process (i.e., in concept, strategy and materials development).¹ Focus groups are usually composed of 8 - 10 people who have characteristics similar to the target audience or subgroups of the target audience. The groups are conducted by a professional moderator who keeps the session on track while allowing respondents to talk openly and spontaneously. The moderator uses a loosely structured discussion outline, which allows him/her to change direction as the discussion unfolds and new topics emerge. Focus groups are valuable in exploring consumer reactions to design and message concepts before additional resources are put into their development.

Background Information on Triads

A triad is a form of group interviewing involving three participants. It offers many of the advantages of a focus group or group interview as well as some of those of an in-depth interview. Triad participants may be recruited to form a homogeneous group. As with the focus group, triads are facilitated by a professional moderator using a loosely structured discussion outline.

Proposed Research: Focus Groups and Triads with Potential Cancer.gov Users

NCI proposes using focus groups to conduct formative qualitative research with cancer patients, friends and family. Focus groups will consist of four in-person groups of 6 to 8 participants in two US cities. These groups will consist of current and former cancer patients, close family members, and close friends of cancer patients who are potential users of the Cancer.gov Web site.

	Washington DC - NCI	2nd Site – Chicago
# of Focus Groups (# of participants)	2 (16)	2 (16)

NCI proposes the use of triad discussions with healthcare professionals (HCPs), cancer researchers, and advocates to test designs and concepts for the Cancer.gov web pages targeted for these audiences. All triads will be conducted by telephone. Four triads will be conducted with each of two types of HCPs. One group of HCPs will be physicians, including family physicians and internists. These doctors use the Cancer.gov Web site on an as-needed basis. The second group of HCPs who will participate in triad discussions will be oncology nurses. These nurses are frequently engaged in patient education and delivering information to cancer patients. Additionally, four triads will be conducted with staff in cancer-related advocacy organizations; these organizations frequently act as information intermediaries or honest brokers for cancer patients, friends and family.

NCI will also use triad discussions to explore how Cancer.gov can meet the information needs of new cancer researchers. These new researchers are defined as researchers with one to three years of post-doctoral experience. Based on previous formative research, they are viewed as being potentially different from more established cancer researchers who have developed networks of collaborators and are most interested in streamlined approaches to identifying and applying for funding opportunities.

	HCP – Physicians	HCP – Oncology Nurses	Advocates	Researchers – New
# of Triads (# of participants)	4 (12)	4 (12)	4 (12)	4 (12)

Focus group and triad participants will participate in discussions that will address the following areas:

• Design, design sub-elements and concepts that are most useful in accomplishing goals

• What the design says to them about NCI and Cancer.gov

• Most appealing design and concepts

• Most memorable part of a design

• General zoning, colors and scale of the design

• Usefulness of ideas for each audience.

Focus group and triad moderator guides (Attachments 1 and 2) will be used to help focus the discussions. These guides are attached.

Proposed Focus Group and Triad Research Study Design, Methodology, and Limitations

NCI will use a contractor, AED, to conduct data collection activities. Four focus groups will be divided between two locations: Washington, DC (at NCI) and Chicago, IL. Four triad discussions will be held with each of cancer researchers and advocates while 8 triad discussions are proposed for healthcare professionals.

Focus groups participants will be screened on several key dimensions. Cancer patients will include those who currently have or have had cancer within the past 5 years; friends and families will be screened to assure that they are friends or family of a current cancer patient or a person with a cancer diagnosis within the past five years (Attachment 3). For these focus groups, participants will include a diverse mix of adults aged 30 to 65 and of men and women. The focus groups will be in-person and will use a prepared flip book to explore audience-appropriate designs and concepts for the Evolution of Cancer.gov.

All triads will be conducted by telephone using an electronic conferencing system, such as GoToMeeting. This will permit the participants to review a prepared flip book on-line and react to the different Cancer.gov concepts and designs developed as part of the Evolution project. Due to some researchers working in laboratories where plug-in applications are blocked, screening for triads will only identify respondents as eligible when they have the ability to use the GoToMeeting tool.

Recruiters will use four screening questionnaires (“screeners”) to identify eligible respondents in each category (Attachments #3 for PFF, #4 for HCP, #5 for new researchers, and #6 for advocates). The screeners are carefully thought out so that the questioning process is short, easy to-understand, friendly, and efficient. In addition, the moderator’s guides for the focus groups and triads will be used to ensure that the questions are easy to understand and answer, well-organized, and flow well. The moderators will also be instructed to keep discussions to 90 minutes to limit respondent burden with respect to their time. Participation will be voluntary and respondents will be asked to participate only once.

All respondents will receive modest remuneration at a flat rate. Research on participation in focus groups indicates that, without providing minimal levels of monetary compensation, insufficient numbers of participants will attend and results will not be useful. In order to gauge the remuneration rates typical in today’s market, OCE has contacted 8 focus group facilities to ask what the minimum remuneration amount would be required to achieve a feasible turn-out to conduct the focus groups with the general public. We were cautioned that a remuneration amount of $60 will not be enough given the current market. Here is a sample of responses we received from focus group facilities regarding this issue:

• “The minimum incentive amount that we will accept for a focus group of 90 minutes to 2 hours is $75. This is a standard amount in most major markets across the country, and show rates are greatly jeopardized if the incentive is reduced, and recruiting fees are higher because the refusal rate is higher. We need to ensure that all potential respondents agree to attend, and that the show rate is as high as possible.”

• “We recommend a $75 incentive across the board even though $85 is the typical incentive for daytime groups. These costs are the norm in our market, so it would be extremely difficult to go any lower than $75. We feel that the incentive fits are market, and the time commitment you are asking for. A higher incentive keeps respondents committed to doing the group, and gives some incentive for responding to our calls.”

• “In order to have a good show rate to produce the necessary results for your research we would highly recommend offering $75.00 per person for groups after 5pm and $100.00 per person for groups before 5pm. These are the standard incentive costs for research in the DC area.”

Based on these responses, we are proposing providing a remuneration rate of approximately $75 for focus group participants from the general public, and $150 for those triad participants who are participating in the groups because of their specialization in the health care field. Two of the four triad groups are made up of physicians and persons with technical Ph.D.s (physicians and researchers).  It has been our experience as well as the experience of other researchers that remuneration must be higher to these groups than to members of the general public to participate in formative research.²The same is true of oncology nurses, who are highly trained and are more like physicians than the general public. In fact, research has indicated that it may be necessary, and it is our experience that it is necessary, to remunerate up to $150 to recruit physicians who are tend to be motivated by the money, not the research. Advocates, due to their vested interest, should be able to be recruited with the same remuneration rate as the general public; their remuneration rate for triad participation will therefore be $75.

This remuneration will be provided in order to defray the cost of participation in the focus groups (parking, transportation, child care, etc.) and to provide participants with a nominal compensation. Light refreshments will also be provided to focus group participants.

All data will be collected by the contractor, the Academy for Educational Development (AED), and all personal identifiers will be excluded from the data records. Participation will be strictly voluntary and individual respondents will not be identified. Any necessary identifying or potentially identifying information (e.g., signed consent agreements) will be secured and kept separate from the data records (Attachment 7). Focus group respondents will be assured that neither their participation/non-participation nor any responses to items will have any effect on their eligibility for, or receipt of, services.

All information provided by respondents will be maintained in a confidential manner, unless compelled by law. AED will do the analysis of the data and write the report of the findings. Findings in the report delivered to NCI will be reported in the aggregate and no identifiable individual responses will be provided. NCI and AED’s Institutional Review Board (IRB) Research Integrity Officers will review the research instruments and ensure that all necessary human subject protection procedures are in place.

To prevent the misinterpretation of the data, particularly the perception that the results are generalizable, NCI will take steps to ensure that the research findings are not taken out of context, misrepresented, or misused. NCI will include the purpose and formative nature of the sub-study, as well as the qualitative research methodology along with its limitations, in any presentation of data and results generated from this sub-study.

Participant Burden

There will be a maximum of 80 respondents (four focus groups with eight or fewer respondents per group and 12 triads with three participants per triad). For each focus group, up to 10 individuals will be recruited and, based on prior focus group experience; we realistically expect six to eight individual to show for the group. However, to ensure we accurately reflect a maximum burden potential, we are estimating the burden total on the assumption that 8 will attend each group. For triads, based on prior experience, HCPs, advocates, and researchers will not be over recruited - three participants will be recruited for each triad scheduled. An average total participation time will be 65 minutes for triads and 95 minutes for focus groups; five minutes of each reflects the recruitment time. This culminates in a maximum total annual hour burden of approximately 103 hours.

Estimates of Hour Burden
Types of Respondents	Number of Respondents	Frequency of Response	Average Response Time (Hours)	Annual Hour Burden
HCP-General Practitioners	12	1	1.083	12.996
HCP-Oncology Nurses	12	1	1.083	12.996
Advocates	12	1	1.083	12.996
Medical Researchers	12	1	1.083	12.996
Patients, Families, and Friends	32	1	1.583	50.656
TOTAL	80	--	--	102.64

The full generic study, approved on February 10, 2010, requested a total of 7,050 burden hours. There has been one previous sub-studies approved by OMB under this umbrella submission, totaling 917 burden hours requested to date. Approval by OMB of this sub-study would bring the total burden hour requested to date for 0925-0046 to approximately1020, which is approximately 14.5% of the total burden hours allowed.

Thank you for your consideration of this proposed sub-study 0925-0046-02. Please feel free to contact me at 301-435-7789 if you have any questions.

Attachments: 1) Moderator’s guide for focus groups

2) Moderator’s guide for triads

3) Recruitment screener for patients, friends and family

4) Recruitment screener for health care professional triads

5) Recruitment screener for new researcher triads

6) Recruitment screener for advocate triads

7) Consent form

1 Krueger, R.A., (1994). Focus groups: A practical guide for applied research, (2nd ed.) Thousand Oaks, CA: Sage Publications.

2 Krueger, R.A., (1994). Focus groups: A practical guide for applied research, (2nd ed.) Thousand Oaks, CA: Sage Publications, p. 92.

6