Accrual CT Justif Memo_Email

Date: March 27, 2012

To: Office of Management and Budget (OMB)

Through: Keith Tucker, Report Clearance Officer, DHHS

Seleda Perryman, Project Clearance Officer, NIH

Vivian Horovitch-Kelley, PRA OMB Project Clearance Liaison, NCI

From: Holly A. Massett, PhD, Behavioral Science Analyst

Nina Goodman, Project Officer

Office of Communications and Education (OCE),

National Cancer Institute (NCI)/NIH

Subject: Generic Sub-study, Revision of Rapid Feedback Tool to Identify Accrual Concerns with Trial Concepts, Protocols, and Active NCI clinical Trials, under “Formative Research, Pretesting, and Customer Satisfaction of NCI’s Office of Communications and Education,” (OMB No. 0925-0046-21, Expiry Date 02/28/2013).

Background/Need and Use of Information

The information described below was submitted to OMB on May 26, 2011 and we received approval June 15, 2011 (OMB No. 0925-0046-15). We have highlighted the amended portions of the memo in yellow to ease your review process. Specifically, we received approval to conduct up to 24 surveys over the course of one year using two approved templates to develop the surveys. These surveys were designed to seek feedback from the field about low accruing trials that had been open for at least a year.

Recently, the Clinical Investigations Branch of the Cancer Therapy and Evaluation Program (CTEP) in the Division of Cancer Treatment and Diagnosis (DCTD) has increased their focus to obtain feedback on NCI trial concepts before a trial is open—if they can identify and address potential accrual concerns and barriers upfront can possibly circumvent accrual hurdles later on in a trial’s lifecycle. This OMB request is seeking to amend the already approved sub-study to add two additional templates—based on the approved versions–but that seek input earlier in a clinical trial process: one template is for the trial concept stage (i.e., trial idea is under consideration) and the second is for a trial protocol under development (i.e., after the concept is approved but before it is launched). We are not asking for additional burden hours but only to add these templates as options for CTEP to use to seek feedback on trials from clinical researchers in the field (i.e., have 4 templates to choose from instead of just the original 2).

Low accrual to oncology clinical trials persists, with over 40% of National Cancer Institute (NCI) sponsored trials failing to achieve minimum accrual goals. As of 2004, NCI has implemented an institute-wide effort to improve the operational efficiency of its clinical trial enterprise system. There are three timeframes in a trial’s life cycle where opportunities exist to positively impact a trial’s accrual: 1) prior to a trial concept being approved; 2) during a protocol’s development; and, 3) once a trial is open and running. NCI is interested in identifying potential accrual problems at the trial concept, protocol and implementation stage; and ultimately to determine if they are worthy to open, reparable or if they should be terminated (resulting in substantial resource and scientific losses to NCI). Critical is the opinion of researchers in the oncology field, who accrue over 60% of patients to NCI’s trials; yet, NCI has few mechanisms to seek their feedback and remain mostly connected to the academic researchers who develop the trial concepts but rarely accrue. Specifically, feedback from the field can help determine the level of interest in a trial concept, determine the feasibility and burden of the trial’s requirements, and help assess the trial’s challenges to open and run. OCE has recently developed four versions of a brief online feedback tool to assist CTEP to explore a concept’s level of interest, a protocol’s feasibility, and why a trial is poorly accruing. These tools are templates that can be tailored to a trial and sent to oncologists and research staff in the community who opened the trial.

In order to assess if the pool of questions on the templates were appropriate, OCE tailored the tools to a trial that CTEP identified as low accruing. OCE then worked with one community program willing to send invitations out to 4 oncologists and 5 staff persons (total 9) who were willing to fill out the surveys and then participate in a debriefing session afterwards. The tools were further refined based on this feedback. One overriding comment from these 9 individuals was the excitement and support of such feedback tools for future trials. One oncologist noted: “It’s good that NCI is asking our opinion about these trials.” Additionally, it took only an average of five (5) minutes to complete each survey.

Participants

NCI’s Cancer Trials Support Unit (CTSU) tracks all cooperative group sites that receive IRB approval to open each of its trials. For each site, a primary Principal Investigator is listed as well as his/her contact information; the PI typically represents 8-10 sub-sites (i.e., the primary site is the parent site to several satellite sites). The number of PIs listed for each trial ranges from 50 into the hundreds. For each feedback survey, OCE will randomly select up to 50 primary PIs and/or clinical research site staff that have either opened the trial of interest or opened a similar trial (in the case of the surveys that are assessing a new trial not yet open). Each respondent will be sent an invitational email with the survey link and asked to complete it or send the link to another oncologist or research staff member on their team (across their satellite sites) who they believe can best respond to the trial’s accrual concerns (Attachment 21E). Due to no known similar tools used for such feedback among oncology teams, it is unclear the response rate to be expected. That said there is evidence that the field is highly motivated to provide accrual feedback to NCI. Additionally, the brevity and anonymity of the survey supports the assumption of a 75% response rate, amounting to 38 completions for each tool, or a total of 76 participants for each trial. Over the course of one year, and assuming two trials a month, the assumption is 1,824 participants completing the surveys.

See Attachment 21A-C for the oncology feedback survey tools; Attachment 21D for the staff feedback survey tool; and Attachment 21F for the concept sheet which provides an overview of the trial to assist the oncologists and staff to complete the surveys.

Methodology and Research Instrument

OCE would like to pilot these tools for one year on two trials a month. For each trial the process will be: 1) NCI identifies a concept or trial at-risk to low accrual; 2) the applicable template is selected and the trial name is inserted into the tool; 3) an email is sent by NCI to those PIs/staff who opened the trial or a similar trial to the one presented in the concept or protocol inviting them to respond to a questionnaire about the trial; 4) a reminder is sent out after one week, with the tool shutdown after two weeks; 5) data are compiled from an excel sheet print out and put into a standard report to give to the PI and NCI stakeholders involved in the trial. NCI can use the report as part of their decision-making to amend or run the concept, protocol or trial. OCE will keep records of the trials over the year and determine the impact of the findings overall. Currently, no other mechanisms exist to solicit this type of feedback from oncology researchers in the field, so their perspective is not represented.

The request put forth in this memo is to determine if such opinions can be collected systematically in an efficient and rapid manner to better guide NCI on its trial portfolio decisions. It is understood that the data are qualitative in nature and designed to guide decisions, but not be the sole or primary determinant of such decisions.

Questions on the templates include: their type of oncology practice; reasons for considering/opening the trial; level of difficulty in opening the trial; scientific interest in the trial (initially and currently); general assessment of number of patients accrued compared to those eligible at their site; level of difficulty consenting patients and why; top reasons patients decline enrollment; degree of competition for patient population; top concerns/reasons for low accrual; recommended changes to the trial to improve accrual; drug availability and support from different practices. These standard template questions will be “tailored” for each trial: tailoring consists of inserting the trial name, cancer type and practice type into the survey when applicable. If a question does not apply to a particular trial (e.g., drug availability), it will be deleted.

For each trial, the close-ended responses will be tabulated and summarized in table format (counts/amounts) on a question-by-question basis. Crosstabs will be included using “organization type” and “accrual rates” (low/high) as the categorical variables. Qualitative responses will be analyzed using a general inductive approach that focuses on condensing raw textual data into brief ‘chunks,’ establishing clear links between research objectives and the data, and developing a framework to describe what the data indicate. Conclusions and recommendations for each trial will be provided to CTEP and the PI. In addition, at the end of the one-year pilot OCE will compile a summary document detailing how the tools were used over the year (e.g., number of trials), the degree of impact the findings had on trial decisions, and lessons learned. A recommendation will be made as to whether or not the process should be expanded permanently.

It is anticipated that the summary findings assessing the tools’ functions, process, use and impact on NCI decision-making for accrual feedback on NCI trials will be submitted for publication in a journal such as the Journal of Clinical Oncology, which is expected to be read by individuals in the community oncology research field who oversee the opening and closing of trials at their sites. It is understood that the summary information collected and compiled over the year will be useful to institutions and organizations that also struggle with accrual to its clinical trials. The publication will include specific discussion of the tools’ limitations, the qualitative nature of the data, and other related lessons learned from the pilot process.

Other Considerations

• A request for Office of Human Subjects Research exemption was submitted on February 7, 2012 amending the application and adding the two additional templates. We received approval on February 9, 2012 for an exemption for the additional templates.

• No PII from respondents will be collected at all from respondents who complete these feedback surveys. Furthermore, no email addresses will be collected or stored. There will be a database behind each survey web-link, but this database is not designed to allow NCI staff, or others connected to the project, the ability to access or search, and it will not have any information stored that identifies the people in any way what so ever.

Burden

A total of 36 oncology surveys and 12 staff surveys will be administered over the 12 months (assumes two trials a month; total = 48 completed surveys). Respondents will be completing one of the four surveys, depending on the trial status (i.e., before the concept has been approved, after the concept has been approved but before the protocol has been developed, or once the trial has started in the field). The previous sub-study requested 0925-0046-15 was approved with 1,824 respondents and 152 burden hours.

No additional respondents or burden hours are needed or being requested at this time beyond what had already been approved, therefore the burden table below includes the numbers in squared brackets [ ]. The surveys will be entered into the electronic system with minimal additional hours.

Estimates of Burden Hours
Types of Respondents	Instrument	Number of Respondents	Frequency of Response	Average Time Per Response (Hours)	Total Hour Burden
Oncologists and Research Staff for NCI-Sponsored Trials	Oncology Feedback Survey (Attachment 21A)	[456]	1	5/60 (0.08)	[38]
	Staff Feedback Survey (Attachment 21B)	[456]	1	5/60 (0.08)	[38]
	Concept Feedback Survey (Attachment 21C)	[456]	1	5/60 (0.08)	[38]
	Prospective Template (Attachment 21D)	[456]	1	5/60 (0.08)	[38]
Total		[1,824]			[152]

List of Attachments (separate files)

21A: Oncologist Survey 21D. Prospective Template

21B: Staff Survey 21E. Invitational Emails (attached below)

21C: Concept Feedback Survey 21F. Concept Sheet (attached below)

Attachment 21E: Invitational Emails

[FOR TRIALS NOT YET OPEN]

Dear Colleague,

The National Cancer Institute’s Cancer Therapy Evaluation Program (CTEP), in conjunction with [insert cooperative group name], would like your opinions about an NCI clinical trial for [insert cancer type] cancer [select: under consideration/not yet open].  This trial is called the [insert nickname] trial ([insert CTEP code]) and will require patients with [insert patient eligibility description].

We are soliciting feedback from clinical oncology researchers who have opened similar trials to learn any potential issues with opening and accruing to this trial.   We have developed a brief online survey to quickly and easily gather your comments anonymously.   

First, please review the brief two-page pdf attached to this email as an overview of the [insert nickname] trial’s concept.  After reviewing this document please click on the link below and take about 5 minutes to answer this short survey.

This is one new approach that CTEP has developed to seek feedback directly from the community about its trials.   Your comments will help us plan in advance for any concerns about the PROSPECT trial identified from this survey. 

****** To access the brief survey, please click this link: [insert survey link]

(or copy/paste link into your browser)

We thank you in advance for your consideration of this important request. If another [oncologist/research staff member] is better suited to answer questions about this trial, please forward this email along for them to respond to the brief survey.

Sincerely,

Holly A. Massett, Ph.D.

Behavioral Science Analyst

Office of Market Research and Evaluation,

Office of Communications and Education

National Cancer Institute

6116 Executive Blvd. Suite 400

Rockville, MD 20892

ph: 240-535-0781

[FOR TRIALS OPEN AND LOW ACCRUING]

Dear Colleague,

The National Cancer Institute’s Cancer Therapy Evaluation Program (CTEP), in conjunction with [insert cooperative group name], would like your opinions about an NCI clinical trial for [insert cancer type] cancer that you have opened and has low patient accrual.  This trial, the [insert nickname] trial ([insert CTEP code]) requires patients with [insert patient eligibility description].

We are soliciting feedback from clinical oncology researchers at sites that have opened the trial via the CTSU and are interested to learn any potential issues with opening and accruing to this trial.   We have developed a brief online survey to quickly and easily gather your comments anonymously.   

The survey will take about 5 minutes to answer.

This is one new approach that CTEP has developed to seek feedback directly from the community about its trials.   Your comments will help us address any concerns about the [insert trial name] trial identified from this survey. 

****** To access the brief survey, please click this link:  [insert survey link]

(or copy/paste link into your browser)

We thank you in advance for your consideration of this important request. If another [oncologist/research staff member] is better suited to answer questions about this trial, please forward this email along for them to respond to the brief survey.

Sincerely,

Holly A. Massett, Ph.D.

Behavioral Science Analyst

Office of Market Research and Evaluation,

Office of Communications and Education

National Cancer Institute

6116 Executive Blvd. Suite 400

Rockville, MD 20892

ph: 240-535-0781

Attachment 21F: Concept-Description Sheet

Phase III NCI Trial [Concept/Description] Sheet

TRIAL NAME: [INSERT CTEP TRIAL CODE NAME]

TRIAL TITLE: [INSERT FULL TRIAL NAME AS LISTED ON CANCER.GOV]

Research Question 1: [INSERT FULL RESEARCH QUESTION]

Research Question 2: [IF MORE THAN ONE RQ EXISTS, ADD THIS LINE]

Primary Objective:

[INSERT BRIEF DESCRIPTION OF TRIAL’S OBJECTIVE(S)]

Design: This is a Phase III study designed to accrue [INSERT #] patients over [INSERT #] years.

[IN THE REMAINING SPACE ON THIS PAGE,

INSERT SCHEMATIC OF THE TRIAL’S PROPOSED/APPROVED DESIGN]

Considerations for Participation:

[IN SPACE BELOW, LIST CRITERIA SITES NEED TO FOLLOW IN ORDER TO OPEN/RUN THE TRIAL]

1. [INSERT TITLE OF CRITERIA #1]. [INSERT BRIEF DESCRIPTION, REFERENCE WHERE IN PROTOCOL THIS CAN BE FOUND, IF APPLICABLE]

2. [INSERT TITLE OF CRITERIA #2]. [INSERT BRIEF DESCRIPTION, REFERENCE WHERE IN PROTOCOL THIS CAN BE FOUND, IF APPLICABLE]

3. [INSERT TITLE OF CRITERIA #3]. [INSERT BRIEF DESCRIPTION, REFERENCE WHERE IN PROTOCOL THIS CAN BE FOUND, IF APPLICABLE]

4. […CONTINUE THROUGH CRITERIA #N].

10