Memo-Justification #12

Date: November 14, 2008

To: Office of Management and Budget (OMB)

Through: Seleda Perryman, DHHS Report Clearance Officer

Marilyn Tuttleman, NIH Project Clearance Officer, OPERA

Vivian Horovitch-Kelley, NCI OMB Project Clearance Liaison Office

From: Nina Goodman, Project Officer

Office of Communications and Education (OCE)

National Cancer Institute/NIH

Subject: Generic Sub-Study, “Survey to Assess Public Understanding of Risk Estimates through Visual Displays” (OMB No. 0925-0046-12)

The National Cancer Institute (NCI) proposes conducting a web-based survey of visual methods for communicating the uncertainty of individualized cancer risk estimates. The objective of this study is to collect formative data to inform development of a new cancer risk prediction model, the Colorectal Cancer Risk Assessment Tool (CCRAT), developed by Dr. Andrew Freedman and colleagues in the Applied Research Program and the Division of Cancer Epidemiology and Genetics. This research consists of a administering a web-based survey designed to pilot test different methods of communicating individualized colon cancer risk estimates, focusing specifically on the visual communication of uncertainty surrounding the reliability and accuracy of the model. The study will examine how different visual/graphical methods of communicating uncertainty influence psychological outcomes including knowledge, interest in using the CCRAT, trust in information, and perceptions of cancer risk and of the model’s accuracy and credibility.

Previous formative research conducted by the Office of Market Research and Evaluation (OMRE) within the Office of Communications and Education (OCE) assessed messages and message formats of risk information to inform the development of the CCRAT. The current study is an independent formative research investigation of a new and different aspect of the information provided by the CCRAT, the visual display of risk estimates, and will be conducted through an existing contract with the Academy for Educational Development (AED). Findings from this study will be used to determine and design the best, most effective visual displays and educational formats for communicating the results of the CCRAT to clinicians and laypersons.

Background information on Web surveys

Web-based surveys represent a standard state-of-the-art formative testing methodology, adapted from marketing and communications research. For this formative research, a self-administered web-based survey will be used, as it is a methodology frequently used by NCI to pretest drafts of NCI concepts and materials that is both reliable and efficient. This survey will be accessed on-line at a designated and secure Internet location.

Proposed Research: Web-based study of visual/graphical methods for communicating uncertainty about individualized cancer risk estimates

NCI proposes using a Web survey to assess how people respond to risk information displayed through five different visuals/graphics communicating uncertainty regarding individualized cancer risk estimates produced by a cancer risk prediction model. Potential individual and contextual mediators and moderators of people’s responses will be explored. For this formative research, NCI proposes conducting a web-based survey with members of the general public, ages 40 and older, with no family history of colon cancer; as these characteristics define the potential target population for the CCRAT. Potential participants will include both men and women residing in geographic regions across the United States, and representing all racial, ethnic, and educational backgrounds.

Because the purpose of the current survey is to collect information for use in formative pretesting of the CCRAT, the sampling methods will focus on obtaining, with the greatest efficiency possible, a sample of respondents that is diverse though not necessarily nationally-representative, and that has demonstrated proficiency in using the web. To this end, the survey will be conducted among members of a professionally-managed web-based survey panel. Panel members have experience conducting online surveys, have already indicated an expressed interest in completing web-based surveys, and have explicitly agreed to participate in surveys presented to them about issues both unrelated and related to health. Panel members eligible to participate in the current survey will be contacted through an e-mail invitation from the panel managers which will include a secure, non-identifiable link to the web-based survey. Recruitment will continue until the target sample size for completed surveys is reached.

Participants will be asked to answer a brief, self-administered, interactive web-based survey which includes one of five potential visual/graphical outputs from the hypothetical colon cancer risk prediction tool. Reactions to the following five visual/graphical methods for communicating uncertainty will be assessed (see attachment for illustrations of each visual/graphical method):

1. Numeric risk range: standard bar graph with confidence interval.

2. Blurred risk range: standard bar graph with confidence interval, but with visually-blurred borders to convey uncertainty.

3. Degraded icons: standard icon array representing a group of people, with confidence interval represented by blurred borders to convey uncertainty.

4. Dynamic icons – version 1: standard icon array representing a group of people, but with real-time, dynamic movement of the array to convey “randomness.”

5. Dynamic icons – version 2: standard icon array representing a group of people, but with real-time, dynamic movement of the array to convey “randomness,” and including blurred borders to also convey uncertainty.

The Web-survey will examine participants’ reactions to one of the five alternative visual/graphic displays from the hypothetical colon cancer risk prediction tool listed above, and explore whether these alternative visual/graphic presentations of uncertainty will have different effects on key perceptions and attitudes. Specific measures on the survey will include the following:

1. Knowledge

2. Perceived cancer risk

3. Worry about cancer risk

4. Confidence in understanding

5. Perceived accuracy of estimate

6. Understanding of the meaning of risk estimates

7. Understanding of the limits of risk estimates

8. Interest in risk reduction

9. Interest in using the risk prediction model

10. Socio-demographic (age, sex, education, family cancer history)

The total sample size of 250 (50 respondents/group) represents the minimum number of participants required for reliable inferences to be drawn to guide NCI’s development of the CCRAT for public use.

Estimates of Hour Burden
Types of Respondents	Number of Respondents	Frequency of Response	Average Response Time	Annual Hour Burden
General Public	250	1	10/60	41.67

Individual respondents will not be identified and participation will be strictly voluntary. Names or images will not be recorded, nor will personal identifying data be maintained, nor will any responses to items will have any effect on their eligibility for, or receipt of, services.

All data will be collected by the contractor, the Academy for Educational Development (AED), and all personal identifiers will be excluded from the data records. Any necessary identifying or potentially identifying information (e.g., signed consent agreements) will be secured and kept separate from the data records. All information provided by respondents will be maintained in a confidential manner, unless compelled by law. The focus group data files that are delivered to NCI will be analyzed in the aggregate and no identifiable individual respondents will be provided. NCI and AED’s Institutional Review Board (IRB) Research Integrity Officers will review the research instruments and ensure that all necessary human subject protection procedures are in place.

The full generic study, approved on January 26, 2007, requested a total of 2010 burden hours. There have been ten previous sub-studies approved by OMB under this umbrella submission, totaling 965 burden hours requested to date. Approval by OMB of this sub-study would bring the total burden hour requested to date for 0925-0046 to 1007; well below the original request of 2010 hours per year.

Thank you for your consideration of this proposed sub-study #12.

Attachments: Web-based survey

4