#9_Justif Memo & Attach C,D,F&G

Date: December 21, 2010

To: Office of Management and Budget (OMB)

Through: Seleda Perryman, DHHS Report Clearance Officer

Marilyn Tuttleman, NIH Project Clearance Officer, OPERA

Vivian Horovitch-Kelley, NCI OMB Project Clearance Liaison, OMAA

From: Bradford Hesse, Project Officer,

Health Information National Trends Survey (HINTS),

Division of Cancer Control and Population Sciences,

National Cancer Institute/NIH

Subject: Health Information National Trends Survey 4 (HINTS 4) Cognitive Testing of Cycle 1 Instrument

Generic Sub-study under “Questionnaire Cognitive Interviewing and

Pretesting,” (OMB No. 0925-0589-09; Expiration Date: 5/31/2011)

The National Cancer Institute’s (NCI) Division of Cancer Control and Population Sciences (DCCPS), Behavioral Research Program (BRP) proposes conducting formative research for the fourth iteration of the Health Information National Trends Survey (HINTS 4)¹. Since many of the questions that are to be included in the Cycle 1 instrument are considered “core” questions, they will be asked during each of the four cycles of data collection. NCI proposes conducting cognitive testing of the Cycle 1 data collection instrument with the objective of identifying any potential sources of measurement or response errors within the questionnaire.

Background, Need and Use of Information. The HINTS data collection program addresses many critical health research and programmatic needs. The HINTS program monitors changes in the rapidly evolving field of health communication. Survey researchers are using the data to understand how adults 18 years and older use different communication channels, including the Internet, to obtain vital health information for themselves and their loved ones. Program planners are using the data to understand and address barriers to health information usage across populations and to create more effective communication strategies. Finally, social scientists are using the data to refine their theories of health communication in the information age and to offer evidence-based recommendations for reducing the burden of cancer throughout the population. The HINTS program develops and disseminates HINTS Briefs showcasing results to ensure that findings from HINTS research can be used to develop evidence-based policies, programs, and practices relevant to health communication at the national, state, and local level.

These important scientific and program functions require that the HINTS program engage in efforts to develop and improve data collection protocols that will result in high-quality and timely data. Based on experimentally controlled methods research conducted as part of the previous round of HINTS, HINTS 4 will use an entirely mail-based data collection procedure that collects data in 4 cycles with 4 independent samples. In addition to switching to an entirely mail-based data collection, the content of the survey itself will not necessarily reflect the identical content as previously implemented in HINTS data collections. There is an ongoing effort to include input from subject matter experts in the broader HINTS research and data user community. Thus, cognitive testing of the survey questionnaire is essential to identify problems in question wording, context or order effects, as well as response difficulties resulting from the design and layout of the new mail form. The results of the cognitive testing will facilitate improvements to both the questions and the design of the questionnaire for use in HINTS 4, cycle 1 data collection².

Participants. Participants for the cognitive interviews will be recruited from the non-institutionalized adult household population. Neither Westat nor NCI employees will be eligible to participate in the cognitive interviews. HINTS 4 will include a Spanish-language questionnaire, so information about the performance of the Spanish version of the questions is needed. Seventeen of the 50 cognitive interviews will be conducted with adults who speak primarily Spanish at home. A racially diverse population will be recruited for the English-speaking cognitive interviews. For both the English and Spanish speaking cognitive interviews, we will seek participation from adults representing a variety of educational attainment levels. Additionally, we will attempt to recruit up to 10 participants in the English-speaking interviews and up to 5 in the Spanish-speaking interviews who have had a prior diagnosis with cancer (of any type) to ensure coverage of the some of the cancer-specific items on the questionnaire.

Data Collection Methods. The face-to-face cognitive interviews will be conducted in the focus group facilities, or in private conference rooms at Westat in Rockville, MD. Seventeen of the 50 cognitive interviews will be conducted in Spanish and 33 will be conducted in English. All interviews will be conducted by a trained and experienced cognitive interviewer whose native language is the one being used for the interview.

All cognitive interviews will be audio recorded with the respondent’s consent. The cognitive interviews will use a mix of concurrent and retrospective probing techniques and will take up-to 90 minutes each to complete. The cognitive interviewer him/herself or an assistant methodologist will review the completed questionnaires, any notes taken by the cognitive interviewer, and the audio recordings to inform completion of the interview summaries. The audio-recordings will only be accessible to project staff directly working on the project, and no names or other personally identifying information (other than the respondent’s voice itself) will be included on the audio recordings. Participants will be given $50 as a thank you for their participation.

Cognitive Interview Materials. Westat will use a semi-structured protocol for conducting the cognitive interviews, focusing primarily on comprehension issues with those questions new to HINTS or modified for the mail administration of HINTS. In addition, the cognitive interviews will assess the potential for instrument navigational difficulties with the form design that might contribute to missing or invalid response data. See Attachment A for the proposed questionnaire content for testing and Attachment B for the proposed interview protocol. Attachment C shows the consent form planned for use with the cognitive interviews. The Spanish-language materials will reflect the same content as the English language materials.

Participant Recruitment. Westat, along with their subcontractor Eureka Facts³, will recruit participants for the cognitive interviews. Recruiters will place advertisements on Internet sources such as Craig’s List, and will print advertisements in local community papers around the Washington DC metropolitan area (see Attachment D). Additionally, we will work with doctor offices that diagnose or treat cancer patients located near the Westat facility and seek their agreement to post or distribute flyers to patients. Similarly, we will also contact local adult education programs to seek permission to distribute flyers at one or more classes that cater to populations with less than a college education. All advertisements will include a toll-free phone number for those interested in finding out more about participation.

Eureka Facts will coordinate recruitment of the Spanish-speaking participants. As a professional market research firm specializing in Spanish speaking and other hard-to-reach populations, EurekaFacts will advertise for participants in Spanish-market papers and locations for which they have had prior success in recruiting one-on-one interview participants. Additionally, EurekaFacts may draw from eligible Spanish-speaking respondents within their established recruitment database, providing individuals have not participated in similar cognitive tests in the last 12 months. All advertisements will include a toll-free number dedicated to Spanish language calls.

When interested individuals call either toll-free number, recruitment staff will explain the purpose of the project, eligibility criteria, and what is involved in participating. Recruitment staff will screen individuals for participation (see Attachment E). One week before the interview, recruitment staff will send individuals who have agreed to participate a reminder letter (see Attachment F) and directions to the Westat facility. Recruitment staff will attempt to contact all participants by phone 1–2 days prior to the interview to remind them of their appointment.

Data Analysis. Westat will analyze the data using qualitative analysis methods. Cognitive interview summaries will be analyzed to identify issues common across respondents or supported by theory to guide the final question wording and form design construction for use in the first cycle of data collection of HINTS 4.

IRB Review. Approval or exemption from both Westat and NCI IRBs is obtained prior to the initiation of any testing. The cognitive testing activities and materials outlined in this memo was approved by Westat’s IRB and is currently under review at NCI’s IRB. The previous HINTS sub-study (0925-0589-06) received Westat IRB approval and amended approval (see Attachment G).

Privacy Considerations. As a necessity for recruitment, we will collect personally identifying information (PII). The completed recruitment questionnaires will be stored in a locked cabinet/drawer. The keyed data from the questionnaires will replace individuals’ names with unique ID numbers for electronic storage. The mapping between the person’s name and the associated ID number will be stored in a separate file. Both the mapping and the electronic version of the recruitment responses will be stored on a restricted-access drive within the Westat firewall. Both the paper and electronic versions of the recruitment information will be destroyed within two weeks of the completion of the final report covering the cognitive testing research.

Additionally, we have been in contact with the NCI Privacy Act Coordinator, Suzanne Milliard, and discussed the need for a Privacy Impact Assessment (PIA). We have been informed that a PIA is not needed for the cognitive testing activities because a system is not being established. A PIA for HINTS 4 will be submitted in coordination with the beginning of data collection for Cycle 1 in the fall of 2011.

Respondent Burden and Costs. The table below displays the burden estimate for this data collection effort. We expect to have up to 50 people participate across the English- and Spanish-language cognitive interviews. Those interviews will last up to 90 minutes each. The total respondent burden for this effort is estimated to be 83 hours. This effort will account for less than 5 percent of the total burden hours (1800) granted in OMB No. 0925-0589.

Estimates of Hour Burden

Type of Respondents	Instrument	Maximum Number of Respondents	Frequency of Response	Estimated Burden (minutes/hour)	Total Burden Hours
General public	Screening Questionnaire (Attachment E)	100	1	5/60 (0.083)	8 hours
	Questionnaire Content and Interview Protocol (Attachments A and B)	50	1	90/60 (1.5)	75 hours
Total		150			83

Thank you for your consideration of this proposed sub-study.

Attachments (attached in a separate file)

A: HINTS 4 Questionnaire Content (Cycle 1)

B: Interview Protocol

E: Screening Questionnaire

Attachments (attached below)

C: Consent Form

D: Recruitment Advertisement

F: Reminder letter to Cognitive Interview Participants

G: Westat IRB Amended Approval for HINTS Material testing (OMB No. 0925-0589-06)

Attachment C: COGNITIVE TESTING CONSENT FORM

We would like to talk with you about a questionnaire Westat is developing as part of the Health Information National Trends Survey (HINTS). Westat is conducting this research for the U.S. Dept. of Health and Human Services (DHHS). The questionnaire asks about health-related topics. By taking part in this interview, you are helping us evaluate how easy or difficult the questions are to understand and answer. Your opinions will help us improve the questionnaire.

• Your participation is completely voluntary. You may stop at any time, and you can skip any questions you do not wish to answer.

• All information obtained during this study will be treated as secure and will only be used to develop and improve the questionnaire. We will not share your answers with anyone outside of the HINTS project research team.

• The interviewer will audiorecord the discussion and take notes. In addition, project researchers may observe the interview. The researchers will destroy the audio recording as soon as they complete the questionnaire development process.

• The interview should take about an hour and a half.

• The report summarizing the findings will not contain any names or identifying information.

• You will receive $50 cash as a token of our appreciation for completing the session.

• If you have questions about this research, please contact Brad Hesse, the Principal Investigator at (301-594-9904; [email protected]). If you have questions about your role as a research participant, please contact Sharon Zack, the Westat Institutional Review Board Administrator (301-251-1500; [email protected]).

• A copy of this consent form has been provided for your records.

If you agree to participate in this interview, please sign the following statement.

I have read this consent form and understand the proposed project. I consent to participate in this study and to have the interview audio taped.

___________________________________ ___________________________________

Participant’s Signature Researcher’s Signature

___________________________________ ___________________________________

Printed Name Printed Name

___________________________________

Date

Attachment D: SAMPLE RECRUITMENT ADVERTISEMENT

Health Information Study

Receive $50

Westat is looking for study participants to review survey questions about how people find information about health topics as well as questions about health in general. Interviews will be conducted at Westat’s Rockville office and will last approximately 90 minutes.

We are interested in adults at least 18 years of age or older.  All participants receive $50 dollars for their time. Call 800-1-888-XXX-XXXX and say you are calling about the Health Information Study.  Please leave a name, telephone number and a good time to reach you. 

ATTACHMENT F: REMINDER LETTER

[Date]

Dear [name],

Thank you for agreeing to participate in our study on health-related topics. Here are details to remind you where and when to appear for the interview session. The session will take about an hour and a half. You will receive $50 as a token of gratitude for your participation in the study.

DATE: XX/XX/XXXX

TIME: XXpm

LOCATION: Westat

Conference Center

1600 Research Boulevard

Rockville, MD 20820

Directions and a map are enclosed.

Kindly allow enough time for travel so that we may begin on time. There will be designated parking spaces marked “(Study)” in the Conference Center Parking lot. Someone will meet you in the lobby of the 1600 building.

If you need further information, please call me at the office (301) XXX-XXXX.

We look forward to seeing you and thank you in advance for your assistance.

Sincerely,

[Westat staff]

Attachment G: Westat IRB Amended Approval for HINTS Advance Materials Testing

(OMB No. 925-0589-06)

1 HINTS 4 (OMB No. 0925-0538) Reinstatement with Changes is planned to be submitted for OMB approval in Summer, 2011.

2 Separate OMB generic sub-study clearance packages will be submitted for testing the mail questionnaires associated with each cycle 2 through 4 data collections.

3 Eureka Facts, LLC is a market research and analysis firm that conducts data collection, market research, and data analysis. Eureka Facts staff has particular experience in focus group facilitation and cognitive testing conducted as part of study development and a track record of engaging and obtaining response from hard to reach populations including Spanish speakers and teenagers. Recently, Eureka Facts has been involved in qualitative and quantitative work for the Census Bureau, the Behavioral Risk Factor Surveillance System, the California Health Interview Survey, and the National Children’s Study.

7