Justification Memo

Date: May 22, 2012

To: Office of Management and Budget (OMB)

Through: Keith Tucker, Report Clearance Officer, HHS

Seleda Perryman, Project Clearance Officer, NIH

Vivian Horovitch-Kelley, PRA OMB Project Clearance Liaison, NCI

From: Linda Nebeling, Branch Chief

Health Behavior Research Branch

Gordon Willis, Project Officer

Applied Research Program

National Cancer Institute (NCI)/NIH

Subject: Generic Sub-study, Cognitive Testing of the Family Life, Activity, Sun, Health, and Eating (FLASHE) Survey under “Questionnaire Cognitive Interviewing and Pretesting,” (OMB No. 0925-0589-04, Expiry Date 4/30/2014).

Background

The National Cancer Institute’s (NCI) Division of Cancer Control and Population Sciences (DCCPS), Behavioral Research Program (BRP) proposes conducting formative research as an integral part of the development of a new data collection instrument, the Family Life, Activity, Sun, Health, and Eating (FLASHE) survey. A separate request for PRA/OMB clearance for a larger research study will be submitted once cognitive interviewing is complete and the surveys and methodology finalized. Specifically, NCI proposes conducting cognitive testing with the objective of identifying potential sources of measurement or response errors within the questionnaires. Many of the questions are new questions that have not been previously tested, or previously-used questions that have never been used with a teenage population.

Need and Use of Information

FLASHE data collection addresses many critical health research and programmatic needs. The Family Life, Activity, Sun, Health, and Eating (FLASHE) survey is a new NCI survey that seeks to examine psychosocial, generational (parent-child), and environmental correlates of cancer preventive behaviors. The goal of the survey is to advance understanding of the dynamic relationship among those factors by providing a public use dataset as a resource to the research community.

These important scientific and program functions require that the FLASHE survey results in high-quality and timely data. Many of the FLASHE survey items are new, previously untested items, while others are previously used items that have never been tested with teenagers. Thus, cognitive testing of the survey questionnaires is essential to identify problems in question wording, context or order effects, as well as response difficulties resulting from the design and layout of survey. The results of the cognitive testing will facilitate improvements to both the questions and the design of the FLASHE questionnaire.

Participants

Before recruitment begins, we will have approval for research with human subjects. Participants for the cognitive interviews will be recruited from the general population, and will include dyads of a parent or caregiver, and his/her teenage child between the ages of 11 and 18. Neither the contractor who will be coordinating the research (Westat) or NCI employees will be eligible to participate in the cognitive interviews. The contractor will recruit respondents using a variety of methods including use of their current database of potential participants and advertising on CraigsList in the local DC area under the Jobs postings within the subheading Et Cetera. The contractor’s digital media team will reach out to 100 local DC area (including lower Pennsylvania) publishers such as mommy and papi bloggers, health websites, giveaway publishers, and Facebook and Twitter users. The team will also look at local MeetUps. The goal will be to encourage these publishers to post about the FLASHE program, the request for parent/caregiver and child participants, and urge their readers to visit the website to sign up. More detailed information about the recruiting effort appears in the Cognitive Testing Plan (Attachment 4E, Section 2, page 2).

A total of 40 individual interviews will be conducted: 20 interviews with parents and 20 interviews with those parents’ children. A screener will be conducted by telephone to seek participation from adults representing a variety of educational attainment levels, ages, race/ethnic backgrounds, and genders (Attachment 4F).

Methodology and Research Instrument

Depending on the preference given by the participants, the contractor will contact the participants via mail or email prior to the interview to remind them of their upcoming appointment (Attachment 4D). The face-to-face cognitive interviews will be conducted in Rockville, MD and in Frederick, MD. All one-on-one interviews will be conducted by a trained and experienced cognitive interviewer. Respondents will complete the survey at the time of the interview.

All cognitive interviews will be audio recorded with the respondent’s consent. The cognitive interviews will use a mix of concurrent and retrospective probing techniques and will take up to 90 minutes each to complete. The cognitive interviewer him/herself or an assistant methodologist will review the completed questionnaires, any notes taken by the cognitive interviewer, and the audio recordings to inform completion of the interview summaries. We will analyze the interview data in a series of iterative steps that identify themes organized by overall survey issues, individual survey items and sections, and respondents’ overall reactions to the survey. For more detailed information about the analysis, please see Attachment 4E, Section 7, page 5.

The audio-recordings will only be accessible to project staff directly working on the project, and no names or other personally identifying information (other than the respondent’s voice itself) will be included on the audio recordings. The contractor will collect personally identifiable information in the form of full name of parent and child, mailing address, and contact information (phone number and/or email address) so that the contractor can contact the participant as a reminder for the upcoming appointment. However, the PII will not be forwarded to NCI and NCI will receive de-identified data. The PII will not be kept in electronic format except the merged reminder letters or sent email message. These will be stored on a secure network drive to which only project staff will have access. Information with PII will be destroyed one month after completion of the project.

Immediately after the interview, adult participants will receive $75 cash as a thank you for their participation, while the teenagers will receive $50 cash. This amount is justified because the length of the interview (90 minutes) is greater than usual and because recruiting parent/child pairs will be logistically challenging.

The contractor will use a semi-structured protocol for conducting the cognitive interviews, focusing primarily on comprehension issues. In addition, the cognitive interviews will assess the potential for instrument navigational difficulties with the form design that might contribute to missing or invalid response data. Respondents will complete the survey instrument and the interviewer will administer a mix of concurrent and retrospective probes. Attachments 4A-1 through 4A-4 contain the survey content for testing, as follows:

• Parent Diet and Demographics Survey

• Parent Physical Activity Survey and Activity Checklist

• Teenager Diet and Demographics Survey

• Teenager Physical Activity Survey and Activity Recall

Attachments 4B-1 and 4B-2 are the proposed cognitive interview guides. Attachment 4C shows the consent forms planned for use with the cognitive interviews. Attachment 4E, Section 6, page 4 contains more detailed information about the interview structure. Westat Institutional Review Board (IRB) approval will be requested by May 25, 2012.

Burden

The cognitive interview will take each of the participants approximately 90 minutes (1.5 hours) to complete. Therefore we expect the total respondent burden for this proposed effort to be 60 hours. This effort will account for less than 5 percent of the total burden hours granted in the full generic OMB clearance package. To date, a total of 172 burden hours have been used of the 3,600 hours that were requested. Estimated cost to the Federal Government is $140,000.

		Estimates of Burden Hours
Types of Respondents	Type of Instrument		Number of Respondents	Frequency of Response	Average Time Per Response (Hours)	Total Burden Hours
Parents and Teenagers	Screener (Attachment 4F)		40	1	5/60	3
	Surveys to Review/Complete (Attachment 4A-1 through 4A-4) and Cognitive Interviews (Attachments 4B-1 and 4B-2)		40	1	1.5	60
Total			80			63

The “Category of Respondents” will be “Individuals.”

List of Attachments

Surveys under Development:

4A-1: Parent Diet and Demographics Survey

4A-2: Parent Physical Activity Survey and Activity Checklist

Note: Graphic designers have intended the checklist to be in a brochure format. The FLASHE page is the cover, then the questions above, and then the last page of the attachment.

4A-3: Teenager Diet and Demographics Survey

4A-4: Teenager Physical Activity Survey and Activity Recall

Note: Graphic designers have intended the checklist to be in a brochure format.

Cognitive Guides:

4B-1: Parent Cognitive Interview Guide

4B-2: Teenager Cognitive Interview Guide

Other Attachments:

4C: Informed Consent Forms (parent and teenager)

4D: Reminder Letter/Email

4E: FLASHE Cognitive Testing Plan

4F: Screener

4