SSB_Generic Theory Development

SUPPORTING STATEMENT B For:

A Generic Submission for Theory Development and Validation

(BRP/DCCPS/NCI)

April, 2014

Rebecca A. Ferrer, PhD

Health Scientist Administrator

Basic Biobehavioral and Psychological Sciences Branch

Behavioral Research Program

Division of Cancer Control and Population Sciences

National Cancer Institute

9609 Medical Center Dr., MSC

Room 3E114, MSC 9761

Bethesda, MD 20892 - mailing address

Rockville, MD 20850 - physical address

Phone: (240) 276-6914
Email: [email protected]

Table of Contents

B. statistical methods 1

B.1 Respondent Universe and Sampling Methods 1

B.2. Procedures for the Collection of Information 1

B.3 Methods to Maximize Response Rates and Deal with Nonresponse 4

B.4 Test of Procedures or Methods to be Undertaken 5

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting

and/or Analyzing Data 6

Attachments

ATTACHMENT 1: Scientific, Expert Panel, and Ongoing Working Group Meetings Information

ATTACHMENT 2: Behavioral Research Program: Mission Statements and Key Initiatives Related to the Advancement of Theory-Based Formative Research

ATTACHMENT 3: List of experts consulted

ATTACHMENT 4: Information on Incentives

ATTACHMENT 5: Previously Approved/Pending Approval Sub-studies

B. statistical methods

B.1 Respondent Universe and Sampling Methods

The universe of potential respondents includes all individuals in the general public. It is difficult to anticipate the sample of potential respondents in advance of study design; as such, each sub-study submission will include a complete description of the target sample and sampling methodology. Respondents could include: members of the general public, mail, or phone; University students; people with cancer, their family members, and/ or their caregivers; Physicians and other healthcare providers; and health researchers. Recruitment may take place via a variety of commonly used recruitment procedures, including: telephone; mail; Internet, University advertisement or participant pool solicitation; print and community advertisement; and clinical advertisement.

As stated in Supporting Statement A of this OMB submission, the collections under this clearance are not intended to be nationally representative, and as such, sampling methodology will not ensure generalizeable results. Power calculations will be undertaken to ensure that the sample size is the smallest required to adequately test the research hypothesis(es) for each study.

B.2 Information Collection Procedures

Formative theory development and validation research involves several information collection procedures, including interviews, focus groups, and surveys, conducted in person, in print, by telephone, and online. Each collection method is outlined briefly below.

All data analysis will be conducted under the advice of an NCI statistician. Data analysis procedures may differ depending on the nature of the study and the questions addressed; however, it is anticipated that data will be analyzed using standard statistical analyses commonly employed in survey research. Weighted analyses would not be undertaken in data obtained under this clearance, as the sub-studies would be formative in nature and not intended to be representative.

Surveys. The majority of the research activities conducted under this generic clearance will be surveys. For these types of surveys, participants will answer questions designed to inform the development of behavioral research theories of cancer prevention and control or to facilitate validation of such theories. Additionally, some surveys may involve presenting messages or other stimuli intended to affect constructs of behavioral theories. For example, a standard message about the benefits of colorectal cancer screening that highlights the potential to catch cancer at an early stage may be presented, and then anticipated regret at not screening may be measured using standard measures. For each survey, NCI will employ the survey method best suited to address the research question(s). Surveys covered under this clearance may be implemented in the following formats:

• Written surveys (mail surveys or questionnaires) are distributed to a sample of participants through the mail. Participants can elect to complete the questionnaires and mail them back to the researcher in a pre-paid envelope. Written surveys are particularly useful for gathering information that may be subject to demand characteristics – for example, participants may be motivated to tell a phone interviewer that their diet is healthier than it actually is in order to report socially desirable information, and may be more willing to admit to poor eating habits when this information is solicited in writing. Additionally, mail surveys are an important way to reach populations that are not easily reached in other mediums (individuals without land lines who cannot be reached in phone surveys; individuals without internet access who cannot be reached in online surveys).

• Online surveys are distributed online to a sample of participants in an existing panel or a panel created for the purposes of the survey. Companies like Knowledge Networks (http://www.knowledgenetworks.com/) administer online surveys for a fee (which covers labor and programming as well as participant incentives). Additionally, projects like TESS (http://www.tessexperiments.org/) can support and administer internet surveys to examine theoretical questions. Emerging platforms like Amazon mTurk (https://requester.mturk.com/) are also useful tools for internet survey research. Amazon mTurk is an internet service that allows researchers to gather survey data (mTurk also supports other purposes related to “crowd-sourcing”). Previous studies under this clearance and in the scientific literature indicate that data collection through mTurk is generally high quality, and the data collection process is relatively fast.¹ Internet surveys are useful to reduce demand characteristics (see above), and also reduce the burden on both participants and researchers (see Supporting Statement A Section A.3).

• Telephone surveys are useful for collecting data that requires interviews – for example, if a questionnaire involves asking about medical procedures that someone might have had but not know the correct name for – such as a fecal occult blood test – an interviewer can describe the procedure and answer questions to determine whether the participant has actually had the medical procedure. In telephone surveys, a sample of potential respondents is drawn and trained interviewers contact them to administer a questionnaire over the phone.

• Computer-assisted telephone interviewing (CATI) surveys, Computer-assisted personal interviewing (CAPI), and Audio-and computer-assisted Self-Interviewing are all types of telephone survey methodologies that could be employed in this type of research (see Supporting Statement A Section A.3). These can involve automated dialing, random selection, scheduling unanswered call dial-back, guided interviewing, computer-assisted interviewing, and automatic data entry.

Focus Groups. Focus groups are useful as a means of theory development, particularly when the theory is being articulated to a specialized population or for a behavior that has not previously been explored in that theoretical context. For example, when attempting to generalize a theory about one behavior to another, a focus group could help to identify differences that could be key in tailoring the theory – such as specific aspects of a behavior that could influence attitudes, which could be very different when the behaviors are very different. In another example, focus groups could be very useful for examining whether attitudes are informed differently for individuals who are a different age, even if the behaviors are the same. Focus groups allow for the collection of rich, in-depth information regarding theoretical constructs. Focus groups traditionally take place in an in-person format, with a moderator to facilitate discussion. However, focus groups may also use a variety of technology-based formats, including videoconferencing, Internet, and teleconferencing (see Supporting Statement A, Section A.3).

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

Several procedures are commonly employed, accepted, and proven effective in maximizing response rate in behavioral research activities employing surveys, interviews, and focus groups.

These procedures involve:

• Informing potential participants about the nature of the studies, including the importance of the research and the purposes and anticipated use of the data

• Recruiting participants though a variety of methods (e.g., newsletters, internet, clinic)

• Obtaining support from key individuals (e.g., obtaining support from healthcare providers at a clinic where recruitment will take place)

• Employing highly-trained staff to conduct interviews and surveys or to facilitate focus groups

• Experienced and well-qualified staff will moderate all focus groups and conduct all interviews and surveys.

• Conducting thorough trainings of all staff, including training concerning techniques for facilitating participant survey completion and cooperation and strategies for engaging potential and current participants

• Targeting recruitments and study methodology appropriately to each sub-study target population and sample

• Employing incentives, where appropriate and approved by OMB (see section A.9)

• Following-up non-response, when appropriate and approved by OMB

• Rescheduling cancellations

• Providing the name and contact information for an official at NCI, who will confirm the importance of participation and answer any questions

• Including postage paid, self-addressed envelopes with mail surveys

• Creating attractive graphics to attract the attention of respondents in mail and internet surveys

• Hand-signing cover letters included in mail surveys

• Employing multiple methods (e.g., telephone and internet) to give participants a choice, when appropriate to the research question

B.4 Tests of Procedures or Methods to be Undertaken

Before most sub-study information collection is undertaken, contractors will pilot-test any new questionnaires, instruments, and methods of data collection. Lessons from pilot tests will be identified and question and design changes will be made as necessary. Pretesting will involve no more than nine individuals unless OMB clearance is sought and obtained.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following individuals, among others, including scientific staff, fellows, and contractors, who may be chosen to collect theory development and validation research, will be responsible for the design of statistical and sampling procedures for each sub-study as part of these data collection activities:

Richard P. Moser, Ph.D.

Office of the Associate Director

Behavioral Research Program

Division of Cancer Control and Population Sciences

National Cancer Institute

6130 Executive Blvd.

Rockville, MD  20852

National Cancer Institute

Phone: 301-496-0273

Email: [email protected]

Rebecca A. Ferrer, Ph.D.

Basic Biobehavioral and Psychological Sciences Research Branch

Behavioral Research Program

Division of Cancer Control and Population Sciences

National Cancer Institute

6130 Executive Blvd., Room 4056

Rockville, MD  20852

301-594-0437

[email protected]

1 Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk a new source of inexpensive, yet high-quality, data?. Perspectives on Psychological Science, 6(1), 3-5.

Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26(3), 213-224.

3