Mini SSB

Mini Supporting Statement B For

Generic Clearance: Questionnaire and Cognitive Interviewing and

Pretesting (NCI)

OMB # 0925-0589, Expiration Date 07-31-2017

Title of Sub-Project: Formative Research to Investigate a Web Panel and to Determine Feasibility of Web-Based Cognitive Pretesting

Dr. Richard Moser, Program Director, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health

List of Attachments

Appendix A: Screenshots for Condition 1

Appendix B: Screenshots for Condition 2

Appendix C: OHSRP Exemption

Section B. Statistical Methods

B1. Respondent Universe and Sampling Methods

The proposed study is methodological in nature. Therefore, data collection will not make use of statistical sampling methods, but as is common with research on questionnaires that is formative in nature, will rely on a convenience sample obtained through use of an existing, opt-in, Internet-based system named Amazon Mechanical Turk (MTurk; wwww.mturk.com). The sample obtained is not to be regarded as representative of the U.S. household population: No statistical estimates of U.S. characteristics will be produced as a result of this data collection, as the objective is to assess differences between experimental conditions.

This study is based on a sample design approach dependent on volunteers via the Amazon Mechanical Turk system (MTurk: www.mturk.com). These respondents are known to be similar to the population of internet users, but differ from a representative sample of the U.S. population. Specifically, MTurk respondents are generally younger and more educated than the general U.S. population and are predominantly White and middle class. The respondent universe consists of the over 500,000 members from over 190 countries registered in the MTurk system. Only English-speaking, U.S. residents will be included in this study because the proposed modules on health information may not be valid for non-English-speaking, non-U.S. samples.

B2. Procedures for the Collection of Information

English-speaking, U.S. residents will be recruited to participate in the study through Amazon’s Mechanical Turk (www.mturk.com). All English-speaking, U.S. residents age 18 and older are eligible to participate. Respondents will complete the 6-item Looking for Health Information Module, the 13-item Using the Internet to Find Information Module, the 8-item Quality of Health Care Module, the 2-item Electronic Medical Records Module, 2-item Cancer Screening Module, 9-item Cancer History and Beliefs Module, the 10-item Physical Activity and Nutrition Module, the 4-item Tobacco Use Module, 13 demographic questions, and 5 cognitive probes with a total average administration time of 30 minutes. Respondents will complete the 30 minute survey one time. As an experimental manipulation, question order will be varied, with half of the respondents first completing questions on health information followed by questions on healthy behaviors, condition 1 (Appendix A). The other half will first be completing questions on healthy behaviors followed by questions on health information, condition 2 (Appendix B). Demographic questions and cognitive probes will be completed at the end for both conditions. Appendices A and B contain the screenshots for the 2 experimental conditions.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Given the nature of the non-probability-based method used for this project, a standard survey response rate cannot be computed:  According to the American Association for Public Opinion Research (http://www.aapor.org/AAPORKentico/AAPOR_Main/media/MainSiteFiles/StandardDefinitions2011_1.pdf), “For non-probability samples, response rate calculations make little sense, given the broader inferential concerns. Further, for many of these surveys, the denominator is unknown, making the calculation of response rates impossible.”  Similarly, because participants will not be selected via a random process, they are not considered respondents in the usual sense, and therefore the concept of nonresponse does not apply.   

B.4 Test of Procedures or Methods to be Undertaken

N/A

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The Principle Investigator and his NCI CRTA fellow will be collecting and analyzing the collected information.

Reliability Testing of the Walking Environment Module – Page 4