Mini SSB

Supporting Statement B for

Generic Sub-study:

Reliability Testing of Survey Questions about the Walking Environment

(NCI)

(OMB No. 0925-0589, Expiry Date 7/31/2017)

David Berrigan, PhD MPH

Behavioral Research Program

Division of Cancer Control and Population Sciences

National Cancer Institute

9609 Medical Center Drive MSC 7344

Room 3E342

Bethesda MD 20892-7344

Phone: 240-276-6752

Fax: 240-276-7906

[email protected]

February 18, 2015

List of Attachments

Attachment 1: Appendix A – Screenshots for Condition 1 (Walking Environment First)

Attachment 2: Appendix B – Screenshots for Condition 2 (Walking Behaviors First)

Attachment 3: Appendix C – Screenshots for Condition 3 (Random Order)

Attachment 4: Appendix D – Cognitive Probes

Mini Supporting Statement B For

“Generic Clearance to Support Programs and Administrative Operations

At the National Cancer Institute (NCI)”

0925-0589, Expiration Date 07-31-2017

Title of Sub-Project: Reliability Testing of the Walking Environment

Dr. David Berrigan, Program Director, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health

Section B. Statistical Methods

B1. Respondent Universe and Sampling Methods

The proposed study is methodological in nature. Therefore, data collection will not make use of statistical sampling methods, but as is common with research on questionnaires that is formative in nature, will rely on a convenience sample obtained through use of an existing, opt-in, Internet-based system named Amazon Mechanical Turk (mTurk; wwww.mturk.com). The sample obtained is not to be regarded as representative of the U.S. household population: No statistical estimates of U.S. characteristics will be produced as a result of this data collection, as the objective is to assess differences between experimental conditions.

This study is based on a sample design approach dependent on volunteers via the Amazon Mechanical Turk system (mTurk: www.mturk.com). These respondents are known to be similar to the population of internet users, but differ from a representative sample of the U.S. population. Specifically, mTurk respondents are generally younger and more educated than the general U.S. population and are predominantly White and middle class. The respondent universe consists of the over 500,000 members from over 190 countries registered in the mTurk system.  Only English-speaking ambulatory U.S. residents will be included in this study because the proposed modules on walking behaviors may not be valid for non-English-speaking, non-U.S. samples.

B2. Procedures for the Collection of Information

English-speaking, ambulatory U.S. residents will be recruited to participate in the study through Amazon’s Mechanical Turk (www.mturk.com). All English-speaking, U.S. residents age 18 and older are eligible to participate. Respondents will be presented with 5 demographic questions, 11 questions on walking behaviors, 6 questions on walking environment, and 3 cognitive probes (only completed at Time 2) with a total average administration time of 5 minutes at Time 1 and 6 minutes at Time 2. Respondents will complete the two surveys at Time 1 and Time 2 approximately 4 weeks apart. If someone completes all of the questions at Time 1 and Time 2, it will take a total of 11 minutes to complete.

As an experimental manipulation, question order will be varied, with a third of the respondents first completing the walking environment questions (and then completing the walking behaviors questions second); a third first completing the walking behaviors questions (and then completing the walking environment questions second); and the final third completing a combination of walking behaviors and walking environment questions integrated in random order. Appendices A-C contain the screenshots for the 3 experimental conditions. Appendix D contains the screenshots for the cognitive probes.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Given the nature of the non-probability-based method used for this project, a standard survey response rate cannot be computed:  According to the American Association for Public Opinion Research (http://www.aapor.org/AAPORKentico/AAPOR_Main/media/MainSiteFiles/StandardDefinitions2011_1.pdf), “For non-probability samples, response rate calculations make little sense, given the broader inferential concerns. Further, for many of these surveys, the denominator is unknown, making the calculation of response rates impossible.”  Similarly, because participants will not be selected via a random process, they are not considered respondents in the usual sense, and therefore the concept of nonresponse does not apply.   

Of the 1,500 total respondents recruited at Time 1, we expect a cooperation rate (as defined by AAPOR) of 80% at Time 2.

B.4 Test of Procedures or Methods to be Undertaken

N/A

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The principle investigator will be collecting and analyzing the collected information.

Reliability Testing of Survey Questions about the Walking Environment– Page 7