Mini SSA

Supporting Statement A for

Generic Sub-study:

Reliability Testing of Survey Questions about the Walking Environment

(NCI)

(OMB No. 0925-0589, Expiry Date 7/31/2017)

David Berrigan PhD MPH

Behavioral Research Program

Division of Cancer Control and Population Sciences

National Cancer Institute

9609 Medical Center Drive MSC 7344

Room 3E342

Bethesda MD 20892-7344

Phone: 240-276-6752

Fax: 240-276-7906

[email protected]

February 18,2015

List of Attachments

Attachment 1: Appendix A – Screenshots for Condition 1 (Walking Environment First)

Attachment 2: Appendix B – Screenshots for Condition 2 (Walking Behaviors First)

Attachment 3: Appendix C – Screenshots for Condition 3 (Random Order)

Attachment 4: Appendix D – Cognitive Probes

Mini Supporting Statement A For

“Generic Clearance to Support Programs and Administrative Operations

At the National Cancer Institute (NCI)”

0925-0589, Expiration Date 07-31-2017

Title of Sub-Project: Reliability Testing of the Walking Environment

Dr. David Berrigan, Program Director, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health

Section A.

A1. Circumstances Making the Collection of Information Necessary

Section 410 of the Public Health Service Act (42 USC § 285) authorizes the collection of the information requested in this sub-study clearance application. The information collection request falls under the Health Behaviors Research Branch, housed within the Behavioral Research Program of the Division of Cancer Control and Population Sciences (HBRB/BRP/DCCPS).

As per NCI Generic Clearance, Questionnaire Cognitive Testing and Interviewing (OMB # 0925-0589, expiration date: 7/31/2017), the formative research proposed in this document focuses on “survey material development and pretesting based on cognitive interviewing methodology” (Supporting Statement Part A, April 2014, p. 1). This information collection is formative research related to questionnaire design.

A2. Purpose and Use of the Information Collection

The purpose of this sub-study is to conduct formative research and cognitive testing on a newly developed questionnaire assessing perceptions of the walking environment. Specifically, we will examine the reliability of responses to walking environment questions and walking behaviors questions, which both contain items of the type administered within Federal surveys such as the CDC/NCHS National Health Interview Survey (NHIS). The proposed information collection also involves methodological research to examine the extent to which question order may influence survey responses. That is, does answering questions about the walking environment first influence how much walking respondents report? To answer this question, we will vary the order that respondents answer these types of questions. Respondents will be randomly assigned to complete Condition 1) walking environment questions first and the walking behavior questions second (Appendix A), Condition 2) walking behavior questions first and the walking environment questions second (Appendix B), or Condition 3) walking environment questions and walking behavior questions in random order (Appendix C).

The initial baseline measures (Time 1) will include the 5 demographic questions, the 6 question Walking Behaviors Module, and the 11 question Walking Environment Module (in varying order based on experimental condition). Then again approximately 4 weeks later (Time 2) respondents will answer the same set of modules as Time 1 with the addition of the 3 question cognitive probe (attachment D)added at the very end. The cognitive probes are web-based and self-administered. Unlike traditional in-person cognitive interviewing in which oral probing is conducted among a small sample of respondents and with a trained cognitive interviewer, this study involves a novel form of probing that is self-administered, web-based, and collected among a large sample of respondents.

Primary objectives include:

1. Formative research involving test-retest reliability of the Walking Environment

2. Formative research involving the factor structure of the Walking Environment

3. Cognitive testing examining if question order influences self-reported walking behavior

4. Cognitive testing involving human-computer interfaces/usability

The results of the information collection will be used to further develop and improve assessments, with the expectation that they will be included within appropriate sections on walking behavior, in future Federal surveys that assess physical activity, such as the NHIS, as well as surveys that may involve computerized self-administration.

A3. Use of Information Technology and Burden Reduction

Because we are examining the human-computer usability as part of cognitive testing, the modules will be administered to respondents electronically, through an online information-collection platform called Amazon Mechanical Turk (mTurk; wwww.mturk.com). mTurk is an internet service that allows researchers to collect survey data in a quick fashion minimizing the burden for respondents. The use of this online information-collection platform will decrease the response burden to the respondent, eliminate the need for manual data entry, and eliminate the use of paper to collect information. Appendices E-G contains the screenshots for the 3 experimental conditions.

A4. Efforts to Identify Duplication and Use of Similar Information

No similar information collection exists.

A5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this study.

A6. Consequences of Collecting the Information Less Frequently

This is a one-time information collection.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances relating to 5 CFR 1320.5

A8. Comments in Response to Federal Register Notice and Efforts to Consult Outside Agency

We have not consulted with any outside agency on this project.

A9. Explanation of Any Payment or Gift to Respondents

Respondents will receive $0.25 for completion of the evaluated survey items at first survey administration, and an additional $0.25 for completion of the same items at the follow-up administration of the same items. At the first time point, it will take respondents an average of 5 minutes to complete all of the items which include the 5 demographic questions, 6 questions on walking behavior, 11 questions on walking environment. At the second time point, it will take respondents an average of 6 minutes to complete because participants will be asked to answer all of the aforementioned questions in addition to 3 cognitive probe questions. A total burden of 11 average minutes is estimated to complete all of the questions at Time 1 and Time 2. The total incentive for respondents completing both item administrations will be $0.50. In order to use the mTurk platform, we are required by Amazon to offer a minimum incentive amount to respondents.

A10. Assurance of Confidentiality Provided to Respondents

The information will be kept private to the extent that is allowed by law. The Privacy Act Coordinator determined that a PIA is not necessary. This project is deemed exempt from human subjects approval by the NIH Office of Human Subjects Research.

A11. Justification for Sensitive Questions

No PII will be collected. No sensitive questions will be asked. The format for the race and ethnicity used is based on the standards set by the Affordable Care Act,

http://aspe.hhs.gov/datacncl/standards/ACA/4302/index.shtml.

A12. Estimates of Hour Burden Including Annualized Hourly Costs

The modules will be administered to individuals twice except for the 3 cognitive probe questions, which will only be administered at Time 2. We estimate a total mean burden of 5 minutes per respondent at Time 1 and 6 minutes per respondent at Time 2. We will recruit a convenience sample of 1,500 respondents, with 500 respondents assigned to each of the 3 experimental conditions. The expected amount of administration time for all questions is approximately 5 minutes with an additional 1 minute for the cognitive probes at Time 2. The total amount of administration time across Time 1 and Time 2 is estimated at 11 minutes (extra minute for cognitive probes at Time 2). If a respondent completed all of the questions at Time 1 and Time 2, it would take a total of 11 minutes to complete. The total respondent burden for this proposed effort is estimated to be no more than 274 burden hours (Table A12-1).

Table A12-1. Estimates of Hour Burden

Type of Instrument	Type of Respondent	Number of Respondents	Number of Responses Per Respondent	Average Burden Per Response (in hours)	Total Burden Hours
Condition 1	General Public	500	2	5/60	83
Condition 2	General Public	500	2	5/60	83
Condition 3	General Public	500	2	5/60	83
Cognitive Probes (All Participants)	General Public	1500	1	1/60	25
Total					274

Table A12-1 illustrates the measures respondents will be answering and the number of hours to complete each type. These 1500 respondents will be completing the demographic questions, walking behaviors questions, and walking environment questions at Time 1 and then again at Time 2 four weeks later. Respondents will also complete the cognitive probes questions at Time 2.

Table A12-2. Cost to Respondents

Type of Respondent	Number of Respondents	Total Burden Hours	Wage Rate	Respondent Cost
Individuals	1500	200	$22.33	$4,466

Table A12-2 contains labor calculations for all occupations. Labor information, obtained from http://www.bls.gov/oes/current/oes_nat.htm#00-0000, provides the average US hourly earnings ($22.33 as of May 2013) for employees of all occupations (occupation code: 00-0000 occupation title “All Occupations”).

A13. Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no capital costs, operating costs, or maintenance costs to report.

A14. Annualized Cost to the Federal Government

The annualized cost to the federal government is $57,141.

Table A14-1. Cost to the Federal Government

Source	Task	Annualized Cost
NCI Staff	NCI GS 15-7 Program Officer (5% of $120,740 for 12 months)	$6,037
	NCI GS 14-8 Program Officer (5% of $105,500 for 12 months)	$5,275
	NCI CRTA Fellow (75% of $60,000 for 12 months)	$45,004
Respondent Costs	Time 1 incentives	$375
	Time 2 incentives	$375
	Mechanical Turk administrative fees	$75
Total		$57,141

Table A14-1 contains the annualized cost to the federal government. The NCI GS-15 Program Office and NCI GS-14 Program Officer will both be dedicating 5% of their total time to the project, engaging in oversight and administrative roles. Salary information for GS-level Program Officers can be obtained at: http://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/2015/general-schedule/. The NCI CRTA Fellow will be dedicating 75% of her total time to the project by administering the survey, collecting, cleaning, and analyzing data, and writing up the results for publication. Salary information for NIH CRTA fellows can be obtained at: https://www.training.nih.gov/trainee_-_postdoc_faqs .

A15. Explanation for Program Changes or Adjustments

This is a new collection of information.

A16. Plans for Tabulation and Publication and Project Time Schedule

The project time schedule is outlined in Table A16-1. Please see below.

Table A16-1. Project Time Schedule

Activity	Months after OMB Approval
Post modules on mTurk (collect information)	0 – 1
Information collection period	2 - 3
Clean and analyze information	4 – 7
Interpret and Summarize information	8 – 11
Write report	12

Descriptive statistics will first be conducted to examine the distribution of scores for both modules.

Second, as part of formative research, test-retest reliability coefficients will be calculated for Time 1 and Time 2 scores on the Walking Environment Module.

Third, as part of formative research, we will conduct an Exploratory Factor Analysis (EFA) to determine the factor structure of the Walking Environment Module.

Finally, as part of cognitive testing and determining if question order influences responses and reliability, we will conduct analysis of variance tests (ANOVAs) to determine if there are condition differences in responses or differences in responses due to time.

No statistical point estimates for the U.S. population (or walking or of any other behavior) will be produced, or published, based on this information collection. The results of this methodological study may be combined and published with the conclusions drawn from previously approved studies relating instrumentation reliability and order effects. Outcomes of this study will also be used to inform future tool development.

A17. Reason(s) Display of OMB Expiration Date Is Inappropriate

We are not requesting exemption from the display of the OMB expiration date.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

This information collection will comply with the requirements in 5 CFR 1320.9.