Mini SSA

Mini Supporting Statement A For

Generic Clearance “Questionnaire Cognitive Interviewing and

Pretesting (NCI)”

OMB #0925-0589, Expiration Date 07-31-2017

Title of Sub-Project: Formative Research to Investigate a Web Panel and to Determine Feasibility of Web-Based Cognitive Pretesting

Dr. Richard Moser, Program Director, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health

List of Attachments

Appendix A: Screenshots for Condition 1

Appendix B: Screenshots for Condition 2

Appendix C: OHSRP Exemption

Section A.

A1. Circumstances Making the Collection of Information Necessary

Section 410 of the Public Health Service Act (42 USC § 285) authorizes the collection of the information requested in this sub-study clearance application. The information collection request falls under the Science of Research and Technology Branch, housed within the Behavioral Research Program of the Division of Cancer Control and Population Sciences (SRTB/BRP/DCCPS).

As per NCI Generic Clearance, Questionnaire Cognitive Testing and Interviewing (OMB # 0925-0589, expiration date: 7/31/2017), the formative research proposed in this document focuses on “survey material development and pretesting based on cognitive interviewing methodology” (Supporting Statement Part A, April 2014, p. 1). This information collection is formative research related to questionnaire design.

A2. Purpose and Use of the Information Collection

The purpose of this sub-study is to examine if question order influences responses to survey questions. That is, does answering questions first about engaging in healthy behaviors (e.g., physical activity) influence how much medical information people report seeking. Or does asking people first about seeking medical information change how they respond to questions about engaging in healthy behaviors? To answer this question, we will vary the order that respondents answer these types of questions. Respondents will be randomly assigned to 1 of 2 conditions where condition 1 (Appendix A), asks questions about health information first followed by questions about engagement in health behaviors. Condition 2 (Appendix B) asks questions about engagement in health behaviors first followed by questions about health information second. The only difference between the two conditions is the order of the questions. All respondents will answer the same questions. This project is deemed exempt from human subject’s approval by the NIH Office of Human Subjects Research Protections.

Respondents will complete one of the two conditions that will consist of the 6-item Looking for Health Information Module, 13-items Using the Internet to Find Information Module, 8-item Quality of Health Care Module, the 2-item Electronic Medical Records Module, 2-item Cancer Screening Module, 9-item Cancer History and Beliefs Module, the 10-item Physical Activity and Nutrition Module, the 4-item Tobacco Use Module, 13 demographic questions. Also, as part of formative research, we will include 5 cognitive probes at the end of the survey asking respondents what they were thinking when responding to the survey questions. Unlike traditional in-person cognitive interviewing in which oral probing is conducted among a small sample of respondents and with a trained cognitive interviewer, this study involves a novel form of probing that is self-administered, web-based, and collected among a large sample of respondents.

We estimate a total mean burden of 30 minutes per respondent. We will recruit a convenience sample of 1,000 respondents, with 500 respondents assigned to one of the 2 experimental conditions: health information questions first or healthy behaviors questions first. The expected amount of time for all questions to be completed is approximately 30 minutes.

Primary objectives include:

1. Formative research examining if question order influences self-reported seeking of health information

2. Formative research involving use of novel methods for cognitive probing

3. Cognitive testing involving human-computer interfaces/usability

The results of the information collection will be used to further develop and improve the evaluated modules.

A3. Use of Information Technology and Burden Reduction

Because we are examining the human-computer usability as part of cognitive testing, the modules will be administered to respondents electronically, through an online information-collection platform called Amazon Mechanical Turk (MTurk; wwww.mturk.com). MTurk is an internet service that allows researchers to collect survey data in a quick fashion minimizing the burden for respondents. The use of this online information-collection platform will decrease the response burden to the respondent, eliminate the need for manual data entry, and eliminate the use of paper to collect information. Appendices A and B contain the screenshots for the 2 experimental conditions.

A4. Efforts to Identify Duplication and Use of Similar Information

No similar information collection exists.

A5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this study.

A6. Consequences of Collecting the Information Less Frequently

This is a one-time information collection.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances relating to 5 CFR 1320.5

A8. Comments in Response to Federal Register Notice and Efforts to Consult Outside Agency

We have not consulted with any outside agency on this project.

A9. Explanation of Any Payment or Gift to Respondents

Respondents will receive $1.85 for completion of the evaluated survey items. It will take respondents an average of 30 minutes to complete their assigned condition. This amount of remuneration is standard for Amazon mTurk participant and is consistent with remuneration offered in previously approved sub-study under 0925-0645 Expiration 11/30/2017 (Refining and Validating a Theory of Risk Perception and 0925-0589 Expiration 7/31/2017 Reliability Testing of Survey Questions about the Walking Environment. This amount will be issued by Amazon to participants’ mTurk “worker” accounts. In order to use the mTurk platform, we are required by Amazon to offer a minimum incentive amount to respondents.

A10. Assurance of Confidentiality Provided to Respondents

The information will be kept private to the extent that is allowed by law. The Privacy Act Coordinator determined that a PIA is not necessary.

The OHSRP has deemed this information collection exempt from human subject research (Appendix C).

A11. Justification for Sensitive Questions

No PII will be collected. The following question topics may appear to be sensitive in nature: occupation status, race, salary, gender, marital status and cancer history for family members or self.

A12. Estimates of Hour Burden Including Annualized Hourly Costs

The total respondent burden for this proposed effort is 500 burden hours for a total of 1,000 respondents. Table A12-1 illustrates the instruments that respondents will be completing and the number of hours to complete each type. These 500 respondents will complete Condition 1 which is estimated to take 30 minutes to complete. 500 respondents will complete Condition 2 which is estimated to take 30 minutes to complete.

Table A12-1. Estimates of Hour Burden

Type of Instrument	Type of Respondent	Number of Respondents	Number of Responses Per Respondent	Average Burden Per Response (in hours)	Total Burden Hours
Condition 1 (Appendix A)	General Public	500	1	30/60	250
Condition 2 (Appendix B)	General Public	500	1	30/60	250
Totals		1,000	1,000		500

The total cost to respondents is $11,355. Table A12-2 contains the calculation for respondents costs by using title “All Occupations” occupation code “00-0000” with a wage rate of $22.71. This information was obtained from the Bureau of Labor Statistics, http://www.bls.gov/oes/current/oes_nat.htm#00-0000.

Table A12-2. Cost to Respondents

Type of Respondent	Total Burden Hours	Wage Rate	Respondent Cost
Individuals	500	$22.71	$11,355
Totals	500		$11,355

A13. Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no capital costs, operating costs, or maintenance costs to report.

A14. Annualized Cost to the Federal Government

The annualized cost to the government is $48,419. Table A14-1 contains the annualized cost to the federal government. The NCI GS-14-9 Program Officer and NCI Title 42 Program Officer will both be dedicating 5% of their total time to the project, engaging in oversight and administrative roles. The NCI CRTA Fellow will be dedicating 50% of her total time to the project by administering the survey, collecting, cleaning, and analyzing data, and writing up the results for publication. Salary information for NIH CRTA fellows can be obtained at: https://www.training.nih.gov/trainee_-_postdoc_faqs .

Table A14-1. Cost to the Federal Government

Staff	Grade/Step	Salary	% of Effort	Fringe (if applicable)	Total Cost to Gov’t
Federal Oversight	14-9	$135,000	5%		$6,750
	Title 42	$153,357	5%		$7,668
	CRTA Fellow	$62,000	50%		$31,000
Cost for Web Panel (including admin fees)					$4,000
TOTAL					$49,418

A15. Explanation for Program Changes or Adjustments

This is a sub-study.

16. Plans for Tabulation and Publication and Project Time Schedule

Descriptive statistics will first be conducted to examine the distribution of scores for all modules.

Second, as part of formative research, we will conduct independent samples t-tests to determine if the order of questions between the two experimental conditions influenced responses.

Third, as part of formative research, we will analyze responses to the cognitive probes which will help us determine how items are functioning within the modules.

No statistical point estimates for the U.S. population (or walking or of any other behavior) will be produced, or published, based on this information collection. The results of this methodological study may be combined and published with the conclusions drawn from previously approved studies relating instrumentation reliability and order effects. Outcomes of this study will also be used to inform future tool development.

The project time schedule is outlined in Table A16-1. Please see below.

Table A16-1. Project Time Schedule

Activity	Months after OMB Approval
Post modules on mTurk (collect information)	0 – 1
Information collection period	2 - 3
Clean and analyze information	4 – 7
Interpret and summarize information	8 – 11
Write report	12

A17. Reason(s) Display of OMB Expiration Date Is Inappropriate

We are not requesting exemption from the display of the OMB expiration date.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

This information collection will comply with the requirements in 5 CFR 1320.9.