HINTS 5 Cognitive Testing Mini SSA

Mini Supporting Statement A For:

Cognitive Testing of the Health Information National Trends Survey V (HINTS V), Cycle 1 Instrument

(NCI)

(OMB No. 0925-0589, Expiry Date 7/31/2017)

Gordon Willis, PhD
Behavioral Research Program

Division of Cancer Control and Population Sciences

National Cancer Institute

9609 Medical Center Drive

Room 3E228

Bethesda MD 20892

Phone: 240-276-6788

[email protected]

April 6, 2016

List of Attachments

A. Westat IRB Approval Letter

B. OHSRP Request for Determination Form

C. Screening questionnaire

D. Consent form

E. Questionnaire content

F. Interview protocol

G. Recruitment advertisement

H. Reminder letter

Section A.

A1. Circumstances Making the Collection of Information Necessary

The National Cancer Institute’s (NCI) Division of Cancer Control and Population Sciences (DCCPS), Behavioral Research Program (BRP) proposes conducting cognitive testing for the Health Information National Trends Survey (HINTS V) data collection instrument which is planned for fielding in late 2016. This activity will be conducted under Generic Clearance #0925-0589, expiration date 7/31/2017: “Questionnaire Cognitive Interviewing and Pretesting,” which states that “The types of activities covered by this Generic request include…Survey material development and pretesting based on cognitive interviewing methodology Although a subset of the proposed HINTS V survey questions have been included in previous HINTS instruments which have already been tested, the testing proposed will focus on items developed specifically for HINTS V, mainly concerning uses of health information technology.

The HINTS data collection program addresses many critical health research and programmatic needs related to health communication. Because there is an ongoing effort to include input from subject matter experts in the broader HINTS research and data user community, the development of the questions for HINTS V involved partnership with the Office of the National Coordinator (ONC) at the Department of Health and Human Services (HHS), and from the National Partnership for Women and Families (NPWF). Given the new content, cognitive testing of the survey questionnaire is essential to identify problems in question wording, context or order effects, as well as response difficulties resulting from the design and layout of the mailed paper form.

A2. Purpose and Use of the Information Collection

The results of the cognitive testing will facilitate improvements to both the questions and the design of the questionnaire for use in HINTS V. It is anticipated that the questions being tested will be used in the instrument repeatedly over survey cycles, so the cognitive testing conducted under this request will inform the HINTS instrument for years to come and ensure that HINTS data are valid and reliable by examining questionnaire wording, flow, and timing. Specifically, we will be seeking each respondent’s thought processes to identify and refine:

• Instructions that are insufficient, overlooked, misinterpreted, or difficult to understand;

• Wordings that are misunderstood or understood differently by different respondents;

• Vague definitions or ambiguous instructions that may be interpreted differently;

• Items that ask for information to which the respondent does not have access; and

• Confusing response option or response formats.

NCI’s contractor, Westat, will be responsible for recruitment, testing and analysis. Specifically, Westat will:

• Recruit testing respondents. Some respondents will come from Westat’s existing database of testing respondents. In addition, Westat will advertise for new respondents (see Attachment G). This advertising will take place primarily via the web on sites such as Craig’s List.

• Screen potential respondents. Westat will administer a screener instrument (see Attachment C) with each potential respondent to ensure that the final pool of respondents includes a mix of ages, races, ethnicities, levels of education, and cancer status.

• Schedule appointments and send reminders. Once a respondent has been screened and determined to be appropriate for HINTS V testing, Westat will schedule a cognitive interview with that person. As a courtesy, Westat will send that person a reminder letter via the US Postal Service or (if the respondent has provided an email address) via email. See Attachment H for the reminder letter.

• Administer informed consent. Once a respondent has arrived at Westat for testing, the Westat tester will provide the respondent with an informed consent form (see Attachment D). This form tells the respondent about the voluntary nature of the testing, what will happen with their information, that the session will be audio taped, and who to contact if they have concerns. The respondent will be asked to sign 2 copies of this form: one for Westat to keep on file and the other for the respondent to take home.

• Conduct the cognitive testing. After providing informed consent, the respondent will start the cognitive testing. The Westat tester will give the respondent a copy of the survey (see Attachment E) ask him/her to fill out the survey as if they were in their own home. The tester will stop the respondent occasionally to ask questions about how the respondent reached a certain answer. These follow-up questions will follow the interview protocol, provided in Attachment F. The cognitive interview will consist of using the interview protocol (Attachment F) while the respondent is completing the survey, which will take 90 minutes.

• Pay the incentive. Once the testing is complete, Westat will provide each respondent the $50 incentive and ask them to sign a receipt.

• Conduct analysis and make recommendations. Once all tests are complete, Westat will compile all the qualitative data, conduct analyses and present the results to NCI. Based on the testing results, Westat will make recommendations to NCI about improvements to be made to the HINTS V, Cycle 1 instrument.

A3. Use of Information Technology and Burden Reduction

Respondents will be recruited via the Internet, and cognitive testing will be conducted face-to-face interviews at Westat’s testing facility.

A4. Efforts to Identify Duplication and Use of Similar Information

The ONC and the NPWF have verified that the questions being tested are new and are not being collected elsewhere in Federal surveys.

A5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this study.

A6. Consequences of Collecting the Information Less Frequently

This is a one-time information collection.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances relating to 5 CFR 1320.5

A8. Comments in Response to Federal Register Notice and Efforts to Consult Outside Agency

This is a sub-study.

In designing the current data collection involving cognitive interviewing, NCI staff consulted with the Office of the National Coordinator (ONC) at the Department of Health and Human Services (HHS. , the primary operational component of the DHHS responsible for implementing strategies for connected interoperability. Conversations began with an early kick-off meeting with the appropriate ONC representatives, and continued through collaborative working sessions on early content drafts. The resulting content contains the results of this purposeful collaboration between agencies.

A9. Explanation of Any Payment or Gift to Respondents

Consistent with procedures Approved under Generic Clearance #0925-0589 (expiration date 7/31/2017), cognitive testing respondents will receive $50 as an incentive for their participation. This incentive amount has been used for conducting previous cognitive testing for HINTS over the past several years, and it has been successful in retaining respondents throughout the testing time frame.

A10. Assurance of Confidentiality Provided to Respondents

All information will be kept secure to the extent allowable under the law. NCI will not receive PII at any time. Attachment D contains the Recruitment Form to be used by Westat interviewers. Westat will collect personally identifying information (PII), only for purposes of recruitment, and this information will be destroyed within two weeks of the completion of the final report covering the cognitive testing research. All procedures, including those for maintaining confidentiality have been Approved by the Westat IRB (Attachment A).

Based on the NIH Office of Human Subjects Protection Determination form (Attachment B), this project is not considered to involve research requiring human subjects, and therefore requires no NIH IRB review.

A11. Justification for Sensitive Questions

There are no sensitive questions.

A12. Estimates of Hour Burden Including Annualized Hourly Costs

The total annual burden for this effort is estimated to be 58 hours. The table below displays the burden estimate for this data collection effort. We expect to screen up to 60 people. The screener is expected to take 10 minutes. We plan to include 30 people in a 90 minute cognitive interview. Initially, respondents will also complete Consent Form which we estimate to take 5 minutes.

Table A12-1: Estimates of Hour Burden

Type of Respondents	Form Name	Number of Respondents	Number of Responses Per Respondent	Average Burden Per Response ( in hours)	Total Burden Hours
General Public	Screening Questionnaire (Attachment C)	60	1	10/60	10
	Consent Form (Attachment D)	30	1	5/60	3
	Interview Protocol (Attachments E and F)	30	1	90/60	45
Total		60	120		58

The annual cost to respondents is $1,317.18. This amount was calculated from labor information, obtained from the Bureau of Labor Statistics, http://www.bls.gov/oes/current/oes_nat.htm#00-0000. The $22.71 wage rate was obtained using occupation code: 00-0000 occupation title “All Occupations”. Table A12-2 below contains labor calculations for all occupations

Table A12-2: Cost to Respondents

Type of Respondent	Total Burden Hours	Wage Rate	Respondent Cost
Individuals	58	$22.71	$1,317.18

A13. Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no capital costs, operating costs, or maintenance costs to report.

A14. Annualized Cost to the Federal Government

The annualized cost to the federal government is $76,685.

Table A14-1 contains the annualized cost to the federal government. The NCI Title 42 Program Officer will be dedicating 5% total time to the project, engaging in oversight and administrative roles. Salary information for Program Officers can be obtained at: http://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/2016/general-schedule/. The NCI CRTA Fellow will be dedicating 10% of her total time to the project by administering the survey, collecting, cleaning, and analyzing data, and writing. Salary information for NIH CRTA fellows can be obtained at: https://www.training.nih.gov/trainee_-_postdoc_faqs.

Table A14-1. Cost to the Federal Government

Staff	Grade/Step	Salary	% of Effort	Fringe (if applicable)	Total Cost to Gov’t
Federal Oversight
Program Manager	Title 42	$153,357	5%		$7,668
CRTA Fellow		$62,000	10%		$ 6,200
Contractor Cost: Fixed price contract includes recruitment, screening, testing, analysis and recommendations					$60,000
TOTAL					$73,868

A15. Explanation for Program Changes or Adjustments

This is a regular sub-study.

A16. Plans for Tabulation and Publication and Project Time Schedule

The project time schedule is outlined in Table A16-1 below. There are no plans to publish the results of the cognitive testing, although the aggregate, qualitative results may be entered into the Q-Bank as a reference for other researchers.

Table A16-1: Timeline

Activity	Months after OMB Approval
Recruit respondents for testing	Immediately after approval
Conduct testing	1
Analyze results and make recommendations for instrument changes	2
Finalize survey questions based on analysis	3

A17. Reason(s) Display of OMB Expiration Date Is Inappropriate

We are not requesting exemption from the display of the OMB expiration date.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

This information collection will comply with the requirements in 5 CFR 1320.9.