SSB RAND revisions 2018-11-20

The World Trade Center Health Program:

Impact Assessment and Strategic Planning for Translational Research

(Part 1, Formative Research: Focus Groups)

Supporting Statement – Section B

November 20, 2018

Program Official/Project Officer

LCDR Pattama Ulrich, RN, MPH

World Trade Center Health Program
DHHS/USPHS/CDC/NIOSH
513-223-0011
[email protected]

Table of Contents

1. Respondent Universe and Sampling Methods

2. Procedures for the Collection of Information

3. Methods to Maximize Response Rates and Deal with Non Response

4. Tests of Procedures or Methods to be Undertaken

5. Statistical and Data Collection Consultants

List of Attachments

1. Att. A Authorizing Legislation

2. Att. B 60-Day FRN

3. Att. C Consent Form

4. Att. D Focus Group Discussion Guide

5. Att. E Recruitment and Reminder emails

6. Att. F RAND IRB Determination

7. Att. G Brief Demographic Survey

8. Att. H NIOSH IRB Determination Form

9. Att. I Privacy Impact Assessment

SUPPORTING STATEMENT B

The World Trade Center Health Program: Impact Assessment and Strategic Planning for Translational Research (Part 1, Formative Research: Focus Groups)

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The focus groups will be conducted with three major stakeholder communities of the World Trade Center Health Program (WTCHP): funders, researchers, and research users. Two of the three major stakeholder communities have key sub-communities. Table 1 shows the communities and sub-communities that we will recruit for participation in focus groups along with rationale for selection. Each focus group meeting will be comprised of WTCHP stakeholders from one sub-community. RAND will conduct 12 focus groups across the stakeholder categories with about 8-10 participants per group, for maximum of 120 participants. Burden calculations are based on a maximum of 110 participants. Ten participants (NIOSH employees) are Federal employees acting within the scope of their job responsibilities, and are thus excluded from the burden estimate.

Table 1: Stakeholder communities and sub-communities

Community	Sub-community [approach]	Rationale	# groups*
Funder	Those affiliated with the WTCHP at NIOSH [in person]	Provides NIOSH’s perspective on translational research efforts.	1
Researchers	WTC Health Registry [in person]	Provides a unique perspective on surveillance of members’ health.	1
	Principal Investigators of WTCHP-supported research [in person]	Provides insights into research priorities and the process of communicating work to research users.	3
Research Users	Clinicians caring for WTCHP Members [phone/webinar]	Provides perspectives on quality of care and service delivery from an on-the-ground experience.	2
	Leadership from the WTC Clinical Centers of Excellence and Data Center representatives [in-person]	Provides insights on how WTCHP-supported research is viewed by health system leadership (who make decisions about clinical care) and leaders of the 3 Data Centers (who make decisions around clinical care and the flow of clinical data).	1
	WTCHP members: responders & survivors [phone and webinar]	Provides perspectives on how research impacts personal care.	3
	“Other” category: policy makers, program advisors, non-profits, government agencies that interact with the WTCHP [phone/webinar]	Stakeholders whose decision making affects the WTCHP as a whole.	1
Total		12 focus groups (96-120 total participants)

*8-10 participants per group

For all stakeholder groups RAND will recruit participants using purposive sampling, a type of non-probability sampling that will generate a non-representative subset of each stakeholder group. For this type of qualitative research, representativeness is not required as the research will complement the data gathered during planned in-depth interviews, in which we will probe on interesting or unusual findings and follow-up on specific themes. The focus group research itself is not meant to be generalizable to a larger population but is exploratory to uncover and elaborate on key themes that will complement the interviews in Part 2 of the qualitative research.

Participants will first be selected through lists both maintained and generated by NIOSH (e.g., principal investigator lists, clinical centers of excellence clinical leadership, member and scientific advisory committees). For stakeholder categories where contact lists may not be readily available (e.g., members), we will contact, through an introduction by NIOSH, selected advocacy organizations including but not limited to Voices of 9/11. We will sample for maximum variation within each stakeholder category, seeking to obtain variation on characteristics such as: experience and familiarity with WTCHP research; type or focus of research conducted (principal investigators); degree of contact or participation with WTCHP activities; and health outcomes experienced (members).

Across the three types of focus group respondents we will recruit a minimum of 96 and maximum of 120 participants, depending on response rate. We will include a maximum of 40 participants in the “researcher” stakeholder category, 70 participants in the “research users” category, and 10 participants in the “funder” category.

2. Procedures for the Collection of Information

RAND will recruit a maximum of 120 participants from the stakeholder categories listed in Table 1. Participants will be asked open-ended questions about the following:

• Conceptualizations of research and “translational research”

• Relevance of WTCHP research topics, potential gaps, and stakeholder priorities

• Uses and usefulness of WTCHP research

• Barriers to conduct and use of WTCHP research

• Understanding of and perspectives on the relevance and usefulness of the Research-to-Care model

One main protocol will be used to guide focus group sessions across all stakeholder categories. The focus group questions have been constructed to be general and applicable to all stakeholder categories, allowing for group comparisons across questions. In some instances, we include probes for specific stakeholder groups. Participants are expected to provide different perspectives and levels of detail in their answers, which then can be analyzed to make connections among the data.

Depending on the timing of OMB approval, we anticipate conducting focus groups shortly after, most likely in the winter/early spring of 2019. If this occurs, results will be analyzed in the spring of 2019. If the timing of OMB approval coincides with one of the twice-yearly NIOSH-sponsored research meetings in NYC, we will plan to hold in-person focus groups with the stakeholder groups in attendance (NIOSH and principal investigators); the remainder of the focus groups will be held by webinar to minimize burden on the participants. For these webinars, RAND will employ electronic technology (e.g., video/webinar- or tele-conferencing) to conduct focus groups and maximize convenience. Each focus group meeting will last approximately 2 hours and will be recorded and transcribed. Focus group meetings will be moderated by one RAND staff member using the standardized focus group protocol/script (including suggested question probes). A second RAND staff member will be available to take notes as a back-up to the recording and to keep time.

3. Methods to Maximize Response Rates and Deal with No Response

The following are examples of procedures that have proven effective in previous studies and will be used when possible to obtain at least a 90% response rate:

• Informing respondents of project purpose and rationale of the evaluation, who will see the evaluation results, and how the results will be used

• Using culturally appropriate data collection instruments and procedures

• Using alternative communication means, such as video-or teleconferencing

• Addressing data security and privacy safeguards with respondents

• Minimizing the time needed for participation in the project

• Informing respondents about the project process and focus group duration and setting, so that they know what to expect

• Limiting participation of each respondent to one interview meeting

• Discussing the importance of the evaluation for different stakeholders and how the findings will be put into action

• Giving participants multiple options for scheduling

In addition, we will employ flexible sampling procedures to maximize responses. For all stakeholder groups in Table 1, RAND will recruit participants using purposive sampling, a type of non-probability sampling that will generate a non-representative subset of each stakeholder group. If initial recruitment efforts do not yield the desired participant numbers across the target stakeholder sub-communities, snowball sampling will be used, where those who have agreed to participate are asked to provide names and contact information for other potential recruits.

NIOSH will initiate contact by email with each prospective participant (see Attachment F Recruitment and Reminder emails). The initial contact will inform the prospective participant that RAND will be contacting them to invite them to participate in a focus group and that participation in this project is voluntary. All NIOSH and RAND recruitment emails will describe the purpose of the research, privacy procedures (see Section 10 in Supporting Statement A), and the benefits that prospective participants can expect from involvement.

At the time participation is confirmed or declined, RAND will ask respondents to provide the names of 2-3 additional referrals in their same stakeholder group, to be contacted first by NIOSH and then by RAND, should the need arise to implement snowball sampling during the recruitment process.

After the first email contact is made, up to 2 reminder emails will be sent (followed by an optional phone call to any difficult-to-reach and under-enrolled stakeholder group) (Attachment F). After second reminder emails have been sent to the initial list of participants, two approaches will be used to intensify recruitment: (1) we will contact the additional potential participants that the invitees provided when they confirmed or declined the invitation and, if needed; (2) we will enlist the assistance of NIOSH to provide additional names of potential participants. Each of these recommended individuals will be contacted by email using the same procedures described above, except that the name of the referring person (at NIOSH, or by an initial invitee) will also be given.

As described above, to maximize convenience for stakeholder communities and thus maximize response rates, teleconference- and webinar-enabled focus groups will be conducted shortly after OMB approval is received. As noted above, if the timing of OMB approval coincides with one of the twice-yearly NIOSH-sponsored research meetings in NYC, we will plan to hold in-person focus groups with the stakeholder groups already in attendance (NIOSH and principal investigators).

4. Tests of Procedures or Methods to be Undertaken

A group of three RAND subject matter experts outside of the research team reviewed the focus group protocol and provided feedback and edits to ensure probes covered important points to gather and questions were clear. In addition, RAND will pre-test the focus group protocol with five members of RAND staff pulled from different occupation types (e.g., administrative staff, facilities/mail staff, research support staff, research staff). RAND will conduct interviews to assess whether certain sections and questions performed as expected, to identify any questions and wording that were unclear, and estimate the length of time required to complete the protocol. As the protocol is designed to be semi-structured, we do not anticipate making large adjustments to the content or format, but may need to clarify the intent of certain questions or probes depending on the feedback.

5. Statistical and Data Collection Consultants

The data collection tools and procedures were designed by the RAND Corporation under the leadership of the individuals named below. The same individuals will also be responsible for collecting and analyzing the data.

Thomas Concannon, Project Director

20 Park Plaza, Suite 920

Boston, MA 02116

[email protected]

617.338.2059 x8615

Laura Faherty, Co-Project Director

20 Park Plaza, Suite 920

Boston, MA 02116

[email protected]

617.338.2059 x8693

Ramya Chari, Focus Group Lead

1200 South Hayes Street

Arlington, VA 22202-5050

[email protected]

703.413.1100 x5216

8