Focus Group Discussion Guide

Form Approved

OMB No. 0920-xxxx

Exp. Date xx/xx/XXXX

Attachment E. Focus Group Discussion Guide

Public reporting of this collection of information is estimated to average 2 hours per response, including the time for reviewing instructions and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a current valid OMB number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer, 1600 Clifton Road NE, MS D-74, Atlanta, GA, 30033; ATTN: PRA (0920-XXXX).

Stakeholder group #1: NIOSH Staff

Stakeholder group #2: WTC Registry and Principal Investigators of WTCHP-supported research

Stakeholder group #3: (Members, Clinicians, Health System Leadership at the Clinical Centers of Excellence, Data Centers, Policymakers)

Welcome, Ground Rules, Consent (15 minutes)

Thank you for joining us for this discussion of the World Trade Center Health Program.

My name is *** and I’m a *** at the RAND Corporation. My research focuses on ****. For those of you who are not familiar with RAND, we are a nonprofit, nonpartisan, research organization that is committed to the public interest. RAND’s research and analysis addresses issues that affect people around the world including health, education, security, justice, and the environment, among many other areas.

[Co-facilitator]: And I’m ****. I’m a *** at the RAND Corporation. In my work, I focus on ****.

The purpose of the discussion today is to hear your perspectives on the World Trade Center Health Program, and specifically, your thoughts on the Program’s research mission. The World Trade Center Health Program contracted with the RAND Corporation to conduct an evaluation of the Program’s translational research efforts, which we’ll talk about more in a few minutes. Your unique perspectives on the Program will help inform our recommendations to the Program on translating research findings into better care for Members.

This meeting will last 2 hours. We’re going to discuss several topics related to your experiences with the World Trade Center Health Program, starting with your thoughts on research and translational research specifically; then focusing on the research topics that the Program has prioritized so far; then the use of research for different purposes; and finishing with your perspectives on the processes the WTCHP uses to achieve its research mission. In order to get through all of the questions we have in our allotted time, my colleague *** will help me move through the topics in a timely fashion. Please be aware that if we are running over time on any one question, we may have to move on to the next topic.

We’ll first explain our procedures for protecting your privacy and then ask for your consent to participate and be recorded. Then, at the end of our meeting, we’ll ask you to fill out a brief paper survey about yourself that should take about a minute or less to complete.

RAND will use the information you provide during this focus group for research purposes only. Specifically, these focus groups will provide important information for the evaluation as a whole and will also be used to develop in-depth interview guides for the next phase of this project. We hope you will feel comfortable sharing your honest opinions based on your own experiences, and there are no right or wrong answers. In our reports, comments will not be linked to you or your organization. In conversations you may have outside of this focus group, we ask that you protect the privacy of others in this meeting. Please do not repeat anything that is said here so that feedback from other groups members is protected. Comments shall not be attributed to a particular person or organization.

This meeting will be recorded because we don’t want to miss anything that you say. The recording will be transcribed and the transcripts will be stripped of any information that could identify you later. After removing all identifying information from the transcripts, the recordings and the original transcripts will be destroyed.

Does anyone have any questions about the purpose of this focus group, our privacy procedures, or anything else that I’ve mentioned so far?

Now, I’ll go around the room and ask you one by one for your consent to participate in this focus group and to be recorded.

[To each participant]: Do you consent to participate in this discussion and to be recorded?

[If everyone consents to participate and be recorded, start the recording and note the start time for the focus group. If anyone does not consent, ask that person to please excuse him/herself and wait until the individual has left the room before starting the recording.]

Focus Group Participant Introductions (10 minutes)

We’re going to start with introductions so everyone knows who’s in the room [on the phone]. Please introduce yourself by giving your first name and then we’d like to know what your connection is to the WTCHP.

1. [Go around the room/phone] Could each of you please describe what your specific role is in the program, or how you interact with the WTCHP?

Mental Models: Research/Translational Research (20 minutes)

Thank you for those introductions. Now this first section focuses broadly on some scientific concepts, that you may or may not already be familiar with. We are interested in knowing what your own understanding is of these concepts. There is no right or wrong answer for any of these questions. We want to know what these concepts mean to you or sound like they mean, even if it’s the first time you are hearing a certain term or a phrase.

2. When you hear the phrase, "scientific research," what do you think of? What comes to your mind?

3. The WTCHP carries out many different activities, including monitoring and treatment of health conditions related to the 9/11 attacks. One of its main missions is to carry out research. Why do you think that the WTCHP includes a research mission?

4. Have you heard of the term or are familiar with the term, “translational research?” Regardless of how familiar you are with the term, what do you think of when I say “translational research?”

Probe:

[For researchers/NIOSH staff only]: are you familiar with any model(s) of translational research? [such as NIH’s T1-T4 model].

5. What is needed to make research “translational?” [refer to terms generated by the previous question, e.g., actionable, useful...]

Probes [potential constructs to explore, suggestions if needed]

• Identifying stakeholders/users of the research

• Creating evidence, studies, and data to support decisions

• Having clear next steps for the research to be acted upon

Translational Research within the WTCHP (45 minutes)

Next, we’ll discuss translational research that’s supported by the WTCHP.

The program focuses on several broad research areas, some of which were specified in the original Zadroga Act [the Act that established the WTCHP] and others that have emerged over time. The research areas include cardiovascular disease (or heart disease), cancer, mental health, and the others shown here: [show slide with the following topics displayed in a grid to avoid ranking them: adult mental health, cardiovascular disease, emerging conditions; note to moderator: includes Hepatitis C, autoimmune disease, cognitive function, kidney disease, among others; WTC youth, respiratory disease (or lung disease), and exposure assessment (or, what things in the environment people at the 9/11 site were exposed to). Show additional slide with these terms defined for reference].

6. What do you think about this list of research topics?

Probes:

• Are these the topics that are most relevant to you?

• Are these the topics that seem the most relevant to decision makers, or people who make policy decisions about the program itself?

• Are these the topics that seem most relevant to users of WTCHP research, or people who make decisions based on the findings of the WTCHP?

• What research areas do you think are missing from this list?

Now I’m going to show a slide that ranks the research topics by the amount of research funding that each has received from the WTCHP. [Show slide that ranks them as follows: 1. Respiratory disease, 2. Adult mental health, 3. Cancer, 4. WTC youth, 5. Cardiovascular disease, 6. Emerging conditions, 7. Exposure assessment].

7. What do you think about this ranking?

Probes:

• Do you agree with the ranking? Disagree?

• Any surprises?

• Why do you think research funding has been allocated this way?

• What would you change? And why?

Across all these research topics, the research supported by the WTCHP aims to answer a number of important questions regarding the health effects of the WTC attacks.

8. What do you think are the most pressing questions about the health effects of the 9/11 attacks? [suggestions if needed: Is a particular health condition more common among people exposed to the 9/11 attacks? How should survivors and responders be screened, monitored over time, and treated most effectively for certain health conditions related to the attacks? Mechanistic links between acute event and chronic conditions? And others.]

Are these questions being addressed by the research that the WTCHP supports?

If not, why do you think those questions aren’t being addressed?

9. Now that we’ve discussed some of the important research topics and questions, we’ll now talk about the use of this research. And when I say “research,” I mean that broadly, including the information that comes out of studies on the health effects of 9/11. Who is using, or who should be using the results of WTCHP research? How are the results being used? How should the results be used?

[Note to moderator: if the group does not identify many user communities, suggest: clinicians, WTCHP members, policy makers at NIOSH, other researchers; allow for an open-ended discussion; multiple suggested probes listed below if discussion is stalling]

Probes

• To what extent is this [the use of WTCHP-supported research] happening? To what extent do you find the research useful?

• Can you provide examples of when you, or a decision maker, used WTCHP research? Another way to think about this is: can you describe how WTCHP-funded research has contributed to better services, or outcomes, for members? [If additional prompts needed]:

• Improved recognition of WTC-related conditions—i.e., changes to the list of covered conditions

• Improved knowledge about how to screen for WTC-related conditions

• Improved knowledge of treatments for WTC-related conditions

• Better health care for members with chronic WTC-related conditions

• Improved member satisfaction

Thank you for your input. Now we’ll focus on the roadblocks to using this research. These barriers may be related to the research itself, or may be related to external factors such as political or financial pressures.

10. In the previous question, the group mentioned [restate answers to the previous question about who uses WTCHP research and how] What do you think are the major barriers that prevent the research from being used, in other words, acted on, by this/these group(s)?

[Follow-up questions related to the research itself. Follow each with “why/why not” and only ask what is relevant to the stakeholders in that focus group:]

• Do you trust the methods that generated the evidence?

• Is the research high-quality and rigorous? How do you determine this?

• [if not already covered earlier]: Is the research relevant to you (and your work)?

• [if not already covered earlier]: Is the research useful to you (and your work)?

• Is the research easy to understand, or overly complex?

• Are the important research results being appropriately disseminated or published?

[Follow-up questions on external factors:] Thinking about external factors:

• Are the research findings shared with the key people who need them? If not, why do you think this isn’t happening?

• Do those decision makers feel empowered to take action based on those findings? Why or why not?

• To what extent do you get pushback when trying to use the research to make changes or develop new practices? I’m talking here about making changes to the way members are cared for OR how the program works. [If so]: From whom do you get pushback? Why do you think there is resistance?

• [follow-up probe]: Are the actions to take based on the research considered not economically or politically feasible, not practical in the real world, or something else?

• [follow-up probe] Are there other competing demands or not enough resources to use the research effectively?

The Research-to-Care Model (with visual aid) (25 minutes)

We’ll now move on to discuss something called the Research-to-Care logic model that guides the WTCHP’s translational research efforts. This is the last section of the discussion.

11. Is the Research-to-Care model familiar to you? Who has heard of it?

[If in person, moderator passes around handouts with a blank logic model inserted, or if conducting a webinar, shows a slide with the blank logic model.] Research-to-Care is a logic model. A logic model is a diagram that shows you how a program is supposed to work. The first step of a logic model is to determine the inputs, or resources necessary for performing activities. Resources could be financial support, people, equipment, infrastructure, etc. The resources help support all the activities that people do within the program. So the next step in the logic model is the activities that take place. Activities could be research or sharing the results of research. The third step in the logic model is the outputs from the activities. The activities should lead to certain products or findings. Finally, everything in the logic model then should lead to short-, intermediate-, and long-term outcomes. These outcomes are really the goals of the program.

The WTCHP has already designed a Research-to-Care logic model and has filled in the inputs, activities, outputs, and outcomes. Before we share with you their model, we first want to hear your thoughts, as if you were designing the process from scratch.

12. We are interested in your ideas for the Research-to-Care model—how would you describe the steps in the process of doing research that leads to better care and outcomes for WTCHP members?

• What do you think should be the major inputs (or ingredients that support research being done within the WTCHP)?

• What, in your view, are the key activities? [if needed, clarify that these are not activities within the WTCHP as a whole, but specifically, activities that relate to the translation of research to better care for members—the research mission]

• What would you like to see as the main outputs of the WTCHP’s research mission (aka the Research-to-Care process)?

• What would you say are the key outcomes, or ultimate results of the research mission, and can you specify if these should be accomplished in the short-term or long-term, or somewhere in between?

[Following discussion, if in person, moderator now shows a large poster of the NIOSH Research-to-Care model and passes around handouts, or shows a slide with the model, if conducting a webinar.] Here is the current Research-to-Care model created by the WTCHP. As you can see, the model lists nine key inputs including groups of people like program members and researchers who are extramural (or outside the program); infrastructure like the Data Centers, Clinical Centers of Excellence, and other investments like the NYC 9/11 Health Registry. The Research-to-Care logic model lists seven important activities that the Program should focus on, such as, to conduct quality research and conduct and analyze health surveillance. The logic model lists ten specific outputs, seven of which are types of research findings shown in the first box. Finally, these inputs, activities, and outputs ideally lead to the outcomes shown here. This is what the WTCHP has come up with to describe the “research to care” process within the program. Now, we want to hear your thoughts on the WTCHP’s model so that we can make recommendations to NIOSH on how it might be improved.

13. What parts of this diagram are relevant to you? Does this model reflect your own experience with the program? Why or why not?

14. What is missing from the Research-to-Care logic model? [note to moderator: restate the categories of the logic model and take the discussion piece-by-piece]: Please think about the inputs, or who needs to be involved and what resources, technologies, and support those groups of people need; the activities, or what people need to do; and the outputs, or what the program should be producing.

15. Last question: all the way on the right here: what do you think about these outcomes? Is there anything that you would add to what’s shown here? Any you would change or take away?

Probes:

• Where would you say the WTCHP research mission is falling short? And why?

• Finally, what would you say is working well?

Concluding Points (5 minutes)

That concludes our prepared questions for this focus group.

16. Before we wrap up, is there anything else that has come to mind during the course of this discussion, or important points about research within the WTCHP that you wanted to bring up?

Thanks to you all for taking time to participate in today’s focus group. Your feedback has been incredibly informative and we appreciate your time. If you have any questions after we conclude, please email the study team at [email protected]. Finally, the timeline for this project spans the next few years or so. Some of you may be contacted in the future for follow-up questions. If you would prefer not to be contacted again for this project, please email us that as well. We’ll present preliminary findings from this work to different communities involved in the World Trade Center Health Program during the course of the project, and you will all be notified when our final report to NIOSH is completed. One last request: we would be grateful if you could all fill out a very brief, anonymous survey with some demographic questions. [If in person, hand out paper copies; if by webinar, state: We’ve just emailed you a link to the survey, which should take just a minute to fill out.] Thanks again for participating and have a great rest of your day.

8