0920-1027 NPIN Evaluation Moderator Guide - Managers

Form Approved

OMB No. 0920-1027

Expiration Date 07/31/2020

NPIN Evaluation Moderator Guide - PUBLIC HEALTH PROFESSIONAL SUPERVISORS/LEADERSHIP

Public reporting burden of this collection of information is estimated to average 2 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; Attn: OMB-PRA (0920-1027)

Guide for Conducting Group Interviews

puBLIC HEALTH PROFESSIONAL Supervisors/Leadership

I. Welcome and Introduction

Thank you for coming today to our session on the National Prevention Information Network, better known to some as NPIN. I’m Mark Herring, and I’m from Schlesinger Associates, a marketing research organization. The Centers for Disease Control and Prevention (CDC) is sponsoring this research.

The purpose of today’s focus group is to hear your views and opinions in order to make it a stronger service and to better meet your prevention information needs. It’s been several years since NPIN was last evaluated, and we know that the landscape of health communication and health technology has changed considerably. The purpose of this effort is to evaluate the awareness, perceptions, and utilization of NPIN services from public health professionals like you who use the platform.

Your insights are very important to us and we appreciate your time today. Please be honest with your feedback. We would like for you to offer both positive and negative feedback in order to help us shape NPIN for future users.

This focus group will last approximately two hours.

Before we begin, I want to review a few ground rules for our discussion.

• Most importantly, there are no right or wrong answers. We want to know your opinions and what you think about the issues we will be discussing. I do not work for the people who are sponsoring this research, so don’t hold back from giving me your honest opinions.

• You have probably noticed the microphones in the room. They are here because we are recording these sessions, and they will be transcribed for analysis. Please speak loudly and clearly. I want to give you my full attention and not have to take a lot of notes. At the end of our discussion, I have to write a report and will refer to the recording when doing so.

• Behind me is a one-way mirror. Some of the people working on this project are observing this discussion so that they can hear your opinions directly from you and take notes so that your opinions are accurately captured. However, your identity and anything you personally say here will remain secure. Your name, address, and phone number, which only the facility knows, will not be given to anyone, and no one will contact you after this interview/group is over.

• To protect your confidentiality, we will assign each participant a number. Be sure to only use participant numbers when addressing other members of the group. Please do not use any names.

• We ask that each participant respect each other’s privacy, as well. Please do not share what was said in today’s group with others after the group is over. Please be respectful of others’ opinions. Any participant who is disrespectful to others or disruptive to the group will be asked to leave.

• Please turn your cell phone to vibrate or silent mode.

• Do we have your permission to record today’s session? If so, please respond with “yes.” [MODERATOR WAITS FOR CONSENT] Thank you.

• Do you have any questions before we begin?

Before we begin our discussion, let’s do some very brief introductions. We know your first name from the tent card in front of you on the table.

• Please tell me your favorite hobby and a show you’re currently watching on Netflix, Hulu, Amazon Prime, etc., and what you like about. If you’re not a big TV person, tell us about a book you’re reading.

II. Awareness of NPIN and Brand Perceptions

Let’s talk about some of your general knowledge, awareness, and thoughts about NPIN.

• How did you first become aware of NPIN?

Probes:

• At a conference

• Email list

• Website search, such as Google

• NPIN social media

• Through training

• From what you know, what services does NPIN provide?

Probes:

• Sending prevention information

• Training Opportunities

• Awareness day information and updates

• Connecting prevention professionals

Now, let’s talk about NPIN and what that brand represents to you.

• From your experiences, what words do you associate with NPIN?

[NOTE: Moderator elicits word and explanation from each respondent individually]

Let’s discuss your responses.

I’d like you to write down three words or short phrases that describe positive qualities of NPIN. After that, I’d like you write down three words or phrases that describe negative qualities of NPIN.

Let’s discuss the positive and negative qualities you all put down.

• What made you write these positive qualities about NPIN?

• What made you write these negative qualities about NPIN?

Ok, let’s move on.

• How many of you perceive NPIN as a primary resource for [HIV/AIDs, STDs, Viral Hepatitis, TB, adolescent sexual health information – ask each topic in turn]?

• What is NPIN doing well, since you consider it a primary resource?

Probes:

• The resources NPIN provides are useful to your work

• It’s a highly trusted source of prevention information

• NPIN keeps you consistently updated with the latest prevention information

• For those of you who do not perceive NPIN as a primary resource on this topic, what could the organization do differently to become a more valuable resource for you regarding [topic]?

Probes:

• Reach out more directly to public health professionals

• Increase marketing efforts

III. Leadership Roles, Responsibilities, and Resource Needs

Today’s discussion is focused on leadership-level public health professionals, and we are interested in how NPIN can better support you in your program management efforts and activities.

• What are some of your main roles and activities in managing disease prevention programs/organizations? [MODERATOR MAKES A LIST ON THE BOARD]

Probe:

• Setting/following policy surrounding prevention activities

• Administrative activities

• Strategy development

• Organizational strategy

• Program/project management

• What types of resources do you find to be most useful in supporting you in your prevention work?

Probe:

• Policy-related information

• Relevant guidance documents

• Templates

• Strategic plan guides

• Relevant disease data

• Program planning tools

• Toolkits

• Are there resources you need that might be specific to the managerial activities we described? [MODERATOR REFERENCES LIST]

  • What are they?

• Where do you obtain the resources you need to support your managerial activities?

Probes:

• Internally

• From a specific website or organization

  • Which?

• From existing standardized templates of specific resources

NPIN PEFORMANCE RELATED TO MANAGERIAL ACTIVITIES

• Do you feel NPIN is a platform for managers of prevention programs at your level?

Probe

• Why or why not?

• In what specific ways does NPIN support you regarding any of the managerial responsibilities we described earlier? [MODERATOR REFERENCES THE LIST]

Probes:

• Providing prevention/disease policy updates from CDC

• Guidelines that help you set team/organizational agenda

• Resources that support your program/project management efforts

• If NPIN does not support you in any of these managerial responsibilities we discussed, what are some ways NPIN could better do this?

We know that many of your activities are collaborative and take place with your team or staff, so we wanted to know a little about that part of your work.

Organizational and Team Resource Needs

• What are some ways you engage with your teams to support your prevention programs?

Probes:

• Providing/implementing necessary guidelines (e.g., organizational, state, or federal guidelines) to direct work

• Providing/implementing policies surrounding the work

• Collaborating to strategize, develop, or implement prevention programs

• What are the credentials of your team members?

Probe:

• Epidemiologists?

• Information Specialists?

• Health Educators?

• Has your team or staff mentioned NPIN to you as a place they go for disease prevention resources?

• In what capacity have your teams or staff discussed NPIN with you?

Probes:

• Website they go to for prevention resources

• Organization that held a training

• Have your staff expressed a need for any prevention resources that NPIN might be able to provide?

Probes:

• Fact sheets

• Templates

• Strategic plan guides

• Relevant disease data

• Program planning tools

• Toolkits

• Training

• Something else

• Are there any resources that your organization needs that NPIN could provide to better support your prevention efforts?

Probes:

• Policy-related resources

• Guidance documents

• Organizational strategic planning

• Program planning

• Program implementation

• Campaigns-related resources

• Communication strategies

• Evaluation

NPIN WEBSITE

The primary location to find NPIN prevention resources is on its website. We would like to get some of your feedback on the website. [MODERATOR PULLS UP NPIN.CDC.GOV SITE]

• Who has visited the NPIN website before? [MODERATOR GETS A SHOW OF HANDS]

• What are your overall impressions of the NPIN website?

Probes:

• Has information on disease topics relevant to my work

• Seems easy to use

• Prevention information seems disorganized

• Information seems out of date

• Too much information/hard to navigate

• What circumstances have made you visit the NPIN website?

Probe:

• NPIN website is a primary source for your work

• It came up on a Google search

• You arrived there through an email link

• Arrived through social media

• When you go to the NPIN website, what actions do you take?

Probe:

• Look for specific prevention resources

• Browse for general prevention information

• Look for certain updates

• What actions have you taken in the past on the NPIN website?

Probe:

• Looked for specific resources

• Browsed for general prevention information

• Looked for certain updates

• Do you feel that the NPIN website helps you with regard to the managerial responsibilities we discussed earlier? [MODERATOR REFERS TO LIST]

• How could NPIN improve its website to better meet you managerial/supervisory needs?

Probe:

• Provide a specific section on the site for managers/supervisors

• If NPIN were to develop a specific section for managers or your role, what types of resources would be most useful to locate there?

Probes:

• Policy-related information

• CDC guidance materials

• Disease prevention program management tools

• If the NPIN website had such a space devoted to managers, what would be the most effective way to call attention to it?

Probes:

• Place a call-out box on the website directed to managers

• Develop a mailing list specifically for program managers

VI. Awareness of Specific NPIN Resources and their Usefulness to Leadership Work

In the next section, we are going to discuss some of the resources and services that NPIN offers. As a reference, I’m going to give you a list of NPIN resources and services with their descriptions. We don’t expect you to have experience with everything we’re going to review.

Newsletters, Announcements, and Dear Colleague Letters

I’d like us to start by reviewing some materials that are developed and distributed by NPIN. Each of these is sent to subscribers electronically.

Connections

[NOTE: Moderator distributes hard copy of Connections]

This is a recent copy of Connections. You can see that this is a bimonthly one-page newsletter. It’s delivered electronically.

• How many of you currently subscribe to the Connections newsletter? [MODERATOR GETS A SHOW OF HANDS]

• What are your perceptions of the Connections newsletter?

• How might the newsletter benefit you in the managerial efforts and activities we listed earlier? [MODERATOR REFERS BACK TO LIST OF LEADERSHIP EFFORTS DEVELOPED BY THE GROUP]

Probes:

• Highlight new disease data that could inform strategy

• Highlights from the Director show direction for CDC

• Reinforce the importance of your organization’s prevention efforts

• What changes or improvements do you recommend?

Funding Announcements

[MODERATOR DISTRIBUTES SAMPLE FUNDING ANNOUNCEMENT]

Funding announcements are regular email announcements with descriptions of and links to funding and grants information.

• Who receives funding announcements from NPIN? [MODERATOR GETS A SHOW OF HANDS]

• How might the funding announcements benefit you in the managerial efforts and activities we listed earlier? [MODERATOR REFERS BACK TO LIST OF LEADERSHIP EFFORTS DEVELOPED BY THE GROUP]

• How have the funding announcements helped to support your team in their work?

• Is there anything you’d recommend be changed or improved about these funding announcements?

Dear Colleague Letters

Dear Colleague letters are email announcements sent out periodically to inform on topics such as new data or special dates for the public health community.

[NOTE: Moderator distributes hard copy of Dear Colleague letter]

• Who receives Dear Colleague letters? [MODERATOR GETS A SHOW OF HANDS]

• How might the Dear Colleague letter support you in the managerial efforts and activities we listed earlier? [MODERATOR REFERS BACK TO LIST OF LEADERSHIP EFFORTS DEVELOPED BY THE GROUP]

Probe:

• Highlight new disease data that might inform strategy

• Provide information that may help organization align with CDC goals or objectives

• Reinforce importance of your organization’s prevention efforts

• What improvements would you suggest to the Dear Colleague letters to make them more useful to your work?

VII. Databases

Now, we will discuss some of the NPIN databases. One is a database of organizations, a second is a resource database showing available materials, and the final database offers campaign resources.

Organizations Database

First, let’s talk about the NPIN Organizations Database. Public health professionals like you can use the Organizations Database to find information about public health and health organizations, including their locations and the types of services they provide.

• How many of you have accessed the Organizations Database? [MODERATOR GETS A SHOW OF HANDS]

• If you accessed the database, were you satisfied with your experience using the Organizations Database?

Probes:

• You find the organizations listed are relevant to your work

• Organization contact information is up to date

• How might the Organizations Database support you in the managerial efforts and activities we listed earlier? [MODERATOR REFERS BACK TO LIST OF LEADERSHIP EFFORTS DEVELOPED BY THE GROUP]

• Are there any specific gaps that you feel the NPIN Organizations Database could address regarding your work?

• Can you suggest any changes to improve this database – either the site itself or the information provided about each organization?

Prevention Resources Database

Now, let’s discuss the Prevention Resources Database. This section contains strategic plans, program descriptions, research papers from CDC and partner organizations, and more.

• Who has accessed the Prevention Resources Database? [MODERATOR GETS A SHOW OF HANDS]

• What are you overall perceptions of this database?

Probes:

• Seems to have resources to support my work

• Easy/difficult to navigate

• How might the Prevention Resources Database support you in the managerial efforts and activities we listed earlier? [MODERATOR REFERS BACK TO LIST OF LEADERSHIP EFFORTS DEVELOPED BY THE GROUP]

Probe:

• Supplying documents that I can use in my work

• Supplying documents to provide to my team for prevention work

• What changes or improvements might make this database more useful to your work?

Campaigns Database

The Campaigns Database is where NPIN partners like you can find prevention campaign materials, like banners, graphics, and other campaign content, from campaigns like Act Against AIDS. Here is an example of the Campaigns Database. [SHOWS CAMPAIGNS DATABASE]

• How many of you have accessed the Campaigns Database? [MODERATOR GETS A SHOW OF HANDS]

• How might the Campaigns Database support you in the managerial efforts and activities we listed earlier? [MODERATOR REFERS BACK TO LIST OF LEADERSHIP EFFORTS DEVELOPED BY THE GROUP]

Probe:

• Supplying prevention campaign resources that my organization may use

• What could make the NPIN Campaigns Database better?

Probes:

• Different types of campaign materials

• Different layout on the site

Learning Opportunities

Webinars/Webcasts/Podcasts

Now, let’s discuss NPIN’s webinars. NPIN supports the delivery of informative webinars on topics relevant for public health partners, such as disease updates...

• How many of you have ever attended a webinar hosted by NPIN? [MODERATOR GETS A SHOW OF HANDS]

• How did you find out about the webinars?

Probe:

• Email

• Social media

• Invited directly by CDC Division

• Was the information you received through the webinar beneficial to your work?

• How could you use NPIN webinars to support your organizational strategic goals? Program goals?

• What would motivate you to encourage your team or staff to attend an NPIN webinar?

Probe:

• The relevance of webinar topic

• Credentials of the presenter

• Surveillance or data updates

• Demonstration of new prevention tools

• Hearing from other prevention organizations

Training Programs

Next let’s talk about training programs. NPIN provides and facilitates trainings on areas such as disease topics, social media evaluation, and health literacy.

• How many of you have ever attended an NPIN training? [MODERATOR GETS A SHOW OF HANDS]

• What topic did the training cover?

• How could you use the NPIN trainings to support your organizational strategic goals? Program goals?

• What improvements might be made to address your leadership efforts more directly?

• What would motivate you to encourage your staff or team to attend more NPIN trainings?

Probes:

• Knowing they could earn CEUs

• Knowing the training schedule further in advance

• Knowing more about the trainer and their credentials

• What training topics would you like to see NPIN provide in the future?

Probes:

• Health communication

• Health literacy and education

• Social and digital media

• Partner engagement and collaboration

• Technology

• Using and communicating effectively with data

• Trainings related to your specific disease topic

VIII. Other web activities

NPIN has multiple sites to support other aspects of prevention, including testing locators and websites to support health observance days. One is Get Tested, and another is the National HIV Testing Day (NHTD) website.

• How important is having websites like these that connect your constituents to local STD or HIV testing resources?

Get Tested Website

Now let’s talk briefly about the Get Tested website.

• How many of you have visited the Get Tested website before? [MODERATOR GETS A SHOW OF HANDS]

• How have you or your team used the site to benefit your work?

Probe:

• Did you link to the site on your organization’s page?

• How does having this type of locator resource help shape your prevention strategies?

• What is your general impression of the site?

Probe:

• Strengths?

• Weaknesses?

• How could the Get Tested website support the managerial efforts and activities that we discussed earlier?

National HIV Testing Day Site

As we all know, health observance and awareness events are useful to help bring attention to a particular health topic at one time during the year.

• Are health observance or awareness events important to you as a program manager?

• Which health awareness events are most important for your team/organization?

Probe:

• NHTD

• STD Awareness Month

• NBHAAD

• Are there any observances that need more support than others?

• How best could NPIN support your organization in conducting your observance activities?

One website NPIN has that supports observances is the National HIV Testing Day website. This is a page for prevention organizations and partners to promote their NHTD activities.

• How many of you have visited the NHTD website before? [MODERATOR GETS A SHOW OF HANDS]

• For those of you who have been on the site, how have you used it to benefit your work?

Probe:

• Did you share the site link with constituents?

• Did you link to the site on your organization’s page to promote NHTD?

• Did you upload your organization to the NHTD site to share testing day activities for the observance?

• In your opinion, what are the strengths of this site?

Probes:

• Great resources for NHTD

• Functionality: the way the site identifies and geo-locates organizations with NHTD activities

• Great to see other organizations that are participating

• In your opinion, what are the weaknesses of this site?

Probes:

• Would like more content

• Functionality: the way the site identifies and geo-locates organizations with NHTD activities could be improved

• How could these sites be improved to support the leadership efforts we discussed above?

Social Media Activities

Finally, NPIN also has a social media presence.

• How many of you use social media in your day-to-day work? [MODERATOR GETS A SHOW OF HANDS]

• Which social media channels do you use most in your work?

Probes:

• Twitter

• Facebook

• Instagram

• How do you use social media in you work?

Probes:

• Send out prevention messages to constituents

• Connect with other prevention organizations

• Stay up to date on the latest prevention information

• Is it important to you that your organization engage on social media?

• How have you engaged with NPIN social media?

Probes:

• Followed one of the NPIN social media handles

• Retweeted NPIN tweets

• Reposted Facebook messages

• Participated in an NPIN Twitter Chat

• Signed up for an NPIN Thunderclap

• Is it important to you that an organization like NPIN communicate to you through social media? Why or why not?

• Where do you think it’s most important for NPIN to have a social media presence – Facebook, Twitter, Instagram, LinkedIn, or other?

XI. Wrap-Up

In thinking of all of the resources we’ve discussed today:

• Is there some targeted effort that should be considered in providing information specifically for public health partners who manage programs?

Probe:

• Provide specific training for program managers

• Targeted communication

• Email messages that highlight certain information that may support manager activities

In closing, do you have any last thoughts you’d like to share?

Excuse me for one moment while I see if the people observing have any questions that I have not asked. I will be right back.

[NOTE: Moderator steps out to check to see if there are any additional questions]

Thank you for your participation. Please leave all your worksheets and other materials on the table. We will be using your written feedback as we write our final report. Have a good day/evening.

25