0920-1279 Key Administrative Staff Discussion Guide

OMB Control No: XXX-XXX

Expiration Date:  XX/XX/XXX

Attachment D1. DISCUSSION GUIDE-
KEY ADMINISTRATIVE STAFF

This page left blank for double-sided copying.

WISEWOMAN Evaluation Site Visit Discussion Guide

Key Administrative Staff
(Program Director, Program Manager, Data Manager)

1. Introduction [5 minutes]

My name is [fill in]. Thank you for your time today. As we mentioned when we scheduled this interview, General Dynamics Information Technology (GDIT) and Mathematica Policy Research are supporting the Centers for Disease Control and Prevention (CDC) in conducting an evaluation of the WISEWOMAN program. The purpose of the evaluation is to find best practices in program implementation and develop evidence of the program’s effect on outcomes. This year, we are visiting [five/six/seven] funded WISEWOMAN programs across the country to meet with administrative staff and clinical and healthy behavior support partners.

This interview will take about 90 minutes. I would like to record the conversation as a back-up for our notes. We will keep the recording private and use it only for reference purposes for this project. We will not attribute any statements or quotes to you without permission. Is it OK for me to begin recording?

[BEGIN RECORDING]

1. First, briefly tell me a bit about yourself and your job. [If multiple respondents are present, collect this information from each person]

   1. What is your position at WISEWOMAN?

   2. What are your roles and responsibilities?

   3. How long have you been in this position?

2. Participant Reach [5 minutes]

Next, I am interested in hearing more about where your program operates and what your program is doing to recruit participants.

2. In which communities or regions does the program operate? [EQI.2]

1. How did you decide to provide WISEWOMAN services in these communities or regions?

2. Are there any communities or regions that are not currently served by your WISEWOMAN program but might benefit from access to WISEWOMAN services?

3. Do you have plans to offer WISEWOMAN services in these or other areas in the future?

3. What populations does your program target? [EQI.3, EQII.3, EQIV.3]

   1. Which of these would you consider as hard-to-reach and why? (for example, are there certain racial or ethnic sub-groups, or other groups of medically underserved women)?

   2. How did you decide to target these populations?

   3. How would you say your program is doing at recruiting and enrolling individuals from these populations?

4. How does your WISEWOMAN program recruit new participants? [EQI.2]

1. Does your program recruit new participants through community outreach (for example, at health fairs or other events in the community)?

2. Do other organizations or programs refer women to the WISEWOMAN program? If so, which organizations or programs? How often does this happen?

3. Does your program use any mass media outreach?

4. Can you tell us about any other outreach strategies you have used?

5. How does your program determine eligibility for new participants? [EQI.2]

   1. Does your program review the Breast and Cervical Cancer Early Detection Program (BCCEDP) list of women and follow up with eligible individuals?

   2. Is eligibility determined by the program before women attend screening visits?

   3. Does the program determine participant eligibility during office visits with clinical providers?

6. How content are you with your program’s outreach and recruitment efforts? [EQI.2]

   1. Would you say your program does a good job reaching its target population, or are there things that could be improved?

   2. Does your program reach the types of participants that it intends to? Or do you struggle to reach some sub-populations or types of participants? Why?

7. What types of innovative approaches is your program using to recruit participants? [EQI.2, IV.6]

   1. What types of approaches are you using to recruit hard-to-reach populations?

   2. Thinking about the strategies you use to recruit new participants, which do you think are the most successful?

   3. Which strategies are the least successful?

8. In your opinion, what makes some recruitment and outreach strategies work better than others? [EQI.2, IV.6]

3. Clinical Services [30 minutes]

Screening Sites

9. What types of clinical sites has your program partnered with to conduct WISEWOMAN screenings (for example, federally qualified health centers, local health departments, mobile clinics, etc.)? [EQI.4]

1. Are most WISEWOMAN providers embedded in primary care clinics, women’s health clinics, or some other type of clinic?

2. Can you tell us more about your experience forming partnerships with clinical sites? Was it easy? Hard? What did you do to overcome any challenges you encountered?

10. At how many sites does your WISEWOMAN program conduct screenings? [EQI.4]

1. Approximately how many providers conduct WISEWOMAN screenings at each site?

11. In what types of settings does your program deliver WISEWOMAN screening services (clinics, health centers, participants’ homes, community-based locations, etc.)?

1. [If screenings are conducted in non-traditional/ non-clinical settings] When did the program begin offering screening services in non-traditional locations? Approximately what percentage of screenings are conducted in non-traditional locations?

12. Have you found that you have had a more effective partnership with some WISEWOMAN clinics/ sites than others? If so, which types of sites have been easier to work with? More difficult to work with? [EQI.4]

13. Does your program have official contracts with these clinical sites? [EQI.4]

1. If not contracts, what other agreements are in place and how do these work?

14. How does your program reimburse clinical sites for WISEWOMAN services? [EQI.4]

1. Have you encountered any issues with this approach to reimbursement?

Screening process

15. What types of staff typically conduct screenings (physicians, NPs, nurses)? [EQI.1, IV.6]

    1. Are patient navigators involved in the screening process? If so, how?

16. Please briefly describe the typical sequence in which WISEWOMAN screenings are conducted. [probe on the following components] [EQI.1, IV.6]

    Component	Probes
    Blood pressure measurement	What types of staff measure participants’ blood pressure? How many measurements are taken? Can you tell us any challenges associated with collecting blood pressure measurements during WISEWOMAN screenings? How has your program addressed these challenges?
    Labs	When are labs drawn? When are lab results shared with participants? Can you tell us about any challenges associated with drawing labs and/or sharing results with participants? How has your program addressed these challenges?
    Health risk assessment	When do participants complete the health risk assessment form? What type of staff typically conduct the health risk assessment? Can you tell us about any challenges associated with completing health risk assessment forms? How has your program addressed these challenges?
    Risk reduction counseling	When is risk reduction counseling conducted? What type of staff typically conduct risk reduction counseling? Can you tell us about any challenges associated with conducting risk reduction counseling? How has your program addressed these challenges?
    Clinical referrals	What types of clinical referrals are made for WISEWOMAN participants who need additional medical attention (for example, cardiology, behavioral health, primary care, etc.)? Can you share examples of times when patients have received clinical referrals? Does your program track or follow-up on clinical referrals? If so, how? Does your program use patient navigators to assist with the clinical referral process – if yes, how?

17. How do provider sites identify participants with undiagnosed hypertension? [EQI.1, IV.6]

1. What proportion of your program’s clinical sites have formal protocols in place to identify participants with hypertension?

2. What has your program done to help sites develop and implement protocols for identifying patients with undiagnosed hypertension?

18. What are the biggest challenges in the screening process? How has your program addressed these challenges? [EQI.1, IV.6]

19. How do you think that the screening process could be improved? [EQI.1, IV.6]

Team-based care

We are interested in hearing how WISEWOMAN clinics are using team-based care to reduce participants’ risk of cardiovascular disease and manage hypertension.

[IF NEEDED: Team-based care is an approach that uses a multi-disciplinary team (including clinicians, nurses, medical assistants, health coaches, pharmacists, social workers, dieticians and other providers) to deliver clinical and health education services to participants. The goal of team-based care is for team members to provide support and share responsibilities of hypertension care to complement the primary care provider’s activities.]

20. What proportion of your program’s WISEWOMAN providers/sites use a team-based care approach? [EQ I.1]

21. Do you provide any training or technical assistance to WISEWOMAN providers on use of team-based care? [EQ I.1]

1. If so, can you tell us more about the types of training/TA provided?

2. Do you think different or additional training/TA would be useful to providers?

3. What else is your program doing to increase the number of WISEWOMAN providers/ sites using a team-based care approach?

22. Who are the different members of the care teams that work with WISEWOMAN participants? [EQ I.1]

1. What is the role of each of the different members?

23. How do team members communicate with patients and one another to coordinate care? (for example, communicate in person, through EHR messaging, care team meetings/ daily huddles, by phone, etc.) [EQ I.1]

    1. In general, how effective is this approach to coordinating care?

24. Can you tell me about how WISEWOMAN providers/sites use a team-based care approach to help participants manage hypertension? [EQ I.1]

2. Is a team-based approach used to provide care for all patients at the sites, or just for WISEWOMAN participants?

25. Can you tell me about any challenges that your providers/sites have experienced using a team-based care approach? [EQ I.1]

1. Have providers/sites experienced turnover in care team staff? If so, how have you helped providers maintain continuity in the team-based care approach despite staff turnover?

26. In your opinion, has using a team-based care model helped your program serve participants? If so, how? If not, why? [EQ I.1]

    1. In your opinion, has using a team-based care model helped WISEWOMAN clinics work with participants to manage hypertension? If so, how? If not, why?

    2. What factors have helped clinics use a team-based care model successfully?

Hypertension self-management

The next questions are about how your program encourages hypertension self-management through medication adherence and blood pressure self-monitoring.

27. What is your program doing to encourage participants to regularly monitor their blood pressure? [EQ I.1, EQIV.5]

1. Do you offer programs that encourage blood pressure self-monitoring with clinical support? Who is eligible for these programs?

2. Has your program purchased blood pressure monitors for participants to use at home?

3. Do you provide tracking sheets to help participants self-monitor their blood pressure with clinical support?

4. What challenges have you encountered in trying to encourage participants to self-monitor their blood pressures and share results with their provider?

28. What is your program doing to help participants take medication as prescribed? [EQ I.1, EQ I.4, EQIV.5]

1. Have you identified partners that offer Medication Therapy Management services?

2. Do your providers refer participants to these services?

3. What else are you doing to encourage medication adherence?

4. What strategies have you found to be most effective for encouraging participants to take medication as prescribed?

29. Does your program partner with any organizations or programs that help provide affordable medications to participants? [EQ I.4]

1. With what types of organizations or programs does your program partner?

2. Can you tell us more about why your program partners with these organizations or programs?

3. In what way or ways do they help you improve access to affordable medications?

4. Healthy Behavior Support Services (HBSS) [20 minutes]

Now let’s talk about your program’s referrals to lifestyle programs, health coaching, and other community-based resources.

HBSS Referrals

30. In general, how does your program determine which participants should be referred to healthy behavior support services, such as lifestyle programs, health coaching, and/or other community-based resources? [Interviewer note: this question is intended to get at broad methods for referring patients. We ask about specific criteria for each program later.] [EQ I.1, IV.4]

1. Do clinical staff decide who to refer to healthy behavior support services?

1. If so, which staff make these referral decisions?

2. Do clinical providers/sites use WISEWOMAN data or electronic health record data to identify participants who could benefit from specific healthy behavior support services?

1. If so, can you provide more detail about how this works?

31. How do clinical providers/sites determine which participants will be referred to lifestyle programs versus health coaching? What about community-based resources?

32. Does your program encourage healthy behavior support service providers to refer women to the WISEWOMAN program? [EQ I.1, IV.4]

1. If so, how do you encourage partner organizations to refer women to WISEWOMAN?

2. Do you have any processes in place to make this easier?

3. How often does this happen?

33. Does your program conduct follow-up assessments for women who complete health coaching programs or lifestyle programs? [EQ I.1, IV.4]

1. How are these completed (in-person, over the phone, etc.)?

2. Who is involved in conducting follow-up assessments?

3. What challenges does your program face in completing follow-up assessments in a timely manner?

Health Coaching

Next I have a few questions specifically about health coaching.

34. Can you tell me more about how your program ensures that WISEWOMAN participants have access to health coaching? For example, does your program employ centralized health coaches, or require clinics to identify and provide an on-site health coach? Or do you have some other type of arrangement in place? [EQ I.1]

1. What types of contracts or agreements do you have in place with health coaches?

35. How do clinical providers/sites determine who to refer to health coaching? [EQ I.1]

36. How do women usually react to health coaching? [EQ I.1. IV.3, IV.5]

1. Do participants usually complete health coaching programs?

2. Do they provide any feedback about their experiences with health coaching?

3. How do participants’ experiences with health coaching vary? (for example, do participants who complete health coaching programs share different characteristics than participants who do not complete programs – either by racial/ ethnic group, CVD risk group, etc.?)

37. Can you tell me about any strategies your program uses to help keep women engaged with health coaching? [EQ I.3]

Lifestyle Programs

Now I have a few questions about the types of lifestyle programs that are available for participants.

38. What types of lifestyle programs are offered to participants? [EQ I.1, I.4]

[PROBE FOR EACH LSP DESCRIBED]

1. What types of services does [LSP] offer participants?

2. Is [LSP] available to all eligible participants, or is it only offered for participants in certain communities/ regions?

3. Can you tell me more about the criteria that clinical providers/sites use when referring women to [LSP]?

4. Does [LSP] use a standard curriculum?

5. Has [LSP] been tailored (or have the ability to be tailored) to meet the needs of the WISEWOMAN participant population? (for example, can sessions be conducted in different languages or otherwise be adapted to be culturally appropriate)

6. What types of contracts or agreements do you have in place with [LSP]?

39. Which lifestyle programs does your program refer women to more frequently? Less frequently? [EQ I.1]

1. Why do you think women are or are not referred to these programs as often?

40. How did your WISEWOMAN program originally select the lifestyle programs that you now offer to participants? [EQ 1.4]

1. What challenges did you face when identifying and partnering with organizations that offer lifestyle programs? How did you address these challenges?

2. What factors helped your program identify and partner with these organizations?

41. Are there any additional lifestyle programs that you would like to offer to participants? [EQ 1.1, 1.4]

1. Will these programs be offered in the future?

42. Can you tell me about any strategies your program uses to help keep women engaged with lifestyle programs? [EQ 1.3]

43. How do women usually react to these lifestyle programs? [EQ I.1. IV.3, IV.5]

1. Do participants usually complete lifestyle programs?

2. Do they provide any feedback about their experiences with lifestyle programs?

3. How do participants’ experiences with lifestyle programs vary? (for example, do participants who complete lifestyle programs share different characteristics than participants who do not complete programs – either by racial/ ethnic group, CVD risk group, etc.?)

Community-based resources

We understand that WISEWOMAN programs may also refer participants to resources in the community, outside of the health coaching and lifestyle program offerings.

44. Are WISEWOMAN participants referred to other community-based resources, such as the state Quitline or other resources to support healthy behaviors? [EQ I.1, I.4]

1. To what types of community-based resources does your program refer participants?

2. Can you tell me more about the criteria that your program uses when referring women to community-based resources?

3. Do you track referrals to these resources?

4. Does your program conduct follow-up to see if women who were referred actually used these resources?

45. Which types of community-based resources does your program refer women to more frequently? Less frequently? [EQ I.1, I.4]

1. Can you tell me about some of the reasons why you think women are (or are not) referred to these resources?

46. Can you tell me how the program originally selected the community-based resources that you now offer to participants? [EQ I.4]

1. What challenges did you face when identifying and partnering with organizations that provide community-based resources? How did you address these challenges?

2. What factors helped your program identify these organizations and partner with them?

47. Are there any additional community-based resources that you would like to offer to participants? [EQ I.4]

1. Will these resources be offered in the future?

5. Participant Engagement [5 minutes]

Next, we have a few questions about ways your program keeps participants engaged in WISEWOMAN services.

48. You’ve mentioned several strategies that your program uses to keep women engaged in WISEWOMAN services (such as health coaching and other healthy behavior support services), and to ensure that women continue to return for annual screening visits. What other strategies do you use to engage participants? [EQ 1.3] [Interviewer note: probe on strategies other than those mentioned previously. If no other strategies are used, skip to next question]

1. Do you offer incentives for participation?

2. Do you use social media to keep participants updated on program offerings?

3. Do you provide supports (such as transportation or child care) to help participants attend in-person sessions or get to appointments?

4. Do you send reminders when women are due for their annual screening visit?

5. Do you use any other strategies to engage participants?

49. Are there sub-populations of participants that are more difficult to engage in the WISEWOMAN program (for example, who are less likely to complete HBSS referrals or return for annual visits)? [EQ 1.3]

    1. What strategies have you used to engage these populations?

50. Which strategies have been most effective for engaging women? [EQ 1.3, IV.6]

51. What have been the biggest challenges in engaging women in the program? [EQ 1.3]

6. Data and evaluation [10 minutes]

Next, I’d like to ask you a few questions about your program’s collection and use of data.

[Note to interviewer: Update to reference grantee-specific data system/processes if we have this information.]

52. How does your program use WISEWOMAN data? [EQ I.1 I.4, IV.4]

POSSIBLE PROBES:

1. To identify participants who could benefit from referrals to healthy behavior support services?

2. To track and monitor clinical measures? (If so, which measures?)

3. To follow-up with participants with uncontrolled hypertension?

4. To encourage referrals between community partners and the WISEWOMAN program? [If needed, explain that we are interested in if/how data is used to encourage bidirectional referrals between community partners and the WISEWOMAN program]

53. What percentage of your WISEWOMAN clinics/ sites use EHRs to manage WISEWOMAN data? [EQ I.1, I.4, IV.4]

1. Do providers use the same EHR systems or do systems vary by provider?

2. Do providers use EHR data to identify women who could benefit from the WISEWOMAN program but are not currently participating?

3. What are the benefits of using EHRs to collect and manage WISEWOMAN data?

4. What are the challenges of using EHRs to collect and manage WISEWOMAN data?

5. Does your program have access to clinic/site EHR data? If so, how do you use these data? Is the EHR integrated with the data system your program uses to manage and report MDE data?

54. In general, what challenges does your program face collecting WISEWOMAN data (including MDE data and other data)? [EQ I.1]

55. What data does your program currently collect in addition to minimum data elements (MDEs)?

56. In which ways does your program use the data that it collects? (Probe on MDE data and other data sources) [EQ I.1. IV.4]

1. Is it for program monitoring, evaluation, quality improvement, program improvement, and so on?

2. Can you provide a few specific examples of ways your program is using the data you collect?

7. Staffing and professional development [5 minutes]

Now we will talk about the staff employed by your WISEWOMAN program and the professional development opportunities that you offer for staff.

57. How many staff members are employed by your program? Include contract staff, such as those who help you develop and maintain your data system and provide evaluation support. [EQI.1]

    1. Do you share an evaluator with another program (such as your 1815 or 1817 grant program)?

58. Including the staff employed by your program and those employed by lifestyle program (LSP) and health coaching (HC) providers, would you say you have enough staff to cover the needs of all participants? [EQ I.1, I.5]

59. How often are staff members required to receive in-house or external trainings? Include both those employed by WISEWOMAN and LSP and HC providers. [EQ I.1]

1. How are topics for training and professional development identified?

2. Are joint trainings ever offered with other organizations in the community?

60. To what extent do staff find professional development to be helpful and effective (either through evaluations or through word of mouth)? [EQ I.1]

61. What are the challenges in finding staff with the right skills or retaining them? [EQ I.1, I.5]

1. How often do staff turn-over?

2. Does this vary by type of staff?

8. Community [5 minutes]

Let’s talk about the communities in which your program operates.

62. Can you describe the demographic, cultural, and linguistic characteristics of your target population? [EQ 1.6, II.3]

1. What languages are most common?

2. In addition to language, are there any special cultural considerations for any populations your program serves?

3. What different cultural approaches do you take for each population that your program serves?

63. What are the biggest barriers to cardiovascular health in the communities where your program operates? [EQ 1.6, II.3]

1. Are there any resources in the community that help promote cardiovascular health?

64. From your perspective, has the WISEWOMAN program helped address disparities in cardiovascular health based on race or socioeconomic status? [EQ II.3]

1. Why or why not?

65. Based on your interactions with WISEWOMAN participants, does it seem that they value WISEWOMAN services? [EQ II.3, IV.5]

66. Does it seem that women engage in healthier behaviors after they enroll in the WISEWOMAN program? [EQ II.2, IV.1, IV.5]

1. Are participants more likely to take medication as prescribed?

2. Are participants more likely to monitor their blood pressure regularly?

67. Can you provide any stories or anecdotes of how the WISEWOMAN program has affected women’s health or behaviors? [EQ IV.1, IV.5]

9. Wrap-up [5 minutes]

We are almost done. We will ask you a few more questions about your perceptions of your WISEWOMAN program’s greatest strengths and weaknesses.

68. From your perspective, what have been the greatest strengths of the WISEWOMAN program in your community? [EQ II.2, II.3, IV.5, IV.6]

1. How much of an impact do you think the WISEWOMAN program makes on the community?

69. What have been the biggest roadblocks to the WISEWOMAN program’s success? (include internal factors like staff/ resources, and external factors like policy changes) [EQ I.5, I.6, II.2, II.3, IV.5, IV.6]

1. What resources would help the WISEWOMAN program become more successful?

70. Of the strategies or processes that you’ve implemented for WISEWOMAN, which do you think could be sustained after the cooperative agreement ends? [EQ IV.7]

For example:

1. Referral networks between clinics and community-based resource providers

2. Team-based care protocols for hypertension control

71. Are there any especially interesting or innovative approaches that your program or the providers you have worked with have used in providing WISEWOMAN services? [EQ IV.6]

72. What else would you like to share about your experiences with the WISEWOMAN program?

1. What advice would you give to another program just starting out in this work?

73. Is there anything else you would like to share with us?

Thank you for taking the time to speak with us today. If you have any additional questions, please feel free to reach out to us.

Public Burden Statement: Public reporting burden of this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid Office of Management and Budget control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Centers for Disease Control and Prevention/Agency for Toxic Substances and Disease Registry Information Collection Review Office, 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-XXXX).