CMS-10436 Protocol - Networks

MULTI-PAYER ADVANCED PRIMARY CARE PRACTICE DEMONSTRATION
EVALUATION
INTERVIEW PROTOCOL
Community-based Care Networks

PRA Disclosure Statement
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a
collection of information unless it displays a valid OMB control number. The valid OMB control
number for this information collection is 0938-XXXX. The time required to complete this
information collection is estimated to average [Insert Time (hours or minutes)] per response,
including the time to review instructions, search existing data resources, gather the data needed,
and complete and review the information collection. If you have comments concerning the
accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS,
7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore,
Maryland 21244-1850.

1

Background for Interviewers
NC-CCN is physician led at each level of its infrastructure: (1) the Board of Directors,
(2) management, (3) networks, and (4) practices. NC-CCN is governed by the Board of Directors
and its four committees: Information Technology, Nominations and Bylaws, Finance, and
Governance. Other leadership positions include members of the executive committee, the
president, and the executive director. NC-CCN is structured around four key programs: the
Information Center, Clinical Programs, Pharmacy Programs, and Administration.
Each community-based network has a clinical director who is a part-time physician that is well
known to the community, meets with physicians to encourage provider participation, provides
oversight to quality improvement in practices, and serves on the state clinical directors’
committee. In addition, all of the networks have network directors who manage daily operations,
care managers who coordinate services for enrollees, and pharmacists who provide medication
management for the most complex patients. All networks plan to hire psychiatrists to assist in
mental health integration.
NC-CCN builds on the 14 community-based networks that the CCNC originally established.
NC-CCN provides technical support to the networks to help identify the patient population and
reporting performance measures. These networks work together and with the state government to
define, track, and report performance measures that gauge the effectiveness of participating
physicians in achieving quality, utilization, and cost objectives. Currently, the CCNC networks
include more than 4,000 physicians in all 100 NC counties. The Medicare demonstration
includes 26 counties within 8 networks.
Community Care Networks and physician practices work together in a number of different ways.
They help patients in transitioning between care settings, such as from the hospital to home and
ambulatory care; assist patients with complex medical and social conditions; reduce medication
problems; support patients and families in the self-management of their diseases; strengthen links
among community providers; and enhance the ability of physicians to manage patients with
chronic conditions.
The networks are established based on a “one size does not fit all” principle, which allows
flexibility and leadership at the community level. NC-CCN does not mandate specific
intervention components for all communities, given that the needs of individual communities are
quite diverse in terms of physician interest and expertise in quality improvement programs, 11
staffing and financial capacity of network practices, interest and compliance of participating
patients, and prevalence and severity of medical conditions. All 14 networks have standard
expectations and are held accountable to the same performance measures for quality, cost, and
utilization.
Some networks built on existing community organizations while others were established as new
entities. The networks make decisions locally about the physician practices with which to
partner, because each county has a different delivery system, culture, and politics. Each local
network has a different relationship with the statewide central NC-CCN office. Local networks
have a strong role in NC-CCN governance. Each network has a minimum of two seats on the
NC-CCN Board of Directors (and larger networks have additional seats). Networks also have

2

autonomy in deciding how to use their funds but submit standard budgets to the program office
on a quarterly basis and are expected to expend most of their funds on care and population
management staff and activities.
All of the NC-CCN local networks have legal status as 501(c)(3) nonprofit organizations. A
typical organizational structure for a local network includes a chairman of the board, a network
director, a medical director, and a medical management committee. The Network Director and
the Medical Director report to the Board Chairman and oversee network clinical and care
management staff who provide services in various counties and practices, and the network’s
administrative staff. For example, one network has 18 chronic care coordinators, 8 pediatric care
coordinators, 2 health coordinators and enrollment specialists, a behavioral health coordinator, 3
clinical pharmacists, an information technology (IT) privacy officer, an IT coordinator, a
finance/human resources officer, an office manager, and an administrative assistant.

3

Introduction
Thank you for making time to speak with us today. As we mentioned in our email to [name of
our contact in their practice / clinic / hospital], we are researchers from RTI, the Urban Institute,
and the National Academy of State Health Policy (NASHP), evaluating the Multi-Payer
Advanced Primary Care Practice (MAPCP) Demonstration.
As previously mentioned in the email we sent you when we set up the interview, we’d like your
candid views about this initiative. Specifically, we’d like to focus on learning about the period
after Medicare joined the state’s initiative and your thoughts on the future. Since we’re asking
about a number of different periods of time, we have created a visual timeline and logic model to
differentiate each time period [show the respondent the state’s timeline and logic model].
We are also particularly interested in the impact of the initiative on Medicare and Medicaid
beneficiaries generally and special populations (such as those dually eligible for Medicare and
Medicaid) specifically. We understand that some changes are system wide and may not affect
these groups differently, but other changes that payers or practices make may be tailored to these
groups to some degree and have different impacts on these patient populations.
As described in the one-page attachment to our email we sent to you to set up the interview, our
evaluation is being funded by the Centers for Medicare and Medicaid Services (CMS). As a
condition of participation, CMS expects state staff and participants to cooperate with the
evaluation team, but ultimately your decision to participate in this aspect of the study is
voluntary. If you do not wish to participate in this interview or answer specific questions, please
let us know immediately.
We believe there are minimal risks to you from participation, and every effort will be made to
protect your confidentiality. We want to assure you that we will not quote you by name. We will
use some quotes in reports, but quotes will not be attributed to an individual or his/her
organization.
There are no direct benefits to you from participating in this study. But your insights will be used
by federal and state policymakers to better understand the effects of the medical home model on
Medicare and Medicaid beneficiaries as well as the challenges involved in running a medical
home initiative and becoming and maintaining a medical home. The Institutional Review Boards
(IRB’s) at RTI International and the Urban Institute have reviewed this research.
Finally, we would like to record our conversation, to ensure our notes from today are complete.
Are these interview conditions OK with you? Do you have any questions before we begin?

4

Introduction/Background Questions
1. How long have you been with your network?
2. Please tell me about your roles and responsibilities with the network.
a. How long have you served in this role at the network?
b. Were you previously with a similar kind of organization?
3. Please tell me about the staff composition of your network, including health care
professionals, care managers, and administrative staff.
4. How many practices are part of your network? How many of these participate in the multipayer medical home demonstration?
5. How does your network communicate with practices?
6. We understand that NC networks connect patients with community-based resources, provide
care coordination for care transitions, support beneficiary self-care, and facilitate practice
improvement and transformation. Is this an accurate capture of what your network does in
support of the practices? What additional services do you provide? Which of these services
are the most important for NC medical home demonstration? Why?
7. How do you track what care is delivered by what provider to which patients?
Progress / State Implementation
8. Prior to 2011—when Medicare joined NC’s medical home demonstration—what were the
most significant problems that practices faced in serving Medicare beneficiaries?
9. What were the most important medical home features in the NC initiative prior to 2011?
10. Now that Medicare has joined the effort, what new features were added to your medical
home efforts?
11. What changes did you have to make to accommodate Medicare’s participation?
12. How is the implementation of the NC demonstration with Medicare’s involvement going so
far?
a. What has gone well?
b. What hasn’t gone so well?
13. What do you think are the strengths and weaknesses of the NC multi-payer medical home
demonstration?
14. What challenges do you anticipate in the future?

5

15. What opportunities for improvement do you anticipate in the future?
Payment and Infrastructure
16. How were community care networks paid prior to 2011—before Medicare joined the
demonstration? What changes did your network or practices have to make to accommodate
Medicare’s participation?
17. What are the strengths and of the current payment methodology for community health
networks and practices? What are the weaknesses?
18. Are the current payments sufficient to support collaboration and linkages between primary
care practices, networks, and CCNC/NCCCN?
19. Are the current payments sufficient to improve quality, utilization, and cost outcomes?
Health Information Technology
20. We understand that as part of the multi-payer medical home initiative, practices are required
to meet Blue Quality Physician Program standards, which require electronic prescribing and
claims submission. We also understand that CCNC’s informatics center provides a webbased portal for practices to monitor data at a state-, network-, practice- and patient-level.
Finally, we have also learned that case managers use a case management system. Are there
other types of health information technology that you use for North Carolina’s multi-payer
medical home initiative?
a. We’d like to learn more about each of these systems.
21. What experience does your network and practices have in meeting the Blue Quality
Physician Program standards? What benefit does this system bring to the network and
practices?
22. How does your network and practices use CCNC’s web-based portal? What benefit does this
portal bring to the network and practices? How often do you use it?
23. How do you use the Case Management Information System?
a. What benefit does this system bring?
b. How often do case managers use it and how?
24. Which of these systems allows you to track services provided to patients?
25. To what extent can you readily exchange health information with primary care practices?
Other practices/facilities (e.g., specialists, hospitals, long-term care facilities)?

6

26. We understand that your network receives daily admission, discharge, and transfer lists for
Medicare patients. How is it working?
a. How far along are you in the process of receiving real time hospital admission and
discharge information for these patients and those covered by BCBS or the State Health
Plan?
27. What challenges pertaining to health information technology do you face? How has your
participation in the NC demonstration changed the amount or frequency with which you use
health information technology?
28. What additional health IT features would help your community health network do a better
job assisting/communicating with patients? Practices?
Practice Transformation
[Note to interviewer: try to ask about (1) the time it took to implement any network changes
discussed; and (2) the cost of implementing or sustaining any network changes discussed]
29. What kinds of structural or organizational changes did your network make pre-2011—before
Medicare became involved in the NC demonstration—in an effort to support the medical
home model?
a. What additional changes are you working on making now that Medicare is involved in
the NC demonstration?
b. What are the major challenges to implementing these additional changes?
30. We understand that CCNC has hosted a series of webinars and developed a toolkit to help
practices achieve NCQA recognition. We also know that the quality improvement coaches
assist practices through the state’s Area Health Education Centers.
a. Can you tell us more about these activities?
b. What is the role of quality improvement coaches?
c. How did these tools and resources help networks and practices?
d. [AccessCare:] How has the ASU PCMH practicum study helped practices to achieve
NCQA PCMH recognition? Is this activity on-going?
Access to Care and Coordination of Care
[Note to interviewer: (1) Try to ask respondents to identify medical home features that
contributed to the observed impacts, or features that could be added to the initiative to improve
outcomes. (2) More specific examples of impacts to care might be discussed during the
interview. For example, timeliness of care is an example of access to care. Other more specific
examples that may be discussed in this section include: discussion of current and potential

7

changes to a patient’s drug regimen, proactive care (e.g. generating lists of patients who need to
come in for an office visit), and active reminders about tests, treatment, and/or appointments.]
31. Now we’d like to discuss with you how beneficiaries access care and their experiences with
this care. Please comment on patients overall, but also on whether there are any differences
for Medicare or Medicaid beneficiaries or special populations of patients. Special populations
can include: Medicare and Medicaid dual eligibles; children; racial and ethnic subgroups;
people living in rural or inner-city areas; and persons with chronic illnesses, mental illnesses,
and disabilities. How do you identify people who need help or how do they get connected
with your network?
a. Please describe how you identify Medicare or Medicaid beneficiaries for case
management, clinical pharmacy, or any additional services?
32. How familiar are you with Medicare’s beneficiary assignment process? [Note to interviewer:
please briefly describe the process, if the interviewee is not familiar, as described below …]
Each participating Medicare beneficiary is attributed to one of the Demonstration practices
using the following algorithm. A beneficiary must have at least one qualifying visit,
determined by one of twelve E&M codes to a participating practice during the
Demonstration period. The beneficiary will be attributed to the practice where he/she
accumulated the greatest number of qualifying visits/E&M codes. If there is an equal number
of qualifying visits/E&M codes among multiple participating practices, the beneficiary will
attributed to the practice where he/she was last seen.
a. Do you limit network services to those patients for whom a participating insurer pays a
monthly fee and/or only those beneficiaries who have been assigned to the practice?
Please describe briefly.
b. How do you currently invite patients to participate?
c. Please tell me about the activities of Patient Outreach Teams.
i.

What patient education tools do they provide to patients? How do they distribute
those tools?

d. How are patients transitioning between two practices participating in the demonstration
identified and handled by the community health network? Please describe briefly.
e. What about patients transitioning from participating practices to non-participating
practices or vice versa? How are these patients identified and handled by community
health network?
f. How does your network coordinate patient care with primary care practices and other
providers or facilities?

8

33. How does the network or its practices:
a. Assign a staff member within your community health network to a patient?
b. Follow up with patients after they are discharged from a hospital? ER?
c. Identify patients for medication reconciliation?
d. Coordinate care with specialists?
e. Provide linkages to other services and facilities like long-term care, mental health,
community services, or social services?
f. Work with patients to address challenges they may face accessing care?
g. Work with patients to address challenges they may face caring for themselves?
h. Work with special populations such as the mentally ill to address any challenges they
may face?
i. Work with patients to address challenges they may face caring for themselves?
j. Other?
k. Are there any differences in how you or the practices coordinate care for Medicare or
Medicaid beneficiaries, or other special populations?
34. How do the practices that are in your network communicate with patients who do not speak
or cannot read English?
a. How do practices identify patients with different language or literacy needs? Who does
the screening? What tools do they use?
b. Do practices that are part of your network and this demonstration offer translation
services?
Beneficiary Experience with Care
[Note to interviewer: (1) Try to ask respondents to identify medical home features that
contributed to the observed impacts, or features that could be added to the initiative to improve
outcomes. (2) More specific examples of impacts to care might be discussed during the
interview. For example, shared decision-making is an example of beneficiary experience with
care. Other more specific examples that may be discussed in this section include: provision of
information such as test results to patients in a way that is easy to understand and health literacy.]

9

35. What strategies do the practices that are part of your network and this demonstration use to
engage patients in their care?
a. To what extent are patients, their families, and/or their caregivers actually able to
participate more effectively in decisions concerning their care as a result of the NC
demonstration? Can you provide an example or do you have any early data on this?
36. How do the network or practice staff teach self-management to patients?
a. How do patients use self-management notebooks?
b. In your opinion, to what extent are patients actually better able to self-manage their
health conditions or engage in healthy behaviors as a result of the NC demonstration?
Can you provide an example or do you have any early data on this?
37. Does your network or participating practices experience any challenges in reaching Medicare
or Medicaid beneficiaries, in an effort to improve quality and safety? If so, please describe
briefly.
Quality of Care and Patient Safety
Now we’d like to discuss health care quality and patient safety. Please comment on patients
overall, but also on whether there are any differences for Medicare or Medicaid beneficiaries or
special populations of patients.
38. What quality and safety data does your network or participating practices collect?
a. Were any quality and safety measures added or dropped since 2011, when Medicare
joined the demonstration?
i.

Which preventive care service data do your network and participating practices
collect or share? These might include cancer screening, smoking cessation, weight
management, influenza vaccination, pneumonia vaccination. Please describe
briefly.

b. What quality and safety measures are reported back to your network? Are any of these
measures new, since 2011?
39. How often does CCNC provide these reports?
40. How do you use the information provided in these reports?
41. What are the most useful features of the quality measure reports you receive?

10

42. Does your network or its assigned practices receive data summarizing other dimensions of
your patients’ care, such as their utilization of health care services? (For example, statistics
on your patients’ hospital or nursing home admissions or readmissions.)
[If yes:]
a. What quality and safety measures are reported back to you?
i.

Which of these are measures did you receive feedback on prior to 2011—before
Medicare joined the NC demonstration?

ii.

Which of these measures (if any) did you begin to receive feedback on in 2011—
when Medicare joined the NC demonstration?

b. What entity provides you with these reports, and how often?
c. How do you use the information provided in these reports?
d. What are the most useful features of the quality measure reports you receive?
i.

How could these quality measure reports be made more useful to you?

[If no:]
e. If you were to receive a quality measure report, what quality measures would be the most
useful to you?
f. If you were to start receiving these types of data on your Medicare patients, how might
you use it?
Wrap-up
43. What major areas will your network focus on in the next year? What changes will you make?
a. What do you see as the facilitators (or critical factors) of successful implementation of
this demonstration?
b. What do you see as the barriers or major challenges to implementing the changes that are
part of this demonstration?
44. What advice would you give to other networks if the NC demonstration were to be extended
or expanded to include them?
45. What advice would you give to CMS (Medicaid and Medicare)? Given what you know now,
what would you have done differently, both prior to and after Medicare’s involvement in the
NC medical home demonstration?

11

46. Is there anything we have not discussed about the NC medical home demonstration or about
the Medicare MAPCP Demonstration that you feel would be important for our National
Evaluation Team to know?

12