Crosswalk

CMS_Response_4-19-13_tables.pdf

Evaluation of the Multi-Payer Advanced Primary Care Practice Demonstration

Crosswalk

OMB: 0938-1211

Document [pdf]

Download: pdf | pdf

Summary of Revisions Made to Site Visit Interview Protocols’ Existing Content in Response to Public Comments
Received Regarding CMS’s Planned Information Collection for the
Evaluation of the Multi-payer Advanced Primary Care Practice Demonstration
(Document Identifier: CMS-10436)
Protocol

Question #

State Officials

15ai

State Officials

21a

State Officials

27a

State Officials

Original Wording

[Optional] Do any of these new
services focus on improving care
coordination? If so, how?

Revised Wording

Do any of these new services focus on
improving access? If so, how?

Reason for Revision

Removed “optional” status due
to increased importance of the
question, and further revised to
probe if the new services focus
on improving access instead of
care coordination. A probe
question about care coordination
was added (15aii) (see table
below).
[Optional] How, if at all, do you feel How, if at all, do you feel these changes
Removed “optional” status due
these changes impact patient
impact patient access? We are
to increased importance of the
access to and coordination of care? particularly interested in the impact on
question, and further revised to
Medicare, Medicaid, and special
probe only about patient access.
populations.
A new probe question about care
coordination was added (21b)
(see table below).
What about in terms of measuring What about in terms of improving and
Revised to ask about improving
performance (in terms of quality,
measuring performance (in terms of
performance in addition to
utilization, and cost)?
quality, utilization, and cost)?
measuring it.
What state and/or federal health
What state and/or federal health ITRemoved language asking about
IT-focused projects or programs
focused projects or programs does [state] incentive programs through
does [state] participate in? (e.g.,
participate in? (e.g., EHR payment
Medicare (now the question only
EHR payment incentive programs
incentive programs through Medicaid,
asks about incentive programs
through Medicare and Medicaid,
regional health extension centers, Health through Medicaid), and added
Health Information Exchange
Information Exchange grants to the state) language asking about regional
grants to the state)
extension centers.

Protocol

State Officials

Question #

35aiii

36aiii

Original Wording

Patient participation or behavior?
(e.g., patients engaging more in
decisions and managing their care)

For patients?
State Officials

Payers

Revised Wording

Patient and family caregiver participation
or behavior? (e.g., patients engaging
more in decisions and managing their
care, use of patient and family advisory
boards for practice quality improvement
efforts)
Patient and family caregiver participation
or behavior? (e.g., patients engaging
more in decisions and managing their
care, use of patient and family advisory
boards for practice quality improvement
efforts)
For patients and family caregivers?

37e

What activities has your [plan /
organization undertaken to
support participating practices or
the [state-specific name of PCMH
initiative] in general? For example,
have you provided more timely or
user-friendly data? Disease
management support? Support for
learning collaboratives? Practice
coaches or consultants?

What activities has your [plan /
organization] undertaken to support
participating practices or the [statespecific name of PCMH initiative] in
general? For example, have you solicited
member input on the medical home
model or provided information to
members on their medical home? Have
you provided more timely or user-friendly
data? Care coordination or disease
management support? Support for
learning collaboratives? Practice coaches
or consultants?

Reason for Revision

Revised to ask about family
caregiver participation or
behavior in addition to patient
participation or behavior.
Revised to ask about family
caregiver participation or
behavior in addition to patient
participation or behavior.
Revised to ask about major
barriers to achieving the goals of
this initiative for family
caregivers in addition to barriers
for patients.
Revised to probe about the
extent to which payers have
solicited input from or provided
information to their members
about the patient-centered
medical home.

Protocol

Payers

Question #

21b

27aiii

Original Wording

Revised Wording

Reason for Revision

Have you reached out to or
undertaken any activities with
consumers and [beneficiaries / plan
members] served by participating
practices? In so, please briefly
describe. If not, why not?
Since the state initiative began,
how successful has the [statespecific name of PCMH initiative]
been in getting practices to change
the way they deliver care?

Have you reached out to or undertaken
any activities with consumers and
[beneficiaries / plan members] served by
participating practices? In so, please
describe. If not, why not?

Revised so that the respondent is
asked to provide a longer
response, not a brief one.

Revised to ask specifically about
the state’s degree of success in
improving access, care
coordination and transitions, and
linkages to community health
resources.

Patient participation or behavior?
(e.g., patients engaging more in
decisions and managing their care)

Since the state initiative began, how
successful has the [state-specific name of
PCMH initiative] been in getting practices
to change the way they deliver care,
particularly improving access, care
coordination and transitions, and making
linkages with community health
resources?
Patient and family participation or
behavior? (e.g., patients engaging more in
decisions and managing their care)

Revised to ask specifically about
the initiative’s impact on family
participation or behavior in
addition to patient participation
or behavior.
[Before Visit] [Optional] Have you
Revised to specifically ask about
conducted, or do you plan on conducting, surveys that assess the medical
a patient experience survey among your
home approach.
[members / beneficiaries]? We are
particularly interested in any survey that
may more specifically assess aspects of
the medical home, such as C-G CAHPS or
the newly developed PCMH CAHPS.

[Before Visit] [Optional] Have you
conducted, or do you plan on
conducting, a patient experience
survey among your [members /
beneficiaries]?

Protocol

Question #

Community
Health [Teams/
Networks]

Original Wording

Prior to [2011/2012]—when
Medicare [and Medicaid (if joined
at the same time0] joined [name of
state]’s [state-specific name of
PCMH initiative]—what were the
most significant problems that
practices involved in this initiative
faced in serving Medicare [and
Medicaid (if joined at the same
time)] beneficiaries?
[Optional] What were the most
important features of [name of
state]’s [state-specific name of
PCMH initiative] implemented prior
to [2011/2012]?
[Optional] Now that Medicare [and
Medicaid (joined at the same
time)] has joined the effort, what
features of [name of state]’s [statespecific name of PCMH initiative]
are most important?
[Optional] [Before Visit] How were
community health [Teams/
networks] paid prior to
[2011/2012]—before Medicare
[and Medicaid (if joined at the
same time)] joined the
demonstration? What major
changes have been made since
Medicare [and Medicaid (if joined
at the same time)] joined, if any?

Revised Wording

Reason for Revision

Prior to [2011/2012]—when Medicare
[and Medicaid (if joined at the same
time0] joined [name of state]’s [statespecific name of PCMH initiative]—what
were the most significant problems that
practices involved in this initiative faced in
serving Medicare [and Medicaid (if joined
at the same time)] beneficiaries and their
families or caregivers?

Revised to ask about the
problems that practices faced in
serving Medicare (and potentially
Medicaid depending on the
specifics of the state initiative)
beneficiaries’ families or
caregivers, in addition to the
beneficiaries themselves.

What were the most important features
of [name of state]’s [state-specific name
of PCMH initiative] implemented prior to
[2011/2012]?

Removed “optional” status due
to increased importance of the
question.

Now that Medicare [and Medicaid (joined Removed “optional” status due
at the same time)] has joined the effort,
to increased importance of the
what features of [name of state]’s [state- question.
specific name of PCMH initiative] are
most important?
[Optional] [Before Visit] How were
Removed “optional” status due
community health [Teams/ networks]
to increased importance of the
paid prior to [2011/2012]—before
question.
Medicare [and Medicaid (if joined at the
same time)] joined the demonstration?
What major changes have been made
since Medicare [and Medicaid (if joined at
the same time)] joined, if any?

Protocol

Question #

Community
Health [Teams/
Networks]

21ai

Community
Health [Teams/
Networks]

21aii

Community
Health [Teams/
Networks]

21aiv

Community
Health [Teams/
Networks]

21avii

Original Wording

[Before Visit] What types of health
information technology capabilities
does [name of organization] use to
carry out its functions? For
example, do you use web portals
maintained by payers or referral
tracking databases developed by
the community health [team /
network]?
Assign a staff member within your
community health [team /
network] to a patient?

Revised Wording

[Before Visit] What types of health
information technology capabilities does
[name of organization] use to carry out its
functions? For example, do you use web
portals maintained by payers or
participating practices or referral tracking
databases developed by the community
health [team / network]?
Assign a staff member within your
community health [team / network] to a
patient and/or their family or caregiver?

Reason for Revision

Revised to probe specifically
about webportals maintained by
practice participating in the
initiative.

Revised to probe specifically
about whether community
health Teams/ networks assign a
staff member to a patient’s
family or caregiver in addition to
asking about whether a network
/ team assigns a staff member to
the patient.
Follow up with patients after they Follow up with patients and/or their
Revised to probe specifically
are discharged from a hospital?
family or caregiver after they are
about whether community
ER?
discharged from a hospital? ER?
health Teams/ networks follow
up with a patient’s family or
caregiver in addition to the
patient.
Coordinate care with specialists?
Coordinate care with specialists (e.g.,
Revised to probe specifically
physical and mental health)?
about whether community
health Teams/ networks
coordinate care with physician
and mental health specialists.
Work with patients to address
Work with patients and families or
Revised to probe specifically
challenges they may face caring for caregivers to address challenges they may about whether community
themselves?
face caring for themselves?
health Teams/ networks work
with families or caregivers in
addition to patients.

Protocol

Question #

Community
Health [Teams/
Networks]

Community
Health [Teams/
Networks]
Community
Health [Teams/
Networks]
Professional
Association

28a

28aii

30a

Original Wording

What are patients’ reactions when
you contact them?

[Optional] Which quality data did
your community health [team /
network] or your assigned
practices begin collecting and
sharing once Medicare [and
Medicaid (if joined at the same
time)] joined the [state-specific
name of PCMH initiative]? Were
any quality and safety measures
added or dropped?
[Optional] Are there any chronic
conditions your community health
[team / network] or your assigned
practices collect or share data on?
What quality and safety measures
are reported back to you?
What do the care coordinators do?
What kind of services do they
provide to the practices and/or the
practice’s patients?

Revised Wording

Reason for Revision

What are patients’ reactions when you
contact them? What do they see as the
potential benefits of working with a
community health network/team? What
do they see as the potential drawbacks, if
any?
Which quality data did your community
health [team / network] or your assigned
practices begin collecting and sharing
once Medicare [and Medicaid (if joined at
the same time)] joined the [state-specific
name of PCMH initiative]? Were any
quality and safety measures added or
dropped?

Revised to ask about patients’
perception of the benefits and
drawbacks of working with a
community health team or
network.

Are there any chronic conditions your
community health [team / network] or
your assigned practices collect or share
data on?
What other measures (e.g., utilization) are
reported back to you?

Removed “optional” status due
to increased importance of the
question.

What do the care coordinators do? What
kind of services do they provide to the
practices and/or the practice’s patients
and families?

Removed “optional” status due
to increased importance of the
question.

Revised to reflect that the
interview protocol asks about
quality and safety measures in an
earlier question.
Revised to probe about patients’
families instead of only patients.

Protocol

Question #

Local Chapters
of Physician
and Clinical
Professional
Associations

Local Chapters
of Physician
and Clinical
Professional
Associations
Local Chapters
of Physician
and Clinical
Professional
Associations
Office of Aging
Staff and Other
Patient
Advocates

Original Wording

Revised Wording

Reason for Revision

How do care coordinators
coordinate with specialists,
hospitals, mental health and
nursing facilities, and community
services?

How do care coordinators work with
specialists, hospitals, mental health and
nursing home facilities, and community
services?

10b

[If needed:] What do you think
patients see as the main strengths
and weaknesses of the model?

[If needed:] What do you think patients
and families see as the main strengths
and weaknesses of the model?

Revised to eliminate redundant
language and also revised to ask
specifically about nursing home
facilities in addition to the other
types of providers already listed
in the question.
Revised to probe specifically
about patients’ families in
addition to patients themselves.

22aiii

Patient participation or behavior?
(e.g., patients engaging more in
decisions and managing their care)

Patient and family or caregiver
participation or behavior? (e.g., patients
engaging more in decisions and managing
their care)

Revised to probe specifically
about patients’ families or
caregivers in addition to patients
themselves.

Patients having access to care
when they need it? Could you give
me an example of that?

Patients and families or caregivers having
access to care when they need it? Could
you give me an example of that?

2ai

Office of Aging
Staff and Other
Patient
Advocates

2aii

Office of Aging
Staff and Other
Patient
Advocates

2aiv

Revised to probe specifically
about patients’ families or
caregivers in addition to patients
themselves.
Effective communication between Effective communication between
Revised to probe specifically
these patients and practice staff
patients, families or caregivers, and
about patients’ families or
(including doctors and other staff)? practice staff (including doctors and other caregivers in addition to patients
Could you give me an example of
staff)? Could you give me an example of
themselves.
that?
that?
Patients being better able to self- Patients and families or caregivers being Revised to probe specifically
manage their health and medical
better able to self-manage their health
about patients’ families or
conditions?
and medical conditions?
caregivers in addition to patients
themselves.

Protocol

Office of Aging
Staff and Other
Patient
Advocates
Office of Aging
Staff and Other
Patient
Advocates

Office of Aging
Staff and Other
Patient
Advocates

Question #

Original Wording

Coordination of care?

8aii
Do you anticipate better access to
care? If so, please explain how.
9a

Do you anticipate more effective
participation in health care
decisions? If so, please explain
how.

Do you anticipate increased
engagement in health behaviors
(e.g., healthy diet and exercise)? If
so, please explain how.
9c

Revised Wording

Coordination of care or care transitions?

Reason for Revision

Revised to ask about evidence of
improvements in care transitions
in addition to coordination of
care.
Do you anticipate better access to care? If Revised to probe about specific
so, please explain how. For example,
ways that a practice can improve
through expanded office hours, additional access.
practice staff, on-line access to the
practice and electronic health
information?
Do you anticipate more effective patient Revised to probe specifically
and family or caregiver participation in
about patients and family or
health care decisions? If so, please explain caregiver participation and to
how. For example, through more active
provide specific examples of
role in care management plans, greater
ways that practices can
use of shared decision-making tools,
encourage more patient and
explicit conversation about whether other family or caregiver participation
family or caregivers should be involved in in health care decisions.
care and how?
Do you anticipate increased engagement Revised to include examples of
in healthy behaviors (e.g., healthy diet
ways of engaging patients in
and exercise)? If so, please explain how.
healthy behaviors.
For example, through motivational
interviewing or brief counseling by
providers, educational materials made
available in appropriate languages and
literacy levels, linkages to local health and
wellness classes provided by local
community groups?

Protocol

Question #

Office of Aging
Staff and Other
Patient
Advocates

Office of Aging
Staff and Other
Patient
Advocates
Office of Aging
Staff and Other
Patient
Advocates

Original Wording

Do you anticipate increased
adherence to preventive services?
For example, cancer screening,
smoking cessation, influenza
vaccination, or pneumonia
vaccination.

Do you anticipate better
management of their chronic
health conditions (e.g., diabetes)?
If so, please explain how.
9e

Do you anticipate better
coordination of care, such as
between primary care and
specialists or physicians and
hospitals? If so, please explain
how.
For beneficiaries/patients?

10a

Revised Wording

Do you anticipate increased adherence to
preventive services? For example, cancer
screening, smoking cessation, influenza
vaccination, or pneumonia vaccination.
For example, through motivational
interviewing or brief counseling by
providers, educational materials made
available in appropriate languages and
literacy levels, linkages to local
community resources?
Do you anticipate better management of
their chronic health conditions (e.g.,
diabetes)? If so, please explain how. For
example, through motivational
interviewing or brief counseling by
providers, educational materials made
available in appropriate languages and
literacy levels, better self-management
skills, linkages to local community health
networks/teams or resources?
Do you anticipate better coordination of
care, such as between primary care and
specialists or physicians and hospitals? If
so, please explain how. For example,
though new care coordinators in practices
or health plans and community health
networks/teams working with practices.
For beneficiaries/patients, including
special or vulnerable populations?

Reason for Revision

Revised to include additional
examples of ways that practices
can improve adherence to
preventive services.

Revised to include examples of
ways in which practices can
improve management of chronic
conditions.

Revised to include examples of
ways that practices can improve
care coordination.

Revised to probe specifically
about major barriers for special
or vulnerable populations

Protocol

Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)

Question #

Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)

13f

Original Wording

[If needed:] What do you think
patients see as the main strengths
and weaknesses of the model?

Revised Wording

Reason for Revision

[If needed:] What do you think patients
Revised to probe specifically
and families or caregivers see as the main about patients’ families or
strengths and weaknesses of the model? caregivers in addition to patients
themselves.

What major changes did your
[office / clinic / hospital] make to
become recognized as a medical
home or to be eligible for the [state
initiative] when it first began in
[year that the state initiative
began] (e.g., focusing on new
conditions, using new care
processes, adopting new health IT
tools, interacting differently with
patients)

What major changes did your [office /
clinic / hospital] make to become
recognized as a medical home or to be
eligible for the [state initiative] when it
first began in [year that the state initiative
began] (e.g., focusing on new conditions,
improving access through additional
evening and weekend hours or same-day
appointments, using new care processes
to improve care coordination and
transitions, adopting new health IT tools,
interacting differently with patients and
families or caregivers)
How have your interactions with patients
and families or caregivers changed since
you became a medical home—before,
during, and after a visit?

Revised to probe specifically
about changes practices made to
improve access through
additional hours and same-day
appointments; care coordination
and transitions; and interaction
with patients’ families or
caregivers in addition to patients
themselves.

Are there any major differences in the
extent to which Medicare or Medicaid
patients or special populations (e.g., dual
eligibles, patients with chronic conditions)
are willing and able to engage in these
patient and family or caregiver
engagement activities? If so, please
describe these major differences.

Revised to probe about
differences in the level of
engagement in these activities
among Medicare or Medicaid
patients or special populations.

How have your interactions with
patients changed—before, during,
and after a visit--since you became
a medical home?
Are there any major differences for
Medicare or Medicaid patients or
special populations (e.g., dual
eligibles, patients with chronic
conditions)?

Revised to probe specifically
about patients’ families or
caregivers in addition to patients
themselves.

Protocol

Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)
Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)

Question #

Original Wording

Revised Wording

Reason for Revision

Why not?

Why not? Who coordinates care for your
patients?

Revised to probe about who
coordinates care for patients.

[Before Visit] Does your practice
work with a community health
[team / network]?

[Before Visit] Does your practice work
with a community health [team /
network] and/or develop linkages with
other local community health resources?

Revised to ask if practices
develop linkages with other local
community health resources.

14f

Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)

15a

Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)

15b

What do they do? What kind of
services do they provide to your
[office / clinic / hospital] and/or
your patients?

What do these community health
[team/network] do and/or how do you
make linkages with other local community
health resources? What kind of services
do they provide to your [office / clinic /
hospital] and/or your patients?
What specific types of patients
What specific types of patients does your
does your community health [team community health [team / network]
/ network] focus on? (e.g.,
and/or other local community resource(s)
Medicare or Medicaid patients,
focus on? (e.g., Medicare or Medicaid
special populations of patients.
patients, special populations of patients.
Special populations can include:
Special populations can include: Medicare
Medicare and Medicaid dual
and Medicaid dual eligibles; children;
eligibles; children; racial and ethnic racial and ethnic subgroups; people living
subgroups; people living in rural or in rural or inner-city areas; and persons
inner-city areas; and persons with with chronic illnesses, mental illnesses,
chronic illnesses, mental illnesses, and disabilities.) How do they decide
and disabilities.) How do they
which patients to focus on?
decide which patients to focus on?

Revised to ask what community
health teams or networks do and
how practices make linkages to
community resources.
Revised to ask about other local
community resources.

Protocol

Question #

Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)

15c

Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)
Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)
Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)

15d

Original Wording

Revised Wording

Reason for Revision

How do you communicate with
your community health team /
network]?

How do you communicate with your
community health team / network]
and/or other local community
resource(s)?

Revised to ask about other local
community resources.

What have been the benefits of
working with a community health
[team / network]?

What have been the benefits of working
with a community health [team /
network] and/or other local community
resource(s)?

Revised to ask about other local
community resources.

15e

What have been some drawbacks What have been some drawbacks of using Revised to ask about other local
of using a community health [team a community health [team / network]
community resources.
/ network]?
and/or working with other local
community health resources?

Does your [office / clinic / hospital]
typically exchange medical records
with other providers and health
care facilities?

Does your [office / clinic / hospital]
Revised to probe about physical
typically exchange medical records with
and mental health specialists,
other providers (e.g., physical and mental diagnostic testing or laboratory.
health specialists, diagnostic testing or
laboratory) and health care facilities, such
as hospitals or nursing homes?

Protocol

Question #

Medical Home
Practices
(Physicians,
Office
Managers and
Other Staff)

Original Wording

Does your [office / clinic / hospital]
typically receive medical records
from other providers and health
care facilities?

Are you participating in the
Medicare or Medicaid EHR
payment incentive program?
24a

28bi

31aiii

Are these quality results broke out
by payer type (e.g., Medicare,
Medicaid, commercial payers?)

Patient participating or behavior?
(e.g., patients engaging more in
decisions and managing their care)

Revised Wording

Reason for Revision

Does your [office / clinic / hospital]
typically receive medical records from
other providers (e.g., physical and mental
health specialists) and health care
facilities (e.g., hospitals, mental health
facilities, diagnostic testing facilities,
laboratories)?
Are you participating in the Medicare or
Medicaid EHR payment incentive
program? If so, from your perspective,
how well aligned are stage 1 meaningful
use criteria and the clinical quality and
other reporting requirements for [statespecific name of PCMH initiative]? To
what extent are the measures for both
initiatives similar or different?
Are these quality results broke out by
payer type (e.g., Medicare, Medicaid,
commercial payers) or demographic (e.g.,
race/ethnicity), language, gender or
disability categories?

Revised to probe about physical
and mental health specialists,
diagnostic testing or laboratory.

Patient and family or caregiver
participating or behavior? (e.g., patients
engaging more in decisions and managing
their care)

Revised to probe specifically
about patients’ families or
caregivers in addition to patients
themselves.

Revised to ask about the extent
to which stage 1 meaningful use
criteria and the initiative’s
reporting requirements are
aligned.

Revised to ask if quality results
are broken down by patient
demographics.

Summary of New Questions Added to Existing Site Visit Interview Protocols in Response to Public Comments
Received Regarding CMS’s Planned Information Collection for the Evaluation of the Multi-payer
Advanced Primary Care Practice Demonstration
(Document Identifier: CMS-10436)
Protocol

Question #

State Officials

15aii

State Officials

20a

New Question Added

Briefly describe the major goals of the [state-specific
name of the PCMH initiative]?
How were key stakeholders, including consumers or
patients, involved in setting the goals for [state-specific
name of the PCMH initiative]?
How are key stakeholders, including consumers or
patients currently involved in monitoring the
implementation of [state-specific name of the PCMH
initiative] and providing input on any aspects that need
to be more fully articulated or refined?
Do any of these new services focus on improving care
coordination or care transition, including transitions
from hospital to home? If so, how?
Please describe briefly the kinds of technical assistance
or supports the [state-specific name of the PCMH
initiative] is providing to support the development of
greater medical home capacity as reflected in medical
home assessment or recognition tool level or score? For
example, learning collaboratives, practice coaching or
consultation, care coordinators, etc.
What are the strengths of the technical assistance and
practice supports put in place by [state-specific name of
PCMH initiative?

Reason for Addition

Added so that the interviewer can probe about
stakeholder involvement, including consumers or
patients, in setting goals and monitoring
implementation.
Added to probe about stakeholder involvement,
including patients and consumers, in setting the
initiative’s goals.
Added to probe about stakeholder monitoring of the
initiative’s implementation and their degree of input, if
any, on implementation issues.
Added to probe specifically about new care
coordination and transition services.
Added to ask specifically about the technical assistance
that the initiative provides to practices trying to
develop greater medical home capacity.

Added to probe about the strengths of the initiative’s
medical home technical assistance to practices.

Protocol
State Officials

State Officials

Question #
20b

21b

21c

State Officials

31c

State Officials

37d

New Question Added

What are the challenges or areas for improvement
needed with respect to technical assistance or practice
supports put in place by [state-specific name of PCMH
initiative]?
How, if at all, do you feel these changes impact care
coordination or care transitions, including from
transitions from hospital to home? We are particularly
interested in the impact on Medicare, Medicaid, and
special populations.
How, if at all, do you feel these changes impact patient
and family engagement? For example, identifying and
involving key family members involved in care, selfmanagement skills, and development of care plans or
shared decision making?
To what extent are EHR requirements and related
clinical quality measures used by the [state-specific
name of PCMH initiative] aligned with Medicare and
Medicaid Meaningful Use (MU) measures? For
example, have stage 1 MU requirements to collect
demographic information, provide on-line access to
patients, or report particular clinical quality measures
been incorporated into the medical home assessment
or recognition criteria?
How aligned are the states’ clinical quality measure and
other reporting requirements with those required for
other CMS programs?
For community health teams or networks?

Reason for Addition

Added to probe about the weaknesses of the initiative’s
medical home technical assistance to practices.
Added to ask in a separate probe question about care
coordination (previously this was included in 21a) and
care transitions (which was not asked about in 21a).
Added to probe specifically about the impact of practice
changes on patient and family engagement.

Added to ask about the degree of alignment between
the initiative’s EHR requirements and Medicare and
Medicaid’s Meaningful Use measures.

Added to probe about the degree of alignment
between state clinical quality measures and others
required by other CMS programs.
Added to probe about major barriers to achieving the
goals of the initiative for community health teams or
networks.

Protocol

Payers

Question #

22a

New Question Added

What activities has your [plan / organization]
undertaken to support community health networks /
teams or other community resources or linkages
between primary care practices and these networks /
teams or other community resources? For example, do
you work directly with state and local social service
agencies such as housing departments, office of aging
or transportation services for the disabled?
How do encourage and help support primary care
practices’ efforts to create linkages to and work with
these community networks / teams or other
community resources?
What state and/or federal health IT-focused projects or
programs does [payer] participate in? (e.g., Health
Information Exchange grants to the state, Beacon
communities)
What has [payer] done to support [medical home /
health care home] practices in health IT
implementation or upgrade? For example, does the
[payer] have any financial incentive for practices to
implement or upgrade their electronic health record
(EHR) and/or electronically exchange data with the plan
and other providers?
To what extent are EHR requirements and quality
measures used by [state-specific name of PCMH
initiative] aligned with the Medicare and Medicaid
Meaningful Use (MU) measures? For example, have
stage 1 MU requirements to collect demographic
information, provide on-line access to patients, or
report particular clinical quality measures been
incorporated into the medical home assessment or
recognition criteria?

Reason for Addition

Added to ask whether and to what extent the payer
engages in any activities that would create links
between primary care practices and community
resources or community health Teams/ networks.

Added to probe about payer activities that support and
encourage practices’ own efforts to create linkages with
the community or community health Teams/ networks.
Added to ask about payer activities related to state
and/or federal health information technology projects.
Added to ask about payer activities in support of
practices in health information technology
implementation or upgrade.

Added to ask about the alignment between 1) the
initiative’s EHR requirements and quality measures and
2) the Medicare and Medicaid Meaningful Use
measures.

Protocol
Payers
Community
Health
[Teams/
Networks]
Community
Health
[Teams/
Networks]
Community
Health
[Teams/
Networks]
Local Chapters
of Physician
and Clinical
Professional
Associations
Local Chapters
of Physician
and Clinical
Professional
Associations
Local Chapters
of Physician
and Clinical
Professional
Associations

Question #
28biii

27a

29b

30b

New Question Added

How aligned is your plan and states’ clinical quality
measure and other reporting requirements aligned with
those required for other CMS programs?
Do you or participating practices ask adult patients
who, if anyone, they want involved in their care and
what role they would play? If so, is that information
recorded in their electronic or paper medical record?
Are these quality and safety measures stratified by
payer and/or other patient characteristics, such as
demographics (e.g., race/ethnicity), language, gender,
or disability?
Are these other measures (e.g., utilization) stratified by
payer and/or other patient characteristics, such as
demographics (e.g., race/ethnicity), language, gender,
or disability?
We understand your state uses [name of state’s
medical home assessment or recognition tool]. Why
was that tool selected or developed?

Reason for Addition

Added to ask about the extent of alignment among the
payer’s quality reporting, state quality reporting, and
requirements for other CMS programs.
Added to find out if community health Teams/
networks record in patient medical records who, if
anyone, patients want involved in their care.
Added to find out if the quality and safety measures
reported back to community health Teams/ networks
are stratified by a range of patient characteristics.
Added to find out if the other measures reported back
to community health Teams/ networks (such as
utilization) are stratified by a range of patient
characteristics.
Added to ask about the state’s reasons for selecting its
medical home assessment or recognition tool.

How well do you feel it assesses a practice’s medical
home capabilities?

Added to determine the respondent’s opinion on how
well the tool assesses medical home capabilities.

How is this medical home assessment or recognition
information used? For example, does the state and/or
your organization use it to guide learning collaborative
activities? To determine practice payment levels?

Added to find out how the medical home assessment or
recognition information is used in the initiative.

Protocol

Local Chapters
of Physician
and Clinical
Professional
Associations
Local Chapters
of Physician
and Clinical
Professional
Associations
Local Chapters
of Physician
and Clinical
Professional
Associations
Local Chapters
of Physician
and Clinical
Professional
Associations
Local Chapters
of Physician
and Clinical
Professional
Associations

Question #

New Question Added

Reason for Addition

Access, such as same-day appointments or extended
evening or weekend hours?

Added to probe specifically about improvements in
access.

Care coordination and care transitions with other
practices, hospitals, and other nearby health providers
such as mental health facilities or nursing homes?

Added to probe specifically about improvements in care
coordination and care transitions.

Linkages with community health [Teams/ networks] and Added to probe specifically about improvements in
resources?
linkages with community health [Teams/ networks] and
resources

Patient and family caregiver engagement at the
individual and practice level? For example, more
involvement in care plans, shared decision-making,
patient and family caregiver input on quality
improvement efforts by the practice?
To what extent are EHR requirements and quality
measures used by [state-specific name of PCMH
initiative] aligned with the Medicare and Medicaid
Meaningful Use (MU) measures? For example, have
stage 1 MU requirements to collect demographic
information, provide on-line access to patients, or
report particular clinical quality measures been
incorporated into the medical home assessment or
recognition criteria?

Added to probe specifically about improvements in
patient and family caregiver engagement.

Added to ask about the extent of alignment between
the initiative’s EHR requirements and quality measures
in Medicare and Medicaid’s Meaningful Use measures.

Protocol

Local Chapters
of Physician
and Clinical
Professional
Associations
Local Chapters
of Physician
and Clinical
Professional
Associations
Office of Aging
Staff and
Other Patient
Advocates
Office of Aging
Staff and
Other Patient
Advocates
Office of Aging
Staff and
Other Patient
Advocates
Office of Aging
Staff and
Other Patient
Advocates
Office of Aging
Staff and
Other Patient
Advocates

Question #

New Question Added

Reason for Addition

19aiv

How aligned are the state’s clinical quality measures
and other reporting requirements aligned with those
required for other CMS programs?

Added to ask about the extent of alignment between
the initiative’s EHR requirements and quality measures
and other CMS programs.

19bii

Are these quality results broken down by patient
demographics (race/ethnicity), gender, language, or
disability status?

Added to find if quality measurements results are
broken down by a range of patient characteristics.

How do practices, community health [teams/networks],
and plans communicate with patients? Families or
caregivers? Patients who do not speak or cannot read
English?
Were you involved in planning of the demonstration,
including goal setting?

Added to probe specifically about practice, payer, and
community health team / network communication with
patients and their families or caregivers.

To what extent are you involved in implementing or
monitoring the implementation of the [state-specific
name of PCMH initiative], formally and informally, if at
all?
To what extent are you asked and given opportunities
to provide input into how to refine the [state-specific
name of PCMH initiative], formally and informally, if at
all?
To what extent do the impacts observed differ
depending on demographics (e.g., race/ethnicity),
language, gender, or disability status?

Added to probe about extent of the respondent’s role
in the planning stages of the demonstration.

2aiii

Added to inquire about the respondent’s role in the
planning stages of the demonstration.

Added to probe about the respondent’s opportunities
to provide input on the initiative.
Added to find out if the initiatives impacts differ based
on patient demographics.

Protocol

Office of Aging
Staff and
Other Patient
Advocates
Practices

Practices

Question #
10c

13b

13c

13d

Practices

13e

Practices

New Question Added

Reason for Addition

For community health networks/teams and/or other
local community health resources linked to
participating practices?

Added to probe about major barriers for community
health teams/networks and/or other local community
resources linked to practices.

What efforts has the practice undertaken to improve
patient self-management skills and/or engage in care
planning and shared decision-making?
How do you communicate with patients who do not
speak or cannot read English? Does the practice screen
for language or literacy problems? Does the practice
arrange for translators or other services or information
products
Do you ask adult patients who, if anyone, they want
involved in their care and what role they would play? If
so, is that information recorded in the electronic or
paper medical record?
To what extent do you involve patients and families or
caregivers in practice quality improvement or redesign
efforts?
Do patients and families or caregivers have on-line
access to the practice and their medical record
information? If so, through what mechanisms (e.g.,
web-portal, personal health record) and what kind of
services (e.g., e-visits) or information can they access?

Added to probe about patient self-management, care
planning, and shared decision-making.
Added to probe about the ways that practices
communicate with patients who do not speak English.

Added to probe about whether practices include in the
patient’s medical record the people the patient wants
involved in their care.
Added to probe about the extent to which practices
involve patients and families or caregivers in quality
improvement efforts.
Added to ask about patient and family or caregiver
access to medical record information online.

Summary of Questions Removed From Existing Site Visit Interview Protocols in Response to Public Comments
Received Regarding CMS’s Planned Information Collection for the
Evaluation of the Multi-payer Advanced Primary Care Practice Demonstration
(Document Identifier: CMS-10436)
Protocol

Community
Health
[Teams/
Networks]

Question #
N/A

Deleted Question

Work with patients to address challenges they may
face caring for themselves?

Reason for Deletion

Deleted because the question was duplicative.

Summary of Questions from New Site Visit Interview Protocols (Added After the 60-day Comment Period)
Protocol

Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks

Communitybased Care
Networks

Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks

Question #

New Question Added

Please tell me about your roles and responsibilities with the network.
How long have you served in this role at the network? Were you
previously with a similar kind of organization?
Please tell me about the staff composition of your network, including
health care professionals, care managers, and administrative staff.

Added to obtain information about the
respondent’s background and
responsibilities.
Added to obtain information about the
respondent’s background and
responsibilities.
Added to obtain information about the
organization’s composition.

How many practices are part of your network? How many of these
participate in the multipayer medical home demonstration?

Added to obtain information about the
organization’s composition.

How does your network communicate with practices?

Added to obtain information about of
the organization’s functions.

We understand that NC networks connect patients and communitybased resources, provider care coordination for care transitions,
support beneficiary self-care, and facilitate practice improvement and
transformation. Is this an accurate capture of what your network does
in support of the practices? What additional services do you provide?
Which of these services are the most important for NC medical home
demonstration? Why?
How do you track what care is delivered by what provider to which
patient?

Added to obtain information about the
organization’s functions.

Prior to 2011—when Medicare joined NC’s medical home
demonstration—what were the most significant problems that
practices faced in serving Medicare beneficiaries?
What were the most important medical home features in the NC
initiative prior to 2011?

Added to obtain information about the
problems facing Medicare beneficiaries.

1
2-2b

7
8
9

How long have you been with your network?

Reason for Addition

Added to obtain information about the
organization’s functions.

Added to provide background
information on the state’s initiative.

Protocol

Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks

Question #

New Question Added

Reason for Addition

Now that Medicare has joined the effort, what new features were
added to your medical home efforts?

Added to provide background
information on the state’s initiative.

What changes did you have to make to accommodate Medicare’s
participation?

Added to get a better sense of the
organization’s functions.

How is the implementation of the NC demonstration with Medicare’s
involvement going so far? What has gone well? What hasn’t gone
well?
What do you think are the strengths and weaknesses of the NC multipayer medical home demonstration?

Added to obtain background
information on the state’s initiative.

What challenges do you anticipate in the future?

Added to obtain background
information on the state’s initiative.

What opportunities for improvement do you anticipate in the future?

Added to obtain background
information on the state’s initiative.

12-12b
13
14
15
16
17
18
19

How were community care networks paid prior to 2011-before
Medicare joined the demonstration? What changes did your network
or practices have to make to accommodate Medicare’s participation?
What are the strengths of the current payment methodology for
community health networks and practices? What are the weaknesses?

Added to obtain background
information on the state’s initiative.

Added to obtain information about how
the networks and practices are paid
under the state’s initiative.
Added to obtain information about how
the networks and practices are paid
under the state’s initiative.
Are the current payments sufficient to support collaboration and
Added to obtain information about how
linkages between primary care practices, networks, and CCNC/NCCCN? the networks and practices are paid
under the state’s initiative..
Are the current payments sufficient to improve quality, utilization, and Added to obtain information about how
cost outcomes?
the networks and practices are paid
under the state’s initiative.

Protocol

Communitybased Care
Networks

Question #

20-20a

Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks

23-23b

Communitybased Care
Networks

26-26a

21
22

24
25

New Question Added

We understand that as part of the multi-payer medical home initiative,
practices are required to meet Blue Quality Physician Program
standards, which require electronic prescribing and claims submission.
What also understand that CCNC’s informatics center provides a webbased portal for practices to monitor data at a state-, network-,
practice- and patient-level. Finally, we have also learned that case
managers use a case management system. Are there other types of
health information technology that you use for North Carolina’s multipayer medical home initiative? We’d like to learn more about each of
these systems.
What experiences does your network and practices have in meeting
the Blue Quality Physician Program standards? What benefit does this
system bring to the network and practices?
How does your network and practices use CCNC’s web-based portal?
What benefit does this portal bring to the network and practices? How
often do you use it?
How do you use the Care Management Information System? What
benefit does this system bring? How often do case managers use it and
how?
Which of these systems allows you to track services provided to
patients?

Reason for Addition

Added to obtain information about
health information technology use
under the state’s initiative.

Added to obtain information about the
Blue Quality Physician Program
standards.
Added to obtain information about the
CCNC’s web-based portal.
Added to obtain information about the
Care Management Information System.

Added to gain a better understanding
of networks’ capacity to track services
provided to patients.
To what extent can you readily exchange health information with
Added to obtain information about
primary care practices? Other practices/facilities (e.g., specialists,
health information technology use
hospitals, long-term care facilities)?
under the state’s initiative.
We understand that your network receives daily admission, discharge, Added to gain a better understanding
and transfer lists for Medicare patients. How is it working? How far
of the hospital admission, discharge,
along are you in the process of receiving real time hospital admission
and transfer lists received by the
and discharge information for these patients and those covered by
networks.
BCBS or the State health Plan?

Protocol

Question #

Communitybased Care
Networks

29-29b

Communitybased Care
Networks

30-30d

31-31a

New Question Added

What challenges pertaining to health information technology do you
face? How has your participation in the NC demonstration changed the
amount or frequency with which you use health information
technology?
What additional health IT features would help your community health
network do a better job assisting/communicating with patients?
Practices?
What kinds of structural or organizational changes did your network
make pre-2011—before Medicare became involved in the NC
demonstration—in an effort to support the medical home model?
What additional changes are you working on making now that
Medicare is involved in the NC demonstration? What are the major
challenges to implementing these additional changes?
We understand that CCNC has hosted a series of webinars and
developed a toolkit to help practices achieve NCQA recognition. We
also know that the quality improvement coaches assist practices
through the state’s Area Health Education Centers. Can you tell use
more about these activities? What is the role of quality improvement
coaches? How did these tools and resources help networks and
practices? How has the ASU PCMH practicum study helped practices to
achieve NCQA PCMH recognition? Is this activity on-going?
Now we’d like to discuss with you how beneficiaries access care and
their experiences with this care. Please comment on patients overall,
but also on whether there are any differences for Medicare or
Medicaid beneficiaries or special populations of patients. Special
populations can include: Medicare and Medicaid dual eligible;
children; racial and ethnic subgroups; people living in rural or innercity areas; and persons with chronic illnesses, mental illnesses, and
disabilities. How do you identify people who need help or how do they
get connected with your network? Please describe how you identify
Medicare or Medicaid beneficiaries for case management, clinical
pharmacy, or any additional services?

Reason for Addition

Added to obtain information about
health information technology use
under the state’s initiative.
Added to obtain information about
health information technology use
under the state’s initiative.
Added to obtain information about the
organizational changes that networks
had to make.

Added to obtain information about
networks’ activities related to NCQA
recognition.

Added to obtain information about
patients’ access to care under the
state’s initiative.

Protocol

Communitybased Care
Networks

Question #

32-32f

33-33k

34-34b

New Question Added

How familiar are you with Medicare’s beneficiary assignment process?
Do you limit network services to those patients for whom a
participating insurer pays a monthly fee and/or only those
beneficiaries who have been assigned to the practice? Please describe
briefly. How do you currently invite patients to participate? Please tell
me about the activities of Patient Outreach Teams. What patient
education tools do they provide to patients? How do they distribute
those tools? How are patients transitioning between two practices
participating in the demonstration identified and handled by the
community health network? Please describe briefly. What about
patients transitioning from participating practices to non-participating
practices or vice versa? How are these patients identified and handled
by community health network? How does your network coordinate
patient care with primary care practices and other providers or
facilities?
How does the network or its practices: assign a staff member within
your community health network to a patient? Follow up with patients
after they are discharged from a hospital? ER? Identify patients for
medication reconciliation? Coordinate care with specialists? Provide
linkages to other services and facilities like long-term care, mental
health, community services, or social services? Work with patients to
address challenges they may face accessing care? Work with patients
to address challenges they may face caring for themselves? Work with
special populations such as the mentally ill to address any challenges
they may face? Work with patients to address challenges they may
face caring for themselves? Other? Are there any differences in how
you or the practices coordinate care for Medicare or Medicaid
beneficiaries, or other special populations?
How do the practices that are in your network communicate with
patients who do not speak or cannot read English? How do practices
identify patients with different language or literacy needs? Who does
the screening? What tools do they use? Do practices that are part of
your network and this demonstration offer translation services?

Reason for Addition

Added to gain a better understanding
Medicare’s beneficiary assignment
process.

Added to gain a better understanding
of a range of networks activities under
the state initiative.

Added to obtain information about how
practices communicate with patients
who cannot speak or read English.

Protocol
Communitybased Care
Networks

Question #

35-35a

Communitybased Care
Networks

36-36b

Communitybased Care
Networks

38-38b

Communitybased Care
Networks
Communitybased Care
Networks
Communitybased Care
Networks

39
40
41

New Question Added

What strategies do the practices that are part of your network and this
demonstration use to engage patients in their care? To what extent
are patients, their families, and/or their caregivers actually able to
participate more effectively in decisions concerning their care as a
result of the NC demonstration? Can you provide an example or do
you have any early data on this?
How do the network or practice staff teach self-management to
patients? How do patients use self-management notebooks? In your
opinion, to what extent are patients actually better able to selfmanage their health conditions or engage in healthy behaviors as a
result of the NC demonstration? Can you provide an example or do
you have any early data on this?
Does your network or participating practices experience any
challenges in reaching Medicare or Medicaid beneficiaries, in an effort
to improve quality and safety? If so please describe briefly.
What quality and safety data does your network or participating
practices collect? Were any quality and safety measures added or
dropped since 2011, when Medicare joined the demonstration? Which
preventive care services data do your network and participating
practices collect or share? These might include cancer screening,
smoking cessation, weight management, influenza vaccination,
pneumonia vaccination. Please describe briefly. What quality and
safety measures are reported back to your network? Are any of these
measures new, since 2011?
How often does CCNC provide these reports?
How do you use the information provided in these reports?
What are the most useful features of the quality measure reports you
receive?

Reason for Addition

Added to obtain information about
patient engagement.

Added to obtain information about
patient self-management.

Added to obtain information about
quality and safety under the state
initiative.
Added to obtain information about
quality and safety under the state’s
initiative.

Added to obtain information about
quality and safety under the state’s
initiative.
Added to obtain information about
quality and safety under the state’s
initiative.
Added to obtain information about
quality and safety under the state’s
initiative.

Protocol

Communitybased Care
Networks

Question #

42-42f

Communitybased Care
Networks

43-43b

Communitybased Care
Networks

New Question Added

Does your network or its assigned practices receive data summarizing
other dimensions of your patients’ care, such as their utilization of
health care services? (for example, statistics on your patients’ hospital
or nursing home admissions or readmissions.) [If yes:] What quality
and safety measures are reported back to you? Which of these are
measures that you received feedback on prior to 2011—before
Medicare joined the NC demonstration? Which of these measures (if
any) did you begin to receive feedback on in 2011—when Medicare
joined the NC demonstration? What entity provides you with these
reports, and how often? How do you use the information provided in
these reports? What are the most useful features of the quality
measure reports you receive? How could these quality measure
reports be made more useful to you? [If no:] If you were to receive a
quality measure report, what quality measures would be the most
useful to you? If you were to start receiving these types of data on
your Medicare patients, how might you use it?
What major areas will your network focus on in the next year? What
changes will you make? What do you see as the facilities (or critical
factors) of successful implementation of this demonstration? What did
you see as the barriers or major challenges to implementing the
changes that are part of this demonstration?
What advice would you give to other networks if the NC
demonstration were to be extended or expanded to include them?

Reason for Addition

Added to obtain information about the
reports that networks and/or practices
receive under the state’s initiative.

Added to obtain information about
future activities under the state’s
initiative

Added to obtain information about
advice that the respondent would give
others.
What advice would you give to CMS (Medicaid and Medicare)? Given
Added to obtain information about
what you know now, what would you have done differently, both prior advice that the respondent would give
to and after Medicare’s involvement in the NC medical home
others.
initiative?
Is there anything we have not discussed about the NC medical home
Added to obtain any other information
demonstration or about the Medicare MAPCP Demonstration that you that was not discussed during the
feel would be important for our National Evaluation Team to know?
interview.

Protocol

NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers
NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers
NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers
NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers
NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers
NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers

Question #

New Question Added

Are you employed by the [name of network]?

Added to obtain background
information on the respondent.

How long have you worked at [name of network]?

Added to obtain background
information on the respondent.

With how many practices or health centers do you work? For how
many patients do you coordinate care per month? Which insurers do
your patients represent?

Added to obtain background
information on the respondent and
his/her responsibilities.

What kind of services do you provide to your practices and/or your
patients? What specific types of patients do you focus on? (e.g.,
patients with comorbidities, Medicare and Medicaid dual eligible;
children; racial and ethnic subgroups; people living in rural or innercity areas; persons with Chronic illnesses, mental illnesses, and
disabilities). How do you decide which patients to focus on?
How do you communicate with physicians and other staff in your
practice? What kind of information do you relay to providers for
patients’ care?

Added to obtain background
information on the respondent and
his/her responsibilities.

What training do you receive? Who provides it?

Added to obtain background
information on the respondent.

1 (Care
Managers)

2 (Care
Managers)

3-3b (Care
Managers)

4-4b (Care
Managers)

5-5a (Care
Managers)

6 (Care
Managers)

Reason for Addition

Added to obtain information about
communication between networks and
practices.

Protocol

NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers
NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers

NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers

Question #
7 (Care
Managers)

8-8b (Care
Managers)

New Question Added

Reason for Addition

What kinds of health information technology do you use in case
management?

Added to obtain information about
health information technology use.

How have EHRs and other health IT changed the way you manage
specific cases? What particular features or capabilities have been
especially helpful? What technical assistance for the health IT is
available to you?

Added to obtain information about
health information technology use.

What are some problems with how care is currently delivered to rural,
aging, or chronically ill patients in North Carolina? To what extent do
you see problems with: Patients having access to care when they need
it? Could you give me an example of that? Effective communication
between these patients and practice staff (including doctors and other
staff)? Could you give me an example of that? Does this include
9-9b (Care communication related to shared decision-making? Does this include
Managers) communication related to self-management? Quality of care? Could
you give me an example of that? Care coordination, specifically in
instances where patients visit a specialist or are seen in a hospitalsetting and require follow-up from their primary care provider?
Patients being able to better self-manage their health and medical
conditions? Do you see any major differences for Medicare
beneficiaries, Medicaid beneficiaries, or other special populations?
For the problems that you’ve already identified in caring for rural,
aging, and chronically ill patient populations, to what extent is
10-10a (Care
Medicare’s participation in the NC demonstration helping to address
Managers)
those issues? How does the NC demonstration improve care for these
special populations? Could you give me an example of that?

Added to obtain information about
problems in care delivery.

Protocol

Question #
11 (Care
Managers)

New Question Added

What do you think are the strengths and weaknesses of the medical
home model of care?

Added to obtain background
information on the medical home
model.

How do you coordinate with BCBS case managers?

Added to obtain information about the
respondent’s responsibilities.

12 (Care
Managers)
We understand that the NC Division of Aging and Adult Services
provided training/education sessions regarding Medicare benefits and
13-13a (Care
community resources. Did you attend that session? [If Yes:] What is
Managers)
helpful? In what ways?

14-14j (Care
Managers)

15 (Care
Managers)

16 (Care
Managers)

Reason for Addition

In the NC Demonstration practices, is there evidence of improvement
in: access to care; coordination of care; increased adherence to
preventive services; reduced acute care utilization, like ED visits,
hospitalizations, readmissions; patient experience/satisfaction; selfmanagement of health conditions; engagement in healthy behaviors;
shared decision making between primary care providers and patients,
their family members, and/or caregivers; health; other?
What recommendations do you have to improve the NC
demonstration?

Added to obtain information about the
respondent’s responsibilities.

Added to obtain information about
evidence of improvement.

Added to obtain information about
areas of the state’s initiative that could
be improved.

What are your goals for improving the care delivered to patients in the Added to obtain information about
next year? What are the facilitators of or barriers to their
future plans.
achievement?

Protocol

NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers
NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers

Question #
17 (Care
Managers)

1-1b (DAAS)

NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers

2-2b (DAAS)

NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers

3 (DAAS)

New Question Added

Is there anything else you would like to share about the NC
demonstration that we did not cover today?

Reason for Addition

Added to obtain information about
anything that might not have been
covered in the interview.

Could you tell us a little about your role? How long have you worked at Added to obtain background
the DAAS? What has your role been in the NC multi-payer medical
information on the respondent and
home demonstration?
his/her responsibilities.
What are some problems with how care is currently delivered to rural, Added to obtain information about
aging, or chronically ill patients in North Carolina? To what extent do
problems in care delivery.
you see problems with: patients having access to care when they need
it? Could you give me an example of that? Effective communication
between these patients and practice staff (including doctors and other
staff)? Could you give me an example of that? Does this include
communication related to shared decision-making? Does this include
communication related to self-management? Quality of care? Could
you give me an example of that? Care coordination, specifically in
instances where patients visit a specialist or are seen in a hospitalsetting and require follow-up from their primary care provider?
Patients being able to better self-manage their health and medical
conditions? Do you see any major differences for Medicare
beneficiaries, Medicaid beneficiaries, or other special populations?
How has the NC Demonstration addressed some of these issues?
Added to obtain information about
problems in care delivery and the
state’s initiative.

Protocol

Question #

New Question Added

Which of these issues remain unaddressed by the NC Demonstration?

Added to obtain information about
problems in care delivery and the
state’s initiative.

In the NC Demonstration practices, is there evidence of improvement
in: access to care; coordination of care; increased adherence to
preventive services; reduced acute care utilization, like ED visits,
hospitalizations, readmissions; patient experience/satisfaction; selfmanagement of health conditions; engagement in healthy behaviors;
shared decision making between primary care providers and patients,
their family members, and/or caregivers; health; other?
How has health IT changed care management in practices that are
participating in the demonstration?

Added to obtain information about
evidence of improvement under the
state’s initiative.

4 (DAAS)

5-5j (DAAS)

6 (DAAS)

Reason for Addition

Added to obtain information about
health information technology use.

7 (DAAS)

How could CMS or North Carolina change the demonstration to better Added to obtain information about
serve the needs of the [list earlier identified special populations]? Why special populations being served under
would that help?
the state’s initiative.

8 (DAAS)

If a colleague in another state was interested in advocating for a
program like the NC demonstration, what advice would you give
them?

Added to obtain information about
areas of the state’s initiative that could
be improved.

What are your goals for improving the care delivered to aging and
chronically ill patients in the context of this demonstration in the next
year? What are the facilitators of or barriers to their achievement?

Added to obtain information about
future plans.

9-9a (DAAS)

Protocol

NC Division of
Aging and Adult
Services (DAAS)
& Care
Managers
Provider
Organization /
Organized
Delivery System
Provider
Organization /
Organized
Delivery System

Provider
Organization /
Organized
Delivery System

Question #
10 (DAAS)

3-3d

New Question Added

Reason for Addition

Is there anything else you would like to share about the NC
demonstration that we did not cover today?

Added to obtain information about
anything that might not have been
covered in the interview.

How long have you been with [name of physician organization /
organized delivery system]?

Added to obtain background
information on the respondent and
his/her responsibilities.

Please tell me about your roles and responsibilities with [name of
physician organization / organized delivery system].

Added to obtain background
information on the respondent and
his/her responsibilities.

Please tell me about our [physician organization / organized delivery
system] overall. For example, how many physicians and practice sites
or locations do you operate? How many hospitals do you own and
operate? What is your approximate payer mix, particularly your
percent Medicare and Medicaid? What is the mix of health care
professionals and other team members or staff in your [name of
physician organization / organized delivery system]? Where is your
[physician organization / organized delivery system] located? Where
do your staff physically work (e.g., on site at the practices)? Please
describe the practices in this Chronic Care Initiative, including how
they compare to other practices in your physician organization or
organized delivery system that are not participating in the Chronic
Care initiative. Please describe what other departments or individuals
in the [name of physician organization / organized delivery system]
provide support to the participating practice for medical home
development and other activities related to the Chronic Care Initiative.
What is your [physician organization / organized delivery system]’s
primary role with respect to medical home development and practice
transformation?

Added to obtain background
information on the organization.

Added to obtain information about the
organization’s medical home activities.

Protocol

Question #

Provider
Organization /
Organized
Delivery System

5-5g

Provider
Organization /
Organized
Delivery System

7-7a

Provider
Organization /
Organized
Delivery System

Provider
Organization /
Organized
Delivery System
Provider
Organization /
Organized
Delivery System

8-8b

9-9b

New Question Added

Reason for Addition

Which of the following activities does your [physician organization /
organized delivery system engage in to support the practices: internal
learning collaboratives? Practice coaching or facilitation? Leadership or
staff training? EHR/HIT and disease registry support? Performance
measurement and monitoring? Performance incentives? Other
contracts with practices or other health care provider organizations?
How does your [physician organization / organized delivery system]
interact with practices?

Added to obtain information about the
organization’s support to practices.

If you were involved in Pennsylvania’s Chronic Care Initiative
implemented prior to January 2012, what features of the Chronic Care
Initiative are most important for your organization and participating
practices? What were the major strategies used prior to January 2012
to implement medical homes? How successfully were they
implemented?
Now that Medicare and Medicaid have joined the effort, what features
of the Chronic Care Initiative are most important for your organization
and participating practices? What changes were made to
accommodate Medicare’s participation, if any, or to accommodate the
Medicare beneficiaries now being served by the Chronic Care
Initiative? How does your own [physician organization / organized
delivery system]’s activities fit with activities sponsored by the state,
such as the learning collaborative, practice coaching, performance
measurement and monitoring?
How is the implementation of the Chronic Care Initiative with
Medicare and Medicaid’s involvement going so far? What has gone
well? What has gone less well?

Added to obtain information about the
organization’s perspective on the
state’s initiative.

What do you think are the strengths and weaknesses of the Chronic
Care Initiative?

Added to obtain information about the
organization’s perspective on the
state’s initiative.

Added to obtain information about the
organization’s support to practices.

Added to obtain information about the
organization’s perspective on the
state’s initiative.

Protocol

Provider
Organization /
Organized
Delivery System
Provider
Organization /
Organized
Delivery System
Provider
Organization /
Organized
Delivery System

Provider
Organization /
Organized
Delivery System

Question #
11

Provider
Organization /
Organized
Delivery System

15-15b

Provider
Organization /
Organized
Delivery System

16-16b

New Question Added

Reason for Addition

What new or persistent challenges do you anticipate in the future, if
any?

Added to obtain information about the
organization’s perspective on the
state’s initiative.

What new opportunities for improvement do you anticipate in the
future, if any?

Added to obtain information about the
organization’s perspective on the
state’s initiative.

Prior to Medicare joining the Chronic Care Initiative, participating
practices received payments from Medicaid and private payers to
engage in medical home-related activities. How do the participating
practices use these medical home payments? For example, did they
invest in disease registries or EHR infrastructure, hired additional staff,
etc?
We understand that as a participant in Pennsylvania’s Chronic Care
Initiative, your [office / clinic / hospital] receives two types of per
member per month (PMPM] payments: the physician coordinated care
oversight services PMPM and the coordinated care fees PMPM. We
also understand that practices are eligible for shared savings
payments. What are the strengths and weaknesses of the shared
savings payment component with respect to supporting continued
practice transformation and key outcomes?
Are participating practices engaging in additional activities now that
they also are receiving medical home payments from Medicare? [If so:]
what are those activities? [If not:] Why not?

Added to obtain information about the
state initiative’s payment model.

Do you think that Medicare’s medical home payments are adequate to Added to obtain information about the
allow your [physician organization / organized delivery system] and
state initiative’s payment model.
the practices you support to continue to invest in medical home
development and sustain effective medical home activities? What
kinds of infrastructure or care processes do the Medicare medical
home payments support? What kinds of activities are the medical
home payments not sufficient to support but that could be beneficial?

Protocol

Provider
Organization /
Organized
Delivery System
Provider
Organization /
Organized
Delivery System

Provider
Organization /
Organized
Delivery System

Provider
Organization /
Organized
Delivery System
Provider
Organization /
Organized
Delivery System

Question #
17

18-18a

19-19d

New Question Added

Reason for Addition

What are the strengths and weaknesses of the current payment
methodology (since Medicare joined the Chronic Care Initiative) for
[physician organizations / organized delivery systems]? For the
participating practices?
How are employed physicians generally paid in [name of physician
organization / organized delivery system]? Is it primarily productivity
based or is there some portion tied to performance metrics or other
factors? Has the [physician organization / organized delivery system]’s
payment model been altered for employed physicians since this
Chronic Care Initiative began? If so, how. Please briefly describe.
What types of health information technology capabilities does [name
of physician organization / organized delivery system] use to carry out
its functions? For example, do you use web portals maintained by
payers, electronic disease registries, or electronic health records? Can
you track services provided to patients in each participating practice
and other owned or affiliated practices? If so, how do you do this? To
what extent can you readily exchange health information with
participating practices? Other practices or facilities (e.g., specialists,
hospitals, long-term care facilities)? What are the major benefits of the
health information technology that you use? What are the primary
challenges you face, either because of any health information
technology capabilities you are lacking or with the health IT that you
have?
What additional health IT features would help your [physician
organization / organized delivery system] do a better job assisting
participating practices? Providing care coordination to patients?

Added to obtain information about the
state initiative’s payment model.

Has your [physician organization / organized delivery system]
registered for Medicare and/or Medicaid meaningful use (MU)? If so,
which program and have you attested to adopt, implement, and
upgrade (AIU) or stage 1 MU?

Added to obtain information about
health information technology use.

Added to obtain information about the
state initiative’s payment model.

Added to obtain information about
health information technology use.

Protocol

Question #

Provider
Organization /
Organized
Delivery System

22-22e

New Question Added

Reason for Addition

We understand that before Medicare joined, practice performance
Added to obtain information about
was measured and monitored on a relatively small set focused on
quality measurement under the state’s
diabetes and pediatric asthma. These were collected by practices and initiative.
reported to Pennsylvania Academy of Family Physicians (PAFP) and the
Improving Performance in Practice (IPIP) initiative. Now that Medicare
has joined, the set of measures has been expanded to preventive
services for children and adults, as well as care for patients with
diabetes, asthma (age 5-40), hypertension, and ischemic vascular
disease. These measures are reported to PAFP and then the state.
What is your organization’s view of the measure set? What are the
strengths and weaknesses of these measures? How does the [name of
physician organization / organized delivery system] assist in these
quality measurement efforts? What type of results do they focus on in
these quality measurement efforts? Are these quality results broken
out by payer type (e.g., Medicare, Medicaid, commercial payers?)
What does your [physician organization / organized delivery system]
usually do with these quality results? How about the participating
practices? What specific changes, if any, have participating practices
made based on these results? Have participating practices taken any
special actions targeted at Medicare, Medicaid, or other special
patient populations? Are there any special issues or challenges to
reaching Medicare or Medicaid beneficiaries or special populations?
Please describe briefly. We understand that under the shared savings
component, practices are eligible for shared savings payments that will
take into consideration practice performance on key quality and cost
metrics. What are the strengths and weaknesses of using these
measures to determine the distribution of shared savings?

Protocol

Question #

Provider
Organization /
Organized
Delivery System

23-23c

Provider
Organization /
Organized
Delivery System

24-24d

Provider
Organization /
Organized
Delivery System

25-25b

Provider
Organization /
Organized
Delivery System

26-26bii

New Question Added

Practices participating in the Chronic Care Initiative have been
receiving beneficiary utilization files from CMS/RTI that show some
quality measures, and hospital and ER utilization information (dates,
principal diagnosis, hospital name) for their Medicare patients. Are you
familiar with these files? If so, what do practices do with these data?
What aspects of those files do you believe have been most useful in
helping participating practices change the way they deliver care? What
features are not as helpful, or need improvement? What specific
changes, if any, have participating practices made based on these
data?
Do you monitor practices’ utilization and cost information? [If yes:]
what do you monitor? What do you do with the data you collect? Do
you provide feedback to practices? Do you help them develop
strategies to improve on any performance measures where they
appear to have quality gaps?
Does [name of physician organization / organized delivery system] hire
care managers to provide care coordination? [If yes:] Please describe
the services they provide. Where do they work? [If no:] Why not?

Reason for Addition

Added to obtain information about
utilization reports received by practices
participating in the state’s initiative.

Added to obtain information about the
organization’s activities related to
utilization and cost monitoring.
Added to obtain information about the
organization’s care management
activities.

Now we’d like to discuss with you how beneficiaries access care
Added to obtain information about the
coordination or management services and their experiences with it.
organization’s care management
Please comment on patients overall, but also on whether there are any activities.
differences for Medicare or Medicaid beneficiaries or special
populations of patients. Special populations can include: Medicare and
Medicaid dual eligible; children; racial and ethnic subgroups; people
living in rural or inner-city areas; and persons with chronic illnesses,
mental illnesses, and disabilities. How do you identify people who
need care coordination and management help or how do they get
connected with these staff in the participating practices or [name of
your organization]? Are there any differences in how your practices or
organization identify or are assigned Medicare or Medicaid
beneficiaries, or other special populations? [If yes:] Please describe. [If
no:] Why not?

Protocol

Question #

Provider
Organization /
Organized
Delivery System

28-28b

Provider
Organization /
Organized
Delivery System
Provider
Organization /
Organized
Delivery System
Provider
Organization /
Organized
Delivery System

29-29a

Provider
Organizations

New Question Added

Reason for Addition

What major areas will your [physician organization / organized delivery
system] be focusing on in the next year, or what changes will you be
making? What do you see as the facilitators (or critical factors) of
successful implementation? What do you see as the barriers or major
challenges to implementing these changes?
What advice would you give to other states or [physician organizations
/ organized delivery systems] if the Chronic Care Initiative were to be
extended or expanded to include them? What advice would you give
to CMS (Medicaid and Medicare)? Any particular advice on the role of
[physician organization / organized delivery systems] in medical home
implementation?
Give what you know now, what would you have done differently, both
prior to and after Medicare and Medicaid’s involvement in the Chronic
Care Initiative?

Added to obtain information about the
organization’s future plans.

Is there anything we have not discussed about the Chronic Care
Initiative or about the MAPCP Demonstration that you feel would be
important for our National Evaluation Team to know?

Added to obtain information about
anything that might not have been
covered in the interview.

How long have you been with [name of physician organization]?

Please tell me about your roles and responsibilities with [name of
physician organization].
Please describe the mix of health care professionals and other team
members or staff in your [name of physician organization].

Added to obtain information about
areas of the state’s initiative that could
be improved.

Protocol

Question #

New Question Added

Provider
Organizations

9-9b

Provider
Organizations

What are your physician organization’s primary activities? What are
your major activities with practices? How does your physician
organization interact with practices? What types of infrastructure and
support does your physician organization provide to practices to
support medical home development and medical home activities?
What role do you have in distributing incentives to MiPCT? For other
incentive programs?
Where is your physician organization located? Where do your staff
physically work (e.g., on site at the practices)? How about your care
coordinators?
What features of the Michigan Primary Care Transformation Project do
you think are the most important? What do you think are the
strengths and weaknesses of the Michigan Primary Care
Transformation Project?
Were you involved in the BCBSM PCMH PGIP prior to January 2012? [If
yes:] What were its most important features? What were the major
strategies used to implement medical homes during that time period
before January 2012? How successfully were those strategies
implemented?
What changes, if any, were made to the BCBSM PCMH PGIP or
Michigan Primary Care Transformation Project to accommodate
Medicare joining?
How is the implementation of the Michigan Primary Care
Transformation Project with Medicare and Medicaid’s involvement
going so far? What has gone well? What hasn’t gone so well?
What new or persistent challenges do you anticipate in the future, if
any?

Provider
Organizations

What new opportunities for improvement do you anticipate in the
future, if any?

Provider
Organizations

4-4d

Provider
Organizations

5-5a

Provider
Organizations

6-6a

7-7b

Reason for Addition

Added to obtain information about the
organizations activities related to
practices.

Added to obtain background
information on the organization.
Added to obtain information about the
respondent’s perspective on the state’s
initiative.
Added to obtain information about the
respondent’s (and organization’s) role
in the state’s initiative.
Added to obtain information about the
organization’s role in the state’s
initiative.
Added to obtain information about the
respondent’s perspective on the state’s
initiative.
Added to obtain information about the
respondent’s perspective on the state’s
initiative.
Added to obtain information about the
respondent’s perspective on the state’s
initiative.

Protocol
Provider
Organizations
Provider
Organizations

Provider
Organizations

Provider
Organizations
Provider
Organizations

Question #
12

15-15b

16-16a

New Question Added

How were physician organizations paying for the support they
provided to practices prior to January 2012—before Medicare and
Medicaid joined MiPCT? What major changes, if any, have been made
since Medicare and Medicaid joined? How are they paid now?
We understand that each payer contributes to an incentive pool that
distributed to the POs based on their performance and improvement.
How are those payments distributed to practices? How do you think
these payments will be used?
Prior to Medicare and Medicaid joining the Michigan Primary Care
Transformation Project, participating practices received payments
from Blue Cross Blue Shield of Michigan to engage in medical homerelated activities. Do you know how the participating practices used
these medical home payments?
What kinds of infrastructure or care processes do the Medicare
medical home payments support at the PO level? At the practice level?
Are participating practices engaging in any additional activities now
that they are receiving Medicare and Medicaid medical home
payments on top of its medical home payments from private payers?
[If so:] What are those activities? [If not:] Why not?
Do you think that Medicare’s medical home payments are adequate to
allow your physician organization and the practices you serve to
continue to invest in medical home development and sustain effective
medical home activities? What kinds of activities are the Medicare
payments not sufficient to support but that could be beneficial?
What are the strengths and weaknesses of the current payment
methodology (since Medicare and Medicaid joined for this
demonstration) for physician organizations? For practice?

Reason for Addition

Added to obtain information about the
payment model under the state’s
initiative.
Added to obtain information about the
payment model under the state’s
initiative.
Added to obtain information about the
payment model under the state’s
initiative.
Added to obtain information about the
organization’s support to practices.

Added to obtain information about the
payment model under the state’s
initiative.
Added to obtain information about the
payment model under the state’s
initiative.

Protocol

Question #

Provider
Organizations

18-18d

Provider
Organizations

20-20e

Provider
Organizations

21-21c

Provider
Organizations

22-22b

New Question Added

What types of health information technology capabilities does [name
of physician organization] use to carry out its functions? For example,
do you use web portals maintained by payers or electronic disease
registries? Can you track services provided to patients? If so, how do
you do this? To what extent can you readily exchange health
information with primary care practices? Other practices/facilities
(e.g., specialists, hospitals, long-term care facilities)? What are the
major benefits of the health information technology that you use?
What are the primary challenges you face, either because of any
health information technology capabilities you are lacking or with the
health IT that you have?
What additional health IT features would help your physician
organization do a better job assisting practices? Providing care
coordination to patients?
We understand POs assist practices with collecting data and
submitting reports to MiPCT. What types of data do practices collect?
Chronic condition data? Quality and safety data? How does your PO
assist in the data collection and submission? What do you think
practices usually do with these quality results? What specific changes,
if any, have participating practices made based on these results? How
is this different from what practices did before January 2012 as part of
BCBSM’s PCIP PCMH initiatives?
Do you monitor practice’s utilization and cost information? [If yes:]
What do you monitor? What do you do with the data you collect? Do
you provide feedback to practices?
Does [name of physician organization] hire care managers to provide
care coordination? [If yes:] Please describe the services you provide. [If
no:] Why not?

Reason for Addition

Added to obtain information about
health information technology use.

Added to obtain information about
health information technology use.
Added to obtain information about
data collection activities.

Added to obtain information about the
organization’s utilization and cost data
monitoring.
Added to obtain information about the
organization’s activities related to care
managers.

Protocol

Provider
Organizations

Provider
Organizations
Provider
Organizations

Question #

23-23aii

25-25e

26-26c

27-27c

28-28b

New Question Added

How do you identify people who need help or how do they get
connected with care managers? Are there any differences in how you
identify or are assigned Medicare or Medicaid beneficiaries, or other
special populations, for care coordination services? Special
populations can include: Medicare and Medicaid dual eligible;
children; racial and ethnic subgroups; people living in rural or innercity areas; and persons with chronic illnesses, mental illnesses, and
disabilities. [If yes:] Please describe. [If no:] Why not?
How do care coordinators coordinate patient care between the
primary care practices you work with and other providers or facilities,
such as hospitals, long-term care facilities, medical or surgical
specialists, and behavioral health providers?
Do you, and if so how do you: Follow up with patients after they are
discharged from a hospital? ER? Perform medication reconciliation?
Provide linkages to community services, or social services? Work with
patients to address challenges they may face accessing care? Work
with patients to address challenges they may face caring for
themselves?
What are some reasons why you might contact patients? How do the y
react when you contact them? How successful are these contacts with
patients? To what extent do you assist patients in getting them access
to non-clinical supports (e.g., transportation to doctor appointments,
social or community-based health services) that could benefit their
health or access to needed health care?
To what extent are you in communication with patient’s family
members or other caregivers? What are some reasons why you might
contact them? How do they react when you contact them? How
successful are these contacts?
What are some reasons why you might contact other providers,
practices, or facilities (e.g., hospitals, pharmacies)? How do they react
when you contact them? How successful are these contacts with other
providers, practices, or facilities?

Reason for Addition

Added to obtain information about the
organization’s care coordination
activities.

Added to obtain information about the
organization’s care coordination
activities.
Added to obtain information about the
organization’s care coordination
activities.

Added to obtain information about the
organization’s care coordination
activities.

Added to obtain information about
communicating with patients’ families.
Added to obtain information about the
organization’s care coordination
activities.

Protocol
Provider
Organizations

Question #
29-29b

Provider
Organizations

30-30b

Provider
Organizations

31-31a

Provider
Organizations

Provider
Organizations
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses

New Question Added

How do you track and manage information you receive from patients,
their caregivers, and their other providers? How are patients’ practice
clinicians and staff made aware of pertinent information you gather
from these various sources? How does this information get
incorporated into the patient’s medical records?
What major areas will your physician organization be focusing on in
the next year, or what changes will you be making? What do you see
as the facilitators (or critical factors) of successful implementation?
What do you see as the barriers or major challenges to implementing
these changes?
What advice would you give to other states or physician organizations
if the Michigan Primary Care Transformation Project were to be
extended or expanded to include them?
Given what you know now, what would you have done differently,
both prior to and after Medicare and Medicaid’s involvement in the
Michigan Primary Care Transformation Project?
Is there anything we have no discussed about the Michigan Primary
Care Transformation Project or about the Medicare MAPCP
Demonstration that you feel would be important for our National
Evaluation Team to know?
How long have you been with [name of organization]?

Added to obtain information about the
organization’s care coordination
activities.

Please tell me about your role with the SASH program.

Added to obtain information about the
respondent’s background and
responsibilities.

How long have you served in this role?

Added to obtain information about the
respondent’s background and
responsibilities.

Reason for Addition

Added to obtain information about the
organization’s future plans.

Added to obtain the respondent’s
advice to others.
Added to obtain general information
about the state’s initiative.
Added to obtain information that the
respondent thinks might be important
but was not covered during the
interview.
Added to obtain information about the
respondent’s background and
responsibilities.

Protocol

SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses

Question #

New Question Added

Were you previously with a similar kind of organization?

5-5a

Added to obtain information about the
respondent’s background and
responsibilities.

Please tell me about the staff composition of your SASH team. Please Added to obtain background
describe the mix of health care professionals and other team members information about the organization.
or staff in your [name of organization].
[For SASH Wellness Nurses:] We understand that the primary
responsibilities of a wellness nurse is to provide in-person coaching on
proper medication management, monitor vital signs to look for early
warnings of health complications, and provide intensive self-care
counseling and education post-discharge from hospitals and nursing
homes. Do you have any other primary responsibilities?
[For SASH Coordinator:] We understand that the primary
responsibilities of a SASH coordinator includes providing in-person
needs assessments, motivational coaching to help residents meet their
personal health aging goals, daily visits with high-risk residents to
ensure medication compliance, and social services coordination. Do
you have any other primary responsibilities?
Where do you physically work?

Added to obtain background
information about SASH wellness
nurses.

From your perspective, what are the most significant problems that
practices involved in the Blueprint face in serving Medicare
beneficiaries in Vermont?

Added to obtain information about the
respondent’s perspective on the state’s
initiative.

From your perspective, what are the most significant problems that
Medicare beneficiaries face in Vermont?

Added to obtain information about the
respondent’s perspective on the state’s
initiative.

Reason for Addition

Added to obtain background
information about SASH coordinators.

Added to obtain information about the
respondent’s activities.

Protocol
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses

Question #

11-11c

New Question Added

Given the challenges faced by practices and beneficiaries, what are the
strengths and weaknesses of the Blueprint for Health initiative? How
well do you think implementation of the Blueprint for Health is going?
Do you think the entrance of Medicare has enhanced the initiative?
What new opportunities for improvement od you anticipate in the
future, if any?
How do SASH staff coordinate patient care with primary care
practices?

Added to obtain information about the
respondent’s perspective on the state’s
initiative.

How do you coordinate patient care between the primary care
practices you work with and the other providers or facilities, such as
hospitals, long term care facilities, medical or surgical specialists, and
behavioral health providers?
What kind of services do you provide to practices in your area?

Added to obtain information about how
the organization coordinates with
practices.

Does your SASH team interact with care coordinators in any of the
practices in your area? If so, what is the nature of the interaction? Are
there coordination issues?

Added to obtain information about how
the organization coordinates with
practices.

15-15a

Reason for Addition

Added to obtain information about how
the organization coordinates with
practices.

Added to obtain information about the
organization’s activities.

What have been the challenges of working with practices in your area? Added to obtain information about how
What is working well?
the organization coordinates with
practices.

Protocol

SASH
Coordinators/
SASH Wellness
Nurses

Question #

17-17e

18-18e

SASH
Coordinators/
SASH Wellness
Nurses

20-20h

SASH
Coordinators/
SASH Wellness
Nurses

New Question Added

Reason for Addition

Please describe how you coordinate with the CHTs for Medicare
beneficiaries that are also participating in the SASH Program? What
kinds of information does your SASH team exchange with the CHT
team or vice versa? How do you interact with the CHT team members?
What is the nature of the interactions? Are there coordination issues?
Is there joint decision making with the CHT teams on who will be doing
what with which patients? How do you communicate with the CHT
team members? What have been challenges of working with the CHT
teams?
Do you do any coordination with the Medicaid Care Coordinators for
Medicaid patients with complex care needs? What kind of information
do you exchange with the Medicaid CCs or vice versa? How do you
interact with the Medicaid CCs? What is the nature of the interaction?
Are there coordination issues? Is there joint decision making with the
Medicaid CCs on who will be doing what with these patients? How do
you communicate with the Medicaid CCs? What have been the
benefits or challenges of working with the Medicaid CCs?
How do you identify people who need help or how do they get
connected with a SASH team?

Added to obtain information about how
the organization coordinates with CHTs.

Do you, and if so how do you: Assign a staff member within your SASH
team to a patient? Follow up with patients after they are discharged
from a hospital? ER? Perform medication reconciliation? Coordinate
care with specialists? Provide linkages to other services and facilities
like long-term care, mental health, community services, or social
services? Work with patients to address challenges they may face
accessing care? Working with patients to address challenges they may
face caring for themselves? Other?
What are patients’ reactions when you contact them?

Added to obtain information about the
organization’s activities.

Added to obtain information about how
the organization coordinates with
Medicaid CCs.

Added to obtain information about how
people are connected to SASH.

Added to obtain information about the
organization’s communications with
patients.

Protocol

SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses

Question #
22

23-23a

27-27a

New Question Added

Are there any special issues or challenges to reaching Medicare
beneficiaries? If so, please describe briefly.

Reason for Addition

Added to obtain information about the
organization’s communications with
patients.

What percentage of your patients have language literacy problems?
Added to obtain information about
How do you communicate with patients who have language or literacy potential literacy problems with
problems?
patients.
To what extent do you interact with caregivers and family members of Added to obtain information about the
Medicare beneficiaries? What is the nature of these interactions?
organization’s interactions with family
members and caregivers.
What strategies have the [SASH Coordinator/SASH wellness nurse]s
used to engage patients more in their care?

Added to obtain information about
patient engagement.

To what extent are patients, their families, and/or their caregivers
actually able to participate more effectively in decisions concerning
their care as a result of the Blueprint for Health initiative? Care you
provide an example or do you have any early data on this?
How are you teaching self-management to patients? To what extent
are patients actually better able to self-manage their health conditions
or engage in healthy behaviors as a result of the Blue-print for Health
initiative? Care you provide an example or do you have any early data
on this?
What role does your SASH team have in monitoring or improving
quality of care and patient safety?

Added to obtain information about
patient engagement.

Are you involved in the EQiuP activities? If so, how?

Added to obtain information about the
organization’s involvement in EQiuP
activities.

Added to obtain information about
patient self-management.

Added to obtain information about the
organization’s quality and patient
safety activities.

Protocol

SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses

SASH
Coordinators/
SASH Wellness
Nurses

Question #
30

32-32eiv

New Question Added

Reason for Addition

Do you use clinical data from the DocSite clinical registry for any
Added to obtain information about the
quality of care or patient safety improvement activities? If so, for what use of clinical data from DocSite.
purpose?
Does your CHT receive any other type of performance data on quality
and safety?

Added to obtain information about
other performance data.

We know that as part of the Vermont Blueprint for health, practices
Added to obtain information about
are required to enter into agreements with Vermont Information
health information technology use.
Technology Leaders (VITL) and demonstrate progress toward being
able to communicate with the Vermont statewide health information
exchange (VHIE) and the DocSite clinical registry. Are you also required
to use DocSite? [If yes:] Are you currently using DocSite? Have there
been any challenges to using DocSite? Do you submit clinical data to
DocSite? If so, what data? What information or services have you used
from DocSite? The visit planner (individualized visit plans based on
age, gender, and diagnosis that provide guideline based
recommendations for annual health maintenance, prevention, and
chronic disease treatment)? The integrated health record which
contains clinical information for patients receiving care also from other
providers? Populations level reports? If so, what kinds of reports?
Comparative performance reports? If so, at what level (providers
within a practice, across independent practices and organizations, or
across HSAs within the state)?

Protocol

SASH
Coordinators/
SASH Wellness
Nurses

Question #

33-33d

SASH
Coordinators/
SASH Wellness
Nurses

35-35c

New Question Added

What types of other health information technology capabilities do
SASH staff use to carry out their functions? For example, do you use
web portals maintained by payers or referral tracking databases? Can
you track services provided to patients? If so, how do you do this? To
what extent can you readily exchange health information with the
practices you work with? Other practices/facilities (e.g., specialists,
hospitals, long-term care facilities)? What are the major benefits of the
health information technology that you use? What are the primary
challenges you face, either because of any health information
technology capabilities you are lacking or with the health IT that you
have?
What additional health IT features would help your SASH team do a
better job assisting/communicating with patients? Practices?

Reason for Addition

Added to obtain information about
health information technology use.

Now I want to talk to you about the beneficiary utilization files that RTI Added to obtain information about
is producing for practices in the demonstration. The utilization files are RTI’s beneficiary utilization files.
Excel files that are posted on the MAPCP Web Portal that show all of
the assigned beneficiaries for the quarter for each practice, the
severity level (HCC risk score), disease status for diabetes and ischemic
vascular disease, some quality measures, and hospital and ER
utilization (dates, principal diagnosis, hospital name). Do you have
access to these files? [If yes:] What aspects or features of the
beneficiary utilization files do you find most useful? Is there
information from Medicare claims that could be added to make them
more useful? How are you using these files? Do you receive utilization
files from any other entity? If so, who?

Protocol
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses
SASH
Coordinators/
SASH Wellness
Nurses

Question #

36-36b

New Question Added

Reason for Addition

Now I want to talk to you about the Practice Feedback Reports that RTI Added to obtain information about
is producing for practices in the demonstration. These are PDF reports RTI’s Practice Feedback Reports.
summarizing each practice’s performance on a selection of quality
measures that are posted quarterly to the MAPCP Web Portal. Do you
have access to these reports? [If yes:] What aspects or features of the
Practice Feedback Reports do you find most useful? How are you using
these reports? Do you receive feedback repots like these from any
other entity? If so, who?
Are there other performance monitoring reports that you receive?
Added to obtain information about
other performance monitoring reports.
Are the payments you’re receiving through the Blueprint for Health
sufficient to support the needs of your patients and in working with
the community and community health teams to provide services?

Added to obtain information about the
payment model under the state’s
initiative.

Is there anything we have not discussed about the Blueprint for Health Added to obtain information that was
or about the Medicare MAPCP Demonstration that you feel would be not already covered in the interview.
important for our Evaluation Team to know?

Protocol

Question #

Organized
Delivery System

27-27b

New Question Added

Reason for Addition

Since Medicare joined the Chronic Care Initiative In January 2012,
Added to obtain information about
what impacts has it had on patients? Is there evidence of
impacts on care.
improvements in: access to care? coordination of care? (e.g. care
transitions). Patient and family caregiver participation or behavior (e.g.
patients engaging more in decisions and managing their care, use of
patient and family advisory boards for practice quality improvement
efforts). Increased delivery of preventive services? (e.g. cancer
screenings, smoking cessation, weight management, influenza
vaccination). Reduced use of acute care? (e.g. emergency department
visits, hospitalizations, readmissions). Improved health care quality,
patient safety, and patient experience and/or satisfaction? Other? If
Medicaid participation in [state-specific name of MAPCP
demonstration] prior to Medicare joining the initiative in [2011/2012],
what impact has the initiative had on Medicaid beneficiaries and
special populations? Special populations can include: Medicare and
Medicaid dual eligibles; children; racial and ethnic subgroups; people
living in rural or inner-city areas; and persons with chronic illnesses,
mental illnesses and disabilities.

File Type	application/pdf
File Title	CMS Response to Public Comments Received Regarding Planned Information Collection for the Evaluation of the Multi-payer Advanced
Subject	MAPCP, site visit, Multi-payer advanced primary care practice, CMS, RTI, Urban, NASHP
Author	Centers for Medicare & Medicaid Services
File Modified	2013-04-23
File Created	2013-04-22