Crosswalk

Table 1. Summary of Existing Focus Group Protocol Questions that Address Public Comments
Received Regarding CMS’s Planned Information Collection for the
Evaluation of the Multi-payer Advanced Primary Care Practice Demonstration
(Document Identifier: CMS-10479)
Commenter’s Feedback

Protocol

Question #

Ready Access: Focus group questions should
seek to learn from patients and their
caregivers:

9. How easy or hard is it for you to get an appointment with a
provider at this practice when you need one? [PROBES: Can
you schedule a same day appointment for urgent needs? Can
you schedule an appointment for nights, or weekends, or
holidays? Did the practice change its hours of operation? If so,
how? Are the practice’s hours more convenient for you?]

• Whether they are able to schedule
appointments promptly—on the same day if
needed—and experience brief waiting
times.

1

• Whether a care team member is available by
phone, email, or in person nights, weekends,
and on holidays, and has ready access to the
patient’s information.
• Whether the patient has had difficulties
accessing providers, either providers they
previously used or new providers they
would like to see.
• Whether the patient has had difficulties
access services, either service they
previously received or new services they
believe they need.

Question Text

Beneficiaries

9, 9abc, 11

What ways can you can now schedule an appointment?
[PROBES: online through a patient portal, leaving a message at
the clinic and someone calls you back, scheduling an
appointment before you leave the hospital.] What do you think
about these different ways of scheduling an appointment?
[PROBES: What do you like about it? What are some things
you don’t like about it?]
Has scheduling an appointment gotten better, worse, or about
the same over the past year or so? How has it gotten better or
worse?
How are wait times for your appointments? Have they gotten
better or worse?
In that past year, have you noticed any other changes in the way
your primary care practice is working now that makes it easier
or harder for you to get the care you need, when you need it?
(continued)

Table 1. continued
Commenter’s Feedback

Protocol

Question #

Ready Access: Focus group questions should
seek to learn from patients and their
caregivers:

How easy or hard is it for you or the person you care for to get
an appointment when he/she needs one? [PROBES: Can they
get a same day appointment for urgent needs? Can they get an
appointment for nights, or weekends, or holidays? Did the
practice change its hours of operation? If so, how? Are the
practice’s hours more convenient for the person you care for?]

• Whether they are able to schedule
appointments promptly—on the same day if
needed—and experience brief waiting
times.
• Whether a care team member is available by
phone, email, or in person nights, weekends,
and on holidays, and has ready access to the
patient’s information.

Caregivers

10, 10abc, 12

2

• Whether the patient has had difficulties
accessing providers, either providers they
previously used or new providers they
would like to see.

In what ways can appointments be scheduled with a provider?
[PROBES: online through a patient portal, leaving a message at
the clinic and someone calls back, scheduling an appointment
before leaving the hospital.]
Has scheduling an appointment gotten better, worse, or about
the same over the past year or so?
How are wait times for appointments? Have they gotten better
or worse?
In the past year, have you noticed any other changes in the way
the primary care practice is working now that makes it easier or
harder for you to help the person you care for to get the care
he/she needs, when he/she needs it?

• Whether the patient has had difficulties
access services, either service they
previously received or new services they
believe they need.
Ready Access: Focus group questions should
seek to learn from patients and their
caregivers whether access goes beyond office
visits or phone calls to include other forms of
access, such as e-visits or secure messaging
between patient, care team member, and
family caregiver as appropriate.

Question Text

Beneficiaries

10

Some practices have added a patient portal to their website
where patients can access lab or test results, contact their
providers electronically, or schedule appointments
electronically. Does your practice have a website that allows
you to do any of these things? [PROBES: Have you used this
website, online tool or patient portal?]
(continued)

Table 1. continued
Commenter’s Feedback
Ready Access: Focus group questions should
seek to learn from patients and their
caregivers whether the practice is
accommodating the needs of patients with
limited physical mobility, limited English
proficiency, low health literacy skills, cultural
differences, disabilities, or other issues that
could impede access to needed examination
and treatment and patient self-management.

3

Ready Access: Focus group questions should
seek to learn from patients and their
caregivers whether the practice is
accommodating the needs of patients with
limited physical mobility, limited English
proficiency, low health literacy skills, cultural
differences, disabilities, or other issues that
could impede access to needed examination
and treatment and patient self-management.

Protocol

Beneficiaries

Question #

6, 6a

Question Text
How well do providers at your primary care practice understand
your own unique views? Do they consider your cultural beliefs
and values when they talk to you about your health condition or
treatment options? [PROBE: Do they understand challenges
that you may have to making a weekly appointment or to
getting care at a facility across town, language barriers, or other
things that are important to you that may get in the way of your
health care?]
How could the staff at your primary care practice better
understand your values, your preferences for treatment, or just
understand your unique needs?

Caregivers

7, 7a

How well do providers understand the unique needs of the
person you care for? Do they consider his/her cultural beliefs
and values when they talk to you or the person you care for
about his/her health condition or treatment options? [PROBES:
Do they understand challenges that you or the person you care
for may have in making a weekly appointment or in getting
care at a facility across town, language barriers, or other things
that are important that may get in the way of the health of the
person you care for?]
How could the providers at the primary care practice better
understand his/her values, his/her preferences for treatment, or
just understand his/her unique needs?
(continued)

Table 1. continued
Commenter’s Feedback
Communication: Focus groups should assess
whether the doctor/care team talks with the
patient and family caregiver—in a way that is
easy to understand—about the patient’s health
condition, test results, treatment options, and
care instructions. Did the provider use medical
words the patient/family caregiver did not
understand?

Protocol

Beneficiaries

Question #

8

Focus Groups should assess whether the
doctor and care team listen carefully to the
patient and family caregiver.

4

Communication: Focus groups should assess
whether written instructions were provided to
follow up on the meetings.

Care coordination: Focus groups should
assess whether the patient/family caregiver
has a “go to” person on the care team to help
them navigate the system.

Care coordination: Focus groups should
assess whether the patient/family caregiver
has a “go to” person on the care team to help
them navigate the system.

Caregivers

3a

17
Beneficiaries

(NOTE:
Language will
be tailored to
specific state
context)
18

Caregivers

(NOTE:
Language will
be tailored to
specific state
context)

Question Text
In what ways does your primary care practice support patients
and their caregivers getting involved in their own care?
[PROBE: Does your provider talk to you about your condition
or treatment options in a way that is easy for you to
understand? Does your provider use medical words that are
easy for you to understand? Does your provider explain the
pros and cons of different treatment options? Does your
provider listen carefully to your concerns? Is your provider
willing to answer your questions?]

When the person you care for last saw his/her provider, did the
provider give you or the person you care for any instructions or
things to work on between visits? [If yes: Were written
instructions provided?] Have they always done this or is this
something that has changed in the past year or so?
Do you have a care coordinator, care manager, social worker,
or someone else who calls you every so often, or that you can
call when you have questions?

Does the person you care for have a care coordinator, care
manager, social worker, or someone else who calls you or
him/her every so often, or that you can call when you have
questions?

(continued)

Table 1. continued
Commenter’s Feedback
Care coordination: Focus groups should
assess whether the patient, and family
caregiver as appropriate, worked
collaboratively with the care team to develop
and update an individualized care plan, as
needed but at least annually.
Care coordination: Focus groups should
assess whether the patient, and family
caregiver as appropriate, worked
collaboratively with the care team to develop
and update an individualized care plan, as
needed but at least annually.

5

Care coordination: Focus groups should
assess whether the patient receives help
choosing specialists and obtaining
appointments.
Care coordination: Focus groups should
assess whether the patient receives help
choosing specialists and obtaining
appointments.
Care coordination: Focus groups should
assess whether multidisciplinary teams are
coordinating the care patients are getting from
all their health care providers—such as
physicians, hospitals, home care agencies,
nursing homes, and community resources.

Protocol

Beneficiaries

Caregivers

Beneficiaries

Caregivers

Question #

Question Text

5c

Has your primary care practice worked with you to develop a
care plan? What kind of information or instructions are in this
care plan? [PROBES: Does it include personal, patient-centered
health goals (e.g., “to live long enough to attend my son’s
wedding” or “to be able to walk to the mailbox without getting
out of breathe”)?]

6c

Has your primary care practice worked with you to develop a
care plan for the person you care for? What kind of information
or instructions are in this care plan? [PROBES: Does it include
personal, patient-centered health goals (e.g., “to live long
enough to attend my son’s wedding” or “to be able to walk to
the mailbox without getting out of breathe”)?]

14

15

How does your primary care practice play a role in getting you
to see a specialist? [PROBES: Do they make referrals? Do they
make the appointment for you?]
What role does their primary care practice play when he/she
needs to see a specialist? [PROBES: Do they make referrals?
Do they make the appointment for the person you care for or
ask you to do it?]
Questions ask about a variety of possible care team members
and their care coordination activities.

Beneficiaries

14–18

(continued)

Table 1. continued
Commenter’s Feedback
Care coordination: Focus groups should
assess whether multidisciplinary teams are
coordinating the care patients are getting from
all their health care providers—such as
physicians, hospitals, home care agencies,
nursing homes, and community resources.
Care coordination: Focus groups should
assess whether there is support for transitions
across care settings (including discharge from
hospital to home) with appropriate
involvement of the patient’s primary care
provider.

6

Care coordination: Focus groups should
assess whether there is support for transitions
across care settings (including discharge from
hospital to home) with appropriate
involvement of the patient’s primary care
provider.

Protocol

Question #

Question Text
Questions ask about a variety of possible care team members
and their care coordination activities.

Caregivers

Beneficiaries

15–19

16, 16a

When you go to your primary care practice for a medical visit,
does your provider know if you’ve visited the emergency room,
been hospitalized or had a nursing home or rehabilitation stay
since your last office visit?
Do you think your primary care practice knows about new
prescriptions or procedures that were done?

Caregivers

17, 17a

When the person you care for goes to his/her primary care
practice for a medical visit, does his/her provider know if
he/she has visited the emergency room, been hospitalized or
had a nursing home or rehabilitation stay since their last office
visit? Has this always been the case or have you noticed any
changes in the past year or so?
Do you think the primary care practice knows about new
prescriptions or procedures that were done? Has this always
been the case or have you noticed any changes in the past year
or so? How do you think they know? Has this always been the
case or have you noticed any changes in the past year or so?
(continued)

Table 1. continued
Commenter’s Feedback
Care coordination: Does the patient know
whether her/his information is being shared
among practices, specialists, and other
facilities (hospitals, EDs, labs, SNFs, home
health agencies, community support services).
For example, do the patient and his/her
providers have access to the patient’s
individualized care plan? Does the patient
know whether the primary care practice is
tracking the patient’s referrals and test results?
Is the patient receiving appropriate follow-up
care and help from the medical home in
understanding results and treatment
recommendations?

7

Care coordination: Does the patient know
whether her/his information is being shared
among practices, specialists, and other
facilities (hospitals, EDs, labs, SNFs, home
health agencies, community support services).
For example, do the patient and his/her
providers have access to the patient’s
individualized care plan? Does the patient
know whether the primary care practice is
tracking the patient’s referrals and test results?
Is the patient receiving appropriate follow-up
care and help from the medical home in
understanding results and treatment
recommendations?

Protocol

Question #

Question Text
You may need to get lab work done, get an x-ray, or other tests
during your office visit with a specialist. How do you usually
learn about the results of these tests? [PROBES: Who tells you
about the results? How do they contact you? How soon do you
usually find out?]

Beneficiaries

15

The person you care for may need to get lab work done, get an
x-ray, or other tests during an office visit. How do you or the
person you care for usually learn about the results of these
tests? [PROBES: Who tells you or him/her about the results?
How do they contact you or him/her? How soon do you or
he/she usually find out?]
Caregivers

16

(continued)

Table 1. continued
Commenter’s Feedback
Inclusion of family caregivers: Focus group
questions can help to elicit whether the
medical home offered appropriate support for
family caregivers as needed (e.g. assistance,
training, education, or connections to
community resources or services).
Connection to Community Resources: Focus
group questions should elicit from patients
and their family caregivers whether patients
are receiving the community-based resources
they want and need when they need them.

Protocol

Caregivers

Beneficiaries

Question Text

5

What do the providers at the primary care practice of the person
you care for do that helps you to take better care of the person
you care for? [PROBES: Reviews the medications that he/she
takes with you; gives you advice on nutrition/meal plans;
teaches you ways to provide care; writes instructions for you;
asks you about household hazards, such as scatter rugs that
someone could trip or slip on]

18

Care coordinators or social workers also may help you find
resources in the community to better manage your care. These
people could help you if you are experiencing some sadness or
challenges in your life, need help getting to the grocery store or
the pharmacy or need help with other basic needs. Has anyone
at your primary care practice told you about non-medical
services in your community that they thought you could benefit
from? [PROBES: Meals on Wheels, housing support, social
activities at the local seniors’ center, support groups?]

19

Care coordinators or social workers also may help you find
resources in the community to help you in your role as
caregiver. Has anyone at your primary care practice told you
about any services or support groups for caregivers? [PROBES:
Support groups that meet through the local hospital, local
senior center, or through a place of worship? On-line support
groups? Respite services that could relieve you of your daily
responsibilities on occasion? Adult daycare options?]

8

Question #

Connection to Community Resources: Focus
group questions should elicit from patients
and their family caregivers whether patients
are receiving the community-based resources
they want and need when they need them.

Caregivers

(continued)

Table 1. continued
Commenter’s Feedback

Protocol

Question #

Some practices have added a patient portal to their website
where patients can access lab or test results, contact their
providers electronically, or schedule appointments
electronically. Does your practice have a website that allows
you to do any of these things? [PROBES: Have you used this
website, online tool or patient portal?]

Health information technology: Focus groups
should seek to elicit
• Whether patients, and family caregivers as
appropriate, have online access and written
permission, via HIPAA authorization forms
recorded by the beneficiary, to access updated
patient medical records and act on behalf of that
beneficiary, should they not be able to advocate
on their own care

Question Text

Beneficiaries

10

If uses: How easy is it to use? What do you like or dislike
about it? What features do you use the most? What
improvements, if any, would you suggest?

9

• Whether patients and family caregivers believe
HIT is enhancing communication and
coordination. For example, if the patient emails a
question, does he/she get a response as soon as
needed? Was the provider’s use of a computer
helpful to the patient? Did the use of a computer
in the exam room interfere with the patient’s
ability to get the provider’s full attention?

If doesn’t use: Why not?

Health information technology: Was the family
caregiver consulted about whether expectations in
the care plan to be carried out by the family
caregiver were acceptable and did she/he get the
training to do what was expected of her/him?

What do the providers at the primary care practice of the
person you care for do that helps you to take better care of the
person you care for? [PROBES: Reviews the medications that
he/she takes with you; gives you advice on nutrition/meal
plans; teaches you ways to provide care; writes instructions for
you; asks you about household hazards, such as scatter rugs
that someone could trip or slip on.]

If the practice doesn’t have one or don’t know: Does this
sound like something that you would find useful? Why or why
not?

Caregivers

5, 6c

Has your primary care practice worked with you to develop a
care plan for the person you care for? What kind of
information or instructions are in this care plan? [PROBES:
Does it include personal, patient-centered health goals (e.g.,
“to live long enough to attend my son’s wedding” or “to be
able to walk to the mailbox without getting out of breathe”)?]
(continued)

Table 1. continued
Commenter’s Feedback

Protocol

Question #

Disparities: Focus groups should asses

How well do providers at your primary care practice understand
your own unique views? Do they consider your cultural beliefs
and values when they talk to you about your health condition or
treatment options? [PROBE: Do they understand challenges
that you may have to making a weekly appointment or to
getting care at a facility across town, language barriers, or other
things that are important to you that may get in the way of your
health care?]

• Whether patients and their family caregivers
believe the care provided by the practice is
tailored to their specific needs, beliefs, and
cultural values.
• Whether patients and family caregivers
believe they can communicate effectively
with their provider and receive information
that they understand. Are patients with
limited English proficiency, cultural
differences, and disabilities receiving
appropriate translation/interpreter services?

Question Text

Beneficiaries

How could the staff at your primary care practice better
understand your values, your preferences for treatment, or just
understand your unique needs?
6, 6a

10

• Whether notices and documents are provided
in alternative formats and effectively
communicated to patients with
communication, print, and mobility
disabilities.
• Whether patients believed they were treated
unfairly because of their race, ethnicity or
language.
• Whether patients believed they were treated
unfairly because of the type of insurance they
have, or because they don’t have insurance.
Patient and Family Caregiver Engagement:
Focus groups should seek to elicit whether the
patient, and family caregiver as appropriate, is
an active member of the care team, and
participates in developing and executing the
care plan.

Beneficiaries

5c

Has your primary care practice worked with you to develop a
care plan? What kind of information or instructions are in this
care plan? [PROBES: Does it include personal, patient-centered
health goals (e.g., “to live long enough to attend my son’s
wedding” or “to be able to walk to the mailbox without getting
out of breathe”)?]
(continued)

Table 1. continued
Commenter’s Feedback
Patient and Family Caregiver Engagement:
Focus groups should seek to elicit whether the
patient, and family caregiver as appropriate, is
an active member of the care team, and
participates in developing and executing the
care plan.

Protocol

Caregivers

Question #

3b

Question Text
In what ways did the provider involve you in the plan for the
person you care for? [PROBES: Did they ask you how you
could help with the plan? Did they ask you if you understood
the plan?] Have they always done this or is this something that
has changed in the past year or so?

11

How well do providers at your primary care practice understand
your own unique views? Do they consider your cultural beliefs
and values when they talk to you about your health condition or
treatment options? [PROBE: Do they understand challenges
that you may have to making a weekly appointment or to
getting care at a facility across town, language barriers, or other
things that are important to you that may get in the way of your
health care?]

Patient and Family Caregiver Engagement:
Focus groups should seek to elicit whether
patients, and family caregivers as appropriate,
are receiving the full range of information
about their treatment options and outcomes
(including benefits, costs, side effects, and/or
alternative options) that is needed to make
informed decisions about their care.
Beneficiaries

6,8

In what ways does your primary care practice support patients
and their caregivers getting involved in their own care?
[PROBE: Does your provider talk to you about your condition
or treatment options in a way that is easy for you to
understand? Does your provider use medical words that are
easy for you to understand? Does your provider explain the
pros and cons of different treatment options? Does your
provider listen carefully to your concerns? Is your provider
willing to answer your questions?]
(continued)

Table 1. continued
Commenter’s Feedback

Protocol

Question #

Question Text

7,8

How well do providers understand the unique needs of the
person you care for? Do they consider his/her cultural beliefs
and values when they talk to you or the person you care for
about his/her health condition or treatment options? [PROBES:
Do they understand challenges that you or the person you care
for may have in making a weekly appointment or in getting
care at a facility across town, language barriers, or other things
that are important that may get in the way of the health of the
person you care for?]

Patient and Family Caregiver Engagement:
Focus groups should seek to elicit whether
patients, and family caregivers as appropriate,
are receiving the full range of information
about their treatment options and outcomes
(including benefits, costs, side effects, and/or
alternative options) that is needed to make
informed decisions about their care.
Caregivers

12

How much of a role do you take in deciding how to best
manage the health or condition of the person you care for?
[PROBES: Do you ask the provider questions, share your views
about what you think is best for them? Does the provider talk to
you about the condition of the person you care for or his/her
treatment options in a way that is easy for you to understand?
Does the provider use medical words that are easy for you to
understand?]
Questions 4–8 address the care plan, care team support for selfmanagement, quality of life goals, etc.

Patient and Family Caregiver Engagement:
Focus groups should seek to elicit
• Whether patients are getting the support
they need to build the skills, knowledge, and
confidence to manage their health and
health care, maintain maximum function,
overcome barriers, and achieve personal
quality of life goals.
• Whether this support is tied to the care plan
they have developed collaboratively with
the care team and reflective of the patient’s
needs, wants, life situation, and goals.

Beneficiaries

4–8

Table 2. Summary of Revisions Made to Focus Group Protocols’ Existing Content in Response to Public Comments
Received Regarding CMS’s Planned Information Collection for the
Evaluation of the Multi-payer Advanced Primary Care Practice Demonstration
(Document Identifier: CMS-10479)
Protocol

Beneficiaries

Question #

2a

13
Caregivers

3a

Original Wording

Revised Wording

Reason for Revision

When the person you care for
last saw his/her provider, did the
provider give you or the person
you care for any instructions or
things to work on between
visits? Have they always done
this or is this something that has
changed in the past year or so?

When the person you care for last saw
his/her provider, did the provider give
you or the person you care for any
instructions or things to work on
between visits? [IF YES: Were written
instructions provided?] Have they
always done this or is this something
that has changed in the past year or so?

Comment: Whether written
instructions were provided to
follow up on the meetings. This
is crucial in a situation where
the family caregiver may not
have been at the appointment,
but needs to know how to
follow up, especially for
beneficiaries with dementia or
cognitive impairment.

When the person you care for
last saw his/her provider, did the
provider give you or the person
you care for any instructions or
things to work on between
visits? Have they always done
this or is this something that has
changed in the past year or so?

When the person you care for last saw
his/her provider, did the provider give
you or the person you care for any
instructions or things to work on
between visits? [IF YES: Were written
instructions provided?] Have they
always done this or is this something
that has changed in the past year or so?

Comment: Whether written
instructions were provided to
follow up on the meetings. This
is crucial in a situation where
the family caregiver may not
have been at the appointment,
but needs to know how to
follow up, especially for
beneficiaries with dementia or
cognitive impairment.
(continued)

Table 2. continued
Protocol

Caregivers

Question #

5

14
Beneficiaries

8

Original Wording

Revised Wording

Reason for Revision

What do the providers at the
primary care practice of the
person you care for do that helps
you to take better care of the
person you care for? [PROBES:
Reviews the medications that
he/she takes with you; gives you
advice on nutrition/meal plans;
asks you about household
hazards, such as scatter rugs that
someone could trip or slip on]

What do the providers at the primary
care practice of the person you care for
do that helps you to take better care of
the person you care for? [PROBES:
Reviews the medications that he/she
takes with you; gives you advice on
nutrition/meal plans; teaches you ways
to provide care; writes instructions for
you; asks you about household hazards,
such as scatter rugs that someone could
trip or slip on]

Comment: Focus group
questions can help to elicit
whether the medical home team
provided appropriate written,
visual, or other instructional
information on medication
management, wound care, postoperative procedures, or other
care the family caregiver will be
providing.

In what ways does your primary
care practice support patients
and their caregivers getting
involved in their own care?
[PROBE: Does your provider
talk to you about your condition
or treatment options in a way
that is easy for you to
understand? Does your provider
explain the pros and cons of
different treatment options? Is
your provider willing to answer
your questions?]

In what ways does your primary care
practice support patients and their
caregivers getting involved in their own
care? [PROBE: Does your provider talk
to you about your condition or treatment
options in a way that is easy for you to
understand? Does your provider use
medical words that are easy for you to
understand? Does your provider explain
the pros and cons of different treatment
options? Does your provider listen
carefully to your concerns? Is your
provider willing to answer your
questions?]

Comment: Did the provider use
medical words the
patient/family caregiver did not
understand? Whether the doctor
and care team listen carefully to
the patient and family caregiver.

(Edited probes to more
specifically ask about types of
assistance to the caregiver)

(Added probes about medical
language and whether care team
listens to concerns to more
specifically ask about patient
experience with their provider’s
communication.)
(continued)

Table 2. continued
Protocol

Caregivers

Question #

8

15
Beneficiaries

9

Original Wording

Revised Wording

Reason for Revision

There are many ways that
caregivers and providers can work
together to manage the patient’s
health or medical condition. For
example, some caregivers rely
completely on the provider to know
what is best for the person they care
for, while others take a more active
role in the decisions that affect the
person they care for. How much of
a role do you take in deciding how
to best manage the health or
condition of the person you care
for? [PROBES: Do you ask the
provider questions, share your
views about what you think is best
for them?]

There are many ways that caregivers and
providers can work together to manage
the patient’s health or medical condition.
For example, some caregivers rely
completely on the provider to know what
is best for the person they care for, while
others take a more active role in the
decisions that affect the person they care
for. How much of a role do you take in
deciding how to best manage the health
or condition of the person you care for?
[PROBES: Do you ask the provider
questions, share your views about what
you think is best for them? Does the
provider talk to you about the condition
of the person you care for or his/her
treatment options in a way that is easy
for you to understand? Does the provider
use medical words that are easy for you
to understand?]

Comment: Focus groups
should assess whether the
doctor/care team talks with
the patient and family
caregiver—in a way that is
easy to understand—about
the patient’s health condition,
test results, treatment
options, and care
instructions. Did the provider
use medical words the
patient/family caregiver did
not understand?

How easy or hard is it for you to get
an appointment with a provider at
this practice when you need one?
[PROBES: Can you schedule a
same day appointment for urgent
needs? Can you schedule an
appointment for nights or
weekends? Did the practice change
its hours of operation? If so, how?
Are the practice’s hours more
convenient for you?]

How easy or hard is it for you to get an
appointment with a provider at this
practice when you need one? [PROBES:
Can you schedule a same day
appointment for urgent needs? Can you
schedule an appointment for nights, or
weekends, or holidays? Did the practice
change its hours of operation? If so,
how? Are the practice’s hours more
convenient for you?]

Edited language to ask about
after-hours access to care on
holidays.

(Added more specific probes
about caregiver experiences
with provider
communication.)

(continued)

Table 2. continued
Protocol

Caregivers

Question #

Original Wording

Revised Wording

10

How easy or hard is it for you or
the person you care for to get an
appointment when he/she needs
one? [PROBES: Can they get a
same day appointment for
urgent needs? Can they get an
appointment for nights or
weekends? Did the practice
change its hours of operation? If
so, how? Are the practice’s
hours more convenient for the
person you care for?]

How easy or hard is it for you or the
person you care for to get an
appointment when he/she needs one?
[PROBES: Can they get a same day
appointment for urgent needs? Can they
get an appointment for nights, or
weekends, or holidays? Did the practice
change its hours of operation? If so,
how? Are the practice’s hours more
convenient for the person you care for?]

Reason for Revision
Edited language to ask about
after-hours access to care on
holidays.

16

Table 3. Summary of New Questions Added to Existing Focus Group Protocols in Response to Public Comments
Received Regarding CMS’s Planned Information Collection for the Evaluation of the Multi-payer
Advanced Primary Care Practice Demonstration
(Document Identifier: CMS-10479)
Protocol

Question #

New Question Added
Did the primary care practice of the person you care
for ask about your role in helping that person? Did
they ask about any circumstances that could affect the
way you care for him/her, such as how close you live
to the person you care for?

Caregivers

1b

Comment: Focus group questions can help to elicit
• Whether the medical home asked about the role of
the family caregiver in helping the patient, and what
kind of support the caregiver provides (e.g., help
with activities like household chores, medical tasks,
bills, bathing, dressing, or preparing food).

17

• Whether the medical home knows of a family
caregiver’s circumstances, such as whether the
caregiver is local or providing care from a distance,
whether the caregiver has health conditions of
his/her own, and other factors that may impact the
caregiver’s ability to provide care.
How are wait times for your appointments? Have they
gotten better or worse?

Beneficiaries

Reason for Addition

9c

Comment: Focus groups should assess whether
patients and their caregivers are able to schedule
appointments promptly—on the same day if needed—
and experience brief waiting times.
(Added to ask specifically about patient experiences
with wait times)

How are wait times for appointments? Have they
gotten better or worse?
Caregivers

10c

Comment: Focus groups should assess whether
patients and their caregivers are able to schedule
appointments promptly—on the same day if needed—
and experience brief waiting times.
(Added to ask specifically about caregiver experiences
with wait times)
(continued)

Table 3. continued
Protocol

Caregivers

Question #

11a,b,c

New Question Added

Reason for Addition

Some practices have added a patient portal to their
website where patients and caregivers can access lab
or test results, contact providers electronically, or
schedule appointments electronically. Does the
practice of the person you care for have a website that
allows you to do any of these things? [PROBES: Have
you used this website, online tool or patient portal?]

Added to ask about caregiver experiences with new
forms of health IT that aim to increase care access and
enhance coordination and communication.

If uses: How easy is it to use? What do you like or
dislike about it? What features do you use the most?
What improvements, if any, would you suggest?
If doesn’t use: Why not?

18

If the practice doesn’t have one or don’t know: Does
this sound like something that you would find useful?
Why or why not?