SupportingStatement-A_852013

Supporting Statement for
Evaluation of the Multi-Payer Advanced Primary Care Practice (MAPCP) Demonstration:
Focus Group Interview Guides

1

A.

BACKGROUND

On September 16, 2009, Secretary of Health and Human Services, Kathleen Sebelius, and the
Director of the Office of Health Reform, Nancy-Ann DeParle, announced the establishment of
the Multi-payer Advanced Primary Care Practice (MAPCP) Demonstration, under which
Medicare joined Medicaid and private insurers as a payer participant in state-sponsored
initiatives to promote the principles that characterize advanced primary care, often referred to as
the “patient-centered medical home” (PCMH). CMS selected eight states to participate in this
demonstration: Maine, Vermont, Rhode Island, New York, Pennsylvania, North Carolina,
Michigan, and Minnesota. These states vary on a number of important dimensions, such as
features of their public (Medicaid) and private insurance markets, delivery system, prior
experience with medical home initiatives, and nature of their state-sponsored multi-payer
initiative.
CMS is conducting an evaluation of the demonstration to assess the effects of advanced primary
care practice when supported by Medicare, Medicaid, and private health plans. As part of this
evaluation, qualitative and quantitative data will be collected and analyzed to answer research
questions focused on: 1) state initiative features and implementation, including various payment
models; 2) practice characteristics, particularly medical home transformation; and 3) outcomes,
including access to and coordination of care, clinical quality of care and patient safety,
beneficiary experience with care, patterns of utilization, Medicare and Medicaid expenditures,
and budget neutrality.
This OMB application seeks approval to conduct in-person focus groups with beneficiaries and
their caregivers to inform CMS’ evaluation of the MAPCP Demonstration. Six focus groups will
be conducted in each of the eight participating states, spread across Medicare and Medicaid
beneficiaries and caregivers of beneficiaries. Special populations in each participating state (e.g.,
persons with chronic illnesses, mental illnesses, and disabilities; Medicare and Medicaid dual
eligibles; and people living in rural or inner-city areas) will be represented.
These focus groups will be conducted by The Henne Group, a subcontractor of CMS’ evaluation
contractor, RTI International. Each group will be observed by a research team member from RTI
International or from one of their subcontractors, the Urban Institute or the National Academy
for State Health Policy. Focus groups will be conducted twice—at the mid-point and at the end
of the demonstration.
A.1

Need and Legal Basis

The focus groups are part of a mixed-methods evaluation strategy for studying the process of,
and barriers and facilitators to, transforming practices into PCMHs and for assessing the effects
of the PCMH model on access, quality, and cost of care. Mixed-methods research is well-suited
for accomplishing the goals of this evaluation, as different methods yield different insights.
While quantitative methods (e.g., Medicare claims data analysis) are well-suited for outcomes or
summative evaluation, qualitative methods (e.g., interviews, focus groups) are necessary for
process or formative evaluation (Patton, 1990 and 1996; Sofaer, 1999). The combination of these
methods can provide a comprehensive understanding of the nature of each state PCMH initiative,
their implementation, the process and degree of practice transformation, and perceived outcomes
for patients, practices, and purchasers (Creswell, 2009). Qualitative methods are particularly

2

useful for evaluating health policy interventions, providing a more complete understanding of the
interventions themselves and the context in which they are taking place, the views of different
stakeholders, the unexpected outcomes, and the state and program conditions or factors more
likely to be associated with success (Ragin, 1999; Rist, 1994; Sofaer, 1999; Yin, 1999).
Because the demonstration is patient-centered, it is critical to understand patients’ experiences
from their perspective (or from their caregiver’s perspective) and how well this model is serving
their needs. For this evaluation, the focus groups will provide us with answers to fundamental
“what, how, and why” questions about beneficiaries’ experiences with care and access to and
coordination of care, such as:

• What are the beneficiaries’ experiences with care under the state initiative?
• Do features of the state initiatives result in more timely delivery of health services? If so,
what features facilitate more timely health care delivery and what outcomes result from
these improvements?

• Are beneficiaries, their family members, and/or their caregivers able to participate more
effectively in decisions concerning their care as a result of the state initiative? How does
the state initiative facilitate this and what impacts are seen as a result of this more
effective participation?

• Are beneficiaries better able to self-manage their health conditions or more likely to
engage in healthy behaviors as a result of the state initiative? How does the state initiative
facilitate this and what impacts are seen as a result?

• How easy or difficult is it for beneficiaries to schedule appointments with their
physician?

• Do beneficiaries think that their physician’s care has improved over the past year? In
what ways?

• How are beneficiaries referred to specialists? Who provides the referral?
• How well do beneficiaries’ physicians know about care received from specialists?
The information collected through these focus groups is critical to CMS in determining whether
the MAPCP Demonstration is effective.
A.2

Information Users

These focus groups will be used by CMS to understand:

• the experiences beneficiaries have with their care under the state initiatives, including
quality of care and their participation in decision-making about their care,

3

• beneficiaries’ perspectives about how their access to care has changed under the state
initiatives,

• beneficiaries’ experiences with coordination of care, including referral management, and
• experiences of special populations of beneficiaries (e.g., persons with chronic illnesses,
mental illnesses, and disabilities; Medicare and Medicaid dual eligibles; and people living
in rural or inner-city areas) with the state initiatives.
The results will also be used by policymakers, payers, healthcare purchasers, primary care
practices, and Medicaid and Medicare beneficiaries in the following ways:

• Local and state governments will have information on how their Medicare and Medicaid
beneficiaries’ experiences with care have changed under their state initiatives, as well as
suggestions about potential areas for program improvement.

• Payers and healthcare purchasers will have information to help them to know whether
their payment models and program activities (e.g., learning collaboratives, practice
coaches) are effective or whether modifications are warranted.

• Primary care practices will have data to inform them about what other practice changes
may be beneficial to enhance the quality and safety of care, efficiency of care delivery,
access to care, and other outcomes.

• Patients will directly benefit from any improvements implemented by policymakers,
payers, purchasers, and their primary care practices.
This information also will facilitate diffusion and implementation of similar initiatives in other
states, if this demonstration is found to be effective.
A.3

Use of Information Technology

The focus groups will make minimal use of information technology (IT). Skilled and
experienced focus group facilitators from RTI International’s subcontractor, The Henne Group,
will lead each discussion group and a dedicated note taker will capture participant responses.
Each group will be audio and video recorded to be used as a back-up to assure the completeness
and accuracy of the notes and to provide members of the research team the opportunity to
observe the groups at a later date. Data will be managed and analyzed in NVivo, a powerful and
widely used qualitative data analysis software program (QSR International, Doncaster, Australia;
Bazeley, 2007; Richards, 2009; Sorensen, 2008). The research team has significant experience in
managing and analyzing large primary qualitative data sets with this type of software.
Enrollment and participation in the focus group will require minimal use of information
technology. Telephonic communication will be used during recruitment, but participants will not
be expected to use IT during their focus group participation.

4

A.4

Efforts to Identify Duplication

The evaluation has been designed to comprehensively address the research questions while
minimizing the burden placed on the states, their partners (e.g., state evaluators), demonstration
participants (e.g., practices and community health teams), and Medicare and Medicaid
beneficiaries and special populations.
Focus groups are designed to complement other primary and secondary data collection and
analysis (see section A-1 for more details). That is, they will build on and fill information gaps
rather than duplicate information from other sources of data. For example, information from
focus groups can be used to help explain results from the survey of beneficiaries in each state
(e.g., the CAHPS survey) and can provide deeper understanding of beneficiaries’ experience that
cannot be understood through quantitative data alone. Focus groups will be conducted only when
primary or secondary data from states or their evaluators cannot be obtained to fully answer the
evaluation research questions.
CMS and its evaluation contractor and subcontractors have taken numerous steps to ensure that
the information to be collected through these focus groups are not readily available from existing
sources. We have examined secondary qualitative documents and resources publicly available
and have reviewed the states’ MAPCP applications and other documentation and
communications provided to CMS. In addition, we are seeking to collaborate with the states on
future data collections. Furthermore, since programs vary by state we will be tailoring each
state’s focus group guides to best understand the experiences that beneficiaries have with the
programs in their state and to minimize the collection of data. Focus group guides will also be
tailored for special populations to better understand how the initiatives have impacted their
experiences with care. By tailor, we mean either deleting questions that are not relevant given a
particular states initiative, making slight modifications to the questions to reflect specific or
unique elements of the state’s initiative (e.g., name of the effort, when it began, provider
payment method), or adding specific questions to reflect the characteristics of a special
population. For example, because childhood asthma is one of the most common chronic diseases
among children, affecting over ten million U.S. children in 2010 (Bloom, et al. 2011), some
states may target Medicaid children with asthma as a special population to focus on in their
initiative. Focus group guides for this special population may hone in on beneficiary’s caregivers
perspectives regarding how accessible, continuous, comprehensive, family-centered,
coordinated, compassionate and culturally effective their child’s care has been throughout the
initiative compared to their care prior to the initiative.
Thus, the information collected through the focus groups should not duplicate any other effort
and should not be obtainable from any other source.
A.5

Involvement of Small Entities

Focus group participants will be limited to Medicare and Medicaid beneficiaries and beneficiary
caregivers. We plan to recruit focus group participants that are enrolled in Medicaid with some
assistance from primary care provider offices that may represent small businesses or other small
entities. These offices will not be asked to respond to any data collection instruments. Rather,
these offices will simply assist in mailing recruitment letters to 50–100 Medicaid enrollees in

5

their patient panel. To accommodate staff time and any resources used by the practice, RTI will
provide a $500 gift card.
A.6

Less Frequent Collection

The focus groups will be conducted twice—at the middle and end of the 3-year demonstration.
This frequency allows for the collection of information and feedback at critical points in the
demonstration that are necessary for addressing the evaluation research questions.
A strength of the qualitative data collection plan is its timeliness for obtaining relatively early
insights about beneficiaries’ experiences with each state’s initiative, which can be used to make
improvements to the MAPCP Demonstration and, in turn, increase the likelihood of program
success.
A.7

Special Circumstances

There will be no special circumstances.
A.8

Federal Register/Consultation Outside the Agency

The 60-day Federal Register Notice was published on April 29, 2013.
A.9

Payments/Gifts to Respondents

Each focus group participant will be provided with a $50 gift card for their participation in the
focus group. To further facilitate participation, focus groups will be held at locations convenient
to beneficiaries (e.g., near hospitals or on a bus line) as well as during convenient times such as
during lunchtime or in the evenings. Beverages and some food will be offered at these focus
groups. RTI’s past experience conducting focus groups indicated that individuals are more
willing to attend a group discussion if a light meal is provided and the discussion is held at
convenient locations and times.
Additionally, primary care provider offices who are assisting with recruitment of Medicaid
enrollees, will be provided a $500 gift card to accommodate staff time and any resources used by
the practice when mailing recruitment letters. These offices will not be asked to respond to any
data collection instruments or participate in the focus groups.
A.10

Confidentiality

Personnel to be given access to focus group data (including notes, summary reports, transcripts,
audio, and video tapes) and/or individual identifiers will be trained on the significance and
protection of confidentiality, particularly as it relates to controlled and protected access to these
data. Further, information will be provided to potential focus group participants describing the
purpose and the voluntary nature of the focus groups and will convey the extent to which
respondents and their responses will be kept confidential. We pledge privacy to the fullest extent
possible. We will use a file-naming convention denoting the state and type of beneficiary or
caregiver group; the full names of individual participants will not be included in the focus group
notes or transcripts. As previously described on page 4, NVivo 9 is a computer software package
used to analyze qualitative data. The notes and the database will be stored on a secured server
and password-protected computers.

6

A.11

Sensitive Questions

Information collected in the focus groups is not of a sensitive nature. Questions are confined to
participant’s experiences, opinions, and perspectives regarding their care received under the
MAPCP Demonstration. Some participants may choose to share information about their health or
medical condition to illustrate how it shaped their experiences with their providers. We will ask
participants to not share any personal information about other participants outside of the room.
Some focus group participants might have views that are critical of state or federal initiatives or
particular participating organizations (e.g., health plans, health systems or practice, community
organizations). We will handle such insights with sensitivity and confidentiality in mind and will
not share nor attribute the identities of those individuals in an identifiable way in any written or
oral communications.
A.12

Burden Estimates (Hours and Wages)

Six focus groups with Medicare and Medicaid beneficiaries and caregivers of beneficiaries will
be conducted in two rounds, in each state. The length of each focus group will be no more than 2
hours, including time to review the focus group processes and to obtain verbal informed consent.
Medicare and dually eligible Medicare and Medicaid beneficiaries will be selected using
Medicare claims data. For Medicaid beneficiaries, we will work with a select group of primary
care practices to generate a list of 50–100 patients who are enrolled in Medicaid. Invitations for
all beneficiaries (Medicare, dually eligible Medicare and Medicaid, and Medicaid) will be sent
by mail to individuals meeting study selection criteria such as number of visits to the primary
care practice and emergency room over the past year, presence of chronic conditions, and
location of residence. Individuals interested in participating will be given a telephone number to
call. During the call, additional screening criteria will be applied to select the final list of
participants to achieve the desired composition of each focus group and to finalize the date and
time. Staff conducting the screenings will be given tracking tables to ensure a mix of participants
with regard to gender, education level, and ethnicity.
To estimate the cost of burden, we used an average of eight participants per focus group. Wage
calculations are based on the mean hourly wages as indicated in the “National Compensation
Survey: Occupational Wages in the United States, May 2011,” by the U.S. Department of Labor,
Bureau of Labor Statistics. The Bureau of Labor Statistics reported the mean hourly wage for
civilian workers in the United States was $21.74 in May 2011.
The maximum number of participants by state that will participate in focus groups during each
round is shown in Exhibit 1. A total of 384 individuals will participate in a focus group in each
round. We will conduct two rounds of focus groups during the course of the demonstration,
meaning that 384 individuals will participate in focus groups in the eight states at two points in
time (approximately twelve months apart), for a total of 768 participants (384 x 2). To minimize
burden on individuals, steps will be taken to ensure that a beneficiary is not invited to participate
in both rounds.

7

Exhibit 1.

Maximum Number of Participants by State per Round
Number of
Focus Groups
per Round

State

Number of Beneficiaries
or Caregivers per Focus
Group, per Round

Total
Participants
in Round 1

Total
Participants
in Round 2

Total
Participants

ME

6

8

48

48

96

MI

6

8

48

48

96

MN

6

8

48

48

96

NY

6

8

48

48

96

NC

6

8

48

48

96

PA

6

8

48

48

96

RI

6

8

48

48

96

VT

6

8

48

48

96

48

64

384

384

768

Total

There will be no more than six focus groups conducted per state in each round of focus groups.
An example of the number and composition of these groups, shown for each participant type and
stratified by the representative of the beneficiary (either the beneficiary or their caregiver), per
state per round, is shown in Exhibit 2. The actual composition of groups in each state may vary,
depending on factors such as whether the state focuses on a special population and the size of the
pediatric Medicaid population.
Exhibit 2.

Participant Type and Numbers per State per Round
Number of
Beneficiaries

Number of
Caregivers

Total Participants per
State per Round

Medicare beneficiaries

8

8

12

Medicaid beneficiaries

8

8

12

Dual eligible beneficiaries

8

0

8

Special population

8

0

16

32

16

48

Participant Type

Total

Estimated annual time and wage burden during each round is shown in Exhibit 3. The total
estimated time burden for each round is 1152 hours, which includes 2 hours for the focus groups
and 1 hour of travel time to and from the focus group site. The total estimated time burden for
two rounds is 2304 hours (1152 x 2). The total estimated wage burden for each round of focus
groups is $25,044.48. The total estimated wage burden for the entire evaluation (2 rounds
combined) is $50,088.96.

8

Exhibit 3.

Estimated Respondent Annual Time and Wage Burden by Participant Type
(1,152 burden hours for an estimated $25,044 wage burden) per Round

Number of
Participants

Length of
Focus
Group
(hours)

Travel Time
to/from Focus
Group (hours)

Total
Burden
Hours

Mean
Hourly
Wage
Rate*

Total
Wage
Burden

Medicare
beneficiaries

64

2.0

1.0

3.0

$21.74

$4,174.08

Medicaid
beneficiaries

64

2.0

1.0

3.0

$21.74

$4,174.08

128

2.0

1.0

3.0

$21.74

$8,348.16

Dual eligible
beneficiaries

64

2.0

1.0

3.0

$21.74

$4,174.08

Special population

64

2.0

1.0

3.0

$21.74

$4,174.08

Participant Type

Caregivers

Total

384

1,152

$25,044.48

*Based upon the mean hourly wages, “National Compensation Survey: Occupational Wages in the United States,
May 2012,” U.S. Department of Labor, Bureau of Labor Statistics.
http://www.bls.gov/oes/current/oes_nat.htm#b29-0000

A.13

Capital Costs

There are neither capital or startup costs, nor are there any operation and maintenance costs to
the focus group participants.
A.14

Costs to Federal Government

Total costs associated with two rounds of focus groups are estimated to be $719,354 for
recruitment, focus group facilitation, participant incentives, travel, meeting notes and analysis.
The annualized costs are approximately $359,677 for each round of focus groups; the two rounds
will occur over a three-year period. These costs are funded through an existing CMS contract
with RTI.
Federal FTE costs are expected to be negligible. The Project Officer for the CMS contract with
RTI may be required to spend 0.2% of her time on the administration of this survey (~$250 of
annual salary).
A.15

Changes to Burden

This is a new data collection for CMS. The focus groups will not result in any recurrent periodic
reporting or recordkeeping costs or time burden.
A.16

Publication/Tabulation Dates

These qualitative results will be reported in the second and third annual reports and the final
report. The Second Annual Report will be completed in March 2014 and the Third Annual
Report will be completed in March 2015. The Final Report will be completed in January 2016.

9

Additionally, the RTI/Urban/NASHP team plan to develop peer-reviewed publications and
conference presentations that will be reviewed and approved by CMS prior to submission.
A.17

Expiration Date

The OMB expiration date will be displayed on all disseminated data collection materials.

10

SUPPORTING STATEMENT—PART B
COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
This information collection does not employ statistical methods.

11

REFERENCES
Bazeley, P. (2007). Qualitative data analysis with NVivo (2nd ed.). London: Sage Publications
Ltd.
Berenson, R. A., Devers, K. J., & Burton, R. A. (2011). Will the patient centered medical home
transform the delivery of health care? Robert Wood Johnson Foundation and Urban
Institute Issue Paper, August. Retrieved from
http://www.urban.org/publications/412373.html.
Bitton, A., Martin, C., & Landon, B. E. (2010). A nationwide survey of patient centered medical
home demonstration projects. Journal of General Internal Medicine, 25(6), 584–592.
Bloom, B., Cohen, R. A., & Freeman, G. (2011). Summary health statistics for U.S. children:
National Health Interview Survey, 2010. National Center for Health Statistics. Vital
Health Statistics, 10(250), 1–80.
Bradley, E. H., Curry, L. A., & Devers, K. J. (2007). Qualitative data analysis for health services
research: Developing taxonomy, themes, and theory. Health Services Research, 42(4),
1758–1772.
Crabtree, B. F., Nutting, P. A., et al. (2010). Summary of the National Demonstration Project and
recommendations for the patient-centered medical home. Annals of Family Medicine,
8(Suppl. 1), S80–S90.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods
approaches (3rd ed.). Thousand Oaks, CA: Sage.
Devers, K. J. (1999). How will we know good qualitative research when we see it? Health
Services Research, 34(5, part 2), 1153–1188.
Miles, M. A., & Huberman, A. M. (1994). Qualitative data analysis: An expanded source book
(2nd ed.). Thousand Oaks, CA: Sage.
Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Thousand Oaks,
CA: Sage.
Patton, M. Q. (1996). Utilization-focused evaluation (3rd ed.). Thousand Oaks, CA: Sage.
Ragin, C. C. (1999). Using qualitative comparative analysis to study causal complexity. Health
Services Research, 34(5 Pt 2), 1225–1239.
Richards, L. (2009). Handling qualitative data (2nd ed.). Thousand Oaks, CA: Sage.
Rist, Ray C. (1994). Influencing the policy process with qualitative research. In N. Denzin & Y.
Lincoln (Eds.), Handbook of qualitative research (pp. 545–557). Thousand Oaks, CA:
Sage Publications, Inc.

R-1

Sofaer, S. (1999). Qualitative methods: What are they and why use them? Health Services
Research, 34(5, part 2), 1101–1118.
Sorensen, A. (2008). Use of QSR NVivo 7 qualitative analysis software for mixed methods
research. Journal of Mixed Methods Research, 2(1), 106–110.
Steiner, B. D., Denham, A. C., et al. (2008). Community care of North Carolina: Improving care
through community health networks. Annals of Family Medicine, 6(4), 361–367.
Yin, R. K. (1999). Enhancing the quality of case studies in health services research. Health
Services Research, 34(5 Pt 2), 1209–1224.

R-2

ATTACHMENT A
60-DAY FEDERAL REGISTER NOTICE
(To be added after issue by CMS)

A-1

ATTACHMENT B
30-DAY FEDERAL REGISTER NOTICE
(To be added after issue by CMS)

B-1

ATTACHMENT C
SCREENER AND SCRIPT FOR RECRUITMENT

C-1

ATTACHMENT D
FOCUS GROUP PROTOCOLS

D-1