Supporting Statement A_102714 clean

Supporting Statement

Part A. Justification

1. Circumstances Making the Collection of Information Necessary

On September 16, 2009, the Secretary of Health and Human Services, Kathleen Sebelius, and the Director of the White House Office of Health Reform, Nancy-Ann DeParle, announced the establishment of the Multi-payer Advanced Primary Care Practice (MAPCP) Demonstration, under which Medicare joined Medicaid and private insurers as a payer participant in state-sponsored initiatives to promote the principles that characterize advanced primary care, often referred to as the “patient-centered medical home” (PCMH) model of care.CMS selected eight states to participate in this demonstration: Maine, Vermont, Rhode Island, New York, Pennsylvania, North Carolina, Michigan, and Minnesota. These states vary on a number of important dimensions, such as features of their public (Medicaid) and private insurance markets, delivery system, prior experience with medical home initiatives, and nature of their state-sponsored multi-payer initiative. Medicare participation in three of the states’ initiatives (Vermont, New York, and Rhode Island) started July 1, 2011. Two additional states (North Carolina and Minnesota) were effective October 1, 2011, and the three remaining states (Maine, Michigan, and Pennsylvania) became operational January 1, 2012. CMS is currently committed to participating in these state initiatives for three years. Approximately 1,200 medical homes serving over 900,000 Medicare beneficiaries are expected to be participating by the end of the demonstration.

The medical home care delivery model is a potentially transformative health system innovation, combining changes in provider payment and primary care structure and care processes. Evaluations of medical home models have shown mixed results to date, with some studies showing positive effects and others not showing statistically significant effects. Many findings to date have been preliminary, have had limited generalizability, and have had some limitations in their study design (e.g., no comparison group). Although some positive outcomes from the medical home model have been shown to be significant, critical questions remain unanswered. For example, the impacts of medical home provider payment models and medical home practice transformation on health outcomes and the U.S. health care system, particularly from a cost perspective, are largely unknown (Berenson et al., 2011; Crabtree et al., 2010; Steiner et al., 2008; Bitton et al., 2010).

CMS is conducting an evaluation of this demonstration to assess the effects of advanced primary care practice when supported by Medicare, Medicaid, and private health plans. As part of this evaluation, qualitative and quantitative data will be collected and analyzed to answer research questions focused on: 1) the features of the state initiative, including various payment models; 2) the features of participating practices, particularly understanding what new care processes or activities providers adopt as a result of participating in this demonstration; and 3) the outcomes produced by this demonstration, including improving access to and coordination of care, clinical quality of care, patient safety, beneficiary experience with care, changing patterns of utilization, reducing Medicare and Medicaid expenditures, and achieving budget neutrality.

This information collection request seeks approval to field a pair of surveys to inform CMS’s evaluation of the MAPCP Demonstration. These two surveys will be targeted to staff of medical practices: a medical home survey will be fielded among health care providers (e.g., medical doctors, doctors of osteopathy, nurse practitioners, and physician’s assistants) who are participating in the MAPCP Demonstration; a companion practice characteristics survey will be fielded among practice managers or administrators (e.g., non-clinical administrative staff). Staff from the full universe of practices participating in the MAPCP Demonstration will be asked to complete this survey, and will be contacted using contact information already provided to the federal evaluators of this demonstration by CMS. Both surveys will be fielded electronically, via emails inviting them to complete our web-based surveys. CMS’s evaluation contractor, RTI International, and its subcontractors, the Urban Institute and the National Academy for State Health Policy (NASHP), are responsible for the development, administration, and analysis of these surveys. Before the surveys are administered, they will be approved by RTI International and the Urban Institute’s Institutional Review Boards (IRBs). (A pilot test of this survey conducted in the Fall of 2013 was approved by these organizations’ IRBs, but not OMB, since only 8 respondents were asked to complete the survey.)

These provider surveys will allow the Administration to identify which aspects of the patient-centered medical home have the largest impact on the quality and efficiency of the care they deliver. The Administration could in turn use these findings to craft policies that incentivize the adoption of the high-value components of the PCMH model on a more widespread basis, across the country.

The MAPCP Demonstration is being conducted under the authority of §402 of the Social Security Amendments of 1967 (as amended). Section 402 authorizes the Secretary of HHS to conduct demonstration projects to evaluate changes in methods of payment for covered services and payment for services not otherwise covered and which are incidental to services for which payment may be made.

CMS takes responsibility for the quality of information that it disseminates and has instituted appropriate content/subject matter, statistical, and methodological review procedures to comply with OMB’s and its own information quality guidelines.

2. Purpose and Use of the Information Collection

These surveys are part of a mixed-methods evaluation strategy for studying the process of transforming practices into medical homes and for assessing the effects of this model on access, quality, and cost of care. Other primary data are being collected as part of this study using site visit interviews (OMB Control #0938-1211), focus groups (OMB Control #0938-1224), and a patient experience of care survey (OMB Control #0938-1223). Secondary data are also being collected – in the form of Medicare and Medicaid claims data, and in some cases reports and survey data previously collected by participating states. These data are allowing researchers to conduct a mixed-methods evaluation, to allow CMS to maximize the lessons it can learn from this demonstration.

Mixed-methods research is well-suited for accomplishing the goals of this evaluation, as different methods yield different insights. While quantitative methods (e.g., statistical analysis of claims data) are well suited for studying outcomes or summative evaluation, qualitative methods (e.g., interviews, focus groups) are necessary for process or formative evaluation (Patton, 1990 and 1996; Sofaer, 1999). The combination of these methods can provide a comprehensive understanding of the each state’s MAPCP Demonstration initiative, the process and degree of transformation to the medical home model that occurred among participating practices, perceived outcomes among patients, providers, and purchasers, and actual quality and cost outcomes produced (Creswell, 2009).

As part of this evaluation, CMS proposes to survey practices participating in the demonstration who are caring for patients insured through Medicaid and fee-for-service Medicare near the end of the demonstration. These surveys will allow evaluators to understand how participating practices’ structures and functions vary, particularly with respect to their adoption of different components of the medical home model of care. Researchers evaluating the MAPCP Demonstration plan to link these survey data with provider-identifiable claims data to run sophisticated statistical analyses that identify which particular medical home care processes are associated with the largest gains in health care quality measures and reductions in health care utilization and/or spending. The data collected using these surveys will also be analyzed to document variation in the degree of MAPCP Demonstration practices’ “medical homeness” during the demonstration, by comparing responses across states and types of practices. The data collected using these surveys may also be compared to practices’ and/or providers’ responses on medical home recognition surveys completed to enter the state initiatives (specifically, NCQA’s PPC-PCMH (2008) and PCMH (2011) practice recognition surveys, Blue Cross Blue Shield of Michigan’s PCMH designation program, and Minnesota’s Health Care Home standards¹), to the extent such data is available for a large enough sample of practices, to allow researchers to identify which medical home care processes practices already had in place and which they adopted once enhanced reimbursement was offered through this demonstration.

The information collected through these surveys is critical to CMS in evaluating the effect of the MAPCP Demonstration on health care quality and cost trends. Specifically, this survey data will be used to answer the research questions:

• What are the features of practices participating in the MAPCP Demonstration?

• All questions in the two companion surveys will be used to answer this research question. We will use the survey data to present basic descriptive statistics about the makeup of the practices and providers participating in the demonstration (e.g., the prevalence of care coordinators in practices, whether participating practices tend to have adopted EHRs many years ago). We will also use the survey data to present descriptive statistics identifying the components of the PCMH model of care that practices adopted most frequently (e.g., actively coordinating care with other providers, using patient registries to identify patients to remind to come in for visits). These descriptive statics will be presented for each state, to show how characteristics of participating practices varied in different demonstration states.

• What changes did practices make to enter and maintain participation in their state’s PCMH initiative?

• The questions that ask about medical home care processes (with answer options of “1” to “9”) will allow us to identify which aspects of the PCMH model of care each provider has adopted and incorporated into his or her daily practice.

• Do features of the participating practices result in more efficient delivery of health services, improved access, or higher quality of care to Medicare and Medicaid beneficiaries? If so, what features facilitate these improved outcomes?

• To answer this question, we will combine data collected from both of our companion surveys (which will give us information about practices’ basic characteristics as well as their adoption of specific PCMH care processes) with identifiable claims data (which will give us information on health care utilization, spending, and quality of care received by patients aligned with the responding practices) and with data from each state’s chosen PCMH recognition surveys (in most cases, the 2008 or 2011 NCQA PCMH recognition tool). Survey, claims and PCMH recognition data for an individual practice will be linked, and statistical analyses performed to identify which state and practice characteristics and/or PCMH activities are associated with the largest increases in health care quality, utilization and/or utilization measures.

We will study variation in the practice characteristics and medical home capabilities of PCMHs across the eight MAPCP Demonstration states, as well as within states (e.g., between metropolitan and nonmetropolitan areas or by practice size). To do this, we will draw on data collected through our provider survey. In a parallel analysis, we will draw on data from each state’s chosen medical home recognition surveys (in most cases, the 2008 or 2011 NCQA medical home recognition tool). These descriptive cross-sectional analyses, using both tabular and multivariate methods, will allow us to determine whether certain states or practice characteristics are more highly associated with implementation of certain medical home capabilities than others.

Our multivariate models would use the practice as the unit of observation and the practice-level variables as the independent variables. The dependent variables in these models would be the key outcomes of interest to this evaluation: total expenditures, acute-care expenditures, emergency department utilization, and hospitalizations). The models would vary depending on whether or not we are examining variation in medical home attributes within or across states. Our primary analysis will be a series of within-state analyses that only include practice-level variables. However, assuming the data allow us to define variables uniformly across states, we would also estimate a model that pools data from all 8 states and includes state-level measures related to the demonstration and other contextual factors.

In modeling the medical home variables within or across states, we understand that they are not likely to be independent of one another and, as such, the estimation of the models needs to take this into account. For example, practices that have systems in place for care coordination may also do well in managing care transitions. This can be done by producing estimates that explicitly correct for correlation in the error terms across individual models (e.g., seemingly unrelated regressions), or adjusts standard errors to prevent overstating significance when making multiple comparisons (e.g., Bonferroni correction). Alternatively, if we see that many medical home characteristics are routinely implemented together, we would develop composite measures from the underlying data and use these as the dependent variables in the modeling.

This part of the pooled modeling could compare the characteristics of PCMHs across states, to assess if certain types of initiatives are attracting practices at different levels of PCMH development than others. For example, we may find that initiatives that are attempting statewide implementation are less selective in the practices they certify to join their initiatives than states that are focused on a narrower geographic region. Our analytical approach could also be structured to allow us to examine the relationship between a number of the state-level contextual variables and practice-level PCMH attributes, as well as how a practice’s PCMH activities may be related to a range of the practice-level contextual variables. For example, we may find that large practices, those in urban areas, and/or those staffed by younger clinicians are more likely to have adopted more medical home capabilities than other practices. However, we recognize that a limitation of these pooled models is that we will not be able to distinguish between demonstration variables that are uniform across all practices in the state and time-invariant state fixed effects.

Broadly speaking, the results of CMS’s evaluation of the MAPCP Demonstration will benefit policymakers, payers, healthcare purchasers, primary care practices, and Medicaid and Medicare beneficiaries in the following ways:

• Payers will have information to help them understand which medical home payment models and technical assistance approaches have the greatest impact on health care quality and cost and utilization measures;

• Primary care practices will have data to inform them about what specific care processes are most associated with improved care quality, access, and patient outcomes;

• Patients will directly benefit from any improvements to how care is delivered that are implemented by policymakers, payers, and their primary care practices.

This information also will facilitate diffusion and implementation of similar initiatives in other states, if this demonstration is successful.

3. Use of Improved Information Technology and Burden Reduction

The proposed surveys will primarily be fielded online – as opposed to by mail, fax, or in-person – to minimize respondent burden. Near the end of the MAPCP Demonstration, practice staff will be sent an email inviting them to complete the surveys, which will include hyperlinks to websites where they can complete the surveys electronically and submit them to CMS’s evaluation contractors instantaneously. The surveys will automatically save respondents’ answers each time they advance to the next page of the survey – thus allowing respondents to partially fill out the survey, close their Internet browser, and return to the survey at a later time to finish completing it. (This is described at the beginning of the survey.) Evaluators are also exploring the possibility of designing these web-based surveys such that they can be completed on smart phones (e.g., iPhones).

4. Efforts to Identify Duplication and Use of Similar Information

The primary potential source of duplication of the proposed surveys are the medical home recognition surveys that practices were required to complete to enter (and in some cases, to maintain participating in) their state’s PCMH initiative.

All eight MAPCP states require practices to complete a medical home recognition survey to enter their demonstration. Our proposed surveys would supplement these existing data, rather than duplicate them, since our surveys would be fielded near the end of the demonstration, and thus allow us to identify what medical home care processes and activities practices had in place near the end of the demonstration (as opposed to at the beginning of the demonstration). In addition, our surveys would allow us to measure other aspects of practices’ medical home capabilities beyond those activities captured by state-endorsed medical home recognition surveys (e.g., NCQA).

In addition to requiring pratices to complete a medical home recognition survey to enter their state’s MAPCP demonstration projects, three MAPCP states (Maine, Pennsylvania, and Rhode Island) also plan to require practices to re-take their chosen survey near the end of Medicare’s three-year involvement in these states’ multi-payer medical home initiatives. Two other states (Minnesota and Michigan) require annual re-certification using state-specific medical home recognition standards. Some of the questions in these state-administered medical home recognition surveys have the potential to be duplicative of our proposed provider survey, however the questions are not likely to overlap completely with our proposed survey questions. (We are in the process of obtaining PCMH practice recognition data for all MAPCP Demonstration states.)

CMS and its evaluators carefully considered using the states’ data as its sole source of medical home recognition data but in the end determined that the proposed surveys are needed for several reasons. First, the diversity of survey instruments and different dates of administration will lessen the comparability of data collected through rounds of state data collection, which will hamper our ability to make cross-state comparisons. Second, three MAPCP states (Vermont, North Carolina, and New York) will not require practices to re-take a medical home recognition survey at any point in their demonstration. We are unable to assess how practices have transformed over the course of the demonstration without a second round of data collection. Third, CMS’s evaluators expect to see a reduction in the number of practices that agree to re-take their state’s medical home recognition survey, especially in states that have opted to use NCQA’s PCMH standards, which are considered extremely burdensome to complete (taking months to prepare for and weeks to upload the required documentation proving each answer they give on the survey) (Burton, Devers, Berenson 2012). Fourth, there is no guarantee that states with plans to collect medical home recognition surveys will actually implement them, especially given the volatility of state budgets in recent years. If the states do not administer their surveys as planned, CMS runs the risk of not having data to answer key research questions that are important for policy decisionmaking.

Because of the various concerns described above, CMS’s evaluators believe their evaluation of the MAPCP Demonstration would be significantly strengthened by the administration of common, short surveys to all participating practices at a consistent point in time near the end of the MAPCP Demonstration.

5. Impact on Small Businesses or Other Small Entities

Some of the providers surveyed will work for or own small businesses (i.e., physicians’ offices). However, the impact of this data collection on small businesses will be attenuated by the fact that 1) these providers have voluntarily chosen to participate in the MAPCP Demonstration, and 2) these providers will be explicitly told that the proposed surveys are voluntary (see recruitment email and the introductory section of the proposed surveys, included as attachments). Also, the short proposed surveys have been designed to be of minimal burden to respondents.

6. Consequences of Collecting the Information Less Frequently

CMS’s evaluators propose to administer these surveys once. The consequences of not fielding these surveys at all would be that CMS’s evaluators would lack consistent data about the medical home activities and care processes that practices participating in the MAPCP Demonstration are engaging in. In addition, evaluators would lack data documenting participating practices’ medical home capabilities near the end of the MAPCP Demonstration in the three states that do not require any subsequent fieldings of medical home recognition surveys, and would likely have incomplete data in several other states (due to the reasons we described above in section 4). Incomplete data would prevent evaluators from being able to capture the effects of medical home capabilities that practices improved upon after the start of the demonstration.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

CMS’s evaluators plan to list the OMB control number on the front page of the proposed survey. They do not anticipate any special circumstances that would require the need for other means to inform potential respondents of the OMB control number associated with this provider survey.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A notice about this proposed information collection, which was originally envisioned as a single survey for providers, was published in the Federal Register on July 12, 2013, on pages 41931-41932, and allowed 60 days for public comment. No public comments were received. However, we did receive feedback from six clinicians participating in the MAPCP Demonstration who pilot-tested our survey (a seventh clinician pilot-tested our survey but had no suggested revisions). In addition, staff from CMS and the firms it contracted with to develop this survey (RTI International and the Urban Institute) have suggested some minor refinements. As a result, minor revisions have been made throughout the survey in response to feedback from these parties. These revisions are documented in Attachment E. After review by OMB, CMS decided to split what was originally envisioned as a single survey for providers into two surveys: one for providers (Attachment D1), which will ask about medical home care processes and activities adopted in the practice; and a survey for non-clinical practice managers (Attachment D2), which will ask about basic practice characteristics. The estimated burden associated with completing the original survey was 18 minutes, based on the length of time it took for pilot-testers to complete the survey; this estimate has been changed to 12 minutes for providers to complete the medical home survey, and 6 minutes for practice managers to complete the practice characteristics survey.

9. Explanation of Any Payment or Gift to Respondents

We have budgeted $50 per practice for gift cards or a check to give the practice manager who will assist with the distribution of emails to providers requesting their participation in this survey. The $50 we propose offering to practice managers is intended to compensate them for labor spent reading emails from us, forwarding our medical home provider survey to the providers in their practice, answering physicians’ questions about the survey, and reminding providers to complete the survey. As noted above, we have also decided to ask practice managers to complete a practice characteristics survey consisting of a few basic practice-level questions that previously appeared in the second half of our survey; these questions will now only be asked of practice managers, and not providers.

We considered requiring practice managers to achieve an 80% within-practice response rate to receive this $50 incentive, but ultimately decided against it. We believe adding such a requirement could cause some practice managers to not bother trying to get any providers to complete the survey, if they assume right from the start that they will not reach the 80% threshold – for example if they think (based on their knowledge of the personalities of the physicians in their practice) they could only get 60% of their practice’s providers to complete the survey. We therefore think that making the $50 incentive contingent on achieving a within-practice response rate of 80% could actually reduce our response rate, and have not opted for such a requirement.

No remuneration will be offered to the providers completing the survey. We anticipate that providers participating in the MAPCP Demonstration will have a sufficient interest that they will be willing to participate without compensation and will feel a duty to respond to the survey given the enhanced reimbursement they are receiving as a result of their participation in the MAPCP Demonstration.

10. Assurance of Confidentiality Provided to Respondents

All personnel who will have access to data collected through the proposed surveys and/or individual identifiers collected through these survey instruments will be trained on the importance of keeping this information private, particularly as it relates to controlled and protected access to survey data and summary files. Further, materials sent to potential survey respondents will describe the purpose and the voluntary nature of the surveys and will convey to respondents that their responses will be kept private, to the extent permitted by law. The survey database will be stored on a secured server with access-limiting firewall protections, including encryption and password requirements. When data collected through these surveys are shared by RTI International with the Urban Institute, they will be transmitted via a secure, password-protected website and then saved on a confidential computer network drive that only Urban staff analyzing these data will have access to. Data collected through these surveys will be retained only long enough to perform analyses associated with CMS’s evaluation of the MAPCP Demonstration, and will then be destroyed.

11. Justification for Sensitive Questions

Information collected in the proposed surveys are not of a sensitive nature. Questions in the proposed surveys are confined to factual questions about whether a respondent’s practice engages in certain care processes and activities or not; no opinion or views are solicited, and the surveys include only a few open-ended answer options – for example, a few questions give respondents the option of answering “Other” and writing in a few words of text (e.g., the question “Please indicate the types of organizations that your practice is part of or affiliated with” includes an “Other” answer option and a blank field where respondents can write in an answer). Nevertheless, CMS’s evaluators will handle data collected through these surveys with sensitivity and with privacy in mind, and will not share nor attribute the responses of those individuals surveyed in an identifiable way in any written or oral communications.

12. Estimates of Annualized Burden Hours and Costs

Estimates of survey burden in terms of hours and annualized costs for this one-time fielding are shown in the table below.

Medical Home Survey for Health Care Providers

	Total Number of Respondents	Number of Responses per Respondent	Time Per Response	Cost Per Response	Total Cost Burden for All Respondents
Reporting	5,799	1	0.2 hours (12 min.)	$18.001	$104,382
Record Keeping	-	-	-	-	-
Third Party Disclosure	-	-	-	-	-
Total	5,799	1	0.2 hours (12 min.)	$18.00	$104,382

Practice Chracteristics Survey for Practice Managers

	Total Number of Respondents	Number of Responses per Respondent	Time Per Response	Cost Per Response	Total Cost Burden for All Respondents
Reporting	803	1	0.1 hours (6 min.)	$4.261	$3,420.78
Record Keeping	-	-	-	-	-
Third Party Disclosure	-	-	-	-	-
Total	803	1	0.1 hours (6 min.)	$4.26	$3,420.78

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

Not applicable. Respondents and record keepers face no capital or startup costs, or operation or maintenance costs.

14. Annualized Cost to the Federal Government

Total costs associated with the development, fielding, and analysis of the proposed surveys are estimated to be $268,669. This total cost estimate was derived from a summation of total estimated number of labor hours times hourly rates and approved indirect costs ($199,377) and direct operational expenses for establishing, administering, and hosting the web-based surveys ($69,292). These costs are funded through an existing CMS contract with RTI.

Federal FTE costs are expected to be negligible. The Project Officer for the CMS contract with RTI may be required to spend 0.2% of her time on the administration of these surveys (~$250 of annual salary).

15. Explanation for Program Changes or Adjustments

Not applicable. This is a new data collection for CMS. The proposed surveys will not result in any recurring periodic reporting, recordkeeping costs, or time burden.

16. Plans for Tabulation and Publication and Project Time Schedule

The results of these surveys will be tabulated and reported by CMS’s evaluation contractors in reports and publications. Results of these surveys are also expected to be linked to practice-identifiable claims data (using well-established quantitative methods) to allow researchers to identify which medical home activities or care processes are most highly correlated with improvements in care quality and reductions in health care cost trends. Results of these statistical analyses will be included in the final report to CMS, to be completed in March 2016. Additionally, the RTI/Urban/NASHP team may produce articles for peer-reviewed journals and conference presentations that will be reviewed and approved by CMS prior to submission.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

Not applicable. The OMB expiration date will be displayed on all survey respondent data collection materials.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Not applicable. CMS is able to certify compliance with the provisions of 5 CFR 1320.9.

1 These four medical home recognition instruments are described in detail in: Burton R, Devers K, and Berenson R. Patient-Centered Medical Home Recognition Tools: A Comparison of Ten Surveys’ Content and Operational Details. Baltimore, MD: U.S. Centers for Medicare and Medicaid Services, March 2012. Available at: http://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/Reports/Downloads/Burton_PCMH_RT_Survey_Compare_March_2012.pdf.

1 This amount assumes the hourly wage of a primary care physician is $90.00, per the Bureau of Labor Statistics, and the proposed survey is estimated to take x.x hours to complete. Source: Bureau of Labor Statistics, U.S. Department of Labor, Occupational Outlook Handbook, 2014-15 Edition, Physicians and Surgeons, on the Internet at http://www.bls.gov/ooh/healthcare/physicians-and-surgeons.htm (visited September 17, 2014).

1 This amount assumes the hourly wage of a medical practice manager is $42.59, per the Bureau of Labor Statistics, and the proposed survey is estimated to take x.x hours to complete. Source: Bureau of Labor Statistics, U.S. Department of Labor, Occupational Outlook Handbook, 2014-15 Edition, Medical and Health Services Managers, on the Internet at http://www.bls.gov/ooh/management/medical-and-health-services-managers.htm (visited September 17, 2014).

11