Response to Comments

Centers for Medicare & Medicaid Services (CMS) Response to the National Health Law Program (NHeLP) Public Comment to Information Collection CMS-10341: Section 1115 Demonstration Projects Regulations at 42 CFR Part 431

November 9, 2021

On May 7, 2021, the Centers for Medicare & Medicaid Services (CMS) released a notice in the Federal Register announcing an opportunity for public comment on CMS’s intention to renew the collection of information for the requirements at 42 CFR Part 431, which are the regulatory requirements for states submitting applications to CMS for new section 1115 demonstrations or to extend existing section 1115 demonstrations. Under the Paperwork Reduction Act of 1995 (the PRA), federal agencies are generally required to publish notice in the Federal Register concerning each proposed collection of information (including each proposed extension or reinstatement of an existing collection of information) and to allow 60 days for public comment on the proposed collection. Interested persons are invited to send comments regarding CMS’s burden estimates or any other aspect of the collection of information. In response, the National Health Law Program (NHeLP) submitted a comment that agreed with the necessity of the information collection, but also suggested that the existing information collection did not sufficiently address all legal requirements of section 1115(d) of the Social Security Act (the Act) and the implementing regulations at 42 CFR Part 431. NHeLP provides recommendations for CMS to revise the proposed information collection.

CMS thanks NHeLP for its comments and we will consider whether and how to further improve upon the transparency requirements of 42 CFR Part 431. However, many comments are not germane to the stated purpose of this information collection. CMS notes that the GAO has studied CMS’s consistency in the application of certain public notice procedures across various section 1115 applications. Many of the provisions at 42 CFR §431.408 and §431.412 were purposefully drafted to permit state flexibility in achieving the statutory objectives of section 1115(d) of the Act while minimizing administrative burden. And in trying to strike the right balance in supporting state flexibility within consistent parameters, CMS acknowledges that there is some intended inconsistency that the GAO nonetheless finds objectionable. In response, CMS has undertaken several activities to improve consistency in how the agency interprets what meets the requirements for state public notice (42 CFR §431.408) and application content (42 CFR §431.412) for a "complete" demonstration application. These efforts to address the GAO’s findings – and any resulting associated burden for states – has already been factored in this proposed information collection for renewal. With one noted exception described below, the current information collection addresses all legal requirements of the final rule provisions at 42 CFR Part 431. CMS also made some minor descriptive edits to the information collection for clarity, where noted, in response to the NHeLP comments. We continue to examine state and federal transparency practices to inform guidance aimed at promoting improved transparency of section 1115 demonstrations and again we appreciate the suggestions provided by NHeLP to inform this effort. Please see CMS’s responses to NHeLP’s specific comments below.

Comment 1: Sufficiency of the Proposed Information Collection

NHeLP stated: “NHeLP agrees that information collection CMS-10341 is ‘necessary to ensure that states comply with regulatory and statutory requirements related to the development, implementation and evaluation of demonstration projects.’ The existing collection, however, does not sufficiently address all legal requirements.” (NHeLP comments, pg. 2).

CMS Response: With the exception of certain reporting requirements under 42 CFR §431.424 and 42 CFR §431.428, as discussed below, the information collection sufficiently addresses each provision of 42 CFR Part 431 that establishes requirements for states in the development of an application for a new section 1115 demonstration or to extend an existing section 1115 demonstration. The information collection describes the burden associated with the overall implementation of each regulatory provision as a whole; and burden estimates account for all requirements within each statutory provision that is necessary to establish full compliance. CMS agrees that the burden estimates associated with certain reporting requirements should have been adjusted to more closely align with CMS’s robust reporting requirements, such as distinguishing the “final evaluation report” requirement (referred to as the “summative evaluation report” in the CMS Special Terms and Conditions (STCs)), and accordingly CMS adjusted the burden hours in the information collection associated with the provisions at 42 CFR §431.424 and 42 CFR §431.428. CMS also made minor clarifying revisions to the “State Public Notice Process (§431.408)” section of the Supporting Statement to identify the several public notice documents that are required by this provision in response to the NHeLP comments.

Action(s) Taken: CMS revised the Supporting Statement to add burden hours associated with certain provisions at 42 CFR §431.424 and 42 CFR §431.428 and made minor clarifying revisions to the “State Public Notice Process” section.

Comment 2: Coverage loss estimates

NHeLP stated: “Federal statute requires that states provide information regarding ‘coverage projections’ as part of an initial application for or a request to extend a Section 1115 demonstration project. 42 C.F.R. § 431.408(a)(1)(i)(C) specifies that states must include ‘[a]n estimate of the expected increase or decrease in annual enrollment.’ States have sometimes failed to provide these estimates. CMS should ensure that the information collection includes a requirement that a state provide enrollment projections for the proposed demonstration.” (NHeLP comments, pg. 2, footnote omitted).

CMS Response: As stated above, many of the provisions at 42 CFR §431. 408 and §431.412 were purposefully drafted to permit state flexibility while minimizing administrative burden. Section 1115 demonstrations vary significantly in nature and impact as the demonstration authority allows a broad scope for states to address a myriad of evolving state-specific program and beneficiary needs. CMS has generally only considered applications to be “complete” when there is some information on enrollment estimates. CMS has accepted enrollment projection information in several forms, such as by providing eligible person counts, eligible member month counts, or by aggregate percentage increases/decreases on enrollment. Irrespective of how the state reflects enrollment projections, the information must be provided in a way that allows CMS to derive the number of individuals expected to be impacted by the demonstration and reflects change, if anticipated, over the course of the demonstration.

The information collection addressed the content of the full public notice process required by 42 CFR §431.408 on page 4 in the section designated “State Public Notice Process (§431.408).” CMS stated that “[t]he public notice must address the information requirements listed at §431.408(a)(1)(i) through (iv)” – which is where the requirement for enrollment estimates resides. However, CMS has amended this paragraph to list out the specific requirements of 42 CFR §431.408(a)(1)(i) through (iv). As indicated above, the burden estimate already factors in these requirements and does not require adjustment.

Action(s) Taken: CMS amended the section designated “State Public Notice Process (§431.408)” in the Supporting Statement to list out the specific requirements of §431.408(a)(1)(i) through (iv).

Comment 3: Specify provisions waived

NHeLP stated: “States are required to include in 1115 applications a list of the waiver authorities necessary for the waiver. Often, however, applications for section 1115 demonstration projects do not specify with precision which provisions of 42 U.S.C. § 1396a they seek to waive. We encourage CMS to require states to include in their applications a waiver list that identifies the specific statutory provisions the state is asking to waive and for what purpose they seek the waiver, including an explanation of why that purpose cannot be achieved through a state plan amendment.” (NHeLP comments, pg. 2, footnote omitted).

CMS Response: CMS has generally only considered applications to be “complete” when there is some information on the proposed waiver or expenditure authorities that the state believes is necessary to achieve the goal(s) of the demonstration project. States are sometimes uncertain about the specific statutory provision(s) that may need to be waived or considered non-applicable for an expenditure authority, and therefore, CMS has permitted the authorities requirement to be met through the identification of program areas where flexibility may be needed to achieve the state’s goal(s) of the demonstration project. And in instances where states are amending existing demonstrations, states have been permitted to provide a statement on whether or not new waiver and/or expenditure authorities are needed to achieve the proposed change without listing the already approved authorities. Irrespective of how the state reflects this information, the information must be provided in a way that allows CMS to determine the applicable provisions of federal law and/or regulation where the state may need flexibility to operationalize the proposed demonstration.

As mentioned in response to Comment 1 above, the information collection addressed the content of the full public notice process required by 42 CFR §431.408 on page 4 in the section designated “State Public Notice Process (§431.408).” CMS stated that “[t]he public notice must address the information requirements listed at §431.408(a)(1)(i) through (iv)” – which is where the requirement for a list of waiver and expenditure authorities resides. However, CMS has amended this paragraph to list the specific requirements of §431.408(a)(1)(i) through (iv). As indicated above, the burden estimate already factors in these requirements and does not require adjustment.

Action(s) Taken: CMS amended the section designated “State Public Notice Process (§431.408)” in the Supporting Statement to list out the specific requirements of §431.408(a)(1)(i) through (iv).

Comment 4: Proposed hypotheses, research design, and evaluation plans

NHeLP stated: “Accordingly, when submitting an application for a Section 1115 demonstration project states should include a comprehensive description of their hypotheses and research design. Likewise, when a state proposes an amendment to the project, the state should update its hypotheses and research design to reflect the amendment. CMS should update this information collection to ensure that states provide sufficient information regarding the state’s hypotheses, research design, and evaluation plans to enable CMS and the public’s review. (NHeLP comments, pg. 3, footnote omitted).

CMS Response: CMS has generally only considered applications to be “complete” when there is some information on the proposed hypothesis and parameters for evaluating the intended goal(s) of the approved demonstration. CMS has permitted broad flexibility in how states provide this information for the purpose of deeming an application "complete." However, CMS minimally requires that state section 1115 submissions list proposed hypotheses for measuring outcomes, that each hypothesis has a quantifiable target for improvement that relates to a demonstration goal, and that associated data source(s) or the research approach for testing each hypothesis are identified. Described in more detail below, section 1115 amendments are not within scope of this information collection, but we note that CMS’s standard STC for amendment submissions includes a requirement for an updated evaluation design that accounts for the proposed change. Further, as part of the federal review process, and to a greater extent after CMS’s approval, states are to comprehensively design an evaluation that meets CMS’s evaluation requirements.

As mentioned in response to Comment 1 above, the information collection addressed the provision of hypothesis and evaluation parameters as required by 42 CFR §431.408 on page 4 in the section designated “State Public Notice Process (§431.408).” CMS stated that “[t]he public notice must address the information requirements listed at §431.408(a)(1)(i) through (iv)” – which is where the requirement for hypothesis and evaluation parameters resides. However, CMS has amended this paragraph to list out the specific requirements of §431.408(a)(1)(i) through (iv). As indicated above, the burden estimate already factors in this requirement and does not require adjustment.

Action(s) Taken: CMS amended the section designated “State Public Notice Process (§431.408)” in the Supporting Statement to list out the specific requirements of §431.408(a)(1)(i) through (iv).

Comment 5: Analysis of racial equity impacts

NHeLP stated: “Accordingly, CMS should require states to provide information sufficient for the Secretary to evaluate the racial equity impacts of their Section 1115 projects.” (NHeLP comments, pg. 4).

CMS Response: As NHeLP indicates, the Biden Administration has prioritized pursuing a comprehensive approach to advancing equity for all, including people of color and others who have been historically underserved, marginalized, and adversely affected by persistent poverty and inequality. Per Executive Order 13985, CMS is examining all programs and agency actions to determine where we can enhance investment in underserved communities. However, the scope of this information collection is limited to the regulations at 42 CFR Part 431, and the policy articulated in this Executive Order is not codified in the regulations at 42 CFR Part 431. Accordingly, CMS has not adjusted the information collection to require this information.

Action(s) Taken: None, as this comment is outside the scope of this information collection.

Comment 6: Final evaluations

NHeLP stated: “Although final evaluations should be submitted for every demonstration, ‘CMS has historically not required states to submit final, comprehensive evaluation results at the end of each demonstration cycle. Instead CMS has allowed states to submit only interim evaluations. But these interim evaluations serve a different purpose from comprehensive, final evaluations. The interim evaluations typically are ‘based on more limited data from the early years of the demonstration cycle,’ and unlike a final evaluation, lack ‘evidence on outcomes and impacts.’ The information collection appears to reinforce this problematic practice: it does not include any burden estimates regarding final, comprehensive evaluations, instead estimating only the time to develop an evaluation plan and an interim evaluation report. CMS should update the information collection to include information regarding the final, comprehensive evaluations described in the regulations.” (NHeLP comments, pgs. 4-5)

CMS Response: CMS first clarifies that all section 1115 evaluation reports must align with the evaluation requirements of 42 CFR §431.424. States are typically required in the STCs to submit the following: 1) the interim evaluation report due when a state requests an extension of the approved demonstration period; and 2) the summative evaluation report due 18 months after the end of a demonstration approval period. If the state does not renew its demonstration and proceeds with a final phase-out of the demonstration, the summative evaluation report that is due 18 months after the end of the demonstration approval period serves as the “final evaluation report.” CMS uses the term “summative evaluation” instead of “final evaluation” for the report due at the end of a demonstration cycle. The GAO (like NHeLP in its comments) used the term “final evaluation report” in its 2018 audit titled, “Medicaid Demonstrations: Evaluations Yielded Limited Results, Underscoring Need for Changes to Federal Policies and Procedures” (GAO-18-220). CMS believes that this difference in terminology has contributed to some misunderstanding about CMS’s implementation of the evaluation requirement due at the end of a state’s demonstration period.

The GAO’s recommendation from audit GAO-18-220 was that CMS should establish written procedures for implementing the Agency's policy that requires all states to submit a “final evaluation report” after the end of each demonstration cycle, regardless of extension status. As noted above, CMS refers to this requirement as the “summative evaluation report” in the standard evaluation STCs incorporated into every section 1115 demonstration approval. What the GAO noted in this audit was CMS’s then practice of not requiring states to submit a completed evaluation report as a follow-up to the interim evaluation report submitted with the state’s extension application. In response, in 2017, CMS began requiring states to submit a summative evaluation report 18 months after the end of the approved demonstration period regardless of renewal status. The summative evaluation report follows the interim evaluation report due at the time a demonstration extension application is submitted (which is generally due one year prior to the demonstration expiration date) and reports on all evaluation activities and findings from the completed demonstration cycle. Both the interim and summative reports are required to align with the CMS-approved evaluation design. In January 2020, after review of CMS’s implementation of the requirement for a summative evaluation report, the GAO formally closed the recommendation as implemented, as “actions that satisfy the intent of the recommendation have been taken.”¹

Action(s) Taken: CMS revised the “evaluation requirements” section of the Supporting Statement to separately identify the requirement for a summative evaluation report. Additionally, after considering NHeLP’s comments on the “final evaluation” report requirement, CMS also made increasing adjustments to the burden hours associated with the requirements for interim evaluations and annual reports to more accurately reflect alignment with CMS’s more robust standards for evaluation reports.

Comment 7: Amendments

NHeLP stated: “The information collection only discusses requirements relating to ‘applications for and extensions of’ Section 1115 demonstration projects. It does not discuss information collection for proposed amendments. We have seen, however, a trend of states submitting proposed ‘amendments’ to an existing project that are, in substance, applications for significantly different projects, with components unrelated to the existing project. CMS should extend the required information collections to amendments and strengthen its initial review to consistently require states to submit new applications when requesting components unrelated to the existing project.” (NHeLP comments, pg. 5)

CMS Response: Section 1115(d) of the Act, as amended by section 10201(i) of the Affordable Care Act and implemented in regulation at 42 CFR Part 431, does not pertain to actions to amend existing section 1115 demonstrations. As CMS expressed in the April 27, 2012 State Health Official Letter on the “Revised Approval Process for Section 1115 Demonstrations” (SHO#12-001): “[t]he final rule left open the question of public notice requirements for proposed amendments to existing demonstrations…In

collaboration with States and other stakeholders, CMS will evaluate the types of demonstration amendments that are submitted and issue further guidance in the future on how the notice and

comment provisions would best be applied at the State and Federal levels to amendments that

have a significant impact.” (pg. 12, emphasis added) Since release of this SHO letter, CMS refined the standard STC for amendments to require more robust application content as well as applies the federal review procedures at 42 CFR §431.416 to demonstration amendments in the same manner as initial demonstrations or demonstration extensions. These practices were implemented to better align amendment applications with the principles of section 1115 transparency; but these practices are not codified in regulation, and accordingly, are not subject to this information collection.

Action(s) Taken: None as this comment is outside the scope of this information collection.

Additional Comments:

In part II of NHeLP’s comments, NHeLP provides suggestions for improving the information collected in state section 1115 applications pertaining to budget neutrality, estimates of administrative costs, quality of evaluations, and standardizing how states address state-level public comments. In part III of NHeLP’s correspondence, suggestions are provided for updating CMS’s existing burden estimates to more sufficiently account for the scope of information states should be providing as part of their section 1115 applications.

CMS Response: As discussed above, this information collection is limited to the regulations at 42 CFR Part 431 – to which budget neutrality and the treatment of administrative costs in the development of budget neutrality are implemented through CMS policy and not codified in regulation at 42 CFR Part 431. CMS has modified the burden associated with the evaluation provisions of 42 CFR §431.424 as described under Comment 6 above to the extent applicable to this information collection. Further, regulations at 42 CFR Part 431 do not regulate the format of how states report on the issues raised by the public during the comment period and how the state considered the comments when developing the demonstration application. For these aspects of NHeLP’s feedback that are not governed by statute, again, CMS appreciates the feedback and it will be considered.

Part III of NHeLP’s correspondence largely duplicates the feedback provided in Part I of NHeLP’s correspondence and CMS has addressed in specific detail above the burden adjustments made in consideration of NHeLP’s feedback. No further revisions were made specifically in response to the feedback provided in this section.

Action(s) Taken: CMS has modified the burden associated with the provisions of 42 CFR Part 431 as described above to the extent applicable to this information collection. However, for the aspects of NHeLP’s feedback on the transparency process that are not governed by statute, again, CMS appreciates the feedback and will be considered.

1 Full GAO report, “Medicaid Demonstrations: Evaluations Yielded Limited Results, Underscoring Need for Changes to Federal Policies and Procedures” (GAO-18-220) and status of recommendations available here: https://www.gao.gov/products/gao-18-220.

Page 4 of 4