CMS-10487 Supporting Statement A_rev_clean_022414

Part A

Supporting Statement

Medicaid Emergency Psychiatric Demonstration Evaluation

A. Background

Since the inception of Medicaid, inpatient care provided to adults ages 21 to 64 in institutions for mental disease (IMDs) has been excluded from federal matching funds. The Emergency Medical Treatment and Active Labor Act (EMTALA), however, requires IMDs that participate in Medicare to provide treatment for psychiatric emergency medical conditions (EMCs), even for Medicaid patients for whose services they cannot be reimbursed. Section 2707 of the Patient Protection and Affordable Care Act (ACA) of 2011 (P.L. 111-148; Attachment A) directs the Secretary of Health and Human Services to conduct and evaluate a demonstration project to determine the impact of providing payment under Medicaid for inpatient services provided by private IMDs to individuals with emergency psychiatric conditions between the ages of 21 and 64. This project, the Medicaid Emergency Psychiatric Demonstration (MEPD), and its evaluation are being implemented by the Centers for Medicare & Medicaid Services (CMS). On May 10, 2011, CMS received Paperwork Reduction Act (PRA) approval to collect application information from states interested in participating in the demonstration (OMB control number 0938-1131, now discontinued) and, in March 2012, selected 11 states and the District of Columbia to participate. Within these states, the participation of 27 private IMDs was approved. The goal of the 3-year demonstration is to assess whether the expansion of Medicaid coverage to include services provided in private, free-standing inpatient psychiatric facilities improves access to and quality of medically necessary care and whether this change in reimbursement policy is cost-effective. Focusing on psychiatric emergencies, the demonstration is also an attempt to explore a potential remedy to alleviate one of the factors contributing to psychiatric boarding, one of the consequences associated with the Medicaid IMD exclusion. The current PRA submission requests approval to collect data in association with the mandated evaluation of the demonstration.

Section 2707 of the ACA specifies that the evaluation shall include the following:

1. An assessment of access to inpatient mental health services under the Medicaid program; average lengths of inpatients stays; and emergency room (ER) visits;

2. An assessment of discharge planning by participating hospitals;

3. An assessment of the impact of the demonstration project on the costs of the full range of mental health services (including inpatient, emergency, and ambulatory care);

4. An analysis of the percentage of consumers with Medicaid coverage who are admitted to inpatient facilities as a result of the demonstration project as compared to those admitted to these same facilities through other means; and

5. A recommendation regarding whether the demonstration project should be continued after December 31, 2013, and expanded on a national basis.

The ACA further mandates that “not later than December 31, 2013, the Secretary shall submit to Congress and make available to the public a report on the findings of the evaluation.”

In addition to the requirements for the evaluation, the ACA specifies the following aspects of the demonstration:

1. The population served by the demonstration is limited to individuals who require medical assistance to stabilize a psychiatric EMC, defined as a situation where the individual “expresses suicidal or homicidal thoughts or gestures, if determined dangerous to self or others.” CMS expanded the eligibility criteria, as of October 1, 2012, to also include Medicaid enrollees who may not have suicidal or homicidal thoughts, gestures, or ideations but are, nevertheless, determined to be dangerous to self or others.

2. States are required to establish a mechanism for determining whether or not participants have been stabilized, meaning that “the EMC no longer exists…and the individual is no longer dangerous to self or others.” The stabilization assessment mechanism must commence before the third day of the inpatient stay.

Understanding the manner in which these requirements have been operationalized and the way in which they may affect outcomes of the demonstration is important for informing possible continuance and expansion of the demonstration on a national basis.

In addition to the ACA specifications regarding the demonstration, many stakeholder groups believe that the IMD exclusion, coupled with a general shortage of specialized inpatient psychiatric beds, contributes to extended psychiatric boarding—the practice of holding a patient with a psychiatric EMC in an ER or general hospital nonpsychiatric medical unit (known as “scatter beds”) because no specialized beds are available. Psychiatric boarding is thought to contribute to overcrowding of ERs and to result in substandard care for beneficiaries in facilities that are not well equipped to treat psychiatric conditions. The expectation is that by increasing access to IMDs, the demonstration will decrease psychiatric boarding and use of scatter beds, thereby improving quality of care for beneficiaries with psychiatric EMCs. Therefore, an assessment of changes in psychiatric boarding, use of scatter beds, and quality of care is essential for understanding the extent to which the demonstration results meet stakeholder expectations.

To respond to the ACA evaluation requirements, CMS is planning a comprehensive, mixed-methods evaluation of the MEPD. CMS is requesting approval from the Office of Management and Budget (OMB) for the collection of qualitative data through site visits and key informant and beneficiary interviews as part of the evaluation.

Fully assessing all of the areas mandated by the ACA as well as the interests of critical stakeholders necessitates a mixed-methods approach. Quantitative data on service utilization and expenditures are critical to successfully evaluating the MEPD’s impact in ACA-mandated evaluation areas A, C, and D. Only a qualitative approach, however, can provide a full assessment of discharge planning by participating hospitals, as mandated by ACA evaluation area B, and of psychiatric EMC determination and stabilization processes utilized to ensure compliance with ACA demonstration requirements; in addition, few if any data are available on the use of scatter beds and psychiatric boarding times in ERs, so an understanding of the extent and impact of these practices may only be possible through qualitative methods.

Quantitative Data Collection

To the extent possible, the evaluation will use publicly available data to minimize burden on the demonstration states and facilities. Medicaid and Medicare enrollment and claims data submitted to CMS will be used to address ACA evaluation areas A and C. Variation in the quality, timeliness, and completeness of Medicaid data across states will necessitate that the demonstration states assist the evaluation contractor to develop a clear understanding of claims data submitted by their particular states. In addition, because not all data needed to address the ACA evaluation mandates are included in claims data, the evaluation will ask the states and facilities to submit relevant administrative data that they collect for other purposes. In particular, because of the IMD exclusion, data on IMD inpatient psychiatric admissions are not available through claims data. Information about admissions as a result of the demonstration will be available through claims that the states submit to CMS for demonstration payment and monitoring purposes, but data for comparison group admissions will have to be obtained from state or facility administrative sources. Comparison data for admissions before the demonstration began are needed to determine the extent to which IMD admissions, lengths of stay, and costs during the demonstration represent a change from IMD admissions, lengths of stay, and costs prior to the demonstration. In addition, data from non-participating psychiatric facilities are needed to determine the extent to which such changes are due to the demonstration itself rather than non-demonstration factors. In addition to IMD admissions, identification of psychiatric EMCs may also not be fully possible through Medicaid and Medicare data alone, and quantitative data on psychiatric boarding times, if available, must also be obtained directly from states or facilities.

Qualitative Data Collection

Information on processes of care that are critical to the success of the demonstration are not available through quantitative data. Nonetheless, CMS has an interest in ensuring the proper conduct of discharge planning to (1) achieve positive health outcomes for Medicaid beneficiaries, (2) limit costs related to readmissions that may occur when discharges are premature, and (3) ensure that clients are served in community-based settings whenever possible. While these outcomes of discharge planning will be assessed through analysis of quantitative data, information about the processes used to conduct discharge planning itself can only be obtained through qualitative approaches. Qualitative data are critical for understanding the relationships among length of stay, initiation of stabilization assessments and discharge planning, stabilization of emergency psychiatric conditions, and discharge. The qualitative data collected will enrich the evaluation’s understanding of quantitative results, permit consideration of alternative explanations for significant changes over time, examine the circumstances under which varying effects might be expected if Congress expands the demonstration, and help generate hypotheses about outcomes for further exploration through quantitative data analysis.

Because the demonstration operates at state, facility, and beneficiary levels, CMS proposes a systematic qualitative data collection approach that addresses each of these levels. Key informant interviews and document review conducted for each level will be used to cross-validate one another. Two rounds of site visits will be conducted. The first will occur about 24 months after the start of the demonstration (spring 2014) and focus on admission, stabilization, and discharge-planning procedures before and after the demonstration. The second round of visits will occur about a year later (spring 2015) to allow the evaluation team to gather detailed information on changes in these procedures, as well as lessons learned and sustainability of changes made. For each state, during each round an evaluation contractor will visit all participating IMDs and, for each IMD, one ER that refers patients to that IMD and one general hospital that admits patients with psychiatric EMCs to general medical units when no psychiatric bed is available.

During the site visits, the evaluation contractor will interview facility staff regarding processes of care that are critical to the success of the demonstration, namely procedures for psychiatric EMC determination, inpatient admissions, stabilization assessment, stabilization, and discharge planning. Interview questions for staff at each type of facility are included in Attachment B. In addition, during each site visit, purposive sampling will be used to select 10 medical records to review at each facility. (See Attachment C for sampling procedures.) From IMD and general hospital records, information regarding stabilization and discharge-planning procedures and interventions administered will be extracted. From ER records, CMS proposes extracting information regarding length of time spent in the ER, psychiatric EMC determination, interventions administered, and inpatient referral procedures. The medical record review tool is included in Attachment D. Records from both pre- and post-demonstration time periods will be reviewed to assess how care has changed.

Prior to each site visit, the evaluation contractor will conduct a semi-structured phone interview with the state demonstration project director, using questions included in Attachment B. The contractor will also review site documents, such as operation plans, psychiatric EMC determination procedures, and stabilization assessment and discharge-planning policies. After each round of site visits, evaluation teams will conduct telephone interviews with five beneficiaries receiving inpatient services through the demonstration from each participating IMD, for a total of 135 interviews. These interviews will be essential to understanding beneficiaries’ experiences with the admission and discharge processes. Moreover, the beneficiaries’ viewpoints are critical to understanding if and how quality of care improves as a result of the demonstration. Beneficiary interview questions are included in Attachment E, along with a draft consent form and the recruitment script for beneficiary interviews.

To manage the voluminous qualitative data collected from interviews and site documents, the evaluation contractor will systematically code and analyze the data using Atlas.ti, a qualitative data analysis software package. Data gathered from medical records will be entered into a data entry program called Viking. These data will then be exported as SAS files for analysis at the facility-level.

Limitations of the Evaluation

The planned evaluation will address all four ACA-mandated areas. The rigor with which each area can be addressed and the generalizability of the results will depend upon the availability and feasibility of obtaining needed data, the completeness and quality of data obtained, and the appropriateness of available comparison groups. Specific limitations of the evaluation include the following:

• Lags in the availability of data and time needed to process and analyze data may not allow all Medicaid and Medicare services provided within later months of the demonstration period to be analyzed; moreover, lags in data reporting mean that data from later years are likely to be less complete and accurate than data from earlier years.

• Data from some states or facilities may lack critical data elements, suffer significant quality issues, or be inaccessible for needed populations or time periods due to privacy concerns, lack of staff time to prepare and send the data, or other reasons. As a result, data for some states or facilities may be inadequate for conducting some analyses.

• Due to resource constraints, any sites added to the demonstration after September 2012 will not be included in data collection and analysis.

• In the broadest sense, the ACA specifies that the evaluation is “to determine the impact on the functioning of the health and mental health service system and on individuals enrolled in the Medicaid program.” Due to limitations on data availability and project resources, analyses will be limited to the functioning of health and mental health service system components funded through Medicaid and Medicare and will not include aspects of the system that are funded by states or other sources, other than through qualitative interviews with state demonstration and facility staff. Examination of impacts on individuals in the Medicaid program will be limited to effects on Medicaid and Medicare service utilization, supplemented by qualitative interview data regarding beneficiary experiences with the demonstration processes. Data on important individual outcomes, such as symptoms and well-being; incidence of suicide, homicide, self-harm, and harm to others; and post-discharge incidence of arrests, homelessness, death, and institutionalization in nonparticipating IMDs, cannot be obtained within project resources.

• Comparison facilities will not always be ideal, and we will not always be able to control adequately for secular trends that affect facilities and outcomes.

• The effects of the demonstration are expected to be specific to or strongest for people with psychiatric EMCs. Data sources, however, are unlikely to include data elements that clearly indicate the presence of psychiatric EMCs as defined for the demonstration. Therefore, analyses will most likely have to be conducted with broader proxies for psychiatric EMCs, such as any use of emergency services associated with a psychiatric diagnosis. Use of such proxies may weaken our ability to detect effects.

• Few quantitative data regarding discharge planning exist. Therefore, the assessment of discharge planning will primarily involve qualitative data analysis. Discharge planning processes used before and during the demonstration in participating facilities and general hospital scatter beds will be described, but impact estimates cannot be calculated.

• The ACA calls for an assessment of the impact of the demonstration on the costs of the full range of mental health services (including inpatient, emergency and ambulatory care). Post-demonstration changes in overall Medicaid and Medicare mental health costs relative to pre-demonstration costs will be examined. Impact estimates, however, cannot be generated. Assessing impacts requires parallel examination of changes in equivalent systems over the same period of time. Lack of comparability of Medicaid systems and potential confounding factors across states, coupled with limited project resources, precludes the analysis of comparison systems.

B. Justification

Need and Legal Basis

Section 2707 of the Patient Protection and Affordable Care Act (ACA) of 2011 (P.L. 111-148; Attachment A) directs the Secretary of Health and Human Services to conduct and evaluate a demonstration project to determine the impact of providing payment under Medicaid for inpatient services provided by private IMDs to individuals with emergency psychiatric conditions between the ages of 21 and 64.

Information Users

The data will be used by CMS to evaluate the MEPD in accordance with the ACA mandates. This evaluation in turn will be used by Congress to determine whether to continue or expand the demonstration. If the decision is made to expand the demonstration, the data collected will help to inform CMS and their stakeholders about possible effects of contextual factors and important procedural issues to consider in the expansion, as well as the likelihood of various outcomes. Although the results of this data collection will not be included in the report to be submitted to Congress by December 31, 2013, we anticipate that Congressional and stakeholder interest will continue until the evaluation results are published. A comprehensive report of the findings will be produced in the final year of the project, and interim results will be described in annual reports and presentations made via webinar during the final two years of the project.

The conceptual framework for the evaluation is presented in visual and narrative form in Attachment F. Table 1, below, lists the specific research questions to be examined, how each relates to the ACA mandates, and the data sources that will be used to answer each question. As shown in the table, CMS Medicaid and Medicare claims data will be the primary data sources for addressing ACA-mandated evaluation areas A and C, state and facility administrative data will contribute importantly to addressing all mandated areas, and the qualitative data sources together will provide the primary information for addressing ACA-mandated area B (discharge planning), as well as psychiatric boarding, which falls under area A, regarding ER visits. In addition to discharge planning, the multiple sources of qualitative data will provide important cross-validating perspectives on processes of care that are critical to understanding the success or failure of the demonstration, including psychiatric EMC determination, inpatient admission, and stabilization procedures.

Table 1. Use of Data to Answer Research Questions Addressing ACA-Mandated Evaluation Areas

Research Question	Data Source	ACA-Mandated Evaluation Area
To what extent do private IMDs increase admissions of Medicaid beneficiaries with psychiatric emergencies as a result of the demonstration?	CMS demonstration payment and monitoring data State or facility administrative data	(A) Medicaid inpatient access, length of stay, and reduced ER visits (D) The percentage of consumers who are admitted to participating IMDs as a result of the demonstration compared to those admitted to the same facilities with other payment arrangements
Does the demonstration decrease admissions to nonpsychiatric units of general hospitals for Medicaid beneficiaries with psychiatric emergencies?	CMS Medicaid and Medicare claims data—pre- and post- demonstration	(A) Medicaid inpatient access, length of stay, and reduced ER visits (C) Impact on system costs of the full range of mental health services, including inpatient, emergency, and ambulatory care
What is the demonstration’s effect on lengths of stay for Medicaid beneficiaries with psychiatric emergencies admitted to private IMDs compared with lengths of stay in these facilities before the demonstration and to lengths of stay in other facilities?	CMS demonstration payment and monitoring data CMS Medicaid and Medicare claims data State or facility administrative data	(A) Medicaid inpatient access, length of stay, and reduced ER visits
What is the demonstration’s effect on lengths of stay for Medicaid beneficiaries with psychiatric emergencies admitted to scatter beds in general hospitals?	CMS Medicaid and Medicare claims data	(A) Medicaid inpatient access, length of stay, and reduced ER visits (C) Impact on system costs of the full range of mental health services, including inpatient, emergency, and ambulatory care
Are fewer Medicaid beneficiaries with psychiatric emergencies seen in ERs as a result of the demonstration?	CMS Medicaid and Medicare claims data—pre- and post- demonstration	(A) Medicaid inpatient access, length of stay, and reduced ER visits (C) Impact on system costs of the full range of mental health services, including inpatient, emergency, and ambulatory care
Does the demonstration reduce psychiatric boarding time in ERs for Medicaid beneficiaries with psychiatric emergencies?	ER administrative data Key informant interviews Beneficiary interviews Medical records review	(A) Medicaid inpatient access, length of stay, and reduced ER visits
Does the demonstration increase the proportion of individuals discharged with a continuing care plan from the participating hospitals?	Quality improvement data obtained from CMS’s Inpatient Psychiatric Facilities Quality Reporting program	(B) Discharge planning
Does the demonstration improve the quality of discharge plans? - Does the demonstration increase the length of time spent developing a discharge plan for Medicaid beneficiaries with psychiatric emergencies in participating IMDs? - Does the demonstration increase the proportion of Medicaid beneficiaries with psychiatric emergencies in participating IMDs who are discharged to community-based residences (compared to before the demonstration and compared to nonparticipating IMDs and nonpsychiatric units of general hospitals)? - Does the demonstration increase the level of detail (e.g., appointment times, names of providers) included in the discharge plans for Medicaid beneficiaries with psychiatric emergencies in participating IMDs? - How does the discharge-planning process in participating IMDs compare (in terms of the previous questions) to the processes in nonpsychiatric units of general hospitals?	CMS demonstration payment and monitoring data Key informant interviews Document review Medical record reviews Beneficiary interviews	(B) Discharge planning
Does the demonstration reduce 30-day readmissions (all cause and psychiatric) for patients discharged from participating IMDs for a psychiatric emergency (compared to before the demonstration and compared to nonparticipating IMDs and nonpsychiatric units of general hospitals)?	CMS Medicaid and Medicare claims data State or facility administrative data	(B) Discharge planning (C) Impact on system costs of the full range of mental health services, including inpatient, emergency and ambulatory care
What effect does the demonstration have on costs to the IMDs, states, Medicaid, and Medicare? - What are the federal Medicaid costs for care provided by private IMDs as a result of the demonstration? - To what extent do costs incurred by the states for Medicaid emergency IMD admissions decrease after the demonstration’s implementation? - To what extent do costs incurred by participating IMDs for Medicaid emergency IMD admissions decrease after the demonstration’s implementation? - What is the demonstration’s effect on overall costs to Medicaid and Medicare for care provided to beneficiaries with emergency psychiatric conditions (perhaps through cost savings in ER utilization, general hospital scatter bed and inpatient psychiatric unit admissions, nursing home admissions, and so forth)? - What additional administrative costs are incurred by states and participating facilities to fully implement the demonstration’s service-delivery models?	CMS demonstration payment and monitoring data CMS Medicaid and Medicare claims data State or facility administrative data Key informant interviews	(C) Impact on system costs of the full range of mental health services, including inpatient, emergency, and ambulatory care
Within participating IMDs, how does the percentage of inpatients who are Medicaid beneficiaries admitted as a result of a psychiatric emergency change relative to the percentage of inpatients admitted through other means (i.e., with payment sources other than Medicaid and/or not as a result of a psychiatric emergency) after the demonstration’s implementation?	CMS demonstration payment and monitoring data State or facility administrative data	(D) The percentage of consumers who are admitted to participating IMDs as a result of the demonstration compared to those admitted to the same facilities with other payment arrangements
How does the process of assessing stabilization in participating IMDs compare to the processes used before the demonstration and to processes in nonpsychiatric units of general hospitals?	Document review Key informant interviews Beneficiary interviews Medical records review	ACA Demonstration Requirement for “Stabilization Review”

Use of Information Technology

States and facilities will submit quantitative administrative data electronically through secure file-transfer programs or encrypted CD-ROMs. These means are necessary to ensure the security of the data in transit.

Digital audio recording of all interviews (with respondents’ permission) will be the primary electronic method for ensuring the completeness and quality of interview data. Recording also enhances efficiency and reduces respondent burden by allowing researchers to review and edit their written or typed notes without calling respondents for clarification or to check quotes.

Obtaining high quality data through semistructured interviews requires flexible exchange and conversational rapport between interviewer and respondent. Although information technology can greatly enhance the smooth administration of large-scale surveys with complex skip patterns, in qualitative interviewing, it is often best to avoid complex skip patterns in the first place. For this data collection, the contractor will minimize the skip patterns an interviewer must navigate during interviews by customizing the protocols in advance. The interview protocols accompanying this package have been customized for five types of respondents: state demonstration project directors; staff of IMDs, general hospitals, and ERs; and beneficiaries. In addition, the site visit teams will be led by trained, experienced interviewers. The interviewers will be thoroughly familiar with protocol content so they can readily move back and forth within the protocol without disrupting the conversational flow or asking questions the respondent has already answered.

After information collection, researchers will use Atlas.ti, an electronic software program that enables systematic coding and retrieval of textual data according to a specified scheme.

Data gathered from medical records will be entered into a data entry program called Viking. These data will then be exported as SAS files for analysis at the facility-level.

Duplication of Efforts

This information collection does not duplicate any other effort and the information cannot be obtained from any other source.

Because the data to be collected are highly specific to the demonstration, no other relevant data collection effort currently exists. Semistructured interviews will be used only to collect evaluation information that cannot be obtained from other sources. Where possible, to address its research questions, CMS will use existing administrative data and secondary data sources, such as states’ demonstration payment data submitted to CMS; Medicaid and Medicare enrollment, claims, and encounter data; and administrative data housed in state data warehouses.

Small Businesses

Many, if not all, of the facilities participating in the demonstration may be small businesses or entities. The site visit and interview protocols have been designed with an effort to minimize burden on these entities. Every effort will be made to schedule site visits and interviews at the convenience of these respondents. Evaluation staff will ensure that visits to each facility last no more than one day. The information being requested will be held to the minimum required for the intended use.

Less Frequent Collection

If this information is not collected, CMS will not be able to address the ACA-mandated evaluation areas or have a complete and objective understanding of the impacts of the demonstration on quality of care and the lives of beneficiaries.

Qualitative data will be collected through site visits and interviews twice during the evaluation, in spring 2014 through summer 2014 and spring 2015 through summer 2015. Implementation of a complex demonstration project like the MEPD at multiple sites necessarily faced obstacles; some sites were as much as four months late in implementing it. Therefore, the first round of qualitative data collection will occur after all sites are fully operational but early enough to gather reliable data about practices prior to the implementation of the demonstration. Operational refinements at the state level are likely as the demonstration progresses; therefore, the second round of data collection is scheduled to capture these changes and ensure a complete, nuanced evaluation. Not collecting the information at all would seriously impede CMS’s ability to answer the questions mandated by the ACA, particularly those regarding discharge planning.

Special Circumstances

This request fully complies with the regulations. There are no special circumstances associated with this information collection.

Federal Register/Outside Consultation

Federal Register Notice

The 60-day notice to solicit public comments was published in the Federal Register on July 26, 2013, vol. 78, No. 144, pp. 45205-45208 (Attachment J). Public comments were received from two individuals. These comments and our responses are summarized in Attachment K.

Consultation Outside the Agency

CMS’s evaluation contractor presented an overview of the evaluation plans, including timelines and requirements regarding data collection, to the state demonstration project directors on October 24, 2012. The contractor also established a nine-member technical expert panel (TEP) composed of representatives of IMDs, including those involved in the demonstration; consumers; and other individuals who regularly work with emergency rooms, community mental health data and systems, and state mental health and Medicaid authorities. Attachment G lists the members of the TEP and their professional affiliations. On January 16, 2013, the contractor held an initial meeting with the TEP via webinar to obtain their feedback on evaluation plans, including the medical record review protocols and beneficiary interview questions. TEP members have also agreed to be available throughout the project for individual consultation on design, measurement, and analytic challenges.

Payments/Gifts to Respondents

The TEP recommended that incentives be offered to beneficiaries to participate in the beneficiary interviews in order to obtain an unbiased sample; otherwise, the TEP suggested, only beneficiaries who are particularly unhappy with the process are likely to participate. The evaluation team will, therefore, provide a $20 incentive in the form of a check for each beneficiary interview. A $300,000 incentive pool has also been established for distribution among ERs and general hospitals to offset the burden of participating in the evaluation. These facilities are not formal participants in the demonstration in most states and, therefore, may have little incentive otherwise to assist with the evaluation. Incentive payments will be offered on an as-needed basis to ensure necessary cooperation. The incentive payment to be provided to each facility will vary, depending on 1) the state demonstration project director’s description of the demonstration’s formal and informal relationships with the facility and his/her assessment of the need for incentive payment, 2) the specific amount of burden entailed for each facility, given variations in their systems, along with facility leadership’s expression of need for compensation, concern with burden on staff time, and/or reluctance to participate, and 3) availability of funds based on the number of facilities requiring incentive payment and overall project resources. Based on preliminary conversations with state project directors, we anticipate that thirty to forty ERs and general hospitals will be offered incentive payments of $5,000 to $10,000 each.

Confidentiality

Individuals and organizations will be told the purposes for which the information they provide is collected and advised that any identifiable information provided by them will not be used or disclosed for any other purpose. The evaluation contractor will comply with CMS privacy guidelines pertaining to personally identifiable information. Mathematica’s Institutional Review Board (IRB), The New England IRB, determined that this evaluation is exempt from IRB review under the category “Research and demonstration projects that are conducted by or subject to the approval of Federal Department or Agency heads, and which are designed to study or evaluate public benefits or services.” If required by individual participating state or local governments or facilities, internal review board approval will be obtained before conducting site visits and/or interviews.

Key informant interviews, including both telephone interviews with state demonstration project directors and interviews with facility staff during site visits, will discuss the procedures utilized in implementing the demonstration and results. Responses are not seen as containing private information, but they will be aggregated to the extent possible so individual answers will not be identifiable. Individual responses may be inferred from individual state profiles and case study narratives, however, because of the limited number of respondents interviewed per state and facility (for example, there is only one project director per state). For each respondent, name, professional affiliation, and title will be collected, but Social Security numbers, home contact information, and similar information that can directly identify the respondent will not be collected.

Participants in beneficiary interviews will be advised that their responses will be kept private and secure to the extent permitted by law. Respondents will be given this assurance during recruitment (see Attachment E, Beneficiary Interview Consent Form and Recruitment Script) and again immediately before their interview. Further, they will receive assurance that the information being gathered is for evaluation purposes only. Name, contact information, and other identifying information will be requested only as needed to contact the individual for the interview and to deliver the incentive payment. Comments made during the interview will not be linked to individual beneficiaries.

During the informed consent process and prior to the interview, all interview respondents will be asked if they give permission to have the conversation audio recorded solely for the purpose of filling in any gaps in the research notes. Only the research team will have access to the recording. The beneficiary will be informed that they may request to listen to the audiotape. The audiotape will be destroyed after the contents are transcribed no later than 90 days after the interview. If the respondent does not wish to have the interview audiotaped, the interviewer will take notes instead. The transcription and interview notes will be maintained in a secure study-specific electronic folder that only a minimum number of research staff members may access.

To maintain patient privacy and security in the medical record reviews, the evaluation contractor will use a unique numbering system to identify patients in the sample. The contractor number will indicate the state, type of facility (IMD, ER, or general hospital), and a two-digit suffix unique to the patient. IMD patients discharged 30–60 days prior to the site visit will be identified by suffixes between 21 and 29, and IMD patients discharged 30–60 days prior to the demonstration by suffixes between 31 and 39. Patients discharged from an ER 30–60 days prior to the site visit will be identified by suffixes between 41 and 49; patients discharged from an ER 30–60 days prior to the demonstration will be given suffixes between 51 and 59; general hospital patients discharged 30–60 days prior to the site visit will have suffixes between 61 and 69; and general hospital patients’ discharged 30–60 days prior to the demonstration will have suffixes between71-79. Site visitors will receive several prenumbered sample labels for each patient sampled. Site visitors will attach a label to the applicable roster next to the patient’s name and will enter the number in the record review data collection protocol. The facility points of contact will be asked to keep the labeled rosters for six months after the site visit in case questions arise regarding the record review after the site visit is completed.

Data from both the medical record review and beneficiary interviews will be kept private and secure to the extent permitted by law. The evaluation contractor, Mathematica Policy Research, has established data security plans for the handling of all personally identifiable information, including administrative data obtained from CMS, states, and facilities; interview notes, audiotapes, coded interview data, and data processing for the interviews; and medical records abstractions. These plans meet the requirements of U.S. federal government agencies and are continually reviewed for compliance with new government requirements and data collection needs. Such security is based on (1) exacting company policy promulgated by the highest corporate officers in consultation with systems staff and outside consultants, (2) a secure systems infrastructure that is continually monitored and evaluated with respect to security risks, and (3) secure work practices of an informed staff who take all necessary precautions when dealing with private data. All employees also sign a general confidentiality pledge, included as Attachment H. During site visits, evaluation researchers will at all times keep notebooks and laptop computers on their persons or in secure, locked locations. Private data are kept in study-specific folders that only a minimum number of staff members may access. All typed or electronically coded qualitative data are periodically backed up and preserved on secure media.

Sensitive Questions

Given the nature of the demonstration and its evaluation, beneficiary interview questions of a sensitive nature concerning the individual’s psychiatric condition, his or her recent and past psychiatric emergencies, and details of medical treatment, are unavoidable. This information is at the center of the qualitative data and is necessary to conduct the evaluation. Beneficiaries will be advised of the nature of these questions in advance of the interview and informed that their participation is strictly voluntary. Beneficiaries will also have the option of declining to answer specific questions without opting out of the interview as a whole; incentive payments will not be affected by choosing not to answer particular questions. In the event that a beneficiary becomes upset during the interview, the interviewer will pause and let them collect their thoughts. The interviewer will ask the beneficiary if they are okay and if they would like to continue, or if they would prefer a callback at another time. If the interviewer determines that the beneficiary is a danger to him/herself (i.e., the beneficiary expresses a plan to harm him/herself or others) the interviewer will stop the interview and give the beneficiary the phone number for the crisis hotline. All privacy and security procedures described in the prior section will apply to the sensitive information collected. Solicitation of sensitive information will be limited to only that needed for evaluation purposes.

Burden Estimates (Hours & Wages)

Table 2, below, shows the estimated burden hours and costs for the respondents’ time to participate in this evaluation. All 12 states will be visited twice. Each site visit will consist of a visit to each participating IMD and, for each IMD, to one ER that refers patients to the IMD and one general hospital that boards patients with psychiatric emergencies in nonpsychiatric general medical units when no psychiatric beds are available. On average, site-visit teams will conduct four 60-minute interviews each day at each facility, with one respondent per interview. Site-visit teams will also conduct medical record reviews of 10 medical records at each IMD, referring ER, and general hospital they visit. In addition to the site visits, estimates are provided for the associated project director and beneficiary telephone interviews, site-visit planning time, assistance with gathering documents to be reviewed, and submitting and assisting the evaluation contractor to understand needed state and facility administrative data.

The total burden for this evaluation is estimated to be 2,613 hours, and the total cost burden is estimated to be $111,706.

Burden hour estimates are based on prior experience of the evaluation contractor with evaluations of a similar nature. Throughout the information collection process, the contractor will monitor the length of the interviews, comments received from participants and field interviewers, and the number of individuals who refuse to be interviewed. If this information indicates that the burden on participants is so great as to undermine the collection of high quality data, procedures will be revised accordingly. For example, the number of questions asked during interviews may be reduced. If procedures require revision, the CMS will seek OMB approval to implement specific changes.

Average hourly wages were drawn from the May 2011 National Occupational Employment and Wage Estimates, United States, as reported by the Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes_nat.htm, accessed February 18, 2013). Psychiatrist, social worker, registered nurse, and administrative assistant rates are the average wages for these positions, respectively. Facility administrator rates were estimated based on wages for chief executive officers; the rate for counselors was based on the average of wages for social workers and psychologists, for project directors it was based on general and operations managers, and for data analysts, it was based on computer programmer wages. The majority of Medicaid beneficiaries to be interviewed are likely to be unemployed; therefore, the beneficiary rate is based on the federal minimum wage.

Capital Costs

There are no capital costs.

Table 2. Estimated Total Burden Hours and Cost Over Three Year

Data Collection Activity/ Respondent Type	Number of States and/or Facilities	Number of Respondents per State and/or Facility	Frequency of Response1	Number of Responses	Average Burden Per Response in Hours	Total Burden Hours	Average Hourly Wage	Total Cost Burden
Site Visit Planning—Facility Administrator	81 (27 IMDs, 27 general hospitals, and 27 ERs)	1	2	162	2	324	$84.88	$27,501
Site Visit Interview—Facility Administrator	81 (27 IMDs, 27 general hospitals, and 27 ERs)	1	2	162	1	162	$84.88	$13,751
Site Visit Interview—Psychiatrist	81 (27 IMDs, 27 general hospitals, and 27 ERs)	1	2	162	1	162	$83.73	$13,564
Site Visit Interview—Counselor (e.g., social worker, psychologist)	81 (27 IMDs, 27 general hospitals, and 27 ERs)	1	2	162	1	162	$32.78	$5,310
Site Visit Interview—Registered Nurse	81 (27 IMDs, 27 general hospitals, and 27 ERs)	1	2	162	1	162	$33.23	$5,383
Medical records assistance—Registered Nurse	81 (27 IMDs, 27 general hospitals, and 27 ERs)	1	2	162	3	486	$33.23	$16,150
Obtaining beneficiary consent to be called by evaluation staff —Social Worker	27 IMDs	25	2	1350	0.2	270	$20.50	$5,535
Telephone Interview—Beneficiary	27 IMDs	5	2	270	1	270	$7.25	$1,958
Telephone Interview—Project Director	12 states	1	2	24	1	24	$55.04	$1,321
Assistance gathering site documents—Administrative Assistant	39 (12 states plus 27 IMDs)	1	2	78	0.5	39	$15.87	$619
Facilitation of administrative data requests—Project Director	12 states	1	1	12	2	24	$55.04	$1,321
Ad-hoc email/phone communication to answer questions about MSIS data—Data Analyst	12 states	1	1	12	2	24	$36.54	$877
Assistance in extracting, sending, and answering questions about state or facility administrative data on IMD admissions and ER boarding—Data Analyst	12 states	1	32	36	14	504	$36.54	$18,416
Totals	93 (12 states, 27 IMDs, 27 general hospitals, and 27 ERs)			2,754		2,613		$111,706

¹The individuals interviewed during the second round of site visits may differ from those interviewed during the first round, but categories of respondents will remain the same. Estimates for conversations regarding MSIS data, which will occur on an ad-hoc basis, are for the total number of hours needed over the course of the three-year evaluation.

²Assistance will be requested on a schedule to be worked out individually with each state and facility from which data are needed. Average burden per response is an estimate of time needed for this assistance during each of the three years of the evaluation.

Cost to Federal Government

Table 3 shows the total and annualized cost for this evaluation. The total cost to the federal government of the entire evaluation contract is $5,468,458 (including a base period and three option periods); the annualized cost is $1,367,114 per year. These costs will be incurred from September 2012 through September 2016.

Table 3. Estimated Total and Annualized Cost for Four-Year Evaluation Contract

Cost Component	Total Cost	Annualized Cost
Evaluation Design	$231,159	$57,790
Data Collection and Analysis	$4,381,091	$1,095,273
Synthesis of Project Findings	$178,379	$44,595
Management and Oversight	$677,828	$169,457
Total	$5,468,458	$1,367,115

Changes to Burden

Based on changes made as a result of the pilot test, the burden estimate has increased since the 60-day Federal Register notice was published, by a total of 567 hours, from a total of 2,046 hours to 2,613 hours. Correspondingly, the estimated cost burden has increased by a total of $17,656, from a total of $94,050 to $111,706. The increase results from three sources.

The largest increase is due to the pilot test’s demonstration of the need for facility staff to assist the evaluation contractor in finding information in the medical records. To reflect this need, we have added 2.5 hours of staff time to assist with the medical record reviews at each of the 81 facilities that we visit in each of the two rounds of site visits, for a total increase in burden of 405 hours, at a cost of $13,458.

A further increase in burden is due to the pilot test’s demonstration of the need to obtain additional informed consents for beneficiary interviews from IMD staff. We originally asked for only 10 consents in hopes of interviewing 5 beneficiaries from each IMD. Difficulties in reaching many of the beneficiaries in the pilot test, however, revealed the need to obtain additional consents in order to complete five interviews. Therefore, we will now ask staff of each of the 27 IMDs to obtain 25 consents for each of the two rounds of site visits. The IMD visited during the pilot test indicated that obtaining the consents was easy to do and that obtaining 25 consents would not be a burden to them. The additional consents will require a total of 162 additional burden hours, at a cost of $3,321.

The remaining $877 increase in the burden costs is due to correction of a calculation error in the original submission.

Publication/Tabulation Dates

CMS expects the site visits to begin in the spring of 2014, pending OMB clearance. CMS’s evaluation contractor will synthesize the interview data for inclusion in annual reports as well as a final evaluation report. These reports will integrate qualitative data from the site visits with quantitative data. The reports will be released to the public only after they have been cleared for release by CMS. The evaluation contractor will also develop interactive webinar presentations for key stakeholders that present cross-cutting analyses of integrated quantitative and qualitative data and provide opportunities for discussion. Webinars will only be scheduled and conducted upon approval from CMS. Table 4 presents the anticipated data collection, analysis, and reporting schedule.

Table 4. Schedule of Proposed Data Collection, Analysis, and Reporting

Task	Dates
Qualitative Data Collection and Analysis
Round 1 Site Visits*
Review state documents prior to round 1 site visits	Jan.—March 2014
Plan round 1 site visits in collaboration with states and IMDs	Jan.—March 2014
Conduct state calls prior to round 1 site visits	Jan.—Feb. 2014
Conduct round 1 site visits	March–June 2014
Conduct beneficiary interviews after round 1 site visits	April–July 2014
Analyze qualitative data from round 1 site visits	April–July 2014
Round 2 Site Visits
Review state documents prior to round 2 site visits	Jan.—March 2015
Plan round 2 site visits in collaboration with states and IMDs	Jan.—March 2015
Conduct state calls prior to round 2 site visits	Jan.—Feb. 2015
Conduct round 2 site visits	March–-June 2015
Conduct beneficiary interviews after round 2 site visits	April–July 2015
Analyze qualitative data from round 2 site visits	April–July 2015
Quantitative Data Collection and Analysis
Process Medicaid and Medicare data	Mar. 2013–Nov. 2015
Obtain and process state data	Sept. 2013–Nov. 2015
Analyze demonstration year 1 data	Mar.–May 2014
Analyze demonstration year 2 data	Mar.–May 2015
Analyze demonstration year 3 data	Mar.–May 2016
Evaluation Reports
First annual report	Aug. 29, 2014
Second annual report	Aug. 28, 2015
Final report	Sept. 2, 2016
Webinar presentations	Sept. 2014–Aug. 2016

*The first round of site visits will not begin until the data collection has been approved in accordance with the Paperwork Reduction Act.

Data described in this clearance package will be analyzed to address the research questions described in Section 1. The first round of site visits and interviews will focus on admission, stabilization, and discharge-planning procedures before and after the demonstration. The second round of site visits will take place a few months before the end of the demonstration to allow the evaluation contractor to gather detailed information on lessons learned, changes in quality of care, and sustainability. Administrative data submitted by the states and facilities will be used to supplement Medicaid and Medicare data in analyses of inpatient admissions, emergency services, and costs.

As noted above, notes from all interviews and document reviews will be typed, uploaded to Atlas.ti, and coded according to a specified scheme. Analysis of the site visit and interview data will emphasize policies and procedures that are critical to the implementation of the demonstration, including psychiatric EMC determination, admissions, stabilization assessment, stabilization, and discharge planning. The analysis will include identification of themes within and across states. Throughout the process of gathering, reviewing, and analyzing qualitative data, quotations will be noted that capture a point of view or an experience particularly well. For each project, findings from the implementation analysis will be used to interpret findings about outcomes and to help establish a basis for causal inference. In brief, the interview data collected under this clearance package, when combined with impact analyses using quantitative administrative data, will fully address the critical aspects of the demonstration, as mandated by the ACA.

Expiration Date

This collection does not lend itself to the displaying of an expiration date.

Part B

Supporting Statement

Collection of Information Employing Statistical Methods

Respondent Universe and Sampling Methods

The information collected under this request is not based on probability samples and may not be generalizable beyond the states included in the demonstration. Interview subjects and medical records to be reviewed are selected purposively and fall into the following categories:

• State demonstration project directors (one telephone call for each of the universe of 12 states). Calls with the project directors will provide an efficient means for collecting information on each state’s Medicaid program, mental health delivery system, implementation procedures, and demonstration successes and challenges. As the individuals most involved in project design and oversight, state project directors will provide insight into the implementation of demonstration projects and relevant contextual factors, and may identify lessons and implications as to the broad application and sustainability of projects.

• Key informants from each facility to which site visits are made (up to four in-person interviews at each facility during site visits). Site visits will be made to all 27 IMDs participating in the demonstration, as well as to one ER that refers patients to each IMD and, for each IMD, one general hospital that admits patients with psychiatric EMCs to nonpsychiatric general medical units (that is, scatter beds) when beds in the IMD are not available. ERs and general hospitals will be selected on the basis of a review of state demonstration operational plans and conversations with state project directors and IMD staff about recommended facilities. Priority will be given to facilities that are active participants in the demonstration and that have the largest expected impact on or from the demonstration. For example, if the majority of demonstration referrals to an IMD are made from one particular ER, that ER would be solicited for the site visit; likewise, general hospitals with particularly high use of scatter beds prior to the demonstration would be prioritized because the demonstration aims to alleviate the need for scatter bed use. Because of the need to understand how the demonstration affects the use of scatter beds, general hospitals selected may or may not operate acute inpatient psychiatric units. The demonstration does not alter Medicaid reimbursement for care provided in general hospital psychiatric units, nor does it aim to divert patients from them or change the care they provide; therefore, we do not expect to see significant changes in these units as a result of the demonstration. Funds will be available to provide incentives on an as-needed basis to encourage selected facilities to participate in the site visits. Interview respondents at each selected facility will include administrators and direct care staff from each site who are involved in facility operations or who provide direct care to demonstration participants and can provide information on factors associated with implementation and outcomes. Administrators may include the chief executive officer, chief nursing officer, or other senior managers. Direct care providers may include psychiatrists, registered nurses, and counselors such as social workers and psychologists. Administrators and direct care providers are important interviewees because they will provide insight into changes in access to and quality of care due to the demonstration.

• Beneficiaries (five receiving inpatient services from each IMD through the demonstration, to be interviewed by telephone following their discharge from the hospital). Beneficiary interviews will be essential to understanding patients’ experiences with the admission and discharge process—for example, the amount of time spent waiting for admission, their level of involvement in discharge planning, and how waiting time and participation in discharge planning compare with previous hospitalizations for psychiatric emergencies. Beneficiary viewpoints are critical to understanding if and how quality of care improves as a result of the demonstration. IMD staff will be asked to solicit demonstration participants as they are being discharged, to procure their consent to be contacted by the evaluation team. All demonstration participants who are discharged 21 days before the date of the site visit will be asked to participate until 25 have agreed. Patients will be asked whether they would be willing to speak with a member of the evaluation team about their admission and discharge experiences; if they agree to speak with the evaluation team, the IMD staff member will document contact information for each patient and obtain signed consent. The consent form will include a discussion of the use of an audio recording during the interview. The IMD staff will inform patients that they will be selected randomly for an interview; that is, signing the consent form does not guarantee that he or she will be called for an interview. Due to logistical complexities, patients discharged to forensic facilities will not be interviewed. For patients assigned legal guardians for decision-making purposes, IMD staff will solicit consent and contact information from both the guardian and the patient. IMD staff will inform patients that, if selected, they will receive a $20 check from the evaluation staff for participating in the interview. Across all states, the demonstration is expected to enroll hundreds to thousands of participants. The 270 beneficiaries selected for interviews over two rounds of site visits will be selected on the basis of proximity of their discharge dates to the timing of the site visit. Provision of incentives will help to encourage participants with a range of experiences to participate, thereby helping to reduce the potential for bias if only patients with negative experiences were to respond. Patients with more positive relationships with the IMD staff soliciting their participation may be more likely to agree to participate. Due to logistics regarding locating and connecting with individuals for interviews, patients with more positive discharge experiences (such as those discharged to stable homes in the community rather than to forensic units, homeless shelters, or other types of institutional care) may be more likely to participate. Despite these potential sources of bias, the beneficiary interviews provide an important cross-validation of information about implementation procedures provided by medical record reviews and participating facility and demonstration staff, each of which is subject to its own unique biases.

• Medical records review (10 medical records reviewed at each of 27 IMDs, 27 EDs, and 27 general hospitals during site visits). Medical records review will cross-validate and provide a more detailed understanding of stabilization assessment and discharge-planning procedures, interventions administered to achieve stabilization, length of time spent in the ER, procedures for determining and documenting the existence of qualifying psychiatric EMCs, and inpatient referral procedures. Medical records are important for determining whether the demonstration was implemented as intended, and will facilitate identification of operational lessons learned. Sampling procedures for medical records to be reviewed are described in detail in Attachment C.

Procedures for the Collection of Information

CMS’s evaluation contractor will use a systematic qualitative data collection approach that will draw from multiple sources including telephone interviews, document review, beneficiary telephone interviews, and site visits, including in-person interviews and medical records review.

CMS’s evaluation contractor will conduct two rounds of site visits during the evaluation period. Pending clearance, the first round will take place about 24 months after the start of the demonstration (spring 2014 through summer 2014) and will focus on admission, stabilization, and discharge-planning procedures before and after the demonstration. The timing of the visits will ensure that states have sufficient time to respond to unforeseen implementation challenges, and that project procedures operate consistently. The second round will take place a few months before the end of the demonstration (spring 2015 through summer 2015) to allow the evaluation contractor to gather detailed information on lessons learned, changes in access and quality of care, and sustainability. The length of each visit will vary based on the number of IMDs involved in each state’s demonstration project. Table 5 details the proposed site visit structure and plans for data collection at each facility.

Table 5. Site Visit Structure and Data Collection

State/Number of Participating IMDs	Days On Site	Site Visit Structure and Data Collection
Alabama (4) California (4)	6	Structure: The evaluation team will spend one day at each participating IMD, one day at the IMD’s primary ER referral source, and one day at a general hospital that admits patients experiencing a psychiatric emergency to a nonpsychiatric unit when beds are not available in a psychiatric unit or IMD. Because of the need to understand how the demonstration effects the use of scatter beds, the general hospital selected may or may not operate an acute inpatient psychiatric unit. Key Informant Interviews: On average, teams will conduct four 60-minute interviews at each facility. Medical Record Reviews: Teams will review 10 medical records at each facility: IMD, the IMD’s primary ER referral source, and the general hospital that admits patients experiencing a psychiatric emergency when beds are not available in a psychiatric unit or IMD.
Maryland (3) Missouri (3) Washington (3)	5
Illinois (2) Maine (2) West Virginia (2)	3
Connecticut (1) District of Columbia (1) North Carolina (1) Rhode Island (1)	3

Note: A four-person team will conduct site visits that involve more than one participating IMD. A two-person team will conduct site visits to states with only one participating IMD.

• State project directors will be interviewed by telephone prior to each round of site visits. One-hour interviews will focus on identifying any changes in the state’s role in administering the demonstration and the associated costs, evolving contextual factors affecting psychiatric emergency and inpatient care in the state, and implementation facilitators and challenges. The evaluation team will also review with each project director the state-specific logic model they developed based on information gathered from document review during the evaluation design phase. The evaluation team will use a standardized set of questions to guide conversations (Attachment B).

• Direct care staff and administrators from IMDs, ERs, and general hospitals will be interviewed in person once during each round of site visits. Each interview will last 60 minutes. Semistructured interview guides will indicate the type of information to be collected but will allow for flexibility across sites in terms of respondents, topics, and questions asked; this flexibility is critically important given the significant variation in demonstration projects across states. Attachment B includes a list of the interview questions to be asked of direct care providers and administrators from IMDs, ERs, and general hospitals.

• Beneficiary telephone interviews will be conducted with five demonstration participants discharged from each IMD after each round of site visits, for a total of 135 interviews. Attachment E details the interview questions for beneficiaries and includes the consent form for beneficiaries and the script IMD staff will use to invite beneficiaries to participate in the interviews.

• Ten medical records will be selected at each facility (IMD, ER, and general hospital), using purposive sampling. Attachment C details the sampling procedures. This technique will enable site visitors to identify records for patients with a wide range of characteristics of interest, such as high-risk behaviors requiring chemical or physical restraint, medical comorbidities, or frequent admissions. Direct care staff at each facility will be asked to assist site visitors in finding information needed for the evaluation within the medical records; this should take approximately 3 hours for staff at each facility. Using a structured tool (Attachment D), the evaluation contractor will abstract from:

• IMD records, information on stabilization assessment procedures, discharge-planning procedures, and interventions administered

• ER records, information on length of time in the ER, EMTALA status determination, interventions administered, and inpatient referral procedures

• General hospital records, information on interventions administered, stabilization assessment procedures, and discharge-planning procedures.

To ensure effective coordination with respondents, the evaluation contractor will use a systematic approach to communicating and coordinating with IMDs, ERs, and general hospitals. Table 6 details the sequence of events. Approximately three months before the scheduled site visit, the contractor will send an email to the demonstration project director and point of contact for each IMD. The email will describe site-visit activities, identify the approximate time frame for the visit, and request a date for a planning meeting via telephone to discuss the logistics of the site visit and all pre-visit activities. During the planning meeting with the IMD point of contact, the evaluation contractor will discuss the schedule for the site visit to the IMD (for example, length of interviews with four key informants and time needed for an overview of medical records) and identify a point of contact for a referring ER and a general hospital that boards patients with psychiatric emergencies in nonpsychiatric general medical units when no beds are available in IMDs. The contractor will inform the IMD contact that, on the first day of the site visit, the review team will request two lists of patients from which medical records will be selected for review.

After the planning meetings with the IMDs are completed, the evaluation contractor will contact the points of contact at the ERs and general hospitals to discuss site-visit activities and to schedule interviews with four staff members and time for medical record reviews. For example, the contractor will ask the ER contacts to provide the team with lists of patients from which medical records will be selected for review.

Table 6. Site-Visit Planning Protocol

Weeks Before Site Visit	Scheduling Activity	Purpose of Activity
12	Send email to demonstration project directors and IMD points of contact (POC)	Provide overview of site-visit activities Propose site-visit dates Propose planning meeting telephone call date(s) with IMD POC(s) during week 10
11	Send follow-up email to demonstration project directors and IMD POCs	Confirm site-visit dates Confirm IMD POC planning meeting call dates and times
9–10	Call IMD POCs to plan site visit	Review site-visit logistics Discuss site-visit activities and schedule, including staff interviews and medical record reviews Request two patient rosters for medical record reviews Request assistance from IMD staff for beneficiary recruitment for interviews Request IMD documents Identify and obtain contact information for ER and general hospital POCs
9	Send email to ER and general hospital POCs	Provide overview of site-visit activities Propose site-visit planning meeting telephone call date(s) and time(s)
9	Send follow-up email to ER and general hospital POCs	Confirm telephone meeting dates and times
7–8	Call ER and general hospital POCs to plan site visit	Review site visit logistics Discuss site visit activities and schedule, including staff interviews and medical record reviews Request two patient rosters from ER POC Request one patient roster from general hospital POC
1–2	Follow up by telephone with IMD, ER, and general hospital POCs	Confirm any information that might have changed Provide site-visit team’s names and contact information Remind POCs about rosters needed from their location Review site-visit logistics one final time

Quality Control Procedures. Customized, comprehensive training is vital for uniform, consistently high quality qualitative data collection. The evaluation contractor will conduct two training sessions in association with each round of telephone interviews and site visits.

The training sessions will review the semistructured interview guides, the medical record review tool, the beneficiary interview guide, and the data coding scheme. The site-visit teams will practice using the medical record review tool, role-play interviews, and discuss how to respond to unexpected events while on site. The training will promote reliability in use of the protocols and will ensure that each contractor staff member shares a common understanding of the goals of the site visits.

After the first site visit, contractor staff will meet to discuss any changes required to the interview guides or medical record review tool, with revisions made as needed. Further, contractor staff will meet after the site visit to review findings and to identify any information that requires further calls with the site. Once all site visits are complete, the evaluation contractor will train teams to code qualitative data using Atlas.ti software. The contractor will follow a thematic coding scheme to be developed by the qualitative research experts (Attachment I).

The site visit team’s lead will ensure quality and consistency of data collection during the site visits by conducting reliability assessments to ensure consistent implementation of the review procedures and accuracy of data collection across team members. At the end of the site visit, the team’s lead will review all data collection protocols for missing or inconsistent data.

Methods to Maximize Response Rates and Deal with Nonresponse

The interview and medical record data collection is not based on probability samples and is not meant to represent anyone other than the respondents. Therefore, a response rate does not apply to these activities. However, in selecting states to participate in the demonstration, CMS stipulated that states cooperate fully in the cross-state demonstration evaluation. Given this, and the evaluation contractor’s experience conducting other process evaluations, CMS expects a high level of participation from state demonstration personnel and facility administrators and direct care providers. A $300,000 incentive pool has also been established for distribution among ERs and general hospitals to offset the burden of participating in the evaluation. These facilities are not formal participants in the demonstration in most states and, therefore, may have little incentive otherwise to assist with the evaluation. Incentive payments will be offered on an as-needed basis to ensure necessary cooperation. The incentive payment to be provided to each facility will vary, depending on 1) the state demonstration project director’s description of the demonstration’s formal and informal relationships with the facility and his/her assessment of the need for incentive payment, 2) the specific amount of burden entailed for each facility, given variations in their systems, along with facility leadership’s expression of need for compensation, concern with burden on staff time, and/or reluctance to participate, 3) availability of equivalent alternative facilities that may be more willing and able to participate; and 4) availability of funds based on the number of facilities requiring incentive payment and overall project resources. Based on preliminary conversations with state project directors, we anticipate that thirty to forty ERs and general hospitals will be offered incentive payments of $5,000 to $10,000 each. To further ensure the cooperation of respondents, contractor staff will attempt to minimize individual burden and develop interview schedules that respect site constraints and pressures.

• Minimize individual burden. Willingness of respondents to participate in in-person interviews may hinge on the time these meetings require. To minimize the burden, guides are designed to gather information that is as complete as possible in as little time as possible. The evaluation contractor has developed separate discussion guides for each respondent type so that respondents are not asked about activities or issues that are not applicable to them. In addition, interviewers will meet with interview respondents in person in their own offices or at a location of their choice. Telephone interviews with facility staff will be scheduled at a time that is convenient for the respondent, and respondents will be provided with the interview questions in advance to allow them to prepare if they so desire.

• Develop interview schedules that respect site constraints and pressures. The contractor will work with each site to determine logistics and a schedule for the in-person interviews. The schedule will avoid conflict with other activities and allow individuals to find time in their calendars to spend with contractor staff.

Although CMS expects a high degree of participation from all respondent types, direct care providers may be less readily available for in-person interviews than other respondent types. The evaluation contractor will offer additional accommodations to this respondent type to increase the likelihood of their participation. They will offer to meet with direct care providers outside of clinical hours, restrict the interview to 30 minutes if 60 minutes is not acceptable, and conduct the interview by telephone if the respondent says that would be more convenient. If only a 30-minute interview is possible, we will review the protocol and select questions for which we feel their input would be most valuable, such as questions that best address the interviewee’s area of expertise, those about which we have the least information from other sources, or those for which the direct care provider perspective is critical. Research reports will note any important gaps in our understanding of the direct care provider perspective due to shortened interviews.

To encourage participation of beneficiaries in the interviews, an incentive payment of $20 in the form of a check will be paid for each interview. This will help to encourage patients with a range of experiences to participate, thereby helping to reduce potential biases if only patients with negative experiences were to respond. IMD staff will be asked to solicit agreement from patients for them to be contacted by the evaluation team as well as their contact information, immediately prior to discharge. Obtaining contact information at this point will greatly facilitate the ability to locate discharged demonstration participants; being asked by hospital staff with whom they are familiar might encourage participation. IMD staff will be asked to obtain consent to be contacted from 25 patients discharged within 21 days of the site visit; of these 25, only 5 will be randomly selected to be interviewed. Beneficiaries who cannot be located or who chose not to be interviewed when contacted by the evaluation team will be replaced from among the remaining pool of those providing initial consent at discharge. The proximity of the interviews to the respondents’ hospital discharge dates will facilitate the evaluation team’s ability to locate potential participants and the respondents’ ability to recall details of their recent hospitalization experiences. Interview respondents may choose not to answer specific questions without consequences; the interview notes will record such decisions.

Test of Procedures or Methods to Be Undertaken

The evaluation contractor pilot tested the protocols by conducting a site visit and associated interviews for the Connecticut demonstration project from May 20-22, 2013. Connecticut was selected because of its proximity to the evaluation contractor’s offices and because only one IMD is participating in the state’s demonstration project, which simplified logistical arrangements and allowed the visit and interviews to be completed on an expedited schedule without violating the PRA.

CMS’s objectives during the pilot test were to assess whether (1) planned procedures allow collection of the information needed in the allotted time, (2) respondents can readily understand and answer the interview questions, (3) interviews flow sensibly from topic to topic, and (4) the questions seem to yield thoughtful, candid responses. The pre-tests were also useful for identifying interviewer training needs and considering refinements to site-visit planning procedures (for example, how best to identify facilities to visit, individuals to interview, and medical records to review). Pilot testing also helped confirm the burden estimates.

The site visit protocols attached to this supporting statement (Attachments B - E) directly reflect the pretest results. Below, we summarize changes made in response to lessons learned during the pilot test.

• The medical record review protocol for the IMD took longer than anticipated. To eliminate redundancy and reduce burden on staff, the contractor removed several items and subquestions from the medical record review protocols and simplified and reworded others.

• Facility staff had lists of medical records prepared for site visits as requested, and staff noted that preparing these lists required little effort. Procedures for sampling these records were simplified to reduce burden on facility staff. Rather than asking staff to recall patient characteristics, the evaluation team will now utilize readily available information on diagnostic codes and length of stay to select participants. In addition, the evaluation contractor initially planned to review two open medical records for patients in the IMD but some information for patients currently receiving treatment at the IMD in the pilot site was maintained on the unit rather in the medical records office. To minimize burden on staff working on the unit, the evaluation contractor will now review only closed medical records during the site visit.

• The point of contact for the pilot site reported that requesting IMD staff to recruit beneficiaries to participate in interviews was feasible and that the process was straightforward and did not require much time. The IMD gave the evaluation contractor 12 signed consent forms, even though only 10 were requested. Subsequent to the site visit, however, the contractor had difficulty contacting many of the beneficiaries who provided consent. To obtain enough consents to ensure that that the desired sample size can be achieved, therefore, the contractor will now ask IMDs to gather at least 25 consents.

• The interview questions were generally understood by respondents, but the majority of respondents were not aware that the state is participating in the MEPD. Therefore, questions that refer to changes since the demonstration began were reworded using the date of implementation as the reference point for changes. Respondents did not generally have difficulty answering the interview questions, but some clarifying modifications were made to the interview protocols.

Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

CMS contracted with Mathematic Policy Research to design and conduct the evaluation of the MEPD. Table 7 identifies the individuals at Mathematica involved in designing, overseeing, and analyzing the data.

Mathematica also consulted with a technical expert panel on the methods and data collection procedures used in this project. Attachment G identifies members of the technical expert panel.

Table 7. Individuals Consulted on Statistical Aspects of the Design

Name	Project Role	Email	Phone
Project Management
Crystal Blyler	Project Director	[email protected]	(202) 250-3502
Quantitative Team
Melissa Azur	Team Leader	[email protected]	(202) 250-3518
Jonathan Brown	Senior Researcher/Data Team Leader	[email protected]	(202) 264-3446
Priyanka Anand	Researcher	[email protected]	(202) 552-6401
Jessica Nysenbaum	Research Analyst	[email protected]	(202) 250-3556
Brenda Natzke	Research Analyst	[email protected]	(202) 484-3287
Frank Yoon	Statistician	[email protected]	(202) 554-7518
Tom Bell	Principal Program Analyst	[email protected]	(312) 994-1010
Bryan Bernecker	Senior Programmer	[email protected]	(617) 674-8370
Lucy Lu	Systems Analyst	[email protected]	(202) 554-7578
Qualitative Team
Angela Gerolamo	Team Leader	[email protected]	(609) 945-3345
Jung Kim	Researcher	[email protected]	(609) 936-3253
Rosalind Keith	Researcher	[email protected]	(609) 716-4397
Grace Ferry	Researcher	[email protected]	(202) 250-3571
Benjamin Fischer	Program Analyst	[email protected]	(312) 994-1047
Jennifer McGovern	Survey Specialist	[email protected]	(609) 275-2200
Nikkilyn Morrison	Survey Specialist	[email protected]	(312) 994-1048
Amy Overcash	Research Analyst	[email protected]	(609) 750-2009
Other
Carol Irvin	Senior Advisor/ Quality Assurance	[email protected]	(617) 301-8972
Jim Verdier	Senior Advisor/Quality Assurance	[email protected]	(202) 484-4520
Bonnie O’Day	Senior Researcher/Reports	[email protected]	(202) 264-3455

Attachment A

SECTION 2707 OF THE AFFORDABLE CARE ACT OF 2010

SECTION 2707 OF THE AFFORDABLE CARE ACT OF 2010:

THE MEDICAID EMERGENCY PSYCHIATRIC DEMONSTRATION PROJECT

(a) AUTHORITY TO CONDUCT DEMONSTRATION PROJECT.—The Secretary of Health and Human Services (in this section referred to as the ‘‘Secretary’’) shall establish a demonstration project under which an eligible State (as described in subsection (c)) shall provide payment under the State Medicaid plan under title XIX of the Social Security Act to an institution for mental diseases that is not publicly owned or operated and that is subject to the requirements of section 1867 of the Social Security Act (42 U.S.C. 1395dd) for the provision of medical assistance available under such plan to individuals who—

(1) have attained age 21, but have not attained age 65;

(2) are eligible for medical assistance under such plan; and

(3) require such medical assistance to stabilize an emergency medical condition.

(b) STABILIZATION REVIEW.—A State shall specify in its application described in subsection (c)(1) establish a mechanism for how it will ensure that institutions participating in the demonstration will determine whether or not such individuals have been stabilized (as defined in subsection (h)(5)). This mechanism shall commence before the third day of the inpatient stay. States participating in the demonstration project may manage the provision of services for the stabilization of medical emergency conditions through utilization review, authorization, or management practices, or the application of medical necessity and appropriateness criteria applicable to behavioral health.

(c) ELIGIBLE STATE DEFINED.—

(1) IN GENERAL.—An eligible State is a State that has made an application and has been selected pursuant to paragraphs (2) and (3).

(2) APPLICATION.—A State seeking to participate in the demonstration project under this section shall submit to the Secretary, at such time and in such format as the Secretary requires, an application that includes such information, provisions, and assurances, as the Secretary may require.

(3) SELECTION.—A State shall be determined eligible for the demonstration by the Secretary on a competitive basis among States with applications meeting the requirements of paragraph (1). In selecting State applications for the demonstration project, the Secretary shall seek to achieve an appropriate national balance in the geographic distribution of such projects.

(d) LENGTH OF DEMONSTRATION PROJECT. — The demonstration project established under this section shall be conducted for a period of 3 consecutive years.

(e) LIMITATIONS ON FEDERAL FUNDING.—

(1) APPROPRIATION.—

(A) IN GENERAL.—Out of any funds in the Treasury not otherwise appropriated, there is appropriated to carry out this section, $75,000,000 for fiscal year 2011.

(B) BUDGET AUTHORITY. — Subparagraph (A) constitutes budget authority in advance of appropriations Act and represents the obligation of the Federal Government to provide for the payment of the amounts appropriated under that subparagraph.

(2) 5-YEAR AVAILABILITY.—Funds appropriated under paragraph (1) shall remain available for obligation through December 31, 2015.

(3) LIMITATION ON PAYMENTS. — In no case may—

(A) the aggregate amount of payments made by the Secretary to eligible States under this section exceed $75,000,000; or

(B) payments be provided by the Secretary under this section after December 31, 2015.

(4) FUNDS ALLOCATED TO STATES.—Funds shall be allocated to eligible States on the basis of criteria, including a State’s application and the availability of funds, as determined by the Secretary.

(5) PAYMENTS TO STATES. — The Secretary shall pay to each eligible State, from its allocation under paragraph (4), an amount each quarter equal to the Federal medical assistance percentage of expenditures in the quarter for medical assistance described in subsection (a). As a condition of receiving payment, a State shall collect and report information, as determined necessary by the Secretary, for the purposes of providing Federal oversight and conducting an evaluation under subsection (f)(1).

(f) EVALUATION AND REPORT TO CONGRESS.—

(1) EVALUATION.—The Secretary shall conduct an evaluation of the demonstration project in order to determine the impact on the functioning of the health and mental health service system and on individuals enrolled in the Medicaid program and shall include the following:

(A) An assessment of access to inpatient mental health services under the Medicaid program; average lengths of inpatient stays; and emergency room visits.

(B) An assessment of discharge planning by participating hospitals.

(C) An assessment of the impact of the demonstration project on the costs of the full range of mental health services (including inpatient, emergency and ambulatory care).

(D) An analysis of the percentage of consumers with Medicaid coverage who are admitted to inpatient facilities as a result of the demonstration project as compared to those admitted to these same facilities through other means.

(E) A recommendation regarding whether the demonstration project should be continued after December 31, 2013, and expanded on a national basis.

(2) REPORT. — Not later than December 31, 2013, the Secretary shall submit to Congress and make available to the public a report on the findings of the evaluation under paragraph (1).

(g) WAIVER AUTHORITY.—

(1) IN GENERAL.—The Secretary shall waive the limitation of subdivision (B) following paragraph (28) of section 1905(a) of the Social Security Act (42 U.S.C. 1396d(a)) (relating to limitations on payments for care or services for individuals under 65 years of age who are patients in an institution for mental diseases) for purposes of carrying out the demonstration project under this section.

(2) LIMITED OTHER WAIVER AUTHORITY.—The Secretary may waive other requirements of titles XI and XIX of the Social Security Act (including the requirements of sections 1902(a)(1) (relating to state wideness) and 1902(1)(10)(B) (relating to comparability)) only to extent necessary to carry out the demonstration project under this section.

(h) DEFINITIONS. — In this section:

(1) EMERGENCY MEDICAL CONDITION.—The term ‘‘emergency medical condition’’ means, with respect to an individual, an individual who expresses suicidal or homicidal thoughts or gestures, if determined dangerous to self or others.

(2) FEDERAL MEDICAL ASSISTANCE PERCENTAGE.—The term ‘‘Federal medical assistance percentage’’ has the meaning given that term with respect to a State under section 1905(b) of the Social Security Act (42 U.S.C. 1396d(b)).

(3) INSTITUTION FOR MENTAL DISEASES. — The term ‘‘institution for mental diseases’’ has the meaning given to that term in section 1905(i) of the Social Security Act (42 U.S.C. 1396d(i)).

(4) MEDICAL ASSISTANCE. — The term ‘‘medical assistance’’ has the meaning given that term in section 1905(a) of the Social Security Act (42 U.S.C. 1396d(a)).

(5) STABILIZED.—The term ‘‘stabilized’’ means, with respect to an individual, that the emergency medical condition no longer exists with respect to the individual and the individual is no longer dangerous to self or others.

(6) STATE.—The term ‘‘State’’ has the meaning given that term for purposes of title XIX of the Social Security Act (42 U.S.C. 1396 et seq.).

Attachment b

key informant Interview PROTOCOLs

Medicaid Emergency Psychiatric Demonstration (MEPD)
Interview Guide: MEPD Project Director

Round of Site Visit:

Site Visit Dates:

State:

Date of MEPD Implementation:

Informant(s) Name: [Using notes from the initial interview conducted during fall 2012, insert name, title, and role and responsibilities in the demonstration]

Informant(s) Title:

Informant(s) Contact Information:

Date of Interview:

Time of Interview:

Interviewer:

Note taker:

I. Introduction

Thank you for agreeing to speak with us. As you know, Mathematica Policy Research is evaluating the Medicaid Emergency Psychiatric Demonstration for the Centers for Medicare & Medicaid Services (CMS) through its Center for Medicare and Medicaid Innovation (CMMI). The evaluation will determine whether and to what extent using Medicaid funding to provide care for adults in private institutions for mental disease (IMDs) impacts service use, quality of care, and Medicaid costs.

We are speaking with you to learn about changes in the state’s role in administering the demonstration and associated costs, evolving contextual factors affecting psychiatric emergency and inpatient care in the state, and implementation facilitators and challenges.

We will be taking notes during the interview and would like to audiotape our discussion to ensure that we have captured your comments accurately. The audio recording will not be shared with anyone outside of the project team and will be destroyed at the conclusion of the study. Is this okay with you?

Do you have any questions before we get started?

II. Role and Responsibilities

1. Has your role and responsibilities changed since we last spoke on [insert date of fall 2012 interview]? If so, please describe.

III. Program Design

2. What specific service improvements are being made as part of the demonstration?

3. Please describe your procedures for monitoring the demonstration.

3a. How is this working?

4. What monitoring procedures have been most useful?

5. What suggestions do you have about demonstration monitoring for other states?

IV. Access to Inpatient Psychiatric Care

Next, I’d like to talk about access to care.

6. How does access to inpatient psychiatric care for Medicaid beneficiaries experiencing a psychiatric emergency compare to access for those beneficiaries before the demonstration?

PROBE: Has access to inpatient psychiatric care increased or decreased? Why or why not?

7. Have there been any changes in patient enrollment estimates since we last spoke on [insert date of fall 2012 interview]?

8. If there has been a change in patient enrollment, what accounts for this change?

V. Boarding Time in ER and General Hospital Scatter Beds

I’d like to shift the discussion to boarding in ERs and general hospital scatter beds.

9. Can you discuss the extent of emergency room boarding in the state?

10. Can you discuss the extent of psychiatric boarding in general hospital scatter beds in the state?

11. How does psychiatric boarding time in ERs for patients with psychiatric emergencies compare to boarding times for psychiatric emergencies before the demonstration?

PROBE: Has boarding time increased or decreased? Why?

12. Is this different for Medicaid beneficiaries?

13. How does psychiatric boarding time in GH scatter beds for patients with psychiatric emergencies compare to boarding times for psychiatric emergencies before the demonstration?

PROBE: Has boarding time increased or decreased? Why?

14. Is this different for Medicaid beneficiaries?

VI. Referral and Admission

Next, I’d like to talk about referral and admission, stabilization, and discharge planning.

15. How do you verify that the patients admitted to the demonstration are suicidal, homicidal, or a danger to themselves or others?

16. How do you verify that the participants in the demonstration are enrolled in Medicaid at the time they are admitted to the IMD?

VII. Stabilization [Insert stabilization assessment requirements identified in the operating plan and/or interview notes]

17. How are you ensuring that IMDs are adhering to stabilization assessment requirements?

18. How is this process going?

18a. What is going well?

18b. What would you like to be done differently?

19. How do stabilization criteria in your state differ from the criteria used for the demonstration?

VIII. Length of Stay

20. How does the average length of stay for patients enrolled in the demonstration compare to21 the average length of stay for patients not participating in the demonstration? (e.g., Medicaid beneficiaries with psychiatric emergencies who are admitted to the public IMDs, general hospitals, or alternatives.)

IX. Discharge Planning

21. What kinds of changes, if any, have occurred regarding post-discharge follow up procedures for Medicaid beneficiaries as a result of the demonstration?

22. How are you monitoring discharge planning for demonstration patients and for non-demonstration psychiatric patients at IMDs?

23. Are you experiencing challenges in monitoring discharge planning? If so, please describe.

24. Have IMDs reported challenges to discharge planning? If so, please describe.

25. Under the demonstration, has the proportion of Medicaid beneficiaries with psychiatric emergencies discharged from participating IMDs to community-based residences changed?

PROBE: How has it changed?

PROBE: To where are demonstration patients being discharged most frequently?

X. Cost

26. Can you describe the effect the demonstration has had on costs to the state?

27. How has care provided by private IMDs impacted state Medicaid costs under the demonstration?

28. How have dollars saved by receiving the federal match been invested by the state?

29. What is your perspective on cost-shifting due to the demonstration?

30. What were the administrative costs to fully implement the demonstration (e.g., for staffing or making changes to the physical environment)?

XI. Context

Next, I’d like to talk about the context in which the demonstration is operating.

31. How are psychiatric emergency services provided in the state?

31a. How many psychiatric emergency providers are in the state?

32. Can you discuss the extent to which there is a shortage of inpatient psychiatric beds in the state?

33. How has the demonstration influenced state hospital bed capacity (e.g., crowding, waiting lists)?

34. Can you describe the levels and types of investments the state is making in community-based behavioral health services (e.g., Assertive Community Treatment programs, mobile crisis treatment teams, partial hospitalization programs)?

35. Can you describe the availability of psychiatric step-down and outpatient services in your state?

35a. Are psychiatric step-down and outpatient services reimbursed by Medicaid?

35b. If not, how are these services reimbursed?

36. Have there been any changes in mental health service delivery that could affect the demonstration (e.g., closure of facilities, new IMDs opening)?

37. Is the state involved in other initiatives that could influence the demonstration (e.g., Institute for Behavioral Health Care Improvement Collaborative)?

38. Are you aware of any state-level initiatives that may be changing the incidence of psychiatric emergencies and access to services for patients experiencing a psychiatric emergency?

39. Are there any planned changes in mental health services at the state level that could affect the demonstration (e.g., change in payment structure)?

40. How will the 2014 Medicaid expansion influence the demonstration (e.g., expenditures and population served)?

XII. Outcomes

I’d like to conclude the interview by talking about outcomes of the demonstration.

41. What are your thoughts about potential short-term effects of the demonstration?

42. What do you think are the two most important changes, if any, resulting from the demonstration?

43. What do you hope the demonstration will do?

XIII. Closing

That completes the questions we have for you today.

• Is there anything we should have asked about but didn’t?

• Do you have anything you would like to tell us, or questions you would like to ask us?

Thank you again for taking the time to speak with us. We appreciate and value your input.

Medicaid Emergency Psychiatric Demonstration (MEPD)
Interview Guide: MEPD IMD Staff Member

Round of Site Visit:

Site Visit Dates:

Facility Name:

Facility State:

Date of MEPD Implementation:

Informant(s) Name: [Note if informant is IMD point of contact interviewed in fall 2012.]

Informant(s) Title:

Informant Contact Information:

Date of Interview:

Time of Interview:

Interviewer:

Note taker:

I. Introduction

Thank you for taking the time to speak with us. We are from Mathematica Policy Research, an independent research firm contracted by the Centers for Medicare & Medicaid Services (CMS) through its Center for Medicare and Medicaid Innovation (CMMI) to evaluate the Medicaid Emergency Psychiatric Demonstration. The three-year demonstration allows eligible, private institutions for mental disease (IMDs) in participating states to receive federal Medicaid reimbursement for adults ages 18 to 64. The purpose of the demonstration is to make inpatient care more accessible to adult Medicaid beneficiaries with psychiatric emergency medical conditions. The evaluation will determine whether and to what extent using Medicaid funding to provide care for adults in private IMDs impacts service use, quality of care, and Medicaid costs.

We are speaking with you to learn about how care is provided in [insert name of IMD] In particular; we are interested in understanding how the referral and admission, stabilization and discharge planning processes differ for Medicaid beneficiaries as a result of the demonstration.

We will be taking notes during the interview and would like to audiotape our discussion to ensure that we have captured your comments accurately. The audio recording will not be shared with anyone outside of the project team and will be destroyed at the conclusion of the study. Is this okay with you?

Do you have any questions before we get started?

II. Role and Responsibilities

1. Please describe your role and responsibilities at [insert name of IMD].

2. How long have you been in this role?

3. How long have you worked at [insert name of IMD]?

4. Are you aware that [insert name of IMD] is participating in the Medicaid Emergency Psychiatric Demonstration?

[Interviewer: If informant is not aware of the demonstration, reword all questions referring to the ‘demonstration’ as the ‘date of implementation.’ See example in Q5 below.]

III. Program Design

5. What specific service improvements are being made as part of the demonstration?

[Interviewer: If informant is not aware of the demonstration, reword this question as: What specific service improvements are being made since [insert month, year of demonstration implementation]]?

6. What organizational changes were made to the facility as a result of the demonstration (e.g., staffing changes, changes in staff responsibilities)?

[Interviewer: Ask Q7 and Q8 only if informant is IMD point of contact, a hospital administrator and/or is familiar with the monitoring of the demonstration.]

7. What are your perceptions about the state’s procedures for monitoring the demonstration?

8. What would you change about the state’s monitoring procedures?

IV. Access to Inpatient Psychiatric Care

I would like to discuss access to care.

[Interviewer: If informant is not aware of the demonstration, reword all questions referring to the ‘demonstration’ as the ‘date of implementation.’]

9. How does access to inpatient psychiatric care for Medicaid beneficiaries experiencing a psychiatric emergency compare to access for those beneficiaries before the demonstration?

PROBE: Has access to inpatient psychiatric care increased or decreased? Why or why not?

10. How has the mix of patients in this hospital changed since implementing the demonstration on [insert date of implementation]?

11. [Ask only if informant is aware of the demonstration.] Are you noticing any trends in the participation of a particular sub-group of populations eligible for the demonstration (e.g., trends by age, race, gender, Medicaid eligibility status)? If so, please describe these trends.

12. [Ask only if informant is aware of the demonstration.] Are you having challenges with implementing patient eligibility criteria? If so, please describe these challenges.

[Interviewer: ask Q13 and Q14 only if the informant is the IMD point of contact we spoke with in fall 2012. Contact state lead to obtain patient enrollment estimates if not known.]

13. Have there been any changes in patient enrollment estimates since we last spoke on [insert date]?

14. If there has been a change in patient enrollment estimates, what accounts for this change?

15. [Ask only if informant is aware of the demonstration.] How has bed capacity changed as a result of the demonstration?

PROBE: Has the facility added beds, opened additional units, or started staffing beds that were previously not used?

V. Boarding Time in ER

[Interviewer: If informant is not aware of the demonstration, reword all questions referring to the ‘demonstration’ as ‘date of implementation.’]

Now I’d like to talk about the amount of time patients spend in the ER or intake department prior to admission.

16. Does this hospital have an ER or a place where someone comes (for example, an intake or assessment department) because they are experiencing a psychiatric emergency condition? Is so, please describe. [Obtain during site visit planning.]

17. Before the demonstration, did this facility ever have to board Medicaid patients in the ER or intake/assessment department while awaiting admission to a hospital for psychiatric emergency?

18. Has this changed since the demonstration was implemented in [insert date of implementation]?

19. If this has changed since the demonstration, on average how long does a patient with a psychiatric emergency currently wait in the ER or intake/assessment department once it has been decided that psychiatric hospitalization is needed?

20. Is this different for Medicaid beneficiaries?

21. Has this changed since the demonstration began in [insert date of implementation]?

PROBE: Have wait times in the ER or intake/assessment department increased or decreased since the demonstration began? Why or why not?

VI. Referral and Admission

[Interviewer: If informant is not aware of the demonstration, reword all questions referring to the ‘demonstration’ as the ‘date of implementation.’]

I’d like to shift the discussion to referral and admission to this hospital.

22. What is the primary source of referral for patients to this hospital?

22a.[Ask only if informant is aware of the demonstration.] Is that the same referral source for demonstration patients? If not, what is the primary referral source for demonstration patients?

23. What are other sources of referral for patients to this hospital?

23a. [Ask only if informant is aware of the demonstration.] Are the other referral sources the same for demonstration patients? If not, what are the other sources of referral for demonstration patients?

24. How has your relationship with other sources of referral for admission of patients with psychiatric emergencies changed as a result of the demonstration?

25. How does the referral process since the demonstration began differ from what you were doing before the demonstration?

26. [Ask only if informant is aware of the demonstration.] What are your primary methods for identifying patients for the demonstration?

VII. Stabilization

Next, I would like to discuss procedures for stabilizing patients.

[Interviewer: If informant is not aware of the demonstration, reword all questions referring to the ‘demonstration’ as the ‘date of implementation’.]

27. Please describe your stabilization assessment procedures.

28. How does the stabilization assessment under the demonstration differ from what you were doing before the demonstration?

29. Are you experiencing any challenges adhering to the stabilization assessment requirements?

30. What types of treatments do patients receive while in this hospital?

PROBE: What types of therapies and modes are offered, for example, psychotherapies (CBT, interpersonal therapy, and behavioral therapy), psychoeducation and individual and/or group psychotherapy, or other therapeutic treatments?

31. How does this treatment compare to the treatment received by non-Medicaid beneficiaries with psychiatric emergencies treated in this hospital?

32. How does treatment of psychiatric emergencies differ from treatment provided to patients not experiencing a psychiatric emergency?

VIII. Length of Stay

[Interviewer: Ask Q33 and Q34 only if time permits.]

33. What is the average length of stay for patients in this hospital?

PROBE: For example, people with psychiatric emergencies with payment sources other than Medicaid and people without psychiatric emergencies.

34. [Ask only if informant is aware of the demonstration.] What is the average length of stay for patients enrolled in the demonstration?

IX. Discharge Planning

Now I’d like to talk about discharge planning and post-discharge care.

Interviewer: If informant is not aware of the demonstration, reword all questions referring to the ‘demonstration’ as the ‘date of implementation.’

35. Could you please describe the hospital’s discharge planning procedures?

35a. [Ask only if informant is aware of the demonstration.] Are the discharge planning procedures the same for demonstation patients? If not, how do they differ?

36. How does the discharge planning process differ now from what you were doing prior to the demonstration?

37. How has the quality of discharge planning changed under the demonstration?

PROBE: Has the quality of discharge planning improved, worsened, or stayed the same?

38. How are patients at your hospital involved in discharge planning?

PROBE: How does patient involvement (or lack of) impact the patient’s discharge experience?

39. Is this different than how patients were involved in discharge planning before the demonstration?

40. How does the amount of time staff spend developing discharge plans now compare to the amount of time staff spent on discharge planning for Medicaid beneficiaries prior to the demonstration?

41. Under the demonstration, has the proportion of Medicaid beneficiaries with psychiatric emergencies discharged from your hospital to community-based residences changed?

PROBE: How has the proportion discharged from your hospital to community-based residences changed?

42. Under the demonstration, has the level of detail included in discharge plans changed?

PROBE: How has the level of included detail changed? What is included?

43. To where is the majority of patients discharged?

PROBE: For example, home, group home or other structured setting, jail, or patients are homeless.

44. [Ask only if informant is aware of the demonstration.] To where is the majority of demonstration patients discharged?

45. What proportion of patients are discharged outside of the local area?

45a. [Ask only if informant is aware of the demonstration.] What proportion of demonstration patients are discharged outside of the local area?

46. What types of aftercare services are provided to patients?

46a. [Ask only if informant is aware of the demonstration.] What types of aftercare services are provided to demonstration patients?

47. Where do the majority of patients typically receive aftercare services?

47a. [Ask only if informant is aware of the demonstration.] Where do the majority of demonstration patients typically receive aftercare services?

48. Could you please describe the post discharge follow up procedures for Medicaid beneficiaries?

49. [Ask only if informant is aware of demonstration.] What kinds of changes, if any, have occurred regarding post-discharge follow up procedures for Medicaid beneficiaries as a result of the demonstration?

X. Cost

I’d like to ask next a few questions about cost.

Interviewer: ask Q50 – Q52 only if informant is IMD point of contact, a hospital administrator, and/or is aware of the demonstration.

50. Can you describe the effect the demonstration has had on costs to your hospital?

51. How has the care provided under the demonstration impacted Medicaid costs?

52. What, if any, were the administrative costs to the hospital to fully implement the demonstration (e.g., for staffing or making changes to the physical environment)?

XI. Context

I’d like to talk about the availability of mental health services.

53. What types of psychiatric step-down and outpatient services are available for patients?

53a. [Ask only if informant is aware of demonstration.] What types of psychiatric step-down and outpatient services are available for demonstration patients?

54. Are psychiatric step-down and outpatient services reimbursed by Medicaid?

54a. If not, how are these services funded?

55. Please describe the working relationship your facility has with psychiatric step-downor outpatient providers.

56. Have there been any changes in mental health service delivery that could affect the demonstration (e.g., closure of facilities, new IMDs/hospitals opening, changes in availability of community-based services)?

57. Are you aware of any local-level events or initiatives that may be changing the incidence of psychiatric emergencies and access to services for patients experiencing a psychiatric emergency?

XII. Outcomes

I’d like to conclude by talking about outcomes of the demonstration.

58. What are your thoughts about potential short-term effects of the demonstration?

59. What do you think are the two most important changes, if any, resulting from the demonstration?

60. What do you hope the demonstration will do?

XIII. Closing

That completes the questions we have for you today.

• Is there anything we should have asked about but didn’t?

• Do you have anything you would like to tell us, or questions you would like to ask us?

Thank you again for taking the time to speak with us. We appreciate and value your input.

Medicaid Emergency Psychiatric Demonstration (MEPD)
Interview Guide: MEPD GH Staff Member

Round of Site Visit:

Site Visit Dates:

Facility Name:

Facility State:

Date of MEPD Implementation:

Informant(s) Name:

Informant(s) Title:

Informant(s) Contact Information

Date of Interview:

Time of Interview:

Interviewer:

Note taker:

I. Introduction

Thank you for taking the time to speak with us. We are from Mathematica Policy Research, an independent research firm contracted by the Centers for Medicare & Medicaid Services (CMS) through its Center for Medicare and Medicaid Innovation (CMMI) to evaluate the Medicaid Emergency Psychiatric Demonstration. The three-year demonstration allows eligible, private institutions for mental disease (IMDs) in participating states to receive federal Medicaid reimbursement for adults ages 21 to 64. The purpose of the demonstration is to make inpatient care more accessible to adult Medicaid beneficiaries with psychiatric emergency medical conditions. The evaluation will determine whether and to what extent using Medicaid funding to provide care for adults in private IMDs impacts service use, quality of care, and Medicaid costs.

We are speaking with you to learn about how care is provided in [insert name of GH]. In particular, we are interested in understanding how care is provided to Medicaid beneficiaries experiencing a psychiatric emergency and the process of referring these individuals for inpatient psychiatric treatment.

We will be taking notes during the interview and would like to audiotape our discussion to ensure that we have captured your comments accurately. The audio recording will not be shared with anyone outside of the project team and will be destroyed at the conclusion of the study. Is this okay with you?

Do you have any questions before we get started?

II. Role and Responsibilities

1. Please describe your role and responsibilities at [insert name of GH].

2. How long have you been in this role?

3. How long have you worked at [insert name of GH]?

4. Are you aware of the state’s participation in the Medicaid Emergency Psychiatric Demonstration?

[Interviewer: If respondent is not aware of the demonstration, reword all questions referring to the demonstration as the date of implementation.]

III. Program Design

5. Have you seen any service improvements since [insert name(s) of participating IMD(s)] began the demonstration?

PROBE: For example, changes in procedures for identifying available inpatient beds, ER diversion, use of peer supports in ER, use of mobile crisis team.

IV. Access to Inpatient Psychiatric Care

Next, I would like to discuss access to care.

6. Have you observed any changes in the number of patients being admitted to non-psychiatric units of this hospital for treatment of a psychiatric emergency?

PROBE: Has it increased or decreased? Why?

7. If a change was noted in either direction, how has this change influenced the quality of care delivered?

V. Boarding Time in ER

Now I’d like to talk about the amount of time patients spend in the ER prior to admission.

8. In your experience, how long do patients admitted to your unit after experiencing a psychiatric emergency wait in the ER before being admitted?

[Interviewer: If long waits are reported, ask why.]

9. Has this changed since [insert start date of demonstration in state]?

10. If a change was observed, what factors do you think account for the change?

VI. Referral and Admission

I’d like to shift the discussion to referral and admission to this hospital.

11. Please describe the process for admitting patients with psychiatric emergencies from the ER to non-psychiatric units of this hospital.

12. Have there been any changes in the admission process recently?

12a.If so, what has changed? Why?

VII. Stabilization

13. Next, please tell me about the types of treatments patients experiencing psychiatric emergencies receive while in non-psychiatric units of this hospital.

14. How is stabilization of the psychiatric emergency assessed?

15. When does the assessment begin?

16. How often are stabilization assessments conducted?

17. Is there anything you would like to see done differently in how patients with psychiatric emergencies are stabilized on non-psychiatric units of this hospital?

VIII. Length of Stay

18. What is the average length of stay for psychiatric patients admitted to non- psychiatric units of this hospital?

19. On average, how long do psychiatric emergency patients stay in non-psychiatric units of this hospital while awaiting admission to a psychiatric unit or psychiatric hospital?

IX. Discharge Planning

Now I’d like to talk about discharge planning and post-discharge care.

20. Please describe the discharge planning process for psychiatric patients admitted to non-psychiatric units of this hospital.

21. When does discharge planning begin?

22. Who is involved in developing a discharge plan for psychiatric patients?

23. How are psychiatric patients receiving care from non-psychiatric units in your hospital involved in discharge planning?

24. To where are psychiatric patients treated in non-psychiatric units of your hospital being discharged most frequently?

25. What types of aftercare services are provided to psychiatric patients?

X. Context

I’d like to talk about the context in which the demonstration is operating.

Interviewer note: ask Q26 – 31 only if hospital has a psychiatric unit.

26. How does having an inpatient psychiatric unit affect the extent of psychiatric boarding in your ER and non-psychiatric units?

27. What are the referral sources for admission to the psychiatric unit of your hospital?

28. Have the sources of referral to the unit changed since the demonstration was implemented [insert date of implementation]?

29. What types of patients are served by the psychiatric unit?

30. What impacts has the demonstration had on the hospital’s psychiatric unit, if any?

PROBE: Are they more likely to serve Medicaid beneficiaries or other patients with psychiatric emergencies?

31. Has the average length of stay or discharge planning process changed since implementing the demonstration on [insert date of implementation]?

32. Have there been any changes in mental health service delivery that could affect the demonstration?

PROBE: For example, closure of facilities, new IMDs opening, changes in how psychiatric emergencies are handled in your hospital or community, changes in availability in community-based services?

33. Are there any planned changes in mental health services at the state level that could affect the demonstration?

PROBE: For example, change in payment structure?

XI. Closing

That completes the questions we have for you today.

• Is there anything we should have asked about but didn’t?

• Do you have anything you would like to tell us, or questions you would like to ask us?

Thank you again for taking the time to speak with us. We appreciate and value your input.

Medicaid Emergency Psychiatric Demonstration (MEPD)
Interview Guide: MEPD ER Staff Member

Round of Site Visit:

Site Visit Dates:

Facility Name:

Facility State:

Date of MEPD Implementation:

Informant(s) Name:

Informant(s) Title:

Informant(s) Contact Information:

Date of Interview:

Time of Interview:

Interviewer:

Note taker:

I. Introduction

Thank you for taking the time to speak with us. We are from Mathematica Policy Research, an independent research firm contracted by the Centers for Medicare & Medicaid Services (CMS) through its Center for Medicare and Medicaid Innovation (CMMI) to evaluate the Medicaid Emergency Psychiatric Demonstration. The three-year demonstration allows eligible, private institutions for mental disease (IMDs) in participating states to receive federal Medicaid reimbursement for adults ages 21 to 64. The purpose of the demonstration is to make inpatient care more accessible to adult Medicaid beneficiaries with psychiatric emergency medical conditions. The evaluation will determine whether and to what extent using Medicaid funding to provide care for adults in private IMDs impacts service use, quality of care, and Medicaid costs.

We are speaking with you to learn about how care is provided in [insert name of ER] In particular; we are interested in understanding how care is provided to Medicaid beneficiaries experiencing a psychiatric emergency and the process of referring these individuals for inpatient psychiatric treatment.

We will be taking notes during the interview and would like to audiotape our discussion to ensure that we have captured your comments accurately. The audio recording will not be shared with anyone outside of the project team and will be destroyed at the conclusion of the study. Is this okay with you?

Do you have any questions before we get started?

II. Role and Responsibilities

1. Please describe your role and responsibilities at [insert name of ER].

2. How long have you been in this role?

3. How long have you worked at [insert name of ER]?

4. Are you aware of the state’s participation in the Medicaid Emergency Psychiatric Demonstration?

[Interviewer: If respondent is not aware of the demonstration, reword all questions referring to the demonstration as the date of implementation].

III. Program Design

5. Have you seen any service improvements since [insert name(s) of participating IMD(s)] began the demonstration?

PROBE: For example, changes in procedures for identifying available inpatient beds, ER diversion, use of peer supports in ER, use of mobile crisis teams.

IV. Access to Inpatient Psychiatric Care

Next, I would like to discuss access to care.

6. How often do individuals experiencing a psychiatric emergency seek treatment in this ER?

7. Please describe how you work with individuals experiencing a psychiatric emergency.

[Interviewer: Ask Q8 only if this hospital has a psychiatric unit.]

8. I understand that this hospital has a psychiatric unit. Do you contact the unit to determine bed availability?

8a. If the psychiatric unit is not contacted, please explain why.

9. Which facilities do you contact for inpatient care for patients with a psychiatric emergency?

10. Are the facilities you contact the same facilities you contact for Medicaid beneficiaries?

PROBE: Why or why not? Is there a particular order in you contact hospitals?

11. What is your experience with the rate at which patients with psychiatric emergencies are accepted by these hospitals?

11.a Is the acceptance rate different for Medicaid beneficiaries?

V. Boarding Time in ER

Now I’d like to talk about the amount of time patients spend in the ER prior to admission.

12. On average, how long does a patient with a psychiatric emergency currently wait in the ER once it has been decided that psychiatric hospitalization is needed?

13. Are wait times different for Medicaid beneficiaries?

14. Has this changed since [insert start date of demonstration in state]?

15. If a change was observed, what factors do you think account for the change?

VI. Referral and Admission

I’d like to shift the discussion to referral and admission of patients experiencing a psychiatric emergency to psychiatric hospitals.

16. How do you determine whether someone in the ER is suicidal, homicidal, or a danger to themselves or others?

17. How do you determine whether someone with a psychiatric emergency is in need of inpatient psychiatric hospitalization?

18. Have you noticed any changes since [insert start of demonstration in state] in how patients who present with a psychiatric emergency in your ER are admitted?

PROBE: Do you contact a different person to assess the patient’s level of need? Are the verification process or eligibility criteria different? Has the timing of the verification process changed?

19. Have there been any changes in the types of patients admitted since [insert start of demonstration in state]?

PROBE: Were there any patients not admitted for inpatient care that you felt should have been?

20. Has the admission process changed under the demonstration?

VII. Stabilization

21. Next, please describe how patients experiencing a psychiatric emergency are stabilized in the ER.

22. Have these processes changed since the demonstration was implemented?

VIII. Cost

[Interviewer: ask Q23 only if informant is a hospital administrator and/or is aware of the demonstration.]

23. What, if any, were your administrative costs to fully implement the demonstration (e.g., for staffing or making changes to the physical environment)?

IX. Context

I’d like to talk about the context in which the demonstration is operating.

24. To what extent is psychiatric boarding an issue in your ER?

25. Is your hospital or department involved in other initiatives that could influence the demonstration (e.g., ER diversion programs)?

26. Have there been any changes in the community that have affected the number of individuals with a psychiatric emergency who present in the ER?

27. Does the [insert name of IMD] participation in the demonstration change how you refer patients?

PROBE: For example, are you more inclined to contact IMDs first?

X. Outcomes

I’d like to conclude the interview by talking about outcomes of the demonstration.

28. What are your thoughts about potential short-term effects of the demonstration?

29. What do you think are the two most important changes, if any, resulting from the demonstration?

30. What do you hope the demonstration will do?

XI. Closing

That completes the questions we have for you today.

• Is there anything we should have asked about but didn’t?

• Do you have anything you would like to tell us, or questions you would like to ask us?

Thank you again for taking the time to speak with us. We appreciate and value your input.

ATTACHMENT C

SAMPLING PROCEDURES FOR MEDICAL RECORD REVIEW

A. Sampling Approach

We will use purposive sampling procedures to select 10 patient medical records for review at each of three facility types: IMDs, referring hospital ERs, and general hospitals (GHs) that admit Medicaid beneficiaries experiencing a psychiatric emergency when no psychiatric beds are available. The total number of records reviewed per state will vary based on the number of participating IMDs. For example, in states with four IMDs, 120 records will be reviewed—10 from each of the four IMDs, and 10 from both an ER and GH associated with each of the four IMDs. For states with only one participating IMD, a total of 30 records will be reviewed—10 from each facility type.

Prior to the site visit, we will ask each facility’s point of contact for specific patient rosters, described below, from which we will select records to review.¹ Purposive sampling will enable site visitors to identify medical records for patients of particular interest, such as those with medical comorbidities or high-risk behaviors requiring chemical or physical restraint, or those whose length of stay was greater than average for a particular IMD.

B. Sampling Procedure

In order to select samples of patient medical records, the procedures below will be carefully followed by site visitors. A description of rosters needed from each facility type is provided, along with the number of patients to be sampled from each roster.

1. Roster Descriptions and Patient Sample Sizes by Facility Type

IMDs—The site visit team will request two different rosters of IMD patients. Each roster will include the patient’s name, age, admission date and time, diagnosis, and the discharge date and length of stay for patients who have been discharged. A total of 10 patients will be chosen from among the two rosters, as follows:

• Five patients will be selected from a roster of demonstration patients discharged 30 - 60 days prior to the start of the site visit (closed medical records).

• Five patients will be selected from a roster of Medicaid beneficiaries who experienced a psychiatric emergency and were discharged 30 - 60 days prior to the implementation of the demonstration (closed medical records). Selecting closed medical records of patients receiving treatment prior to the implementation of the demonstration will allow site visitors to assess changes in quality of care.²

ERs—The site visit team will request two rosters from each referring ER that is visited. Selecting closed medical records of patients receiving care at the ER prior to and during the demonstration will allow site visitors to assess changes in the amount of time Medicaid patients spend in the ER, care received, and discharge disposition. Rosters will include the patients’ name, age, admission date, diagnosis, discharge date, and length of time spent in the ER. A total of 10 patients will be chosen from the two rosters, as follows:

• Five patients will be selected from among Medicaid patients discharged from the ER with psychiatric emergencies 30 - 60 days prior to the start of the site visit.

• Five patients will be selected from among Medicaid patients discharged from the ER with psychiatric emergencies 30 - 60 days prior to the implementation of the demonstration.

GHs—The site visit team will request two rosters from each GH visited. The roster will include the patients’ name, age, admission date and time, admitting diagnosis, discharge date, and length of stay. A total of 10 patients will be chosen from the two rosters, as follows:

• Five patients will be selected from a roster of Medicaid beneficiaries with psychiatric emergencies who were discharged from the GH 30 - 60 days prior to the start of the site visit. If, as a result of the demonstration, fewer than 5 Medicaid beneficiaries with psychiatric emergencies have been admitted to the GH, we will request to also examine records for non-Medicaid beneficiaries admitted with psychiatric emergencies. If psychiatric boarding in the GH has been eliminated, we will record the date of the last admission of a Medicaid beneficiary for a psychiatric emergency, as well as the date of the last admission for a psychiatric emergency of any kind.

• Five patients will be selected from a roster of Medicaid patients with psychiatric emergencies who were discharged from the GH 30 - 60 days prior to implementation of the demonstration.

2. Sample Selection

Using the list of patient characteristics in priority order below, we will choose patients from each roster until the sample number specified for the roster is reached. For IMD and GH patients, we will choose only one patient from each category unless no patients exist in the other categories. For ER patients, we will select patients with the longest length of stay. Although only the first category (length of stay) will be indicated explicitly on the roster, we will review the roster with facility staff to identify patients with the other characteristics.

31. Long length of stay (if the length of stay is abnormal or above average)

• High suicide risk

• High homicide risk

• Medical comorbidities

• Co-occurring substance use diagnosis

If multiple patients on a roster fall within a given characteristic category, we will use any other information that is provided to select a patient that is a more complicated or unique case.

3. Sampling Labels

To maintain patient privacy and security we will use a unique Mathematica numbering system to identify the patients in our sample. The Mathematica number will indicate the state, type of facility (IMD, ER, or GH), and a 2-digit suffix unique to the patient. We will identify IMD patients discharged 30 - 60 days prior to the site visit by suffixes between 21 and 29, and IMD patients discharged 30 - 60 days prior to the demonstration by suffixes between 31 and 39. We will identify patients discharged from an ER 30 - 60 days prior to the site visit by suffixes between 41 and 49 and patients discharged from an ER 30 - 60 days prior to the demonstration by suffixes between 51 and 59. We will identify Medicaid beneficiaries with psychiatric emergencies who were discharged from the GH 30 – 60 days prior to the site visit by suffixes between 61-69 and Medicaid beneficiaries with psychiatric emergencies discharged from the GH 30 – 60 days prior to the implementation of the demonstration by suffixes between 71 and 79. Site visitors will receive pre-numbered sample labels, with several labels for each patient sampled. Site visitors will attach a label on the applicable roster next to the patient’s name and will enter the number in the computerized record review data collection protocol. The facility points of contact will be asked to keep rosters, with labels attached, for six months after the site visit in case questions arise regarding the record review after the site visit is completed.

attachment d

Medical Record Review ToolS

Medicaid Emergency Psychiatric Demonstration (MEPD)
Medical Record Review: MEPD Institution of Mental Disease (IMD)

Round of Site Visit:

Site Visit Dates:

IMD Name:

State:

IMD Point of Contact:

IMD Point of Contact Information:

Date of MEPD Implementation:

Type of Information System:

__ Electronic, __ Paper, __ Combination

Brief description of system: ______________________________________________

Name of Information System:

Site Visitor:

Record Review Date:

RECORD 1

Mathematica Patient ID: [attach label or enter number]

Description of patient characteristics:

A. Access to Inpatient Psychiatric Care

1. Source of referral to this IMD:

2. Was the patient previously admitted to this IMD?

Yes  [Enter date of most recent prior admission]

No

Unable to determine

3. Has the patient been hospitalized twice or more during the past year?

PROBE: During the 12 months prior to the date of this admission.

Yes

No

Unable to determine

Reviewer’s comments/notes about this section:

B. Boarding Time in Emergency Room

4. When was this IMD contacted about bed availability for the patient’s most recent visit?

a. Date hospital contacted:

b. Time hospital contacted: am/pm

c. Unable to determine

5. When was the patient transferred to this IMD for the most recent admission?

a. Date transferred to hospital:

b. Time transferred to hospital: am/pm

c. Unable to determine

6. How was the patient transported to this hospital?

a. Ambulance

b. Receiving hospital’s transportation

c. Other

Specify:

d. Unable to determine

Reviewer’s comments/notes about this section:

C. Admission to IMD

7. When was the patient admitted to this hospital?

a. Date of admission:

b. Time of admission: am/pm

c. Unable to determine

8. Did patient exhibit signs and symptoms of intoxication and/or withdrawal from drugs or alcohol upon admission?

Yes

No

Unable to determine

8a. If Yes, describe symptoms of withdrawal exhibited by patient.

9. When was the initial nursing assessment completed?

a. Date of initial nursing assessment:

b. Time of initial nursing assessment: am/pm

c. Unable to determine

10. When was the initial medical history and physical completed?

a. Date of initial medical history and physical:

b. Time of initial medical history and physical: am/pm

c. Unable to determine

11. When was the initial psychiatric evaluation completed?

a. Date of initial psychiatric evaluation:

b. Time of initial psychiatric evaluation: am/pm

c. Unable to determine

12. Which diagnoses were identified in the initial psychiatric evaluation completed at this hospital?

	Dimension	Diagnoses (Include DSM code and description if provided.)	Not documented
1.	Axis I
2.	Axis II
3.	Axis III
4.	Axis IV
5.	Axis V

Reviewer’s comments/notes about this section:

D. Stabilization

13. Does the medical record include documentation that the patient was assessed for stabilization (that is, to determine whether they remained suicidal, homicidal, or a danger to themselves or others) by the third day of IMD admission?

Yes

No  GO TO Q.15

Unable to determine  GO TO Q.15

14. Enter date(s) of stabilization assessment documentation provided in the medical record regarding whether the patient was suicidal, homicidal, or a danger to themselves or others. [Interviewer: Ask person assisting with chart review how the hospital defines stabilization assessment.]

Stabilization Assessment Date	Patient expressed suicidal or homicidal thoughts or gestures, or is dangerous to self or others
a. MM/DD/YYYY	Yes No Not Documented
b. MM/DD/YYYY	Yes No Not Documented
c. MM/DD/YYYY	Yes No Not Documented
d. MM/DD/YYYY	Yes No Not Documented
e. MM/DD/YYYY	Yes No Not Documented
f. MM/DD/YYYY	Yes No Not Documented

15. Was the patient chemically restrained, that is given psycho-active medication to subdue behavior while at this IMD?

Yes, patient requested medication

Yes, staff initiated medication

No GO TO Q.17

Unable to determine GO TO Q.17

16. Enter the date(s) and time(s) of chemical restraint, name of pharmacological agent(s) administered, dosage, and mode of administration.

    	Date	Time	Name of Pharmacological Agent(s)	Dose	Mode of Administration (IM, IV, PO, or SQ)
    1.
    2.
    3.
    4.
    5.

17. Was the patient physically restrained while at this IMD?

Yes

No GO TO Q.19

Unable to determine  GO TO Q.19

18. Enter the date(s), time(s), and mode of physical restraint.

	Date	Time	Mode of Restraint (Four point leather or cloth restraint, physical hold, hand mitts, other)
1.
2.
3.
4.
5.
6.

19. Was consultation ordered for evaluation of an active or chronic medical condition?

Yes

No  GO TO Q.21

Unable to determine  GO TO Q.21

20. Was treatment provided for an active or chronic medical condition as a result of the consultation?

Yes, treatment provided at this facility

Yes, treatment provided at a different facility

No

21. Did an injury or infection occur during the patient’s stay in this hospital?

Yes

No  GO TO Q.23

Unable to determine GO TO Q.23

22. What type of injury or infection did the patient have?

a. Self-inflicted injury

b. Nosocomial injury only

c. Nosocomial infection only

d. Both nosocomial injury and infection

Reviewer’s comments/notes about this section (describe the stabilization process):

E. Discharge Planning

23. What was the earliest date discharge plans, or a patient meeting with a discharge planner, was documented?

Date:

Not documented  GO TO Q25

24. Does the discharge plan include documentation of patient’s preferences after discharge?

Yes

Not documented

25. When was the patient discharged from this IMD?

a. Date of discharge:

b. Time of discharge: am/pm

26. Does the medical record include documentation that IMD staff contacted the patient’s other providers for input into the discharge plan?

Yes

No

Unable to determine

27. Does the discharge plan include a follow-up aftercare appointment scheduled within 7 days of the discharge date?

Yes

Yes, but not scheduled for within 7 days of the discharge date

No  GO TO Q.29

Unable to determine  GO TO Q.29

28. Record date of appointment and provider.

a. Appointment date:

b. Provider’s name:

29. Does the medical record include documentation that medication reconciliation was conducted upon discharge?

Yes

No

Unable to determine

30. Does the discharge plan include discharge medications?

Yes

No

Unable to determine

31. Does the discharge plan include the reason for hospitalization?

Yes

No

Unable to determine

32. Does the discharge plan include the principal discharge diagnosis?

Yes

No

Unable to determine

33. Does the discharge plan include the next level of care recommendations?

Yes

No

Unable to determine

34. Does the discharge plan include documentation that the discharge plan was sent to patient’s aftercare provider?

Yes

No

Unable to determine

35. Does the discharge plan include the patient’s signature?

Yes

No

Unable to determine

Reviewer’s comments/notes about this section:

END

Medicaid Emergency Psychiatric Demonstration (MEPD)
Medical Record Review: MEPD General Hospital (GH)

Round of Site Visit:

Site Visit Dates:

GH Name:

State:

GH Point of Contact:

GH Point of Contact Information:

Date of MEPD Implementation:

Type of Information System:

__ Electronic, __ Paper, __ Combination):

Brief description of system:_____________________________________________

Name of Information System:

Site Visitor:

Record Review Date:

RECORD 1

Mathematica Patient ID: [attach label or enter number]

Description of patient characteristics:

A. Access to Inpatient Psychiatric Care

1. Source of referral to this general hospital:

2. Was the patient previously admitted to this general hospital for psychiatric treatment in a non-psychiatric unit?

Yes  [Enter date of most recent prior admission]

No

Unable to determine

3. Has the patient been hospitalized twice or more during the past year?

PROBE: During the 12 months prior to the date of this admission.

Yes

No

Unable to determine

Reviewer’s comments/notes about this section:

B. Boarding Time in Emergency Room

4. When was this general hospital contacted about bed availability for the patient’s most recent visit?

a. Date hospital contacted:

b. Time hospital contacted: am/pm

c. Unable to determine

5. When was the patient transferred to this general hospital for the most recent admission?

a. Date transferred to hospital:

b. Time transferred to hospital: am/pm

c. Unable to determine

6. How was the patient transported to this general hospital?

a. Ambulance

b. Receiving hospital’s transportation

c. Other

Specify:

d. Unable to determine

Reviewer’s comments/notes about this section:

C. Admission to GH

7. When was the patient admitted to this general hospital?

a. Date of admission:

b. Time of admission: am/pm

c. Unable to determine

8. Did patient exhibit signs and symptoms of intoxication and/or withdrawal from drugs or alcohol upon admission?

Yes

No

Unable to determine

8a. If Yes, describe symptoms of withdrawal exhibited by patient.

9. When was the initial nursing assessment completed?

a. Date of initial nursing assessment:

b. Time of initial nursing assessment: am/pm

c. Unable to determine

10. When was the initial medical history and physical completed?

a. Date of initial medical history and physical:

b. Time of initial medical history and physical: am/pm

c. Unable to determine

11. When was the initial psychiatric evaluation completed?

a. Date of initial psychiatric evaluation:

b. Time of initial psychiatric evaluation: am/pm

c. Unable to determine

12. Which diagnoses were identified in the initial psychiatric evaluation completed at this hospital?

	Dimension	Diagnoses (Include DSM code and description if provided.)	Not documented
1.	Axis I
2.	Axis II
3.	Axis III
4.	Axis IV
5.	Axis V

Reviewer’s comments/notes about this section:

D. Stabilization

13. Does the medical record include documentation that the patient was assessed for stabilization (that is, to determine whether they remained suicidal, homicidal, or a danger to themselves or others) by the third day of admission?

Yes

No  GO TO Q.15

Unable to determine  GO TO Q.15

14. Enter date(s) of stabilization assessment documentation provided in the medical record regarding whether the patient was suicidal, homicidal, or a danger to themselves or others. [Note: Site visitor will need to ask person assisting with chart review how the hospital defines stabilization assessment]

Stabilization Assessment Date	Patient expressed suicidal or homicidal thoughts or gestures, or is dangerous to self or others
a. MM/DD/YYYY	Yes No Not Documented
b. MM/DD/YYYY	Yes No Not Documented
c. MM/DD/YYYY	Yes No Not Documented
d. MM/DD/YYYY	Yes No Not Documented
e. MM/DD/YYYY	Yes No Not Documented
f. MM/DD/YYYY	Yes No Not Documented

15. Was the patient chemically restrained, that is given psycho-active medication to subdue behavior while at this general hospital?

Yes, patient requested medication

Yes, staff initiated medication

No GO TO Q.17

Unable to determine GO TO Q.17

16. Enter the date(s) and time(s) of chemical restraint, name of pharmacological agent(s) administered, dosage, and mode of administration.

	Date	Time	Name of Pharmacological Agent(s)	Dose	Mode of Administration (IM, IV, PO, or SQ)
1.
2.
3.
4.
5.
6.

17. Was the patient physically restrained while at this general hospital?

Yes

No GO TO Q.19

Unable to determine  GO TO Q.19

18. Enter the date(s), time(s), and mode of physical restraint.

	Date	Time	Mode of Restraint (Four point leather or cloth restraint, physical hold, hand mitts, other)
1.
2.
3.
4.
5.
6.

19. Was consultation ordered for evaluation of an active or chronic medical condition?

Yes

No  GO TO Q.21

Unable to determine  GO TO Q.21

20. Was treatment provided for an active or chronic medical condition as a result of the consultation?

Yes, treatment provided at this facility

Yes, treatment provided at a different facility

No

21. Did an injury or infection occur during the patient’s stay in this hospital?

Yes

No  GO TO Q.23

Unable to determine GO TO Q.23

22. What type of injury or infection did the patient have?

a. Self-inflicted injury

b. Nosocomial injury only

c. Nosocomial infection only

d. Both nosocomial injury and infection

Reviewer’s comments/notes about this section (describe the stabilization process):

E. Discharge Planning

23. What was the earliest date discharge plans, or a patient meeting with a discharge planner, was documented?

Date:

Not documented  GO TO Q25

24. Does the discharge plan include documentation of patient’s preferences after discharge?

Yes

Not documented

25. When was the patient discharged from this general hospital?

a. Date of discharge:

b. Time of discharge: am/pm

26. Does the medical record include documentation that general hospital staff contacted the patient’s other providers for input into the discharge plan?

Yes

No

Unable to determine

27. Does the discharge plan include a follow-up aftercare appointment scheduled within 7 days of the discharge date?

Yes

Yes, but not scheduled for within 7 days of the discharge date

No  GO TO Q.29

Unable to determine  GO TO Q.29

28. Record date of appointment and provider.

a. Appointment date:

b. Provider’s name:

29. Does the medical record include documentation that medication reconciliation was conducted upon discharge?

Yes

No

Unable to determine

30. Does the discharge plan include discharge medications?

Yes

No

Unable to determine

31. Does the discharge plan include the reason for hospitalization?

Yes

No

Unable to determine

32. Does the discharge plan include the principal discharge diagnosis?

Yes

No

Unable to determine

33. Does the discharge plan include the next level of care recommendations?

Yes

No

Unable to determine

34. Does the discharge plan include documentation that the discharge plan was sent to patient’s aftercare provider?

Yes

No

Unable to determine

35. Does the discharge plan include the patient’s signature?

Yes

No

Unable to determine

Reviewer’s comments/notes about this section:

END

Medicaid Emergency Psychiatric Demonstration (MEPD)
Medical Record Review: MEPD Emergency Room (ER)

Round of Site Visit:

Site Visit Dates:

ER Hospital Name:

State:

ER Point of Contact:

ER Point of Contact Information

Date of MEPD Implementation:

Type of Information System:

__ Electronic, __ Paper, __ Combination

Brief description of System:

___________________________________________________________

Name of Information System:

Site Visitor:

Record Review Date:

RECORD 1

Mathematica Patient ID: [attach label or enter number]

Description of patient characteristics:

A. Admission to Emergency Room (ER)

1. When was the patient admitted to the ER?

a. Date of admission to ER:

b. Time of admission to ER: am/pm

2. Was the patient’s Medicaid number identified in the medical record?

Yes

No

Unable to determine

3. When was the initial medical history and physical examination completed?

a. Date of initial medical history and physical examination:

b. Time of initial medical history and physical examination: am/pm

c. Unable to determine

4. When was the patient medically cleared by a provider?

a. Date of medical clearance:

b. Time of medical clearance: am/pm

c. Unable to determine

5. Upon admission to the ER, was the patient identified as…

a. Suicidal?

b. Homicidal?

c. Dangerous to themselves?

d. Dangerous to others?

e. Unable to determine

6. When was the patient assessed by a provider to determine whether inpatient psychiatric treatment was necessary?

a. Date psychiatric emergency determined:

b. Time psychiatric emergency was determined: am/pm

c. Unable to determine

7. What type of provider determined the presence of a psychiatric emergency?

a. MD/DO

b. NP/CNS/PA

c. RN

d. LCSW

e. Psychologist

f. Licensed mental health professional (e.g., licensed counselor or therapist)

g. Other

Specify:

h. Unable to determine

8. Was eligibility for the demonstration indicated in the ER medical record?

Yes, patient eligible

Yes, patient not eligible

Not documented

Not applicable, pre-demonstration

9. Which diagnoses were identified in the initial psychiatric evaluation completed at this ER?

	Dimension	Diagnoses (Include DSM code and description if provided.)	Not documented
1.	Axis I
2.	Axis II
3.	Axis III
4.	Axis IV
5.	Axis V

Reviewer’s comments/notes about this section:

B. Stabilization

10. Was the patient evaluated for active substance use while in the ER?

Yes

No  GO TO Q.12

Unable to determine  GO TO Q.12

11. What type of evaluation was conducted?

a. Specialist consult

b. Laboratory diagnostics

c. Other

Specify:

d. Unable to determine

12. Was the patient treated for active substance use while in the ER?

Yes

No  GO TO Q.14

Unable to determine  GO TO Q.14

13. What type of treatment was provided to the patient?

a. Pharmacologic treatment

b. Other

Specify:

c. Unable to determine

14. Was the patient evaluated for an active or chronic medical condition while in the ER?

Yes

No  GO TO Q.16

Unable to determine  GO TO Q.16

15. What type of evaluation was conducted?

a. Specialist consult

b. Laboratory diagnostics

c. Radiographic or ultrasonic diagnostics

d. Other

Specify:

e. Unable to determine

16. Was the patient treated for an active or chronic medical condition while in the ER?

Yes

No  GO TO Q.18

Unable to determine  GO TO Q.18

17. What type of treatment was provided to the patient?

a. Pharmacologic treatment

b. Education/support

c. Other

Specify:

18. Was the patient chemically restrained, that is, given psycho-active medication to subdue behavior while at this ER?

Yes, patient requested medication

Yes, staff initiated medication

No  GO TO Q.20

Unable to determine  GO TO Q.20

19. Enter the date(s) and time(s) of chemical restraint, name of pharmacological agent(s) administered, dosage, and mode of administration.

	Date	Time	Name of Pharmacological Agent(s)	Dose	Mode of Administration (IM, IV, PO, or SQ)
1.
2.
3.
4.
5.
6.

20. Was the patient physically restrained while at this ER?

Yes

No  GO TO Q.22

Unable to determine  GO TO Q.22

21. Enter the date(s), time(s), and mode of physical restraint.

	Date	Time	Mode of Restraint (Four point leather or cloth restraints, physical hold, hand mitts, other)
1.
2.
3.
4.
5.
6.

Reviewer’s comments/notes about this section:

C. Access to Inpatient Psychiatric Care

22. To where was the patient discharged or transferred from the ER?

Specify: __________________________________________

23. What facilities were contacted to see whether a bed was available for the patient?

	Name of Facility	Date contacted for bed availability	Time contacted for bed availability	Date patient accepted for bed	Time patient accepted for bed
1.
2.
3.

24. When was the patient discharged from the ER?

a. Date of discharge from ER:

b. Time of discharge from ER: am/pm

25. How was the patient transported to their discharge placement?

a. Ambulance

b. Receiving facility transportation

c. Other

Specify:

d. Unable to determine

Reviewer’s comments/notes about this section:

END

Attachment E

BENEFICIARY INTERVIEW Protocol, CONSENT FORM, AND RECRUITMENT SCRIPT

MEPD Beneficiary Interview Guide

(Approximate length: 30-60 minutes)

Round of Site Visit:

Site Visit Dates:

Facility Name:

Facility State:

Date of MEPD Implementation:

Informant ID Number:

Informant Contact Information:

Date of Interview:

Time of Interview:

Interviewer:

Note taker:

SOC station number:

Introduction

[If this is a scheduled interview, start here]

Hi, can I please speak with [beneficiary first and last name]?

If beneficiary answers the phone: This is [interviewer name] from Mathematica Policy Research. I’m calling because you agreed to participate in an interview. Does this sound familiar to you? [Interviewer pause and wait for recognition to ensure we have correct person on the phone].

[If no/unsure recognition] Is there another [beneficiary first and last name] in your household? Is that person available to speak with me? [If no] Do you know when might be a good time to reach him/her? Ok, thank you. I’ll try calling back another time.

I’d like to hear your perspective on the experience you had recently at [IMD]. You mentioned that you were available to talk with us today - is this still a good time? [If not, schedule another day/time and confirm contact information].

If someone else answers and questions the purpose of the call: I’m calling in relation to an interview that [beneficiary name - Mr./Ms. X] agreed to participate in. Is [he/she] available? [If not] Do you know when might be a good time to reach him/her? Thank you, I’ll call back another time.

If someone else continues to probe about purpose: I’m sorry, I would like to be able to answer your questions but we are committed to maintaining the privacy of the people we interview. Is it possible for [beneficiary name - Mr./Ms. X] to talk with me? [If not] Do you know when might be a good time to reach him/her? Thank you, I’ll call back another time.

[If interview is conducted during the initial contact, start here]

Thanks so much for taking the time to talk with me today. You will receive a $20 check in the mail for completing the interview. [If there is a note taker on the phone] I have another staff member [colleague’s name] from our company on the phone today to take notes during our discussion. Is that OK with you? [If not, have colleague hang up and the interviewer will take notes].

Are you comfortable with our discussion being audio taped to ensure that we remember everything correctly? The audio tape will be destroyed after 90 days. I want to remind you that, to the extent permitted by law, your answers will be kept private and secure; that is, your information will be used only for this study, and your name will not be associated with your answers. [If respondent consents to recording, start recorder] [If respondent does not consent to recording and the interviewer is using a phone line with automatic audio recording, then (1) turn off the recording feature, or (2) notify the beneficiary that they should stay on the line and hold while the interviewer transfers the call to a non-recorded phone line, or (3) request that the beneficiary hang up the phone and the interviewer will call them back from a non-recorded line].

Your answers are really important to help us learn about quality of care for people experiencing psychiatric emergencies. If I go through the questions too quickly or you don’t understand something, please stop me at any point. Talking about your hospital stay may bring up sensitive issues. If there are any questions you do not want to answer, we can skip them or end the discussion at any time. Please just let me know, and I will move on to the next question. Do you have any questions before we begin?

[Interviewer note: beneficiary will receive $20 incentive if they participate for 30 minutes. Schedule another call to try to finish if they can only complete 15 minutes at a time, even if it takes 3 calls to finish. If they don’t like the questions and don’t want to answer them note it below the question(s) and at the end of the interview guide. Do not give the incentive if they never show up for later interviews or hang up without explanation after only completing 15 minutes].

Access to Inpatient Psychiatric Care

1. I know that you were recently hospitalized for a psychiatric crisis at [name of IMD]. Was it your choice to go to [IMD]?

[Follow-up]

1. If so, why did you choose to go there?

2. If not, how was it decided that you would go there? (Probe: Who decided, and why?)

3. How many other times since [state demonstration start date] did you seek help for an emotional or mental crisis through an emergency room, hospital, or other crisis service?

4. [If sought help other times since the demonstration began] When you had other crises, were you also admitted to a hospital?

   1. If so, did you go to [IMD]?

   2. [If did not go to [IMD]] Where did you go instead of [IMD]? How did it compare to [IMD]?

   3. Where would you prefer to go in the future? Why?

2. Before [state demonstration start date], how many times did you seek help for an emotional or mental crisis through an emergency room, hospital, or other crisis service?

1. [If sought help at any time prior to [state demonstration start date] and used an emergency room] About how many times per year did you use the emergency room for a psychiatric emergency before [state demonstration date]? How many times have you used the emergency room for a psychiatric emergency since [state demonstration start date]? (Probe: Do you think you went to the emergency room more or less this past year compared to years before?]

2. [If experienced any crisis before demonstration start date] When was the last time before [state demonstration start date] that you sought help for an emotional or mental crisis through an emergency room, hospital, or other crisis service?

3. Were you admitted to the hospital?

4. If so, did you go to [IMD]?

5. [If did not go to [IMD]] Where did you go instead of [IMD]? How did it compare to [IMD]? (Probe: admission process, types of treatment received)

[Interviewer note: If beneficiary has not experienced crises within 3 years prior to [date of demonstration] that required hospitalization, omit all questions regarding prior crises throughout the remainder of the protocol. If beneficiary has experienced a crisis within 3 years prior to [date of demonstration] that required hospitalization, note the approximate date of that crisis and any other details provided so that you can refer clearly to that event throughout the interview. We are interested in comparing (1) the hospitalization that occurred just prior to the site visit and (2) the most recent hospitalization (if any) before the demonstration date].

Boarding Time in the ER

3. I know that the hospital admission process can often be quite challenging. In your situation, do you recall going to an emergency room right before going into [IMD]? If so, which emergency room did you use? If not, how did you get into the hospital? [Interviewer note: keep the discussion focused on their hospital admission before the site visit]

(Probe: Did a doctor admit you directly into the hospital? Did a mobile crisis team take you there? Did you go directly to the hospital yourself (walk-in)?)

[If used emergency room or alternative, ask a-h]

1. Why did you go to this particular emergency room (or alternative)?

2. Before going to [IMD], how long did you wait in the emergency room (or alternative)?

3. [If ER] Before you went to [IMD], did the staff move you to a bed in the main part of the hospital? If so, was it in a psychiatric unit or some other kind of hospital unit? How long did you stay there?

4. To the best of your ability, could you describe what your experience was like while waiting in the emergency room (or general medical unit or alternative)? What type of treatment did you receive (e.g., counseling, medication)? What was the environment like?

5. [If experienced a crisis prior to [demonstration date], ask the following questions from e-h; otherwise, skip to the Referral and Admission section]: For [the crisis prior to demonstration date], did you use the same emergency room (or alternative)? If not, how did you get help?

6. How did your experiences waiting for admission during your most recent crisis compare to your experiences during [the crisis prior to the demonstration date]?

7. For [the crisis prior to demonstration date], when you went to an emergency room (or alternative) for an emotional or mental crisis and needed hospitalization, did you wait a longer or shorter time to be admitted to a hospital than the most recent time?

8. [If ER was used for crisis prior to demonstration date] For [the crisis prior to demonstration date], did the emergency room ever move you to a bed in the main part of the hospital? If so, what kind of unit was it? (Probe: psychiatric unit, other unit?) How long did you stay there?

[If walk-in to IMD, ask i-m]

9. To the best of your ability, could you describe what your experience was like while waiting to be admitted to [IMD]? What type of treatment did you receive (e.g., counseling, medication)? What was the environment like?

10. [If experienced a crisis prior to demonstration date ask i-l]: For [the crisis prior to demonstration date], how did you get help? (Probe: walk-in, emergency room or alternative)

11. How did your experiences waiting for admission during your most recent crisis compare to your experiences during [the crisis prior to the demonstration date]?

12. For [the crisis prior to demonstration date], did you wait a longer or shorter time to be admitted to a hospital than the most recent time?

13. [If ER was used for crisis prior to demonstration date] For [the crisis prior to demonstration date], did the emergency room ever move you to a bed in the main part of the hospital? If so, what kind of unit was it? (Probe: psychiatric unit, other unit?) How long did you stay there?

Referral and Admission

4. [If used ER or alternative] Why did you first go to the emergency room (or alternative) before you were hospitalized at [IMD]?

[If walk-in to IMD] What led you to go to [IMD]?

[Interviewer note: keep the discussion focused on their hospital admission before the site visit]

1. Do you recall feeling suicidal, homicidal, or that you were a danger to yourself or others? Did the emergency staff (or alternative or IMD staff) ask you questions about this?

2. How do you recall the process of your admission? (Probe: Who decided? Why?)

3. How were you involved in the decision to go to the hospital? Were you accompanied by someone? Did anyone ask you where you would prefer to receive treatment?

4. [If used emergency room or alternative] Did you give your Medicaid card to someone at the emergency room (or alternative)? Did someone explain to you which hospital you would go to and what was happening?

[If walk-in to IMD] Did you give your Medicaid card to someone at [IMD]?

5. [If experienced a crisis prior to [demonstration date], ask the following questions; otherwise, skip to the Stabilization section]: How did your experience with referral and admission to [IMD] during this most recent crisis compare to [the crisis prior to demonstration date]? Did you notice anything different this time?

Stabilization

5. What types of group or individual activities did you engage in while you were at [IMD]? [Interviewer note: keep the discussion focused on their hospital admission before the site visit]

1. Were these activities helpful? If so, how? If not, why not?

2. Did someone explain your treatment plan to you?

3. How frequently were you offered the opportunity to speak with a doctor?

4. How was the decision made that you were ready to leave the hospital?
   (Probe: Who made the decision? How were you involved in making the decision?)

5. When the hospital told you that you could leave, did you feel safe to leave the hospital?

6. [If experienced a crisis prior to [demonstration date], ask the following questions; otherwise, skip to the next section]: How did the care you received compare to care you have received during the hospitalization for your mental health prior to [the demonstration date]?

Length of Stay

6. For your recent admission to [IMD], how long did you stay in the hospital? [Interviewer note: keep the discussion focused on their hospital admission before the site visit]

1. [If experienced a crisis prior to [demonstration date], ask the following questions; otherwise, skip to the next section]: How does this length of stay compare to the time when you were hospitalized for a psychiatric emergency prior to [demonstration date]?

Discharge Planning

7. When patients are ready to leave the hospital, the hospital may give them instructions about what to do after leaving the hospital. This is called a discharge plan. Sometimes it includes instructions about which medications to take, when to see the doctor, or where to go if you have questions or need help. Did you receive instructions like this before you left [IMD]? [Interviewer note: keep the discussion focused on their hospital discharge before the site visit]

1. If so, did the instructions seem to cover all of your questions or concerns? Was there anything you wished was in the instructions but wasn’t? Did anyone talk to you about your preferences and goals when developing the discharge plan? Did you feel that staff listened to you?

2. Did you feel that you were ready to leave the hospital when you were discharged? Why or why not?

3. Where did you go after you were discharged from the hospital? How did you get there?

4. What kinds of services or support did you receive after you left the hospital?

5. Did the instructions you received give you enough information?
   (Probe: Too much or too little information; was it clearly written or did it use a lot of medical words?)

6. Were you offered resources or techniques that you could use after discharge to help you manage uncomfortable feelings? If so, please describe.

7. [If experienced a crisis prior to [demonstration date]]Did the services or support you received after you left the hospital seem different from what you received when you left the hospital back in [date of crisis prior to demonstration]?

Closing/Follow-Up

That completes the questions we have for you today. [If there is remaining time: Is there anything we should have asked about but didn’t? Do you have anything else you would like to tell us, or questions you would like to ask us?] I’d like to give you the phone number for the crisis hotline so that you can contact someone who can help you if, for any reason, you feel upset after ending the call with us. Do you have something to write it down? [wait until they are ready or, if no writing implement say “It’s pretty easy to remember—it’s 1-800-273-TALK,” skipping saying the numbers.] It’s 1-800-273-8255. It’s pretty easy to remember if you need it because it spells out 1-800-273-TALK.

I also just want to make sure that the information I have is correct so that I can send you a check in appreciation for your completing the interview. [Go over spelling of name, address, and, if relevant, fiduciary guardian information]. OK, so we will process this as soon as possible to get you your check [if respondent wants to know when they will receive the check say “you should receive the check in about 6 weeks”]. Thank you so much for taking the time to speak with us - we really appreciate and value your input.

Post-Interview Notes and Impressions

[Interviewer use this space to document additional information such as reasons why the beneficiary did not complete the interview, questions the beneficiary asked that you could not answer, observations regarding accuracy of responses, or anything else that could be of importance]

Responding to Beneficiary in Crisis during the Interview

Situation	Interviewer Action	Follow-Up
Consumer becomes upset/agitated	Pause to let the consumer collect their thoughts. Ask, as needed: Are you alright? Would you like to continue? Would you prefer I call back at another time? Provide crisis hotline number in case consumer experiences distress after the call: 1-800-273-TALK (1-800-273-8255)	Use the “Post-Interview Notes” section in the interview guide to describe this interaction and the resolution. Use the interview tracking document on the secure N drive to indicate partially completed interview and whether/when interview was rescheduled.
Consumer is a danger to him/herself (expresses a plan to harm him/herself or others)	Terminate the interview using the following script: Let’s stop the interview and I’d like to give you the phone number for the crisis hotline so that you can talk to someone and get help. The phone number is 1-800-273-TALK (1-800-273-8255). I’m going to hang up the phone now so that you can call the hotline number, it’s 1-800-273-TALK. Thank you for talking with me today, take care.	Inform Crystal Blyler (Project Director) and Bonnie O’Day (Qualitative Team Lead) of this event. The team will debrief. Crystal (202) 250-3502 Bonnie O’Day (202) 264-3455

B ENEFICIARY INTERVIEW CONSENT FORM

The Centers for Medicare & Medicaid Services (CMS) is sponsoring a study called the Medicaid Emergency Psychiatric Demonstration (MEPD). The study will look at expanding Medicaid coverage to include psychiatric inpatient services to adults experiencing psychiatric emergencies.

As part of the study, CMS wants to learn about your recent experiences in the emergency room and with the hospital admission and discharge processes. CMS would also like to learn how these experiences compare with your previous hospitalizations for psychiatric emergencies.

Mathematica Policy Research is an independent research company hired by CMS to conduct the study. Mathematica is a leader in policy research and has been conducting studies about health for more than 40 years. You can learn more about Mathematica by visiting its website at http://www.mathematica-mpr.com.

Your participation is completely voluntary, but very important. If you would like to participate, a study team member from Mathematica may call you to set up a time that is convenient for you to participate in a 30- to 60-minute interview over the telephone. Because Mathematica will randomly select individuals to participate in this study, there is a chance that you will not be selected.

If you are selected for an interview, to the extent permitted by law, your answers will be kept private and secure; that is, your information will be used only for this study, and your name will not be associated with your answers. If you are comfortable with it and give the interviewer permission, the interview will be audio taped to ensure that the interviewer remembers correctly everything said during the interview. No one will listen to the audio tape except the Mathematica study team members who transcribe it (that is, the person[s] who writes down what was said on the audio tape) and who check to make sure that the written notes are accurate. The audio tape will be destroyed after the contents are transcribed, no later than 90 days after the interview. You may request to listen to the audio tape before it is destroyed. If you are not comfortable having the interview audio taped, the interviewer will conduct the interview without taping it; instead, notes will be taken about your answers. The written version of the interview and interviewer notes will be kept in a secure study-specific electronic folder to which only a few members of the Mathematica study team who need to use them for study purposes will have access.

Your decision to participate in the study will not change any of your Medicaid benefits or any other benefits you currently receive or may qualify for in the future. As a token of appreciation, you will receive a $20 check for participating in the interview.

If you would like to be part of the study, please review the information on the reverse side of this form. Print your name and telephone number in the spaces provided so a member of the Mathematica study team can call you to schedule a time to talk to you. You will receive a copy of this form for your records.

For more information about the study, please call Amy Overcash at Mathematica Policy Research at (609) 750-2009.

Signature and Contact Information

• I understand I have been invited to take part in an interview about my recent experiences in the emergency room and with the hospital admission and discharge processes.

• I have read the information on this form, or someone read it to me.

• I understand that I do not have to take part in the study.

• I understand that, if I am comfortable with it and give the interviewer permission, the interview will be audio taped.

• If I am not comfortable having the interview audio taped, the interviewer will conduct the interview without taping it and notes will be taken instead.

• I give the study team from Mathematica Policy Research permission to call me at the telephone number provided, if I am selected to participate.

• I may change my mind and take back my permission at any time.

• If I take back my permission, the Mathematica study team will not pursue an interview with me.

Signature Date

Print Name

Telephone Number

Email Address

Witness Date

If the beneficiary has a legal guardian and cannot legally provide consent, the guardian must sign below; the beneficiary must also sign above to indicate his or her agreement to participate.

If the beneficiary can legally provide consent but has a financial guardian, please provide the financial guardian’s contact information below so that the study team can contact him or her to make arrangements for the $20 payment for the beneficiary’s participation in the interview. Note that the financial guardian does not have to sign the form unless he or she also serves as the guardian for personal decision-making purposes.

Guardian’s Signature Date

Print Name

Telephone Number

Relationship to Beneficiary

Email Address

Please indicate whether guardianship pertains to financial or personal decision-making purposes. If both apply, please check each line.

Financial Decisions _________

Personal Decisions _________

RECRUITMENT SCRIPT FOR STAFF MEMBER TO READ TO BENEFICIARY⁰ BEFORE DISCHARGE

[Mr./Ms./Mrs.] [Fill in name],

I would like to see if you are interested in participating in an interview about your experiences in the emergency room and with the admission and discharge processes at this hospital. The interview is part of a study that the Centers for Medicare & Medicaid Services (or CMS) is sponsoring to learn more about inpatient psychiatric treatment. The study is called the Medicaid Emergency Psychiatric Demonstration. The information you provide about your experiences may help others in the future. An interviewer from the study team would like to talk to you over the telephone in the next few weeks, at a time that is convenient for you.

If you want to participate, all you have to do is provide a phone number at which you can be reached. Someone from the study team may call you to schedule a time that is convenient for you to talk. The team will select people randomly so there is a chance you will not be called. If you are selected for an interview, to the extent permitted by law, your answers will be kept private and secure; that is, your information will be used only for this study, and your name will not be associated with your answers. Your decision to participate in the study will not change any of your Medicaid benefits or any other benefits you currently receive or may qualify for in the future.

STAFF MEMBER, HAND FACT SHEET TO BENEFICIARY AND SAY: This sheet provides information about the study.

Do you think you might like to participate?

YES STAFF MEMBER, TURN PAGE OVER AND FOLLOW INSTRUCTIONS

NO STAFF MEMBER REPLY TO BENEFICIARY: Thank you for your consideration.

STAFF MEMBER INSTRUCTIONS: IF RESPONDENT ANSWERED “YES”, PLEASE FOLLOW THESE INSTRUCTIONS:

1. Read the consent form to the beneficiary, or ask beneficiary to read the consent form.

• If the beneficiary agrees to participate in the study, ask the beneficiary to read the consent form. Print the beneficiary’s name, phone number, and email address on the consent form, and have the beneficiary sign and date the consent form. Ask the witness (this might be you) to sign and date the consent form.

• If the beneficiary does not have a personal phone (home, work, or cell phone), inquire about other phones the beneficiary might use or have access to—for example, a phone belonging to a relative or someone the beneficiary lives with.

• If the beneficiary agrees to participate and cannot legally provide consent on his or her own behalf, but has a legal guardian, please obtain consent, a signature, and contact information from the guardian.

• If the beneficiary can legally provide consent but has a financial guardian, please obtain the financial guardian’s contact information so that the study team can contact him or her to make arrangements for the $20 payment (check) for the beneficiary’s participation in the interview. Note that the financial guardian does not have to sign the form unless he or she also serves as the guardian for personal decision-making purposes.

2. Tell the beneficiary that someone from Mathematica Policy Research may call him or her in a few weeks to schedule an interview at a convenient time.

3. Give the beneficiary a copy of the consent form. If the beneficiary has questions about the study, refer him or her to the fact sheet and/or the Mathematica contact person listed on the consent form.

Attachment F

Conceptual Framework

Conceptual Framework for Understanding the Goals and Objectives of the Medicaid Emergency Psychiatric Services Demonstration (MEPD)

Key to developing an effective evaluation design is a clear understanding of the goals and objectives of the demonstration. As depicted in our conceptual framework, the demonstration is aimed at reducing a number of undesirable aspects of the current system of care for psychiatric emergencies by increasing the use of private IMDs. The typical path for Medicaid beneficiaries with psychiatric EMCs in the current system begins in a medical emergency room (ER). Once the ER determines that the beneficiary is in need of inpatient services, the search for an available inpatient bed begins. The lack of available beds often leads to long periods of boarding in the ER (depicted by the wide red bar) or inappropriate placement in available beds scattered throughout general hospital medical units. Stabilization in such units may take longer than it would if more appropriate care was provided, leading to higher costs. Discharge planning by non-specialized staff may result in lower quality placements. Inadequate care following a discharge that occurs before the beneficiary is fully stabilized can result in readmission to the ER and a recurrence of the cycle. The MEPD seeks to break this cycle by increasing the use of private IMDs. Increased availability of beds in these specialized facilities would be expected to decrease both the time spent in ERs awaiting inpatient services and inappropriate placements in general medical units. Receipt of specialized care may be expected to decrease the time needed for stabilization and increase time spent on and quality of discharge planning which, in turn, would be expected to result in better quality post-discharge care and a reduction in the need for readmission. Decreased use of ERs and stabilization times, along with reduced use of inpatient care due to readmissions, could result in net savings to overall Medicaid costs.

Attachment G

Technical Expert Panel

Technical Expert Panel

Name	Organization	Expertise
Michael H. Allen	Professor of Psychiatry and Emergency Medicine, University of Colorado School of Medicine; Director of Research, University of Colorado Depression Center; Senior Investigator, Veterans Integrated Services Network 19 Mental Illness Research Education and Clinical Center	Emergency psychiatry research
Alisa Busch	Director of Integration of Clinical Measurement and Health Services Research at McLean Hospital; Chief, Health Services Research Division, Partners Psychiatry and Mental Health, a division of Partners HealthCare; Associate Professor of Psychiatry and Health Care Policy, Harvard Medical School	Psychiatry, quality of care, health services research
Richard Dougherty	Chief Executive Officer, DMA Health Strategies	Mental health and Medicaid policy and systems
Jonathan Edwards	Director of Peer Counseling, Division of Wellness, Recovery, and Community Integration, Kings County Hospital Center, New York, NY	Consumer perspectives on emergency and inpatient services, mental health recovery and service delivery systems
Karen Johnson	Senior Vice President of Clinical Services, Behavioral Health Division, Universal Health Services, Inc.	IMDs across many states
Theodore Lutterman	Director of Research, National Association of State Mental Health Program Directors Research Institute	Data systems of state mental health authorities
Kathleen McCann	Director, Quality and Regulatory Affairs, National Association of Psychiatric Health Systems	IMDs
Steve Sharfstein	President and Chief Executive Officer, Sheppard Pratt Health System; Clinical Professor and Vice Chair of Psychiatry, University of Maryland	IMDs, public mental health policy
Laura van Tosh	Independent Consultant; Former Director of Consumer Affairs, Western State Hospital, Washington; Former Consumer Affairs Coordinator, Greater Oregon Behavioral Health Care, Inc.	Consumer perspectives on inpatient care, mental health policy and program development

Attachment H

Confidentiality Pledge

C ONFIDENTIALITY PLEDGE

I understand that the names, and any other identifying facts or information, of individuals, businesses, organizations, and families participating in projects conducted by Mathematica, Inc. or its subsidiaries are confidential information. I agree that I will not reveal such confidential information, regardless of how or where I acquired it, to any person unless such person has been authorized by the cognizant Mathematica Project Director or the Mathematica Project Manager to have access to the information.

I further understand that the unauthorized access to, use, or disclosure of any confidential information is a breach of the terms of my employment, or my consultant agreement with Mathematica and may subject me to court action by any interested party or to other sanctions by Mathematica. I acknowledge that this agreement shall continue to bind me even after the project(s) is (are) completed and/or even though my employment or my consultant agreement with Mathematica has terminated.

In addition, in the course of my employment I may have access to personal information, electronic and otherwise, about fellow employees. I agree that I will treat that information as having the highest confidentiality, and not communicate it to fellow employees or others outside Mathematica. Final determination of whether or not there is a business purpose requiring that I access a fellow employees’ records will be made in consultation with the Director of Human Resources. Failure to uphold this standard is a breach of trust and may subject me to disciplinary action, including termination of employment.

Other than in the course of my authorized employment or my consultant agreement, I further agree that I will not use, nor facilitate the use by any third party, in any way any information deemed confidential by the terms of any contract or other written agreement between Mathematica and any other organization, except by written authorization by both parties. It is my understanding that Mathematica and the contracting organization(s) have the exclusive right to all information acquired or developed under such a contract or other written agreement. I acknowledge that I acquire no right, title, or interest in and to any data or information to which I have access by reason of my employment or my consultant agreement and that I may not remove such data from my assigned work location without prior authorization.

I agree to promptly notify the cognizant Mathematica Project Director or Project Manager, the Survey Operations Center Manager or Supervisor for survey work, and the Mathematica Security Officer of any unauthorized disclosure, use, or alteration of confidential information that I observe.

Nothing herein shall be construed to prevent divulgence of information to any court or governmental agency, provided such divulgence is required by law. However, if I am subpoenaed, or if I have reason to believe that I may be called upon to make such divulgence, I agree to notify the President of Mathematica promptly in writing and, upon his request, to cooperate in all lawful efforts to resist such divulgence.

Name: ______________________________ Signature: ______________________________

Date: ______________________________

Attachment I

QUALITATIVE DATA CODING SCHEME

Medicaid Emergency Psychiatric Demonstration Evaluation

Document Families and Code List

Document Families

Document Families	DEFINITION
State	All primary documents should belong to a “State” family
Alabama
California
Connecticut
District of Columbia
Illinois
Maine
Maryland
Missouri
North Carolina
Rhode Island
Washington
West Virginia
Multi-state
FMAP	All primary documents should belong to a “FMAP” family
FMAP 50%	State has a federal Medicaid matching rate of 50%
FMAP 51-60%	State has a federal Medicaid matching rate between 51% and 60%
FMAP 61-75%	State has a federal Medicaid matching rate between 61% and 75%
Document Type	All primary documents should belong to a “Document Type” family
Report	Report submitted by the State to CMS. Includes proposals, State Operational Plans, and IMPAQ progress reports
MEPD PD interview	State or county demonstration staff interview
IMD POC interview	Interview with the point of contact at the participating IMD
IMD front interview	Interview with frontline staff at the participating IMD
IMD admin interview	Interview with administrator at the participating IMD
ER front interview	Interview with frontline staff at the emergency room
ER admin interview	Interview with administrator at the emergency room
GH front interview	Interview with frontline staff at the GH
GH admin interview	Interview with administrator at the GH
UR interview	Interview with Utilization Review Vendor or ASO staff
Beneficiary interview	Beneficiary interview
Site Visit Round	Interview notes should belong to a “Site Visit Round” family
Round 1
Round 2
IMD Size	IMD interviews and Medical record reviews should belong to a “IMD Size” family
Beds: 17-25	IMD with 17 to 25 beds
Beds: 26-50	IMD with 26 to 50 beds
Beds: 51-100	IMD with 51 to 100 beds
Beds: 101-200	IMD with 101 to 200 beds
Beds: 201-300	IMD with 201 to 300 beds
Beds: 301+	IMD with 301 or more beds

Code List

CODE	DEFINITION
Respondent Context
Responsibilities	Interviewee’s job duties and/or role (use sub-code for MEPD specific responsibilities)
Responsibilities: MEPD	MEPD specific job duties or role
IMD	IMD context needed to understand the environment the demonstration is operating within (use sub-codes when possible)
IMD: Characteristics	Describes characteristics of participating IMDs (e.g., ALOS, daily census, number of beds, payer mix, type of hospital )
IMD: Outpatient	Describes IMD outpatient or step-down services that patients might be discharged to
IMD: QI other	Non-demonstration quality improvement activities driven by the hospital Do NOT code federal, state, or regional activities
IMD: Patient	Describes characteristics of patients at IMDs (e.g., physical comorbidities, substance use, residence) Do NOT code for consumer description of their own characteristics
ER	Emergency room context needed to understand the environment the demonstration is operating within (use sub-codes when possible)
ER: Characteristics	Describes characteristics of emergency rooms (e.g., boarding times, time spent in waiting rooms, number of beds, payer mix, presence of inpatient psychiatric unit for in-house transfer)
ER: Patient	Describes characteristics of patients at ERs (e.g., physical comorbidities, substance use, insurance status, residence)
GH	General hospital context needed to understand the environment the demonstration is operating within (use sub-codes when possible)
GH: Characteristics	Describes characteristics of general hospitals (e.g., boarding times, number of beds, payer mix)
GH: Patient	Describes characteristics of patients at ERs (e.g., physical comorbidities, substance use, insurance status, residence)
Beneficiary	Beneficiary descriptions of themselves during beneficiary interviews (e.g., physical comorbidities, substance use, insurance status, residence)
State Context
State	State environmental factors including attitudes, geography (urban, rural), and income. (Use sub-code if possible)
State: Event	A natural disaster or other crisis in the state that may impact the delivery, financing, or utilization of mental health services. Events include the Sand Hook shooting and Hurricane Sandy.
State: Politics	Elected officials and political appointees, state budget issues
State: Policies	Legislation or regulation that is proposed or passed
MH Sector	Mental health system context needed to understand the environment the demonstration is operating within. (Use sub-codes if possible.)
MH Sector: Payment	Funding of mental health services in the state. Includes funding of Medicaid stays at IMDs prior to the demonstration and reimbursement for other level of care mental health services under Medicaid (fee for service versus managed care, types of covered services, payment rates)
MH Sector: QI-other	Non-demonstration quality improvement activities at the federal, state, or regional level, including initiatives to reduce emergency room readmissions and to integrate primary and behavioral health care
MH Sector: Demand	Demand for inpatient psychiatric services before the demonstration and changes in demand during the demonstration
MH Sector: Beds	Availability of inpatient psychiatric beds. Include discussion of bed shortages and emergency room and general hospital psychiatric boarding.
MH Sector: Outpatient	Availability and characteristics of community-based services including mental health clinics, intensive outpatient programs (IOPs), partial hospitalization programs, residential programs, and group homes
MH Sector: Workforce	Availability of licensed mental health providers including psychologists, psychiatrics, social workers, advanced practice nurses, and registered nurses
PH Sector	Physical health system context needed to understand the environment the demonstration is operating within. Efforts could include medical home demonstrations or state health IT activities. Do NOT code for mental health system context.
Processes and Procedures
New	State or providers implemented a new process or changed existing procedures. (Double-code with relevant process or procedure code)
Payment	Medicaid payments for services provided under the demonstration. (Use sub-codes if possible.)
Payment: IMDs	Medicaid reimbursement to IMDs for services provided under the demonstration. Include discussion of new or adapted systems for processing payments.
Payment: State	CMS provision of federal matching funds to states for services provided under the demonstration. Include discussion of changes to MMIS systems.
Outreach	Efforts to encourage participation in the demonstration. (Use sub-codes if possible.)
Outreach: Counties	State-level outreach to counties or regions. Include descriptions of outreach activities and reasons for targeting particular counties.
Outreach: IMDs	State or county outreach to IMDs. Include descriptions of how IMDs were selected and outreach activities to IMDs.
Outreach: Referral	Outreach to referral providers, such as emergency departments at general hospitals, community-based providers, or crisis stabilization services. Include descriptions of outreach activities and reasons for targeting particular referral providers. (Use sub-codes if possible
Outreach: Referral from State	State or county outreach to referral providers.
Outreach: Referral from IMD	IMD outreach to referral providers.
Eligibility	Discussion of eligibility criteria for the demonstration (Use sub-codes if possible)
Eligibility: Condition	Discussion of the change in eligibility criteria to allow beneficiaries who are a danger to self or others but not homicidal or suicidal to participate
Eligibility: Enrollment	Discussion of the change in eligibility criteria to allow for individuals who are eligible for Medicaid but not enrolled to participate
Referral	Process for referral and admission under the demonstration. Includes description of pre-authorization procedures. (Use sub-code if process was changed for the demonstration)
Stabilization	Process for stabilization and ongoing/concurrent authorization for patients under the demonstration. (Use sub-code if process was changed for the demonstration)
Discharge	Description of discharge planning activities that occur during the inpatient stay. Do NOT code for activities that occur post-discharge. (Use sub-code if process was changed for the demonstration)
Post-discharge	Post-discharge activities including after-care placements and follow-up care with community-based providers. (Use sub-code if process was changed for the demonstration)
Oversight	State oversight of stabilization, discharge planning, and/or post-discharge requirements. Oversight may be completed by a state agency, case manager, utilization review vendor, or behavioral health organization. (Use sub-code if process was changed for the demonstration)
Stakeholders
Consumer	Consumer involvement in treatment or discharge planning procedures. (Double-code with process or procedure code)
Social support	Family or other social support network involvement in during treatment, discharge planning, or post-discharge procedures. (Double-code with process or procedure code)
Peer support	Peer-support provided during treatment, discharge planning, or post-discharge. (Double-code with process or procedure code)
Community	Involvement of community-based providers or resources (e.g., social workers, case workers, psychologists) during treatment,, discharge planning, or post-discharge procedures. (Double-code with process or procedure code)
Collaboration	Collaboration between the State, IMDs, and/or community-based services to plan or implement the demonstration. (Double-code with process or procedure code)
Experiences in Demo	Double-code with other codes
Unexpected Demo Changes	State or IMD is doing something different than they had originally planned (e.g., adjusting strategies, timelines, or turnover in demo leadership)
Facilitators	Things that went well; positive factors that are helping the demonstration succeed. Do NOT code facilitators of non demonstration activities.
Barriers	Things that haven’t gone well; negative factors/challenges; how overcame them; Do NOT code barriers to non demonstration activities.
Good quote	A good quote or example that nicely illustrates a point; a key insight
Outcomes	Double-code with other codes
Goals	Goals of the MEPD program
Out	Things that happened as a result of demonstration (Use sub-codes if possible.)
Out: Access	Change in access to inpatient psychiatric care. Includes description of the change in use of IMDs
Out: Continuity	Change in the continuity of care for beneficiaries in need of inpatient psychiatric care. Includes discussions of beneficiaries receiving inpatient care closer to their home and/or outpatient services
Out: ER boarding	Change in psychiatric boarding at emergency rooms
Out: GH boarding	Change in psychiatric boarding at general hospitals
Out: Quality	Change in quality of care provided by IMD
Out: ALOS	Change in average length of stay at the IMD
Out: Readmission	Change in readmission rate for psychiatric emergencies
Out: Health	Change in beneficiary health or functional status
Out: Costs	Change in cost of inpatient or outpatient services (Use sub-codes if possible.)
Out: Costs: State	Change in costs incurred by the state
Out: Costs: IMD	Change in costs incurred by the IMD
Out: Costs: Community	Change in costs incurred by community-based providers and referral providers
Out: Satisfaction	Change in beneficiary satisfaction with inpatient psychiatric care. Includes changes in satisfaction with treatment, discharge planning, and post-discharge procedures.
Other
Other	Use if no suitable code and text seems important for analysis (but it’s OK to not code)

Attachment J

60 Day Federal Register Notice

60-Day Federal Register Notice will be inserted here in the final PDF

Attachment K

Response to Public Comments

CMS Response to Public Comments Received for CMS-10487—Medicaid Emergency Psychiatric Demonstration (MEPD) Evaluation

The Centers for Medicare and Medicaid Services (CMS) received comments from two mental health services researchers. Both offered support for the evaluation, the use of mixed-methods techniques, and the use of interviews to obtain feedback from service users with mental illness. This is the reconciliation of the comments.

Comment:

Both of the comments that CMS received suggested that interviews and focus groups be used to obtain the opinions of service users with mental illness and (one of the commenters added) their families.

Response:

CMS appreciates the importance of obtaining the perspectives of Medicaid beneficiaries with mental illness on their experiences with the services provided under the demonstration. Therefore, after each site visit, the evaluation contractor will conduct telephone interviews with five beneficiaries who have been recently discharged from each participating IMD. We opted for individual telephone interviews with Medicaid beneficiaries rather than focus groups for several reasons, including (1) the significant logistical difficulty in bringing beneficiaries together (including coordinating transportation for potential participants across large service areas) and (2) potential reluctance of beneficiaries to honestly discuss their experiences in a group setting. We opted not to speak with family members due to further logistical difficulties involved in obtaining written permission and the needed information from beneficiaries to be able to contact and interview family members about the beneficiary’s psychiatric hospitalizations. Moreover, the beneficiaries themselves will be in the best position to report on their experiences with services provided under the demonstration and how the quality of such services compares to any services received for psychiatric emergencies before the demonstration.

Comment:

One of the comments that CMS received further suggested that we obtain beneficiary perceptions of the quality of their linkage with community-based outpatient mental health, rehabilitation, and peer support services upon discharge as well as their ability to avoid re-entering inpatient settings within the 30-90 days following discharge.

Response:

The interview guide for conducting interviews with Medicaid beneficiaries contains a section on discharge planning. This section of the interview provides the opportunity for beneficiaries to comment on the discharge plan, whether their goals and preferences were included in discharge planning, and the extent to which they were linked with doctors and other service providers, such as outpatient mental health and other programs. The interviewer will ask whether the beneficiary received information about where to go if he or she needs help managing uncomfortable feelings. The interviewer will also ask what services the beneficiary actually received upon discharge. These open-ended questions provide the opportunity for beneficiaries to describe the quality of their linkage with services upon discharge.

1 We will begin discussions with the states and facilities regarding any participant protection requirements well in advance of the site visit to ensure adequate time to obtain internal review board or other approvals that may be necessary.

2 If the IMD was not serving Medicaid beneficiaries prior to the demonstration, we will ask to review records of non-Medicaid beneficiaries experiencing a psychiatric emergency.

0 Please seek consent only from Medicaid beneficiaries receiving services as a result of a psychiatric emergency through the Medicaid Emergency Psychiatric Demonstration.

5