CMHS Supporting Statement A (SSA)_11-17-2021

SUPPORTING STATEMENT FOR

Center for Mental Health Services (CMHS)

MENTAL HEALTH CLIENT/CONSUMER OUTCOME MEASURES

AND

INFRASTRUCTURE, PREVENTION, AND PROMOTION INDICATORS

JUSTIFICATION

A1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA) is requesting approval from the Office of Management and Budget (OMB) for revisions to the previously approved instruments and data collection activities for the Government Performance and Results Act (GPRA) Center Mental Health Services (OMB No. 0930-0285) that expires on February 28, 2022.

To be fully accountable for the spending of federal funds, SAMHSA requires all programs to collect and report data to ensure that program goals and objectives are met. Data is collected and used to monitor and improve performance of each program and ensure appropriate and thoughtful spending of federal funds.

SAMHSA is requesting the following changes:

• Merge the CMHS NOMS Child Client-Level Measures for Discretionary Programs data collection instrument with the CMHS NOMS Adult Client-Level Measures for Discretionary Programs data collection instrument, resulting in one NOMS Client-Level Measures data collection instrument.

• Significantly revise the NOMS Client-Level Measures for Discretionary Program data collection instrument to reduce grantee burden and ensure that program goals and objectives are met.

• Revise the Infrastructure, Prevention, and Mental Health Promotion indicators to delete measures that are not being used, refine measures for improved clarity, and add measures to reflect program development and refinement during the past three years.

A broad summary of the changes are as follows:

1. The CMHS NOMs Adult Client-level Measures for Discretionary Programs Providing Direct Services (Attachment A)

Revisions:

• Merged the CMHS NOMS Child Client-Level Measures for Discretionary Programs data collection instrument with the current Adult tool.

• Deleted questions for data not being utilized for program monitoring and quality improvement.

• Reduced grantee burden by shifting questions with a five-point psychometric response scale to “Yes”, “No”, or “No Response”.

• Modified ICD-10 diagnoses to expand F40-48, F60-63, and F90-99 codes to allow for more specificity. Also, ICD-10 “Z” codes were added to allow for a focus on social determinants of health that may affect the diagnosis, course, prognosis, or treatment of a client’s/consumer’s mental disorder.

• Shifted reporting NOMS data to baseline assessment, 3- month or 6-month reassessment, and a final clinical discharge assessment. CMHS grant programs will elect to conduct a 3-month OR 6-month reassessment based on the average length of stay for that program

• Reduced the number of physical health indicators and reporting frequency from quarterly to three points in time – baseline, either 3- or 6-month reassessment, and clinical discharge. The intent is to further reduce grantee burden

2. The CMHS NOMs Child Client-level Measures for Discretionary Programs Providing Direct Services (Child/Caregiver Version)

• Merged with the Adult client-level performance tool and will no longer be used by grantees as a stand-alone data collection tool.

3. The Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators (Attachment B and Attachment C).

Attachment B identifies the recommended deletions, additions, and modifications to existing indictors. Attachment C is the revised IPP form. The proposed changes are as follows:

• Delete four measures not used by any CMHS grant programs

  • WD1: The number of organizations or communities implementing mental health-related training programs as a result of the grant.

  • WD4: The number of changes made to credentialing and licensing policies in order to incorporate expertise needed to improve mental health-related practices and activities.

  • F1: The amount of additional funding obtained for specific mental-health related practices/activities that are consistent with the goals of the grant.

  • O2: The total number of contacts made through program outreach efforts.

• Revise two measures (A3 and A1) to provide more clarity

  • A3: The number of grant activities in which fidelity is monitored as a result of the grant.

  • A1: The number of communities that enhance health information-sharing for provision of services between agencies and programs.

• Add eight measures (R2, R3, S2, S3, T5, T6, T7, and T8) to reflect program developments during the past three years.

  • R2: The number of individuals referred to trauma-informed care services as a result of the grant.

• R3: The number of individuals referred to crisis or other mental health services for suicidality.

• S2: The number of individuals screened for trauma-related experiences as a result of the grant.

• S3: The number of individuals screened for suicidal ideation as a result of the grant.

• T5: The number of activities modified, adapted, and/or change to reflect trauma-informed practices to the population(s) being served.

• T6: The number of activities modified, adapted, and/or changed to reflect culturally-appropriate practices to the population(s) being served.

• T7: As a result of the grant, the percentage of individuals who died by suicide.

• T8: As a result of the grant, the percentage of individuals who attempted suicide.

These changes will lessen grantee burden with data collection and improve capacity to report qualitative performance and quantitative outcomes for all discretionary grant programs, including: demographic characteristics of individuals served; clinical characteristics of individuals served before, during, and after receipt of services; numbers of individuals served; and characteristics of services and activities provided.

Approval of this information collection will allow SAMHSA to continue to meet Government Performance and Results Modernization Act of 2010 (GPRMA) reporting requirements that quantify the effects and accomplishments of its programs, which are consistent with OMB guidance.

SAMHSA’s legislative mandate is to increase access to high quality substance abuse and mental health prevention and treatment services and to improve outcomes. Its mission is to improve the quality and availability of treatment and prevention services for substance abuse and mental illness. To support this mission, the Agency’s overarching goals are:

• Accountability—Establish systems to ensure program performance measurement and accountability

• Capacity—Build, maintain, and enhance mental health and substance abuse infrastructure and capacity

• Effectiveness—Enable all communities and providers to deliver effective services

Each of these key goals complements SAMHSA’s legislative mandate. All of SAMHSA’s programs and activities are geared toward the achievement of these goals, and GPRA performance monitoring is a collaborative and cooperative aspect of this process.

This request represents another stage in SAMHSA’s continuing efforts to improve its ability to assess the impact of its programs, and to use data collected from its discretionary grant portfolio to enhance grantee performance and to improve the lives of Americans with mental health and substance use disorders. The current request seeks approval to revise the current data collection tools as summarized above.

A2. Purposes and Use of Information

All SAMHSA/Center for Mental Health Services grant programs collect and report Infrastructure, Prevention, and Mental Health Promotion data from discretionary grant programs. A sub-set of CMHS discretionary grant programs that provide direct client treatment services also collects and reports client-level performance measures to measure the performance outcomes of these grants. Attachment E is a listing of current CMHS grant programs, identifying the types of data collected.

These data are used by individuals at three different levels within SAMHSA: the Assistant Secretary of Substance Abuse and Mental Health Services; the Center-level administrators, respective leadership teams, and Program Staff (i.e., Government Project Officers [GPOs]); and grantees.

Assistant Secretary Level: The information informs the Assistant Secretary for Mental Health and Substance Use on the performance and outcomes of the all funded programs. Performance is linked to the goals and objectives of all grant programs. The intent is for this information to serve as the basis of the annual GPRA report to Congress contained in the Justifications of Budget Estimates.

Center Level Administrators, Leadership Teams, and Government Project Officers Level: In addition to providing information about the performance of the various programs, the information is used to monitor and manage individual grant projects within each program. The information is also used by GPOs to identify program strengths and weaknesses, provide an informed basis for providing technical assistance and other support to grantees, inform continuation of funding decisions, and identify potential issues for additional evaluation.

Grantee Level: In addition to monitoring performance and outcomes, the grantee staff can use the information to improve the quality of services provided to their selected population(s) of focus within their projects.

SAMHSA uses the data collected for annual reporting required by the Government Performance and Results Modernization Act (GPRMA) of 2010. SAMHSA’s report for each fiscal year will include results of performance monitoring for the three preceding fiscal years. The information collected through this new data collection process will allow SAMHSA to report on the results of these performance indicators and outcomes in a manner that is consistent with SAMHSA specific performance domains, and to assess the accountability and performance of all discretionary grant programs.

CMHS response to 60-day Federal Register Notice

In response to the 60-day FRN, SAMHSA/CMHS received correspondence and inquiries from forty organizations (refer to Attachment C: CMHS/GPRA Public Comment Response Matrix). In response to these comments, three changes were made to the proposed data collection tools: (1) IPP T7: As a result of the grant, reduce the percentage of individuals who died by suicide; (2) IPP T8: As a result of the grant, reduce the percentage of individuals who attempted suicide; and (3) for the NOMS, change response options to “Yes”, “No”, or “No response”.

Data Collection for Client-level National Outcome Measures

Client-level national outcome measures data reflect the Agency’s desire for consistency in data collected within the Agency. SAMHSA has identified nine specific performance domains to assess the accountability and performance of its discretionary and formula grants. These domains represent SAMHSA’s focus on the factors that contribute to the success of mental health treatment, and are:

• Functioning

• Stability in Housing

• Education and Employment

• Crime and Criminal Justice

• Perception of Care

• Social Connectedness

• Reassessment Status

• Program-specific measures

• Services Received and Clinical Discharge Status

SAMHSA/CMHS services grant programs that provide direct treatment to consumers currently have an OMB approved data collection tools in place. This request for approval is for a revision to the existing CMHS NOMS Adult Client-level Measures for Discretionary Programs Providing Direct Services. In summary, we have revised this tool to include measures for both children and adults and are dropping “Adult” from the title of the tool to indicate that the tool will be used for all clients/consumers, regardless of age.

We are also proposing for data collection of performance and health data to occur at three points of time: client entry into the program (baseline assessment); reassessment at either three- or six- months depending on the average length of stay for each program; and at the completion of client’s treatment (clinical discharge assessment). This is a change as performance data is currently collected at baseline, every six months, and at clinical discharge. Physical health data is currently collected and reported every three months and we are now proposing to collect physical health data at the time of the baseline assessment, reassessment, and clinical discharge assessment.

Data Collection for Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators

To facilitate CMHS reporting of GPRA data for programs engaged in substantial infrastructure development, prevention, and mental health promotion activities, the agency has identified 13 categories of interest for accountability and performance monitoring. The categories are as follows:

• Accountability

• Access

• Awareness

• Financing

• Knowledge, Attitudes, and Beliefs

• Outreach

• Organizational Change

• Partnership and Collaboration

• Policy Development

• Referral

• Screening

• Training

• Workforce Development

Table 1. summarizes the total number of indicators for each category that may or may not apply to each grant program depending on the type of grant and program goals:

Table 1. Data Collection for Infrastructure, Prevention, and Mental Health Promotion Indicators

Category	Current # Indicators	Proposed # Indicators
Accountability	6	6
Access	1	1
Awareness	1	1
Financing	3	2
Knowledge, Attitudes and Beliefs	1	1
Outreach	2	1
Organizational Change	1	1
Partnership and Collaboration	2	2
Policy Development	2	2
Referral	1	3
Screening	1	3
Training	1	4
Types and Targets of Practice	4	8
Workforce Development	5	3
TOTAL	31	38

SAMHSA intends to compare infrastructure, prevention, and mental health promotion targets set at the project period start date with data collected quarterly. These data are used as an indicator of performance

Proposed Changes to NOMS Data Collection Tool

SAMHSA is proposing to revise data collection instrument (OMB No. 0930-0285) to improve performance monitoring and outcome measurement of its programs supporting recovery from mental illness and substance use disorders and to reduce grantee burden.

SAMHSA is requesting approval to modify the current NOMS data collection tool as included as Attachment A. A complete list of all changes can be found in Attachment D.

All CMHS data collection activities are intended to promote the use of consistent measures among CMHS-funded grantees and contractors. These measures are a result of extensive examination and recommendations, using consistent criteria, by panels of staff, experts, and grantees. Wherever feasible, the measures are consistent with or build upon previous data development efforts within CMHS. These data collection activities are organized to reflect and support the domains specified for SAMHSA’s NOMs for programs providing direct services, and the categories developed by CMHS to specify the infrastructure, prevention, and mental health promotion activities.

A3. Uses of Information Technology

Information technology is used to reduce program respondent burden. A web-based data collection and entry system, SAMHSA’s Performance Accountability and Reporting System (SPARS), is currently used and available to all programs for data collection. This web-based system allows for easy data entry, submission, and reporting to all those who have access to the system. Levels of access have been defined for users based on their authority and responsibilities regarding the data and reports. Access to the data and reports is limited to those individuals with a username and password. A screenshot of the data entry screen on SPARS is below:

Electronic submission of the data promotes enhanced data quality. With built in data quality checks and easy access to data outputs and reports, users of the data can feel confident about the quality of the output. The electronic submission also promotes immediate access to the dataset. Once data are entered into SPARS, it is available for access, review, and reporting by all those with access to the system.

A4. Efforts to Identify Duplication

The items collected are necessary in order to assess grantee performance. SAMHSA is promoting the use of consistent performance and outcomes measures across all programs. This effort will result in less overlap and duplication, substantially reducing the burden on grantees that can result from data demands.

A program-level review of current measures and methods of data collection was conducted to identify duplication of these data collection efforts. With the goal of creating standardized indicators and methods for monitoring grantee performance across the Center, existing measures were considered for use where appropriate. The proposed revisions to the data collection instruments were developed in consultation with, reviewed, and approved by Director of the National Mental Health and Substance Use Policy Laboratory and SAMHSA’s Chief Medical Officer as meeting the performance monitoring and management needs of individual programs and the Center.

SAMHSA will work closely with grantees to identify whether other data are being collected by the grantee, which may be redundant to the GPRA instrument. When duplication is identified, SAMHSA and the grantees will identify a priority action plan to leverage the duplicative efforts and streamline the process to reduce client burden.

A5. Involvement of Small Entities

Individual grantees vary from small entities to large provider organizations. Every effort has been made to reduce the number of data items collected from grantees to the least number required to accomplish the objectives of the effort and to meet GPRA reporting requirements. Therefore, there is no significant impact for small entities.

A6. Consequences if Information Collected Less Frequently

NOMS Client-level Data

Mental health programs typically collect client-level data at admission and then conduct periodic reassessments of consumers while the individual remains in treatment. When feasible, mental health providers also conduct an assessment when the consumer is discharged. The data collected for the revised client-level tools parallels this model. All programs that provide direct services will collect data at baseline/intake, at three or six months of project enrollment, and at clinical discharge. Physical health data, which is reported by a limited number of grant programs, will be reported quarterly.

The baseline data collection is critical for measuring change. Extending the interval for the periodic reassessment beyond the requested intervals could lead to loss of contact with consumers, significantly diminishing the response rates and lowering the value of the data for performance reporting use by losing measurement of intermediate effects.

Infrastructure Development, Prevention, and Mental Health Promotion Data

These are no consequences if information is collected less frequently. The quarterly data collection and reporting requirement for the infrastructure development, prevention, and mental health promotion performance indicators provides CMHS with sufficient information for appropriate program monitoring/management and GPRA performance reporting. There is not a need to report data less than quarterly.

A7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

This information collection fully complies with the guidelines in 5 CFR 1320.5(d)(2).

A8. Consultation Outside the Agency

The notice required by 5 CFR 1320.8(d) was published in the Federal Register on August 2, 2021 (86 FR 41492). SAMHSA received 39 public comments from 44 entities.

CMHS consulted both external and internal stakeholders in developing the proposed measures and data collection methodology. CMHS obtained feedback and consultation regarding the availability of data, methods and frequency of collection, and the appropriateness of data elements. Development of the measures involved extensive consultation with staff within CMHS and SAMHSA.

A9. Payment to Respondents

Per SAMHSA policy, grant funds cannot be used to make direct payments to a client/consumer to enter treatment of continue to participate in prevention or treatment services. However, a grantee may provide up to a $30 non-cash incentive to a client/consumer to participate in required data collection follow-up.

A10. Assurance of Confidentiality

The information from grantees and all other potential respondents will be kept private through all points in the data collection and reporting processes. However, SAMHSA cannot ensure complete confidentiality of client data. All data will be closely safeguarded, and no institutional or individual identifiers will be used in reports. Only aggregated data will be reported. SAMHSA and its contractors will not receive identifiable client records. Provider-level information will be aggregated to, at least, the level of the grant funding announcement.

SAMHSA has statutory authority to collect data under the Government Performance and Results Act (Public Law 1103(a), Title 31) and is subject to the Privacy Act for the protection of data. Federally assisted substance abuse treatment providers are subject to the federal regulations for alcohol and substance abuse patient records (42 CFR Part 2) (OMB No. 0930-0092) which govern the protection of patient identifying data. In some cases, these same providers meet the definition of a HIPAA covered entity and are additionally subject to the Privacy Rule (45 CFR Parts 160 and 164) for the protection of individually identifiable data.

A11. Questions of a Sensitive Nature

SAMHSA’s mission is to improve the quality and availability of prevention, early intervention, treatment, and rehabilitation services for substance abuse and mental illnesses, including co-occurring disorders, in order to improve health and reduce illness, death, disability, and cost to society. In carrying out this mission, it is necessary for service providers to collect sensitive items such as criminal justice involvement as well as issues of mental health. The data that will be submitted by each grantee will be based in large part on data that most of the programs are already routinely collecting. This primarily includes data on consumer demographics, mental health condition/illness and treatment history, services received, and consumer outcomes. These issues are essential to the service/treatment context.

Grant projects use informed consent forms as required and as viewed appropriate by their individual organizations. They use the appropriate forms for minor/adolescent participants requiring parental approval. Client data are routinely collected and subject to the Federal Regulations on Human Subject Protection (45 CFR Part 46; OMB No. 0925-0404). Alcohol and drug abuse client records in federally supported programs are also protected by 42 CFR Part 2.

The informed consent forms contain the following elements:

• A description of the purpose of the program and the procedures to be followed.

• The expected duration of client/consumer participation.

• Whether or not client/consumer participation is voluntary. If participation is voluntary, a statement that the client/consumer may receive services even if they do not participate in or complete the data collection component of the program.

• A client/consumer right to leave the project at any time without adverse effect.

• Any possible risks from project participation and the plan(s) to protect the client/consumer from these risks.

• A disclosure of appropriate alternative procedures or courses of treatment.

• A statement describing the extent to which confidentiality of records identifying the subject will be maintained.

• Contact names and telephone numbers for the client/consumer to ask questions about program, participant rights, and injury.

A12. Estimates of Annualized Hour Burden

The time to complete the instruments is estimated in Table 2. These estimates are based on current funding and planned fiscal year 2021 Funding Opportunity Announcements (FOAs) and the number of consumers served in fiscal year 2020. The amount of time required to complete the new NOMS tool and IPP measures is based on an informal pilot and prior SAMHSA/CMHS experience in collecting similar data.

Table 2: Estimates of Annualized Hour Burden

SAMHSA Tool	Number of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost
Client-level baseline interview	40,280	1	40,280	0.33	30,901	$24.781	$765,727
Client-level 3- or 6-month reassessment interview	40,280	1	40,280	0.33	30,901	$24.78	$765,727
Client-level clinical discharge interview	6,668	1	6,668	0.33	2,200	24.78	$54,516
Section H Physical Health Data Baseline	39,231	1	39,231	.10	3,923	$24,.78	$97,212
Section H Program Specific Data: baseline, 3- or 6-month reassessment, and clinical discharge	14,800	2	29,600	.08	2,368	$24.78	$58,679
Subtotal	141,259		154,059		68,673		$1,741,861
Infrastructure development, prevention, and mental health promotion quarterly record abstraction 2	942	4	3,768	2.0	7,536	$24.78	$186,742
Total	142,201		157,827		76,209		$1,928,603

A13. Estimates of Annualized Cost Burden to Respondents

There will be no capital, start-up, operation, maintenance, nor purchase costs incurred by the mental health programs participating in this CMHS data collection, or by consumers receiving CMHS-funded treatment services.

A14. Estimates of Annualized Cost to the Government

The principal additional cost to the government for this project is the cost of a contract to collect the data from the various programs and to conduct analyses, which generate routine reports from the data collected. For NOMS, the reports examine baseline characteristics as well as the changes between baseline, discharge, and each of the follow-up periods. For IPP, the reports provide number

The estimated annualized cost for a contract for the GPRA mandate is $2,200,000 and the cost of 1 FTE staff (GS-14 100%) responsible for the data collection effort is approximately $132,000 per year. The estimated annualized total cost to the government will be $2,332,000.

A15. Changes in Burden

Currently, there are 194,564 total burden hours in the OMB-approved NOMS inventory and SAMHSA is now requesting a reduction to 157,827 burden hours. The decrease of 27,959 burden hours is due to the following:

• Deleted questions for which the data was not being utilized.

• Changed client/consumer responses from a five-point psychometric scale to “yes”, “no”, or “no response” .

• Changed frequency of reassessments to either a 3- or 6-month reassessment followed by a clinical discharge assessment.

• Reduced the number of physical health data measures.

The estimated time to complete the baseline, reassessment, and clinical discharge assessment has decreased from 40 to 20 minutes, the estimated time to complete Section G health data has decreased from 15 to 6 minutes, and the estimated time to complete Section G program-specific measures has decreased from 15 minutes to 5 minutes. The estimated amount of time to collect and report IPP data remains unchanged at 2 hours.

A16. Time Schedule, Publication and Analysis Plans

SAMHSA/CMHS will utilize the data collected from this collection on an ongoing basis to monitor performance and to respond to GPRMA and other Federal reporting requirements. These data are used to provide the agency with information to document the overall Center performance requirements and to provide information that will assist CMHS in planning and monitoring program goals. Descriptive information obtained from program reporting requirements will be reviewed for monitoring and program management. Information is used internally by the agency and for performance reports.

Data for the annual GPRA plan/report are needed by SAMHSA on an ongoing basis. Data collection will commence with approval from OMB. Data are provided by CMHS for the most recently completed calendar year to SAMHSA each May in order to assure analysis in time for the annual GPRA report. The annual GPRA report must be submitted to the Department of Health and Human Services and to OMB by September and is included in the President's Annual Budget Request, which is released to the public February 1st. Data may be refined and added to the final Presidential Budget Request after the Department submits its initial GPRA report.

Data will be available to CMHS staff and grantees through a series of reports available through the web-based system. Roles will determine user access. Individual grantees will only be allowed detailed access to data from their grant. They will also have access to summary information across all grantees in their program. CMHS staff access will be determined by their span of responsibility.

The web-based reports on the SPARS system will include information on the number of consumers served, their demographic characteristics, baseline status, and change scores for the various domains. The data items collected will be analyzed and presented in GPRA reports using basic descriptive statistics. On principle outcome items, a comparison of client status after treatment with baseline data will be used to assess any change in status. The web-based reports will also allow users to create basic cross tabulations of the data.

Data will be used to report to Congress regarding the GPRA as specified in the SAMHSA Annual Justifications of Budget Estimates. They will also allow CMHS staff to examine performance longitudinally, by program, or individual grantee.

In addition to the reports on the SPARS web site, data will be utilized for specialized analyses as needs emerge. Individual grantees will be able to download their own data in into an Excel spreadsheet for further manipulation or to transfer to a statistical package.

The expectation is that over time the results will be examined for subpopulations of interest within individual activities (e.g., by age, gender, or diagnosis) or in response to emerging policy issues. With these analyses, the data would be exported to a statistical package for more elaborate analytic work.

A17. Display of Expiration Date

The expiration date for OMB approval will be displayed on all data collection instruments.

A18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.

Attachments:

A = NOMS Client-Level Data Collection Tool

B = IPP Measures

C = Summary of 60-day Federal Register Notice Comments

D = Listing of all Changes to NOMS Client-Level Data Collection Tool and IPP measures

1 The hourly wage estimate is $24.78 based on the Occupational Employment and Wages, Mean Hourly Wage rate for Substance Abuse, Behavioral Disorder, and Mental Health Counselors (https://www.bls.gov).

2 Grantees are required to report these data quarterly as a condition of their grant.

11