0285-ss_a-cmhs 12.11.15

Transformation Accountability (TRAC) Reporting System

Supporting Statement

Justification

1.Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration’s (SAMHSA), Center for Mental Health Services (CMHS) is requesting approval from the Office of Management and Budget (OMB) for approval for revision to continue the collection of performance data with current and future SAMHSA/CMHS grantees using the TRAC reporting system instruments (OMB No. 0930-0285) that expires on January 1, 2016.

SAMHSA/CMHS is requesting the continuation of the current three data collection instruments:

1. The CMHS NOMs Adult Client-level Measures for Discretionary Programs Providing Direct Services (Attachment 1);

2. The CMHS NOMs Child Client-level Measures for Discretionary Programs Providing Direct Services (Child/Caregiver Version) (Attachment 2)

3. The Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators (Attachment 3).

This information collection will allow SAMHSA to continue to meet the Government Performance and Results Act (GPRA) of 1993 reporting requirements that quantify the effects and accomplishments of its programs, which are consistent with OMB guidance. In order to carry out section 1105(a) (29) of GPRA, SAMHSA is required to prepare a performance plan for its major programs of activity. This plan must:

a) Establish performance goals to define the level of performance to be achieved by a program activity;

b) Express such goals in an objective, quantifiable, and measurable form;

c) Briefly describe the operational processes, skills and technology, and the human, capital, information, or other resources required to meet the performance goals;

d) Establish performance indicators to be used in measuring or assessing the relevant outputs, service levels, and outcomes of each program activity;

e) Provide a basis for comparing actual program results with the established performance goals; and

f) Describe the means to be used to verify and validate measured values.

In addition, this data collection supports the GPRA Modernization Act of 2010 which requires overall organization management to improve agency performance and achieve the mission and goals of the agency through the use of strategic and performance planning, measurement, analysis, regular assessment of progress, and use of performance information to improve the results achieved. Specifically, this data collection will allow CMHS to have the capacity to report on a consistent set of performance measures across its various grant programs that conduct each of these activities. SAMHSA’s legislative mandate is to increase access to high quality substance abuse and mental health prevention and treatment services and to improve outcomes. Its mission is to improve the quality and availability of treatment and prevention services for substance abuse and mental illness. To support this mission, the Agency’s overarching goals are:

• Accountability—Establish systems to ensure program performance measurement and accountability

• Capacity—Build, maintain, and enhance mental health and substance abuse infrastructure and capacity

• Effectiveness—Enable all communities and providers to deliver effective services

Each of these key goals complements SAMHSA’s legislative mandate. All of SAMHSA’s programs and activities are geared toward the achievement of these goals and performance monitoring is a collaborative and cooperative aspect of this process.

SAMHSA will strive to coordinate the development of these goals with other ongoing performance measurement development activities. This information collection is needed to provide objective data to demonstrate SAMHSA’s monitoring and achievement of its mission and goals.

2.Purpose and Use of Information

These proposed data activities are intended to promote the use of consistent measures among CMHS-funded grantees and contractors. These measures are a result of extensive examination and recommendations, using consistent criteria, by panels of staff, experts, and grantees. Wherever feasible, the measures are consistent with or build upon previous data development efforts within CMHS. These data collection activities are organized to reflect and support the domains specified for SAMHSA’s NOMs for programs providing direct services, and the categories developed by CMHS to specify the infrastructure, prevention, and mental health promotion activities.

Individuals at three different levels will use the information: the SAMHSA Administrator and staff, the Center Directors and Project Officers, and grantees:

• SAMHSA level—This information will be used to inform the administration on the performance of the programs funded through the Agency. Assessment of performance will be based on the new measures in line with the grant’s program goals as set by program leadership. The intent is that the information will serve as the basis of the annual performance report to Congress contained in the Justifications of Budget Estimates.

• Center level—In addition to providing information on the performance of the various programs, the information can be used to monitor and manage individual grant projects within each program. The information can be used to identify strengths and weaknesses and provide an informed basis for providing technical assistance and other support to grantees, informing continuation funding decisions, and identifying potential subjects for further evaluation.

• Grantee level—In addition to monitoring performance outcomes, the grantee staff can use the information to improve the quality of services that are provided to consumers within their projects, to promote service system capacity and infrastructure development, to prevent negative impacts of mental health problems, and to promote mental wellness.

To fulfill GPRA requirements SAMHSA develops a report for each fiscal year that includes results of performance monitoring for the three preceding fiscal years. The additional information collected through this process will allow SAMHSA to report on the results of these performance outcomes as well as be consistent with the specific performance domains that SAMHSA is implementing to assess the accountability and performance of its discretionary and formula grant programs.

Client-level Data Collection

To facilitate SAMHSA-wide reporting, the agency has identified ten domains of particular interest for accountability and performance monitoring of client-level data for programs providing direct services. These domains are:

• Access/Capacity

• Functioning

• Stability in Housing

• Education and Employment

• Crime and Criminal Justice

• Perception of Care

• Social Connectedness

• Retention

• Cost-Effectiveness

• Evidence-Based Practices

As stated above, the SAMHSA CMHS programs that provide direct treatment to consumers, or Services programs, currently have an OMB-approved data collection in place. Consequently, this request for approval of the two Services instruments is for revisions to the existing data collection instruments. This data collection includes separate data collection forms that are parallel in design for use in interviewing adults and children (or their caregivers for children under the age of 11 years old); named the CMHS NOMs Adult Client-level Measures for Discretionary Programs Providing Direct Services and the Child Client-level Measures for Discretionary Programs Providing Direct Services, respectively. These SAMHSA TRAC data will be collected at baseline, at six month reassessments for as long as the consumer remains in treatment, and at discharge. The data collection encompasses eight of the ten SAMHSA NOMs domains.

Table 1. Data Collection for Client-level Measures

Domain	Number of Questions: Adult	Number of Questions: Child
Access/Capacity	4	4
Functioning	28	26
Violence and Trauma	6	0
Military Family Deployment	6	6
Stability in Housing	1	2
Education and Employment	4	3
Crime and Criminal Justice	1	1
Perception of Care	15	14
Social Connectedness	4	4
Retention	5	5
Total Number	75	65

Data Collection for Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators

To facilitate CMHS reporting of GPRA data for programs engaged in substantial infrastructure development, prevention, and mental health promotion activities, the agency has identified 14 categories of particular interest for accountability and performance monitoring. No changes are proposed for these categories. These categories are:

• Policy Development

• Workforce Development

• Financing

• Organizational Change

• Partnerships/Collaborations

• Accountability

• Types/Targets of Practices

• Awareness

• Training

• Knowledge/Attitudes/Beliefs

• Screening

• Outreach

• Referral

• Access

Proposed Changes to Data Collection Tool

The following table summarizes the total number of indicators for each category that may or may not apply to each grant program:

Table 2. Data Collection for Infrastructure, Prevention, and Mental Health Promotion Indicators

Category	Number of Indicators
Policy Development	2
Workforce Development	5
Financing	3
Organizational Change	1
Partnerships/Collaborations	2
Accountability	6
Types/Targets of Practices	4
Awareness	1
Training	1
Knowledge/Attitudes/Beliefs	1
Screening	1
Outreach	2
Referral	1
Access	1
Total Number	31

SAMHSA and CMHS intend to compare infrastructure, prevention, and mental health promotion targets set at baseline with data collected quarterly. These outcomes will be used as the indicator of performance.

Changes

NO changes to the instruments.

3.Use of Information Technology

Information technology will be used to reduce program respondent burden. The existing TRAC System is a web-based data entry and reporting system designed to support web-based data collection efforts for CMHS. The system will be updated to incorporate proposed changes to the client-level data collection and the infrastructure development, prevention, and mental health promotion performance indicators. 100% of responses are expected to be submitted electronically through the web-based system. The TRAC System also provides a data repository service that includes methods for receiving the data, data quality checks, storage, and data presentation in reports by individual performance indicator or grouped with other performance indicators. The TRAC system complies with the requirements of Section 508 of the Rehabilitation Act to permit accessibility to people with disabilities.

This web-based system is intended to allow for easy data entry and access to reports for grantees that are required to submit TRAC data to CMHS. Entering and accessing data and viewing reports will be limited to those individuals with a username and password. A user’s level of access to the data and reports will be defined based on his or her authority and responsibilities.

Electronic submission of the data promotes enhanced data quality. With built-in data quality checks and easy access to data outputs and reports, users of the data can feel confident about the quality of the output. The electronic submission also promotes immediate access to the dataset. Once the data are put into the web-based system, it will be available for access, review, and reporting by all those with access to the system from Center staff to the grantee staff.

4.Efforts to Identify Duplication

A program-level review of current measures and methods of collection was conducted to identify duplication of these data collection efforts. With the goal of creating standardized indicators and methods for monitoring grantee performance across the Center, existing measures were considered for use where appropriate. However, modification of current measures was necessary in some cases to generalize across varied programs. Each of these data collection instruments was reviewed and approved by the Government Project Officers, Branch Chiefs, and CMHS senior leadership as meeting the performance monitoring and management needs of individual programs and the Center. Since many of the grantees engaged in infrastructure development, prevention, and mental health promotion activities already collect data for the proposed indicators, the creation of this system will provide them with a standardized method for reporting to CMHS.

5.Involvement of Small Entities

Individual grantees vary from small entities through large provider organizations. Every effort has been made to reduce the number of data items collected from grantees to the least number required to accomplish the objectives of the effort and to meet performance and GPRA reporting requirements and therefore, there is no significant impact involving small entities in general. Based on the pilot test and input and feedback from CMHS Project Officers, however, we understand that it may be difficult for some American Indian/Alaska Native Tribes and tribal organizations to report on the infrastructure development, prevention, and mental health promotion performance indicators on a quarterly basis. CMHS will, therefore, develop a waiver process to allow such grantees to request, through their Project Officers, to report on these indicators every six months rather than quarterly.

6.Consequences If Information Collected Less Frequently

Client-level data

Mental health programs typically collect client-level data at admission and then conduct periodic reassessments of consumers while the individual remains in services. When feasible, mental health providers also conduct an assessment when the consumer is discharged. The data collection schedule for the client-level measures parallels this model. All programs that provide direct services will collect data every six months while the consumer is receiving services; this is a reduction from the prior requirement of quarterly data collection for three of the CMHS programs (the National Child Traumatic Stress Initiative, Meeting the Needs of Elderly Americans, and HIV/AIDS Minority Mental Health Services programs.)

The baseline data collection point is critical for measuring changes. Extending the interval for the periodic reassessment beyond the requested intervals could lead to loss of contact with consumers, significantly diminishing the response rates and lowering the value of the data for performance reporting use by losing measurement of intermediate effects.

Infrastructure development, prevention, and mental health promotion data

This quarterly data collection requirement for the infrastructure development, prevention, and mental health promotion performance indicators is necessary to provide CMHS with the information when needed for appropriate program monitoring and management, as well as for GPRA performance reporting.

7.Consistency with the Guidelines in 5 CFR 1320.5(d) (2)

This information collection fully complies with the guidelines in 5 CFR 1320.5(d) (2).

8.Consultation Outside the Agency

The notice required by 5 CFR 1320.8(d) was published in the Federal Register on September 22, 2015 (80 FR 57197). No comments were received in response to this notice. Both external and internal stakeholders were consulted by CMHS in the development of these indicators and the data collection methodology. CMHS obtained feedback and consultation regarding the availability of data, methods and frequency of collection, and the appropriateness of data elements.

9. Payment to Respondents

No monetary incentives are provided to grantees.

10.Assurance of Confidentiality

SAMHSA’s grantees do not collect individually identifiable information for these programs. Only aggregated data will be reported by grantees, therefore, SAMHSA and its contractors will not receive identifiable client records. Provider-level information will be aggregated to, at the least, the level of the grant/cooperative agreement-funding announcement.

11.Questions of a Sensitive Nature

No questions of a sensitive nature are asked of State or U.S. Territories grantees. However, grantees who are service providers collect information such as use of alcohol and other drugs and mental health conditions routinely as part of the provision of services. While some of this information may appear sensitive, program participants are service recipients.

12.Estimates of Annualized Hour Burden

The time to complete the instruments is estimated in Table 3. These estimates are based on current funding and planned fiscal year 2014 notice of funding announcements (NOFA), SAMHSA CMHS discretionary programs that will use these measures in fiscal years 2015 through 2016 to collect client-level data, and grantee reports of the amount of time required to complete the currently approved instruments accounting for the additional time required to complete the new questions, as based on an informal pilot and prior CMHS experience in collecting similar data.

Table 3. Estimates of Annualized Hour Burden

Type of Response	Number of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost
Client-level baseline interview	35,845	1	35,854	0.45	16,130	$20.131	$324,697
Client-level 6-month reassessment interview2	23,658	1	23,658	0.45	10,646	$20.13	$214,304
Client-level discharge interview3	10,753	1	10,753	0.45	4,838	$20.13	$973,950
PBHCI- Section H Form Only Baseline	14,000	1	14,000	.08	1,120	$20.13	$22.55
PBHCI- Section H Form Only Follow-Up 4	9,240	1	9,240	.08	739	$20.13	$148.76
PBHCI – Section H Form Only Discharge 5	4,200	1	4,200	.08	336	$20.13	$6764
HIV Continuum of Care Specific Form Baseline	200	1	200	0.33	66	$20.13	$1,304
HIV Continuum of Care Follow-Up6	148	1	148	.033	49	$20.13	$986
HIV Continuum of Care Discharge 7	104	1	104	0.33	34	20.13	$684
Infrastructure development, prevention, and mental health promotion quarterly record abstraction8	982	4.0	3928	2.0	7,856	$20.13	$9869
Total	36,827		102,139		48,814		$785,832

1. The hourly wage estimate is $20.13 based on the Occupational Employment and Wages, May 2014 Mean Hourly Wage Rate for 21-1011 Substance Abuse and Behavioral Disorder Counselors  = $20.13/hr. as of May, 2014.  (http://www.bls.gov/oes/current/oes211011.htm  (Accessed on August 24, 2015.

2. It is estimated that 66% of baseline clients will complete this interview.

3. It is estimated that 30% of baseline clients will complete this interview.

4. It is estimated that 74% of baseline clients will complete this interview.

5. It is estimated that 52% of baseline clients will complete this interview.

6 It is estimated that 52% of baseline clients will complete this interview.

7. It is estimated that 30% of baseline clients will complete this interview.

8. Grantees are required to report this information as a condition of their grant. No attrition is estimated.

Note: Numbers may not add to the totals due to rounding and some individual participants completing more than one form.

13.Estimates of Annualized Cost Burden to Respondents

There will be no capital, start-up, operation, maintenance, nor purchase costs incurred by the mental health programs participating in this CMHS data collection, or by consumers receiving CMHS-funded services.

14.Estimates of Annualized Cost to the Government

The principal additional cost to the government for this project is the cost of a contract to collect the data from the various programs and to conduct analyses which generate routine reports from the data collected. The reports examine baseline characteristics as well as the changes between baseline, discharge, and each of the follow-up periods. It is the responsibility of the contractor to work with the Government Project Officer (GPO) when preparing reports that combine the client services data with the annual reports of the project.

The estimated annualized cost for a contract for the GPRA mandate is $1,859,161 and the cost of 1 FTE staff (GS-14 100%) responsible for the CMHS data collection effort is approximately $132,000 /year. The estimated annualized total cost to the government will be $1,991,161.

15.Changes in Burden

Currently, there are 29,298 total burden hours in the OMB inventory and CMHS is requesting 48,814 burden hours.  The increase of 19,516 is a program change in burden due to the number of overall grant programs and number of respondents receiving mental health treatment services.

16.Time Schedule, Publication and Analysis Plans

SAMHSA/CMHS utilizes the NOMs data on an ongoing basis to monitor performance and to respond to GPRAMA and other Federal reporting requirements. These data are used to provide the agency with information to document the overall Center performance requirements and to provide information that will assist CMHS in planning and monitoring program goals. Descriptive information obtained from program reporting requirements will be reviewed for monitoring and program management. Information is used internally by the agency and for performance reports. There are no formal publication plans.

The time frame for submission of the reporting requirements varies by grant cycle and grant program period of performance throughout the year.

17.Display of Expiration Date

The expiration date for OMB approval will be displayed on all data collection instruments.

18.Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.

1

10