Transformation Accountability (TRAC) Reporting System
Supporting Statement
The Substance Abuse and Mental Health Services Administration’s (SAMHSA), Center for Mental Health Services (CMHS) is requesting approval from the Office of Management and Budget (OMB) for approval for revision to continue the collection of performance data with current and future SAMHSA/CMHS grantees using the TRAC reporting system instruments (OMB No. 0930-0285) that expires on January 1, 2016.
SAMHSA/CMHS is requesting the continuation of the current three data collection instruments:
1. The CMHS NOMs Adult Client-level Measures for Discretionary Programs Providing Direct Services (Attachment 1);
2. The CMHS NOMs Child Client-level Measures for Discretionary Programs Providing Direct Services (Child/Caregiver Version) (Attachment 2)
3. The Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators (Attachment 3).
This information collection will allow SAMHSA to continue to meet the Government Performance and Results Act (GPRA) of 1993 reporting requirements that quantify the effects and accomplishments of its programs, which are consistent with OMB guidance. In order to carry out section 1105(a) (29) of GPRA, SAMHSA is required to prepare a performance plan for its major programs of activity. This plan must:
a) Establish performance goals to define the level of performance to be achieved by a program activity;
b) Express such goals in an objective, quantifiable, and measurable form;
c) Briefly describe the operational processes, skills and technology, and the human, capital, information, or other resources required to meet the performance goals;
d) Establish performance indicators to be used in measuring or assessing the relevant outputs, service levels, and outcomes of each program activity;
e) Provide a basis for comparing actual program results with the established performance goals; and
f) Describe the means to be used to verify and validate measured values.
In addition, this data collection supports the GPRA Modernization Act of 2010 which requires overall organization management to improve agency performance and achieve the mission and goals of the agency through the use of strategic and performance planning, measurement, analysis, regular assessment of progress, and use of performance information to improve the results achieved. Specifically, this data collection will allow CMHS to have the capacity to report on a consistent set of performance measures across its various grant programs that conduct each of these activities. SAMHSA’s legislative mandate is to increase access to high quality substance abuse and mental health prevention and treatment services and to improve outcomes. Its mission is to improve the quality and availability of treatment and prevention services for substance abuse and mental illness. To support this mission, the Agency’s overarching goals are:
Accountability—Establish systems to ensure program performance measurement and accountability
Capacity—Build, maintain, and enhance mental health and substance abuse infrastructure and capacity
Effectiveness—Enable all communities and providers to deliver effective services
Each of these key goals complements SAMHSA’s legislative mandate. All of SAMHSA’s programs and activities are geared toward the achievement of these goals and performance monitoring is a collaborative and cooperative aspect of this process.
SAMHSA will strive to coordinate the development of these goals with other ongoing performance measurement development activities. This information collection is needed to provide objective data to demonstrate SAMHSA’s monitoring and achievement of its mission and goals.
These proposed data activities are intended to promote the use of consistent measures among CMHS-funded grantees and contractors. These measures are a result of extensive examination and recommendations, using consistent criteria, by panels of staff, experts, and grantees. Wherever feasible, the measures are consistent with or build upon previous data development efforts within CMHS. These data collection activities are organized to reflect and support the domains specified for SAMHSA’s NOMs for programs providing direct services, and the categories developed by CMHS to specify the infrastructure, prevention, and mental health promotion activities.
Individuals at three different levels will use the information: the SAMHSA Administrator and staff, the Center Directors and Project Officers, and grantees:
SAMHSA level—This information will be used to inform the administration on the performance of the programs funded through the Agency. Assessment of performance will be based on the new measures in line with the grant’s program goals as set by program leadership. The intent is that the information will serve as the basis of the annual performance report to Congress contained in the Justifications of Budget Estimates.
Center level—In addition to providing information on the performance of the various programs, the information can be used to monitor and manage individual grant projects within each program. The information can be used to identify strengths and weaknesses and provide an informed basis for providing technical assistance and other support to grantees, informing continuation funding decisions, and identifying potential subjects for further evaluation.
Grantee level—In addition to monitoring performance outcomes, the grantee staff can use the information to improve the quality of services that are provided to consumers within their projects, to promote service system capacity and infrastructure development, to prevent negative impacts of mental health problems, and to promote mental wellness.
To fulfill GPRA requirements SAMHSA develops a report for each fiscal year that includes results of performance monitoring for the three preceding fiscal years. The additional information collected through this process will allow SAMHSA to report on the results of these performance outcomes as well as be consistent with the specific performance domains that SAMHSA is implementing to assess the accountability and performance of its discretionary and formula grant programs.
To facilitate SAMHSA-wide reporting, the agency has identified ten domains of particular interest for accountability and performance monitoring of client-level data for programs providing direct services. These domains are:
Access/Capacity
Functioning
Stability in Housing
Education and Employment
Crime and Criminal Justice
Perception of Care
Social Connectedness
Retention
Cost-Effectiveness
Evidence-Based Practices
As stated above, the SAMHSA CMHS programs that provide direct treatment to consumers, or Services programs, currently have an OMB-approved data collection in place. Consequently, this request for approval of the two Services instruments is for revisions to the existing data collection instruments. This data collection includes separate data collection forms that are parallel in design for use in interviewing adults and children (or their caregivers for children under the age of 11 years old); named the CMHS NOMs Adult Client-level Measures for Discretionary Programs Providing Direct Services and the Child Client-level Measures for Discretionary Programs Providing Direct Services, respectively. These SAMHSA TRAC data will be collected at baseline, at six month reassessments for as long as the consumer remains in treatment, and at discharge. The data collection encompasses eight of the ten SAMHSA NOMs domains.
Table 1. Data Collection for Client-level Measures
Domain |
Number of Questions: Adult |
Number of Questions: Child |
Access/Capacity |
4 |
4 |
Functioning |
28 |
26 |
Violence and Trauma |
6 |
0 |
Military Family Deployment |
6 |
6 |
Stability in Housing |
1 |
2 |
Education and Employment |
4 |
3 |
Crime and Criminal Justice |
1 |
1 |
Perception of Care |
15 |
14 |
Social Connectedness |
4 |
4 |
Retention |
5 |
5 |
Total Number |
75 |
65 |
To facilitate CMHS reporting of GPRA data for programs engaged in substantial infrastructure development, prevention, and mental health promotion activities, the agency has identified 14 categories of particular interest for accountability and performance monitoring. No changes are proposed for these categories. These categories are:
Policy Development
Workforce Development
Financing
Organizational Change
Partnerships/Collaborations
Accountability
Types/Targets of Practices
Awareness
Training
Knowledge/Attitudes/Beliefs
Screening
Outreach
Referral
Access
Proposed Changes to Data Collection Tool
The following table summarizes the total number of indicators for each category that may or may not apply to each grant program:
Table 2. Data Collection for Infrastructure, Prevention, and Mental Health Promotion Indicators
Category |
Number of Indicators |
Policy Development |
2 |
Workforce Development |
5 |
Financing |
3 |
Organizational Change |
1 |
Partnerships/Collaborations |
2 |
Accountability |
6 |
Types/Targets of Practices |
4 |
Awareness |
1 |
Training |
1 |
Knowledge/Attitudes/Beliefs |
1 |
Screening |
1 |
Outreach |
2 |
Referral |
1 |
Access |
1 |
Total Number |
31 |
SAMHSA and CMHS intend to compare infrastructure, prevention, and mental health promotion targets set at baseline with data collected quarterly. These outcomes will be used as the indicator of performance.
Changes
NO changes to the instruments.
Information technology will be used to reduce program respondent burden. The existing TRAC System is a web-based data entry and reporting system designed to support web-based data collection efforts for CMHS. The system will be updated to incorporate proposed changes to the client-level data collection and the infrastructure development, prevention, and mental health promotion performance indicators. 100% of responses are expected to be submitted electronically through the web-based system. The TRAC System also provides a data repository service that includes methods for receiving the data, data quality checks, storage, and data presentation in reports by individual performance indicator or grouped with other performance indicators. The TRAC system complies with the requirements of Section 508 of the Rehabilitation Act to permit accessibility to people with disabilities.
This web-based system is intended to allow for easy data entry and access to reports for grantees that are required to submit TRAC data to CMHS. Entering and accessing data and viewing reports will be limited to those individuals with a username and password. A user’s level of access to the data and reports will be defined based on his or her authority and responsibilities.
Electronic submission of the data promotes enhanced data quality. With built-in data quality checks and easy access to data outputs and reports, users of the data can feel confident about the quality of the output. The electronic submission also promotes immediate access to the dataset. Once the data are put into the web-based system, it will be available for access, review, and reporting by all those with access to the system from Center staff to the grantee staff.
A program-level review of current measures and methods of collection was conducted to identify duplication of these data collection efforts. With the goal of creating standardized indicators and methods for monitoring grantee performance across the Center, existing measures were considered for use where appropriate. However, modification of current measures was necessary in some cases to generalize across varied programs. Each of these data collection instruments was reviewed and approved by the Government Project Officers, Branch Chiefs, and CMHS senior leadership as meeting the performance monitoring and management needs of individual programs and the Center. Since many of the grantees engaged in infrastructure development, prevention, and mental health promotion activities already collect data for the proposed indicators, the creation of this system will provide them with a standardized method for reporting to CMHS.
Individual grantees vary from small entities through large provider organizations. Every effort has been made to reduce the number of data items collected from grantees to the least number required to accomplish the objectives of the effort and to meet performance and GPRA reporting requirements and therefore, there is no significant impact involving small entities in general. Based on the pilot test and input and feedback from CMHS Project Officers, however, we understand that it may be difficult for some American Indian/Alaska Native Tribes and tribal organizations to report on the infrastructure development, prevention, and mental health promotion performance indicators on a quarterly basis. CMHS will, therefore, develop a waiver process to allow such grantees to request, through their Project Officers, to report on these indicators every six months rather than quarterly.
Client-level data
Mental health programs typically collect client-level data at admission and then conduct periodic reassessments of consumers while the individual remains in services. When feasible, mental health providers also conduct an assessment when the consumer is discharged. The data collection schedule for the client-level measures parallels this model. All programs that provide direct services will collect data every six months while the consumer is receiving services; this is a reduction from the prior requirement of quarterly data collection for three of the CMHS programs (the National Child Traumatic Stress Initiative, Meeting the Needs of Elderly Americans, and HIV/AIDS Minority Mental Health Services programs.)
The baseline data collection point is critical for measuring changes. Extending the interval for the periodic reassessment beyond the requested intervals could lead to loss of contact with consumers, significantly diminishing the response rates and lowering the value of the data for performance reporting use by losing measurement of intermediate effects.
Infrastructure development, prevention, and mental health promotion data
This quarterly data collection requirement for the infrastructure development, prevention, and mental health promotion performance indicators is necessary to provide CMHS with the information when needed for appropriate program monitoring and management, as well as for GPRA performance reporting.
This information collection fully complies with the guidelines in 5 CFR 1320.5(d) (2).
The notice required by 5 CFR 1320.8(d) was published in the Federal Register on September 22, 2015 (80 FR 57197). No comments were received in response to this notice. Both external and internal stakeholders were consulted by CMHS in the development of these indicators and the data collection methodology. CMHS obtained feedback and consultation regarding the availability of data, methods and frequency of collection, and the appropriateness of data elements.
No monetary incentives are provided to grantees.
SAMHSA’s grantees do not collect individually identifiable information for these programs. Only aggregated data will be reported by grantees, therefore, SAMHSA and its contractors will not receive identifiable client records. Provider-level information will be aggregated to, at the least, the level of the grant/cooperative agreement-funding announcement.
No questions of a sensitive nature are asked of State or U.S. Territories grantees. However, grantees who are service providers collect information such as use of alcohol and other drugs and mental health conditions routinely as part of the provision of services. While some of this information may appear sensitive, program participants are service recipients.
The time to complete the instruments is estimated in Table 3. These estimates are based on current funding and planned fiscal year 2014 notice of funding announcements (NOFA), SAMHSA CMHS discretionary programs that will use these measures in fiscal years 2015 through 2016 to collect client-level data, and grantee reports of the amount of time required to complete the currently approved instruments accounting for the additional time required to complete the new questions, as based on an informal pilot and prior CMHS experience in collecting similar data.
Table 3. Estimates of Annualized Hour Burden
Type of Response |
Number of Respondents |
Responses per Respondent |
Total Responses |
Hours per Response |
Total Hour Burden |
Hourly Wage Cost |
Total Hour Cost |
Client-level baseline interview |
35,845 |
1 |
35,854 |
0.45 |
16,130 |
$20.131 |
$324,697 |
Client-level 6-month reassessment interview2 |
23,658 |
1 |
23,658 |
0.45 |
10,646 |
$20.13 |
$214,304 |
Client-level discharge interview3 |
10,753 |
1 |
10,753 |
0.45 |
4,838 |
$20.13 |
$973,950 |
PBHCI- Section H Form Only Baseline |
14,000 |
1 |
14,000 |
.08 |
1,120 |
$20.13 |
$22.55
|
PBHCI- Section H Form Only Follow-Up 4 |
9,240 |
1 |
9,240 |
.08 |
739 |
$20.13 |
$148.76
|
PBHCI – Section H Form Only Discharge 5 |
4,200 |
1 |
4,200 |
.08 |
336 |
$20.13 |
$6764 |
HIV Continuum of Care Specific Form Baseline |
200 |
1 |
200 |
0.33 |
66 |
$20.13 |
$1,304 |
HIV Continuum of Care Follow-Up6 |
148 |
1 |
148 |
.033 |
49 |
$20.13 |
$986 |
HIV Continuum of Care Discharge 7 |
104 |
1 |
104 |
0.33 |
34 |
20.13 |
$684 |
Infrastructure development, prevention, and mental health promotion quarterly record abstraction8 |
982 |
4.0 |
3928 |
2.0 |
7,856 |
$20.13 |
$9869 |
Total |
36,827 |
|
102,139 |
|
48,814 |
|
$785,832 |
1. The hourly wage estimate is $20.13 based on the Occupational Employment and Wages, May 2014 Mean Hourly Wage Rate for 21-1011 Substance Abuse and Behavioral Disorder Counselors = $20.13/hr. as of May, 2014. (http://www.bls.gov/oes/current/oes211011.htm (Accessed on August 24, 2015.
2. It is estimated that 66% of baseline clients will complete this interview.
3. It is estimated that 30% of baseline clients will complete this interview.
4. It is estimated that 74% of baseline clients will complete this interview.
5. It is estimated that 52% of baseline clients will complete this interview.
6 It is estimated that 52% of baseline clients will complete this interview.
7. It is estimated that 30% of baseline clients will complete this interview.
8. Grantees are required to report this information as a condition of their grant. No attrition is estimated.
Note: Numbers may not add to the totals due to rounding and some individual participants completing more than one form.
There will be no capital, start-up, operation, maintenance, nor purchase costs incurred by the mental health programs participating in this CMHS data collection, or by consumers receiving CMHS-funded services.
The principal additional cost to the government for this project is the cost of a contract to collect the data from the various programs and to conduct analyses which generate routine reports from the data collected. The reports examine baseline characteristics as well as the changes between baseline, discharge, and each of the follow-up periods. It is the responsibility of the contractor to work with the Government Project Officer (GPO) when preparing reports that combine the client services data with the annual reports of the project.
The estimated annualized cost for a contract for the GPRA mandate is $1,859,161 and the cost of 1 FTE staff (GS-14 100%) responsible for the CMHS data collection effort is approximately $132,000 /year. The estimated annualized total cost to the government will be $1,991,161.
Currently, there are 29,298 total burden hours in the OMB inventory and CMHS is requesting 48,814 burden hours. The increase of 19,516 is a program change in burden due to the number of overall grant programs and number of respondents receiving mental health treatment services.
SAMHSA/CMHS utilizes the NOMs data on an ongoing basis to monitor performance and to respond to GPRAMA and other Federal reporting requirements. These data are used to provide the agency with information to document the overall Center performance requirements and to provide information that will assist CMHS in planning and monitoring program goals. Descriptive information obtained from program reporting requirements will be reviewed for monitoring and program management. Information is used internally by the agency and for performance reports. There are no formal publication plans.
The time frame for submission of the reporting requirements varies by grant cycle and grant program period of performance throughout the year.
The expiration date for OMB approval will be displayed on all data collection instruments.
This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.
| File Type | application/msword |
| File Title | Transformation Accountability (TRAC) Reporting System |
| Author | Jessica Taylor |
| Last Modified By | Windows User |
| File Modified | 2015-12-11 |
| File Created | 2015-09-02 |