0285-ss_a-11.19. 13

Transformation Accountability (TRAC) Reporting System

Supporting Statement

Justification

1.Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration’s (SAMHSA), Center for Mental Health Services (CMHS) is requesting approval from the Office of Management and Budget (OMB) for approval for an extension to continue the collection of performance data with current and future SAMHSA/CMHS grantees using the TRAC reporting system instruments (OMB No. 0930-0285) that expires on February 28, 2013.

SAMHSA/CMHS is requesting the continuation of the current three data collection instruments:

1. The CMHS NOMs Adult Client-level Measures for Discretionary Programs Providing Direct Services (Attachment 1);

2. The CMHS NOMs Child Client-level Measures for Discretionary Programs Providing Direct Services (Child/Caregiver Version) (Attachment 2)

3. The Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators (Attachment 3).

SAMHSA/CMHS is requesting the deletion of the following question in both the Adult and Child Client-level instrument:

Has the Service Member experienced any of the following (select all that apply).

1. Deployed in support of combat operations (e.g. Iraq or Afghanistan)

2. Was physically injured during combat operations

3. Developed combat stress symptoms/difficulties adjusting following deployment, including PTSD, depression, or suicidal thoughts

4. Died or was killed

SAMHSA/CMHS is requesting the addition of the following question in the Adult Client-level instrument only:

Which one of the following do you consider yourself to be?

Heterosexual, that is straight;

[IF FEMALE THEN “Lesbian or”] Gay

Bisexual

DK/REF

This information collection will allow SAMHSA to continue to meet the Government Performance and Results Act (GPRA) of 1993 reporting requirements that quantify the effects and accomplishments of its programs, which are consistent with OMB guidance. In order to carry out section 1105(a) (29) of GPRA, SAMHSA is required to prepare a performance plan for its major programs of activity. This plan must:

• Briefly describe the operational processes, skills and technology, and the human, capital, information, or other resources required to meet the performance goals;

• Establish performance indicators to be used in measuring or assessing the relevant outputs, service levels, and outcomes of each program activity;

• Provide a basis for comparing actual program results with the established performance goals; and

• Describe the means to be used to verify and validate measured values.

In addition, this data collection supports the GPRA Modernization Act of 2010 which requires overall organization management to improve agency performance and achieve the mission and goals of the agency through the use of strategic and performance planning, measurement, analysis, regular assessment of progress, and use of performance information to improve the results achieved. Specifically, this data collection will allow CMHS to have the capacity to report on a consistent set of performance measures across its various grant programs that conduct each of these activities. SAMHSA’s legislative mandate is to increase access to high quality substance abuse and mental health prevention and treatment services and to improve outcomes. Its mission is to improve the quality and availability of treatment and prevention services for substance abuse and mental illness. To support this mission, the Agency’s overarching goals are:

• Accountability—Establish systems to ensure program performance measurement and accountability

• Capacity—Build, maintain, and enhance mental health and substance abuse infrastructure and capacity

• Effectiveness—Enable all communities and providers to deliver effective services

Each of these key goals complements SAMHSA’s legislative mandate. All of SAMHSA’s programs and activities are geared toward the achievement of these goals and performance monitoring is a collaborative and cooperative aspect of this process.

SAMHSA will strive to coordinate the development of these goals with other ongoing performance measurement development activities. This information collection is needed to provide objective data to demonstrate SAMHSA’s monitoring and achievement of its mission and goals.

2.Purpose and Use of Information

These proposed data activities are intended to promote the use of consistent measures among CMHS-funded grantees and contractors. These common measures recommended by CMHS are a result of extensive examination and recommendations, using consistent criteria, by panels of staff, experts, and grantees. Wherever feasible, the measures are consistent with or build upon previous data development efforts within CMHS. These data collection activities will be organized to reflect and support the domains specified for SAMHSA’s NOMs for programs providing direct services, and the categories developed by CMHS to specify the infrastructure, prevention, and mental health promotion activities.

Individuals at three different levels will use the information: the SAMHSA Administrator and staff, the Center Directors and Project Officers, and grantees:

• SAMHSA level—This information will be used to inform the administration on the performance of the programs funded through the Agency. Assessment of performance will be based on the new measures in line with the grant’s program goals as set by program leadership. The intent is that the information will serve as the basis of the annual performance report to Congress contained in the Justifications of Budget Estimates.

• Center level—In addition to providing information on the performance of the various programs, the information can be used to monitor and manage individual grant projects within each program. The information can be used to identify strengths and weaknesses and provide an informed basis for providing technical assistance and other support to grantees, informing continuation funding decisions, and identifying potential subjects for further evaluation.

• Grantee level—In addition to monitoring performance outcomes, the grantee staff can use the information to improve the quality of services that are provided to consumers within their projects, to promote service system capacity and infrastructure development, to prevent negative impacts of mental health problems, and to promote mental wellness.

To fulfill GPRA requirements SAMHSA develops a report for each fiscal year that includes results of performance monitoring for the three preceding fiscal years. The additional information collected through this process will allow SAMHSA to report on the results of these performance outcomes as well as be consistent with the specific performance domains that SAMHSA is implementing to assess the accountability and performance of its discretionary and formula grant programs.

Client-level Data Collection

To facilitate SAMHSA-wide reporting, the agency has identified ten domains of particular interest for accountability and performance monitoring of client-level data for programs providing direct services. These domains are:

• Access/Capacity

• Functioning

• Stability in Housing

• Education and Employment

• Crime and Criminal Justice

• Perception of Care

• Social Connectedness

• Retention

• Cost-Effectiveness

• Evidence-Based Practices

As stated above, the SAMHSA CMHS programs that provide direct treatment to consumers, or Services programs, currently have an OMB-approved data collection in place. Consequently, this request for approval of the two Services instruments is for revisions to the existing data collection instruments. This data collection includes separate data collection forms that are parallel in design for use in interviewing adults and children (or their caregivers for children under the age of 11 years old); named the CMHS NOMs Adult Client-level Measures for Discretionary Programs Providing Direct Services and the Child Client-level Measures for Discretionary Programs Providing Direct Services, respectively. These SAMHSA TRAC data will be collected at baseline, at six month reassessments for as long as the consumer remains in treatment, and at discharge. The data collection encompasses eight of the ten SAMHSA NOMs domains.

Table 1. Data Collection for Client-level Measures

Domain	Number of Questions: Adult	Number of Questions: Child
Access/Capacity	4	4
Functioning	28	26
Violence and Trauma	6	0
Military Family Deployment	6	6
Stability in Housing	1	2
Education and Employment	4	3
Crime and Criminal Justice	1	1
Perception of Care	15	14
Social Connectedness	4	4
Retention	5	5
Total Number	75	65

Data Collection for Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators

To facilitate CMHS reporting of GPRA data for programs engaged in substantial infrastructure development, prevention, and mental health promotion activities, the agency has identified 14 categories of particular interest for accountability and performance monitoring. No changes are proposed for these categories. These categories are:

• Policy Development

• Workforce Development

• Financing

• Organizational Change

• Partnerships/Collaborations

• Accountability

• Types/Targets of Practices

• Awareness

• Training

• Knowledge/Attitudes/Beliefs

• Screening

• Outreach

• Referral

• Access

The following table summarizes the total number of indicators for each category that may or may not apply to each grant program:

Table 2. Data Collection for Infrastructure, Prevention, and Mental Health Promotion Indicators

Category	Number of Indicators
Policy Development	2
Workforce Development	5
Financing	3
Organizational Change	1
Partnerships/Collaborations	2
Accountability	6
Types/Targets of Practices	4
Awareness	1
Training	1
Knowledge/Attitudes/Beliefs	1
Screening	1
Outreach	2
Referral	1
Access	1
Total Number	31

SAMHSA and CMHS intend to compare infrastructure, prevention, and mental health promotion targets set at baseline with data collected quarterly. These outcomes will be used as the indicator of performance.

3.Use of Information Technology

Information technology will be used to reduce program respondent burden. The existing TRAC System is a web-based data entry and reporting system designed to support web-based data collection efforts for CMHS. The system will be updated to incorporate proposed changes to the client-level data collection and the infrastructure development, prevention, and mental health promotion performance indicators. 100% of responses are expected to be submitted electronically through the web-based system. The TRAC System also provides a data repository service that includes methods for receiving the data, data quality checks, storage, and data presentation in reports by individual performance indicator or grouped with other performance indicators. The TRAC system complies with the requirements of Section 508 of the Rehabilitation Act to permit accessibility to people with disabilities.

This web-based system is intended to allow for easy data entry and access to reports for grantees that are required to submit TRAC data to CMHS. Entering and accessing data and viewing reports will be limited to those individuals with a username and password. A user’s level of access to the data and reports will be defined based on his or her authority and responsibilities.

Electronic submission of the data promotes enhanced data quality. With built-in data quality checks and easy access to data outputs and reports, users of the data can feel confident about the quality of the output. The electronic submission also promotes immediate access to the dataset. Once the data are put into the web-based system, it will be available for access, review, and reporting by all those with access to the system from Center staff to the grantee staff.

4.Efforts to Identify Duplication

A program-level review of current measures and methods of collection was conducted to identify duplication of these data collection efforts. With the goal of creating standardized indicators and methods for monitoring grantee performance across the Center, existing measures were considered for use where appropriate. However, modification of current measures was necessary in some cases to generalize across varied programs. Each of these data collection instruments was reviewed and approved by the Government Project Officers, Branch Chiefs, and CMHS senior leadership as meeting the performance monitoring and management needs of individual programs and the Center. Since many of the grantees engaged in infrastructure development, prevention, and mental health promotion activities already collect data for the proposed indicators, the creation of this system will provide them with a standardized method for reporting to CMHS.

5.Involvement of Small Entities

Individual grantees vary from small entities through large provider organizations. Every effort has been made to reduce the number of data items collected from grantees to the least number required to accomplish the objectives of the effort and to meet performance and GPRA reporting requirements and therefore, there is no significant impact involving small entities in general. Based on the pilot test and input and feedback from CMHS Project Officers, however, we understand that it may be difficult for some American Indian/Alaska Native Tribes and tribal organizations to report on the infrastructure development, prevention, and mental health promotion performance indicators on a quarterly basis. We will, therefore, develop a waiver process to allow such grantees to request, through their Project Officers, to report on these indicators every six months rather than quarterly.

6.Consequences If Information Collected Less Frequently

Client-level data

Mental health programs typically collect client-level data at admission and then conduct periodic reassessments of consumers while the individual remains in services. When feasible, mental health providers also conduct an assessment when the consumer is discharged. The data collection schedule for the client-level measures parallels this model. All programs that provide direct services will collect data every six months while the consumer is receiving services; this is a reduction from the prior requirement of quarterly data collection for three of the CMHS programs (the National Child Traumatic Stress Initiative, Meeting the Needs of Elderly Americans, and HIV/AIDS Minority Mental Health Services programs.)

The baseline data collection point is critical for measuring changes. Extending the interval for the periodic reassessment beyond the requested intervals could lead to loss of contact with consumers, significantly diminishing the response rates and lowering the value of the data for performance reporting use by losing measurement of intermediate effects.

Infrastructure development, prevention, and mental health promotion data

This quarterly data collection requirement for the infrastructure development, prevention, and mental health promotion performance indicators is necessary to provide CMHS with the information when needed for appropriate program monitoring and management, as well as for GPRA performance reporting.

7.Consistency with the Guidelines in 5 CFR 1320.5(d) (2)

This information collection fully complies with the guidelines in 5 CFR 1320.5(d) (2).

8.Consultation Outside the Agency

The notice required in 5 CFR 1320.8(d) was published in the Federal Register on December 18, 2012 (77 FR 74854). No comments were received.

Both external and internal stakeholders were consulted by CMHS in the development of these indicators and the data collection methodology. CMHS obtained feedback and consultation regarding the availability of data, methods and frequency of collection, and the appropriateness of data elements.

9. Payment to Respondents

No monetary incentives are provided to grantees.

10.Assurance of Confidentiality

SAMHSA’s grantees do not collect individually identifiable information for these programs. Only aggregated data will be reported by grantees, therefore, SAMHSA and its contractors will not receive identifiable client records. Provider-level information will be aggregated to, at the least, the level of the grant/cooperative agreement-funding announcement.

11.Questions of a Sensitive Nature

No questions of a sensitive nature are asked of State or U.S. Territories grantees. However, grantees who are service providers collect information such as use of alcohol and other drugs and mental health conditions routinely as part of the provision of services. While some of this information may appear sensitive, program participants are service recipients.

12.Estimates of Annualized Hour Burden

The time to complete the instruments is estimated in Table 3. These estimates are based on grantee reports of the amount of time required to complete the currently approved instruments accounting for the additional time required to complete the new questions, as based on an informal pilot and prior CMHS experience in collecting similar data.

Table 3. Estimates of Annualized Hour Burden

Type of Response	Number of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost
Client-level baseline interview	15,681	1	15,681	0.48	7,527	$151	$112,903
Client-level 6-month reassessment interview	10,637	1	10,637	0.367	3,904	$15	$58,557
Client-level discharge interview2	4,508	1	4,508	0.367	1,776	$15	$26,644
Client-level baseline chart abstraction3	2,352	1	2,352	0.1	235	$15	$3,528
Client-level reassessment chart abstraction4	8,703	1	8,703	0.1	870	$15	$13,055
Client-level discharge chart abstraction5	8,241	1	8,241	0.1	824	$15	$12,362
Client-level Subtotal6					15,137	$15	$227,048
Infrastructure development, prevention, and mental health promotion quarterly record abstraction	942	4	3,768	4	15,072	$357	$527,520
TOTAL	16,623				29,298		$885,135

13.Estimates of Annualized Cost Burden to Respondents

There will be no capital, start-up, operation, maintenance, nor purchase costs incurred by the mental health programs participating in this CMHS data collection, or by consumers receiving CMHS-funded services.

14.Estimates of Annualized Cost to the Government

The total contract award to cover all aspects of the design of the study, sampling design, data collection, and development of the data files, data tapes, and technical documentation is $8,807,858 over a 36-month period. Thus, the annualized contract cost is $2,935,953.

Additional costs will be incurred indirectly by the government in personnel costs of staff involved in oversight of data collection. It is estimated that one CMHS employee will each be involved for 100 percent of their time. Costs of CMHS staff time will approximate $120,000 annually.

The estimated annualized total cost to the government will be $2,989,494.

15.Changes in Burden

There are no burden changes.

16.Time Schedule, Publication and Analysis Plans

SAMHSA/CMHS utilizes the NOMs data on an ongoing basis to monitor performance and to respond to GPRAMA and other Federal reporting requirements. These data are used to provide the agency with information to document the overall Center performance requirements and to provide information that will assist CMHS in planning and monitoring program goals. Descriptive information obtained from program reporting requirements will be reviewed for monitoring and program management. Information is used internally by the agency and for performance reports. There are no formal publication plans.

The time frame for submission of the reporting requirements varies by grant cycle and grant program period of performance throughout the year.

17.Display of Expiration Date

The expiration date for OMB approval will be displayed on all data collection instruments.

18.Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.

1 Based on minimum wage.

2 Based on an estimate that 35 percent will leave the program annually, and it will be possible to conduct discharge interviews on 40 percent of those who leave the program.

3 Based on 13 percent non-response for those eligible at baseline (18,033); baselines are required for all consumers served or an admin baseline for non-responders.

4 Based on 40 percent non-response for those eligible for six-month reassessment.

5 Based on 60 percent non-response for those discharged.

6 This is the maximum burden if all consumers complete the baseline and periodic reassessment interviews.

7 To be completed by grantee Project Directors, hence the higher hourly wage.

10