CSAT SSA GPRA OMB Supporting Statement revised 8.12.24 clean

SUPPORTING STATEMENT FOR THE

GOVERNMENT PERFORMANCE AND RESULTS ACT

CLIENT/PARTICIPANT OUTCOME MEASURE

Check off which applies:

☐ New

☒ Revision

☐ Reinstatement with Change

☐ Reinstatement without Change

☐ Extension

☐ Emergency

☐ Existing

JUSTIFICATION

A1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA) is requesting approval from the Office of Management and Budget (OMB) for a revision to the previously approved instrument and data collection activities for the Government Performance and Results Act (GPRA) Center for Substance Abuse Treatment (CSAT) Client/Participant Outcome Measure (OMB No. 0930–0208) which expires on March 31, 2025. SAMHSA is requesting approval to extend its existing CSAT Client-level GPRA instrument.

The tool reflects CSAT’s desire to elicit pertinent client and program level data that can be used to not only guide future programs and practice, but to also respond to stakeholder, congressional, and agency enquiries.

This information is collected using a client-level instrument that provides SAMHSA with the capacity to report on the performance and outcomes for all its discretionary programs, including:

• Numbers of individuals served;

• Demographic characteristics of individuals served;

• Clinical characteristics of individuals served before, during, and after receipt of services; and

• Types of services and activities provided.

To be fully accountable for the spending of federal funds, SAMHSA requires all discretionary programs to collect and report data on all clients served to ensure program goals and objectives are being met. Data collected as part of this package will be used to monitor performance through the applicable grant periods for each program listed.

Approval of this information collection will allow SAMHSA to continue to meet the Government Performance and Results Modernization Act of 2010 (GPRA) reporting requirements that quantify the effects and accomplishments of its discretionary grant programs, consistent with OMB guidance.

To carry out section 1105(a) (29) of the GPRA, SAMHSA is required to prepare a performance plan for its major programs of activity.

SAMHSA’s legislative mandate is to increase access to high quality prevention and treatment services and to improve outcomes. Its mission is to lead public health and service delivery efforts that promote mental health, prevent substance misuse, and provide treatments and supports to foster recovery while ensuring equitable access and better outcomes.

All SAMHSA’s programs and activities are geared toward the achievement of goals related to reducing the impact of substance use and mental health disorders. GPRA performance monitoring is a collaborative and cooperative aspect of this process.

A2. Purposes and Use of Information

SAMHSA uses this measure to report on the performance and outcomes of its discretionary services grant programs. The information is used by individuals at three different levels: the Assistant Secretary and SAMHSA staff, the Center administrators and Government Project Officers (GPOs), and grantees:

Assistant Secretary Level – The information is used to inform the Assistant Secretary for Mental Health and Substance Use of the performance and outcomes of the programs funded through the Agency. The performance is based on the goals of the grant program. This information serves as the basis of the annual GPRA report to Congress contained in the Justifications of Budget Estimates.

Center Level – In addition to providing information about the performance of the various programs, the information is used to monitor and manage individual grant projects within each program. The information is used by GPOs to identify program strengths and weaknesses, to provide an informed basis for providing technical assistance and other support to grantees, to inform funding decisions, and to identify potential issues for additional evaluation.

Grantee Level – In addition to monitoring performance and outcomes, the grantee staff uses the information to improve the quality of treatment and recovery services provided to clients within their projects.

SAMHSA and its Centers will use the data for annual reporting required by GPRA to describe and understand changes in outcomes from baseline to follow-up to discharge. GPRA requires that SAMHSA’s report for each fiscal year include actual results of performance monitoring for the three preceding fiscal years. The information collected through this revised data collection process allows SAMHSA to report on the results of performance and outcomes in a manner that is consistent with SAMHSA specific performance domains, and to assess the accountability and performance of its discretionary and formula grant programs.

Outcomes data reflect the Agency’s desire for consistency in data collected across the Agency. These domains represent SAMHSA CSAT’s focus on the factors that contribute to the success of substance use disorder treatment. The CSAT Client/Participant Outcome Measure will address the following performance domains:

• Abstinence from Drug / Alcohol Use

• Employment / Education

• Crime and Criminal Justice

• Family and Living Conditions

• Social Connectedness

• Access / Capacity

• Retention

• Recovery

SAMHSA and its Centers will use the data from the current data collection instrument (OMB No. 0903-0208) for annual reporting required by GPRA and comparing baseline with discharge and follow-up data. GPRA requires that SAMHSA’s fiscal year report include actual results of performance monitoring for the three preceding fiscal years. The additional information collected through this process will allow SAMHSA to: 1) report results of these performance outcomes; 2) maintain consistency with SAMHSA-specific performance domains, and 3) assess the accountability and performance of its grant programs including a focus on health equity.

The tool reflects SAMHSA’s desire to elicit pertinent client and program level data that can be used to not only guide future CSAT programs and practice, but to also respond to stakeholder, congressional and agency enquiries.

Proposed Changes to Data Collection Instrument

SAMHSA is proposing to revise the current approved version of the data collection instrument (OMB No. 0930-0208) to maintain performance monitoring and outcome measurement of its programs supporting recovery from mental illness and substance use disorders while SAMHSA completes development of the new client-level services information collection instrument.

SAMHSA’s data collection activities are intended to promote the use of consistent measures among CSAT-funded grantees and contractors. These measures are a result of extensive examination and recommendations, using consistent criteria, by panels of staff, experts, and grantees. Wherever feasible, the measures are consistent with or build upon previous data development efforts within CSAT. These data collection activities are organized to reflect and support the National Outcome Measures domains specified for SAMHSA’s programs providing direct services.

A3. Use of Improved Information Technology

Programs collect client information using a variety of methods, including paper-and-pencil and electronic methods. This project will not interfere with ongoing program collection operations that facilitate information collection at each site.

A web-based data collection and entry system, SAMHSA’s Performance Accountability and Reporting System (SPARS), has been developed and is currently used and available to all programs for data collection. This web-based system allows for easy data entry, submission, and reporting to all those who have access to the system. Levels of access have been defined for users based on their authority and responsibilities regarding the data and reports. Access to the data and reports is limited to those individuals with a username and password. A screenshot of the data entry screen on SPARS is below:

Programs may submit their data electronically through an upload process. This facilitates the submission of data while avoiding duplication of the data entry process.

Electronic submission of the data promotes enhanced data quality. With built-in data quality checks, and easy access to data outputs and reports, users of the data can feel confident about the quality of the output. The electronic submission also promotes immediate access to the dataset. Once the data are put into the web-based system, it is available for access, review, and reporting by all those with access to the system from Center staff to the grantee staff.

A4. Efforts to Identify Duplication

The items collected are necessary to assess grantee performance. CSAT is promoting the use of consistent performance and outcomes measures across all programs; this effort will result in less overlap and duplication and will substantially reduce the burden on grantees that results from data demands associated with individual programs.

A program-level review of current measures and methods of data collection was conducted to identify duplication of these data collection efforts. With the goal of creating questions for more precise monitoring of grantee performance across the Center, existing questions were considered for use where appropriate. Each of the proposed questions was reviewed and approved by CSAT senior leadership as meeting the performance monitoring and management needs of individual programs and the Center.

SAMHSA will work closely with the grantees to identify whether other data are being collected by the grantee, which may be redundant to the data collection instrument. When duplication is identified, SAMHSA and the grantees will identify a priority action plan to reduce the duplicative efforts and streamline the data items to reduce client burden.

A5. Involvement of Small Entities

Individual grantees vary from small entities to large provider organizations. Every effort has been made to minimize the number of data items collected from all programs down to the least number of items necessary to accomplish the objectives described within and meet GPRA reporting requirements. Therefore, there is no significant impact to small entities.

A6. Consequences if Information Collected Less Frequently

Substance use disorder treatment programs collect data at three time points: intake, six-month follow-up, and discharge. Of note, the six-month follow-up data collection may occur after the client has been discharged from the program. These times points are part of regular program activity.

These data collection points are generally accepted intervals for client assessment and the participants will be asked to respond to the items according to this schedule. The grantees for adolescent substance use disorder treatment programs are required to collect information additionally at three months follow-up due to the transitory nature of adolescents. It is more difficult to locate adolescents than adults and, therefore, locating them more frequently and closer to their intake date should increase their follow-up rates. The data will be reported to SAMHSA on an annual basis in keeping with the GPRA requirements for annual reporting.

A7. Consistency with the Guidelines in 5 CFR1320.5(d)(2)

This information collection fully complies with 5 CFR 1320.5(d) (2).

SAMHSA is in the process of establishing a new data collection tool for client services which will be used by discretionary grants from both CMHS and CSAT. While working towards this new combined data collection for client services, SAMHSA is requesting to extend the time period for data collection using the currently approved tools without change to the tools. As a result, SAMHSA also requests exemption from the OMB Statistical Policy Directive No. 15: Standards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity for this request to renew without change to the tool.

A8. Consultation Outside the Agency

The notice required by 5 CFR 1320.8(d) was published in the Federal Register on September 11, 2024 (89 FR 73665). No comments were received in response to this notice.

A9. Payment to Respondents

Grantees are asked to budget for data collection in their grant applications and individual grantees are not prohibited from providing payments to their respondents for follow-up data collection, which is customary practice in the field. If the grantees do provide payment for the follow-up, the maximum incentive is $30.00 or the equivalent in coupons, transportation tokens, or other items per follow-up.

Survey research literature suggests that monetary incentives have a strong positive effect on response rates and no known adverse effect on reliability. Individuals with substance use disorder research have shown improved response rates when remuneration is offered to respondents. Individuals with substance use disorder are typically a harder-to-reach population for whom out-of-pocket costs of participation (e.g., transportation, childcare) are significant barriers.

A10. Assurance of Confidentiality

Data will be kept private to the extent allowed by law. The information from Grantees and all other potential respondents will be kept private through all points in the data collection and reporting processes. However, SAMHSA cannot ensure complete confidentiality of client data. SAMHSA will work with each grantee to prepare an impact assessment protocol. All data will be closely safeguarded, and no institutional or individual identifiers will be used in reports. Only aggregated data will be reported. SAMHSA and its contractors will not receive identifiable client records. Provider-level information will be aggregated to, at least, the level of the grant/cooperative agreement-funding announcement.

SAMHSA has statutory authority to collect data under the GPRA (Public Law 1103(a), Title 31) and is subject to the Privacy Act for the protection of data. Federally assisted substance use treatment providers are subject to the federal regulations for alcohol and substance use patient records (42 CFR Part 2) (OMB No. 0930-0092) which govern the protection of patient identifying data. In some cases, these same providers meet the definition of a Health Insurance Portability and Accountability Act covered entity and are additionally subject to the Privacy Rule (45 CFR Parts 160 and 164) for the protection of individually identifiable data.

A11. Questions of a Sensitive Nature

SAMHSA’s mission is to lead public health and service delivery efforts that promote mental health, prevent substance misuse, and provide treatments and supports to foster recovery while ensuring equitable access and better outcomes. In carrying out this mission, it is necessary for service providers to collect sensitive items such as experiences with violence and trauma, legal involvement, use of alcohol or other drugs, as well as issues of mental health. The data that will be submitted by each grantee will be based in large part on data that most of the programs are already routinely collecting. This primarily includes data on client demographics, substance use and treatment history, services received, and client outcomes. These issues are essential to the service/treatment context. Grant projects use informed consent forms as required and as viewed appropriate by their individual organizations. They also use the appropriate forms for minor/adolescent participants requiring parental approval. Client data are routinely collected and subject to the Federal Regulations on Human Subject Protection (45 CFR Part 46; OMB No. 0925-0404). Alcohol and drug use client records in federally supported programs are also protected by 42 CFR Part 2. The informed consent forms usually contain the following elements:

• Explanation of the purpose of the program or research.

• Expected duration of the subject’s participation.

• Description of the procedures to be followed.

• Identification of any procedures that are experimental.

• Description of any reasonably foreseeable risks or discomforts to the subject.

• Disclosure of appropriate alternative procedures or courses of treatment.

• Statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained.

• Contact names & phone numbers for participants to ask questions about program, participant rights, and injury.

A12. Estimates of Annualized Hour Burden

The time to complete the instruments is estimated in Table 1. These estimates are based on current funding and planned fiscal year 2024 notice of funding announcements and the average number of consumers served in fiscal years 2022-2023; the amount of time required to complete the new questions is based on an informal pilot and prior SAMHSA/CSAT experience in collecting similar data.

The estimated time to complete the baseline, follow-up, and discharge interviews is 45 (0.75) minutes each. This includes the completion of the administrative sections of the tool for all clients including those who decline an interview. The estimated time to complete the SBIRT program-specific measures was increased from 12 (0.2) minutes to 15 minutes (0.25).  

The estimates in this table reflect the maximum annual burden for currently funded discretionary services programs. The number of clients served in following years is estimated to be the same assuming level funding of the discretionary programs, resulting in the same annual burden estimate for those years.

Table 1: Estimates of Annualized Hour Burden

SAMHSA Tool	Number of Respondents	Responses per Respondent	Total Number of Responses	Burden Hours per Response	Total Burden Hours	Hourly Wage [1]	Total Hour Cost
Baseline Interview Includes SBIRT Brief TX, Referral to TX, and Program-specific questions	337,857	1	337,857	0.75	253,393	$28.89	$7,320,523
Follow-Up Interview with Program-specific questions [2]	270,286	1	270,286	0.75	202,715	$28.89	$5,856,436
Discharge Interview with Program-specific questions [3]	175,686	1	175,686	0.75	131,765	$28.89	$3,806,431
SBIRT Program –Screening Only	150,296	1	150,296	0.17	25,550	$28.89	$738,140
SBIRT Program – Brief Intervention Only Baseline	31,481	1	31,481	0.25	7,870	$28.89	$227,364
SBIRT Program – Brief Intervention Only Follow-Up2	25,184	1	25,184	0.25	6,296	$28.89	$181,891
SBIRT Program – Brief Intervention Only Discharge3	16,370	1	16,370	0.25	4,093	$28.89	$118,247
CSAT Total	1,007,160		1,007,160		631,682		$18,249,032

^[1] The hourly wage estimate is based on the Occupational Employment and Wages, Mean Hourly Wage Rate for 21-1011 Substance Abuse and Behavioral Disorder Counselors = $28.89/hr. as of May 11, 2023.  (http://www.bls.gov/oes/current/oes211011.htm. Accessed on June 20, 2024.)

^[2] It is estimated that 80% of baseline clients will complete this interview.

^[3] It is estimated that 52% of baseline clients will complete this interview. This estimate is based on the results in the following: Substance Abuse and Mental Health Services Administration (SAMHSA): Treating Concurrent Substance Use Among Adults. SAMHSA Publication No. PEP21-06-02-002. Rockville, MD: National Mental Health and Substance Use Policy Laboratory. Substance Abuse and Mental Health Services Administration, 2021

Note: Numbers may not add to the totals due to rounding and some individual participants completing more than one form.

A13. Estimates of Annualized Cost Burden to Respondents

There are no capital or startup costs, nor are there any operation and maintenance costs.

A14. Estimates of Annualized Cost to the Government

The principal additional cost to the government for this project is the cost of a contract to collect the data from the various programs and to conduct analyses, which generate routine reports from the data collected. The reports examine baseline characteristics and changes between baseline, discharge, and each of the follow-up periods. It is the responsibility of the contractor to work with the GPO when preparing reports that combine the client services data with the annual reports of the project.

The estimated annualized cost for a contract for the GPRA mandate is $7.2 million and the cost of one full-time equivalent staff (25% for one GS-14 Step 5, $34,716.50 and 75% for one GS-12 Step 5, $74,120.25) responsible for the CSAT data collection effort is approximately $108,837/year.

A15. Changes in Burden

Currently, there are 379,037 total burden hours in the OMB-approved CSAT Client-level GPRA instrument. SAMHSA is now requesting an increase to 631,682 burden hours. The increase of 252,645 burden hours is due to the following: 

• Additional time allocated for interviews, but also improved estimates of the number of clients who would likely consent to complete the interview; and 

• Additional time allocated for administrative collection of data by grantees, including the information that is collected for all clients regardless of whether they completed the client-portion of the interview or not. 

A16. Time Schedule, Publication and Analysis Plans

Data for the annual GPRA plan/report are needed by SAMHSA by September of each year. The discretionary services program data are readily available through the web-based system. Data are provided for the most recently completed calendar year to SAMHSA in May to assure analysis in time for the annual GPRA report. The annual GPRA report must be submitted to the U.S. Department of Health and Human Services and to OMB by September and is included in the President's annual budget request which is released to the public February 1st. Data may be refined and added to the final Presidential budget request after the Department submits its initial GPRA report.

Analysis/Publication Plans

Client outcome data will be collected through the web site. Data will be used to report to Congress regarding the GPRA as specified in the SAMHSA Annual Justifications of Budget Estimates. The data might also be used for specific comparisons relative to the Office of National Drug Control Policy’s National Drug Control Strategic Goals, especially for some of the secondary treatment outcomes (e.g., homelessness).

In the future, the indicators for clients served under these programs might be compared to similar indicators for clients served under block grant programs as a general indicator of whether the programs are doing better than "typical" services. This could be done for discretionary services programs as a group or for specific programs.

SAMHSA and each of its Centers specifically will use the data for annual reporting required by GPRA on the previously stated items, comparing baseline with discharge and follow-up data. The GPRA dataset will consist of each element coded into the reporting categories as seen in Attachment 1. These data are at the client record level. The SAMHSA GPRA performance and client outcome data will be aggregated at the following levels: Project/Grantee, Program/Division, and Activity. The analyses will be organized around SAMHSA's GPRA measures and the measures relating to the National Outcome Measures.

Baseline-level analyses involve using frequency distributions and measures of central tendency to describe the populations across the GPRA client outcomes and by various demographic groups (e.g., gender identity, sexual orientation, race, ethnicity, age, and level of education). The client will be followed longitudinally, with the GPRA client outcome items re-administered again at discharge and six months after baseline. The follow-up data also will be described using frequency distributions and measures of central tendency. Change will be addressed by comparing the discharge and follow-up measurements with baseline data for each client. The percent of clients showing the target changes will be calculated on each of the GPRA client outcome measures that are categorical. For continuous items, mean differences will be calculated. Tables will be constructed to describe the change across projects on client outcomes.

There will also be program-specific analysis of these data because each program may have unique programmatic and performance goals. The data items collected will be analyzed and presented in GPRA reports using basic descriptive statistics. On key outcomes (e.g., drug use, criminal involvement, and employment), the proportion of individuals showing improvement from baseline to discharge and follow-up (baseline to discharge, baseline to six months) will be calculated and aggregated at the program level (e.g., discretionary services). If deemed necessary for CSAT specific issues, the data will be examined at the individual activity level. The results will be examined for subpopulations of interest within individual activities (e.g., by age, by sexual orientation, by gender identity, by race/ethnicity, etc.).

A17. Display of Expiration Date

The expiration date for OMB approval will be displayed on all data collection instruments.

A18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.

Hourly Wage [

[3]

10