0208-ss-a- Csat 12.8.16

SUPPORTING STATEMENT FOR THE

GOVERNMENT PERFORMANCE AND RESULTS ACT

CLIENT/PARTICIPANT OUTCOME MEASURES

JUSTIFICATION

A1. Circumstances Making the Collection of Information Necessary

The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Treatment (CSAT) is requesting from the Office of Management and Budget (OMB) approval for revisions to the previously approved instrument and data collection activities for the Government Performance and Results Act (GPRA) Client/Participant Outcome Measures for Discretionary Programs and Instructions (OMB No. 0930–0208) which expires on March 31, 2019. Specifically, CSAT is requesting approval to add eight new recovery questions to its existing GPRA instrument.

This information is collected using a client tool that provides CSAT the capacity to report for all of its discretionary programs: particular populations served, numbers of people served, types and locations of particular activities supported, effectiveness across programs for particular populations, and the characteristics and effectiveness across programs of activities relative to national, subpopulation and geographic area data and trends. In order to be fully accountable for the spending of federal funds, SAMHSA/CSAT requires all its programs to collect and report data on all clients served to ensure program goals and objectives are being met. Data collected as part of this package are used a tool to monitor performance through the grant period and to ensure appropriate spending of federal funds.

Approval of this information collection will allow SAMHSA to continue to meet the Government Performance and Results Modernization Act of 2010 (GPRMA) reporting requirements that quantify the effects and accomplishments of its discretionary grant programs which are consistent with OMB guidance.

In order to carry out section 1105(a) (29) of the GPRA, SAMHSA is required to prepare a performance plan for its major programs of activity. This plan must:

a) Establish performance goals to define the level of performance to be achieved by a program activity;

b) Express such goals in an objective, quantifiable, and measurable form;

c) Briefly describe the operational processes, skills and technology, and the human, capital, information, or other resources required to meet the performance goals;

d) Establish performance indicators to be used in measuring or assessing the relevant outputs, service levels, and outcomes of each program activity;

e) Provide a basis for comparing actual program results with the established performance goals; and

f) Describe the means to be used to verify and validate measured values.

SAMHSA’s legislative mandate is to increase access to high quality prevention and treatment services and to improve outcomes. Its mission is to reduce the impact of substance use and mental illness on our communities.

All SAMHSA’s programs and activities are geared toward the achievement of goals related to reducing the impact of substance use and mental health disorders. GPRA performance monitoring is a collaborative and cooperative aspect of this process.

SAMHSA is striving to coordinate the development of these goals with other ongoing activities in performance measurement development, for example, development of performance measures for reporting of activities. This information collection is needed to provide objective data to demonstrate SAMHSA’s monitoring and achievement of its mission and goals.

Revisions have been made to the tool to include recovery measure questions.

A2. Purposes and Use of the Information Collection

In support of its Recovery Support Strategic Initiative, SAMHSA first developed a working-definition of recovery which was arrived at via stakeholder input, a series of facilitated discussions, and a comprehensive review of the literature. These efforts resulted in the following definition of recovery: “A process of change through which individuals improve their health and wellness, live a self-directed life, and strive to reach their full potential.” After developing the recovery definition and supporting guiding principles, SAMHSA leadership sought a means for operationalizing recovery among the SAMHSA grantee population.

In consultation with SAMHSA’s Recovery Measure Expert Panel, a group comprised of content experts from the recovery field as well as consumers in recovery themselves, the expert team identified a need to develop an instrument that can be used to assess the degree to which recovery is occurring in SAMHSA funded programs. Staff from SAMHSA’s Center for Behavioral Health Statistics and Quality (CBHSQ) was then charged with identifying an appropriate recovery instrument to capture the four dimensions of recovery (i.e. home, health, purpose and community) that contribute to SAMHSA’s working definition of recovery. Based on an extensive literature review, SAMHSA identified the World Health Organization’s Quality of Life (WHOQOL8) as the existing tool best suited to capture these four dimensions.

SAMHSA uses the performance measures to report on the performance of its discretionary services grant programs. The performance measures information is used by individuals at three different levels: the SAMHSA administrator and staff, the Center administrators and government project officers, and grantees:

SAMHSA Administrator Level—The information is used to inform the administration of the performance of the programs funded through the Agency. The performance is based on the goals of the grant program and includes the National Outcome Measure (NOMs). This information serves as the basis of the annual GPRA report to Congress contained in the Justifications of Budget Estimates.

Center Level—In addition to exploring the performance of the various programs, the information is used to monitor and manage individual grant projects within each program. The information informs the government project officers of the program staff’s abilities to meet their individual goals. The information has been used by government project officers to make funding continuation decisions.

Grantee Level—In addition to monitoring performance outcomes, the grantee staff uses the information to improve the quality of treatment and prevention services provided to clients within their projects.

SAMHSA and its Centers will use the data for annual reporting required by GPRA and for comparing NOMs baseline with discharge and follow-up data. GPRA requires that SAMHSA’s report for each fiscal year include actual results of performance monitoring for the three preceding fiscal years. The additional information collected through this process will allow SAMHSA to report on the results of these performance outcomes as well as to be consistent with the specific performance domains that SAMHSA is implementing as the NOMs, and to assess the accountability and performance of its discretionary and formula grant programs. The CSAT client-level data items were initially identified from widely used data collection instruments.

Outcome data reflect the Agency’s desire for consistency in data collected within the Agency. SAMHSA has implemented specific performance domains called NOMs to assess the accountability and performance of its discretionary and formula grant programs. These domains represent SAMHSA CSAT’s focus on the factors that contribute to the success of substance abuse treatment. The CSAT Client/Participant Outcome Measures will address the following performance domains:

• Abstinence from Drug / Alcohol Use

• Employment / Education

• Crime and Criminal Justice

• Family and Living Conditions

• Social Connectedness

• Social Consequences from Drug / Alcohol Use

• Access / Capacity

• Retention

• Recovery

Based on current funding and planned fiscal year 2016 notice of funding announcements (NOFA), the CSAT programs that will use these measures in fiscal years 2016 through 2019 include: Access to Recovery (ATR)3 and 4; Adult Treatment Court Collaboratives (ATCC); Enhancing Adult Drug Court Services, Coordination and Treatment (EADCS); Offender Reentry Program (ORP); Treatment Drug Court (TDC); Office of Juvenile Justice and Delinquency Prevention – Juvenile Drug Courts (OJJDP-JDC); ; HIV/AIDS Outreach Program; Targeted Capacity Expansion Program for Substance Abuse Treatment and HIV/AIDS Services (TCE-HIV); Addictions Treatment for the Homeless (AT-HM); Cooperative Agreements to Benefit Homeless Individuals (CABHI); Cooperative Agreements to Benefit Homeless Individuals – States (CABHI- States); Recovery-Oriented Systems of Care (ROSC); Targeted Capacity Expansion- Peer to Peer (TCE – PTP); Pregnant and Postpartum Women (PPW); Screening, Brief Intervention and Referral to Treatment (SBIRT); Targeted Capacity Expansion (TCE); Targeted Capacity Expansion- Health Information Technology (TCE-HIT); Targeted Capacity Expansion Technology Assisted Care (TCE-TAC); Addiction Technology Transfer Centers (ATTC); International Addiction Technology Transfer Centers (I-ATTC); State Adolescent Treatment Enhancement and Dissemination (SAT-ED); Grants to Expand Substance Abuse Treatment Capacity in Adult Tribal Healing to Wellness Courts and Juvenile Drug Courts; and Grants for the Benefit of Homeless Individuals – Services in Supportive Housing (GBHI). Grantees in the Adult Treatment Court Collaborative program (ATCC) will also provide program-level data using the CSAT Aggregate Instrument.

Proposed Changes to Data Collection Tool

SAMHSA is proposing the revision of this data collection instrument (OMB No. 0903-0208) in order to improve outcome measurement of its strategic initiative supporting recovery from mental health and substance use disorders. SAMHSA has developed a standard measure of recovery that can be used as part of its grantee performance reporting activities. SAMHSA’s Center for Behavioral Health Statistics and Quality (CBHSQ) previously conducted a pilot test of its Recovery Measure (OMB No. 0930-0342). The Center for Substance Abuse Treatment is now requesting approval to include the WHOQOL8 questions into its regular performance data collection as an ongoing measure of recovery.

CSAT is seeking approval to add the followingWHOQOL8 questions to its currently approved data collection:

Question Number	Item
1	How would you rate your quality of life?
2	How satisfied are you with your health?
3	Do you have enough energy for everyday life?
4	How satisfied are you with your ability to perform your daily activities?
5	How satisfied are you with yourself?
6	How satisfied are you with your personal relationships?
7	Have you enough money to meet your needs?
8	How satisfied are you with the conditions of your living space?

All of CSAT’s data collection activities are intended to promote the use of consistent measures among CSAT-funded grantees and contractors. These measures are a result of extensive examination and recommendations, using consistent criteria, by panels of staff, experts, and grantees. Wherever feasible, the measures are consistent with or build upon previous data development efforts within CSAT. These data collection activities are organized to reflect and support the domains specified for SAMHSA’s NOMs for programs providing direct services.

A3. Use of Improved Information Technology and Burden Reduction

Most programs collect their client information using a variety of methods from paper and pencil to electronic methods. This project will not interfere with ongoing program collection operations that facilitate information collection at each site.

A web-based data collection and entry system has been developed through CSAT and is available to all programs for data collection. This web-based system allows for easy data entry, submission, and reporting to all those who have access to the system. Levels of access have been defined for users based on their authority and responsibilities regarding the data and reports. Access to the data and reports is limited to those individuals with a username and password. A sample data entry screen is below:

A few programs submit their data electronically through an upload process. This facilitates the submission of data while avoiding duplication of the data entry process. Programs that collect these data for other purposes are spared an additional collection burden.

Electronic submission of the data promotes enhanced data quality. With built-in data quality checks, easy access to data outputs and reports, users of the data can feel confident about the quality of the output. The electronic submission also promotes immediate access to the dataset. Once the data are put into the web-based system, it is available for access, review, and reporting by all those with access to the system from Center staff to the grantee staff.

A4. Efforts to Identify Duplication and Use of Similar Information

The items collected are necessary in order to assess grantee performance. SAMHSA is promoting the use of performance measures across all programs; this effort will result in less overlap and duplication, and will substantially reduce the burden on grantees that results from data demands associated with individual programs. SAMSHA will work closely with the grantees to identify whether other data are being collected by the grantee, which may be redundant to the GPRA instrument. When duplication is identified, SAMHSA and the grantees will identify a priority action plan to leverage the duplicative efforts, and streamline the data items to reduce client burden.

A5. Involvement of Small Entities

Individual grantees vary from small entities through large provider organizations. Every effort has been made to minimize the number of data items collected from programs to the least number required to accomplish objectives of the effort and to meet GPRA reporting requirements and therefore, there is no significant impact involving small entities.

A6. Consequences of Collecting the Information Less Frequently

The data collection points remain unchanged from the previous submission. Substance abuse treatment programs collect data at three time points: intake, discharge, and 6-months post intake. These times points are part of regular program activity.

These data collection points are generally accepted intervals for client assessment and the participants will be asked to respond to the items according to this schedule. The grantees for adolescent substance abuse treatment programs are required to collect information additionally at three months post-intake due to the migratory nature of adolescents. It is more difficult to locate adolescents than adults and, therefore, locating them more frequently and closer to their intake date should increase their follow-up rates. The data will be reported to SAMHSA on an annual basis in keeping with the GPRA requirements for annual reporting.

A7. Consistency with the Guidelines in 5 CFR1320.5(d)(2)

This information collection fully complies with 5 CFR 1320.5(d) (2).

A8. Consultation Outside the Agency

The notice required by 5 CFR 1320.8(d) was published in the Federal Register on October 7, 2016 (81 FR 69836). No comments were received in response to this notice.

A9. Payment to Respondents

Grantees are asked to budget for data collection in their grant applications and individual grantees are not prohibited from providing payments to their respondents for follow-up, which is customary practice in the field. If the grantees do provide payment for the follow-up, the maximum incentive is $20.00 or the equivalent in coupons, transportation tokens, or other items per follow-up.

Survey research literature suggests that monetary incentives have a strong positive effect on response rates and no known adverse effect on reliability. In particular, substance abuse research has shown improved response rates when remuneration is offered to respondents. Substance abusers are typically a harder-to-reach population for whom out-of-pocket costs of participation (e.g., transportation, child care) are significant barriers.

A10. Assurance of Confidentiality Respondents

The information from Grantees and all other potential respondents will be kept private through all points in the data collection and reporting processes. However, SAMHSA cannot ensure complete confidentiality of client data. SAMHSA will work with each grantee to prepare an impact assessment protocol. All data will be closely safeguarded, and no institutional or individual identifiers will be used in reports. Only aggregated data will be reported. SAMHSA and its contractors will not receive identifiable client records. Provider-level information will be aggregated to, at least, the level of the grant/cooperative agreement-funding announcement.

SAMHSA has statutory authority to collect data under the Government Performance and Results Act (Public Law 1103(a), Title 31) and is subject to the Privacy Act for the protection of data. Federally assisted substance abuse treatment providers are subject to the federal regulations for alcohol and substance abuse patient records (42 CFR Part 2) (OMB No. 0930-0092) which govern the protection of patient identifying data. In some cases, these same providers meet the definition of a HIPAA covered entity and are additionally subject to the Privacy Rule (45 CFR Parts 160 and 164) for the protection of individually identifiable data.

A11. Questions of a Sensitive Nature

SAMHSA’s mission is to improve the quality and availability of prevention, early intervention, treatment, and rehabilitation services for substance abuse and mental illnesses, including co-occurring disorders, in order to improve health and reduce illness, death, disability, and cost to society. In carrying out this mission it is necessary for service providers to collect sensitive items such as experiences with violence and trauma, criminal justice involvement, use of alcohol or other drugs, as well as issues of mental health. The data that will be submitted by each grantee will be based in large part on data that most of the programs are already routinely collecting. This primarily includes data on client demographics, substance abuse and treatment history, services received, and client outcomes. These issues are essential to the service/treatment context. Grant projects use informed consent forms as required and as viewed appropriate by their individual organizations. They also use the appropriate forms for minor/adolescent participants requiring parental approval. Client data are routinely collected and subject to the Federal Regulations on Human Subject Protection (45 CFR Part 46; OMB No. 0925-0404). Alcohol and drug abuse client records in Federally supported programs are also protected by 42 CFR Part 2. The informed consent forms usually contain the following elements:

• Explanation of the purpose of the program or research.

• Expected duration of the subject’s participation.

• Description of the procedures to be followed.

• Identification of any procedures that are experimental.

• Description of any reasonably foreseeable risks or discomforts to the subject.

• Disclosure of appropriate alternative procedures or courses of treatment.

• Statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained.

• Contact names & phone numbers for participants to ask questions about program, participant rights, and injury.

A12. Estimates of Annualized Hour Burden

A typical grantee currently collects intake, or pre-intervention information at the beginning of program contact, and many also collect standard discharge and follow-up information with similar items. Data are usually collected through interviews for the programs. Across all the SAMHSA discretionary services grants to which this application applies, it is estimated that these customary and usual business practices for services and treatment take about 28 minutes (0.46 hours). Additional burden will only be created where grants are required to collect GPRA core measures at either intake, discharge, or follow-up points that are not customary and usual practices. In these cases the client’s time and effort are required to gather additional information that would not have been part of normal treatment or service activities.

The first value computed is the proportion of additional core GPRA items for a typical Center grant. This is done using the following formula:

Added Burden Proportion For Grant Programs. There are 88 items (including record management) in the CSAT GPRA Client/Participant Outcome Measures for Discretionary Programs, which will take approximately 28 minutes per client to administer at each of the 3 or 4 data collection points. However, 42 of the items are taken from the ASI, which is used in the substance abuse treatment field by researchers and providers as a baseline and follow-up instrument, or are considered standard items in the field. The resulting Added Burden Proportion is then (88-42)/88, or .52.

Estimates of Annualized Hour Burden ¹

CSAT GPRA Client Outcome Measures for Discretionary Programs

SAMHSA Program Title	Number of Respondents	Responses per Respondent	Total Number of Responses	Burden Hours per Response	Total Burden Hours	Hourly Wage 1	Total Hour Cost
Baseline Interview Includes SBIRT Brief TX and Referral to TX	179,668	1	179,668	0.52	75,460	$20.13	$1,928,340.71
Follow-Up Interview 2	132,954	1	143,734	0.52	60,386	$20.13	$1,426,968.69
Discharge Interview 3	93,427	1	94,720	0.52	39,782	$20.13	$1,002,733.31
SBIRT Program –Screening Only 4	594,192	1	594,192	0.13	77,244	$20.13	$1,594,336
SBIRT Program – Brief Intervention Only 5 Baseline	111,411	1	111,411	.20	22,282	$20.13	$459,905
SBIRT Program – Brief Intervention Only Follow-Up 2	82,444	1	82,444	.20	16,489	$20.13	$340,329
SBIRT Program – Brief Intervention Only Discharge 3	57,934	1	57,934	.20	11,587	$20.13	$239,152
CSAT Total	1,252,030		1,252,030		338,748		$6,991,763.67

NOTES:

1. The hourly wage estimate is $20.64 based on the Occupational Employment and Wages, March 2016 Mean Hourly Wage Rate for 21-1011 Substance Abuse and Behavioral Disorder Counselors  = $20.64/hr. as of March, 2016.  (http://www.bls.gov/oes/current/oes211011.htm  (Accessed on July 28, 2016).

2. It is estimated that 80% of baseline clients will complete this interview.

3. It is estimated that 52% of baseline clients will complete this interview.

4. The estimated number of SBIRT respondents receiving screening services is 80% of the total number SBIRT participants. No further data are collected from these participants.

5. The estimated number of SBIRT respondents receiving brief intervention services is 15% of the total number SBIRT participants.

Note: Numbers may not add to the totals due to rounding and some individual participants completing more than one form.

The estimates in this table reflect the maximum annual burden for currently funded discretionary services programs. The number of clients served in following years is estimated to be the same assuming level funding of the discretionary programs, resulting in the same annual burden estimate for those years.

A13. Estimates of Cost Burden to Respondents

There are no capital or startup costs, nor are there any operation and maintenance costs.

A14. Estimates of Annualized Cost to the Federal Government

The principal additional cost to the government for this project is the cost of a contract to collect the data from the various programs and to conduct analyses which generate routine reports from the data collected. The reports examine baseline characteristics as well as the changes between baseline, discharge, and each of the follow-up periods. It is the responsibility of the contractor to work with the Government Project Officer (GPO) when preparing reports that combine the client services data with the annual reports of the project.

The estimated annualized cost for a contract for the GPRA mandate is $7.2 million and the cost of 1 FTE staff (25% for the midpoint of one GS-14 $25,899 and 75% for one GS-12 $48,786) responsible for the CSAT data collection effort is approximately $74,685/year.

A15. Changes in Burden

Currently there are 321,085 burden hours in the OMB inventory. SAMHSA is now requesting 338,748 hours.  This increase of 17,663 hours is due to the following program changes:

• A decrease in 2,485 respondents, 1,044 hours in the elimination of the Teen Drug Court (TDC) program.

• Addition of the WHO-QOL 8 recovery measure questions (16,619 additional burden hours).  The estimated time to complete the client interview with the revised tool has been revised to 28 minutes from 25.

A16. Time Schedule, Publication and Analysis Plans

Data for the annual GPRA plan/report are needed by SAMHSA by September of each year. The discretionary services program data are readily available through the web-based system. Data are provided for the most recently completed calendar year to SAMHSA in May in order to assure analysis in time for the annual GPRA report. The annual GPRA report must be submitted to the U.S. Department of Health and Human Services (the Department) and to OMB by September and is included in the President's annual budget request which is released to the public February 1st. Data may be refined and added to the final Presidential budget request after the Department submits its initial GPRA report.

Analysis/Publication Plans

Client outcome data will be collected through the web site. Data will be used to report to Congress regarding the GPRA as specified in the SAMHSA Annual Justifications of Budget Estimates. The data might also be used for specific comparisons relative to ONDCP National Drug Control Strategic Goals, especially for some of the secondary treatment outcomes (e.g., homelessness).

In the future, the indicators for clients served under these programs might be compared to similar indicators for clients served under block grant programs as a general indicator of whether the programs are doing better than "typical" services. This could be done for discretionary services programs as a group or for specific programs.

SAMHSA and each of its Centers specifically will use the data for annual reporting required by GPRA on the previously stated items, comparing baseline with discharge and follow-up data. The GPRA dataset will consist of each element coded into the reporting categories as seen in Attachment 1. These data are at the client record level. The SAMHSA GPRA client outcome data will be aggregated at the following levels: Project/Grantee, Program/Division, and Activity. The analysis will be organized around SAMHSA's GPRA measures and the measures relating to the Family Drug Courts and the NOMs.

Baseline level analysis involves using frequency distributions and measures of central tendency to describe the populations across the GPRA client outcomes and by various demographic groups (e.g., gender, race, ethnicity, age, and level of education). The client will be followed longitudinally, with the GPRA client outcome items re-administered again at discharge, 6 and 12 months after baseline. The follow-up data also will be described using frequency distributions and measures of central tendency. Change will be addressed by comparing the discharge and follow-up measurements with baseline data for each client. The percent of clients showing the target changes will be calculated on each of the GPRA client outcome measures that are categorical. For continuous items, mean differences will be calculated. Tables will be constructed to describe the change across projects on client outcomes.

It is important to note that each Center is responsible for its own analyses of the data. Common analyses will be used as appropriate for GPRA purposes, but control of the data rests with the Center funding the grant. The Centers submit a GPRA report to SAMHSA Office of the Administrator, and SAMHSA then synthesizes results from the Centers in a descriptive manner for the GPRA report.

There also will be Center unique analysis of these data because each Center has a distinct set of programs. The data items collected will be analyzed and presented in GPRA reports using basic descriptive statistics. On the principal outcome items (e.g., drug use, criminal involvement, and employment), the proportion of individuals showing improvement from baseline to discharge and follow-up (baseline to discharge, baseline to 6 months, baseline to 12 months) will be calculated and aggregated at the program level (e.g., discretionary services). If deemed necessary for CSAT specific issues, the data will be examined at the individual activity level. Occasionally, the results will be examined for subpopulations of interest within individual activities (e.g., by age or by gender).

A17. Display of Expiration Date

The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.

A18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

11