0208-ss-b 11.14.13

GOVERNMENT PERFORMANCE AND RESULTS ACT

CLIENT/PARTICIPANT OUTCOME MEASURES

SUPPORTING STATEMENT

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B1. Respondent Universe and Sampling Methods

All SAMHSA grantees are required to collect and report certain data so that the Agency can meet its obligations under their funding agreements. 

In SAMHSA programs, which are often client level/participant interventions that are dramatically different from one group to another may have insufficient population receiving a specific intervention to justify a sample. Populations from each group may not be similar and would not be appropriate to infer general findings about the successes or failures of a program because of the uniqueness of each group. Within populations, sample sizes may be too small to properly sample, leading to large sample variance and errors in findings about the programs. In these cases, where programs differ from group to group, it is important to gather data sufficient to draw statistically accurate conclusions about how the programs are performing and about what characteristics of the program may matter to the success of the program.

In addition to the GPRA, data collected by grantees will be used to demonstrate how SAMHSA’s grant programs are reducing disparities in access, service use, and outcomes nationwide.  To accomplish this, SAMHSA expects grantees to utilize their data to (1) identifying subpopulations (i.e., racial, ethnic, sexual/gender minority groups) vulnerable to health disparities and (2) implement strategies to decrease the differences in access, service use, and outcomes among those subpopulations.  There will be subpopulations where sample size is too small to properly sample and where a census would be necessary obtain relevant and reliable outcome data.

B2. Information Collection Procedures

Information collection procedures will vary by type of program. The client outcome measures for most providers will be extracted from previously established databases. Intake/baseline information is obtained by intake workers and/or counselors. For clients still in treatment 6 months later, the information will be obtained in the same way. In instances where clients are no longer in direct contact with the service provider, staff from the program will locate the clients and conduct the follow-up interviews. These interviews are to be conducted face-to-face.

Some programs collect their client information using paper and pencil methods. This project will not interfere with ongoing program operations. Programs will submit their data electronically via a web-based data entry process or upload process. The data for those clients with baseline, discharge, and follow-up data are matched using a unique encrypted client identifier.

B3. Methods to Maximize Response Rates

Each grantee will have established its own client follow-up procedures as part of the original protocol. At the time of intake, information is typically obtained from clients to assist with locating them later. This includes information on current residents plus information on one or two other individuals who are likely to know where they are if they have re-located. In addition, some providers are adept at using other community resources to assist with locating clients. Clients are typically quite cooperative with provider staff because of the relationship established during treatment. Since all participating grant programs propose a census at initial intake, considerable options also exist for non-respondent analysis and associated adjustments to the data such as weighting.

Follow-up has been a challenge to some grantees given the remote locations that they serve and the challenge of locating clients as far out as 6 months. For grantees that have not been aware of the strategies they can employ to begin the follow-up process at intake, how to maintain contact with clients, and the importance of good locator forms, several strategies have been implemented to assist the grantees with followup. First, follow-up training is offered which assists grantees in learning about and conducting follow-up at their sites. This program is offered to all grantees and after the grantees are trained through the grantee orientation process, monthly follow-up trainings are offered for those that need additional training or for new project staff. Individual grantee technical assistance is also available for sites that need additional follow-up instruction. These group and individual trainings are conducted by follow-up experts. Each grantee receives a follow-up tracking manual at these trainings that may be used as a future reference. A second strategy provides the grantees with data status reports on how close they are to meeting their follow-up goals. These reports are available from the web-based system to the grantees and Government Project Officers for the grants they are responsible. A third strategy is the automatic, system generated notice of when follow-up interviews are due for each client/participant. A fourth strategy provides technical assistance at national meetings. Experts, including grantees, have been identified and asked to make presentations at national grantee meetings on how to conduct follow-up. These sessions are well attended by grantees.

B4. Test of Procedures

Most of the data elements in the data sets have been taken from established data collection instruments that have a long history of use in the substance abuse field and have already been tested for validity and reliability, (i.e., ASI).

Feedback from the grantees also indicates that they routinely collect the same information requested of this data collection tool and some have integrated this tool into other tools that they routinely use to gather information. Some grantees report that they collect information in greater detail, (i.e., more response alternatives), but these are collapsed into standard categories.

B5. Statistical Consultants

Sarah Ndiangui

Government Project Officer

Center for Substance Abuse Treatment

SAMHSA

1 Choke Cherry Lane

Rockville, MD 20850

(240) 276-2918.

Scott Novak, Ph.D.

SAIS Associate Project Director

RTI International

3040 Cornwallis Road

P.O. Box 12194

Research Triangle Park, NC 27709-2194

(919) 541-3129.

ATTACHMENTS

Attachment 1: CSAT GPRA Client Outcome Measures for Discretionary Programs and Instructions

3