0208-ss-9-6-11-b

SUPPORTING STATEMENT FOR THE

GOVERNMENT PERFORMANCE AND RESULTS ACT

CLIENT/PARTICIPANT OUTCOME MEASURES

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B1. Respondent Universe and Sampling Methods

The Center for Substance Abuse Treatment will utilize the following strategies to facilitate data collection. The CSAT estimated universe of individual respondents is 85,233 individuals per year across all the discretionary grant programs included in the request. For all the programs included, the starting point for use of the measures is a census of all clients at intake (or initial contact). Beyond the initial census, there will be considerable variation in the number of respondents receiving the core measures since there is variability in the measurement strategy used for each program. The options include no follow-up or sampling follow-up for some discretionary services programs. Grantees will be directed to achieve a minimum of an 80 percent response rate for all clients targeted for follow-up efforts. This guidance will apply to the response rate as defined against the intake or baseline census. The sampling strategy for the GPRA measures will be specific to CSAT and the capabilities of the programs under review.

CSAT’s strategy for conducting follow-up is presented here. For CSAT’s discretionary services grantee portfolio, the strategy for baseline, discharge, and follow-up will either be a complete census or a sample, depending on the number of clients the program plans to serve, the amount of variation expected, and the local project activities. The guidance to be given to grantees in this area will be based on project minimum cell size for each analysis of interest. SAMHSA will provide guidance for deciding whether to census or sample for follow-up as part of its instructions to grantees. A sampling rate will be recommended based on the anticipated program population. For example, if it is determined in further analysis of GPRA performance plans that a minimum cell size of 40 is needed for appropriate sensitivity to examine performance on a given measure, a sampling rate will be recommended so that 1 of every x clients/participants will be followed-up to achieve a sample size of at least 40 for each appropriate cell. Where it is not clear what the number of participants will be, an alternate strategy of following up the first 40 in each cell of interest would be recommended, with a sampling strategy initiated once the minimum was reached. Later adjustments could be made if a larger number of participants were actually served. This approach will reduce the actual burden because the total number of clients followed-up will be less than the total number collected at baseline.

B2. Information Collection Procedures

Information collection procedures will vary by type of program. The client outcome measures for most providers will be extracted from previously established databases. Intake/baseline information is obtained by intake workers and/or counselors. For clients still in treatment 6 months later, the information will be obtained in the same way. In instances where clients are no longer in direct contact with the service provider, staff from the program will locate the clients and conduct the follow-up interviews. These interviews are to be conducted face-to-face but can be conducted over the phone in extenuating circumstances and with permission of the Government Project Officer.

Some programs collect their client information using paper and pencil methods. This project will not interfere with ongoing program operations. Programs will submit their data electronically via a web-based data entry process or upload process. The data for those clients with baseline, discharge, and follow-up data are matched using a unique encrypted client identifier.

B3. Methods to Maximize Response Rates

Each grantee will have established its own client follow-up procedures as part of the original protocol. At the time of intake, information is typically obtained from clients to assist with locating them later. This includes information on current residents plus information on one or two other individuals who are likely to know where they are if they have re-located. In addition, some providers are adept at using other community resources to assist with locating clients. Clients are typically quite cooperative with provider staff because of the relationship established during treatment. Since all participating grant programs propose a census at initial intake, considerable options also exist for non-respondent analysis and associated adjustments to the data such as weighting.

Follow-up has been a challenge to some grantees given the remote locations that they serve and the challenge of locating clients as far out as 6 months. For grantees that have not been aware of the strategies they can employ to begin the follow-up process at intake, how to maintain contact with clients, and the importance of good locator forms, several strategies have been implemented to assist the grantees with followup. First, follow-up training is offered which assists grantees in learning about and conducting follow-up at their sites. This program is offered to all grantees and after the grantees are trained through the grantee orientation process, monthly follow-up trainings are offered for those that need additional training or for new project staff. Individual grantee technical assistance is also available for sites that need additional follow-up instruction. These group and individual trainings are conducted by follow-up experts. Each grantee receives a follow-up tracking manual at these trainings that may be used as a future reference. A second strategy provides the grantees with data status reports on how close they are to meeting their follow-up goals. These reports are available from the web-based system to the grantees and Government Project Officers for the grants they are responsible. A third strategy is the automatic, system generated notice of when follow-up interviews are due for each client/participant. A fourth strategy provides technical assistance at national meetings. Experts, including grantees, have been identified and asked to make presentations at national grantee meetings on how to conduct follow-up. These sessions are well attended by grantees. It is anticipated that these strategies will continue to improve the follow-up rates and it is continually stressed to the grantees that a minimum 80 percent follow-up rate is expected.

B4. Test of Procedures

Most of the data elements in the data sets have been taken from established data collection instruments that have a long history of use in the substance abuse field and have already been tested for validity and reliability, (i.e., ASI).

Feedback from the grantees also indicates that they routinely collect the same information requested of this data collection tool and some have integrated this tool into other tools that they routinely use to gather information. Some grantees report that they collect information in greater detail, (i.e., more response alternatives), but these are collapsed into standard categories.

B5. Statistical Consultants

Deepa Avula,

Government Project Officer

Center for Substance Abuse Treatment

SAMHSA

1 Choke Cherry Lane

Rockville, MD 20850

(240) 276-2961.

Scott Novak, Ph.D.

SAIS Associate Project Director

RTI International

3040 Cornwallis Road

P.O. Box 12194

Research Triangle Park, NC 27709-2194

(919) 541-3129.

ATTACHMENTS

Attachment 1: CSAT GPRA Client Outcome Measures for Discretionary Programs and Instructions (REVISED)

3