State Targeted Response to the Opioid
Crisis (Opioid STR) Evaluation
The respondent universe for the proposed state-level evaluation includes all State Targeted Response to the Opioid Crisis (Opioid STR) grantees (n = 57). Up to 20 communities/programs that receive STR funding will be selected for site visits and participation in client-level data collection. These 20 communities/programs will be selected based on factors such as HHS Regional representation; urban/rural status, program model/type, program maturity, and ability to collect and report required data. These 20 programs are not intended to represent or generalize to all STR-funded programs.
The respondents to each data collection effort include the following:
State Surveys: No sampling will be used for the State Surveys. State Surveys will be administered to all Opioid STR State Project Directors/Program Managers (n = 57). The State Survey will be administered three (3) times.
State Interviews: No sampling will be used for the State Director Interviews. Interviews will be conducted with all Opioid STR State Project Directors/Program Managers (n = 57). The interviews will be conducted two (2) times.
Community/Program Surveys: The Community/Program Survey will be administered to the program director/manager of up to 20 communities/programs. The Community/Program Survey will be administered three (3) times.
Community/Program Interview Protocols: Two rounds of site visits interviews will be conducted with the 20 communities/programs that participate in the community/program-level evaluation. Up to six (6) staff members in each community/program will participate in interviews during each site visit, including the Project Director/Project Manager, data manager, and approximately four (4) clinical staff. These interviews will be conducted two (2) times.
CSAT
GPRA Client Outcome Measure:
Client-level data will be collected from individuals receiving
Opioid STR-funded services from up to 20 communities/programs. The
CSTA GPRA measure is administered when a client enters STR-funded
services and every 6 months thereafter until discharged from
STR-funded services. We estimate that client-level data will be
collected from approximately 2000 individuals (1000 per year) at
intake/baseline (100 clients across 20 communities/programs), 1600
individuals will complete a follow-up assessment, and 1040 will
complete a discharge assessment. .
The evaluation includes five complementary data collection activities that will answer the evaluation questions. A brief description of each of the activities follows:
State Surveys: Opioid STR State Project Directors/Manager from all STR-funded states/territories will be asked to complete a survey at three (3) time points. The evaluation team will reach out to Project Directors and invite them to complete a web-based survey, with a paper option available, if requested. Survey data will be collected and maintained on a secure web-based system and hard copy paper instruments will be data entered by professional data entry staff at the evaluation contractor’s survey operation center. All data entered from hard copy instruments will be 100 percent double entered by separate data entry staff to verify accuracy and prevent data entry errors. Evaluation staff will work with Opioid STR program staff to encourage staff to complete these surveys in a timely manner.
State Telephone Interviews: The state official designated as the Opioid STR Project Director/Program Manager will be asked to participate in two telephone interviews at two (2) time points. The evaluation team will reach out to the designated state official to conduct each interview, audio recording the interview if permission is obtained. Interviews will be transcribed and analyzed using qualitative software such as NVivo.
Community/Program Surveys: Project Directors from up to 20 participating communities/programs will be asked to complete the Community/Program survey at three (3) time points. The evaluation team will reach out to participating Project Directors to invite them to complete a web-based survey, with a paper option available if requested. Survey data will be collected and maintained on a secure web-based system and hard copy paper instruments will be data entered by professional data entry staff at the evaluation contractor’s survey operation center. All data entered from hard copy instruments will be 100 percent double entered by separate data entry staff to verify accuracy and prevent data entry errors. Evaluation staff will work with Opioid STR program staff to encourage completion of these surveys in a timely manner.
Community/Program Site Visit Interviews: In-person semi-structured interviews will be conducted with up to six (6) staff members (Project Directors and/or Program Managers, data manager, and up to four (4) clinical staff) during two (2) rounds of site visits to 20 participating communities/programs. Each site visit will be two (2) days in duration. Evaluation staff will reach out to these communities/programs to invite them to participate in this component of the evaluation and schedule the site visits. With permission, each interview will be audio recorded. Interviews will be transcribed and analyzed using qualitative software such as NVivo.
CSAT GPRA Client Outcome Measure: Information collection procedures will vary by community/program. Program staff will administer this measure to clients who receive STR-funded services. Program staff will administer this measure at intake/baseline and every 6 months thereafter until the client is discharged from STR-funded services. Program staff will also administer this measure upon discharge from STR-funded services. In instances where clients are no longer in direct contact with the program, program staff will attempt to locate the clients and conduct the follow-up interviews either in person or over the phone.
Programs will submit their data electronically through the web-based SPARS data portal maintained by SAMHSA. Data for each data collection time point (intake, each six-month follow-up, and discharge) will be linked using a unique encrypted client identifier that programs enter at intake. Participating programs will be instructed not to use personally identifiable information (i.e., Social Security Number, Medicaid or health plan identifiers, or initials) as the client identifier.
In the event that some communities/programs wish to collect data using paper and pencil methods, SAMHSA will provide downloadable paper versions of the data collection instruments to facilitate this process, but program/communities will be responsible for entering this data into the web-based SPARS data portal.
Required data collection points are:
BASELINE: For clients who have not previously received STR-funded services, or who are returning to STR-funded services following a discharge from the program, baseline data will be collected. For clients already enrolled in the program and receiving services funded by Opioid STR funds, the GPRA CSAT measure should be submitted by the program within 30 days of initiating data collection. The timing of any subsequent data collection point(s) will be anchored to the baseline/intake point the program indicates in this administrative record.
6 MONTHS FOLLOW-UP: Consistent with other SAMHSA grant programs, participating STR-funded programs will submit client-level GPRA CSAT data every six months while the client is receiving STR-funded services. Ongoing periodic status review is viewed as consistent with good clinical practice.
DISCHARGE: Programs will provide information on the type of discharge on all clients who are discharged. When the discharge is a planned event, the client will also be asked the questions on the client-level data collection instrument. The one exception to this requirement is when a client had responded to these same questions within the past 30 days as part of a reassessment.
Several steps will be taken to maximize response rates and reduce non-response bias for all data collection efforts. Efforts to maximize response rates are presented below.
Identifying Respondents. The Evaluation Team will work closely with SAMHSA and the Project Director of each community/program to identify appropriate staff members to interview.
Scheduling Interviews. The Evaluation Team will be flexible in scheduling telephone interviews and site visits, provide a copy of the interview schedule ahead of time, and respect the specified time limits. To make the best use of informants’ time, the Evaluation Team will review available documents and conduct web searches to collect publicly available information prior to the interview.
Client-level Data: Each community/program may have established its own client follow-up procedures. At the time of intake, information is typically obtained from clients to assist with locating them later. This includes information on current residence plus information on one or two other individuals who are likely to know where they are if they have re-located. In addition, some providers are adept at using other community resources to assist with locating clients. Clients are typically quite cooperative with provider staff because of the relationship established during treatment.
Historically, the collection of follow-up client-level has been a challenge for some SAMHSA grantees given the challenge of locating clients as far out as 6 months. Participating communities/programs will be expected to provide discharge and follow-up data for 80% of clients who receive STR-funded services. To help participating communities/programs meet these goals, several strategies will be employed. First, group training will be offered to communities/programs to help them about collecting and reporting client-level data. These group and individual trainings are conducted by follow-up experts. Secondly, communities/programs will also be provided with data status reports on how close they are to meeting their follow-up goals. A third strategy is the automatic, system generated notice of when provides follow-up interviews are due for each client/participant. A fourth strategy will be targeted technical assistance provided by a contractor.
Data collection instruments including the State and Community/Program Surveys and key informant interviews were reviewed by the internal technical advisors who provided feedback on measurement quality, potential burden, and ease of administration. The client-level data collection instrument was developed by SAMHSA’s Center for Substance Abuse Treatment (CSAT) and is based on questions that that have a long history of use in the substance abuse field and have already been tested for validity and reliability, (i.e., ASI).
Name |
Affiliation |
Patrick High, Dr.PH
|
Epidemiologist, SAMHSA |
Crystal Barksdale, Ph.D., MPH |
Statistician, SAMHSA |
Attachment 1: Opioid STR State Director Baseline Survey
Attachment 2: Opioid STR State Director Time 2 Survey
Attachment 3: Opioid STR State Director Time 3 Survey
Attachment 4: Opioid STR State Director Time 1 Interview Protocol
Attachment 5: Opioid STR State Director Time 2 Interview Protocol
Attachment 6: Opioid STR Community/Program Director Baseline Interview Protocol Attachment 7: Opioid STR Community/Program Clinical Staff Baseline Interview Protocol
Attachment 8: Opioid STR Community/Program Data Manager Baseline Interview Protocol
Attachment 9: Opioid STR Community/Program Director Time 2 Interview Protocol Attachment 10: Opioid STR Community/Program Clinical Staff Time 2 Interview Protocol
Attachment 11: Opioid STR Community/Program Data Manager Time 2 Interview Protocol
Attachment 12: Opioid STR Community/Program Director Baseline Survey
Attachment 13: Opioid STR Community/Program Director Time 2 Survey
Attachment 14: Opioid STR Community/Program Director Time 3 Survey
Attachment 15: CSAT GPRA Client Outcome Measures Tool
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Sonji Hogan |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |