Justification

The Modular Survey on Consumer Perceptions of Care (referred to as “the Modular Survey”) received OMB approval on June 25, 2009 (OMB No. 0930-0197; Expiration Date 01/31/2011). The currently approved version of the Modular Survey contains 31 questions (i.e., 21 survey questions and 10 questions that ask for background information).

In this clearance request, the Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Treatment (SAMHSA/CSAT) is requesting to pilot test 28 newly developed opioid treatment program (OTP)-specific perception of care questions along with the currently approved Modular Survey questions. The Modular Survey with OTP-specific questions contains these 31 questions plus an additional 28 OTP-specific perception of care questions (see Attachment 1).

A. Product/Activity to be Assessed

SAMHSA/CSAT recognizes the importance of the perceptions of consumers concerning the quality of care and insight on improvement and outcomes that are associated with care received. As such, the Modular Survey with OTP-specific questions will assess the psychometric properties of a group of questions concerning the perceptions of individuals (age 18 and older) who receive medication-assisted treatment and related services for opioid dependence/addiction in an OTP setting. Domains measured in the survey include Stigma, Integration, Wellness, Access, Quality, Perceived Outcomes, Relationship to Program, Commitment to Change, and Social Connectedness.

2. Statement of Objectives

Data from the Modular Survey with OTP-specific questions will be analyzed to assess the validity and psychometric properties of each question. The objective is to identify a small set of public domain, non-proprietary questions that address patients’ perceptions of care related to treatment received at their OTPs. Specifically, it is expected that the pool of 28 potential OTP-specific questions will be reduced to a final set of approximately 10 – 15 questions based on psychometric testing.

C. Methodology

Data Collection

Appropriate treatment/service systems and settings that might be willing to pilot test the Modular Survey with OTP-specific questions (e.g., SAMHSA/CSAT grantee sites, the American Association for the Treatment of Opioid Dependence [AATOD], State and county substance abuse programs or OTPs) will be contacted to seek assistance piloting the Modular Survey with OTP-specific questions. Up to 18 programs that represent a mix of financial status (i.e., For profit or nonprofit) and sizes (i.e., 0 – 100, 101 – 299, 300+) will be recruited for participation. If necessary, survey questions will be read to clients with low literacy.

Data will be collected via paper-and-pencil rather than electronically to allow for thorough quality checks of the paper-and-pencil data (e.g., for completeness, errors in completing the survey) by Contractor staff before the data are analyzed.

Respondents

Respondents for the Modular Survey with OTP-specific questions pilot test will be adults (age 18 years and older) who receive treatment, including medication-assisted treatment, at participating OTPs. Participation for all will be voluntary.

Sample Size and Rationale

The sample is targeted to individuals (age 18 and older) with opioid use diagnoses (i.e., opioid abuse or dependence) receiving treatment/services at participating OTPs. Convenience sampling procedures will be used to select individuals receiving opioid use treatment/services with the targeted sample characteristics. In total, up to 18 treatment sites from up to 18 States may be selected to have patients complete the survey. From those treatment sites, up to 1,200 adults will be asked to participate in the pilot test of the Modular Survey with OTP-specific questions.

Timeline of Data Collection

The data collection for both pilot tests will be conducted between January and March 2010 over a four week-period for each data collection site. Respondents will be requested to complete the survey only one time during the four-week data collection period. The timeframe was chosen to allow the participating OTPs a range of time in which to conduct the survey from willing patients.

Response Rate

SAMHSA/CSAT anticipates receiving responses from approximately 50-60% of patients from OTPs participating in data collection efforts.

Confidentiality

The Modular Survey with OTP-specific questions does not include any identifying information and all responses will be kept private. Patients will be instructed to place the completed paper-and-pencil survey in an envelope, seal it and hand it to the survey administrator, a clerical staff member at the treatment organization. At the end of each day, the survey administrator, will place the envelopes into a larger envelope and store in a locked filing cabinet. The surveys will be mailed back to the contractor and will only be accessible to contractor staff and data analysis staff. In addition, results of the survey will only be presented in aggregate form so that individual responses cannot be identified.

D. Estimates of Annualized Hour Burden

This data collection effort will not involve start-up or operational/maintenance costs to respondents. The maximum number of responses has been estimated for each potential contact with the respondents.

Estimates of Annualized Hour Burden and Annualized Cost Burden to Respondents

Instrument	Respondents	Number of Respondents	Responses per Respondent	Average Burden Hours Per Response	Total Burden Hours	Hourly Wage*	Total ** Respondent Cost
Modular Survey with OTP-specific questions	Individuals receiving services in OTPs	1,200 Adults	1	.25	300	$7.25	$2,175

*The mean hourly wage is estimated at $7.25, the Federal minimum wage as of July 24, 2009.

** Total Respondent Cost is calculated as hourly wage × time spent on survey × number of respondents.

E. Methods Used to Develop and Test the Survey

Development of Currently Approved Modular Survey Questions

The Modular Survey was developed by a group of nationally recognized experts in the areas of substance abuse and mental health. The experts were enlisted to contribute their knowledge of the evidence base to inform a national consensus process to select questions appropriate for inclusion in the Modular Survey. The survey was refined via the consensus process several times for both content and clarity in wording. In addition, a focus group was conducted with a group of six substance abuse treatment and co-occurring substance abuse and mental health treatment providers to provide feedback on content and clarity in wording.

Following administration of the previously approved version of the Modular Survey, analyses were conducted by George Washington University, an IDIQ subcontractor, to identify questions that met acceptable psychometric standards and would have applicability across diverse behavioral healthcare (mental health and substance abuse) systems and settings. The questions that were identified are the 21 questions included in the present version of the Modular Survey.

Development of OTP-specific Questions

More recently, a group of experts in the treatment of opioid abuse/dependence was convened including representatives of State Opioid Treatment Authorities, patient advocacy, accreditation organizations, opioid treatment program staff experts, and CSAT staff. The goals of this meeting were to:

• Discuss challenges related to measuring patient perceptions of care in opioid treatment programs.

• Brainstorm to develop potential questions for measuring patient perception of care in opioid treatment programs.

Meeting participants generated 28 OTP-specific questions that addressed areas including stigma, access, quality/appropriateness, social connectedness, and wellness. Subsequently, Kevin O’Grady, Ph.D., an expert measurement consultant provided his opinions regarding finalization of the OTP-specific questions for pilot testing with a focus on language, content, and reading level. The questions were refined and finalized in consultation with the Dr. O’Grady and the expert group.

Statistical Consultants and Federal Project Officers

Responsibility for Data Collection

JBS International, Inc. will receive the survey databases from participating treatment programs via email at the end of the six-week data collection period. The representative of the Contractor responsible for overseeing the planned data collection is:

Amanda Gmyrek, Ph.D.

Senior Researcher

JBS International, Inc.

5515 Security Lane, Suite 800

North Bethesda, MD 20852

240-645-4848

[email protected]

The representative of the subcontractor responsible for the question development feedback and psychometric analyses for the OTP-specific questions is:

Kevin, O’Grady Ph.D.
QuantAid, LLC

14190 Triadelphia Road
Glenelg, MD 21737

410-489-6045
[email protected]

Agency Responsibility

Within the agency, the following individual will have oversight responsibility for all contract activities, including the data analysis:

Fran Cotter, M.A., M.P.H.

Task Order Officer

SAMHSA/CSAT

1 Choke Cherry Road, Suite 5-1085

Rockville, MD  20857

240-276-1569

[email protected]

5