Ss-hiv 9.20.11- B

Cross-Site Evaluation of the Minority Substance Abuse/HIV Prevention Program

OMB Supporting Statement

COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B1. Respondent Universe and Sampling Methods

The plan is to sample the whole universe. Table 3 displays the aggregate baseline sample sizes for program participant groups. As noted, the expected response rates at exit are 62% and at 37% at the 3-6 month follow-up, based on the HIV Cohort 6 results.

B2. Information Collection Procedures

Common Measures - Youth and Adult Questionnaires (Completed by Program Clients)

The study uses a common protocol for collecting program and participant/client level data and submitting it to CSAP via an online, web-based data entry system (PMRTS). The DITIC Technical Assistance Center also has been established to assist grantees with online data entry. These data provide the basis for categorizing programs on characteristics of program design and implementation (e.g., one-on-one, group format). These are described below:

Youth and Adult Outcome Questionnaires. Two common questionnaires will be administered to program participants/clients. Each questionnaire has three versions, as follows:

• Youth Questionnaire: For persons aged between 12 and 17 - Baseline, Exit, and Follow-up for participants in interventions lasting 30 days or longer,baseline and exit for participants in interventions lasting between 2 and 29 days and at exit only for participants in single session interventions.

• Adult Questionnaire: For persons aged 18 and older - Baseline, Exit, and Follow-up for individuals participating in 30 day or longer interventions, baseline and exit for participants in interventions lasting between 2 and 29 days and at exit only for participants in single session interventions.

For all common measures, administration guides have been prepared to assist program sites with implementation.

The major constructs for the youth outcome questionnaire include demographics, 30-day substance use (ATOD), age of first use, disapproval of ATOD use, perception of risk, ATOD use, family cohesion, perception of peer behavior, sexual behavior, school connectedness, knowledge of HIV, and health care (See Attachment 1 for a copy of the Youth Baseline Questionnaire and the Administration Guide.)

The major constructs for the adult questionnaire include demographics, employment, 30-day substance use (ATOD), age of first use, disapproval of ATOD use, perception of risk, ATOD use, age of first use, family cohesion, sexual behavior, knowledge of HIV, health care. See Attachment 2 for a copy of the Adult Baseline Questionnaire and the Administration Guide.)

The GPRA and NOMs measures have already been approved by OMB (OMB No. 0930-0230), and the remaining HIV-related questions have been approved under OMB No.: 0930-0298. The youth questionnaire contains 125 questions, of which 28 relate to HIV/AIDS and the adult questionnaire contains 118 items, 47 of which relate to HIV/AIDS. The planned analysis will assess the degree to which program strategies reduce SA/HIV/AIDS and increase SA, HIV/AIDS protective factors among at-risk populations. Two new questions have been added to address SAMHSA’s need to collect information on binge drinking behavior, not covered under any prior OMB package.

These questions are:

During the past 30 days, on how many days did you have 4 or more drinks on the same occasion? [By 'occasion,' we mean at the same time or within a couple of hours of each other]

and

During the past 30 days, on how many days did you have 5 or more drinks on the same occasion? [By 'occasion,' we mean at the same time or within a couple of hours of each other].

The information obtained from these questionnaires will generate data to determine the effectiveness of the program in reducing SA, injection drug use, and high-risk sexual behaviors as well as increasing participant knowledge. Pilot tests have been conducted to assist in controlling respondent burden and ensuring the user-relevance of questions.

Data Collection

Time Points (Efficacy) for Data Administration: The common design for participants in interventions lasting 30 days or longer includes assessments at baseline, program exit, and three to six months post-exit (follow-up). The common questionnaires will be administered to all 30-day intervention (program participants) youth and adults at baseline (first data collection point), program exit (second data collection point), and follow-up (third data collection point).For participants in interventions lasting between 2 and 29 days questionnaires will be administered at baseline and exit. For single session interventions an exit only questionnaire will be administered.

• Baseline Questionnaires: This questionnaire (first data collection point), should be administered within 30 days of intake or before core program services begin.

• Exit Questionnaires: This questionnaire (second data collection point), should be administered again within 10 days post program exit or after core program services have ceased.

• Follow-up Questionnaires: This questionnaire (third data collection point), should be administered within 30 days of the planned follow-up (post-exit). Most study sites have a planned follow-up administration for three to six months after program exit.

The common instrument assessment schedule is depicted in Table 5.

Table 5. Common Instrumentation Assessment Schedule

Intervention Duration	Instrument	Baseline	Exit	Follow-up
Single Session Intervention (1day or less)	Youth and Adult Questionnaire		X
Multiple Session Brief Intervention (Between 2 and 29 days)	Youth and Adult Questionnaire	X	X
Multiple Session Long Intervention (30 days or more)	Youth and Adult Questionnaire	X	X	X

This time series analysis is depicted graphically below:

01 X 02 03

Where: 01= Baseline - First Data Collection Point

X = Program Prevention Intervention Activities

02 = Exit - Second Data Collection Point

03 = Follow-Up – Third Data Collection Point

Sample size, respondent burden, and intrusiveness have been minimized to be consistent with cross-site analytical objectives. Procedures are employed to safeguard the privacy and security of participant responses. Every effort has been made to coordinate cross-site data collection with local data collection efforts to minimize respondent burden. Pilot study results indicate that the questionnaires require an administration time of approximately 50 minutes.

Data Management

Storage System: Over the life of this initiative, each grantee will be collecting information that must be documented and organized. Each local evaluation team or the person responsible for data management will be required to store:

• Individual questionnaires (Youth and Adult and Individual and Group Dosage) until they are entered or uploaded into PMRTS

• Consent forms

• Tracking forms for each study participant.

Before initiating data collection, each site is responsible for setting up a filing and storage system that will accommodate these needs.

Web-Based Data Entry Upload System: Prevention Management Reporting and Training System (PMRTS), an online data entry system which provides prevention information, data collection tools, documents, data entry functions, and access to reporting statistics and tracking. All of the HIV instruments can be found in the “Tools” section of this website. Common questionnaires are available in both Microsoft Word and PDF format for individual grant sites to download and make copies for administration to clients or participants. Site evaluators or data collectors are expected to enter client or participant responses to questionnaires through the PMRTS website. Sites will also be able to upload response databases through PMRTS that use the appropriate variable/value numbering (questionnaire codebooks are also available on the “Tools” section of PMRTS website). Dosage data will also be entered online via the PMRTS website by the project director or evaluator of the grant site.

Once data have been entered into PMRTS, the DACCC will clean the data and each grant site will have access to their downloadable, clean, electronic data files. Once data are entered by the grant site and cleaned by the DACCC, data will be available for download by the grant site for use in local data analysis and reporting. Sites will be able to enter the data online, or upload if necessary, on a continual basis. The DACCC will abstract data bi­annually for cleaning, analysis, and reporting purposes; however the data will remain accessible for local evaluations.

CSAP has established a technical assistance center to assist program sites. Any questions regarding the common instruments or submission to the DACCC should be addressed to DITIC Support via telephone, 1-888-DITIC-4-U (1-888-348-4248), or via email, [email protected].

B3. Methods to Maximize Response Rates

Based on the HIV Cohort 6 results, the expected response rate at exit is 62% and 37% at three to six month follow-up. As noted above, each grantee has developed a package of incentives to maximize their response rates. These incentives have been selected on the basis of each grantee’s past experience with their client population. Typically, prevention programs have lower retention rates than treatment programs, so these estimates fall below the OMB expected norm of 80%. In order to achieve these retention rates, it is also necessary to provide incentives.

CSAP's program officers will monitor monthly participation numbers at baseline and exit through the online PMRTS system. The PMRTS system is the online data collection tool that grantees use to upload their qualitative and quantitative data. This system will also be used as a communication tool for grantees to identify to the program officer any potential barriers to retention of participants. Furthermore, CSAP provide annual training and at hoc training to programs dealing with the retention of participants.

B4. Tests of Procedures

CSAP and its contractor have reviewed the adult and youth questionnaires, clarifying terminology and language, and rewriting or eliminating unclear or unnecessary questions. Questionnaires were then pilot-tested on a small sample (less than 10 individuals) to ensure that the multi-site assessment requirements and procedures were consistent with activities conducted at education sites. Pilot-testing was designed to collect information on the overall evaluation design and to draft feedback forms.

All study sites were encouraged to participate in pilot testing because of the diversity in target populations and program settings. (Note: It was only after local sites had completed their pilot testing that project staff learned that not more than nine persons could be included in testing without OMB approval). Pilot testing included administration of draft instruments and solicitation of comments on the instrument from respondents.

The purpose of the pilot testing was to:

• Identify instructions and/or questions within the instrument that were unclear or confusing

• Obtain suggestions for improving questions or instructions

• Identify data collection procedures that were unclear or confusing

• Measure respondent burden based on the average time to complete instruments

• Identify ways to minimize respondent burden and improve accuracy regarding instrument completion.

Comments on the draft feedback forms included collecting information on the likelihood of obtaining specific responses, overall instrument layout, item flow, and administration.

B5. Statistical Consultants

Virginia Mulkern, Ph.D.

Senior Vice President

Human Services Research Institute

2336 Massachusetts Avenue

Cambridge, MA 02140

Nilufer Isvan, Ph.D.

Senior Research Fellow

Human Services Research Institute

2336 Massachusetts Avenue

Cambridge, MA 02140

P. Allison Minugh, Ph.D.
President/CEO
Datacorp, Inc.
Two Richmond Square, Suite 100A
Providence, Rhode Island 02906

Robynn Battle, MPH, Ed.D.

Associate Research Scientist

1995 University Avenue, Suite 450

Berkeley, CA 94704

Faye Belgrave, Ph.D.

Professor

Department of Psychology

820 West Franklin St.

Virginia Commonwealth University

Richmond, VA 23284

Richard Cervantes, Ph.D.

President and CEO, Behavioral Health Assessments, Inc.

Behavioral Assessment, Inc.

291 South La Cienega Blvd. Suite 308

Beverly Hills, CA 90211

Jeffrey Guidry, Ph.D.

Associate Professor

Texas A&M University, Read 159 Bldg.

Department of Health & Kinesiology

College Station, TX 77843

Davis Ja, Ph.D.

President

Davis Ja and Associates

362 Victoria St.

San Francisco, CA 94132

Velma Kameoka, Ph.D.

Faculty, Department of Psychology and Director, Social Science Research Institute,

University of Hawaii at Manoa

2424 Maile Way

Saunders Hall 704

Honolulu, HI 96822

Sally Stevens, Ph.D.

Executive Director of the Southwest Institute for Research on Women (SIROW)

Department of Women’s Studies

College of Social and Behavioral Sciences

University of Arizona

925 N. Tyndall

Tucson, AZ 85721

Tiffany Townsend, Ph.D.

Assistant Professor of Psychiatry

Georgetown University School of Medicine

3800 Reservoir Rd., NW

612 Kober-Cogan Hall

Washington, DC 20007

Rachael Gerber, MPH

DACCC Research Analyst

Human Services Research Institute

2336 Massachusetts Avenue

Cambridge, MA 02140

Katie Howard, MPH

DACCC Research Analyst

Human Services Research Institute

2336 Massachusetts Avenue

Cambridge, MA 02140

Katie McInerney, B.S.

DACCC Research Assistant

Human Services Research Institute

2336 Massachusetts Avenue

Cambridge, MA 02140

Nelly Oliver, Ph.D.

DACCC Senior Researcher

Human Services Research Institute

2336 Massachusetts Avenue

Cambridge, MA 02140

ATTACHMENTS

1. Youth Questionnaire and Youth Questionnaire Administration Guide

2. Adult Questionnaire and Adult Questionnaire Administration Guide

3. Overarching Administration Guide

Minority HIV/AIDS Initiative Page 8