MAI Overarching Guide

Substance Abuse and Mental Health Services Administration

Center for Substance Abuse Prevention

National Minority SA/HIV Prevention Initiative

Participant-Level Instruments and Data Collection Guide

Overview

The National Minority Substance Abuse and HIV/AIDS Prevention Initiative (MAI) administered by the Center for Substance Abuse Prevention’s (CSAP) within The Substance Abuse and Mental Health Services Administration (SAMHSA) supports an array of activities to assist grantees in building a solid foundation for delivering and sustaining effective substance abuse and HIV prevention and related services. While grantees have substantial flexibility in designing their grant projects, all are required to base their project on the five steps of SAMHSA’s Strategic Prevention Framework (SPF). Grantees must also conduct on-going monitoring and evaluations of their projects to meet CSAP’s reporting requirements.

This guide provides a detailed framework for planning and preparing for the participant-level data collection process. The actual arrangements must be worked out by each local program to maintain consistency within the coding and completion of the instruments. Major issues and solutions concerning the administration of the instruments (e.g., which sections of the questionnaire to administer based on intervention duration, targeted group size, the use of translated versions) will be determined by your local data collection team in consultation with your assigned CSAP Project Officer.

General Administration Guidelines

Develop Storage System

Over the life of this initiative, each grantee will be collecting information that must be documented and organized. Each local evaluation team or the person responsible for data management will be required to store:

• Individual questionnaires and forms (Youth & Adult Questionnaires and Individual & Group Dosage Forms) until they are entered online or uploaded as batch databases

• Consent forms

• Tracking forms for each participant (intervention and control/comparison) in the study

Before initiating data collection, set up a filing and storage system that will accommodate these needs. Any form containing a participant’s name and unique identification number should be kept in a locked location. Completed questionnaires and dosage forms should not be stored in the same location as forms which contain the unique participant identification number and the respondent’s name.

Make Arrangements for Data Collection

For administration of the questionnaire, develop a clear understanding with program staff concerning the time and place of administration well ahead of time. The room where administration takes place should have adequate seating to accommodate the respondents, adequate lighting and ventilation, and seating should be spaced to ensure privacy. The administrative portion of the questionnaires should be completed by the program staff responsible for coordinating data collection before questionnaires are distributed to respondents.

A system should be implemented for the collection of dosage data. The program staff responsible for coordinating data collection should determine which individuals are responsible for the collection of dosage data during a session or encounter.

Familiarity with the Instruments

It is critical that the program staff responsible for data collection be familiar with questionnaire and dosage form format and content prior to administering the instrument. Individuals proctoring the questionnaire should read it carefully until an understanding of the wording of all items is established. The questionnaire item structure (outlined in the corresponding administration guide) should also be reviewed before the survey is administered. The individual proctoring the questionnaire should be prepared to answer questions from respondents regarding the questionnaire items. For dosage forms, the program staff responsible for data collection should ensure that all staff who fill out dosage forms understand each service category and the appropriate duration code.

Possible Issues during the Survey Process

The program staff responsible for data collection should train those administering the questionnaires on how to deal with any issues of emotional distress that may arise as a result. Although this is not expected, some questionnaire items ask about personal issues such as partner abuse or accepting money or other goods in exchange for sex. Participants should be instructed that their answers are private and will be very helpful to program evaluators, but that completing the questionnaire is optional. Services or referrals to appropriate services should be provided in the event of emotional distress.

Record Management

Assign a unique identification number to each respondent, whether they are in the intervention or control/comparison group. A 5-digit unique identification number (ID) is required on each form in order to track the responses of an individual and the corresponding dosage data over time. On the questionnaire, each participant’s name and unique 5‑digit ID should be written on the face (cover) sheet of the questionnaire and the same 5 digit ID should be entered on page 2 of the instrument. This unique ID should also be used on every dosage form with which the participant is associated. All record management information should be entered on the questionnaires by the program staff responsible for data collection prior to distributing the questionnaire to the respondent. For dosage forms, a system should be created by the grantee to designate what the program staff responsible for coordinating data collection fills out and what the individual recording the dosage data completes on the forms. Participant names must not be written on any other page but the face sheet on the questionnaire and should never appear on dosage forms. Record management information has the following components:

• Grant Identification Number: Each grantee will use its assigned grant identification number provided by CSAP.

• Study Design Group: Record whether the respondent is receiving an HIV funded intervention (pre-filled as “1” on dosage record forms) or is a control/comparison group participant. As discussed in more detail below, the use of a control/comparison groups is optional. However, even if your site is not using a control/comparison group and all of your participants are receiving services, it is important to complete this field (by entering a “1” for all of your participants) for data management purposes. A data record that has the Study Design Group left blank will not be included in the multisite analysis.

• Unique Participant Identification Number: Unique identification numbers should be assigned to each program participant and used for all data records associated with the participant. Numbers can range from 00001 to 99999. Programs with multiple service locations may want to consider assigning a range of individual identifiers to each location to allow for easy identification of a participant’s service location. For example, one location could be assigned numbers 1000 to 1999, numbers 2000 to 2999 to a second location, and so forth.

• Administration Date: The month (2-digit), day (2-digit), and year (4-digit) of the data collection should be entered.

• Interview Type (Questionnaires Only): Record whether this is a baseline, exit, or followup survey.

• Service Duration (Questionnaires Only): Record whether the total duration of services provided to the participant is Single Session (lasting one day or less), Multiple Session Brief (2-29 day duration), or Multiple Session Long (30 days or longer duration) (See Table 1 below). Service duration is the period between the first and last service encounters with the participant and may not be accurately known at baseline. The duration should be estimated at baseline based on the participant’s needs. The actual duration should be reported on the exit and follow-up surveys.

• Intervention Name(s) (Questionnaires Only): The name(s) of the intervention(s) used by the grantee to construct the sessions delivered to the participant. The intervention name is not the name of a specific service (i.e. HIV education or case management), but rather the name of the curriculum or evidence-based program used by the grantee to guide its intervention. Some examples of intervention names are: Guiding Good Choices, Healthy Workplace, Keepin’ It REAL, Life Skills Training,, Parenting Wisely, Positive Action, Many Men Many Voices, Voices/Voces, SISTA, RESPECT, and BASICS. Up to three intervention names may be entered for each participant.

Overview of CSAP Data Collection Instruments

Youth and Adult Questionnaires

1. Questionnaires

CSAP has created two instruments for HIV Grant participants: the Youth Questionnaire and the Adult Questionnaire. The questionnaires provide CSAP and grantees with a tool that can be used to compare participants across grant sites. These instruments should be filled out by program participants and are meant to document basic characteristics about the participants and/or changes in their behavior and attitudes relevant to substance abuse and HIV prevention.

Some of the items in the Youth and Adult Questionnaires are identical and have been shown to be appropriate for both youth and adults. However, the Youth Questionnaire was created to be cognitively appropriate for youth and contains construct measurements that are specific to this age group. The Adult Questionnaire contains items that are cognitively more advanced and asks about behaviors and beliefs that may not be relevant or appropriate for youth.

The Youth and Adult Questionnaire have been divided into three sections. These sections group items by common theme. The three sections are:

• Section One: Facts about You -- demographic items as well as information relevant to CSAP’s target population

• Section Two: Attitudes & Knowledge -- attitudes and knowledge about drug use, sexual behavior, HIV/AIDS, and other relevant topics

• Section Three: Behavior & Relationships – behavioral items about drug use, sexual behavior, HIV/AIDS, and other relevant topics

Sections will be administered to program participants based on the service duration the participant receives. CSAP has identified three mutually exclusive service duration categories: Single Session, Multiple Session Brief (2-29 day duration), and Multiple Session Long (30 days or longer duration) (Table 1). In this context, service duration refers to the entire period during which the participant received services from funded programs, that is, the period between the first and last service encounters. During this duration, an individual may participate in multiple interventions. The three sections in the Youth and Adult Questionnaire correspond to these service duration categories:

Table 1: Sections of the Questionnaire

Service Duration	Length	Definition	Sections of Survey To be Administered
Single Session	1 day or less	Total services last one day or less. Participant may receive multiple services during the session, but do not continue in a CSAP MAI grant funded activity for more than one day.	Section One: Facts about You 3 to 5 questions from Section Two
Multiple Session Brief	2-29 days	The participant should receive at least two MAI Grant funded sessions or service encounters. The period of time between the first session or encounter and the last session or encounter should be two to 29 days.	Section One: Facts about You Section Two: Attitudes & Knowledge
Multiple Session Long	30 days or more	The participant should receive at least two MAI Grant funded sessions or service encounters. The period of time between the first session/encounter and the last session/encounter should be 30 days or more.	Section One: Facts about You Section Two: Attitudes & Knowledge Section Three: Behavior & Relationships

The staff responsible for data collection, in consultation with the CSAP Project Officer, should determine how to identify what type of intervention(s) a participant will receive. At baseline, service duration should be estimated by counting the number of days the program plans to have with the participant during the course of total services. Only sessions or encounters that are funded by the MAI grant should be included in the count. Once the duration of services has been determined, the staff responsible for data collection should prepare the Youth or Adult Questionnaire for the participant. Sections of the questionnaire that are not applicable for a participant based on the duration of services he or she will receive should be removed and discarded prior to administration.

The same sections should be given at every time point the survey is administered. The appropriate sections of the questionnaire to administer should be determined before baseline (1^st data collection point). For example, if the participant’s services are expected to last between two and 29 days, the participant should receive Section One: Facts About You and Section Two: Attitudes & Knowledge at all the appropriate time points. Additional local evaluation instruments may be administered to participants in conjunction with, but not instead of, the CSAP questionnaires.

Occasionally, a participant who took the baseline survey, completed the intervention, and was given an exit survey, later comes back for further services. In such cases, the participant should be given the same Participant ID that was assigned during the first intervention but a new set of baseline and exit surveys appropriate for the expected duration of the new set of services he or she is receiving.

If the participant has taken a baseline survey but has not completed an intervention or an exit survey and returns for the same intervention as previously planned, the participant should:

• not receive a new baseline survey but should complete the appropriate set of surveys if a baseline survey was taken within six months of the participant’s return;

• receive a new baseline survey and all appropriate surveys if the original baseline survey was taken more than six months before the participant’s return.

2. Data Collection Protocol for Single-Day Services

Grantees are required to collect demographic and limited outcome data from participants whose total service duration is a single day. Data collection should occur once, immediately following the intervention, that is, at exit. To assess outcomes that are directly relevant to the intervention’s content, grantees are instructed to choose 3 to 5 survey items from Section Two of the Youth or Adult Questionnaire to be administered in addition to Section One. No participant-level data collection is required if the participant is only receiving HIV testing and testing counseling services. Grantees are required to keep records of the individuals receiving testing services for purposes of aggregate reporting as part of their quarterly progress reports.

Questions from Section Two must be chosen by the staff responsible for data collection prior to delivery of the service. Complete these steps to prepare questionnaires for participants:

1. Carefully fill out the Record Management section, making sure to mark ‘Single Session’ under Service Duration.

2. For Interview Type in the Record Management section, indicate ‘Exit’.

3. Highlight or circle the selected items and corresponding instructions from Section Two, making sure they are clearly marked.

4. Tear off and discard Section Three of the questionnaire.

Following delivery of the single session services, instruct participants to complete the entire Section One and only the questions selected from Section Two. Dosage forms should be completed by the appropriate project staff.

The 3 to 5 selected items should be directly relevant to the intervention’s content, that is, to the specific attitudes and/or knowledge the intervention is targeting. For example, if you are implementing VOICES/VOCES, which aims to prevent HIV transmission by encouraging condom use, appropriate question items from the Adult Questionnaire might be:

• Perception of risk of harm from unprotected sex (item 21)

• Confidence in ability to negotiate condom use (items 29 and 30)

• Knowledge about HIV transmission through unprotected intercourse (items 32 and 33)

Participants should not be given a mix of items from the Youth and Adult Questionnaires. For example, if Section One of the Adult Questionnaire is being administered to participants, please choose the additional Section Two items from the Adult Questionnaire. For more detailed guidance on how to choose survey items that are appropriate for your single session intervention, please refer to the ‘Single Session Intervention Data Collection Guide’.

The 3 to 5 items from Section Two must be selected by the staff responsible for data collection prior to delivery of the single session intervention.

nce the appropriateipant shouldns or encounters the program plans to have with the participant.ts across grant sites. The ites
. For example, if someone partnsidered to be mutually exclusive. For example, if someone participants in a one session int

3. When to Administer the Questionnaires

Once the planned service duration has been determined, the same sections should be given at every time point the survey is administered. Three possible data collection time points exist in which to collect data:

Table 2: When to Administer Questionnaires

Service Duration	Baseline (1st Data Collection Point)1	Exit (2nd Data Collection Point)2	Follow-up (3rd Data Collection Point)3
Single day (Single Session)	N/A	Immediately following the single session intervention	N/A
2-29 days (Multiple Session Brief)	Within 30 days preceding program exposure (i.e. the first service encounter)	Within 10 days after program exposure has ended (i.e. after the final service encounter)	N/A
30 days or longer (Multiple Session Long)	Within 30 days preceding program exposure (i.e. the first service encounter)	Within 10 days after program exposure has ended (i.e. after the final service encounter)	3-6 months after program exit

¹ No baseline data collection is required for single-day services.

² This is the only data collection point for single-day services.

³ No follow-up data collection is required for participants whose service duration is shorter than 30 days.

4. Selecting Questionnaire Proctors

CSAP strongly recommends that program staff who provide direct services to those taking the survey, such as case managers, should not be involved in administering questionnaires. Ideally, a staff member who is responsible for data collection but not directly involved in delivering services will be selected to proctor administration of the questionnaire. Staff who serve as proctors should be fully trained according to the administration procedures set out in this guide.

At the beginning of a questionnaire administration session, a program staff member should be onsite to introduce the proctor. The program staff member may then leave the room and return later to talk to participants and conduct the debriefing. The proctor should be familiar with the questionnaire’s corresponding administration guide and have it accessible during the session. Proctors should report to the staff responsible for data collection if respondents have serious problems understanding the questionnaire. If this occurs, please contact your assigned CSAP Project Officer.

5. Administration Time

The amount of time required to complete the questionnaire depends on the number of sections that are included.

Table 3: Administration Time

Service Duration	Sections of Survey To be Administered	Approximate Amount of Time To Complete*
Single Session	Section One: Facts about You 3 to 5 questions from Section Two: Attitudes & Knowledge	13 minutes (Youth & Adult)
Multiple Session Brief	Section One: Facts about You Section Two: Attitudes & Knowledge	26 minutes (Youth) 23 minutes (Adult)
Multiple Session Long	Section One: Facts about You Section Two: Attitudes & Knowledge Section Three: Behavior & Knowledge	37 minutes (Youth) 32 minutes (Adult)

*These are estimates; respondents with limited reading and/or English language ability may require more time.

Please allow extra time for distributing of the questionnaires, reading the instructions, collecting completed questionnaires, and any additional local administration activities (e.g., collecting tracking information). Administering the questionnaire to respondents with limited reading abilities (those with a reading level less than 8^th grade) may require full proctoring (reading the entire questionnaire aloud). Full proctoring requires more time than self-administration. If the questionnaire is read aloud, the proctor should practice reading the questions aloud several times beforehand. At the beginning of the survey, inform the respondents on how the questionnaire will be read to them. It is important to tell the respondents not to answer the questions out loud, but to simply mark their answers in the questionnaire.

6. Makeup Administration

If there are missing respondents when a questionnaire is being administered, take the following steps:

• At the next program session, plan to speak with the respondent(s) who did not attend the questionnaire administration session.

• Arrange a makeup session that is convenient for respondents; this can be done in person, by phone, or in writing.

• If a respondent misses a makeup session, or if a group makeup cannot be arranged, make reasonable efforts to administer the questionnaire individually.

7. Translated Instruments

Currently, the Youth and Adult Questionnaires are available in English only. Grantees that serve participants with other language needs should plan to translate the questionnaires and instructions locally after obtaining authorization from the Project Officer. A copy of the translated instruments should be sent to CSAP as an e-mail attachment addressed to your CSAP Project Officer.

Dosage Data: Individual and Group Dosage Forms

1. Dosage Forms

Documenting participant exposure to program services is an important part of the data collection process. Dosage data are designed to reflect the actual prevention practices of each grantee. Dosage is a measure of the type and amount of contact that a participant has in each direct encounter with the program. Programs are required to collect dosage (services) data for each individual participating in the intervention, using the Individual and Group Dosage Forms.

2. Collection of Dosage Data

Individual Dosage Forms should only be used for individual service encounters in which the participant is directly involved with staff in a one-on-one basis. Group Dosage Forms should be used for services delivered to multiple participants during a single encounter, such as a group education session attended by more than one participant. CSAP recognizes that health services are typically not delivered in a group format; however, health services can be recorded on either the Individual Dosage Form or the Group Dosage Form. For example, if identical health services (same service type and duration) are provided for multiple individuals on the same day, it is acceptable to create a single Group Dosage Form and list the unique identification numbers of all the participants receiving the service during that day. Dosage information is only collected on services that are partially or fully funded by the MAI grant.

Dosage data are collected over the course of participants’ engagement in the funded program. Once the participant’s services have ended, the collection of dosage data should cease for this participant.

Dosage data collection is performed on an ongoing basis throughout the service duration. Dosage forms are completed by the program staff responsible for data collection or by the service provider. A standard dosage data collection procedure should be implemented by the data collection team.

3. Completing the Dosage Forms

Those who fill out dosage forms should have an understanding of the service definitions outlined in the corresponding dosage administration guide and the convention for recording service duration codes. For each service type provided during the service encounter, the duration of exposure to that service should be recorded in minutes, rounded up to the next 5-minute interval. For example, the duration code for a service that lasts for 14 minutes should be 15, and a service that lasts 17 minutes should be recorded as 20 minutes.

If a grantee finds that recording duration for each session or encounter of a multi-session intervention places too much burden on staff, service providers should identify the services that are typically provided in these encounters or sessions and define a time estimate for each. These time allocations can be applied to all service encounters (i.e., all participants who complete a specific service would be given the same dosage codes, and the dosage codes would be the same across all program encounters). A participant should not receive a time estimate for a service that they never received. Although this approach would not be as accurate as determining the dosage codes for each encounter, over the course of the entire program the dosage data would reflect approximately how much exposure each participant received for each service. In either case, precautions should be made to maintain consistency in the definition of service types and duration code assignments.

If the grantee chooses to implement this system, the participant identification number on the Group Dosage Form should be filled out by the person conducting the session or encounter to indicate who was present at each session. If this is not possible, an attendance sheet should be kept by the person conducting the session or encounter, and the program staff responsible for data collection should enter the corresponding participant identification numbers on the Group Dosage Form using the attendance sheet.

Control/Comparison Groups

A comparison/control group can be used to examine the effect of an intervention. It is comprised of individuals from the same population as participants who receive the intervention; yet individuals in this group do not receive intervention services. While not required by CSAP, if your grant site is using a comparison/control group for local evaluation purposes, these individuals should receive the same questionnaires as program participants (intervention group). Surveys for comparison/control participants should be administered within two weeks before or after the administration of the intervention group instruments. Dosage data should not be collected for these individuals as they do not receive intervention services.

Web-Based Data Entry and Upload System

CSAP’s Program Evaluation for Prevention Contract (PEP-C) is currently responsible for the cross-site evaluation of the MAI program. An online data entry and batch upload system is under development; grantees will be informed when the system becomes available for data submissions.

In addition to item-by-item data entry of completed questionnaires and dosage forms, the system will provide batch file upload capability that the grantees can use to upload response databases, provided that they use the appropriate variable naming and value coding conventions described in the cross-site codebooks and use the standard Excel templates developed for entering data. Once data have been entered into the system, grantees will be able to download their raw data in spreadsheet form The PEP-C team will extract and clean the data periodically and each grantee will have access to their cleaned downloadable, electronic data files.

Questions about the Instruments and Submission to CSAP

Questions regarding the instruments or data submissions to CSAP should be addressed to the PEP-C Technical Assistance Team via telephone to 1-866-558-0724, or via email to [email protected].

CSAP MAI

Updated August, 2015 Page 12