Mai_mrt_ss_b 10.28.15

Quarterly Progress Reporting and Indirect Services Outcome Data Collection for the Minority AIDS Initiative (MAI) Substance Abuse and HIV Prevention Programs

OMB Supporting Statement

PART B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B1. Respondent Universe and Sampling Methods

Given that the primary function of the Quarterly Progress Report and ISO as a grant management and monitoring tool, all grantees are required to provide quarterly progress reports and annual updates to their community-level outcome measures through the ISO. SAMHSA’s response universe for this cross-site evaluation includes all active grantees and those initially funded at the end of FY 2015, throughout the duration of their grant periods. In order to ensure accountability for the spending of federal funds, SAMHSA has employed the use of these data as a performance management tool to ensure that grantees are meeting the goals and objectives of the initiative. Data are used to monitor performance throughout the grant period. The Public Health Service Act Sec. 501 [290aa] (d) (13) with respect to grant programs authorized under this title, assure that “all grants that are awarded for the provision of services are subject to performance and outcome data collections.” SAMHSA interprets these requirements to indicate the need for data to be collected on all grantees receiving federal funds in order to provide services.

B2. Information Collection Procedures

The Quarterly Progress Report and the Indirect Services Outcomes instruments will be initially launched as paper instruments disseminated to grantees. At the end of each quarter, grantees will complete the relevant module(s) of the instrument and submit their reports to their POs for review and feedback. At the end of the fourth quarter of each year, grantees will also submit a completed ISO instrument for each of the community-level outcome measure they wish to submit to SAMHSA (minimum requirement is a single measure updated every year). The submitted reports will also be coded and digitized by the PEP-C MAI Cross-Site Evaluation Team for incorporation into the main analysis database.

SAMHSA plans to develop an online system to facilitate data entry, retrieval, and linkage to the participant-level instruments. Once the system is in place, grantees and SAMHSA staff will be trained in its use and ongoing technical support will be available to all users.

The Quarterly Progress Report

The Quarterly Progress Report (Attachment 1) is a modular instrument structured around the Strategic Prevention Framework (SPF). Each section or module corresponds to a SPF step with an additional section dedicated to cultural competence and efforts to address behavioral health disparities, which is an overarching principle of the framework guiding every step. A section-by-section description of the instrument is provided in Exhibit 4 below.

Exhibit 4. Structure and Frequency of Quarterly Progress Reports

Section	Contents	Frequency
Health Disparities	Activities implemented during the reporting period to address behavioral health disparities in the community Specific accomplishments and barriers related to health disparities	Twice a year, as part of the second and fourth quarterly progress reports
Assessment	Synopsis of the needs assessment conducted by the grantee and key findings: High-risk groups identified Infrastructure and capacity enhancement targets Quarterly accomplishments and barriers associated with needs assessment Quarterly updates as needed Accomplishments and barriers related to planning	At least once during the needs assessment phase of the SPF, updated as needed in subsequent quarters
Capacity	Staff, advisory group, governing board, and collaborator rosters Advisory council meetings Training and technical assistance events Accomplishments and barriers related to capacity building	At least once during the capacity building phase of the SPF, updated as needed in subsequent quarters
Planning	Planned target population(s) Project goals and objectives Targeted outcome measures Planned direct-service interventions Planned indirect services (i.e. environmental strategies and information dissemination) Planned HIV/HCV testing and hepatitis vaccination activities Accomplishments and barriers associated with strategic planning	At least once during the strategic planning phase, updated as needed in subsequent quarters
Implementation	Numbers served through direct services and numbers reached through indirect services, by demographic group Grant expenditures Direct-service intervention implementation (timing, frequency, dosage, location, adaptations) Detailed list of direct services delivered Numbers tested for HIV and viral hepatitis, by demographics Numbers vaccinated for viral hepatitis Environmental strategies and information dissemination activities implemented Numbers referred outside the grantee organization for further services, by service type Outreach/recruitment activities Accomplishments and barriers associated with intervention implementation and service delivery	Quarterly during the implementation phase
Evaluation	Link to upload evaluation plan (online only) Link to the Indirect Services Outcome Instrument (online only) Accomplishments and barriers associated with evaluation	At least once during the evaluation phase

Before implementing any intervention, the grantees are required to enter planning information about that intervention in the Planning module. The Planning and Implementation modules are similarly structured to allow the evaluators to assess the degree to which the implementation followed the plan (i.e. fidelity to the plan). The grantees are provided with a list of the most common evidence-based direct-service interventions (Attachment 2) in order to standardize the way in which intervention names are reported across time and across grantees. Once the online system is in place, the list will be available as a dropdown menu in the Planning section. The Implementation section will provide each grantee with a dropdown menu consisting of the interventions they entered in the Planning section. This dynamically updated grantee-specific dropdown menu will also be linked to the online participant-level instruments in order to link each participant’s pre-post and follow-up data to the direct-service intervention(s) they received.

Indirect Services Outcomes Instrument (ISO)

The Indirect Services Outcomes Instrument (Attachment 3) is a brief instrument to collect community-level outcome data for use in assessing the effectiveness of funded environmental strategies and information dissemination activities in changing community norms, policies, behaviors, and/or systems of care. The grantees are encouraged to work with local agencies that collect administrative and epidemiological data and with their state’s Epidemiological Outcomes Workgroups to obtain relevant data describing their targeted community’s relevant characteristics. These data are typically either community survey results measuring risk and protective factors and behaviors associated with substance abuse and HIV/HCV transmission, or event/surveillance data maintained by law enforcement, health, or educational agencies and institutions, such as alcohol/drug related arrests or traffic crashes/fatalities, HIV or viral hepatitis diagnoses, substance abuse related emergency room visits, or relevant campus records (in cases where the grantee is a minority-serving educational institution).

The ISO provides the grantee with data fields that describe the data source and enter the calculated value annually. The description of the data source (including population age range and size, sampling ratio, etc.) are used by the cross-site evaluation team to calculate effect sizes and if necessary, introduce weights into the community outcomes analysis data file.

Data Collection

As noted earlier, the Quarterly Progress Report and ISO will initially be launched as paper instruments to be completed and submitted by grantees. They will be incorporated into an online data entry system in the future, contingent on resource availability. The last column of Exhibit 4 indicates the frequency with which each section (or module) of the Quarterly Progress Report will be completed. The ISO will be completed for at least one community-level outcome measure. Data collection will start with a baseline data submission preceding the onset of funded activities and will be updated every year during the grant period. In cases where data updates are only available every other year (e.g. id the source of the data is the Youth Risk Behavior Surveillance System), the grantee will be able to file a request to waive the annual requirement. SAMHSA will review these waiver requests and make decisions on a case-by-case basis. The PEP-C MAI Cross-Site Team will provide training and technical assistance to grantees in selecting, obtaining, and submitting data on community-level outcome measures in line with their strategic plans.

Data Management

Storage System: Over the life of this initiative, Quarterly Progress Report and ISO data submitted by the grantees either in paper form or as extracts from an online data submission system will be filed in a secure location at SAMHSA together with other grant management and monitoring records. Digitized versions of the data used in the cross-site evaluation will be integrated with participant-level cross-site data and stored in secured databases in accordance with the security guidelines defined by the Federal Information Security Management Act (FISMA) and NIST Special Publication 800-53 for Moderate information system security.

B5. Statistical Consultants

Sara Azimi-Bolourian, Ph.D.

Public Health Analyst

Performance and Technical Assistance Branch

SAMHSA/CSAP

1 Choke Cherry Road, Room 4-1031

Rockville, MD 20857

Linda D. Youngman, Ph.D.

Branch Chief

Division of Community Programs

SAMHSA/CSAP

1 Choke Cherry Road, Room 4-1115

Rockville, MD 20857

Patricia A. Sabry

Public Health Advisor

Division of Community Programs

SAMHSA/CSAP

1 Choke Cherry Road, Room 4-1111

Rockville, MD 20857

Phillip W. Graham, Dr.PH

Director, Drugs, Violence, and Delinquency Prevention Research Program,

Center for Justice, Safety, and Resilience,

RTI International

PO Box 12194

Research Triangle Park, NC 27709

Elvira Elek, Ph.D

Research Public Health Analyst

RTI International

701 13th St NW #750

Washington, DC 20005

Nilufer Isvan, Ph.D.

Senior Research Fellow

Human Services Research Institute

2336 Massachusetts Avenue

Cambridge, MA 02140

Mindy Herman Stahl, Ph.D.

Research Psychologist,

RTI International

PO Box 12194

Research Triangle Park, NC 27709

Darigg Brown, Ph.D.

Research Public Health Analyst,

RTI International

2951 Flowers Rd South, Suite 119

Atlanta, GA 30341

Jeffrey Guidry, Ph.D.

Associate Professor

Department of Health & Kinesiology

Texas A&M University, Read 159 Bldg.

College Station, TX 77843

William DeJong, Ph.D.

Professor, Community Health Sciences,

Boston University School of Public Health

801 Massachusetts Avenue

Boston, MA 02118

ATTACHMENTS

1. MAI Quarterly Progress Reporting Tool

2. MAI Indirect-Service Intervention Name List

3. MAI Retrospective Reporting Tool

6