Monitoring Data Collection Tools for the Minority AIDS Initiative (MAI)
Supporting Statement
The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Behavioral Health Statistics and Quality (CBHSQ) is requesting a revision from the Office of Management and Budget (OMB) for the collection of monitoring data from the Center for Substance Abuse Prevention’s Minority AIDS Initiative (MAI) programs. This request consolidates instruments approved under OMB No. 0930-0298 (expiration date 03/31/2019) and OMB No. 0930-0357 (expiration date 05/31/2019). The current package removes instruments that are no longer needed and streamlines the others. The three instruments supported by the present statement are:
Quarterly Progress Report (Attachment 1)
Adult Questionnaires (Attachment 2)
Youth Questionnaires (Attachment 3)
The MAI program is authorized by Section 516 [290bb-22] Priority Substance Abuse Prevention Needs of Regional and National Significance - of the Public Health Service Act, as amended, and subject to the availability of funds. It was supported by the Congressional Black Caucus through its Conference Report on H.R. 4328, Making Omnibus Consolidated and Emergency Supplemental Appropriations Act, for FY 1998 (House of Representatives, October 19, 1998), to address prevention and treatment needs of minority communities that are disproportionately affected by HIV/AIDS. It builds on previously authorized programs addressing these issues (discussed below).
Also, this data collection supports the four primary goals of the National HIV/AIDS Strategy which include: 1) reducing new HIV infections, 2) increasing access to care and improving health outcomes for people living with HIV/AIDS, 3) reducing HIV-related disparities and health inequities, and 4) achieving a coordinated national response to the HIV epidemic.
Quarterly Progress Report:
The quarterly progress report is organized around the Strategic Prevention Framework (SPF). This framework consists of five interrelated steps and two overarching principles. The first step for all grantees is to conduct a needs assessment in their target communities and to submit a report summarizing their findings. Grantees next work on building their prevention capacity to meet the needs identified in the previous step. The third step is strategic planning which includes identifying targets, selecting effective community prevention programs, policies, and practices that best align with the needs of the community. The strategic plan developed during this phase is submitted to CSAP and reviewed by the grantee’s Project Officer (PO). The plan is revised in line with the PO’s feedback and once approved by the PO, it is put into action, initiating the fourth SPF step, that is, implementation. The final step is evaluation of outcomes. All grantees are required to submit an evaluation report at the end of their grant period. The two guiding principles of the SPF are sustainability and cultural competence. Grantees are encouraged to direct their efforts toward infrastructure building and program implementation strategies that are likely to sustain their effects after the end of the grant. Additionally, all activities have to be planned and executed with careful consideration of the specific cultural and linguistic needs of the targeted groups.
Given that the success of each SPF step is dependent on the competence with which the previous step was executed, there is need for constant monitoring of grantees’ progress through the process. This is especially true of grantees that are new to the SPF, but all grantees moving through the SPF steps need to be closely monitored, their barriers promptly addressed, and their training and technical assistance needs met without delay. The overall purpose of the Quarterly Progress Report is to facilitate communication between grantees and their POs about the progress of the grantee through the steps of the SPF. Although POs also communicate with their grantees through regular conference calls and site visits, the Quarterly Progress Report provides them with a standard tool for assessing their grantees’ progress. It also regulates the exchange of information about the grantees’ accomplishments and barriers.
Youth and Adult Questionnaires:
The adult and youth questionnaires aim to assess changes in attitudes, knowledge and behaviors among clients at baseline and follow-up. In addition, the questionnaires capture information specific to the intervention dosage by recording the service types and program dosage that each participant receives in direct service interventions. These will be completed by clients after each service encounter at baseline and follow-up.
MAI Grantees
There will be two grant programs reporting using the tools. The first is the Secretary’s Minority AIDS Initiative Fund (SMAIF). The purpose of SMAIF is to reduce new HIV infections, improve HIV-related health outcomes, and to reduce HIV-related health disparities for racial and ethnic minority communities. SMAIF achieves these objectives by supporting innovation, collaboration, the integration of best practices, effective strategies, and promising emerging models in the response to HIV among minority communities. SMAIF is competitive funding open to federal agencies and offices within the Department of Health and Human Services and subject to the requirements and conditions of award outlined in this funding opportunity announcement. SAMHSA was awarded SMAIF funding for 1-2 years.
The second program is MAI’s Capacity Building Initiative (CBI), which funds community-level domestic, public and private nonprofit entities, federally recognized American Indian/Alaska Native Tribes and tribal organizations, and urban Indian organizations. CBI grants focus on building a solid infrastructure for integrated substance abuse (SA), HIV, and VH prevention service provision and implementation of evidence-based prevention interventions. The target population for the CBI grantees is at-risk minority adolescents and young adults.
The purpose of this data collection is to inform program direction and identify and address program weaknesses. SAMHSA must also collect these data to meet its federal requirements specified in the Government Performance and Results Modernization Act (GPRMA) of 1993 and the GPRA Modernization Act of 2010 (PL 111-352). The information collected through the Quarterly Progress Report, Dosage Forms, and will also be used for various federal government reports including:
Congressional HIV Testing Report (annual)
National HIV/AIDS Strategy progress report (annual)
Viral Hepatitis Action Plan (VHAP) progress report (annual)
Office of HIV/AIDS and Infectious Disease Policy (OHAIDP) African American Inventory report (annual)
White House Minority AIDS Initiative progress report (annual)
White House Minority Serving Institutions progress report (annual)
Secretary’s Minority AIDS Initiative Fund (SMAIF) progress report (annual for ongoing SMAIF-funded programs –
Office of National AIDS Policy (ONAP) implementation updates (semi-annual)
In addition, SAMHSA provides data for various Office of HIV/AIDS and Infectious Disease Policy (OHAIDP) reports and numerous ad hoc reports, as well as reports issued by the Government Accountability Office (GAO). Most of the ad hoc requests for which SAMHSA anticipates using these data require specific information on ethnicity and populations served, sometimes combining these numbers with budgetary information to estimate the costs associated with serving individuals from various demographic backgrounds or sexual orientations. SAMHSA also uses HIV program data for its own program policy, planning and development purposes.
HHS has requested that Federal agencies coordinate their efforts regarding HIV data collection and use of data to reduce burden to grantees and to better utilize collected data. To meet these requests, SAMHSA is collaborating with other Federal agencies that also have ongoing HIV programs, predominantly Health Resources and Services Administration (HRSA), Centers for Disease Control and Prevention (CDC) and National Institutes of Health (NIH), to streamline data collection efforts. For example, SAMHSA staff participated with colleagues in CDC to harmonize data collection efforts for HIV testing. SAMHSA also participates in numerous other inter-agency working groups regarding use of and reporting of HIV data. SAMHSA has already informally shared HIV testing data with CDC and HRSA and is working to harmonize HIV testing data collection so that these data may be formally shared and utilized by other Federal agencies.
The information will be used to influence public policy studies and programming as they relate to the provision of youth and adult services. More specifically, the data will support the following uses by CSAP:
Annual reports to Congress
Information regarding SPF implementation and community-level change will be used in conjunction with participant-level outcome data to assess the effectiveness of currently funded prevention programs. These data will also help program planners and policy makers identify the types of strategies and combinations of strategies that are most effective in the prevention, delay or reduction of substance use, and in reducing risk factors & enhancing protective factors associated with SA and HIV transmission (e.g. knowledge, attitudes, norms, and risky sexual behaviors)
Findings concerning program inputs (intervention strategies, frequency, and length) will be used to provide program guidelines and to plan appropriate technical assistance services for programs
Findings will support CSAP publications and materials on prevention practices that are an important resource for public and private organizations involved in the design and implementation of prevention programming for youth and adults
In sum, the findings from these reports will be a crucial resource for CSAP in setting prevention policy priorities, measuring program performance, and designing and promoting optimally effective prevention program initiatives. SAMHSA will ensure that the data on HIV positivity rates will be shared with CDC, HRSA and other relevant HHS Agencies.
CSAP plans to continue to enhance the current knowledge base on the effectiveness of prevention programs for minority populations at risk for SA, hepatitis transmission and HIV/AIDS as well as increase public awareness about factors associated with substance use and HIV risk behaviors among minority populations. Information collected under CSAP’s programs will be used by CSAP and other Federal agencies in their efforts to assess specific intervention services in the prevention or reduction of substance use and HIV/AIDS among minority populations.
Beyond HHS, CSAP plans to share lessons learned with:
The Department of Justice (DOJ) and their Office of Juvenile Justice and Delinquency Prevention (OJJDP), which funds projects that target high-risk youth and often involve SA prevention interventions.
The Department of Housing and Urban Development (HUD), which supports low-income persons and families living with HIV/AIDS through its “Housing Opportunities for Persons with AIDS Program.”
The Department of Education (DOE), one of the collaborators in the multi-agency “Safe Schools/Healthy Students" effort (focused on violence and substance abuse prevention) under the Drug Free Schools and Communities Act.
State and local program planners and the public through publications and a public-use data set.
Changes:
Revisions to the monitoring tools approved under OMB 0930-0357 include the following:
Quarterly Progress Report (QPR):
Added opioid items to lists for targeted outcome measures, name of direct services list, indirect services - environmental strategy list and environmental strategy purpose
Added fields to capture Project Director (PD), Coordinator and Evaluator name, email, and phone numbers
Added Promising Approaches and Innovations Section (2 questions)
Added upload screen for Final Evaluation Report (for closeout grantees only)
Revisions to the monitoring tools approved under OMB 0930-0298 include the following:
Adult Questionnaire
Added questions to capture details on the intervention and the referrals to the record management section (completed by grantee staff)
Aligned questions with CSAT/CMHS tools & the Rapid HIV/Hepatitis Testing (RHHT) form, where possible
Removed demographic questions related to language, education, employment status, health, military details, and relationship status as they were not central to assessing program performance
Removed knowledge & attitude questions about peer behavior & how they feel about it, sex refusal skills, & HIV knowledge as they were not central to assessing program performance
Removed behavior questions related to other tobacco products, electronic vapor products, synthetic marijuana, mental health, experience with alcohol use, sexual abuse as they were not central to assessing program performance
Added opioid drug questions
Youth Questionnaire
In addition to all items listed above, on the youth questionnaire, SAMHSA also removed questions related to the following which were deemed not central to assessing program performance:
Interest in school & feelings about ethnic identity
Relationships with parents or guardians
Friend substance abuse and sexual behavior
Exposure to prevention education messages
The following tools were removed:
Indirect Service Outcomes (ISO)
Removed indirect service outcomes module as data was not needed for GPRA reporting
HIV Testing Retrospective Reporting Tool
Removed HIV Testing Retrospective Reporting Tool as data was not needed for GPRA reporting
The following tools were removed from OMB 0930-0298
Individual Dosage Form
Removed the Individual Dosage Form by incorporating the relevant questions about the intervention into the Adult and Youth Questionnaires, as noted above.
Group Dosage Form
Removed the Group Dosage Form by incorporating the relevant questions about the intervention into the Adult and Youth Questionnaires, as noted above.
A3: Use of Technology
To maximize data accuracy and reliability, the data collection instruments will be web-based tools that grantees complete online. In case there is a lag between the time approval is obtained and the time resources become available for systems development, the instruments will be sent to grantees as paper questionnaires to be completed. The online tools will be made available to grantees as soon as system development is complete.
The system being planned for the instruments will require a web browser and access to the Internet. Users will be able to access the system 24 hours a day, 7 days a week, aside from scheduled maintenance windows, through the use of an encrypted username and password. Upon logging into a system-assigned account, grantees will be able to: enter data on their program; upload documents for the project officer review; and generate quarterly and annual reports of their activities. Skip patterns will facilitate navigation through the instrument by only displaying items that apply to the respondent, based on information already entered into the system. The system will also allow SAMHSA’s project officers to review and approve submitted progress reports or ask the grantee to provide additional information regarding their activities. Project officers will also have the capability to generate online summary reports on their grantees’ progress.
SAMHSA conducted an extensive literature search, consulted with staff in Federal agencies and organizations that work with substance use and HIV/AIDS prevention programs, and discussed the proposed program with substance abuse prevention experts. Specifically, CSAP:
Reviewed studies to identify any methodological problems that might detract from the validity, generalizability, or policy application of results.
Consulted with staff in CSAT, CDC, NIAID, NIDA, ACF, OJJDP, HUD, DOE and DOJ. None of these Federal organizations had collected data on prevention and early intervention programs targeting minority youth and minority re-entry youth similar to that being proposed in this submission.
In summary, SAMHSA did not identify any redundancy in that there were no precedents for a data collection effort similar to the one being proposed. Thus, it is clear that the data to be collected will be unique to the SAMHSA/CSAP MAI programs. In other words, the data collected through these instruments will be non-duplicative and will minimize burden on grantees.
This data collection will have no significant impact on small entities.
The Quarterly Progress Report is a modular instrument structured around the SPF steps, and designed to be updated quarterly. Only the modules corresponding to the steps that the grantee actively worked on during any given quarter will be completed at each wave of data collection. The module on cultural competence, one of the overarching guiding principles of the SPF that affects every step, will be completed twice a year, as part of the second and fourth quarters’ progress reports. Each module contains data elements on the grantee’s accomplishments and barriers associated with the associated phase or principle of the SPF. If these data are collected less frequently SAMHSA/CSAP’s ability to promptly respond to inappropriate strategies and activities with corrective action and to meet grantees’ training and technical assistance needs in a timely fashion will be negatively affected. Delays in these responses will, in turn, have an impact on grantees’ subsequent SPF steps, causing a cascading effect on overall program effectiveness.
Another reason for quarterly reports of implementation activities is that some of the data are used to meet national data collection needs, especially on HIV/AIDS. For example, information on the numbers of individuals tested for HIV and those with positive test results are typically updated frequently to maintain as close to real-time data as possible.
The client level questionnaires
This information collection fully complies with 5 CFR 1320.5(d) (2).
The notice required in 5 CFR 1320.8(d) was published in the Federal Register on August, 10 2018 (83 FR 39766).
CSAP consulted with experts on Substance Abuse (SA), viral hepatitis (VH), and HIV/AIDS within HHS, as well as other Federal agencies with related programs or mandates, including NIDA, ACF, CDC, DOJ, OJJDP, HUD, and the DOE. Consultations resulted in the refinement of the instruments based on current Federal data reporting needs.
SAMHSA received a public comment from the 60-Day Notice (Attachment 4). The Truth Initiative requested the following two changes:
1) Ask about cigarettes and other tobacco products separately. (Please see questions 26 in the adult questionnaire and 23 in the youth questionnaire for the revised questions).
2) Include brand examples in the help text to clarify what types of vapor electronic products may be included. Both of these edits were made.
Please see question 24 in the youth questionnaire and question 27 in the adult questionnaire
SAMHSA agrees with these revisions and incorporated the change into the instruments.
No payment is received by respondents.
SAMHSA has statutory authority to collect data under the Government Performance and Results Act (Public Law 1103(a), Title 31) and is subject to the Privacy Act for the protection of these data. Only aggregate data will be collected with the Quarterly Progress Report and client level instruments, hence protecting the privacy and confidentiality of program clients and participants.
The information from grantees and all other potential respondents will be kept private and secure through all points in the data collection and reporting process. All data will be closely safeguarded, and no institutional or individual identifiers will be used in reports. It is critical to note that only aggregate data on HIV positivity will be reported to SAMHSA (e.g., 100 tests were done and 3 were positive giving 3% HIV positivity). SAMHSA and its contractors will not receive identifiable client records. A PIA and SORN application have been submitted for this data collection for review at the U.S. Department of Health and Human Services. Provider-level information will be aggregated to, at the least, the level of the grant/cooperative agreement-funding announcement.
Any data of a sensitive nature about individual clients will be aggregated to the program level (e.g., total numbers served, reached, tested, tested for the first time, and tested positive) before being reported.
As mentioned earlier, the Quarterly Progress Report is a modular instrument that will be updated quarterly as needed. Grantees will only update the modules corresponding to the SPF steps that they actively worked on during any given quarter. The cultural competence section will be completed every other quarter. The client level questionnaires will be answered as they are completed by clients.
Grantees included in MAI data collection include the following:
CBI: 2015: 54
CBI: 2016: 19
CBI: 2017: 12
Navigators 2017: 20
CBI: 2018: 40
Secretary Minority AIDS Initiates Fund (SMAIF) 10
Exhibit 1 below displays the calculation of annualized burden for the three instruments. In calculating the total number of respondents, it was assumed that a single staff member within each grant site will be the respondent for the entire data collection effort, rather than different respondents for each instrument.
Exhibit 1: Total Estimated Annualized Burden by Instrument
Type of respondent activity |
Number of Respondents |
Responses per Respondent |
Total Responses |
Hours per Response |
Total Burden Hours |
Wage Rate |
Total Hour Cost |
Quarterly Progress Report |
155 |
4 |
620 |
4 |
2,480 |
$21.79 |
$9,920 |
Adult questionnaire |
12,000 |
2 |
24,000 |
.20 |
4,800 |
$21.79 |
$105,592 |
Youth questionnaire |
3,000 |
2 |
6,000 |
.10 |
600 |
$21.79 |
$13,074 |
Total |
15,155 |
|
30,620 |
|
7,880 |
|
128,586 |
There will be no capital, start up, or operation and maintenance costs.
The annualized cost is approximately $361,202. This includes approximately $200,000 for developing the instruments online; and $105,600 for providing data collection training to grantees and housing the data. In addition, approximately $105,600 per year represents SAMHSA costs to monitor and approve grantee reporting in these instruments (10% time of 10 Project Officers at $108,600 annual salary). The annualized cost is approximately $361,202
Currently, there are 2,848 burden hours in the OMB inventory. The Program is requesting 7,880 burden hours. This increase of 5,032 hours is a program change due to adding the Adult Questionnaire and the Youth Questionnaire to this data collection.
When this data collection is approved, SAMHSA will request a discontinuation of OMB No. 0930-0298, which currently has 35,139 burden hours. This will result in a net reduction of 30,107 hours. This reduction will be achieved by the removal of the Indirect Service Outcomes (ISO) instrument (316 hours) and the HIV Testing Retrospective Reporting Tool (4 hours) from OMB No. 0930-0357 and streamlining the Adult Questionnaire (from 24,378 hours to 4,800) and the Youth Questionnaire (from 1,203 hours to 600 hours) from OMB No. 0930-0298.
Analysis Plans
As previously noted, the main purpose of the monitoring data is to provide SAMHSA with timely information about the progress of the MAI grantees through the SPF steps, to identify and promptly respond to training and technical assistance needs of the grantees, and to recommend corrective action in a timely fashion in cases where grantees’ activities do not comply with the SPF. Some of the data fields in the Quarterly Progress Report and adult and youth questionnaires are also used to meet SAMHSA’s reporting requirements. In the rest of this section, planned analysis methods for each of these functions is discussed separately.
The utilization of the data for grant management and monitoring purposes involves narrative and qualitative reviews of the information rather than quantitative analyses. POs qualitatively compare the quarterly performance of each grantee to their performance in previous quarters and to the expected progress trajectory suggested by the SPF model. Grantees that are not performing as expected are provided with customized technical assistance from a variety of sources depending on the nature of the specific challenges.
The proposed analysis utilizing the Quarterly Progress Report and Client Level Instrument data includes several distinct steps:
Descriptive analysis of grantee targets, organizational structure, training and technical assistance activities, and implemented interventions will be conducted and the results presented separately by the Funding Opportunity Announcement (FOA) to which the grantees responded.
Pooled analyses of participant-level outcomes will be conducted to assess overall program effects and their sustainability. Program effects will be evaluated through paired comparisons of baseline and exit values. Sustainability of effects will be evaluated through paired comparisons of baseline and follow-up values. Past analyses have suggested that some measures continue to improve after program exit. To continue to assess this post-exit improvement, paired comparisons between exit and follow-up values will also be conducted.
Site-specific data obtained from the Quarterly Progress Report, such as types and combinations of interventions implemented, fidelity of implementation, and grantee organizational characteristics, will be introduced into the multilevel models to investigate the sensitivity of effectiveness models to differences in intervention characteristics, fidelity, and grantees’ organizational characteristics. The planned multilevel multivariate models will also test hypotheses about interactions between individual and site-specific factors in determining participant outcomes.
Analysis Techniques and Statistical Test Determination
In assessing overall participant- and community-level improvement based on pooled data, paired comparison tests appropriate to the level of measurement will be employed. For dichotomous outcome measures, significance will be tested using McNemar’s test (Lidell, 1976; Yang, Sun, & Hardin, 2010). For normally-distributed continuous outcome measures, matched-pairs t-tests will be used to assess significance. The significance of change in ordinal or skewed continuous measures will be tested using the Wilcoxon signed-rank test (Wilcoxon, 1945; Blair & Higgins, 1980).
Reporting and Dissemination Plan
Data will primarily be used for Project Officer monitoring and assessment of how grantees progress through the SPF process. Some data will also be reported for GPRA purposes needed in the Congressional Justification. Finally, some data may be used for presentations that SAMHSA staff may make to other staff holders or for short impact reports for senior leadership.
Timeline
CBI grants are funded up to five years. Typically, there is an initial period devoted to Steps 1, 2 and 3 of the SPF, namely conducting needs assessment, capacity building, and strategic planning. Grantees begin implementation and data collection only after their proposed strategic prevention plans are approved by their SAMHSA project officers. The approximate dates of the SPF milestones for the grantees that will be using the instruments are presented in Exhibit 2 below.
Exhibit 2. Project Timelines
|
CBI Awarded in 2015 (5-year grants) |
CBI Awarded in 2016 (5 year grants |
CBI |
Prevention Navigators |
CBI Awarded in 2018 |
SMAIF |
Needs Assessment, Capacity Building, Strategic Planning |
10/2015 – 6/2016 |
10/2016 – 6/2017 |
10/2017 – 6/2018 |
10/2017 – 6/2018 |
10/2018 – 6/2019 |
10/2018- 10/2019 |
Implementation, Data Collection |
6/2016 – 9/2020 |
6/2017-6/2021 |
6/2018-6/2022 |
6/2019-6/2023 |
6/2020-6/2023 |
N/A |
Follow-up, Evaluation, and Reporting |
Ends 9/30/2020 |
Ends 9/30/2021 |
Ends 9/30/2022 |
Ends 9/30/2022 |
Ends 9/30/2023 |
Ends 9/30/2019 (OR 2020) |
The expiration date will be displayed.
No exceptions are required.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | HIV OMB Supporting Statement |
| Author | proddy |
| File Modified | 0000-00-00 |
| File Created | 2021-01-20 |