Monitoring Data Collection Tools for the Minority AIDS Initiative (MAI)
Supporting Statement
The Substance Abuse and Mental Health Services Administration (SAMHSA) is requesting approval for from the Office of Management and Budget (OMB) for the revised collection of monitoring data from Center for Substance Abuse Prevention (CSAP)’s Minority AIDS Initiative (MAI) (OMB No. 0930-0357) programs which expires March 31, 2022. The three instruments supported by the present statement are:
Quarterly Progress Report
Adult Questionnaire
Youth Questionnaire
The MAI program is authorized by Section 516 of the Public Health Service Act, as amended, and subject to the availability of funds. It was supported by the Congressional Black Caucus through its Conference Report on H.R. 4328, Making Omnibus Consolidated and Emergency Supplemental Appropriations Act, for FY 1998 (House of Representatives, October 19, 1998), to address prevention and treatment needs of minority communities that are disproportionately affected by HIV/AIDS. It builds on previously authorized programs addressing these issues (discussed below).
Also, this data collection supports the four primary goals of the National HIV/AIDS Strategic Plan:
Prevent new HIV infections
Improve HIV-related health outcomes of people with HIV
Reduce HIV-related disparities and health inequities
Achieve integrated and coordinated efforts that address the HIV epidemic among all partners and stakeholders. 1
Quarterly Progress Report:
The quarterly progress report is organized around the Strategic Prevention Framework (SPF).2This framework consists of five interrelated steps and two overarching principles. The first step for all grantees is to conduct a needs assessment in their target communities and to submit a report summarizing their findings. Next, grantees work on building their prevention capacity to meet the needs identified in the previous step. The third step is strategic planning which includes identifying targets and selecting effective community prevention programs, policies, and practices that best align with the needs of the community. The strategic plan developed during this phase is submitted to CSAP and reviewed by the grantee’s Government Project Officer (GPO). The plan is revised in line with the GPO’s feedback and, once approved by the GPO, it is put into action, that is, implemented, initiating the fourth SPF step. The final step is evaluation of outcomes. All grantees are required to submit an evaluation report at the end of their grant period. The two guiding principles of the SPF are sustainability and cultural competence. Grantees are encouraged to direct their efforts toward infrastructure building and program implementation strategies that are likely to sustain their effects after the end of the grant. Additionally, all activities have to be planned and executed with careful consideration of the specific cultural and linguistic needs of the targeted groups.
Given that the success of each SPF step is dependent on the competence with which the previous step was executed, there is need for constant monitoring of grantees’ progress through the process. While this is especially true of grantees that are new to the SPF, all grantees moving through the SPF steps need to be closely monitored, their barriers promptly addressed, and their training and technical assistance needs met without delay. The overall purpose of the Quarterly Progress Report is to facilitate communication between grantees and their GPOs about the progress of the grantee through the steps of the SPF. Although GPOs also communicate with their grantees through regular conference calls and site visits, the Quarterly Progress Report provides GPOs with a standard tool for assessing their grantees’ progress. It also regulates the exchange of information about the grantees’ accomplishments and barriers.
Youth and Adult Questionnaires:
The adult and youth questionnaires aim to assess changes in knowledge, attitudes, and behaviors among clients at baseline and follow-up. In addition, the questionnaires capture information specific to the intervention dosage by recording the service types and program dosage that each participant receives in direct service interventions. These questionnaires will be completed by clients after each service encounter at baseline and follow-up.
The questionnaires will be administered to direct-service program participants. The Youth Questionnaire is designed for persons aged between 12 and 17 and the Adult Questionnaire is designed for persons aged 18 and older. Although the questions often differ for age appropriateness, the major constructs for both questionnaires include demographics, 30-day substance use, perception of risk of substance use, perception of peers’ risky behaviors, sexual behavior, and knowledge of HIV.
MAI Grantees
Two reporting grant programs will be using quarterly progress reports and youth/adult questionnaires. The first is the MAI’s Capacity Building Initiative (HIV-CBI) which funds domestic community-level public and private nonprofit entities, federally recognized American Indian/Alaska Native Tribes and tribal organizations, and urban Indian organizations. These grants focus on building a solid infrastructure for integrated substance abuse (SA), HIV, and VH prevention service provision and implementation of evidence-based prevention interventions. The target population is at-risk minority adolescents and young adults.
The second program is the HIV Prevention Navigator Program for Racial/Ethnic Minorities Ages 13–24 Cooperative Agreement (Prevention Navigator). The Prevention Navigator program provides services to those at highest risk for HIV and substance use disorders using a navigation approach (Community Health Workers, Neighborhood Navigators, and Peer Support Specialists) to expedite services for these populations.
The purpose of this data collection is to inform program direction and identify and address program weaknesses. SAMHSA must also collect these data to meet its federal requirements specified in the Government Performance and Results Modernization Act (GPRMA) of 1993 and the GPRA Modernization Act of 2010 (PL 111-352). The information collected through the Quarterly Progress Report will also be used for various federal government reports, including:
Congressional HIV Testing Report
National HIV/AIDS Strategy progress report
Viral Hepatitis Action Plan (VHAP) progress report
Ending the HIV Epidemic Initiative reports
In addition, SAMHSA provides data for various Office of Infectious Disease and HIV/AIDS Policy (OIDP) reports and numerous ad hoc reports, as well as reports issued by the Government Accountability Office (GAO). Most of the ad hoc requests for which SAMHSA anticipates using these data require specific information on ethnicity and populations served, sometimes combining these numbers with budgetary information to estimate the costs associated with serving individuals from various demographic backgrounds or sexual orientations. SAMHSA also uses HIV program data for its own program policy, planning and development purposes.
The information will be used to influence future public policy studies and programming as they relate to the provision of youth and adult services. More specifically, the data will support the following uses by CSAP:
Annual reports to Congress
Information regarding SPF implementation and community-level change will be used in conjunction with participant-level outcome data to assess the effectiveness of currently funded prevention programs. These data will also help program planners and policy makers identify the types of strategies and combinations of strategies that are most effective in the prevention, delaying or reducing substance use, and lowering risk factors and enhancing protective factors associated with SA and HIV transmission (e.g. knowledge, attitudes, norms, and risky sexual behaviors).
Findings concerning program inputs (intervention strategies, frequency, and length) will be used to provide program guidelines and to plan appropriate technical assistance services for programs.
Findings will support CSAP publications and materials on prevention practices that are an important resource for public and private organizations involved in the design and implementation of prevention programming for youth and adults.
In sum, the findings from these reports will be a crucial resource for CSAP in setting prevention policy priorities, measuring program performance, and designing and promoting optimally effective prevention program initiatives. SAMHSA will ensure that the data on HIV positivity rates are shared with CDC, HRSA and other relevant HHS Agencies.
CSAP plans to continue to enhance the current knowledge base on the effectiveness of prevention programs for minority populations at risk for SA, hepatitis transmission and HIV/AIDS, as well as increase public awareness about factors associated with substance use and HIV risk behaviors among minority populations. Information collected under CSAP’s programs will be used by CSAP and other Federal agencies in their efforts to assess specific intervention services in the prevention or reduction of substance use and HIV/AIDS among minority populations.
Changes:
Demographics:
Update ethnicity, race, gender, and sexual orientation to align with changes made to the client level GPRA tools for mental health and substance use treatment grant programs.
Quarterly Progress Report (QPR):
Updated references
Changed output targets to annual reporting from lifetime of grant reporting.
A3: Use of Technology
To maximize data accuracy and reliability, the data collection instruments are web-based tools that grantees complete online. The system requires a web browser and access to the Internet. Users are able to access the system 24 hours a day, 7 days a week, aside from scheduled maintenance windows, through the use of an encrypted username and password. Upon logging into a system-assigned account, grantees are able to: enter data on their program; upload documents for the project officer review; and generate reports of their activities. Skip patterns facilitate navigation through the instrument by only displaying items that apply to the respondent, based on information already entered into the system. The system also allows SAMHSA’s project officers to review and approve submitted progress reports or ask the grantee to provide additional information regarding their activities. Project officers also have the capability to generate online summary reports on their grantees’ progress.
For the original OMB request, SAMHSA conducted an extensive literature search, consulted with staff in Federal agencies and organizations that work with substance use and HIV/AIDS prevention programs, and discussed the proposed program with substance use disorder prevention experts. Specifically, CSAP:
Reviewed studies to identify any methodological problems that might detract from the validity, generalizability, or policy application of results.
Consulted with staff in SAMHSA/CSAT, CDC, NIAID, NIDA, ACF, OJJDP, HUD, DOE and DOJ. None of these Federal organizations had collected data on prevention and early intervention programs targeting minority youth and minority re-entry youth similar to that being proposed in this submission.
In summary, SAMHSA did not identify any redundancy in that there were no precedents for a data collection effort similar to the one being renewed. Thus, it is clear that the data to be collected will be unique to the SAMHSA/CSAP MAI programs. In other words, the data collected through these instruments will be non-duplicative and will minimize burden on grantees.
This data collection will have no significant impact on small entities.
The Quarterly Progress Report is a modular instrument structured around the SPF steps and designed to be updated quarterly. Only the modules corresponding to the steps that the grantee actively worked on during any given quarter will be completed at each wave of data collection. Each module contains data elements on the grantee’s accomplishments and barriers associated with the specific phase or principle of the SPF. If these data are collected less frequently SAMHSA/CSAP’s ability to promptly identify and respond to inappropriate strategies and activities with corrective action and to meet grantees’ training and technical assistance needs in a timely fashion will be negatively affected. Delays in these responses will, in turn, have an impact on grantees’ subsequent SPF steps, causing a cascading negative effect on overall program effectiveness.
Another reason for quarterly reports on implementation activities is that some of the data are used to meet national data collection needs, especially on HIV/AIDS. For example, information on the numbers of individuals tested for HIV and those with positive test results are typically updated frequently to maintain as close to real-time data as possible.
The client level questionnaires
This information collection fully complies with 5 CFR 1320.5(d) (2).
The notice required in 5 CFR 1320.8(d) was published in the Federal Register on August 13, 2021 (86 FR 44737).
In the development of the original instruments, CSAP consulted with experts on substance use disorder , viral hepatitis (VH), and HIV/AIDS within HHS, as well as other Federal agencies with related programs or mandates, including NIDA, ACF, CDC, DOJ, OJJDP, HUD, and the DOE. Consultations resulted in the refinement of the instruments based on current Federal data reporting needs.
No payment is received by respondents.
SAMHSA has statutory authority to collect data under the Government Performance and Results Act (Public Law 1103(a), Title 31) and is subject to the Privacy Act for the protection of these data. Only aggregate data will be collected with the Quarterly Progress Report and client level instruments, hence protecting the privacy and confidentiality of program clients and participants.
The information from grantees and all other potential respondents will be kept private and secure through all points in the data collection and reporting process. All data will be closely safeguarded, and no institutional or individual identifiers will be used in reports. It is critical to note that only aggregate data on HIV positivity will be reported to SAMHSA (e.g., 100 tests were done and 3 were positive giving 3% HIV positivity). SAMHSA and its contractors will not receive identifiable client records. Provider-level information will be aggregated to, at the least, the level of the grant/cooperative agreement-funding announcement.
Any data of a sensitive nature about individual clients will be aggregated to the program level (e.g., total numbers served, reached, tested, tested for the first time, and tested positive) before being reported.
As mentioned earlier, the Quarterly Progress Report is a modular instrument that will be updated quarterly as needed. Grantees will only update the modules corresponding to the SPF steps that they actively worked on during any given quarter. The cultural competence section will be completed every other quarter. The client level questionnaires will be answered as they are completed by clients.
Grantees included in MAI data collection include the following:
CBI: 2017: 12 grantees
CBI: 2018: 36 grantees
Prevention Navigators 2017: 20 grantees
Prevention Navigators 2019: 6 grantees
Prevention Navigators 2020: 84 grantees
Prevention Navigators 2021: 37 grantees
Exhibit 1: Total Estimated Annualized Burden by Instrument.
Type of respondent activity |
Number of Respondents |
Responses per Respondent |
Total Responses |
Hours per Response |
Total Burden Hours |
Wage Rate3 |
Total Hour Cost |
Quarterly Progress Report |
197 |
4 |
788 |
4 |
3,152 |
$24.78 |
$78,107 |
Adult questionnaire |
10,000 |
2 |
20,000 |
.20 |
4,000 |
$24.78 |
$99,120 |
Youth questionnaire |
2,500 |
2 |
5,000 |
.20 |
1000 |
$24.78 |
$24,780 |
Total |
12,697 |
|
25,788 |
|
8,152 |
|
$200,007 |
There will be no capital, start up, or operation and maintenance costs.
The annualized cost is approximately $335,651. This includes approximately $162,900 per year necessary to monitor and approve grantee reporting in these instruments (15% time of 10 Project Officers at $108,600 annual salary).
The burden level requested is slightly lower than the previously approved burden hour of 30,620 due to a more accurate understanding of actual response rate of adult and youth questionnaire respondents.
Analysis Plans
As previously noted, the main purpose of the monitoring data is to provide SAMHSA with timely information about the progress of the MAI grantees through the SPF steps, to identify and promptly respond to training and technical assistance needs of the grantees, and to recommend corrective action in a timely fashion in cases where grantees’ activities do not comply with the SPF. Some of the data fields in the Quarterly Progress Report and adult and youth questionnaires are also used to meet SAMHSA’s reporting requirements. In the rest of this section, planned analysis methods for each of these functions are discussed separately.
The utilization of the data for grant management and monitoring purposes involves narrative and qualitative reviews of the information rather than quantitative analyses. GPOs qualitatively compare the quarterly performance of each grantee to their performance in previous quarters and to the expected progress trajectory suggested by the SPF model. GPOs and Grants Management Specialist provide technical assistance to grantees who experience both programmatic and fiscal challenges.
The proposed analysis utilizing the Quarterly Progress Report and Participant Level Instrument data includes several distinct steps:
Descriptive analysis of grantee targets, organizational structure, training and technical assistance activities, and implemented interventions will be conducted and the results presented separately by the Funding Opportunity Announcement (FOA) to which the grantees responded.
Pooled analyses of participant-level outcomes will be conducted to assess overall program effects and their sustainability. Program effects will be evaluated through paired comparisons of baseline and exit values. Sustainability of effects will be evaluated through paired comparisons of baseline and follow-up values. Past analyses have suggested that some measures continue to improve after program exit. To continue to assess this post-exit improvement, paired comparisons between exit and follow-up values will also be conducted.
Site-specific data obtained from the Quarterly Progress Report, such as types and combinations of interventions implemented, fidelity of implementation, and grantee organizational characteristics, will be introduced into the multilevel models to investigate the sensitivity of effectiveness models to differences in intervention characteristics, fidelity of implementation, and grantees’ organizational characteristics. The planned multilevel multivariate models will also test hypotheses about interactions between individual and site-specific factors in determining participant outcomes.
Analysis Techniques and Statistical Test Determination
In assessing overall participant- and community-level improvement based on pooled data, paired comparison tests appropriate to the level of measurement will be employed. For dichotomous outcome measures, significance will be tested using McNemar’s test (Lidell, 1976; Yang, Sun, & Hardin, 2010). For normally-distributed continuous outcome measures, matched-pairs t-tests will be used to assess significance. The significance of change in ordinal or skewed continuous measures will be tested using the Wilcoxon signed-rank test (Wilcoxon, 1945; Blair & Higgins, 1980).
Reporting and Dissemination Plan
Data will primarily be used for GPO monitoring and assessment of grantees’ progress through the SPF process. Some data will also be reported for GPRA purposes toward the Congressional Justification. Finally, some data may be used for presentations by SAMHSA staff to other stakeholders or for reports to senior leadership.
Timeline
MAI grants are funded for up to five years. Typically, there is an initial period devoted to Steps 1, 2 and 3 of the SPF, namely conducting needs assessment, capacity building, and strategic planning. Grantees begin implementation and data collection only after their proposed strategic prevention plans are approved by their SAMHSA project officers.
The expiration date will be displayed.
No exceptions are required.
1 HIV National Strategic Plan (2021-2025). https://www.hiv.gov/federal-response/hiv-national-strategic-plan/hiv-plan-2021-2025
2 Substance Abuse and Mental Health Services Administration: A Guide to SAMHSA's Strategic Prevention
Framework. Rockville, MD: Center for Substance Abuse Prevention. Substance Abuse and Mental Health
Services
Administration,
2019.
https://www.samhsa.gov/sites/default/files/20190620-samhsa-strategic-prevention-framework-guide.pdf
3 The hourly wage estimate is based on the Occupational Employment and Wages, Mean Hourly Wage Rate for 21-1011 Substance Abuse and Behavioral Disorder Counselors = $24.78/hr. as of May 11, 2021. (http://www.bls.gov/oes/current/oes211011.htm. Accessed on May 11, 2021.)
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | HIV OMB Supporting Statement |
| Author | proddy |
| File Modified | 0000-00-00 |
| File Created | 2021-10-26 |