Project Determination

Project Determination

Capacity Building Assistance Program Data Management, Monitoring, and Evaluation

Project ID:	0900f3eb8231f54a
Accession #:	NCHHSTP-IIT-2/29/24-1f54a
Project Contact:	Sherese Garrett
Organization:	NCHHSTP/DHP/TEB/IIT
Status:	Project In Progress
Intended Use:	Project Determination
Estimated Start Date:	03/01/24
Estimated Completion Date:	02/28/29
CDC/ATSDR HRPO/IRB Protocol#:
OMB Control#:	0920-1322

Description

Priority

Standard

Date Needed

03/15/24

Determination Start Date

02/29/24

Description

This is a non-research program evaluation project. A project determination request was previously submitted and approved as a program evaluation in October 2019, and this request is for continuation of the same project. This project is a systematic and ongoing assessment of HIV prevention capacity building assistance (CBA) services provided via the notice of funding opportunity PS19-1904 (see below for details) to health departments, community-based organizations, and other members of the HIV prevention workforce. CBA services such as training and technical assistance (TA) are currently offered to various health departments and community-based organization (CBOs) through the notice of funding opportunity PS19-1904: Capacity Building Assistance (CBA) for High Impact HIV Prevention Program Integration. PS19-1904 funds 17 organizations that form the national CBA Provider Network (CPN). This program supports the Ending the HIV Epidemic in the U.S. initiative by strengthening the capacity of the HIV prevention workforce so they can implement interventions and strategies to reduce HIV-related morbidity, mortality, and health disparities across the U.S. and its territories. This project collects, manages, analyzes, interprets, and reports the CBA program performance measures and outcomes. CDC is currently engaged in collecting data for this project. The data collection is approved by OMB (OMB Approval No. 0920-1322). The OMB approval for this data collection expires February 29, 2024.The extension/renewal request for this OMB approval has been submitted and is currently under the 60-day FRN review.

IMS/CIO/Epi-Aid/Lab-Aid/Chemical Exposure Submission

No

IMS Activation Name

Not selected

Select the primary priority of the project

Not selected

Select the secondary priority(s) of the project

Not selected

Select the task force associated with the response

Not selected

CIO Emergency Response Name

Not selected

Epi-Aid Name

Not selected

Lab-Aid Name

Not selected

Assessment of Chemical Exposure Name

Not selected

Goals/Purpose

The goals of this CBA program evaluation project are to: (1) Determine the effectiveness of the CBA program, (2) Gather feedback about areas for CBA program improvements, and (3) Identify new or emerging HIV prevention training and technical assistance needs of health departments and CBOs.

Objective

The program objectives do not relate to a public health emergency, or vaccines and immunization activities. The project objectives are to: (1) Establish and maintain a standard and systematic approach to collecting performance data for training and technical activities conducted under CDC-RFA-PS19-1904: Capacity Building Assistance (CBA) for High Impact HIV Prevention Program Integration and its succeeding NOFO. (2) Routinely create standard and ad hoc datasets to assist CDC staff with frequent program monitoring for the purpose of continuous quality improvement. (3) Periodically develop and disseminate reports to assist CDC staff with administrative and technical performance reporting. (4) Annually, produce scientific presentations and publications that inform external stakeholders of the short- and long-term outcomes of PS19-1904 and its succeeding NOFO’s program activities.

Does your project measure health disparities among populations/groups experiencing social, economic, geographic, and/or environmental disadvantages?

No

Does your project investigate underlying contributors to health inequities among populations/groups experiencing social, economic, geographic, and/or environmental disadvantages?

No

Does your project propose, implement, or evaluate an action to move towards eliminating health inequities?

No

Activities or Tasks

New Collection of Information, Data, or Biospecimens

Target Population to be Included/Represented

Other-HIV Prevention Workforce

Tags/Keywords

HIV; Primary Prevention; Capacity Building; training; technical assistance (TA)

CDC's Role

Activity originated and designed by CDC staff, or conducted at the specific request of CDC, or CDC staff will approve study design and data collection as a condition of any funding provided; CDC employees or agents will obtain data by intervening or interacting with participants; CDC employees will participate as co-authors in presentation(s) or publication(s); CDC employees will provide substantial technical assistance or oversight; CDC is providing funding

Method Categories

Survey

Methods

The Evaluation Design and Protocol (Att 17) outlines the data collection protocol for this project. This project primarily uses quantitative methods approach to collect and analyze existing data. Data collection for the CBA program evaluation consists of four instruments. There are two instruments associated with training activities (the Learning Group Registration, or LGR and the Post-Training Evaluation, or PTE, see Att 3 & Att 6), one instrument associated with TA events (the Post-TA Evaluation, see Att 9), and one instrument associated with follow-up for both training and TA (the Training and TA Follow-up Survey or TTAFS, see Att 12). We use a utilization-focused evaluation approach to yield regular data and reporting to inform decision making and identify areas for program improvements. The CBA Program Evaluation Data Management Plan (DMP) (Att 18) describes provisions for protection of privacy, confidentiality, and security of data. our overall approach allows for the identification and analysis of key elements that should be considered in understanding the process of training and TA, and the contextual factors influencing how that training and TA are applied. The HIV prevention workforce is the intended population. The HIV prevention workforce consists of positions such as (but are not limited to) HIV educator; clinical supervisor; HIV-prevention specialist; clinician; outreach worker; case manager director; program coordinator; program manager; disease intervention specialist; partner services provider; physicians; nurses; and health educators. For the LGR, PTE, and PTAE all training participants and TA recipients are invited to participate in the information collection. Program managers of training participants and TA recipients are the only respondents for the TTAFS. Specifically, the CBA Tracking System (CTS) sends the TTAFS to program managers within CDC-funded organizations. Program managers are identified by DHP’s Program Development and Implementation Branch (PDIB). PDIB provides telephone numbers and email addresses for one program manager within each CDC-funded program. The contact information is stored within the CBA Tracking Systems (CTS). CDC TRAIN users who are not members of the “HIV CBA” learning group within CDC TRAIN are excluded from receiving the LGR. CBA recipients who do not complete training or TA are excluded from receiving the PTE and PTAE. CBA recipients are also excluded from receiving the PTE if they have not completed an LGR. Program managers with zero staff participation in training and TA are excluded from receiving the TTAFS. Efforts were made to collect and use the minimum information needed to evaluate this project. The project maintains full, accurate, and transparent documentation of all data collected. Several standards are used to ensure the data collected are high-quality, useful, and accurate. Coding standards for each instrument are outlined in the project data codebooks and dictionaries. The codebooks and dictionaries are used to ensure all data have appropriate documentation that describes the method of collection, what the data represent, and potential limitations for use. Evaluators collaborate with subject matter experts and program operations staff to agree upon vocabulary that establishes standardized terms with consistent semantic definitions. Quality standards are checked and tracked through data quality reports. Each data quality report describes required data editing.

Collection of Info, Data, or Bio specimens

This project collects information using 4 surveys: 1) Learning Group Registration (LGR) (Att 3); 2) Post-Training Evaluation (PTE) (Att 6); 3) Post-TA Evaluation (PTAE) (Att 9); and 4) Training and Technical Assistance Follow-up Survey (TTAFS) (Att 12). CBA training participants can complete the LGR when enrolling in a CBA training offered in CDC TRAIN, a learning management system. The LGR collects training participants’ business contact information and employment setting. The respondents for the LGR and PTE are tracked within CDC TRAIN. Training completion data is transferred to, and stored within, the CBA Tracking Systems (CTS). CTS exists within the CDC network. CTS uses the CDC TRAIN data to electronically send the PTE to individuals who have completed their training(s). PTE responses are stored within CTS. The respondents for the PTAE are identified and tracked within CTS. CTS is the online portal used to request and manage TA requests. Using internal system data, CTS electronically sends the PTAE to TA recipients whose TA requests are marked in the system as complete. PTAE responses are stored within CTS. Non-responders for PTE and PTAE do not receive follow-up phone calls to complete the surveys. Using CDC TRAIN and CTS datasets stored within its database, CTS identifies CDC-funded organizations whose staff completed one of more training or TA event. Every 6 months, CTS electronically sends the TTAFS to program managers of those organizations. If program managers do not complete the TTAFS within two weeks of the email invitation, they are added to a non-responder list. A CDC project team member uses the non-responder list to conduct telephone and email follow-up. The follow-up consists of a combination of two telephone calls and two email reminders. TTAFS responses are stored within CTS. This project applies privacy standards in accordance with the Privacy Act of 1974 (5 U.S.C. § 552a) and Federal Regulation 45 CFR 46 “Protection of Human Subjects”, referred to as the 'Common Rule’. No sensitive information is collected. Respondents’ PII are stored in CTS. While performing data transfer and data processing activities, team members will not transfer or use project files containing PII outside of the CDC network environment. PII will be eliminated from datasets prior to analysis or dissemination. Aggregate data are used for all reports and publications. The project team will extract, manage, clean, and analyze all data within the CDC’s secure network to avoid security threats to the data or its release. All clean datasets or reports will remain within the CDC firewall throughout the project. To ensure and maintain the security of the data, the project team will comply with the HHS Information Technology General Rules of Behavior. Contracted staff within the project team will obtain and maintain access to the government’s intranet for the purpose of accessing and downloading program data from the secure password-protected online applications, including CTS and CDC TRAIN. The project has an estimated 319 total burden hours. Approximately 1,500 people will complete the LGR (Att 3) (125 total burden hours). Approximately 500 people will complete the PTE (Att 6) (84 total burden hours). Approximately 300 people will complete the PTAE (Att 9) (50 total burden hours). For the TTAFS (Att 12), it is estimated that 100 program managers will complete the survey every six months (60 total burden hours). Estimates are annual.

Expected Use of Findings/Results and their impact

The CBA program evaluation data provides critical feedback that can be used for continuous program quality improvement and the information also informs us about the unmet and changing implementation capacity building needs of CDC funded state and local health departments and CBOs. The CBA program evaluation findings/results are disseminated to internal and external collaborators in the forms of reports and presentations. Internal collaborators (e.g., branch leadership, program consultants, health scientists, and health educators) can use the program data for strategic planning, content development, and information integration. To disseminate findings beyond CDC, we present evaluation findings at national, peer-reviewed professional conferences and will prepare a manuscript for publication in a peer-reviewed journal. Materials that will be available to the public include conference presentations and peer-reviewed journal publications that contain aggregate data. Information sharing will adhere to OMB privacy standards by eliminating the use of information that can be used to identify an individual respondent. CDC will share with CBA Providers the data collected from the PTE (Att 6) and PTAEs (Att 9) via the CBA Tracking System (CTS) in the form of an automated, quarterly roll-up summary. The intent for this data sharing is to provide CBA Providers with timely feedback to inform changes to or improve their products or services. CBA Providers will only have access to their own organization’s roll-up summary. Specifically, the quarterly roll-up reporting summarizes the raw PTE (Att 6) and PTAE (Att 9) data while excluding potentially identifiable fields, such as name of survey respondent, their respective organization, and the associated CTS request number. Additionally, a minimum reporting threshold of more than 1 organization and 5 completed surveys must be met for data to be included in the roll-up summaries to further prevent identifiability and ensure de-identification of the source data. Upon request, raw PTE (Att 6) and PTAE (Att 9) data files can be provided, but only after potentially identified fields (name of survey respondent, their respective organization, and the associated CTS request) are removed, reviewed, and approved by the CTS Business Steward.

Could Individuals potentially be identified based on Information Collected?

Yes

Will PII be captured (including coded data)?

Yes

Does CDC have access to the Identifiers (including coded data)?

Yes

Is this project covered by an Assurance of Confidentiality?

No

Does this activity meet the criteria for a Certificate of Confidentiality (CoC)?

No

Is there a formal written agreement prohibiting the release of identifiers?

No

Funding
Funding Type	Funding Title	Funding #	Original Fiscal Year	# of Years of Award	Budget Amount
CDC Funding Intramural	Capacity Building Assistance Program Data Management, Monitoring, and Evaluation		2019

HSC Review

HSC Attributes

Program Evaluation

Yes

Other - This activity has been reviewed by NCHHSTP OADS and was determined to not meet the definition of research as defined in 46.102(l). The purpose of this activity is to (1) Determine the effectiveness of the CBA program, (2) Gather feedback about areas for CBA program improvements, and (3) Identify new or emerging HIV prevention training and technical assistance needs of health departments and CBOs. The activity is not designed to develop or contribute to generalizable knowledge. Major changes (amendments) should be submitted for re-review to ensure the changes do not affect this determination.

Yes

Regulation and Policy

Do you anticipate this project will need IRB review by the CDC IRB, NIOSH IRB, or through reliance on an external IRB?

No

Will you be working with an outside Organization or Institution? No

Institutions
Institution	FWA #	FWA Exp. Date	Funding	Funding Restriction Amount

Institution	Funding Restriction Percentage	Funding Restriction Reason	Funding Restriction has been lifted

Institution	Institution Role(s)	Institution Project Title	Institution Project Tracking #	Prime Institution

Institution	Regulatory Coverage	IRB Review Status

Institution	Registered IRB	IRB Registration Exp. Date	IRB Approval Status

Institution	IRB Approval Date	IRB Approval Exp. Date	Relying Institution IRB

Staff
Staff Member	SIQT Exp. Date	Citi Biomedical Exp. Date	Citi Social and Behavioral Exp. Date	Citi Good Clinical Exp. Date	Staff Role	Email	Phone #	Organization/ Institution
KarenKun	06/26/2026		08/23/2016		Co-Investigator	[email protected]	404-639-2639	INFORMATION INTEGRATION TEAM

DMP
Proposed Data Collection Start Date	03/01/24
Proposed Data Collection End Date	02/28/29
Proposed Public Access Level	Public
Public Access justification	CBA program evaluation data is available to the public. Internal and external requests for CBA program evaluation data may be submitted via email to the project’s general mailbox: [email protected]. Requestors will receive direct access to clean and custom training and TA participant-level data in a manner consistent with the security and confidentiality practices in place at the time of the request. Data will be shared after identifiers are suppressed. The release of the data will be conducted based on guidelines that maintain high standards of quality, protect individuals’ confidentiality and their right to privacy, and safeguard personal information. Data for public use will be de-identified before release, and cell sizes will be sufficiently large to prevent identification of individuals. The release of data for public use will not occur until data quality (i.e., test for completeness, validity, reliability, and reproducibility) is thoroughly scrutinized and evaluated. Public data requests must be approved by the TEB Information Integration Team Lead before being fulfilled. The Information Integration Team will make every effort to respond to data requests in a timely manner, but a specific turnaround time is not guaranteed.
How Access Will Be Provided for Data	CDC evaluation team members have ongoing access to the program evaluation data. All individuals with access are required to complete HHS and CDC mandatory annual trainings including the CDC Security Awareness Training, CDC Records Management, and the Scientific Integrity and Quality Training (SIQT). CDC evaluation team members also complete the Annual Assurance of Confidentiality Training for HIV Surveillance and Related Data, and the National HIV Prevention Program Monitoring and Evaluation (NHM&E) Data. Evaluators with access to electronic data files will read and sign a “Rules of Behavior” document as well as develop and use documentation for collecting and protecting data security. The technical controls for this evaluation are data accesses within CDC TRAIN and CTS are password protected functions that require registration and are only accessible by authorized CDC and contractor staff. CDC TRAIN and CTS contain data that can be downloaded via an Excel file format. The type and availability of data that can be accessed varies by user types. User type, and related permissions, are based on need-to-know. Roles are assigned by a federal system administrator. Datasets are stored on a secure CDC shared drive location with restricted access as they include indirect identifiers such as race/ethnicity, gender identity, and business contact information. No respondent names are retained in clean and custom datasets. The data stored on CDC computer systems and cloud environments are protected from accidental alteration and unauthorized access. No personal computers or personal electronic media will be used for data storage. Whether stored in an application or on the shared drive, the data is maintained in a location with restricted access. The administrative controls include that access is restricted to a limited number of users and is governed by CDC Privacy and Confidentiality policies and the Confidential Information Protection and guidelines. The physical controls include that the data system is protected with physical access controls, software and hardware firewalls, and user access authentication, including user ID and password. Based on their assigned role in CTS, CDC-funded CBA Providers will have limited access to only their PTE/PTAE (Att 6/Att 9) evaluation data via the CTS platform using a single sign on (SSO) validation. CBA Providers with a signed and approved CTS User Agreement, and CDC Assurances of Compliance for AIDS-related materials (CDC Form 0.1113 (E) Rev LVHG-DQXDU), will be able to view and download a roll-up summary of their PTE (Att 6) and/or PTAE (Att 9) data results. Any publication or use of the associated data will require review and clearance by CDC. The roll-up summaries have been remediated to exclude any potential for PII and to de-identify potential identifiable data. In their final form, they have been validated to contain no PII, research, or sensitive data. As a result, a CBA Provider’s roll-up summaries do not meet the threshold nor requirements to establish CDC Data Use Agreement, Assurances of Confidentiality 308(d), or Certificates of Confidentiality 301(d) to release their data to them. Data will be stored and managed based on current CDC/OCISO requirements and standards (Att 16_Security Controls). This includes protecting stored data within the CDC Internet Firewall and following the process for handling security incidents and the event monitoring and incident response.
Plans for archival and long-term preservation of the data	This project creates datasets for program evaluation purposes, and the datasets may be required for follow-up or reference for a moderate period of time. Records are retained according to the General Records Schedule, items 010 (DAA-GRS-2013-0005-0006), 011 (DAA-GRS-2013-0005-0007), and 020 (DAA-GRS2013-0005-0004) and may be destroyed 5 years after the project is terminated, but longer retention is authorized if required for business use. Archived data is available within the Translation and Evaluation Branch shared drive.

Spatiality (Geographic Location)
Country	State/Province	County/Region
United States

Determinations
Determination	Justification	Completed	Entered By & Role
HSC: Does NOT Require HRPO Review	Not Research / Other 45 CFR 46.102(l) Program Evaluation Other - This activity has been reviewed by NCHHSTP OADS and was determined to not meet the definition of research as defined in 46.102(l). The purpose of this activity is to (1) Determine the effectiveness of the CBA program, (2) Gather feedback about areas for CBA program improvements, and (3) Identify new or emerging HIV prevention training and technical assistance needs of health departments and CBOs. The activity is not designed to develop or contribute to generalizable knowledge. Major changes (amendments) should be submitted for re-review to ensure the changes do not affect this determination.	03/07/24	Dodson_Janella R. (jhd7) CIO HSC
PRA: PRA Applies		03/08/24	Bonds_Constance (akj8) CTR OMB/PRA Coordinator
ICRO: PRA Applies	OMB Approval date: 02/04/21 OMB Expiration date: 02/29/24	03/08/24	Zirger_Jeffrey (wtj5) ICRO Reviewer