Capacity Building Assistance Program: Data Management, Monitoring, and Evaluation
OMB No. 0920-1322
Supporting Statement – Section B
July 8, 2024
Program Official/Project Officer
Contact:
Program Official/Project Officer
Name: Karen E. Kun
Title: Behavioral Scientist/Evaluator
CIO: Centers for Disease Control and Prevention, National Center for HIV, Viral Hepatitis, STD, and TB Prevention
Division: Division of HIV Prevention
Branch: Translation and Evaluation Branch
Address: 1600 Clifton Road, MS H24-5
Phone: 404-639-2639
Email: [email protected]
The respondent universe consists of recipients of capacity-building assistance (CBA) services (i.e., training and technical assistance). These are health professionals from community-based organizations (CBOs) and health departments involved in HIV-prevention service delivery and are funded directly or indirectly by the CDC. These health professionals provide essential HIV-prevention services in communities and local areas where HIV is most heavily concentrated to achieve the greatest impact in reducing new HIV infections; increasing access to care for people with HIV; improving outcomes and reducing mortality for people living with HIV; and reducing HIV-related disparities. Their positions include HIV educator; clinical supervisor; HIV-prevention specialist; clinician; outreach worker; case manager director; program coordinator; program manager; disease intervention specialist; partner services provider; physicians; nurses; and health educators. The CDC-funded CBA providers offer classroom, web-based, and blended trainings, as well as one-on-one technical assistance (TA) to build and maintain the capacity of health professionals and their organizations to control and prevent HIV. Since April 1, 2019, TA and training has been provided as requested to health professionals within CBOs and health departments, most of whom are funded directly or indirectly by the CDC.
For the Learning Group Registration, Post-Training Evaluation, and Post-TA Evaluation, all training and/or TA recipients will be invited to participate in the information collection. To minimize impact of the data collection on training and TA recipients, non-responders for Post-Training Evaluation and Post-TA Evaluation do not receive follow-up in the form of emails or phone calls.
Program managers of training and TA recipients are the only respondents for the Training and TA Follow-up Survey. Specifically, the Training and TA Follow-up Survey is sent to program managers of CDC-funded organizations. Program managers will be identified by DHP’s Program Development and Implementation Branch (PDIB). PDIB will provide telephone numbers and email addresses for one program manager at each of the 189 CDC-funded programs. The information provided by PDIB is stored within CTS.
We anticipate collecting Learning Group Registration, Post-Training Evaluation, Post-TA Evaluation, and Training and TA Follow-up Survey data. Approximately 3,800 people will complete the Learning Group Registration. Similarly, approximately 3,800 people will complete the Post-Training Evaluation. It is estimated that approximately 3,650 people will complete the Post-TA Evaluation. Each of the estimates are annual estimates. For the Training and TA Follow-up Survey, it is estimated that 189 program managers will complete the survey every six months during a 3-year period. The estimates are based on previous years’ data for the number of people provided with CBA services, and the current number of program managers for CDC directly-funded programs.
The respondents for the Learning Group Registration and Post-Training Evaluation are identified and tracked within CDC TRAIN. Training completion data is transferred to, and stored within, the CBA Tracking System (CTS). CTS uses the CDC TRAIN data to electronically send the Post-Training Evaluation to training recipients who have completed their training(s). Post-Training Evaluation responses are stored within CTS.
The respondents for the Post-TA Evaluation are identified and tracked within CTS. CTS is the online portal used to request and manage TA delivery. Using internal system data, CTS will electronically send the Post-Training Evaluation to TA recipients whose TA request is marked in the system as complete. Post-TA Evaluation responses are stored within CTS.
The respondents for the Training and TA Follow-up Survey are identified and tracked within CTS. Using data within the CDC TRAIN and CTS datasets, CTS will identify CDC-funded organizations whose staff completed one of more training or TA event. Every 6 months, CTS will electronically send the Training and TA Follow-up Survey to program managers of those CDC-funded organizations. If program managers do not complete the Training and TA Follow-up Survey within two weeks of the email invitation, they will be added to a non-responder list in CTS. A contractor will use the non-responder list to conduct telephone and email follow-up (i.e., 8 contact attempts in 5 weeks) to try to convert non-responders to responders. The eight contact attempts will consist of a combination of four telephone calls and four email reminders (i.e., phone, phone, email, phone, email, phone, email, email). Training and TA Follow-up Survey responses will be stored within CTS.
We anticipate an 80% response rate for the instruments. The Learning Group Registration is requested when training recipients request registration or access to a CBA-related courses. Participation in the Post-Training Evaluation (PTE) will be encouraged by providing a survey invitation before issuing a training certificate of completion. Therefore, we anticipate the response rate for this instrument will be 80% or higher. For the Post-TA Evaluation, TA providers will be provided a short script to inform TA recipients that they will receive a survey and to emphasize the value of their input. We anticipate an 80% response rate for the Post-TA Evaluation. To achieve an 80% response rate with the Training and TA Follow-up Survey, we will follow-up with reminder emails and telephone calls with non-responders. We will not be following-up via telephone or email for the Post-Training Evaluation or the Post-TA Evaluation due to efforts to minimize burden.
The Post-Training Evaluation, Post-TA Evaluation, and the Training and TA Follow-up Survey will allow CDC to assess short- and intermediate-term outcomes for training and TA in the areas of clinical HIV testing and prevention for people with HIV; non-clinical HIV testing and prevention for HIV-negative people; integrated HIV activities; and structural interventions.
The collection of information consists of four instruments administered to the recipients of CBA services (i.e., training and TA) and select program managers. A summary description of each instrument is provided below:
When an individual requests training, they complete the following data collection instrument:
Learning Group Registration
Who responds: Training recipients
Where will the instrument be hosted: Web-based; CDC TRAIN1
When will recipients respond: Upon initial learning group registration
Will there be follow-up via email or phone: No
Instrument Description: The Learning Group Registration Form collects demographic information about training recipients' including: 1) business contact information (e.g., email and telephone number); 2) primary [employment] functional role; 3) employment setting; and 4) programmatic and population areas of focus. The CDC TRAIN system will store the information for use during future training registrations. The demographic information can be updated by the training recipient as needed.
Attachments:
Attachment
3: Learning Group Registration-Word version
Attachment 4:
Learning Group Registration-Screenshots
After an online or in-person training event is completed, training recipients complete the following data collection instrument:
Post-Training Evaluation (PTE)
Who responds: Training recipients
Where will the instrument be hosted: Web-based; CBA Tracking System2
When will recipients respond: Immediately following the completion of a training event
Will there be follow-up via email or phone: No
Instrument Description: The PTE is designed to collect information about training recipients’ satisfaction with the training delivery method and course content, their organization’s implementation status, and additional training or TA needs.
Attachments:
Attachment
5: Post-Training Evaluation Introductory Email
Attachment 6:
Post-Training Evaluation-Word version
Attachment 7:
Post-Training Evaluation-Screenshots
When a CBA provider indicates completion of a TA event in the CBA Tracking System, a link to the following instrument is emailed to TA recipients:
Post-Technical Assistance Evaluation (PTAE)
Who responds: TA recipients
Where will the instrument be hosted: Web-based; CBA Tracking System
When will recipients respond: Immediately following the completion of a TA event
Will there be follow-up via email or phone: No
Instrument Description: The purpose of this instrument is to obtain feedback on TA-event delivery from CBA recipients. It assesses TA recipients’ satisfaction with CBA providers (e.g., provider’s knowledge, provider’s responsiveness, provider’s communication), suggestions for improvement of TA services, barriers to utilizing the TA received, and additional TA needs.
Attachments:
Attachment
8: Post-TA Evaluation Introductory Email
Attachment 9: Post-TA
Evaluation-Word version
Attachment 10: Post-TA
Evaluation-Screenshots.
To follow-up on training implementation and TA utilization, a link to the following data collection instrument is emailed:
Training and TA Follow-up Survey (TTAFS)
Who responds: Training and TA recipients' program managers
Where will the instrument be hosted: Web-based; CBA Tracking System
When will recipients respond: Every 6 months for Training and TA administered in that 6-month period
Will there be follow-up via email or phone: Yes, via phone and email
Procedure for follow-up: 7 business days after the invitation email for the Training and TA Follow-up Survey is sent to program managers, a reminder to complete the survey will be emailed to respondents who have not completed the online instruments. Seven business days after the reminder emails are sent, a government contractor will contact the non-responders by telephone to provide another survey reminder and offer a computer assisted telephone interview (CATI)to complete the survey. The contractor will use a telephone script to complete the telephone version of the survey. Given the typically low response rate to online survey instruments, this telephone follow-up strategy increases the responses to the survey.
Instrument Description: The TTAFS seeks feedback about how the training and/or TA that was provided impacted their organization. Respondents are asked to provide information about the barriers to implementing an intervention and about additional TA that is needed to facilitate implementation of interventions and/or public health strategies.
Sampling strategy: All program managers that have staff who attended a training or TA event will receive a TTAFS.
Attachments:
Attachment
11: Training and TA Follow-up Survey Introductory Email
Attachment
12: Training and TA Follow-up Survey-Word version
Attachment 13:
Training and TA Follow-up Survey-Screenshots
Attachment 14:
Training and TA Follow-up Survey Reminder Email
Attachment 15:
Training and TA Follow-up Survey Telephone Script for non-responders.
Although participation in the data collection process is voluntary, every effort will be made to maximize the rate of response to the data collection. To maximize response rates for the Post-Training Evaluation and Post-TA Evaluation, the survey has a low number of questions/items that are measured and the response options are mostly multiple choice, Likert scale, or a similar design that does not rely on open text boxes that collect qualitative data. To maximize response rates for the Training and TA Follow-up Survey, email reminders will be sent to program managers who do not respond to the initial email invitation within one week. One week after the reminder email, a contractor will follow-up with program managers by telephone to provide a second reminder and to offer a CATI. This strategy significantly raises the response rate to the web-based data collection.
The data collection tools were reviewed by public health and evaluation experts in CDC DHP’s Capacity Building Branch, as well as by an evaluation contractor to ensure that content and readability are appropriate. The estimate of burden hours is based on a pilot test of the information collection instruments by seven public health professionals for the Post-Training and Post-TA Evaluations and six public health professionals for the Training and TA Follow-up Survey. In the pilot test, the average time to complete the instruments, including time for reviewing instructions, gathering needed information, and completing the instrument, was approximately 5 minutes for the Post-Training Evaluation, 5 minutes for the Post-TA Evaluation and 18 minutes for the Training and TA Follow-up Survey.
The data collection was designed by CDC’s DHP Translation and Evaluation Branch. CDC TRAIN is responsible for maintaining the online training registration system. SeKON/Maximus is responsible for CTS capabilities to administer surveys, collect survey responses, and store data. Copial Business Strategists supports evaluation design and execution, leads the development and implementation of data management protocols and processes; conducts telephone interviews; and provides data reporting. CDC performs administrative oversight, provides technical guidance, ensures scientific integrity, and provides programmatic expertise.
Point of Contact:
Karen Kun
Behavioral Scientist/Evaluator
Centers for Disease Control and Prevention
National Center for HIV, Viral Hepatitis, STD and TB Prevention
Division of HIV Prevention, Translation and Evaluation Branch
Phone: 404-639-2639
Email: [email protected]
1 CDC TRAIN (https://www.train.org/cdctrain/welcome), which is a password-protected online portal that allows users to access over 1,000 courses developed by CDC programs, grantees, and funded partners.
2 The Post-Training Evaluation, Post-TA Evaluation, and Training and TA Follow-up Survey will be hosted within the CBA Tracking System application. This application was developed by DHP and is the portal currently used by CBA recipients to request training and TA. The system will send the email invitation to complete the Post-Training Evaluation, Post-TA Evaluation, and Training and TA Follow-up Survey, as applicable.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | MPhields |
| File Modified | 0000-00-00 |
| File Created | 2024-09-06 |