National Network of Sexually Transmitted Disease Clinical Prevention Training Centers (NNPTC): Evaluation
OMB No. 0920-0995
SUPPORTING STATEMENT B
October 28, 2016
Contact
Jami Fraze Risley
Health Scientist
DSTDP/PDQIB/OID
Centers for Disease Control and Prevention
1600
Clifton Road, NE, MS-E-02
Atlanta, GA 30333
Phone: 404-639-1826
Fax: 404-639-5210
Table of Contents
B. Collections of Information Employing Statistical Methods.
Respondent Universe and Sampling Methods
Procedures for Collection of Information
Methods to Maximize Response Rates and Deal with No Response
Tests of Procedures or Methods to be Undertaken
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Section B. Collections of Information Employing Statistical Methods.
This collection of information is used for program management and evaluation.
1. Respondent Universe and Sampling Methods
The respondent universe consists of health care clinicians, social service providers, administrators, researchers, and students, working in private health care practices, publicly-funded health care facilities, hospitals, universities, medical centers, state and federal agencies, and state and local health departments, who apply for training to learn current STD prevention, diagnostic, and treatment practices offered by the National Network of Sexually Transmitted Clinical Prevention Training Centers (NNPTC). The CDC-funded NNPTC offer classroom and experiential training, web-based training, and clinical consultation, to maintain and enhance the capacity of health care providers to prevent and treat STDs.
All persons who voluntarily register to participate in an NNPTC training or technical assistance event are included in the registration sample. Recruitment for training by PTCs targets healthcare organizations/clinics, public health organizations, and professional organizations and takes the form of personal communications, email notifications, flyers, and collaborative planning of trainings.
All persons who complete an NNPTC training or TA event are given the opportunity to complete an immediate post-course and 3 month long-term evaluation if they have provided a working email address upon registration as requested.
2. Procedures for the Collection of Information.
Respondents complete the NNPTC Abbreviated Health Professional Application for Training (NNPTC HPAT) (Att 3 & 4) online the first time the respondent registers for training. NNPTC HPAT data are entered into a centralized electronic database to facilitate generating certificates of attendance or continuing education credits, to allow participants to register for additional courses without re-entering their registration information, and to aggregate participant demographic data, which are transmitted to the CDC and used for evaluation analysis. No identifying information is sent to the CDC.
Respondents are invited by email (Att 33-36) to complete an NNPTC Evaluation Instrument (Att 5-32) within days after completion of the training and again approximately 90 days post training.
Almost all NNPTC HPAT and evaluation instruments are completed online through a centralized learning management system (LMS) managed and operated by the NNPTC (see screenshot attachments). Participants may complete instruments online using a computer, tablet, or smartphone. In rare instances (less than 5% of courses), instruments are completed on paper at the site of the training and data are entered into the electronic system by PTC staff.
3. Methods to Maximize Response Rates and Deal with Nonresponse
The ease of completing the 3 minute NNPTC HPAT online via computer, smart phone, or tablet reduces participant burden and access barriers, thereby maximizing response rates. The NNPTC HPAT is a prerequisite for obtaining training and continuing education credit. Therefore, registration information from nearly all trainees is expected. Similarly, the ease of using the online survey system and continuing education requirement to complete a post-course evaluation encourages a high rate of return of post-course evaluation information from trainees (100% registration response rate expected and 80% post-course response rate expected).
The large number of national trainees invited to complete an evaluation enables a robust analysis with a 60% - 80% response rate. Trainer encouragement and the requirement to complete the evaluation in order to receive continuing education credit should result in >60% post-course response rate, which is higher than reported in the literature for email surveys.
To increase response rates, and address initial non-response, the LMS automatically sends up to three emails within a 10 day period after training to each trainee and again 90 days after training. Emails invite completion of the appropriate NNPTC Evaluation Instrument by clicking on an embedded link. The emails are sent at approximate three-workday intervals, unless an evaluation response has already been received by the LMS. This timing has been suggested by the literature to encourage the highest response rates. The email invitations and online format of the NNPTC Evaluation Instruments contribute to the ease of response and reduce burden. Emails are customized to include the trainee’s first name to increase the likelihood they are opened.
If response rates fall below 80% as is seen with most online surveys, a chi square analysis will compare professions, direct care of at-risk populations, and worksite county STD morbidity rates for responders and non-responders. Chi square analysis tells us whether those who responded to the evaluation surveys are different from those who did not.
4. Tests of Procedures or Methods to be Undertaken
No part of this project entails an experimental design; rather, the design is descriptive in nature.
The NNPTC Evaluation Instruments were developed in consultation with the CDC and NNPTC and the various forms were reviewed for clarity and time to complete by 108 healthcare professionals and 22 PTC staff. No data were collected from the reviewers, only comments. Feedback from reviewers was used to refine questions as needed, ensure accurate programming and skip patterns, and establish estimated time required to complete the NNPTC Evaluation Instruments.
Quantitative data from the NNPTC HPAT will be analyzed using the Statistical Packages for the Social Sciences (SPSS) software program to calculate descriptive statistics (numbers and percentages) of responses to questions about profession, work setting, functional role, primary focus of work program, types of patients and types of patients served. Addresses and zip codes will be used to determine whether trainees come from a county with STD rates greater than or less than the national average. These descriptive statistics plus morbidity rates in the county where the trainee works will be used to determine whether the NNPTC is training the appropriate kinds of healthcare professionals from the places with the greatest STD burden.
Quantitative data from the NNPTC Evaluation Instruments will be analyzed using the Statistical Packages for the Social Sciences (SPSS) software program to calculate number and proportion of responses to evaluation questions for the post training instruments as well as the 90-day evaluation instruments using frequencies, cross-tabulations, comparisons of confidence levels using Wilcoxon Signed Rank and Student t-tests, and chi square analyses in the aggregate and by type of training course and provider categories. Regression analysis may also be used. This means we will look to see whether trainees were satisfied with the training, whether they were more confident to demonstrate skills and knowledge learned in the training after the training as compared to before the training, whether they intend to change their clinical practices (that is, implement the STD Treatment Guideline recommended practices), and after 90 days whether they actually did change their clinical practices.
Qualitative data (from open-ended questions) will be analyzed by theme. At least two reviewers will agree on the assignment of a thematic code to comments so that we can identify, for example, how many trainees experienced audio difficulties on webinars, or how many said they learned how to conduct a sexual history better.
The data from the NNPTC HPAT and the NNPTC Evaluation Instruments can be linked automatically through the learning management system computer for comparative analyses by provider characteristics or type of course. As a result we can determine whether trainees with little previous STD care experience change their practices following training more or less than trainees whose work focuses on treating STDs.
Results from all data collection instruments will be presented in graphic, written and verbal forms with quarterly and annual written reports distributed to the CDC and regional PTCs, and an annual power point presentation presented to the CDC and regional PTCS. . Data from all trainees will be grouped together with no individual data reported. Reports will include successes, challenges and recommendations. If the results of this evaluation are of interest to others outside the CDC and NNPTC, results may be shared with other training organizations, government agencies, and/or healthcare professionals
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Agency personnel responsible for receiving data and approving contract deliverables:
Jami Risely, PhD
Centers for Disease Control and Prevention, DSTDP/PDQIB/OID
Phone: 404-718-8525, Fax: 404-639-0944, Email: [email protected]
Contractors responsible for instrument development, data cleaning and analysis:
Sharon Devine, PhD
University of Colorado Denver
Phone: 303- 724-5955, Email: [email protected]
Susan Dreisbach, PhD
University of Colorado Denver
Phone: 303-721-1817, Email: [email protected]
Christopher Voegeli, PhD
Denver Public Health
Phone: 303-602-3613, Email: [email protected]
| File Type | application/msword |
| File Title | SUPPORTING STATEMENT |
| Author | jug1 |
| Last Modified By | Bonds, Constance (CDC/OID/NCHHSTP) |
| File Modified | 2016-10-28 |
| File Created | 2016-10-28 |