National Network of Sexually Transmitted Disease Clinical Prevention Training Centers (NNPTC): Evaluation
OMB No. 0920-0995
Expiration Date: 05/31/2020
March 25, 2020
Contact:
Project Officer
Name: Alexandra Coor, MPH
Title: Health Scientist
CIO:
Centers for Disease Control and Prevention,
National Center for
HIV, Viral Hepatitis, STD, and TB (NCHHSTP)
Division: Division of STD Prevention, PDQIB
Address: 12 Corporate Square, US12-2
Phone: 404-639-6423
Fax: 404-639-8622
Email: [email protected]
Table of Contents
Section
A. Justification
Circumstances Making the Collection of Information Necessary
Purpose and Use of the Information Collection
Use of Improved Information Technology and Burden Reduction
Efforts to Identify Duplication and Use of Similar Information
Impact on Small Businesses or Other Small Entities
Consequences of Collecting the Information Less Frequently
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
Explanation of Any Payment or Gift to Respondents
Protection of the Privacy and Confidentiality of Information Provided by Respondents
Institutional Review Board (IRB) and Justification for Sensitive Questions
Estimates of Annualized Burden Hours and Costs
Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers
Annualized Cost to the Government
Explanation for Program Changes or Adjustments
Plans for Tabulation and Publication and Project Time Schedule
Reason(s) Display of OMB Expiration Date is Inappropriate
Exceptions to Certification for Paperwork Reduction Act Submissions
Tables
Table 12A Estimated Annual Burden Hours
Table 12B Estimated Annual Cost Burden to Respondents
Table 14A Estimated Annualized Cost to the Federal Government
Table 16A Project Time Schedule
List of Attachments
1: Authorizing Legislation
2: 60-Day FRN
3: NNPTC Abbreviated Health Professional Application for Training (NNPTC HPAT)-Word version
3A: Previously approved HPAT
3B: Table of Revisions to ICR.
4: NNPTC Abbreviated Health Professional Application for Training (NNPTC HPAT) - Screenshot
5: Intensive Complete Post-Course Evaluation -Word version
6: Intensive Complete Post-Course Evaluation -Screenshot
7: Intensive Complete Long-Term Evaluation -Word version
8: Intensive Complete Long-Term Evaluation -Screenshot
9: Intensive Didactic Post-Course Evaluation –Word version
10: Intensive Didactic Post-Course Evaluation –Screenshot
11: Intensive Didactic Long-Term Evaluation -Word version
12: Intensive Didactic Long-Term Evaluation -Screenshot
13: Practicum Post-Course Evaluation -Word version
14: Practicum Post-Course Evaluation -Screenshot
15: Practicum Long-Term Evaluation -Word version
16: Practicum Long-Term Evaluation -Screenshot
17: Wet Mount Post-Course Evaluation -Word version
18: Wet Mount Post-Course Evaluation -Screenshot
19: Wet Mount Long Term Evaluation -Word version
20: Wet Mount Long Term Evaluation -Screenshot
21: STD Treatment Guidelines Complete Post-Course Evaluation -Word version
22: STD Treatment Guidelines Complete Post-Course Evaluation -Screenshot
23: STD Treatment Guidelines Complete Long-Term Evaluation -Word version
24: STD Treatment Guidelines Complete Long-Term Evaluation -Screenshot
25: STD Treatment Guidelines Short Post-Course Evaluation -Word version
26: STD Treatment Guidelines Short Post-Course Evaluation -Screenshot
27: STD Treatment Guidelines Short Long-Term Evaluation -Word version
28: STD Treatment Guidelines Short Long-Term Evaluation -Screenshot
29: Basic Post-Course Evaluation -Word version
30: Basic Post-Course Evaluation -Screenshot
31: Basic Long-Term Evaluation -Word version
32: Basic Long-Term Evaluation -Screenshots
33: Immediate Post-Course email invitation - Word version
34: Immediate Post-Course email invitation - Screenshot
35: 3 Month Long-Term email invitation – Word version
36:
3 Month Long-Term email invitation – Screenshot
37:
Project Determination (IRB File)
38: Table of Attachments and Descriptions
|
Section A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
The Centers for Disease Control and Prevention (CDC), Division of STD Prevention (DSTDP) requests a 3-year renewal for to the previously OMB approved information collection request (ICR) entitled, “National Network of Sexually Transmitted Disease Clinical Prevention Training Centers (NNPTC): Evaluation”/ This Extension consists of minor changes to the Abbreviated Health Professional Application for Training HPAT form; now called the “NNPTC HPAT”.
Data will be collected from a total of 4,500 health professionals annually who provide STD screening, diagnosis, and treatment or provide services to populations at risk of STD and receive training or technical assistance delivered by the NNPTC. In addition to completing the NNPTC HPAT, these 4,500 healthcare professionals will receive an email (Att. 33 & 34) inviting them to complete an evaluation within days after training, may voluntarily complete that evaluation, will receive an email inviting them to complete a follow-up evaluation 3 months after training, and may voluntarily complete that long-term evaluation. As a result, we anticipate 11,769 total respondent instances.
This information collection is authorized by Section 301 of the Public Health Service Act (42 U.S. Code, Sec. 792[295k] (a)) (Att 1). This information collection falls under the essential public health service of 1) informing, educating, and empowering people about health issues; 2) mobilizing community partnerships to identify and solve health problems; 3) linking people to needed personal health services and assure the provision of health care when otherwise unavailable; 4) assuring a competent public health and personal health care workforce.
Despite widely available diagnostic tests and simple treatment for chlamydia, gonorrhea and syphilis, many at-risk individuals in the U.S. receive sub-optimal clinical care for these STDs. For example, even though evidenced-based guidelines have promoted annual chlamydia screening for women <25 years since 1993, in 2010 less than half of commercially insured women aged 21-24 years were screened for chlamydia. Congenital syphilis cases can occur when prenatal care providers fail to screen pregnant women for syphilis as recommended. Many HIV-infected MSM who access clinical care do not receive recommended rectal and pharyngeal STD screening. Some STD-infected patients do not receive the recommended therapy in a timely fashion or are lost to follow up, and for some, partner notification and treatment are lacking. These examples highlight the need for STD clinical training to improve the care that infected and at-risk individuals need.
With widespread availability of health insurance, STD care has shifted to primary care (PC) providers in some areas of the country, while STD clinics continue to provide the majority of STD care in others. Even for the previously insured, health insurance coverage will has extended to STD clinical preventive services. Since 2011, health plans are required to provide coverage without cost-sharing for recommended preventive services. These services include chlamydia screening for sexually active females aged 16-24; prenatal screening for syphilis, gonorrhea, chlamydia, and hepatitis B; behavioral counseling and syphilis, gonorrhea, chlamydia, and HIV screening for high risk individuals; and HPV and hepatitis B vaccines. Consequently, there has been an expanded pool of clinicians who need to provide STD clinical preventive services and who need additional training and resources to increase their knowledge, skill, and capacity to screen, diagnose, treat and counsel those with STDs.
Guidelines and training are necessary but insufficient to change an individual provider’s clinical practice. In 2001, the IOM committee on the Quality of Health Care in America released a report entitled “Crossing the Quality Chasm: A New Health System for the 21st Century.” The report documents the causes of the quality gap, identifies current practices that impede quality care, and explores how systems approaches can be used to implement change, for example by promoting evidence-based practice and strengthening clinical information systems.
2. Purpose and Use of the Information Collection
The purpose of this information collection is to monitor and evaluate the performance of the NNPTC grantees and the NNPTC program. The information will be collected from health professionals that can include physicians, nurses, nurse practitioners etc. that provide STD screening and treatment who register to attend NNPTC training and/or technical assistance sessions. Specifically, this information will be used to:
Determine whether NNPTC grantees are reaching their target audiences
Measure trainee satisfaction with NNPTC services
Measure the impact of training on health professionals’ implementation of recommended STD screening, diagnosis, and treatment practices in the short and long term
Identify factors that correlate with practice changes
Identify and address programmatic areas needing improvement
Provide timely, current, and accurate information in response to requests from Executive Branch officials, the Congress, constituents, or other federal, state, and local agencies concerning the needs of grantees and the types and quality of NNPTC services delivered.
DSTDP will disseminate the summarized information through reports to DSTDP and the NNPTC, and possibly publications or presentations. All data will be published in the aggregate. No individual health professional will be identified in publications.
The information collection system consists of fifteen instruments administered to the recipients of NNPTC training and technical assistance services. Health professionals wanting to attend NNPTC training will complete the NNPTC Abbreviated Health Professional Application for Training (NNPTC HPAT) (Att 3 & 4)) as part of registration for training and technical assistance services. This registration form is a revision of the formerly approved “Abbreviated Health Professional Application for Training (HPAT) - OMB No. 0920–0995, Expiration: 5/31/2020” (Att3A&3B), which has been modified to add the option to provide a telephone number and additional instructions or categories for selection to seven questions. The Abbreviated HPAT is administered online through an NNPTC Learning Management System 95% of the time (or by paper 5% of the time if the registration process needs to be administered at the training site).
After NNPTC training or technical assistance has been delivered, the Learning Management System automatically sends each registrant an email invitation to complete an on-line Post-Course Evaluation Instrument corresponding to the type of training attended or type of technical assistance received. The invitation contains a link to the appropriate online Post-Course Evaluation Instrument. Email invitations (Att 33 & 34) are sent after the end of the training session and again approximately 3 and 6 days after the first email to those who have not responded. Approximately 90 days after the training or technical assistance the NNPTC’s Learning Management System sends an email invitation with a link to an online Long Term Evaluation Instrument corresponding to the training or technical assistance attended (Att 35 & 36). This invitation is sent again approximately 3 and 6 days after the first invitation to those who have not completed the Long Term Evaluation Instrument.
In the 2015 development phase, all data collection tools were informally reviewed by 108 NNPTC healthcare professional trainees and 22 NNPTC staff. NO data were collected, only comments. Feedback from reviewers was used to refine questions as needed, ensure accurate programming and skip patterns and establish the estimated time required to complete the information collection instruments.
The information collection system consists of fifteen instruments administered to the recipients of NNPTC training and TA: 1) NNPTC HPAT (Att 3 & 4) and 2) 14 Post-Course and Long term Evaluation Instruments (Att 5-32). Only one set of post-course and long-term evaluation instruments is administered per training or TA event. The type of information collected with each instrument is described below.
The Abbreviated HPAT (Att 3 & 4) is a modified version of the previously approved OMB Control No. 0920-0995 Abbreviated HPAT. This data collection standardizes the training registration process across the NNPTC. The Abbreviated HPAT instrument collects information from trainees on their professions, functional roles, principal employment settings, location of their work settings, programmatic and population foci of their work, and characteristics of patients/clients served. The Abbreviated HPAT also collects the trainee’s workplace contact information, race, ethnicity, and gender. This data collection provides CDC with information to determine whether the NNPTC is reaching its target audiences in terms of provider type, the types of organizations in which participants work, the focus of their work and the population groups and geographic areas served. The Abbreviated HPAT is also used to monitor and evaluate performance of Prevention Training Center grantees funded by CDC/Division of STD Prevention (DSTDP) that offer STD prevention training and TA to health professionals.
The Evaluation Instruments (Att 5-32) consist of 80 questions: 37 closed-ended questions, 38 paired retrospective before-after practice pattern questions with questions on the follow-up evaluation, and 5 open-ended questions designed to elicit information from NNPTC clients about their satisfaction with training and trainers, changes in knowledge, skills, self-efficacy as a result of training, intention to implement at least one change in practice based on the training or TA, and barriers to change. Questions are of various types including multiple response, Likert scale, retrospective assessments of confidence to demonstrate knowledge and skills, and open ended. Effort was made to limit questions requiring narrative responses and include optional narrative questions for respondents to elaborate on their feedback if they choose to do so. There are seven Post-Course Evaluation Instruments that correspond to the content and length of training session and seven corresponding Long-Term Evaluation Instruments. Collection of data using the NNPTC HPAT and the Evaluation Instruments estimates the burden to a total of 502 hours (see Table 12A for the estimate of burden associated with each instrument).
Attachments 5 & 7: Intensive Complete Post-Course and Long-Term Evaluations
Attachments 9 & 11: Intensive Didactic Post Course and Long-Term Evaluations
Attachments 13 & 15: Practicum Post-Course and Long-Term Evaluations
Attachments 17 & 19: Wet Mount Post-Course and Long-Term Evaluations
Attachments 21 & 23: STD Treatment Guidelines Complete Post-Course and Long-Term Evaluations
Attachments 25 & 27: STD Treatment Guidelines Short Post-Course and Long-Term Evaluations
Attachments 29 & 31: Basic Post-Course and Long-Term Evaluations
3. Use of Improved Information Technology and Burden Reduction
Online versions of the survey collection tools will be used as much as possible to reduce the overall burden on respondents. All instruments may be completed comfortably by computer, tablet, or smartphone. Paper instruments may be used when access to the Internet is not available or advance registration is not possible. These information collection instruments were designed to collect the minimum information necessary for the purposes of this project and to collect data using the same questions for multiple types of courses so that data may be aggregated during analysis when appropriate.
4. Efforts to Identify Duplication and Use of Similar Information
The information being collected is specific to DSTDP’s NNPTC program. This information collection represents the DSTDP’s attempt to monitor and evaluate the NNPTC services and program. There is currently no information available that can substitute for the responses to the data collection instruments and provide essential program improvement information.
5. Impact on Small Businesses or Other Small Entities
This data collection will not involve small businesses or other small entities.
6. Consequences of Collecting the Information Less Frequently
The Attachment 3 will be completed one time by individuals at the time of registration for any given course. The post-course and long-term evaluations will collect information from each recipient of training or technical assistance from the NNPTC. It is necessary to collect registration information from all trainees in order to plan for class size and provide continuing education credit when applicable. Since completing the evaluation is voluntary, we estimate a 60% rate of completion for post-course evaluation information. Consequently, less frequent collection of registration information would impede preparation for training activities and hinder monitoring and evaluation of the program.
If information is not collected, there will be no systematically obtained information for the NNPTC to make timely and essential corrections, if needed, to better meet the needs of its trainees and achieve the objectives of the program. Specifically, not collecting this information would hinder DSTDP’s ability to:
• Monitor and evaluate the NNPTC grantee performance;
• Assess trainee satisfaction with services delivered by CDC-funded NNPTC;
• Identify and address programmatic areas needing improvement;
• Provide timely, current, and accurate information in response to requests from Executive Branch officials, the Congress, constituents, or other federal, state, and local agencies on the needs of grantees and the types and quality of NNPTC services delivered.
7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This request fully complies with the regulation 5 CFR 1320.5.
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
The 60-day federal register notice to solicit public comments was published in the Federal Register Vol. 84, No. 213, pg. 59381 / Monday November 4, 2019 (Att 2: 60-Day FRN). No public comments were received. No consultations outside CDC occurred.
9. Explanation of Any Payment or Gift to Respondents
CDC will not provide payments or gifts to respondents.
10. Protection of the Privacy and Confidentiality of Information Provided by Respondents
The CDC NCHHSTP Associate Director of Science Office reviewed this submission and determined that the Privacy Act does not apply to this ICR because they respondents are providing business contact information and not personally identifiable information. CDC does have an applicable System of Records Notice is 09-20-0161, “Records of Health Professionals in Disease Prevention and Control Training Programs,” last published in entirety in the Federal Register, Vol. 51, No. 226, November 24, 1986, pp. 42485-87 and last updated in 1994 that covers the collection of information requested for this collection. The NNPTC HPAT (Att 3) is the only instrument that collects categories of information in identifiable format from individual respondents such as: name, work mailing address, work phone numbers, work email address, work organization, race and ethnicity. These identifiable NNPTC HPAT data elements are needed to send information about the training or technical assistance for which they have registered, send electronic invitations to complete evaluation instruments, and identify county of workplace and type of employment setting to analyze reach.
Data on race and ethnicity are collected on the NNPTC HPAT (Att 3) because STDs disproportionally impact African Americans and Latino/as. As such it is important to make sure the NNPTC is training African American and Latino/a healthcare providers who work with these diverse racial and ethnic populations.
Analysis of reach and evaluation measures is done by the NNPTC’s National Evaluation Center. The NNPTC HPAT data transmitted to CDC will include no identifying information. A unique identifier will be generated for all data collection instruments to enable linking data from multiple data collection tools.
The identifiable information collected by the National Evaluation Center is stored behind firewalls at the NEC in password protected files and are available only to authorized users of the NEC. As noted above, no IIF is provided to the DSTDP.
There are several safeguards in place to handle de-identified data submitted to the CDC. Data will be stored and managed based on current CDC/OCISO (Office of the Chief Information Security Officer) requirements and standards. This includes protecting stored data within the CDC Internet Firewall. The data are stored and managed based on current CDC/OCISO requirements and standards which also includes the process for handling security incidents and the event monitoring and incident response. All administrative controls required by OCISO are validated through a “Certification and Authorization” (C&A) process as conducted by OCISO prior to moving any software application into “Production” on the CDC network.
Files are backed up daily and stored both onsite and offsite in accordance with CDC standards and OCISO guidelines. Contractors who operate and use the system are managed via the “CDC Information Management Services” (CIMS) contract which requires signed confidentiality agreements. All users’ access is “role based” and reflects a “need to know” policy established by CDC. Accountability is maintained with a user access log file which tracks users’ access to the system. Records will be retained and destroyed in accordance with the applicable CDC Records Control Schedule as mandated by OCISO.
(http://www.cdc.gov/about/leadership/leaders/seligman.htm” (http://aops-mas-iis.od.cdc.gov/Policy/Doc/policy449.htm)
11. Institutional Review Board (IRB) and Justification for Sensitive Questions
This project is an evaluation activity for the purpose of monitoring program activity and it does not involve human subjects research. The Funding Announcement PS14-1407 included a non-research determination, therefore, IRB review is not required (Att 37). The demographic data are needed to complete registration and conduct training assistance. None of the identifiable HPAT data are transmitted to CDC.
No sensitive information is being collected.
12. Estimates of Annualized Burden Hours and Costs
The estimate for burden hours is based on informal review of the information collection instruments by multiple and various types of health professionals (e.g. nurses, physicians, and nurse practitioner, etc. See Att. 3 box 1). Table 12A summarizes the average time to complete the instruments, including time for reviewing instructions, gathering needed information and completing the instrument. (see Att. 38) for a description of the instruments.
The estimated number of respondents is 4,500 health professionals. The estimated number of respondents is based on the annual average number of health professionals, trained by the NNPTC during one year. We estimate 4,500 health professionals will provide one response each per NNPTC HPAT (Att3) per year at an estimated 3 minutes per response for a total of 225 burden hours. Out of the 4,500 respondents, approximately 60% will complete the appropriate Post-Course Evaluation based upon the course they complete and approximately 30% of Post-Course respondents will complete the appropriate Long-Term Evaluation instrument for each episode of training or technical assistance in which they participate. The additional collection instruments total 191 burden hours. Respondents will also receive email notifications with link. Immediate Post-Course email invitation, Att 33 & 34 will be sent to the 4,500 course participants. Reading of the email is estimated at 1 minute per response for a total burden of 75 hours and the 3 Month Long-Term email invitation Att 35 & 36, will be sent to 660 respondents, with 1 minute response time; equaling 11 total burden hours. The total burden hours for the email are 86 burden hours.
The total annualized burden hours for the annual information collection is 502. Table 12A shows estimated time burden.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Table 12A: Estimated Annual Burden Hours
Estimates for the average hourly wage for respondents are based on the Bureau of Labor Statistics May 2018 mean estimate for healthcare practitioners and technical occupations of $39.42 (http://www.bls.gov/oes/current/oes290000.htm). The following table shows estimated time burden and cost information.
Table 12B: Estimated Annual Cost Burden to Respondents
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
There will be no direct costs to the respondents other than their time to participate in each information collection.
14. Annualized Cost to the Federal Government
There are no equipment or overhead costs. The only cost to the federal government would be the salary of CDC staff and contractors supporting the data collection activities and associated tasks.
Annually the estimated cost of the assessment is $ 102,783.50. The personnel cost of the CDC oversight of the project and contractors will be $2,783.50 for the Lead Behavioral Scientist. The cost of the NEC contractor (Denver Health & Hospital Authority) to provide assistance in the preparation of the OMB package, assessment design, instrument development, data collection, quality control, data analysis, and report preparation will be $100,000.
Table 14: Estimated Annualized Cost to the Federal Government
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15. Explanation for Program Changes or Adjustments
The previously approved HPAT will remain unchanged in terms of overall evaluation instruments and evaluation email notifications, with the total time burden for respondents remaining at 502 hours. The burden has not changed from the burden shown in the current inventory. This extension is necessary to monitor and evaluate the NNPTC program on a systematic national basis.
The only revision included in this request consist of refining wording of question instructions and answer options to seven questions on Att 3.
16. Plans for Tabulation and Publication and Project Time Schedule
Both quantitative and qualitative analyses will be performed. Prior to conducting any formal analyses on quantitative data, exploratory univariate and bivariate tests will be performed first to determine trends and patterns in the data. This will be accomplished using frequencies and cross-tabulations, and by examining univariate distributions and correlations. The frequency analysis will give various chi-squared tests for association for categorical ordinal or nominal data, while the ANOVA will provide F-tests for continuous data. We also expect to use various non-parametric tests on the Likert scale interval data.
Qualitative data include open-ended responses within the web-based information collection tools,. Open-ended responses in otherwise categorical questions within the information collection instruments (e.g., “Other, please specify”) will be abstracted and grouped by thematic categories, and analyses will be used to determine the frequency of categories. Other qualitative responses (e.g., how can this training be improved) will undergo content analysis.
Table 16: Project Time Schedule
|
|
|
|
|
|
17. Reason(s) Display of OMB Expiration Date is Inappropriate
The display of the OMB expiration date is not inappropriate.
18. Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | M Phields |
| File Modified | 0000-00-00 |
| File Created | 2022-02-24 |