Revision
A. JUSTIFICATION
This request is for approval from the Office of Management and Budget (OMB) to continue data collection activities with current and prospective health professions awardees of the Health Resources and Services Administration’s (HRSA) Bureau of Health Workforce (BHW). The current approval (OMB #0915-0061) expires on 06-30-2019 and covers data collection efforts through progress reports, as well as annual performance reports for grants and cooperative agreements (PRGCA).
BHW seeks approval from OMB to continue these efforts over the next three years. The supporting statement for this request has been updated and contains discussion about the utility of data collected during July 1, 2015 through June 30, 2018; lessons learned from data collection efforts during this time; as well as details regarding BHW’s proposed strategies for reducing the overall burden associated with its data collecting efforts (i.e. progress reports and the PRGCA).
HRSA is obligated to collect performance information on its grants and cooperative agreements as mandated in the Government Performance and Results Act of 1993 (GPRA)1 and the GPRA Modernization Act of 2010 (GPRAMA)2. The health professions grant programs are governed by the Public Health Service Act (42 U.S.C. 201 et seq.), specifically Titles III, VII, and VIII, which specify additional reporting requirements that are specific to certain programs.
For this revised information collection request, BHW has proposed to add a question on two of its forms indicating which Department of Health and Human Services priorities (HHS Priorities) were met by an activity. The first question appears on the IND-GEN form, column 26b, and asks if an individual trainee has received training on any priority area such as training for substance use disorder treatment. The second question appears on the EXP form, column 14, and asks if a site provides services in an HHS priority area such as substance use disorder treatment. See Appendix C for additional detail. Additionally, after having success in asking for National Provider Identifier (NPI) numbers for its Graduate Medical Education programs, BHW will expand collection of NPI numbers across other eligible professional training programs such as Advanced Nursing Education programs and Residency training programs in Dentistry where individual-level training data is collected and trainees are eligible to receive NPI numbers.
Copies of all performance measures and forms are included in this package. No program completes all forms or items and only sees forms and items specific to them when completing their PRGCA. All measures, forms, and program-specific manuals are also located on the HRSA website at https://bhw.hrsa.gov/grants/reportonyourgrant.
2. Purpose and Use of Information Collection
BHW is statutorily tasked with responding to issues specific to the training and supply of the current and future US healthcare workforce (see 42 USC 292 et seq). Currently, BHW funds over 40 different health professions training and loan programs that aim to increase the supply, diversity, and distribution of the current and future US healthcare workforce. Generally, these programs fall into three distinct categories3:
Infrastructure: refers to programs that are designed to enhance the scope and/or quality health professions training programs. These programs do not provide direct financial support to students; rather, awardees use funds in a variety of ways including enhancing curriculum and clinical training opportunities, as well as offer faculty development opportunities.
Direct Financial Support: refers to programs that are designed to provide students of health professions training programs with a financial award to cover costs associated with tuition and/or allowable living expenses. Depending on the nature of the program, awardees of these programs provide scholarships, stipends, or loans to students pursuing health profession-related training or degrees.
Multipurpose or Hybrid programs: refers to programs that, in accordance with their authorizing statute, may fund a variety of activities to include enhancing training infrastructure, providing direct financial support to health professions students, or support enhancements to clinical rotations and training.
In order to carry out its functions, BHW has historically collected data from funded awardees at two specific phases of a grant cycle:
Phase I: Mid-Year Progress Reports
Data collected in the form of progress reports serve as the official record of communication between government project officers and awardees and highlight awardees' successes and challenges in meeting the goals of each program. Information provided through progress reports are reviewed by government project officers in BHW and are used to determine progress toward implementing required grant activities; as well as technical assistance needs. In addition, information provided through progress reports also assists BHW in understanding fluctuations in program outcomes reported through the PRGCA.
This request seeks approval to continue collecting information through progress reports from BHW-funded awardees on an annual basis (Table 2). Submission of progress reports will not coincide with the submission of the PRGCA and will afford government project officers and awardees an additional opportunity for dialogue regarding progress toward program requirements and goals, as well as respond in a timelier fashion to technical assistance needs. Measures to be used in progress reports can be found in Appendix B.
Table 2.
|
Performance Period |
Progress Report Due Date |
Reporting Schedule |
July 1 through February 28 |
March |
*Actual due date will vary, but it is typically the second week of March.
Phase II: End-of-Year Annual Performance Reports
Data collected through the PRGCA serve a number of critical functions and are essential for responding to Federal reporting requirements (e.g., GPRAMA); understanding emerging issues in the health professions; ensuring compliance with grant and statutory requirements; strengthening overall program performance; and responding to congressional and public inquiries regarding outcomes associated with health professions training and loan programs. For example:
Providing key metrics for the performance budget. The PRGCA provides all of the metrics for BHW’s performance budget that is included in HRSA’s Congressional Justification. BHW currently has 44 GPRA performance budget measures and over 45 additional output measures that are all calculated based on PRGCA data. Additionally, every program in BHW has a program accomplishments paragraph included that is also based on the metrics and data provided by awardees in the PRGCA. Many of the metrics are included in a new program dashboard that is scheduled to go live in the summer of 2019 on the HRSA website.
Informing program management decisions. For example, data collected over the last 3 years from the Primary Care Training and Enhancement (PCTE) program were analyzed and used to make significant changes to the upcoming program Notice of Funding Opportunity. These PRGCA data-driven decisions will align this program with primary care physician workforce projections produced by the National Center for Health Workforce Analysis so that program funding only goes to physician specialties in short supply and to areas of the country showing greatest shortages in FY 2020.
Enhancing the agency's understanding about the distribution of individuals receiving direct financial assistance. As a result of the performance measures being collected at the individual level, BHW has been able to more accurately compare training distribution rates across its programs. In addition, we are able not only to identify a student’s intent to practice in an underserved area, but have been able to collect counts/percentages of graduates who are actually practicing in underserved, rural and primary care areas. BHW has also begun to collect National Provider Identifier data to assist in efforts related to the distribution of graduates of the health professions programs. This effort is helping the Bureau produce stronger outcome data for its programs.
In an effort to implement a stronger performance management strategy throughout the Bureau, BHW changed to an annual PRGCA reporting schedule (see Table 3) in 2016 from a semi-annual reporting schedule. This annual reporting system has served the Bureau well over the past three years and reduced burden on awardees compared with the semi-annual reporting. BHW will continue to utilize technical assistance calls, quarterly calls, and progress reports to proactively and systematically respond to program performance throughout the fiscal year. In addition, BHW has implemented a process called Rapid Cycle Quality Improvement in their Funding Opportunities. The RCQI process requires applicants to use quality improvement techniques to ensure grant activities achieve their intended purposes, and promote continuous assessment and improvement of activities as needed over the grant periods. Measures to be used in the PRGCA can be found in Appendix C and are presented separately for each BHW-funded program.
Table 3.
|
Performance Period |
PRGCA Due Date |
Current Annual Reporting Schedule |
July 1 through June 30 |
July 31 |
3. Use of Improved Information Technology and Burden Reduction
Consistent with the previous reporting cycle, BHW will continue to use HRSA's Electronic Handbook (EHB) as the portal for data collection. The EHB serves as the system of record for HRSA’s grants and cooperative agreements. As a technological enhancement, several of the forms have the option to update previous information reported—reducing the need to re-report information which does not vary during the life of a specific grant. For example, demographic information about individuals receiving direct financial support (e.g., stipends, loans, or scholarships) will only have to be reported once. Using awardee-developed unique identifiers, each awardee will only be required to update specific fields—such as financial award amounts, attrition status, graduation status, and 1-year follow-up.
BHW has also invested in a new data entry portal system that is currently being piloted with one of the programs, Children’s Hospital Graduate Medical Education program (CHGME). If this endeavor proves successful, it may be rolled out to other large grant programs as well. This new data collection portal was approved by OMB on 4/06/2018. The following describes this new innovative system as was reported in the change memo approved by OMB.
To reduce the reporting burden on high volume awardees who are required to report individual level data on a large number of trainees, BHW has developed a data collection portal that will allow awardees to collect individual-level trainee data (consisting of the trainee’s name, training program, demographic information, aspects of their training, and employment information upon completion of training) directly from trainees via online surveys and/or bulk data upload directly into a portal via an Excel spreadsheet. Only data from the INDGEN and INDGEN-PY forms (see Appendix C for these forms) will be collected through the new TRIP portal. Use of the new portal is voluntary for the HRSA awardee. Awardees may continue only to utilize the Electronic Handbook (EHB) system currently in place to provide their performance data to HRSA if they choose. The new TRIP portal is essentially a voluntary and alternative input pathway for an awardee’s INDGEN and INDGEN-PY forms data. Only the Children’s Hospital program will be initially utilizing the portal as that program has the largest number of INDGEN records. BHW hopes to eventually expand the portal for use by other programs that also heavily rely on the INDGEN form.
For awardees that decide to communicate with trainees for this data collection via the surveys, a trainee email addresses would be required. The survey responses will be collected, monitored, and managed in the portal, and awardees will be able to transmit and submit the data electronically into EHBs. Awardees will decide if they want to utilize the survey feature of the portal and may send their trainees up to three voluntary surveys that consist of the INDGEN and INDGEN-PY questions. A trainee profile survey would be sent at the beginning of their training experience. A trainee completion plans survey would be sent at the end of their training experience, and a graduate follow up survey would be sent approximately 6-10 months after their training. Awardees will be able to send reminders or notifications to the trainees for initial surveys or any follow-up reminders. As the surveys are voluntary, awardees will still be responsible for gathering and reporting the data to HRSA in the event a trainee does not respond to the surveys. A list of the INDGEN and INDGEN-PY questions for each survey can be found on pages 26-27 of the TRIP user guide. Awardees will always complete INDGEN questions pertaining to the training program (that a trainee might not be aware of). Those items are listed on page 26 of the TRIP user guide located on the HRSA website at https://bhw.hrsa.gov/grants/reportonyourgrant.
Awardees will also have the ability to directly upload bulk individual-level data rather than key in every required data field or utilize the survey function described previously. This upload feature will significantly reduce awardee burden and has been a feature that awardees have sought. A small group of volunteer awardee organizations (7-8) worked with HRSA to help develop features of this portal that will reduce their reporting burden and still allow HRSA to effectively evaluate its programs. Further, the same group also helped participate in user acceptance testing of the new portal. The feedback HRSA received was unanimously positive and awardees are anxiously waiting to utilize this new reporting tool for their individual-level data, particularly the upload feature.
Data elements collected in the portal about individual trainees will be the same as those already being collected in the EHB on the INDGEN and INDGEN-PY forms; only the source and retrieval method are changing. Enabling awardees to collect individual level trainee data directly from trainees may result in more accurate annual reports to BHW. There are three additional, optional fields contained in the portal that are not in the EHB INDGEN form. They are first name of trainee, last name of trainee, and trainee email address. The first and last name fields were added at the request of many awardees who wanted an easier method to identify individuals on the INDGEN form as some found the trainee unique ID field and NPI field (the current identifiers) not to be easy methods for identification from an awardee’s perspective. The awardee has the option to populate these fields in the new portal as a means to easily identify a record if they choose. They may also choose not to utilize them. The third new element is the email address that is also an optional field for the awardee. For awardees who choose to contact trainees directly via the survey response function, the email address of the trainee would be required to accomplish this function. This function and the email address field are again optional methods that HRSA have included to ease reporting burden on the awardee. Use of the portal is not a requirement, and a awardee could continue to exclusively utilize the EHB for their performance reporting.
Data from these three new fields will remain in the portal and not be transmitted to the INDGEN form within EHB as they are not required of the awardee. As such, the new portal has been designed to meet the PII requirements for this new identifying information. OMB approved the SORN for the TRIP portal on 02/13/2018. The system name and number is the HRSA Trainee Data Collection Portal System, 09-15-0092. The SORN may also be found here:
BHW’s ability to follow trainees after the completion of their training to find out if they are employed in health care and/or work in underserved areas is critical to evaluate the effectiveness and success of BHW health professions programs. The information gathered via the new portal will help HRSA/BHW to comply with the mandate for longitudinal follow-up of trainees. Section 5103 of P.L. 111-148 requires a longitudinal evaluation of individuals who have received education, training, or financial assistance from programs funded through the Bureau of Health Workforce (BHW). In addition, the GPRA Modernization Act of 2010 (GPRAMA) provides a stronger, more precise framework for performance management within the federal government. In accordance with this law, and as directed by the Office of Management and Budget, federal agencies must place a stronger emphasis on performance management and evaluation activities to help support their annual budget requests and demonstrate to the public the effectiveness of their programs. As part of a larger performance management framework, the BHW is committed to longitudinal evaluation of its programs to gain better understanding of factors associated with recruiting, retaining, and diversifying the healthcare workforce. The results of this effort will be used to inform the continued offering of current BHW programs and the development of future programs.
4. Efforts to Identify Duplication and Reduce Burden
BHW has engaged in a thorough analysis of its tools to identify redundancy and/or duplication of measures across its various data collection activities. Below are summaries of strategies used with each data collection activities to eliminate duplication and reduce burden.
Reducing duplication and burden associated with progress reports
Over the last 3 years, BHW has used the revised progress report and this has provided programs with valuable information and has streamlined reporting for awardees. In both 2012 and 2015, BHW eliminated the duplication of measures between the progress reports and the PRGCA. Awardees and government project officers have been very pleased with this reduction in burden and progress reports are being used more frequently to assess program performance. The progress report in Appendix B has not been changed since the last OMB submission, it does not contain any duplicative measures and focuses on assessing activities implemented; achievements and barriers encountered for each activity; as well as technical assistance needs of awardees.
Reducing duplication and burden associated with the PRGCA
Based on feedback from awardees, staff, evaluators, and public comment, BHW continues to revise the PRGCA to eliminate duplicative efforts. Migrating from semi-annual to annual reporting significantly reduced burden on awardees, government project officers, and data scientists in the National Center for Health Workforce Analysis. In addition, technological enhancements have been made to the reporting system where values are automatically totaled for the awardee. In the past, the awardees had to enter individual values and totals, but with the enhancements the system automatically totals and auto-populates cells for the awardees.
One of the major changes over the past 3 years was the ability to have the awardees only report individual level data instead of also reporting aggregate data, BHW scientists have been calculating aggregate-level estimates for each program. This process automatically reduces the number of tables being required for the awardees of direct financial support programs to complete thus significantly reducing burden for the awardees and government project officers who are required to review all the forms submitted by awardees. Additionally, the individual level data helps the Bureau achieve its longitudinal evaluation plans as collection of this data allows BHW to determine whether a graduate of our programs participated in one or more of our federally funded programs, is currently practicing in primary care, is currently practicing in a rural and/or underserved community, and the primary discipline under which they are certified or licensed. This information also helps BHW use data-driven strategies for implementing its programs as programs can focus on areas and disciplines of short supply as well as helping its graduates find employment in underserved areas, all top priorities for HRSA and BHW.
Similarly, over the last 3 years BHW has conducted an extensive analysis to identify other areas of duplication among forms that capture program-level characteristics for infrastructure and multipurpose or hybrid programs. BHW has revised the breadth and depth of measures in each form to ensure that only measures that are most salient to program management and performance reporting are captured in a manner that is appropriate to the purpose, design and impact of each program. Appendix C contains measures and related instructions for each of BHW's health professions training and loan programs.
This project does not involve small businesses or other small entities.
6. Consequences of Collecting the Information Less Frequently
Progress Reports
Data collected in the form of progress reports is a key element of BHW's performance management strategy and serves as an official record of communication between government project officers and awardees. These data provide time-sensitive information about the successes and challenges encountered by awardees in implementing required activities. Progress reports also serve as an instrument for determining awardee-specific technical assistance needs. Collecting data annually will allow BHW to provide a timely response to awardee-specific concerns and technical assistance needs, as well as respond to emerging issues across the health professions. Annual data collection is the minimum allowed under GPRA.
Performance Reports for Grants and Cooperative Agreements (PRGCA)
Over the past 3 years, data have been collected from awardees on an annual basis to meet Federal reporting requirements; respond to congressional inquiries; and strengthen program performance. The implementation of an annual reporting schedule for PRGCA was a critical step in improving BHW’s performance management strategy and reducing awardee burden across the bureau. Analysis of annual vs. semi-annual reporting showed that collection of data on a semi-annual basis imposed additional burden on awardees, government project officers and social scientists. During this past three-year period, BHW scientists have improved the reporting processes in the EHB system and have updated performance measures to enable a reduction in burden with the move to an annual reporting process. Awardees and government have all responded very positively to the annual reporting. In short, the move to annual reporting reduced burden, cost, and showed no difficulty with performance management. In fact, data from annual reporting has shown to be more accurate as it aligns with the Academic Year that all of our awardees work on.
7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
The proposed data collection is consistent with guidelines set forth in 5 CFR 1320.5(d) (2).
8. Comments in Response to the Federal Register Notice & Outside Consultation
Section 8A: Federal Register Notice
The 60-day l notice was published in the Federal Register on October 18, 2018, vol. 83, No. 202; pg. 52840 (Appendix D). Notification was sent to all awardees regarding the posting of the Federal Register notice and an overview of the proposed revisions to performance measurement activities. During the 60-day notice period, only one comment (Appendix E) and one request for materials were received. The comment originated from a professional organization and was actually related more to the National Center for Health Workforce Analysis’ healthcare workforce projection model rather than the performance measurement of our grant programs. There were no specific requests in the comment for changes to the performance measures; nevertheless, HRSA is reaching out to the organization to discuss any opportunities that we may share our data and best practices.
Section 8B: Outside Consultation
In developing the proposed updates to BHW's data collection activities, scientists from BHW's National Center for Health Workforce Analysis (NCHWA) met with government project officers in BHW to discuss updates to the measures, as well as any programmatic changes that were necessary. Government project officers provided critical feedback that assisted NCHWA in updating measurement activities, as well as reducing redundancy and burden. Government project officers have been extraordinarily pleased with the change to an annual the reporting schedule.
In collaboration with government project officers, NCHWA staff held TA sessions to discuss the performance reporting change to annual reporting. Overall, awardees responded very positively to BHW's efforts to update measurement activities and reduce burden and redundancy. Some awardees expressed relief that we were moving to annual reporting because it was very difficult for schools to report in the first reporting period due to school closures for the holidays.
In addition to meeting with government project officers and groups of awardees, NCHWA also consulted with the following individuals about the instruments, burden, and the reporting schedule.
John Snyder, MD, MS, MPH (FACP) Chief Medical Officer HRSA/OA/Office of Policy, Analysis, and Evaluation Department of Health and Human Services Ph: 301-443-4773 Email: [email protected] Consulted 2018 |
Patricia Pittman, PhD Director of George Washington Health Workforce Institute Professor of Health Policy and Management George Washington University Milken Institute School of Public Health Ph: (202) 994-4295 Email: [email protected] Consulted 2018 |
Angela Beck, PhD, MPH Director of Behavioral Health Workforce Research Center Clinical Assistant Professor of Health Behavior and Health Education Assistant Dean for Student Engagement and Practice University of Michigan School of Public Health Ph: (734) 764-8775 Email: [email protected] Consulted 2018 |
Sheila Pradia-Williams, CAPT, Pharm.D. Senior Advisor to the Associate Administrator, Bureau of Health Workforce Health Resources and Services Administration Ph: (301) 443-3709 Email: [email protected] Consulted 2018 |
9. Explanation of any Payment/Gift to Respondents
No payments or gifts are to be provided to respondents. Data collection activities are required as part of the cooperative agreement with awardees and are authorized under 45 CFR Part 74.
10. Assurance of Confidentiality Provided to Respondents
All data collected by BHW awardees (i.e. program level and/or individual level) will be reported through BHW’s PRGCA system that is built on a secure web-based enterprise framework. Program-level data reported by BHW awardees are aggregate in nature. Individual-level data reported by awardees are de-identified by the awardee and reported to BHW using awardee-specific unique identifiers. To ensure confidentiality, awardees are not asked or required to provide a list that corresponds unique identifiers with actual student names; rather, data is reported and will always remain de-identified. For programs that also report NPI numbers on individual-level data, NPI numbers are publicly available via the Centers for Medicare and Medicaid Services (CMS) and when reported to HRSA, are stored on the EHB system that is equipped to handle this level of PII.
11. Justification for Sensitive Questions
Data collection efforts through progress reports and the PRGCA do not obtain information of a sensitive nature. Demographic-related data (e.g., race, ethnicity, age, and gender) will be collected in accordance with standards authorized under Section 4302 of the Patient Protection and Affordable Care Act. Veteran status will be measured in a manner that is consistent with the Veteran’s Administration while disadvantaged status will continue to capture financial disadvantaged status, as well as educational disadvantaged status.
12. Estimates of Annualized Hour and Cost Burden
The estimated annualized burden for the proposed data collection activities vary by activity, as well as the types of awardees providing the required information. Table 4 summarizes the estimated burden by fiscal year, data collection activity, and type of grant program.
12A. Estimated Annualized Burden Hours
Table 4a. Response for Awardees of Direct Financial Support Programs
Type of Respondent |
Form Name |
No. of Respondents |
No. Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Awardee (Direct Financial Support Program) |
Training Program Form |
500 |
1 |
.15 |
75 |
Awardee (Direct Financial Support Program) |
Program Characteristics Form |
500 |
1 |
.50 |
250 |
Awardee (Direct Financial Support Program) |
IND-GEN |
500 |
1 |
1.25 |
625 |
Awardee (Direct Financial Support Program) |
EXP |
500 |
1 |
.50 |
250 |
Awardee (Direct Financial Support Program) |
Curriculum Development & Enhancement Form |
100* |
1 |
.25 |
25 |
Awardee (Direct Financial Support Program) |
Program Curriculum Changes |
58* |
1 |
.50 |
29 |
Awardee (Direct Financial Support Program) |
Hospital Data |
58* |
1 |
.50 |
29 |
Awardee (Direct Financial Support Program) |
Faculty Development, Instruction & Recruitment Form |
100* |
1 |
.50 |
50 |
Awardee (Direct Financial Support Program) |
Progress Report |
500 |
1 |
.50 |
250 |
SUB-TOTAL |
|
500 |
1 |
3.166 |
1,583 |
*Note: Total number of respondents for Awardee Direct Financial Support Programs is 500; however, not all awardees are required to complete all forms due to the nature and purpose of their programs. Number of respondents may be equal to or less than 500 for any form. The completion of all required forms is considered a response to this data collection activity.
Table 4b. Response for Awardees of Infrastructure Programs
Type of Respondent |
Form Name |
No. of Respondents |
No. Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Awardee (Infrastructure Program) |
Grant Purpose Form |
57* |
1 |
.15 |
9 |
Awardee (Infrastructure Program) |
Training Program Form |
100 |
1 |
.15 |
15 |
Awardee (Infrastructure Program) |
Program Characteristics Form |
100 |
1 |
.50 |
50 |
Awardee (Infrastructure Program) |
LR-1 |
100 |
1 |
.25 |
25 |
Awardee (Infrastructure Program) |
LR-2 |
100 |
1 |
.25 |
25 |
Awardee (Infrastructure Program) |
DV-1 |
100 |
1 |
.25 |
25 |
Awardee (Infrastructure Program) |
DV-2 |
100 |
1 |
.25 |
25 |
Awardee (Infrastructure Program) |
DV-3 |
100 |
1 |
.25 |
25 |
Awardee (Infrastructure Program) |
EXP |
100 |
1 |
.50 |
50 |
Awardee (Infrastructure Program) |
CE |
57* |
1 |
3 |
171 |
Awardee (Infrastructure Program) |
Curriculum Development & Enhancement Form |
91* |
1 |
.25 |
23 |
Awardee (Infrastructure Program) |
Faculty Development, Instruction & Recruitment Form |
91* |
1 |
.50 |
46 |
Awardee (Infrastructure Program) |
Progress Report |
100 |
1 |
.50 |
50 |
SUB-TOTAL |
|
100 |
1 |
5.39 |
539 |
*Note: Total number of respondents for Awardee Infrastructure Programs is 100; however, not all awardees are required to complete all forms due to the nature and purpose of their programs. Number of respondents may be equal to or less than 100 for any form. The completion of all required forms is considered a response to this data collection activity.
Table 4c. Response for Awardees of Multipurpose/Hybrid Programs
Type of Respondent |
Form Name |
No. of Respondents |
No. Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Awardee (Multipurpose or Hybrid Program) |
Grant Purpose Form |
285* |
1 |
.15 |
43 |
Awardee (Multipurpose or Hybrid Program) |
Training Program Form |
900 |
1 |
.15 |
135 |
Awardee (Multipurpose or Hybrid Program) |
Program Characteristics Form |
900 |
1 |
.50 |
450 |
Awardee (Multipurpose or Hybrid Program) |
LR-1 |
700* |
1 |
.25 |
175 |
Awardee (Multipurpose or Hybrid Program) |
LR-2 |
700* |
1 |
.25 |
175 |
Awardee (Multipurpose or Hybrid Program) |
DV-1 |
700* |
1 |
.25 |
175 |
Awardee (Multipurpose or Hybrid Program) |
DV-2 |
700* |
1 |
.25 |
175 |
Awardee (Multipurpose or Hybrid Program) |
DV-3 |
700* |
1 |
.25 |
175 |
Awardee (Multipurpose or Hybrid Program) |
IND-GEN |
900 |
1 |
.50 |
450 |
Awardee (Multipurpose or Hybrid Program) |
EXP |
900 |
1 |
.50 |
450 |
Awardee (Multipurpose or Hybrid Program) |
CE |
123* |
1 |
.25 |
31 |
Awardee (Multipurpose or Hybrid Program) |
Curriculum Development & Enhancement Form |
900 |
1 |
.25 |
225 |
Awardee (Multipurpose or Hybrid Program) |
Faculty Development, Instruction & Recruitment Form |
900 |
1 |
.50 |
450 |
Awardee (Multipurpose or Hybrid Program) |
State Oral Health Activities |
30* |
1 |
.5 |
15 |
Awardee (Multipurpose or Hybrid Program) |
Progress Report |
900 |
1 |
.50 |
450 |
Awardee (Multipurpose or Hybrid Program) |
State Primary Care Offices |
56* |
1 |
.25 |
14 |
SUB-TOTAL |
|
900 |
1 |
3.98667 |
3,588 |
*Note: Total number of respondents for Awardee Multipurpose/Hybrid Programs is 900; however, not all awardees are required to complete all forms due to the nature and purpose of their programs. Number of respondents may be equal to or less than 900 for any form. The completion of all required forms is considered a response to this data collection activity.
|
|
Number of Respondents |
Number of Responses per Respondent |
Total Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
TOTAL |
|
1,500 |
1 |
1,500 |
3.80667 |
5,710 |
12B. Estimated of Annualized Cost to Respondents
Based on the estimated total number of burden hours, it is estimated that the annualized cost to respondents is approximately $136,469 (Table 5). This result was obtained by multiplying the number of burden hours by the average hourly wage rate of an individual employed in an academic setting. (Note: Wage rates were obtained from the Department of Labor. Average Hourly Rate for this labor category is $23.90). Data collection and reporting activities are a grant requirement authorized under 45 CFR Part 74.
Table 5. Annualized Cost by Awardee Program
Type of Respondent
|
Total Burden Hours
|
Hourly Wage Rate
|
Total Respondent Costs
|
Awardee (Direct Financial Support Program) |
1,583 |
23.90 |
$37,834 |
Awardee (Infrastructure Program) |
539 |
23.90 |
$12,882 |
Our study is not a federally funded study. It is a doctoral dissertation funded by the doctoral student, William T. (Tom) Allen Jr. Awardee (Multipurpose/Hybrid Program) |
3,588 |
23.90 |
$85,753 |
|
|
|
|
Total |
5,710 |
|
$136,469 |
(Hourly rate determined using Labor Category ID CES6500000008, Education and Health Services).
13. Estimates of Capital Costs and Operations & Maintenance to Respondents or Recordkeepers.
There will be no capital costs or costs associated with operations and maintenance to respondents as all data are reported through a web-based enterprise system owned by and maintained at HRSA.
14. Annualized Cost to Federal Government
The systems used to collect information in the form of progress reports and the PRGCA are maintained by HRSA. It is estimated that the amount of staff time needed for the review and approval of progress reports and PRGCA submitted on an annual basis is equivalent to 2 FTEs at the GS-13 level—for a total of $193,940. Collectively, the estimated annualized cost to the government in staff time is estimated to be $193,940.
15. Explanation for Program Changes or Adjustments
There are currently 5,992 total burden hours approved by OMB for this activity. This request is for approval of roughly 5,710 burden hours, a decrease of nearly 300 hours. The decrease in burden is primarily due to 1) an increase in efficiency of the electronic reporting system and 2) decreases in specific form requirements for multipurpose grant programs. This reduction is a result of annual reviews conducted by the National Center for Health Workforce Analysis scientists. Specifically, as programs have moved to collecting individual-level data on trainees, they no longer must report aggregated counts across five additional forms. We have also maintained the significant burden reduction of over 1,700 hours in 2016 by continuing to use the annual reporting strategy.
BHW has improved the electronic reporting process by providing user-friendly templates on certain forms as well as pre-populated data fields on forms that do not utilize templates. The new web-based reporting system features reduce the need for manual data entry thus reducing burden. The system is designed to pre-populate fields with previously entered data thus reducing date re-entry by the user; automates the calculation of total counts; and allows awardees the ability to enter data into spreadsheets that are available in the web-based reporting system. As a technological enhancement to previous years, several of the revised forms will have the option to update previous information reported—reducing the need to re-report information which does not vary during the life of a specific grant. For example, demographic information about individuals receiving direct financial support (e.g., stipends, loans, or scholarships) will only have to be reported once. Using awardee-developed unique identifiers or NPI numbers, each awardee will only be required to update specific fields—such as financial award amounts, attrition status, graduation status, and 1-year follow-up. Cumulative financial award amounts for individuals are automatically calculated by academic year and over multiple years. Additionally, program financial disbursement totals are also provided to awardees so they may compare this amount with the amount they provide on their Federal financial forms.
There are approximately 20 different forms being used for reporting. These forms were redesigned in 2012, and they have provided BHW with exceptional results. More importantly they have improved the quality and accuracy of data reported. Over the last 3 years, BHW has developed and updated PRGCA manuals and revised the instructions for reporting in order to reduce the amount of awardee data entry time when submitting the performance data. BHW has written and updated a PRGCA manual for each of its programs that contain screenshots of HRSA’s Electronic Handbook (EHB) as well as specific instructions for awardees of each program. As each manual is tailored to the specific program reporting needs, awardees only see forms and items they must complete and are not confused by forms or fields they do not have to complete. The awardees have provided very positive feedback on the user-friendliness of the manuals and the improved efficiency when they are reporting performance data. The manuals as well as the applicable performance measures are available on line at http://bhw.hrsa.gov/grants/reporting/index.html. These manuals serve to increase reliability and accuracy of the performance data and serve an important role in BHW’s performance management strategy.
16. Plans for Tabulation, Publication, and Project Time Schedule
Phase I: Mid-Year Progress Reports - Data collected in the form of progress reports will serve as the official record of communication between government project officers and awardees, and will be used to respond to awardee-specific concerns and technical assistance needs.
Phase II: End-of-Year Annual Performance Reports - Data collected in the form of PRGCA serves a number of important purposes including strengthening program performance; responding to Federal reporting requirements (e.g., GPRAMA); responding to congressional inquiries. Since programs are publicly-funded, data collected through the PRGCA may be showcased in peer-reviewed articles, conferences, and/or reports published through and/or sponsored by HRSA. The process for cleaning, analyzing, and reporting data will consist of the following steps4:
Step 1: Data completeness and accuracy. BHW utilizes a multi-level approach to ensure that data/information used for performance measures is complete. Awardees enter required data according to an established data reporting process which, for the majority of its workforce programs, includes reporting through HRSA’s grant system, Electronic Handbooks (EHB). During data entry in EHB, validation checks on data ensure report completeness and consistency across related measures. The awardee is notified during report completion of any errors that must be resolved prior to submission. Following report submission, government project officers (POs) review the performance report for accuracy and completeness. If the information is incomplete, the PO will request changes from the awardee and the report is sent back to the awardee. The awardee will then resubmit a report after addressing the PO’s concerns. Lastly, the data are reviewed again for completeness and accuracy by scientists in the National Center for Health Workforce Analysis who maintain the performance measures and are knowledgeable about program requirements. Deadlines for reporting are consistent with the Uniform Guidance (45 CFR 75) and shown in Table 3.
Data collected through performance reports serve a number of critical functions such as informing program management decisions, monitoring the types of activities implemented by awardees, and enhancing HRSA’s understanding about the diversity and distribution of the individuals receiving direct support from HRSA programs. In addition, performance reports are essential for:
responding to Federal reporting requirements;
understanding emerging issues in the health professions;
ensuring compliance with statutory requirements, applicable regulations, and terms and conditions of award.
Identifying potential promising or best practices;
strengthening overall program performance; and
responding to congressional and public inquiries regarding outcomes associated with health professions training and loan programs.
The programs receive and review raw data from awardees, aggregate these data for HRSA-level reporting, have ongoing oversight of reporting entities and have direct contact with them to resolve potential data problems. For instance, if a awardee is missing its performance targets, the PO will work to understand why and may work with the awardee to develop a corrective action plan. These data are also reviewed by the National Center for Health Workforce Analysis (NCHWA) who use their data and evaluation expertise to assess how the data relate to national trends. In addition, these measures are entered into the HHS Program Performance Tracking System (PPTS). It is this system (PPTS) that is used to produce information for the HHS Annual Performance Report.
Step 2: Analysis5 & Reporting. The analysis of all PRGCA-related data will be conducted by doctoral-level scientists in NCHWA according to the following priority-based schedule:
Priority I. Data that is essential for performance management and budgeting will be analyzed with the highest priority. Results from these analyses will be provided to government project officers and BHW leadership in the form of briefs and/or reports within 30 days of NCHWA completing the data cleaning process.
Priority II. Data that can be used to respond to inquiries from Congress, stakeholders, and/or the public will be analyzed and reported in accordance with the urgency of the request (usually 1 to 3 days).
Priority III. Data that can enhance the agency's understanding of emerging trends in the health professions will be analyzed and provided to BHW leadership in the form of briefs or presentations within 60 days of NCHWA completing the data cleaning process.
Priority IV. Data that can be used to inform the development of articles or conferences will be analyzed and abstracts of findings will be provided to the requesting staff usually 2-4 weeks from the day of the request.
17. Exemptions for Not Displaying OMB Expiration Date
No exemption is requested. Respondents will see the OMB number and expiration date on each table that exists in the EHB and/or TRIP systems.
18. Exceptions to Certification for Paperwork Reduction Act Submissions
This information collection fully complies with the guidelines set forth in 5 CFR 1320.9. There are no exceptions to the certification statement.
1 Pub. L. No. 103-62, 107 Stat. 285 (Aug. 3, 1993).
2 Pub. L. No. 111-352, 124 Stat. 3866 (Jan. 4, 2011).
3 See Appendix A for a complete listing of BHW funded programs by category.
4 Steps apply to each reporting period (FY 2016 and beyond). Please see Table 3 for an overview of beginning and ending periods of reporting.
5 The analyses of PRGCA data have historically been primarily descriptive in nature (e.g., frequencies, percentages, ratios).
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement |
| Author | Namratha Swamy |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |