0915-0061 - Supporting Statement A_final rev

Supporting Statement A

Bureau of Health Workforce (BHW) Performance Report for Grants and Cooperative Agreements

OMB Control No. 0915-0061

Revision

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

This request is for approval from the Office of Management and Budget (OMB) to continue data collection activities with current and prospective health professions awardees of the Health Resources and Services Administration’s (HRSA) Bureau of Health Workforce (BHW). The current approval (OMB #0915-0061) expires on 03-31-2022 and covers data collection efforts through progress reports, as well as annual performance reports for grants and cooperative agreements (PRGCA).

BHW seeks approval from OMB to continue these efforts over the next three years. The supporting statement for this request has been updated and contains discussion about the utility of data collected during July 1, 2018 through June 30, 2021; lessons learned from data collection efforts during this time; as well as details regarding BHW’s proposed strategies for reducing the overall burden associated with its data collecting efforts (i.e., progress reports and the PRGCA).

HRSA is obligated to collect performance information on its grants and cooperative agreements as mandated in the Government Performance and Results Act of 1993 (GPRA)¹ and the GPRA Modernization Act of 2010 (GPRAMA)². The health professions grant programs are governed by the Public Health Service Act (42 U.S.C. 201 et seq.), specifically Titles III, VII, and VIII, which specify additional reporting requirements that are specific to certain programs.

BHW plans to continue with its current performance management strategy and make only minor changes that reduce burden, simplify reporting, and reflect new Department of Health and Human Services and HRSA priorities as well as elements that enable longitudinal analysis of program performance. One such change is the plan to expand implementation of the excel upload option for the INDGEN form to a majority of programs. Additionally, BHW plans to split profession/discipline questions into two parts making it easier for respondents to locate the appropriate response option. The profession/discipline question appears on the following forms: INDGEN (Column 26c), EXP2 (Column 2a), EXP3 (Column 3a), CE2 (Column 1a), CDE2 (Column 1a), FD-1a (Column 1a), FD-1b (Column 1a), FD-2b (Column 1a), FD-4b (Column 1a). In response to additional funding received to address the COVID-19 pandemic, BHW proposes the inclusion of one question “Was COVID-19 Telehealth Supplement Funding Used?” on the CDE-1 form (Column 13), CE-1 form (Column 16), FD-1a form (Column 12), FD-2a form(Column 8), and FD-3 form (Column 10). This is necessary to distinguish results of the COVID-19 supplement funding from discretionary appropriations. Similarly, as a result of the increased emphasis on telehealth, BHW proposes the addition of three questions to the INDGEN form – Select Whether the Individual Received Training in a Telehealth Setting (Column 27d), Enter # of Contact Hours (Column 27e), and Enter # of Patient Encounters (Column 27f). These questions are complementary to existing questions about training in primary care, rural, and medically underserved communities. Not all awardees answer the contact hours and patient encounters questions. On the EXP form, BHW proposes including street address columns (Columns 7a/7b) so address data for clinical training sites can merge into HRSA’s Enterprise Site Repository (ESR). Lastly, in effort to incorporate reporting for the National Health Service Corp – State Loan Repayment Program into the PRGCA, BHW proposes the addition of 18 program specific questions on INDGEN (Columns 21d, 21e, 21h, 22a, 23a, 27aa, 80-82, and 84-92) and four program specific questions on EXP-1 (Columns 15-18). These questions are only for the State Loan Repayment Program. See Appendix C for additional detail.

Copies of all performance measures and forms are included in this package. No program completes all forms or items and the program awardees only sees forms and items specific to them when completing their PRGCA. All measures, forms, and program-specific manuals are also located on the HRSA website at https://bhw.hrsa.gov/grants/reportonyourgrant.

2. Purpose and Use of Information Collection

BHW is statutorily tasked with responding to issues specific to the training and supply of the current and future US healthcare workforce (see 42 USC 292 et seq). Currently, BHW funds over 40 different health professions training and loan programs that aim to increase the supply, diversity, and distribution of the current and future US healthcare workforce. Generally, these programs fall into three distinct categories³:

• Infrastructure: refers to programs that are designed to enhance the scope and/or quality health professions training programs. These programs do not provide direct financial support to students; rather, awardees use funds in a variety of ways including enhancing curriculum and clinical training opportunities, as well as offering faculty development opportunities.

• Direct Financial Support: refers to programs that are designed to provide students of health professions training programs with a financial award to cover costs associated with tuition and/or allowable living expenses. Depending on the nature of the program, awardees of these programs provide scholarships, stipends, or loans to students pursuing health profession-related training or degrees.

• Multipurpose or Hybrid programs: refers to programs that, in accordance with their authorizing statute, may fund a variety of activities to include enhancing training infrastructure, providing direct financial support to health professions students, or supporting enhancements to clinical rotations and training.

In order to carry out its functions, BHW has historically collected data from funded awardees at two specific phases of a grant cycle:

• Phase I: Mid-Year Progress Reports

• Data collected in the form of progress reports serve as the official record of communication between government project officers and awardees and highlight awardees' successes and challenges in meeting the goals of each program. Information provided through progress reports are reviewed by government project officers in BHW and are used to determine progress toward implementing required grant activities; as well as technical assistance needs. In addition, information provided through progress reports also assists BHW in understanding fluctuations in program outcomes reported through the PRGCA.

• This request seeks approval to continue collecting information through progress reports from BHW-funded awardees on an annual basis (Table 2). Submission of progress reports will not coincide with the submission of the PRGCA and will afford government project officers and awardees an additional opportunity for dialogue regarding progress toward program requirements and goals, as well as to respond in a timelier fashion to technical assistance needs. Measures to be used in progress reports can be found in Appendix B.

Table 2.

	Performance Period	Progress Report Due Date
Reporting Schedule	July 1 through February 28	March

*Actual performance period and due date will vary, but it is typically the second week of March.

• Phase II: End-of-Year Annual Performance Reports

• Data collected through the PRGCA serve a number of critical functions and are essential for responding to Federal reporting requirements (e.g., GPRAMA and Evidence Act), understanding emerging issues in the health professions, ensuring compliance with grant and statutory requirements, strengthening overall program performance, and responding to congressional and public inquiries regarding outcomes associated with health professions training and loan programs. For example:

• Providing key metrics for the performance budget. The PRGCA provides all of the metrics for BHW’s performance budget that is included in HRSA’s Congressional Justification. BHW currently has 49 GPRA performance budget measures and over 30 additional output measures that are all calculated based on PRGCA data. Additionally, every program in BHW has a program accomplishments paragraph included that is also based on the metrics and data provided by awardees in the PRGCA. Many of the metrics are included in the new Health Professions Training Programs dashboard that went live on the HRSA website in 2019 and gets updated annually - https://data.hrsa.gov/topics/health-workforce/training-programs.

• Informing program management decisions. In 2021, a new Grantee Scorecard went live within the EHB system which streamlines the program management process. It visually displays data from a variety of locations within EHB in one place allowing Project Officers, National Center for Health Workforce Analysis (NCHWA) scientists, and BHW leaders to quickly review important information on awardees. The Scorecard includes key performance metrics, submission timeliness information, and other awardee monitoring details from Academic Year 2015 to present. Visualizations allow users to quickly see whether performance metric results are increasing, decreasing, or staying the same year to year. Another important feature on these dashboards is the inclusion of targets. This allows users to see how a particular awardee, grant program, or portfolio of grant programs are performing as compared to program- and BHW-level targets. Combined, this information is intended to inform program management decisions. The scorecards are also available to Awardees so they can see how their particular grant program(s) are doing as compared to the program- and BHW-level targets and make adjustment, as needed.

• Enhancing the agency's understanding about the distribution of individuals receiving direct financial assistance. As a result of the performance measures being collected at the individual level, BHW has been able to more accurately compare training distribution rates across its programs. In addition, we are able not only to identify a student’s intent to practice in an underserved area, but have been able to collect counts/percentages of graduates who are actually practicing in underserved, rural and primary care areas. BHW is also collecting National Provider Identifier data to assist in efforts related to the distribution of graduates of the health professions programs. This effort is helping the Bureau produce stronger outcome data for its programs.

In an effort to implement a stronger performance management strategy throughout the Bureau, BHW changed to an annual PRGCA reporting schedule (see Table 3) in 2016 from a semi-annual reporting schedule. This annual reporting system has served the Bureau well and continues to be a strong performance management strategy with less burden on awardees compared with the semi-annual reporting. BHW will continue to utilize technical assistance calls, quarterly calls, and progress reports to proactively and systematically respond to program performance throughout the fiscal year. In addition, BHW has implemented a process called Rapid Cycle Quality Improvement in their Funding Opportunities. The RCQI process requires applicants to use quality improvement techniques to ensure grant activities achieve their intended purposes, and promote continuous assessment and improvement of activities as needed over the grant periods. Measures to be used in the PRGCA can be found in Appendix C and are presented separately for each BHW-funded program.

Table 3.

	Performance Period	PRGCA Due Date
Current Annual Reporting Schedule	July 1 through June 30	July 31

3. Use of Improved Information Technology and Burden Reduction

Consistent with the previous reporting cycle, BHW will continue to use HRSA's Electronic Handbook (EHB) as the portal for data collection. The EHB serves as the system of record for HRSA’s grants and cooperative agreements. Several of the forms have the option to update previous information reported—reducing the need to re-report information which does not vary during the life of a specific grant. For example, demographic information about individuals receiving direct financial support (e.g., stipends, loans, or scholarships) will only have to be reported once. Using awardee-developed unique identifiers, each awardee will only be required to update specific fields—such as financial award amounts, attrition status, graduation status, and 1-year follow-up.

To reduce the reporting burden on high volume awardees who are required to report individual level data on a large number of trainees, BHW developed an Excel upload. The Excel upload allows awardees to collect individual-level trainee data (consisting of the trainee’s unique ID, training program, demographic information, aspects of their training, and employment information upon completion of training) and complete a bulk data upload directly into EHB rather than keying in every required data field within EHB. Only data from the INDGEN form (see Appendix C for the form) has this option. Use of the spreadsheet is voluntary for the HRSA awardee and the spreadsheet captures the same information as required within EHB on the INDGEN form. Awardees that prefer to enter their data directly into EHB may continue to do so. The Excel upload is essentially a voluntary and alternative input pathway for an awardee’s INDGEN form data. This upload feature significantly reduces awardee burden while allowing HRSA to effectively evaluate its programs. So far, feedback HRSA received has been unanimously positive. Initially, only the Children’s Hospital program received this option as that program has the largest number of INDGEN records. In 2021, BHW successfully expanded the excel upload option to ten additional grant programs that also heavily rely on the INDGEN form. In 2022, BHW expects to expand this option to nearly all grant programs. Awardees who do not have this option are anxiously waiting to use it for their individual-level data.

BHW’s ability to follow trainees after the completion of their training to find out if they are employed in health care and/or work in underserved areas is critical to evaluate the effectiveness and success of BHW health professions programs. Section 5103 of P.L. 111-148 requires a longitudinal evaluation of individuals who have received education, training, or financial assistance from programs funded through the Bureau of Health Workforce (BHW).  In addition, the GPRA Modernization Act of 2010 (GPRAMA) provides a stronger, more precise framework for performance management within the federal government. In accordance with this law, and as directed by the Office of Management and Budget and most recently the Evidence Act of 2018, federal agencies must place a stronger emphasis on performance management and evaluation activities to help support their annual budget requests and demonstrate to the public the effectiveness of their programs. As part of a larger performance management framework, the BHW is committed to longitudinal evaluation of its programs to gain better understanding of factors associated with recruiting, retaining, and diversifying the healthcare workforce. The results of this effort will be used to inform the continued offering of current BHW programs and the development of future programs.

Two additional technological advancements are planned for 2022. Currently, EHB is only compatible with Internet Explorer (IE) and Mozilla Firefox. Microsoft announced that IE will be retired in June 2022, so BHW is proactively updating the EHB system and the PRGCA report to ensure compatibility with Mozilla Firefox, Google Chrome, and Microsoft Edge. Awardees have long requested additional browser options. This upgrade will prevent awardees from having to download additional browsers to complete their reports and ensure a smoother reporting cycle.

BHW is also working on an address standardization feature for the EXP form. Clinical training site addresses that are entered into the EXP form by awardees are checked against the service objects locator. The service objects locator returns standardized addresses. Awardees can review and save the updated address data in their EXP form. This new feature will greatly improve data accuracy and will reduce awardee burden by preventing them from having to look up zip code extensions and manually enter them into the system.

4. Efforts to Identify Duplication and Reduce Burden

BHW has engaged in a thorough analysis of its tools to identify redundancy and/or duplication of measures across its various data collection activities. Below are summaries of strategies used with each data collection activities to eliminate duplication and reduce burden.

Reducing duplication and burden associated with progress reports

Over the last six years, BHW has used the revised progress report and this has provided programs with valuable information and has streamlined reporting for awardees. In both 2012 and 2015, BHW eliminated the duplication of measures between the progress reports and the PRGCA. Awardees and government project officers have been very pleased with this reduction in burden and progress reports are being used more frequently to assess program performance. The progress report in Appendix B has not been changed since the last OMB submission, it does not contain any duplicative measures and focuses on assessing activities implemented; achievements and barriers encountered for each activity; as well as technical assistance needs of awardees.

Reducing duplication and burden associated with the PRGCA

Based on feedback from awardees, staff, evaluators, and public comment, BHW continues to revise the PRGCA to eliminate duplicative efforts. Migrating from semi-annual to annual reporting significantly reduced burden on awardees, government project officers, and data scientists in the National Center for Health Workforce Analysis (NCHWA). In addition, technological enhancements have been made to the reporting system where values are automatically totaled for the awardee. In the past, the awardees had to enter individual values and totals, but with the enhancements the system automatically totals and auto-populates cells for the awardees. Similarly, on forms that request location data, city and state are auto-populated after zip code is entered. This improves data accuracy and reduces awardee burden of having to manually enter city and state information for records.

One of the major changes in 2019 was the ability to have the awardees only report individual-level data instead of also reporting aggregate data. NCHWA scientists calculate aggregate-level estimates for each program removing that burden from the awardee. This process automatically reduces the number of tables required from the awardees of direct financial support programs to complete thus significantly reducing burden for the awardees and government project officers who are required to review all the forms submitted by awardees. Additionally, the individual level data helps the Bureau achieve its longitudinal evaluation plans as collection of this data allows BHW to determine whether a graduate of our programs participated in one or more of our federally funded programs, is currently practicing in primary care, is currently practicing in a rural and/or underserved community, and the primary discipline under which they are certified or licensed. This information also helps BHW use data-driven strategies for implementing its programs as programs can focus on areas and disciplines of short supply as well as helping its graduates find employment in underserved areas, all top priorities for HRSA and BHW.

Similarly, in 2019 BHW conducted an analysis to identify other areas of duplication among forms that capture program-level characteristics for infrastructure and multipurpose or hybrid programs. BHW revised the breadth and depth of measures in each form to ensure that only measures that are most salient to program management and performance reporting are captured in a manner that is appropriate to the purpose, design and impact of each program. Over the last three years, BHW continued this process of removing unnecessary measures from individual grant programs to streamline reporting and reduce burden. Appendix C contains measures for each of BHW's health professions training and loan programs.

5. Impact on Small Businesses or Other Small Entities

This project does not involve small businesses or other small entities.

6. Consequences of Collecting the Information Less Frequently

Progress Reports

Data collected in the form of progress reports is a key element of BHW's performance management strategy and serves as an official record of communication between government project officers and awardees. These data provide time-sensitive information about the successes and challenges encountered by awardees in implementing required activities. Progress reports also serve as an instrument for determining awardee-specific technical assistance needs. Collecting data annually allows BHW to provide a timely response to awardee-specific concerns and technical assistance needs, as well as respond to emerging issues across the health professions. Annual data collection is the minimum allowed under GPRA.

Performance Reports for Grants and Cooperative Agreements (PRGCA)

Over the past three years, data have been collected from awardees on an annual basis to meet Federal reporting requirements, respond to congressional inquiries, and strengthen program performance. The implementation of an annual reporting schedule for PRGCA was a critical step in improving BHW’s performance management strategy and reducing awardee burden across the bureau. Analysis of annual vs. semi-annual reporting showed that collection of data on a semi-annual basis imposed additional burden on awardees, government project officers and NCHWA scientists. Awardees and government have all responded very positively to the annual reporting. In short, the move to annual reporting reduced burden, cost, and showed no difficulty with performance management. In fact, data from annual reporting has shown to be more accurate as it aligns with the Academic Year that all of our awardees work on.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

The proposed data collection is consistent with guidelines set forth in 5 CFR 1320.5(d) (2).

8. Comments in Response to the Federal Register Notice & Outside Consultation

Section 8A: Federal Register Notice

The 60-day formal notice was published in the Federal Register, 86 Fed. Reg. 53069 (September 24, 2021). (Appendix D). Notification was sent to all awardees regarding the posting of the Federal Register notice which included an overview of the proposed revisions to performance measurement activities. During the 60-day notice period, no comments were received.

Section 8B: Outside Consultation

In developing the proposed updates to BHW's data collection activities, scientists from BHW's National Center for Health Workforce Analysis (NCHWA) met with government project officers in BHW to discuss updates to the measures, as well as any programmatic changes that were necessary. Government project officers provided critical feedback that assisted NCHWA in updating measurement activities, as well as reducing redundancy and burden. Government project officers and awardees have been extraordinarily pleased with the change to an annual the reporting schedule.

In collaboration with government project officers, NCHWA staff held TA sessions to discuss updates to performance reporting. Overall, awardees responded very positively to BHW's efforts to update measurement activities and reduce burden and redundancy.

In addition to meeting with government project officers and groups of awardees, NCHWA also consulted with the following individuals about the instruments, burden, and the reporting schedule.

John Snyder, MD, MS, MPH (FACP)

Director of the Division of Data Governance and Strategic Analysis

HRSA/OA/Office of Policy, Analysis, and Evaluation

Department of Health and Human Services

Ph: 301-443-4773

Email: [email protected]

Consulted 2021

Patricia Pittman, PhD

Director of George Washington Health Workforce Research Center

Professor of Health Policy and Management

George Washington University Milken Institute School of Public Health

Ph: (202) 994-4295

Email: [email protected]

Consulted 2021

Angela Beck, PhD, MPH

Director of Behavioral Health Workforce Research Center

Clinical Assistant Professor of Health Behavior and Health Education

Assistant Dean for Student Engagement and Practice

University of Michigan School of Public Health

Ph: (734) 764-8775

Email: [email protected]

Consulted 2021

Sheila Pradia-Williams, CAPT, Pharm.D.

Director of Office of Strategy, Programs and Partnerships

Senior Advisor to the Associate Administrator, Bureau of Health Workforce

Health Resources and Services Administration

Ph: (301) 443-3709

Email: [email protected]

Consulted 2021

9. Explanation of any Payment/Gift to Respondents

No payments or gifts are to be provided to respondents. Data collection activities are required as part of the grant or cooperative agreement with awardees and are authorized under 45 CFR Part 74.

10. Assurance of Confidentiality Provided to Respondents

All data collected by BHW awardees (i.e., program level and/or individual level) will be reported through BHW’s PRGCA system that is built on a secure web-based enterprise framework. Program-level data reported by BHW awardees are aggregate in nature. Individual-level data reported by awardees are de-identified by the awardee and reported to BHW using awardee-specific unique identifiers. To ensure confidentiality, awardees are not asked or required to provide a list that corresponds unique identifiers with actual student names; rather, data is reported and will always remain de-identified. For programs that also report NPI numbers on individual-level data, NPI numbers are publicly available via the Centers for Medicare and Medicaid Services (CMS) and when reported to HRSA, are stored on the EHB system that is equipped to handle this level of PII. EHB uses role-based permissions to further protect the data. Data can only be downloaded for analysis by NCHWA Administrative users. Once downloaded, data is saved on secure sites that are only accessible using two-factor authentication by qualified NCHWA scientists. In accordance with the Health Professions data statute (42 USC 295k et seq.), raw data files are never shared.

11. Justification for Sensitive Questions

Data collection efforts through progress reports and the PRGCA do not obtain information of a sensitive nature. Demographic-related data (e.g., race, ethnicity, age, and gender) will be collected in accordance with standards authorized under Section 4302 of the Patient Protection and Affordable Care Act. Veteran status will be measured in a manner that is consistent with the Veteran’s Administration while disadvantaged status will continue to capture financial disadvantaged status, as well as educational disadvantaged status.

12. Estimates of Annualized Hour and Cost Burden

The estimated annualized burden for the proposed data collection activities vary by activity, as well as the types of awardees providing the required information. Table 4 summarizes the estimated burden by fiscal year, data collection activity, and type of grant program.

12A. Estimated Annualized Burden Hours

Table 4a. Response for Awardees of Direct Financial Support Programs

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Total Responses	Average Burden per Response (in hours)	Total Burden Hours
Awardee (Direct Financial Support Program)	Training Program Form	549*	1	549	.15	82.35
Awardee (Direct Financial Support Program)	Program Characteristics Form	549*	1	549	.50	274.5
Awardee (Direct Financial Support Program)	IND-GEN	699	1	699	1	699
Awardee (Direct Financial Support Program)	EXP	394*	1	394	.50	197
Awardee (Direct Financial Support Program)	Curriculum Development & Enhancement Form	209*	1	209	.25	52.25
Awardee (Direct Financial Support Program)	Program Curriculum Changes	59*	1	59	.50	29.5
Awardee (Direct Financial Support Program)	Hospital Data	59*	1	59	.50	29.5
Awardee (Direct Financial Support Program)	Faculty Development, Instruction & Recruitment Form	230*	1	230	.50	115
Awardee (Direct Financial Support Program)	Progress Report	699	1	699	.50	349.5
Awardee (Direct Financial Support Program)	Grant Purpose Form	219*	1	219	0.15	32.85
SUB-TOTAL		699	1	699	2.6631	1,861.5

*Note: Total number of respondents for Awardee Direct Financial Support Programs is 699; however, not all awardees are required to complete all forms due to the nature and purpose of their programs. Number of respondents may be equal to or less than 699 for any form. The completion of all required forms is considered a response to this data collection activity.

Table 4b. Response for Awardees of Infrastructure Programs

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Total Responses	Average Burden per Response (in hours)	Total Burden Hours
Awardee (Infrastructure Program)	Grant Purpose Form	48*	1	48	.15	7.2
Awardee (Infrastructure Program)	Training Program Form	142	1	142	.15	21.3
Awardee (Infrastructure Program)	Program Characteristics Form	142	1	142	.50	71
Awardee (Infrastructure Program)	LR-1	142	1	142	.25	35.5
Awardee (Infrastructure Program)	LR-2	142	1	142	.25	35.5
Awardee (Infrastructure Program)	DV-1	142	1	142	.25	35.5
Awardee (Infrastructure Program)	DV-2	142	1	142	.25	35.5
Awardee (Infrastructure Program)	DV-3	142	1	142	.25	35.5
Awardee (Infrastructure Program)	EXP	142	1	142	.50	71
Awardee (Infrastructure Program)	CE	112*	1	112	3	336
Awardee (Infrastructure Program)	Curriculum Development & Enhancement Form	117*	1	117	.25	29.25
Awardee (Infrastructure Program)	Faculty Development, Instruction & Recruitment Form	94*	1	94	.50	47
Awardee (Infrastructure Program)	Progress Report	142	1	142	.50	71
Awardee (Infrastructure Program)	IND-GEN	112*	1	112	.50	56
SUB-TOTAL		142		142	6.2486	887.3

*Note: Total number of respondents for Awardee Infrastructure Programs is 142; however, not all awardees are required to complete all forms due to the nature and purpose of their programs. Number of respondents may be equal to or less than 142 for any form. The completion of all required forms is considered a response to this data collection activity.

Table 4c. Response for Awardees of Multipurpose/Hybrid Programs

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Total Responses	Average Burden per Response (in hours)	Total Burden Hours
Awardee (Multipurpose or Hybrid Program)	Grant Purpose Form	516*	1	516	.15	77.4
Awardee (Multipurpose or Hybrid Program)	Training Program Form	726*	1	726	.15	108.9
Awardee (Multipurpose or Hybrid Program)	Program Characteristics Form	726*	1	726	.50	363
Awardee (Multipurpose or Hybrid Program)	LR-1	260*	1	260	.25	65
Awardee (Multipurpose or Hybrid Program)	LR-2	260*	1	260	.25	65
Awardee (Multipurpose or Hybrid Program)	DV-1	260*	1	260	.25	65
Awardee (Multipurpose or Hybrid Program)	DV-2	260*	1	260	.25	65
Awardee (Multipurpose or Hybrid Program)	DV-3	260*	1	260	.25	65
Awardee (Multipurpose or Hybrid Program)	IND-GEN	735*	1	735	.50	367.5
Awardee (Multipurpose or Hybrid Program)	EXP	726*	1	726	.50	363
Awardee (Multipurpose or Hybrid Program)	CE	361*	1	361	.25	90.25
Awardee (Multipurpose or Hybrid Program)	Curriculum Development & Enhancement Form	704*	1	704	.25	176
Awardee (Multipurpose or Hybrid Program)	Faculty Development, Instruction & Recruitment Form	714*	1	714	.50	357
Awardee (Multipurpose or Hybrid Program)	State Oral Health Activities	35*	1	35	.50	17.5
Awardee (Multipurpose or Hybrid Program)	Progress Report	789	1	789	.50	394.5
Awardee (Multipurpose or Hybrid Program)	State Primary Care Offices	54*	1	54	.25	13.5
SUB-TOTAL		789		789	3.3632	2,653.6

*Note: Total number of respondents for Awardee Multipurpose/Hybrid Programs is 789; however, not all awardees are required to complete all forms due to the nature and purpose of their programs. Number of respondents may be equal to or less than 789 for any form. The completion of all required forms is considered a response to this data collection activity.

	Number of Respondents	Number of Responses per Respondent	Total Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Total	1,630	1	1,630	3.31436	5402.4

12B. Estimated of Annualized Cost to Respondents

Based on the estimated total number of burden hours, it is estimated that the annualized cost to respondents is approximately $146,837 (Table 5). This result was obtained by multiplying the number of burden hours by the average hourly wage rate of an individual employed in an academic setting. (Note: Wage rates were obtained from the Department of Labor. Average Hourly Rate for this labor category is $27.18, as of August 2021). Data collection and reporting activities are a grant requirement authorized under 45 CFR Part 74.

Table 5. Annualized Cost by Awardee Program

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Awardee (Direct Financial Support Program)	1,861.5	27.18	$50,595.57
Awardee (Infrastructure Program)	887.3	27.18	$24,116.81
Awardee (Multipurpose/Hybrid Program)	2,653.6	27.18	$72,124.85
Total	5,402.4		$146,837.23

(Hourly rate determined using Labor Category ID CES6500000008, Education and Health Services, August 2021).

13. Estimates of Capital Costs and Operations & Maintenance to Respondents or Recordkeepers.

There will be no capital costs or costs associated with operations and maintenance to respondents as all data are reported through a web-based enterprise system owned by and maintained at HRSA.

14. Annualized Cost to Federal Government

The systems used to collect information in the form of progress reports and the PRGCA are maintained by HRSA. It is estimated that the amount of staff time needed for the review and approval of progress reports and PRGCA submitted on an annual basis is equivalent to 2 FTEs at the GS-13 level—for a total of $207,380. Collectively, the estimated annualized cost to the government in staff time is estimated to be $207,380.

15. Explanation for Program Changes or Adjustments

There are currently 5,710 total burden hours approved by OMB for this activity. This request is for approval of roughly 5,402 burden hours, a decrease of over 300 hours. The decrease in burden is primarily due to 1) implementation of an Excel upload option for individual-level data reporting for a majority of awardees, 2) more precise estimates of the numbers of awardees using each form, and 3) National Center for Health Workforce Analysis scientists’ responsiveness to feedback on the data collection process and ongoing efforts to improve efficiency. Specifically, providing a majority of programs with the ability to enter and upload individual-level data via an Excel spreadsheet allows awardees to work offline and complete data entry more quickly. It also allows for copy/paste functionality from external sources. Awardees have provided very positive feedback about the Excel upload feature and have stated that it greatly reduces the time needed to complete their reporting requirements. Additionally, BHW plans to split profession/discipline questions into two parts. Currently, the number of selection options for this question is lengthy due to the large number of health professions/disciplines trained in BHW programs. Awardees reported that it was difficult and time consuming to find the option they were looking for. In response, BHW plans to create an intermediary question for “type” of profession (i.e., dental, medical, etc.) to make it easier for respondents to then locate the more specific discipline within the existing profession question. BHW has also maintained the significant burden reduction of over 1,700 hours in 2016 by continuing to use the annual reporting strategy.

Additional proposed changes and additions do not increase burden for awardees. In response to supplemental funding received to address the COVID-19 pandemic, BHW proposes the inclusion of a yes or no question (indicating COVID-19 supplement funding) on the CDE, CE, and FD forms. This is important because it will allow BHW to distinguish results of the COVID-19 supplement funding from discretionary appropriations. Similarly, as a result of the increased emphasis on telehealth, BHW proposes the addition of three questions to the INDGEN form. Applicable awardees will respond yes or no as to whether individuals received training in a telehealth setting. Awardees who also report the number of contact hours and the number of patient encounters for other settings (primary care, rural, and medically underserved communities) will be asked also to provide that information for telehealth settings. On the EXP form, BHW proposes including the street address so address data for clinical training sites can merge into HRSA’s Enterprise Site Repository (ESR) as street address is a required field. For clinical training sites already on the EXP forms, awardees will be offered a one-time opportunity to have the street address loaded into EHB for them. This will greatly reduce burden for the grantees. Lastly, in an effort to incorporate reporting for the National Health Service Corp – State Loan Repayment Program into the PRGCA, BHW proposes the addition of 18 program-specific questions on INDGEN and four program-specific questions on EXP-1. These questions are only for the State Loan Repayment Program. In the past, data for this program was collected through Excel spreadsheets that were emailed to awardees. Awardees and Project Officers are eagerly anticipating this upgrade as it will improve data accuracy, reduce reporting burden, and simplify integration with the National Health Service Corp Field Strength Dashboard - https://data.hrsa.gov/topics/health-workforce/field-strength.

There are approximately 20 different forms being used for reporting. These forms were redesigned in 2012, and they have provided BHW with exceptional results. More importantly they have improved the quality and accuracy of data reported. The EHB system continues to provide user-friendly templates on certain forms as well as pre-populated data fields on forms that do not utilize templates. The web-based reporting system features reduce the need for manual data entry thus reducing burden. The system pre-populates fields with previously entered data thus reducing data re-entry by the user; automate the calculation of total counts; and allow awardees the ability to enter data into spreadsheets that are available within the web-based reporting system. Several forms have the option to update previous information reported—reducing the need to re-report information which does not vary during the life of a specific grant. For example, demographic information about individuals receiving direct financial support (e.g., stipends, loans, or scholarships) will only have to be reported once. Using awardee-developed unique identifiers or NPI numbers, each awardee will only be required to update specific fields—such as financial award amounts, attrition status, graduation status, and 1-year follow-up.

In 2020, BHW launched HTML versions of PRGCA manuals. Moving from a PDF to an HTML version of the PRGCA manual provides awardees a more dynamic experience. Manuals can be updated in real-time and awardees can access them directly from the Electronic Handbooks (EHB). By linking the manuals within EHB, awardees automatically get directed to the section of the manual that relates to the form they are working on. These enhancements reduce time spent searching for answers to common questions. Manuals can also be updated in real-time to address confusion during the reporting cycle. The PRGCA manuals contain screenshots of HRSA’s Electronic Handbook (EHB) as well as specific instructions for awardees of each program. As each manual is tailored to the specific program reporting needs, awardees only see forms and items they must complete and are not confused by forms or fields that are not relevant to their grant program. The awardees have provided very positive feedback on the user-friendliness and the improved efficiency of the manuals when they are reporting performance data. The manuals as well as the applicable performance measures are available online at http://bhw.hrsa.gov/grants/reporting/index.html. These manuals serve to increase reliability and accuracy of the performance data and serve an important role in BHW’s performance management strategy.

16. Plans for Tabulation, Publication, and Project Time Schedule

Phase I: Mid-Year Progress Reports - Data collected in the form of progress reports will serve as the official record of communication between government project officers and awardees, and will be used to respond to awardee-specific concerns and technical assistance needs.

Phase II: End-of-Year Annual Performance Reports - Data collected in the form of PRGCA serves a number of important purposes including strengthening program performance; responding to Federal reporting requirements (e.g., GPRAMA); responding to Congressional inquiries. Since programs are publicly-funded, data collected through the PRGCA may be showcased in peer-reviewed articles, conferences, reports, and/or dashboards published through and/or sponsored by HRSA. The process for cleaning, analyzing, and reporting data will consist of the following steps⁴:

Step 1: Data completeness and accuracy. BHW utilizes a multi-level approach to ensure that data/information used for performance measures is complete. Awardees enter required data according to an established data reporting process which, for the majority of its workforce programs, includes reporting through HRSA’s grant system, Electronic Handbooks (EHB). During data entry in EHB, validation checks on data ensure report completeness and consistency across related measures. The awardee is notified during report completion of any errors that must be resolved prior to submission. Following report submission, government project officers (POs) review the performance report for accuracy and completeness. If the information is incomplete, the PO will request changes from the awardee and the report is sent back to the awardee. The awardee will then resubmit a report after addressing the PO’s concerns. Lastly, the data are reviewed again for completeness and accuracy by scientists in the National Center for Health Workforce Analysis who maintain the performance measures and are knowledgeable about program requirements. Deadlines for reporting are consistent with the Uniform Guidance (45 CFR 75) and shown in Table 3.

Data collected through performance reports serve a number of critical functions such as informing program management decisions, monitoring the types of activities implemented by awardees, and enhancing HRSA’s understanding about the diversity and distribution of the individuals receiving direct support from HRSA programs. In addition, performance reports are essential for:

• responding to Federal reporting requirements;

• understanding emerging issues in the health professions;

• ensuring compliance with statutory requirements, applicable regulations, and terms and conditions of award.

• identifying potential promising or best practices;

• strengthening overall program performance; and

• responding to congressional and public inquiries regarding outcomes associated with health professions training and loan programs.

The programs receive and review raw data from awardees, aggregate these data for HRSA-level reporting, have ongoing oversight of reporting entities and have direct contact with them to resolve potential data problems. For instance, if an awardee falls short of its performance targets, the PO will work to understand why and may work with the awardee to develop a corrective action plan. These data are also reviewed by the National Center for Health Workforce Analysis (NCHWA) scientists who use their data and evaluation expertise to assess how the data relate to national trends. In 2021, NCHWA launched Grantee Scorecards that allow awardees, POs, NCHWA scientists, and other BHW staff to see program performance as compared to program and national targets. The Scorecard provides a visual snapshot of performance that can be used for program management and evaluative purposes. In addition, measures are entered into the HHS Data Analytics System (DAS). It is this system (DAS) that is used to produce information for the HHS Annual Performance Report.

Step 2: Analysis⁵ & Reporting. The analysis of all PRGCA-related data will be conducted by doctoral-level scientists in NCHWA according to the following priority-based schedule:

1. Priority I. Data that is essential for performance management and budgeting will be analyzed with the highest priority. Results from these analyses will be provided to government project officers and BHW leadership in the form of briefs and/or reports within 30 days of NCHWA scientists completing the data cleaning process.

2. Priority II. Data that can be used to respond to inquiries from Congress, stakeholders, and/or the public will be analyzed and reported in accordance with the urgency of the request (usually 1 to 3 days).

3. Priority III. Data that can enhance the agency's understanding of emerging trends in the health professions will be analyzed and provided to BHW leadership in the form of dashboards, briefs, or presentations within 60 days of NCHWA scientists completing the data cleaning process.

4. Priority IV. Data that can be used to inform the development of articles or conferences will be analyzed and abstracts of findings will be provided to the requesting staff usually 2-4 weeks from the day of the request.

17. Exemptions for Not Displaying OMB Expiration Date

No exemption is requested. Respondents will see the OMB number and expiration date on each table that exists in the EHB system.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

This information collection fully complies with the guidelines set forth in 5 CFR 1320.9. There are no exceptions to the certification statement.

Attachments

• APPENDIX A: BHW-funded Health Professions Training and Loan Programs by Program Type

• APPENDIX B: Measures to Be Collected from Funded Awardees through Progress Reports

• APPENDIX C: Measures to Be Collected from Funded Awardees through the PRGCA

• APPENDIX D: 60-day Federal Register Notice

APPENDIX A

BHW-funded Health Professions Training and Loan Programs by Program Type

Public Burden Statement: The BHW Performance Report for Grants and Cooperative Agreements (PRGCA) is an annual performance and progress report required from each health professions and nursing education grantee that has an approved, funded project with a project period of one year or more. The report is required to determine the extent to which objectives of the project have been met so that a decision regarding continuation funding can be made. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this information collection is 0915-0061 and it is valid until 03/31/2025. This information collection is required to obtain or retain a benefit (Government Performance and Results Act (GPRA) of 1993 and the GPRA Modernization Act of 2010). Public reporting burden for this collection of information is estimated to average .50 hours per response to the progress report, including the time for reviewing instructions, searching existing data sources, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to HRSA Reports Clearance Officer, 5600 Fishers Lane, Room 14N136B, Rockville, Maryland, 20857 or [email protected].

5600 Fishers Lane, Room 10-29, Rockville, Maryland, 20857.

APPENDIX B

Progress Report

FORM NAME: PROGRESS REPORT

TYPE OF RESPONDENT: Awardees of Infrastructure Programs

Awardees of Direct Financial Support Programs

Awardees of Multipurpose or Hybrid Programs

SECTION I. PROJECT OBJECTIVES AND ACCOMPLISHMENTS⁶

Objective A

Description of Objective

Accomplishments

Objective B

Description of Objective

Accomplishments

Objective C

Description of Objective

Accomplishments

SECTION II. BARRIERS & RESOLUTIONS⁷

Barrier A

Description

Activities Taken to Resolve

Barrier B

Description

Activities Taken to Resolve

SECTION III. TECHNICAL ASSISTANCE NEEDS

Please identify any technical assistance needs that will assist your organization in meeting project objectives and/or improve performance.

APPENDIX C

PERFORMANCE REPORT FOR GRANTS AND COOPERATIVE AGREEMENTS

See attached documents. Attached are the performance measures and mapping documents. The mapping documents indicate which forms and items on the form that each awardee completes.

APPENDIX D

60-day Federal Register Notice

1 Pub. L. No. 103-62, 107 Stat. 285 (Aug. 3, 1993).

2 Pub. L. No. 111-352, 124 Stat. 3866 (Jan. 4, 2011).

3 See Appendix A for a complete listing of BHW funded programs by category.

4 Steps apply to each reporting period (FY 2016 and beyond). Please see Table 3 for an overview of beginning and ending periods of reporting.

5 The analyses of PRGCA data have historically been primarily descriptive in nature (e.g., frequencies, percentages, ratios).

6 Note: awardees will have the ability to list up to 9 objectives and related accomplishments.

7 Note: awardees will have the ability to list up to 9 barriers and related solutions.

Page 18 of 18