Summary of Public Comments

Attachment G: 60-Day FRN Public Comments and Responses

A 60-day Federal Register Notice was published in the Federal Register on May 20, 2011, vol. 71, No. 7; pp. 3383-84. There were 129 respondents to the Federal Register notice. A response to each category of comments received is provided below.

Topic: Clarify purpose of data collection and instructions for information collection instruments, including definitions of data elements and lead time needed to prepare for data collection.

Multiple comments were received requesting more information and clarification on the instructions pertaining to data tables for specific programs and various data elements. Respondents indicated concern about the lack of time being given to grantees to prepare for the proposed level of data collection. Respondents indicated that significant changes would be needed at the program level to design and implement a data collection system that would support the level of data collection proposed. Implementing data collection at the individual-level will likely require IRB approval which will add to the burden and also delay the ability to collect data.

To address this issue, BHPr held Web-based technical assistance sessions with grantees of each program and with federal advisory committees associated with the programs. Each comment and question presented by the grantees that participated was addressed. Suggestions made by the grantees and the federal advisory groups to improve the clarity of the instructions and definitions of data elements were carefully considered and the information collection forms were modified as appropriate. The Web-based sessions and other materials are archived on the HRSA website and accessible to grantees for use at any time. In addition, detailed information on the revised BHPr performance reporting system, including the grantee- and individual-level data elements and data collection, will be available on the HRSA BHPR Website (http://bhpr.hrsa.gov/grants/). The Website will include the instruction manual for the revised system that provides detailed information on all the reporting requirements and the submission process.

In order to respond to the new requirements set forth in Section 5103 of the Affordable Care Act (ACA), it is necessary for BHPr to implement the proposed data collection immediately. To meet these requirements, it is necessary for HRSA/BHPr to obtain data and information that will better describe and link BHPr program activities and program participant characteristics and experiences to their short-term and long-term outcomes related to their BHPR program involvement. Strengthening BHPr data collection in this way will also improve the agency's ability to better identify needed program improvements and to be accountable to the public.

Topic: Additional clarification needed about longitudinal evaluation data collection, including the need for informed consent and institutional IRB approval and burden estimate.

Several comments acknowledged the need for comprehensive performance data to respond to policymakers and decision makers and to ensure that the programs are effective in helping to achieve HRSA’s goals and objectives. However, respondents requested additional information about the individual-level data collection proposed and the need for informed consent and institutional IRB approval. Respondents also stressed the need for extensive communications between HRSA and the grantees prior to implementing the individual-level data collection about the use of the individual-level data over the long-term and justification to institutional IRBs of this data collection, including the development of an IRB form supplied by HRSA to use with the institutional IRBs. Multiple comments were received claiming that data collection at the individual level will be onerous and result in significant increased burden to grantees. Respondents also commented that HRSA has underestimated the level of burden for this data collection and the complexity of following program completers/graduates post-graduation to track participant outcomes.

BHPr acknowledges the added complexity of this additional data collection for the grantees, however, data collection at the individual-level is necessary to respond to the new requirements set forth in Section 5103 of the Affordable Care Act (ACA). The new requirements necessitate performance data that is descriptive of the BHPr programs, activities, and participant outcomes and able to show linkages between the program participants and program outcomes. Before establishing this data collection at the individual-level as a requirement across selected programs, BHPr will collaborate with the grantees and pilot test the proposed data elements and individual-level data collection with a subset of grantees. National associations will also be contacted by BHPr senior leadership to inform them of these activities and obtain feedback regarding the performance measures and data collection. In addition, BHPr is forming an expert panel to explore IRB requirements, determine data security safeguards, identify appropriate data elements for this level of data collection, and explore ways that will help incentivize program participants to submit this level of data. These discussions will help to determine the level of burden imposed on grantees and identify mechanisms to streamline the data collection process. At this time, BHPr intends to scale back its approach to implementing the individual-level data collection and limit it to those programs that require long-term tracking of trainees to determine trainee outcomes. The pilot test of this data collection is expected to begin in Spring 2012.