SUPPORTING STATEMENT
Part A
Evaluation of the Effectiveness of the Agency for Healthcare Research and Quality’s (AHRQ) Grant-Supported Research on Healthcare Costs, Productivity, and Market Forces
March 17, 2008
Revised, June 24, 2008
Agency of Healthcare Research and Quality (AHRQ)
Table of Contents
A. Justification 3
1. Circumstances that make the collection of information necessary 3
2. Purpose and use of information 5
3. Use of Improved Information Technology 6
4. Efforts to Identify Duplication 6
5. Involvement of Small Entities 6
6. Consequences if Information Collected Less Frequently 7
7. Special Circumstances 7
8. Consultation outside the Agency 7
9. Payments/Gifts to Respondents 7
10. Assurance of Confidentiality 8
11. Questions of a Sensitive Nature 8
12. Estimates of Annualized Burden Hours and Costs 8
13. Estimates of Annualized Respondent Capital and Maintenance Costs 9
14. Estimates of Annualized Cost to the Government 9
15. Changes in Hour Burden 10
16. Time Schedule, Publication and Analysis Plans 10
17. Exemption for Display of Expiration Date 10
A. Justification
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services and access to such services through the establishment of a broad base of scientific research and the promotion of improvements in clinical and health systems practices. The authorizing legislation states that AHRQ shall promote health care quality improvement by conducting and supporting:
• Research that develops and presents scientific evidence regarding all aspects of health care;
• The synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policymakers, and educators; and
• Initiatives to advance private and public efforts to improve health care quality.
Also, the authorizing legislation states that AHRQ shall conduct and support research and evaluations and demonstration projects with respect to (a) the delivery of health care in inner-city and rural areas (including frontier areas); and (b) health care for priority populations, including low-income and minority groups, women, children, the elderly, and persons with special health care needs, including individuals with disabilities, chronic health problems or end-of-life health care.
The proposed information collection will support AHRQ’s systematic review of the research that it has funded in the area of healthcare cost, productivity, organization, and market forces since 1998. AHRQ’s authorizing legislation lists research on “health care costs, productivity, organization, and market forces” as one of nine areas specified in the agency’s scope of work. The Center for Delivery, Organization, and Markets (CDOM) at AHRQ helps the agency fulfill its mission by overseeing most of this research activity.
In the first phase of its ongoing evaluation, AHRQ determined that it has funded 149 grants in the area of cost, productivity, organization and market forces since the late 1990s (Krissik, Lake and Gold, 2007). However, AHRQ does not have a system for identifying, categorizing, or tracking these studies. The agency also has no systematic way of knowing, or sharing with the health research field, what findings have been generated through its investigator-initiated grants, and how these findings can be accessed. Some AHRQ databases (Grant Information and Tracking System (GIAnT), Grants Online Database (GOLD), and Grants Reporting System (GRS)) track application submission, review, funding, award, and progress on grants and cooperative agreements. These databases interface with other Department of Health and Human Services databases, such as IMPAC. Most components of the AHRQ databases represent downloads of information from IMPAC, which is then augmented with AHRQ-specific coding. The need for the planned survey was identified based on the following points about the current AHRQ databases:
Only one database, GRS, gathers information directly from grantees. GRS provides a way for grantees to report progress within the period of funded activity. Unlike the proposed survey, progress reported in GRS is focused on specific and immediate project objectives, not on the long-term impact of the activities supported by grants. Furthermore, information reported in GRS is specific to defined initiatives (e.g., is defined by one-time Requests for Applications) rather than to the broad set of individually developed projects that are the target of the current evaluation.
Other databases (i.e., GIAnT, GOLD) do not rely on any input from grantees -- with the exception of grantees’ initial submissions of applications for grant support and their yearly submission of applications for continued grant support. This information, while important in defining the projects to be evaluated under the contract, does not provide an assessment of the impact of the research supported.
Agency databases do not track grants or classify grants specifically in accordance with the Agency's authorizing language "to support . . . research . . . on cost, productivity, organization, and market forces." These words and the impact of such research define the scope-of-work and the focus of the study that the survey supports. Such grant-supported research in the Agency is coordinated primarily in two program offices, is reviewed mainly by one external peer-review committee, and is developed and monitored most closely by several program officers. However, relevant projects for such grant-supported research are assigned to all Agency program offices, are reviewed by all four external peer-review committees, and are developed and monitored by program officers throughout the Agency.
In addition, prior to this evaluation, no assessment of these grants and their effects has been conducted. Collecting such information through a survey of the 149 grantees will assist AHRQ in its mission of supporting the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policymakers, and educators. The survey will help AHRQ better aggregate information so that it is available to the field and to private and public policy decision makers.
A survey of grantees represents a comprehensive and scientific approach to collecting data not currently available. This survey will be an integral part of AHRQ’s overall evaluation, which attempts to describe the research and pathways through which research findings are disseminated and used. Lacking an operational definition of the grants or identifying database, Mathematica Policy Research, Inc. (MPR), as AHRQ’s contractor, compiled a list of grantees that have received AHRQ support for research on cost, productivity, organization, or market forces. The proposed survey will involve interviews with principal investigators on the grant research projects and attempt to capture data that systematically track grant outcomes and provide information on: (1) the main substantive findings from the work and how that has been communicated; (2) known impacts of the work to date; (3) linkage of work to other research in the field; (4) grantee ratings of the support that AHRQ provided before, during, and after award and how service could be improved; and, (5) grantee perceptions of AHRQ’s role in research funding in this area and how sponsor interest influences the topics being addressed.
The survey, in addition to creating a better understanding of AHRQ-funded research in the areas of healthcare costs, productivity, organization and market forces, will also provide the agency with feedback on how their grantees view the support they receive from AHRQ and what ideas the grantees have about how processes relevant to the award, management, and dissemination of grants could be enhanced. The survey will inform AHRQ about what knowledge has been developed with these grants and how it can be accessed. The survey will also inform AHRQ as to what grantees have or have not done to support the dissemination and use of their findings (e.g., in what publications grantees have published and to what organizations they have presented). AHRQ’s grants aim to build the knowledge base on healthcare costs, productivity, organization and market forces and the survey will reveal whether this has occurred.
MPR will use the survey data to synthesize, analyze, and develop findings for a final report that will be submitted to AHRQ in September 2008. The report will provide information on how this research has been disseminated and used, what niche AHRQ has filled in this area, and how AHRQ’s internal processes can be improved to support grant work in this area, especially in dissemination. AHRQ may also use the survey data to develop a tracking system for documenting and synthesizing grant-supported research.
The data collection plan is designed to obtain reliable information in an efficient way that minimizes respondent burden. A web-based survey of AHRQ grantees will be used to obtain pertinent information. A self-administered mode was selected for this survey because respondents may not have information easily accessible for answering certain questions and it allows respondents to complete the survey at their own pace and schedule. The web mode was selected because the survey has a number of open-ended questions and respondents may find completing a web-based questionnaire is less burdensome than handwriting responses. However, if a respondent prefers to be mailed a hardcopy questionnaire, MPR will do so. Respondents will also have the option of printing a hardcopy version of the survey from the web if they wish to do so.
No other survey data collection effort has been conducted by AHRQ or has been planned to collect similar information about this group of sample members. This survey is being conducted because AHRQ’s grant tracking system provides no way of systematically tracking outcomes once grants are over. AHRQ staff worked with MPR to convene a technical expert panel and provide guidance and support. The panel encouraged that a survey of the grantees be conducted.
This data collection will produce no burden on small businesses or other entities.
This is a one-time data collection. If the collection is not conducted, AHRQ will not be able to track grant outcomes of these grantees.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on December 28th, 2007, Vol. 72, page 73825, for 60 days (see Attachment B). No comments were received.
AHRQ has twice convened an expert panel to provide guidance on the survey (including reporting format and data elements). The panel consisted of a balanced set of leaders in the health services research field who are knowledgeable about the dissemination of economic research beyond a research audience as well as its use in the policy process. Panel members were: Sharon Arnold of Changes in Health Care Financing and Organization (HCFO), Jon Christianson of the University of Minnesota, Paul Ginsburg of the Center for Studying Health Systems Change, Robert Helms of the American Enterprise Institute, and Gail Wilensky of Project HOPE.
The study is not planning to provide payments or gifts to respondents.
10. Assurance of Confidentiality
Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be informed of the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.
Individuals and organizations contacted will be further assured of the confidentiality of their replies under 42 U.S.C. 1306, and 20 CFR 401 and 4225 U.S.C.552a (Privacy Act of 1974). In instances where respondent identity is needed, the information collection will fully comply with all respects of the Privacy Act.
Respondents will be informed that their feedback on AHRQ’s performance will be kept confidential and be reported to AHRQ only in summary form and that information they provide on publications resulting from their research and other grant outcomes will be made available to AHRQ and may be reported publicly. This information is presented to respondents in an advance letter (see Attachment C), an email with a web link to the survey (see Attachment D), and the survey instrument (see Attachment E) prior to their beginning the survey.
None of the forms used in the data collection contain items considered to be of a sensitive nature. Respondents are informed about the nature of the study, that their participation is voluntary, and that there are no known benefits, risks, or other consequences to participation.
The total estimated reporting time burden for respondents is 298 hours. This estimate is based on an estimated two hours of time per respondent to complete the questionnaire, including follow-up to some respondents. The survey was pretested with fewer than 10 respondents to learn about problems respondents might have with the questionnaire and to assess the time burden. The estimated reporting burden time is based on the burden time reported by pretest respondents.
Exhibit 1. Estimated Annualized Burden Hours
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Hours per Response |
Total Burden Hours |
AHRQ Grantee Survey |
149 |
1 |
2.0 |
298 |
Total |
149 |
na |
na |
298 |
There is no direct cost to individual participants. The total burden cost of collecting this information is $12,808. This cost represents two hours to complete the survey multiplied by the number of completers and by an estimated hourly wage of $42.98.
Exhibit 2. Estimated Annualized Cost Burden
Form Name |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage Rate* |
Total Cost Burden |
AHRQ Grantee Survey |
149 |
298 |
$42.98 |
$12,808 |
Total |
149 |
298 |
na |
$12,808 |
*Based upon the mean of the average wages for teachers (college and university), National Compensation Survey: Occupational Wages in the United States 2005, U.S. Department of Labor, Bureau of Labor Statistics.
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
The proposed information collection is part of a larger evaluation of the effectiveness of AHRQ’s grant-supported research on healthcare costs, productivity, and market forces, which includes a systematic review of the research that AHRQ has funded, in-depth interviews with grantees and grant document review, case studies to assess the effects and dissemination pathways of market forces research, and preparation of reports and briefings. The cost to conduct the survey of identified grantees is $38,962.
This is a new collection of information.
The anticipated conclusion date for the project is September 30, 2008. MPR will draft and finalize a report and executive summary, and deliver an oral briefing to AHRQ staff. The report will contain an analysis using descriptive statistics of the survey results in addition to case studies that will be conducted as part of MPR’s evaluation. The report will profile AHRQ’s work in this area of research, how it is disseminated and used, and implications for future efforts.
AHRQ does not seek this exemption.
Attachments:
Attachment A: AHRQ's Authorizing Legislation
Attachment B: 60-Day Federal Register Notice
Attachment C: Advance Letter from AHRQ
Attachment D: Advance E-mail from MPR
Attachment E: Questionnaire/Data Collection Instrument
References
Krissik, Tara, Tim Lake, and Marsha Gold. “Evaluation of the Effectiveness of AHRQ’s Grant-Supported Research on Health Care Costs, Productivity, and Market Forces: Interim Report.” Washington, DC: Mathematica Policy Research, Inc., November 2007.
| File Type | application/msword |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| Last Modified By | AHRQ |
| File Modified | 2008-06-25 |
| File Created | 2008-06-25 |