SUPPORTING STATEMENT
Part A
The Agency for Healthcare Research and Quality (AHRQ)
Health Care Innovations Exchange
Innovator Interview and Innovator Email Submission Guidelines
OMB CONTROL NO. 0935-0147
Version: December 19, 2013
Agency for Healthcare Research and Quality (AHRQ)
Table of Contents
Introduction……………………………………………………………………………..3
A. Justification 3
1. Circumstances that Make the Collection of Information Necessary 3
2. Purpose and Use of Information 7
3. Use of Improved Information Technology 8
4. Efforts to Identify Duplication 8
5. Involvement of Small Entities 9
6. Consequences if Information Collected Less Frequently 9
7. Special Circumstances 9
8. Federal Register Notice and Outside Consultations 9
9. Payments/Gifts to Respondents 10
10. Assurance of Confidentiality 10
11. Questions of a Sensitive Nature 11
12. Estimates of Annualized Burden Hours and Costs 11
13. Estimates of Annualized Respondent Capital and Maintenance Costs 12
14. Estimates of Annualized Cost to the Government 12
15. Changes in Hour Burden 13
16. Time Schedule, Publication and Analysis Plans 13
17. Exemption for Display of Expiration Date 14
List of Attachments 14
This request for Office of Management and Budget (OMB) review is for renewal of the existing collection that is currently approved under OMB Control No. 0935-0147, AHRQ Health Care Innovations Exchange Innovator Interview and AHRQ Health Care Innovations Exchange Innovator Email Submission Guidelines, which expires on May 31, 2014.
The mission of the Agency for Healthcare Research and Quality (AHRQ), set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
The 2012 AHRQ National Healthcare Quality Report1 emphasized the continuing need to accelerate progress if the Nation is to achieve higher quality and more equitable health care. The report indicated that:
Health care quality and access continue to be suboptimal, especially for minority and low-income groups;
Urgent attention is warranted to ensure continued improvements in the quality of diabetes care, maternal and child health care, and adverse events; disparities in cancer care; and quality of care among states in the South.; and
Overall quality is improving, but access is getting worse and health care disparities are not changing.
The health care environment is ripe for novel changes to shape how health care is delivered and funded.2 Innovation is at the heart of these novel changes and AHRQ seeks to find health care service delivery and policy innovations and support their accelerated diffusion and adoption, with the goal of having a more profound impact on improvements in the quality of the nation’s health care and significant reductions in its disparities.
The Health Care Innovations Exchange provides a national-level information hub to foster the implementation and adaptation of innovative strategies and policies that improve health care quality and reduce disparities in the care received by different populations. The Innovations Exchange’s target audiences, broadly defined, are current and potential change agents in the U.S. health care system, including clinicians (e.g., physicians, nurses, and other providers), health care administrators, quality improvement professionals, researchers, educators, and policymakers.
The goals of the Health Care Innovations Exchange are to:
1) Identify health care service delivery and policy innovations and provide a national level repository of searchable innovations and tools that enables health care decision makers to quickly identify ideas and tools that meet their needs. These innovations come from many care settings including inpatient facilities, outpatient facilities, long term care organizations, health plans, and community care settings. They also represent many patient populations, disease conditions, and processes of care such as preventive, acute, and chronic care.
2) Foster the implementation and adoption of health care service delivery and policy innovations that improve health care quality and reduce disparities in the care received by different populations.
To achieve the first goal of the Innovations Exchange the following data collections will be implemented:
Email submission – Based on experience during the current approval period, approximately 10% of the health care innovations considered for inclusion annually, and their associated innovators, will submit their innovations via email to the Innovations Exchange without prior contact (about 8 annually). Innovators who submit their innovations for possible publication through the email submission process (see Attachment B) will be considered as will innovations identified by project staff through an array of sources that include: published literature, conference proceedings, news items, list serves, Federal agencies and other government programs and resources, health care foundations, and health care associations.
To meet the publication target of 75 new innovation profiles per year, a purposive sample of approximately 76 health care innovations will be identified and selected annually, in addition to the email submissions, for a total of 84 innovations considered annually for potential consideration. These innovations will be selected to ensure that innovations included in the Innovations Exchange cover a broad range of health care settings, care processes, policies, priority populations, and clinical conditions. Based on experience, approximately 10% of the candidate innovations either will not meet the inclusion criteria or their innovators will decide not to continue their participation after the interview. Therefore, 90% (75) of the 84 candidate innovations will move into the publication stage each year.
2) Health care innovator interview – To collect and verify the information required for the innovation profiles, health care innovators will be interviewed by telephone (see Attachment C) about the following aspects of their innovation: health care problem addressed, impetus for the innovation, goals of the innovation, description of the innovation, sources of funding, evaluation results for the innovation, setting for the innovation, history of planning and implementation for the innovation, and lessons learned concerning the implementation of the innovation. Interviews will be conducted with innovators identified by project staff and those identified through email submission.
3) Annual follow-up reviews – After the innovation profile is published, on a yearly basis, innovators will be contacted by email to review and update their profiles (see Attachment D).
The second goal of the Innovations Exchange is achieved by serving as a “one-stop shop” that provides:
1) Digested and reliable information about innovations in an adoption-friendly format;
2) Learning resources including expert commentaries, articles, adoption guides and educational Web events, and
3) Networking opportunities that allow innovators and potential adopters to share information about implementation strategies and lessons learned, including in-person meetings, interactive online events, and the ability for users to post comments on specific innovations.
To be included in the Innovations Exchange all identified health care delivery and policy innovations must meet six inclusion criteria with respect to the nature of the activity, the level of documentation, and the participation of the innovator, which are minimum requirements to participate.
The six inclusion criteria are:
The innovation relates to patient care delivery processes or involves health care policy.
The innovation aims to improve one or more domains of health care quality.
The activity or policy is innovative in the context of its setting or target population.
Information about the innovation is publicly available.
The innovator (or a representative) will contribute information to the Health Care Innovations Exchange.
There is reason to believe that the innovation will be effective.
Over the course of the current approval period, these six inclusion criteria were modified slightly to include health care policy innovations, which were added to the Web site in 2011. Project staff have continued to refine their understanding of these criteria and their ability to apply them.
In addition to criteria for Innovation Profiles, the Innovations Exchange also solicits Innovation Attempts (i.e., innovations that did not work/were not effective). They are separated in the database and each type of innovation is clearly labeled. The distinction between Innovation Profiles and Attempts is clearly explained in the FAQs displayed on the Web site at http://www.innovations.ahrq.gov/faq.aspx.
Innovations that fall into any one of the following categories are excluded from the Innovations Exchange:
Product or technical innovations;
Educational innovations;
Clinical innovations; and
Innovations without any evidence of effect.
The rationale for excluding submissions is documented based on the criteria above, and submissions are inventoried monthly and provided to AHRQ for review.
The ultimate decision to publish a detailed profile of an innovation depends on several factors, including an evaluation by AHRQ, AHRQ’s priorities, and the number of similar ideas in the Innovations Exchange. AHRQ’s priorities include identifying and highlighting innovations (1) that will help reduce disparities in health care and health status; (2) that will have significant impact on the overall value of health care; (3) where the innovators have a strong interest in participating; and (4) that have been supported by AHRQ. The AHRQ priorities are provided as a part of the instructions to applicants found in the “Submit Your Innovation” section of the Web site, accessible at: http://www.innovations.ahrq.gov/submit/submit.aspx?#1. Of note, however, while the AHRQ Health Care Innovations Exchange is a database of innovative interventions, it is not intended to be an exclusive showcase for AHRQ’s funded projects.
AHRQ’s definition of “innovative” is provided as a part of the instructions to applicants, as shown in the “Submit Your Innovation” section of the Web site at: http://www.innovations.ahrq.gov/submit/submit.aspx?#1
As noted above, AHRQ aims to provide reliable information about innovations. Each innovation is systematically assigned a level of evidence, which is prominently shown on each profile in the Summary section. The Innovations Exchange has three categories: strong, moderate and suggestive, described on the Web site at: http://www.innovations.ahrq.gov/evidencerating.aspx
Over the course of the current project period, AHRQ has gained experience that has allowed project staff to refine processes and increase project efficiency. For example, experience with the profile development process has provided valuable insight for identifying new leads and conducting interviews, and the annual review process has been streamlined to reduce the number of hours required for project staff to update profiles. The information contained within this request for renewal is based on actual project experience.
These research activities are not required by regulation, and will not be used by AHRQ to regulate or sanction its customers.
The Innovations Exchange was recently re-certified by the Health on the Net Foundation, which promotes and guides the deployment of useful and reliable online health information3.
This data collection is being conducted by AHRQ through its contractor, Westat, pursuant to AHRQ’s statutory authority (1) to conduct and support research on, and disseminate information on, health care and on systems for the delivery of such care, 42 U.S.C. 299a(a), and (2) to promote innovation in evidence-based health care practices and technologies by promoting education and training and providing technical assistance in the use of health care practice results, 42 U.S.C. 299b-5(a)(4).
Users of the site may contact program staff through the [email protected] e-mail address. The address is staffed during business hours. In most cases correspondents receive a personalized reply within one business day. In addition, for the assistance of users, the Web site offers frequently asked questions (FAQs) related to an array of questions in part that have been submitted by users and include: Submitting an Innovation, Subscribing to Email Updates, QualityTools, and Other Topics. The complete list of FAQs is available at http://www.innovations.ahrq.gov/faq.aspx.
Access to the AHRQ Health Care Innovations Exchange is freely available to the public at http://www.innovations.ahrq.gov/. Diverse groups use the Innovations Exchange, ranging from nurses and health administrators, quality improvement professionals, researchers and educators. See http://www.innovations.ahrq.gov/about.aspx which displays information about Innovations Exchange users by role for 2012-2013.
The AHRQ Health Care Innovations Exchange uses the services of an independent company, ForeSee, to conduct randomized customer satisfaction surveys as a means to evaluate user satisfaction. Satisfaction scores for the Innovations Exchange have consistently averaged close to or above 80 over the course of the project period. As a point of comparison, the average score for government sites for Q2 2013 was 75, which is cited as an all-time high in customer satisfaction with Federal government Web sites. Sites that score over 80 are cited as high performers.
ForeSee customer satisfaction surveys also provide a mechanism for users to submit free text comments about their experience using the Web site.
Additional sources of evaluation can be derived from the innovators themselves, since a part of the data collection process includes gathering their names and contact information to post on the Innovations Exchange Web site in an effort to encourage collaboration.
In addition, AHRQ intends to gathers user feedback through:
Social media strategies (e.g., Twitter)
A virtual meeting of innovators
Web event evaluations
User comments submitted within Innovation Profiles and Innovation Attempts
Emails submitted through the email line ([email protected]).
Web site usability testing
The Innovations Exchange currently offers guidelines to innovators on how to submit their innovations by email for the purposes of providing project staff with preliminary information about their innovations. Innovators are provided with an email address to which they are asked to send a description of the innovation.
Based on experience during the current project period, about 10% of innovation profiles are developed based on innovators submitting their innovations by email without prior contact.
Duplication of this effort neither exists with respect to other Federal agencies nor with respect to the private sector. There are a number of Web sites that present information on health care innovation, sponsored by educational institutions, foundations, and the private sector. These Web sites tend to be limited in their focus. They may consider innovations that focus on one or a few diseases, funding sources, patient populations, or theoretical foundations. A list of health care innovation-oriented Web sites is located in Attachment E.
The Innovations Exchange is the only readily accessible, single source of information concerning innovations in patient care service delivery and policy, multiple patient populations, multiple care settings, multiple diseases, multiple care processes, and multiple funding sources.
While we anticipate that innovators within small entities will continue to be included in the pool of respondents, participation in and submissions to the Innovations Exchange are voluntary. We do not expect that our data collection procedures will result in any significant additional burden for small entities. Rather the data collection efforts are designed to minimize burden on all respondents.
Without this data collection effort and annual updates, detailed information about current innovations in health care service delivery and policy would not be available in one location. It is essential that AHRQ continues these data collection activities, which are even more important in the context of health care reform.
This data collection will be consistent with the general information collection guidelines of 5 CFR 1320.4(d)(2). No special circumstances apply.
8. a. Federal Register Notice
In developing this project AHRQ consulted a broad range of experts, including staff in-house, in other Federal agencies such as the (National Library of Medicine, National Institutes of Health), and in other organizations who have extensive expertise in health service delivery systems, workflow processes, identifying and disseminating innovative processes across industries, and in health classification systems. Ongoing consultations in these areas are necessary to guide appropriate practices for identifying, organizing, classifying and disseminating innovations that result in health care quality improvements and disparities reduction. Several examples of expert consultations in support of the Innovations Exchange include the following.
Guided by AHRQ officials, Westat, the contractor for this project, consulted targeted senior AHRQ staff to identify prospective program sources that might yield eligible innovations through AHRQ grant and contract programs. Additionally, AHRQ staff have been instrumental in helping to establish criteria that refine various definitions of innovation, degrees of innovativeness, and applications of innovations in addressing various care processes in diverse clinical and community-based settings.
Silverchair, a subcontractor to Westat, is a nationally recognized health information technology firm with a substantial track record in developing high-value, information-critical products for the health care community. For the Innovations Exchange, Silverchair’s role is to help create domain specific taxonomies, facilitate the adaptation of legacy systems, and to operate and maintain the Web site. Supported by a cadre of professional medical indexers, Silverchair provides assistance with taxonomy refinement and semantic tagging.
Dr. Brian Mittman, PhD, an internationally regarded senior social scientist with expertise in implementation science, provides insight into service systems for veterans via his position as senior scientist at Kaiser Permanente Southern California and senior advisor for the VA Center for Implementation Practice and Research Support, Department of Veterans Affairs.
Dr. Herbert Smitherman, MD, MPH, FACP, a practicing physician and President/CEO of Health Centers Detroit Foundation, Inc., a Federally Qualified Health Center Look Alike, provides expertise in underserved and uninsured populations and their access to appropriate health care.
Tamra Minnier, RN, MS, FACHE, Chief Quality Officer for the University of Pittsburgh Medical Center, an integrated health care delivery system with over 18 hospitals and a 1.3 million member health plan, provides expertise in nursing and quality improvement.
Dr. Lisa Simpson, MB, BCh, MPH, President/CEO of AcademyHealth and nationally recognized health policy researcher and pediatrician, provides leadership to the project as Chair of the Expert Panel. In this capacity, Dr. Simpson facilitates dialogue and garners input from a broad range of health care expert in support of the Innovations Exchange.
The above descriptions represent a few examples of the experts consulted, the types of issues discussed and queries that have been made. Attachment G provides a more extensive list of experts who have been engaged during both the development and ongoing operations of the Innovations Exchange project.
9. Payments/Gifts to Respondents
No payments or gifts will be offered to respondents.
No assurance of confidentiality will be made to respondents.
No questions of a sensitive nature are included in the interview guide or email submission guidelines.
Exhibit 1 shows the estimated annualized burden hours for the respondents’ time to participate in this project. Approximately 84 innovators will participate in the initial data collection each year with 75 of those being published to the Innovations Exchange Web site. About 8 innovations will be submitted by email, which requires 30 minutes. All 84 potential innovators will participate in the health care innovator interview, including the 8 submitted via email. The interview will last about 75 minutes, and an average additional 30 minutes is typically required for the innovator to review, comment on, and approve the written profile.
Based on experience, approximately 10% of the candidate innovations either will not meet the inclusion criteria or their innovators will decide not to continue their participation after the interview. Therefore, 90% (75) of the 84 candidate innovations will move into the publication stage each year. Annual follow-up reviews will be conducted with all innovations that have been in the Innovations Exchange for at least one full year. With an expected total of 825 innovations in the Exchange by the end of the current approval period, and an additional 225 to be added over the course of the next 3-year approval period (75 per year), an average of 800 reviews will be conducted annually4 and will require about 15 minutes to complete. The total annualized burden is estimated to be 347 hours.
Exhibit 1: Estimated annualized burden hours
Form name |
Number of Respondents |
Number of Responses per Respondent |
Hours per response |
Total Burden Hours |
Email submission |
8 |
1 |
30/60 |
4 |
Health care innovator interview |
84 |
1 |
75/60 |
105 |
Innovator review and approval of written profile |
75 |
1 |
30/60 |
38 |
Annual follow-up reviews |
800 |
1 |
15/60 |
200 |
Total |
967 |
— |
— |
347 |
Exhibit 2 shows the estimated annualized cost burden associated with the respondents’ time to participate in this project. The total annualized cost burden is estimated to be $21,220.
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of Respondents |
Total Burden hours |
Average Hourly Wage Rate* |
Total Cost Burden |
Email submission |
8 |
4 |
$61.15 |
$245 |
Health care innovator interview |
84 |
105 |
$61.15 |
$6,421 |
Innovator review and approval of written profile |
75 |
38 |
$61.15 |
$2,324 |
Annual follow-up reviews |
800 |
200 |
$61.15 |
$12,230 |
Total |
967 |
347 |
-- |
$21,220 |
*Average hourly wage rate for health care innovators is based upon statistics from the Bureau of Labor Statistics, U.S. Department of Labor, Occupational Employment and Wages, May 2012 (http://www.bls.gov/oes/current/oes290000.htm), and was calculated as an average of the mean hourly wage rate for Family and General Practitioners and the mean hourly wage for all occupations in the major group, “Healthcare Practitioners and Technical Occupations”.
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
Exhibit 3a shows the estimated annualized costs to the Government. The total cost to the Government of this data collection is approximately $314,729 over three years (on average, $104,910 per year). These costs cover data collection efforts for contacting candidate health care innovators, conducting innovator interviews, and contacting innovators annually to update profiles.
Exhibit 3a. Estimated Total and Annualized Cost
Cost Component |
Total Cost |
Annualized Cost |
Data Collection Activities |
$89,886 |
$29,962 |
Website Maintenance |
$64,173 |
$21,391 |
Project Management |
$13,053 |
$4,351 |
Overhead |
$147,618 |
$49,206 |
Total |
$314,730 |
$104,910 |
|
|
|
The final innovation profiles are reviewed and approved by the designated Program Analyst that monitors the program. The profiles are reviewed by the Program Analyst before they are moved into the publication stage. The average hourly salary for the position of Public Health Analyst at the GS-13-Step 7 grade level is $51.71 per hour. The Federal hourly salary information is available on the OPM web site at https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2014/DCB_h.pdf.
Exhibit 3b
|
Total Responses |
Estimated Review Time |
Estimated Hourly Federal Salary |
Total Cost Burden |
Innovation Profiles |
75 |
1 hr |
$51.71 |
$5,171.00 |
Total Burden |
75 |
1 hr |
$51.71 |
$5,171.00 |
Schedule. This data collection is scheduled for June 2014 through May 2017. The data collection follows a four-month cycle, shown below in Table 1. The Innovations Exchange will publish new profiles to the Web site every two weeks, averaging 25 publication issues a year. AHRQ will publish approximately 3 new profiles per issue for a total of 75 innovations per calendar year. The timeline includes reviewing material from the email submission or identifying the potential innovation and conducting background research, contacting the innovator and conducting the innovator interview, developing the profile, requesting innovator review, and publishing the innovation to the Innovations Exchange.
Activity |
Time schedule |
Background research on innovation |
4 months prior to publication |
Innovator interview |
3 months prior to publication |
Develop profile |
2 months prior to publication |
Innovator review |
1 month prior to publication |
Publish profile |
Every 2 weeks |
Analysis. The Innovations Exchange portfolio of innovations will be systematically reviewed to ensure a broad representation of various settings of care, patient populations, disease conditions, and processes of care.
AHRQ does not seek this exemption.
List of Attachments
Attachment A: Healthcare Research and Quality Act of 1999
Attachment B: Innovator Email Submission Guidelines
Attachment C: Health Care Innovator Interview Guide
Attachment D: Annual Follow-up Guide
Attachment E: Web Sites that Offer Health Care Quality Improvement and Innovation Information
Attachment F: Federal Register notice
Attachment G: Expert Consultation for the AHRQ Health Care Innovations Exchange
1Agency for Healthcare Research and Quality. 2012 National Healthcare Quality Report. Rockville, MD: U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality; May 2013. AHRQ Pub. No. 13-0002. http://www.ahrq.gov/research/findings/nhqrdr/nhqr12/2012nhqr.pdf. Accessed 10/31/2013.
2Dougherty D, Conway PH. The “3T’s” Road Map to Transform US Health Care: The “How” of High-Quality Care. JAMA, May 21, 2008—Vol 299, No. 19
3https://www.healthonnet.org/HONcode/Conduct.html?HONConduct434989
4 The number of profiles undergoing annual review will increase annually from 825 in the first year, to 900 in the second year, and 975 in the third year. The average annualized number of annual follow-up reviews is projected to be 800 as it is anticipated that approximately 100 profiles will be archived over three years. Archived profiles are excluded from annual review.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| File Modified | 0000-00-00 |
| File Created | 2021-01-27 |