SUPPORTING STATEMENT
Part A
Evaluation of the Emergency Severity Index
Version 1Revised - March 27July 31, 2008
Agency of Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 3
1. Circumstances that make the collection of information necessary 3
2. Purpose and use of information 4
3. Use of Improved Information Technology 5
4. Efforts to Identify Duplication 5
5. Involvement of Small Entities 5
6. Consequences if Information Collected Less Frequently 6
7. Special Circumstances 6
8. Consultation outside the Agency 6
9. Payments/Gifts to Respondents 6
10. Assurance of Confidentiality 7
11. Questions of a Sensitive Nature 7
12. Estimates of Annualized Burden Hours and Costs 8
13. Estimates of Annualized Respondent Capital and Maintenance Costs 9
14. Estimates of Annualized Cost to the Government 9
15. Changes in Hour Burden 9
16. Time Schedule, Publication and Analysis Plans 9
17. Exemption for Display of Expiration Date 10
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
AHRQ is proposing to examine uptake and use of an emergency room triage tool, the Emergency Severity Index (ESI). The hospital emergency department (ED) represents a critical point in care delivery for patients across the United States. According to the Institute of Medicine, Over the past decade, however, the dramatic influx of patients into EDs has seriously challenged the ability of these departments to deliver timely, quality, and safe emergency health care services.1 Moreover, with most emergency departments operating at or over capacity it may prove difficult for them may find it difficult to respond to the surge in emergency room demand created by natural and man-made disasters. Development of increasingly refined and validated triage methods is one potential key to addressing overcrowding by speeding up the care delivery to the most acute ED patients while helping hospitals assess, carefully allocate and plan the amount of human and other resources needed to care for all patients. The ESI is an emergency department (ED) triage tool developed in 1995 by Richard C. Wuerz, MD (Department of Emergency Medicine at the Brigham and Women’s Hospital and the Harvard Medical School) and David R. Eitel, MD (Department of Emergency Medicine, The York Hospital WellSpan Health System) in response to a need to standardize the triage process and improve the flow of patients. The ESI is unique in its focus on appropriate resource allocation and its consideration of necessary resource utilization in assigning acuity. To encourage adoption of the ESI, AHRQ developed an implementation handbook (Emergency Severity Index, Version 4) and companion DVDs. These materials are intended to provide hospitals and triage nurses with background on the ESI, and offer recommendations on the implementation process and staff training.
In concert with our partners at The George Washington University (GWU), and with oversight from AHRQ, NORC is conducting an assessment of the ESI training materials developed by AHRQ, which include an implementation handbook (Emergency Severity Index, Version 4) and companion DVDs. AHRQ is seeking approval from the Office of Management and Budget to conduct:
A short survey using a self-administered questionnaire (SAQ) offor ED clinicians and managers that requested a copy of the ESI training materials from AHRQ, and agreed to be contacted to participate in a survey about the ESI; and
Four focus groups with 8 ED professionals in each focus group.
This study supports AHRQ’s mission to foster improvements in clinical and health systems practices and to improve health care quality. The assessment will develop new evidence about the uptake and adoption of the ESI triage system in EDs across the U.S. The results of the assessment will be used to provide a synthesis of information about the uptake of the ESI in EDs for use by patients, clinicians, researchers, and policymakers.
The purpose of this data collection is fourive-fold:
To measure the acceptance of the training materials by EDs and others;
To measure the satisfaction with the presentation, content, and clarity of the training materials;
To determine the extent to which the materials have improved emergency services, surge planning and preparation;
To compare usefulness of the ESI with other similar triage tools; and
To determine what improvements users would like to see in the next version of the products.
The survey will assess the extent to which the ESI has been implemented by EDs and the factors associated with implementation; satisfaction with the content of the ESI training materials; the extent to which the product has improved emergency services and surge planning; the usefulness of the ESI as compared to other triage tools; and the improvements users would like to see in the next version of the ESI (Attachment B).
The focus groups with ED professionals will serve as a complement to the survey, enabling AHRQ to learn in greater detail about acceptance of the ESI tool in EDs; the characteristics that might predict uptake and use of the ESI in EDs; the usefulness of the ESI compared to other triage tools; recommendations for improving up-take; and familiarity regarding AHRQ’s role in ED surge planning (Attachment C).
AHRQ will use the study findings to advance its dissemination of the ESI tool, in light of any identified barriers to and/or drivers for the ESI’s implementation. AHRQ will also use the findings to revise the ESI tool in its subsequent versions. Findings from this study will help to guide AHRQ’s efforts in enhancing and disseminating the ESI tool, and to provide AHRQ with guidance related to its emergency medical surge planning activities.
Information from this study will also be useful to ED clinicians, managers, and administrators who are considering implementing the ESI.
The collection of information will use a self administered questionnaire (SAQ) that involves the use of technological collection techniques. The SAQ questionnaire will be completed by people who have requested a copy of the ESI training tools from AHRQ, and indicated that they would be willing to be contacted to participate in a survey about the ESI. Individuals who requested the tool provided contact information and indicated their preferred survey mode (hard-copy via the mail, email, and telephone). We are permitting all respondents to electronically submit their responses to the survey via the online version of the survey – regardless of their initial stated preference in the AHRQ ESI tool requester’s database. We will also accommodate participants’ preferences should they change. It is anticipated that respondents will choose the option of least personal burden, thereby reducing the overall burden of the study.
NORC conducted a literature review, and the search did not identify any systematic studies of the acceptance of and satisfaction with the ESI tool in EDs across the U.S. Researchers have tested the validity and reliability of the ESI algorithm at a small number of ED sites.2,3,4,5 While these studies found that the ESI is a reliable, valid, and useful tool, they have not explored its up-take and use in EDs across the U.S. Further, AHRQ, NORC, GWU, and consulted experts in ED care are unaware of any research that explores the up-take of the ESI tool in EDs across the U.S. We are not aware of any studies that explore why some EDs have implemented the ESI and others have not. No assessments have been conducted to explore the satisfaction with the AHRQ-developed ESI educational training handbook and accompanying DVDs and the impact of these materials on the adoption of the ESI. The proposed research is the first of its kind, and is needed to provide AHRQ with information that can be used to improve the ESI in subsequent versions of the tool, and to help AHRQ to understand the uptake of the tool nationally and whether EDs are using the ESI to plan for every day medical surges.
No small entities will be involved in this study.
The design of this study requires only one data collection activity per respondent. There are no technical or legal obstacles to reducing burden. Without collecting this data, AHRQ will not have access to a comprehensive assessment of the acceptance of and satisfaction with the ESI in EDs, the barriers to its implementation, the uses of ESI in medical surge planning, and the overall utility of the ESI educational handbook and training DVDs. The federal government will benefit from having information available about whether EDs are implementing the ESI, and the satisfaction with the ESI educational handbook and DVDs. Additionally, without this data collection, AHRQ will not know whether the ESI is effective in improving medical surge planning in EDs. Furthermore, this study is needed to provide valuable information about how aware ED clinicians and managers are of AHRQ and its role in ED surge planning. The data collection will allow AHRQ to enhance the ESI tool and improve the agency’s ability to disseminate information about the ESI to EDs.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on January 22, 2008 on pages 3726 – 3727 for 60 days (Attachment D). No comments were received.
NORC consulted with its partners at the George Washington University and a range of ED professionals to obtain their views about the type of data to be collected using the survey instrument and focus group protocol. There are no unresolved issues. A list of the outside consultants can be found in Attachment E of this document.
One honoraria payment in the amount of $75100 will be made to each ED clinician or manager that participates in the focus groups. ED health care professionals have competing priorities for their time and effort. The use of a monetary incentive has been employed as an effective strategy for increasing response rates among medical professionals. Blumenthal (2007) conducted focus groups with physicians and nurses, providing a $100 honorarium for physicians and a $50 honorarium for nurses, plus the cost of meals, to recognize their participation in a 60 to 90 minute focus group.6 For this study, we will use an honorarium of $10075 for each focus group participant. There will be no payments or gifts to respondents of the self-administered questionnaire.
Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.
Individuals and organizations contacted will be further assured of the confidentiality of their replies under 42 U.S.C. 1306, and 20 CFR 401 and 4225 U.S.C.552a (Privacy Act of 1974). In instances where respondent identity is needed, the information collection will fully comply with all respects of the Privacy Act.
Information that can directly identify the respondent, such as name and/or social security
number will not be collected. For the survey and focus groups, individuals will be asked to report his/her position/title, but not the name of the hospital/organization in which they work. This information will be used solely by NORC to categorize and summarize types of respondents for comparison purposes during the analysis phase of the project.
A cover letter accompanying the survey to ED professionals is provided in Attachment F for respondents completing the survey via mail or email, and in Attachment G for respondents completing the survey via telephone. The letter informs participants that the survey data will be held strictly confidential; participants’ identities will be separated from the responses to the survey; and information gathered will be used solely by AHRQ, or its representatives for research, and will not be disclosed or released to other persons for any purpose except as required by law.
Focus group participants will be asked to complete a written informed consent prior to participating in the focus groups (Attachment H). The informed consent indicates: there are no foreseeable risks to participation; participation is completely voluntary; participants have the right to withdraw from the focus group at any time; if at any point during the focus group the participant withdraws, previous responses will remain part of the record; participants are free to refrain from answering any questions or commenting on any discussion topics that may arise; whether or not the participant chooses to participate in the focus group, or decides to withdraw at any point, will not affect him/her in any way. The informed consent will also ask participants for permission to be audio recorded.
The data collection instruments (survey questionnaire and focus group protocol) will not include any questions of a sensitive or personal nature. The questions are designed to solicit information solely regarding the acceptance of and satisfaction with the ESI.
In Exhibit 1, we provide estimates of the collection burden on participants for the survey and focus groups. The survey will be completed by 405 respondents. ED nurses, ED physicians, and ED medical and health services managers will complete the survey, in addition to a small number of people (less than 1%) who do not work in a hospital, but did request the ESI educational training materials. The survey data collection instrument is the same for all respondents, though respondents may choose to complete the questionnaire via an online option, hard-copy option, or over the telephone with the assistance of a trained telephone interviewer. The frequency of response is one survey per person who requested the ESI tool and was listed in AHRQ’s database of tool requesters as someone who agreed to be contacted to participate in the survey. Initial timing tests conducted by NORC indicate the survey will require approximately 20 minutes of each participant’s time to complete. NORC will also verify that the survey will require approximately 20 minutes of time to complete through a pilot test with 6 respondents. The total number of burden hours for the survey is equal to 135 hours.
The ED professionals focus groups will include 32 respondents. This is a one time effort that will require less than 90 minutes of each participant’s time. Note that focus group respondents will not participate in the ED professionals survey. Participants will include ED nurses, physicians, and medical and health services managers. The total number of burden hours for the focus groups is equal to 48 hours.
In Exhibit 2, we provide the estimated annualized cost burden for the survey and focus groups. For both data collection efforts, the total cost burden is $6,307.26.
Exhibit 1. Estimated annualized burden hours
Data collection effort |
Number of respondents |
Number of responses per respondent |
Hours per response |
Total burden hours |
ED professionals survey |
405 |
1 |
20/60 |
135 |
ED professionals focus groups |
32 |
1 |
1.5 |
48 |
Total |
437 |
na |
na |
183 |
Exhibit 2. Estimated annualized cost burden
Data collection effort |
Number of respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
ED professionals survey |
405 |
135 |
$33.70 |
$4,549.50 |
ED professionals focus groups |
32 |
48 |
$36.62 |
$1,757.76 |
Total |
437 |
183 |
na |
$6,307.26 |
*Total cost burden for the survey is based upon the weighted average of 13 physicians at $58.76/ hr, 95 nurses at $29.10/hr, and 27 medical and health services managers at $37.82/hr. Total cost burden for the focus groups is based on the weighted average of 6 ED physicians at $58.76/ hr, 21 nurses at $29.10/hr, and 21 medical and health services managers at $37.82/hr. National Compensation Survey: Occupational wages in the United States 2006, “U.S. Department of Labor, Bureau of Labor Statistics.”
There are no direct costs to respondents other than their time to participate in the study.
All aspects of this project will be completed within one year of OMB approval. All costs for conducting the assessment of the Emergency Severity Index (ESI) are included in the contract between the Agency for Healthcare Research and Quality (AHRQ) at the U.S. Department of Health and Human Services and NORC under contract number HHSP233200700002T. Exhibit 3 provides a detailed overview of the estimated annualized cost to the government.
Exhibit 3. Estimated Cost to the Government |
|
Developing and implementing the survey |
$183,305.00 |
Developing and conducting focus groups |
$69,669.00 |
Analyzing the data and report production |
$26,172.00 |
Associated personnel costs |
$17,073.00 |
Total cost to the government |
$296,219.00 |
Annualized cost to the government |
$296,219.00 |
This is a new collection of information.
This study will use both univariate, and where possible, multivariate techniques to analyze the data. Data analysis will focus on identifying the results of the established key research questions from each of the research objectives. A sample of the key research questions is provided below:
What is the awareness of the Emergency Severity Index (ESI)?
Do the individuals who requested a copy of the ESI products use the tool?
Are ESI tool requestors aware of the ESI training products produced by AHRQ?
Are requestors satisfied with the ESI training product’s presentation and clarity?
What are the advantages/disadvantages of the ESI relative to other triage tools?
Have the ESI tools improved the delivery and efficiency of emergency services?
To what extent have the ESI tools been used in every day ED surge planning?
What improvements can be made to the presentation of the ESI training tools?
How can AHRQ improve awareness of its role in ED medical surge planning?
Data obtained from survey respondents will be subject to preliminary cleaning and editing, and will be keyed into a SAS database for ease of analysis. Simple descriptive statistics will be used to analyze the survey data. Content analysis and univariate frequency and means testing will be used to calculate the percentage of respondents who provided a particular answer to a question, i.e., the proportion of respondents who are using the ESI as opposed to another triage system. For closed-ended survey items, simple statistical tests of significance, such as the chi-square test, will be used. (We further assume that since an equal-probability sample will be identified that the sample will be self-weighting). To the extent that items have numerical responses means and standard deviations will be computed in the aggregate and, as appropriate, for subgroups. Content analysis will be used to explore the themes in the focus groups.
The data collected in the survey and focus groups will be analyzed and interpreted to produce preliminary and final reports as well as a presentation for AHRQ. NORC will deliver the final report to AHRQ in hardcopy and a print-ready electronic format. Publication of findings on the internet is at AHRQ’s discretion. Exhibit 4 provides an overall timetable for data collection, analysis and the final report.
Exhibit 4. Timetable for Data Collection, Analysis, and Publication
Activity |
Expected Date of Completion |
Obtain IRB approval for study |
During OMB approval process |
Complete Pilot Testing of Survey |
During OMB approval process |
Conduct Survey |
1-2 months following OMB approval |
Conduct Focus Groups |
1-3 months following OMB approval |
Analyze Data |
3-4 months following OMB approval |
Prepare Draft Reports |
4-5 months following OMB approval |
Final Report *Ready for Internet publication at AHRQ’s discretion |
6 months following OMB approval |
Presentation to AHRQ |
6 months following OMB approval |
AHRQ does not seek this exemption.
Attachments for Supporting Statement, Parts A and B:
Attachment A: AHRQ's Authorizing Legislation
Attachment B: ED Professional Survey
Attachment C: ED Focus Group Protocol
Attachment D: 60 Day Federal Register Notice
Attachment E: Outside Consultants
Attachment F: Survey Cover Letter For Email/ Mail Based Survey
Attachment G: Survey Cover Letter For Telephone Based Survey
Attachment H: Written Informed Consent for Focus Groups
Attachment I: One-Week Post Card Reminder for Mail Survey Nonrespondents
Attachment J: One-Week Email Card Reminder for Email Survey Nonrespondents
Attachment K: One-Week Phone Call Reminder for Telephone Survey Nonrespondents
Attachment L: Three-Week Post Card Reminder and Subsequent Post Card Reminders for Mail Survey Nonrespondents
Attachment M: Three-Week Email Reminder and Subsequent Email Reminders for Email Survey Nonrespondents
Attachment N: Three-Week Phone Call Reminder and Subsequent Phone Reminders for Telephone Survey Nonrespondents
Attachment O: Flyer for Focus Group Recruitment
Attachment P: Focus Group Letter
1 Institute of Medicine. 2006. Hospital-Based Emergency Care: At the Breaking Point. Washington ,D.C.: National Academies Press.
2 Wuerz R. 2001. Emergency severity index triage category is associated with six-month survival. ESI triage study group. Academic Emergency Medicine, 8(1), 61-64.
3 Travers DA, Waller AE, Bowling JM, Flowers D, Tintinalli J. 2002. Five-level triage system more effective than three-level in tertiary emergency department. Journal of Emergency Nursing, 28(5), 395-400.
4 Eitel DR, Travers DA, Rosenau A, Gilboy N, Wuerz RC. 2003. The emergency severity index version 2 is reliable and valid. Academic Emergency Medicine, 10(10), 1079-1080.
5 Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG. 2004. Reliability and validity of scores on the emergency severity index version 3. Academic Emergency Medicine, 11(1), 1-7.
| File Type | application/msword |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| Last Modified By | Infante-Alycia |
| File Modified | 2008-07-03 |
| File Created | 2008-07-03 |