SUPPORTING STATEMENT
Part A
Pilot Test of an Interview Protocol Designed to Evaluate Emergency Preparedness and Response Capabilities
Version: November 15th, 2013
Agency of Healthcare Research and Quality (AHRQ)
A. Justification 3
1. Circumstances That Make the Collection of Information Necessary 3
2. Purpose and Use of Information 4
3. Use of Improved Information Technology 4
4. Efforts to Identify Duplication 5
5. Involvement of Small Entities 5
6. Consequences if Information Collected Less Frequently 5
7. Special Circumstances 5
8. Federal Register Notice and Outside Consultations 5
8.a. Federal Register Notice 5
8.b. Outside Consultations 5
9. Payments/Gifts to Respondents 6
10. Assurance of Confidentiality 6
11. Questions of a Sensitive Nature 6
12. Estimates of Annualized Burden Hours and Costs 7
13. Estimates of Annualized Respondent Capital and Maintenance Costs 7
14. Estimates of Annualized Cost to the Government 7
15. Changes in Hour Burden 8
16. Time Schedule, Publication and Analysis Plans 8
17. Exemption for Display of Expiration Date 8
18. List of Attachments 9
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. Research that develops and presents scientific evidence regarding all aspects of health care; and
2. The synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. Initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
For this project ASPR/OPP/Division of Health Systems Policy has requested AHRQ’s collaboration. AHRQ’s generic pretest clearance mechanism will be used to support this project in a timely manner.
Understanding pre and post incident response capabilities in a disaster impact area is essential to the planning and operational aspects of preparedness and response entities. These entities are often the only resource available to at risk populations who cannot care for themselves or get out of harm’s way. Having a standardized tool by which voluntary self-assessments may be completed after an incident assists the end user in trending data for purpose of risk assessments, planning, preparedness and response. By achieving accurate risk assessments, a response and preparedness entity may efficiently plan and prioritize resources to meet the needs of any priority populations who require assistance.
This proposed pilot project is designed to:
1) Examine the efficacy and efficiency of a peer assessment interview protocol used to evaluate preparedness and response capabilities following a disaster.
2) Modify the interview protocol as necessary based on the results of the pilot test.
To achieve the goals of this project the following data collection will be implemented:
Emergency Preparedness and Response Capabilities Interview – A pilot test of the Emergency Preparedness and Response Capabilities Interview will be conducted with 60 individuals responsible for responding to disasters and other emergency incidents (see Attachment A for the interview protocol). Respondents will include Federal delivery partners, peers, customers and stakeholders who utilize or provide assistance during a disaster. Interviews will be conducted in Boston Massachusetts and West Texas.
The Boston Massachusetts and West Texas working groups were selected to pilot test this tool because both recently experienced zero warning, real world disasters which required an integrated response between state, Federal and local ESF-8 capabilities. The group members were selected to reflect the typical makeup of a disaster response peer assessment group. Each individual member is selected to reflect the type of support personnel, response personnel, levels of care, and entities typically represented during disaster response operation. In terms of recent disaster response incidents, the Boston and West Texas groups may act as peers to compare, contrast and improve the application of this data collection tool.
This data collection tool pilot project is being conducted by ASPR Division of Health System Policy through a contract with RAND.
The outcome of the interview protocol work group will be to improve the communications tools contained in the interview protocol template and to refine and clarify suggested scope and wording, of questions posed to the disaster assessment team/ work group.
A report including recommendations for enhancing and improving the template will be filed. The report will provide results about the perceived usefulness of the peer assessment interview process. Results will be produced separately for the working groups and for ASPR. The report will also include specific suggestions on how to revise the peer assessment template and interview protocol to make it more useful to its intended audiences.
ASPR Division of Health System Policy will collect data through an established qualitative evaluation methodology, which includes review of After Action Reports and other operational documents, and in-person interviews and observations of work group participants. Because most interview questions are open-ended to allow for in-depth exploration of issues, electronic submission of all responses is not a viable option.
In addition, to reduce reporting burden on participants, we will use Google Analytics to provide supplementary data about direct use of the template, including tracking page views, time spent on page and on site, content, and navigation analysis to examine how participants are using the template Web site.
The Peer Assessment Template that is under evaluation in this proposed project was developed by RAND. To date, ASPR has not conducted a systematic evaluation of this template/interview guide and is not aware of any other entity conducting a similar evaluation.
As noted above, this evaluation will directly involve staff from two working groups. ASPR will target participation from members that do not have a strong affiliation with a health system and that are a privately owned business, and customers of disaster and preparedness services.
Participation is voluntary, and ASPR has designed a participation schedule that is intended to minimize the impact of the template / interview evaluation process. Interviews and observations will be scheduled at times convenient for the working groups. The interview protocols consist of the minimum questions required to evaluate and provide improvement feedback. The established interview time limits for each respondent type will be respected, and the interviews will not exceed one hour. Similar interview guides have been successfully used with groups similar to the ones being included in this evaluation.
This is a one-time collection.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
The following experts were consulted on various aspects of the design of the data and the pilot study collection effort. This effort included the key research question, approaches to identify and recruit participants, methods of data collection and analysis, and interview protocol development:
Christopher Nelson, PhD, The Rand Corporation, Division of Health
Gregg Margolis, PhD, The Assistant Secretary of Preparedness and Response , Division of Health System Policy
The respondents in this template and interview protocol pilot test project are federal partners, peers and consumers of preparedness and response services. As such they will invoice for activities related to the project including pre-evaluation and evaluation activities, but no additional gift or incentive to participation will be provided. The pre-evaluation activity is establishing the work group members. Evaluation activities include participating in interviews; pre-meeting reviews of records and AARs , review of the template and interview protocols; maintaining a log of issues; and participating in the assigned work groups
We estimate the cost of respondent participation as approximately $2765.70 based on ~60 hours of labor at $30.73 per hour. The hourly rate is calculated based on average salaries of the roles we will invite to participate in the project: clinicians, bureaucrats, first responders, managers, civil servants and consumers.
10. Assurance of Confidentiality
Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act, 42 USC 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied. The project will collect information from respondents about the usefulness of the peer assessment Interview Protocol (Attachment A). It will not collect any information about either the respondent or any individual in the establishment. ASPR, DHSP and RAND will collect the respondent’s name, organizational affiliation, organizational phone number, and role. This information will be used for respondent tracking purposes or for clarification call backs. All electronic files will be password protected and accessible only from within a secured network. Electronic files containing study data from the Pilot Study will be transmitted for data management and analysis to RAND, the contractor leading data collection and analysis. These files will be encrypted and will be transmitted through a secure messaging portal. Paper files will be sent via certified mail or delivered by hand to project staff. When not in use by project staff, all printed information or materials that could be used to identify participants in the study will be stored in locked cabinets that are accessible only to project team members.
All respondent involvement will be voluntary. Informed consent will be obtained from each respondent from each organization prior to participation. Respondents will be informed that: (1) the project team will not share their name, their organization’s name, or copies of the interview notes with anyone outside of the team; and (2) respondent comments may be included in reports, but will not be attributed to specific individuals or organizations.
All project team members are required to complete human subjects training coursework through Institutional Review Boards.
No questions of a sensitive nature will be asked.
Exhibit 1 shows the estimated burden hours for each respondent’s time to participate in this pilot test. Interviews will be completed with 60 stakeholders and will last for about one hour. The total burden is estimated to be 60 hours.
Exhibit 2 shows the estimated cost burden associated with the respondents’ time to participate in the pilot test. The total cost burden is estimated to be $1,844.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of respondents |
Number of responses per respondent |
Hours per response |
Total burden hours |
Emergency preparedness and response capabilities interview |
60 |
1 |
1 |
60 |
Total |
60 |
na |
na |
60 |
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Emergency Preparedness and Response Capabilities Interview |
60 |
60 |
$30.73 |
$1,844 |
Total |
60 |
60 |
na |
$1,844 |
*The hourly wage for the participants across the four data collections (pre-workgroup it interviews, observations, usage logs, and post workgroup interview) is based upon a mean of the average hourly wages for Paramedics and EMTs ($17.70 per hour) physicians and surgeons (; $70.01 per hour);); nurses ( $40.50 per hour); community and social service specialists ( $17.73per hour)social workers rs ( $25.40 per hour); fire fighters ( $25.85 per hour); information technology specialists ( $23.43 per hour); quality improvement directors ( 25.12 per hour); and technical staff ( $33.14 per hour) for Boston, Massachusetts from the U.S. Department of Labor, Bureau of Labor Statistics, May 2010 National Compensation Surveys for the United States, http://data.bls.gov/cgi-bin/dsrv?nw (accessed September 2013).
Capital and maintenance costs include the purchase of supplies and computer software or services, travel, postage, and storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
The estimated total cost to the Federal Government for this 1 year project over this 12 -month period is ~$55,000 -~$75,000.
Exhibit 3 provides a breakdown of the estimated total costs by category.
Exhibit 3. Estimated Total and Annual Cost* to the Federal Government
Cost Component |
Total Cost |
Project Development |
~$15,000- $20,000 |
Data Collection Activities |
~$10,000-~$15,000 |
Data Processing and Analysis |
~$10,000- ~$15,000 |
Project Management |
~$20,000- $25,000 |
Total |
$55,000 - $75,000 |
This is a new collection of information.
Time schedule and publication plans. The anticipated schedule for this project is shown in Exhibit 4. Once clearance from the Office of Management and Budget is obtained, AHRQ will begin identifying appropriate respondents and scheduling and conducting evaluation activities.
Exhibit 4. Anticipated Schedule
Activity |
Estimated timeline following OMB clearance |
Recruit for Field Evaluation |
Month 1 |
Conduct Field Evaluation |
Months 2 – 6 |
Analyze Results |
Months 7-11 |
Brief AHRQ on Results |
Month 11 |
Submit Final Report on Results |
Month 13 |
Analysis plans. Project staff will employ the immersion-crystallization approach to qualitative data analysis, in an iterative process that begins at the outset of data collection and continues throughout the data collection period. The RAND team will meet weekly to conduct preliminary analyses of the field notes, interview recordings and notes, and any project documents. Analysis sessions will assess and ensure data quality, and analyze data to address the research question. Analysis will also attempt to identify:
How the template questions and interview protocol may be improved
How understanding of the importance of assessing post incident response and preparedness capabilities changed
How may the respondents utilize the template in the future
Whether the template and interview protocol is regarded as useful
Suggestions for improvement
Atlas.ti software (Version 5.0) will be used to store, code and search the interview data for analysis. Data reduction will be achieved by summarizing coded interview data from Atlas.ti in data tables and practice summaries, which will then be analyzed to refine themes, align them with the evidence supporting each finding, and identify respondent disagreements and disconfirming evidence.
AHRQ does not seek this exemption.
Attachment A -- Emergency Preparedness and Response Capabilities Interview Protocol
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |