SS Part A -- Applying Novel Methods to Better Understand the Relationship Between HIT 4-17-2013

SUPPORTING STATEMENT

Part A

Applying Novel Methods to Better Understand the Relationship between Health IT and Ambulatory Care Workflow Redesign

December 17, 2012

Agency for Healthcare Research and Quality (AHRQ)

Table of contents

1. Circumstances that make the collection of information necessary 3

2. Purpose and Use of Information 6

3. Use of Improved Information Technology 6

4. Efforts to Identify Duplication 6

5. Involvement of Small Entities 7

6. Consequences if Information Collected Less Frequently 7

7. Special Circumstances 7

8. Federal Register Notice and Outside Consultations 7

9. Payments/Gifts to Respondents 7

10. Assurance of Confidentiality 7

11. Questions of a Sensitive Nature 8

12. Estimates of Annualized Burden Hours and Costs 8

13. Estimates of Annualized Respondent Capital and Maintenance Costs 9

14. Estimates of Annualized Cost to the Government 9

15. Changes in Hour Burden 9

16. Time Schedule, Publication and Analysis Plans 9

17. Exemption for Display of Expiration Date 10

A. Justification

1. Circumstances that make the collection of information necessary

The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:

1. Research that develops and presents scientific evidence regarding all aspects of health care; and

2. The synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and

3. Initiatives to advance private and public efforts to improve health care quality.

Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.

Request for information collection approval. The Agency for Healthcare Research and Quality (AHRQ) requests that the Office of Management and Budget (OMB) approve, under the Paperwork Reduction Act of 1995, AHRQ’s collection of information for the project Applying Novel Methods to Better Understand the Relationship between Health IT and Ambulatory Care Workflow Redesign. The data to be collected consists of interviews and focus groups with clinical, non-clinical, and management staff about their experiences with new health information technology (IT) in an ambulatory care facility. The overall goal of this study is to characterize the relationship between health IT implementation and health care workflow in six (6) small and medium-sized ambulatory care practices implementing patient-centered medical homes (PCMH), with a focus on the influence of behavioral and organizational factors and the effects of disruptive events.

Background on health information technology. AHRQ is a lead Federal agency in developing and disseminating evidence and evidence-based tools on how health IT can improve health care quality, safety, efficiency, and effectiveness. Health IT has been widely viewed as holding great promise to improve the quality of health care in the U.S.^1,2 Health IT can improve access to information for both patients and providers, empowering patients to become involved in their own self-care. Increased patient safety can result from health IT when records are shared, medications are reconciled, and adverse event alerts are in place. When health IT improves efficiency, providers can spend more time directly caring for patients, ultimately improving the quality of care patients receive.

In redesigning an ambulatory office practice as a patient-centered medical home (PCMH), health IT is intended to allow for a seamless and organized flow of information among providers. The health IT system is critical, because under the PCMH model, a team of clinicians aims to provide continuous and coordinated care throughout a patient's lifetime.

Unfortunately, health IT systems can fail to generate anticipated results^3,4,5 and even carry unintended consequences thereby undermining usability and usefulness.^{6,7,8,9,10,11} Directly or indirectly, health IT may create more work, new work, excessive system demands, or inefficient workflow (the sequence of clinical tasks). Electronic reminders and alerts may be timed poorly. Software may require excessive switching between screens, leading to cognitive distractions for end users. Providers may spend more time on health IT system-related tasks than on direct patient care.

The literature also suggests that the ambulatory health care environment is full of unpredictable yet frequently occurring events requiring actions that deviate from normal practice.¹⁴ Unpredictable events such as interruptions requiring a provider’s immediate attention, or disruptions in the normal functioning of the health IT system (exceptions) divert health care workers from the usual course of workflow. The inability of health IT to properly accommodate these events could cause compromises to clinical work.^6,7,8,11

Rationale for the information collection. Because of adverse, unintended and disruptive consequences, developing an understanding of how health IT implementation alters clinical work processes and workflow is crucial. Unfortunately, research is scarce, and methods of investigation vary widely.^12,13 As Carayon and Karsh comment, empirical evidence of health IT’s impact on clinical workflow has been “anecdotal, insufficiently supported, or otherwise deficient in terms of scientific rigor.”¹²

This study aims to examine more systematically the impact of health IT on workflow in six (6) small and medium-sized ambulatory care practices varying in their characteristics but all implementing PCMH. It will employ the complementary quantitative and qualitative methods of previous research.^{14,15,16,17,18} The combination of methods produces quantitative results and allows validation of them through observation and solicitation of qualitative participant opinions.

The specific goals of this study are to identify:

1. the relationship between health IT implementation and ambulatory care workflow

2. the behavioral and organizational factors and the role they play in mitigating or augmenting the impact of health IT on workflow

3. how the impacts of health IT are magnified through disruptive events such as interruptions and exceptions

To achieve the goals of this project the following data collections will be implemented:

1. Mapping of Study Practices. This activity will detect any changes made to the physical layout as a result of implementing PCMH and health IT. Practices will be mapped at the beginning of the study and maps will be updated as needed. Recording this information will not burden the clinic staff and is not included in the burden estimates in Section 12.

2. Staff Observation. Clinicians (physicians, nurse practitioners, physician assistants, nurses, medical assistants, pharmacists, and case managers) and non-clinical office personnel will be observed to delineate the overall characteristics of clinical workflow before, during, and after health IT implementation. Particular attention will be paid to interruptions and exceptions. If necessary and if the situation allows, observers will as unobtrusively as possible ask clinic staff to clarify certain observed actions. See Attachment A: Observation Guide. Recording this information will not burden the clinic staff and is not included in the burden estimates in Section 12.

3. Before–After Time and Motion Study. This activity quantifies staff's time expenditures on different clinical activities and delineates the sequence of task execution. It will be conducted before and after health IT implementation. See Attachment B: Screenshot of the Time and Motion Data Collection Tool. This data will be collected by observation only. Recording this information will not burden the clinic staff and is not included in the burden estimates in Section 12.

4. Extraction of Clinical Data. Logs, audits trails, and time-stamped clinical data will be extracted from the health IT system to reconstruct clinical workflow related to the health IT system. This information validates and supplements the data recorded by human observers. Extracting this data will not burden the clinic staff and is not included in the burden estimates in Section 12.

5. Semi-Structured Interviews. This data collection will be conducted post-health IT implementation to solicit attitudes and perceptions by health IT end- users including clinical staff, non-clinical personnel, and management regarding how health IT has changed their workflow. Particular attention will be paid to behavioral and organizational factors. See Attachment C: Semi-Structured Interview Guide.

6. Focus Group. A focus group will be conducted after health IT implementation with the clinical staff, non-clinical personnel, and management team to ensure the research findings, as well as the interpretation of the findings, accurately reflect their experiences using health IT. See Attachment D: Focus Group Guide.

The qualitative study components of this project, namely staff observations, semi-structured interviews, and focus groups, will generate qualitative data in the form of observation notes and interview transcripts. The time-and-motion study and the electronic clinical data will produce quantitative information in the form of sequences of clinical activities and information about the duration, location, and performer of each action. Mapping will create annotated floor plans delineating the physical layout of each study clinic, which will be incorporated in the collection and analysis of the data of the other study components.

This study is being conducted by AHRQ through its contractor, Billings Clinic, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).

2. Purpose and Use of Information

Information collected via staff observation, time and motion study, qualitative interview, and focus group, combined with comprehensive data on staff interaction with the health IT system is necessary in order to determine how health IT characteristics and behavioral and organizational attributes might contribute to clinical workflow. By understanding the nature in which health IT can enhance or impede care workflow in small-to-medium ambulatory care settings, findings will help similar health care organizations identify the technical, behavioral and organizational complexities for implementing health IT systems as a part of practice redesign. Particularly, this research will identify technical factors – such as the temporal order of tasks, the position of items on the computer screen, and the number of “clicks” required to complete a sequence of tasks – and the behavioral and organizational influences and effects – individuals’ different and unique workflow processes followed when performing a specific task, how multiple tasks are ordered, mandating of certain tasks by some organizations and not by others – for how health IT can impact workflow as a part of practice redesign. Furthermore, this research will help identify disruptive events and unintended consequences as a part of health IT implementation.

3. Use of Improved Information Technology

To the extent possible, this research has been designed to utilize technological methods of data collection. The time-and-motion data will be recorded using electronic software that allows the observer to explicitly label activities and to record location information. Clinical data are electronically captured from the health IT system. Semi-structured interview and focus group data will be audio recorded, as well as recorded in text electronically using tablet computers. This will allow direct importation into the NVivo qualitative analysis software package.

4. Efforts to Identify Duplication

AHRQ has conducted a systematic review of the literature and conferred with internal and external experts on current and planned research on the topic of workflow and has found that rigorous research on workflow in ambulatory care settings is lacking. From this work, AHRQ has concluded that data does not exist that specifically addresses the interaction of workflow and health IT in small and medium-sized ambulatory practices engaged in practice redesign efforts. Previous work in this area has focused on large clinics affiliated with academic medical centers, health maintenance organizations, and national health systems outside the US.^13,15

5. Involvement of Small Entities

The clinics in which the research is taking place are small entities. Smaller practices are understudied and have the greatest need due to lack of resources. However, time between collections is spaced to avoid requiring frequent response from subjects. Different individuals are queried at different time periods to reduce the burden on each respondent. The majority of the data will be collected unobtrusively by observation, time and motion study, and from the health IT system itself.

Observers will be trained not to interfere with the work of the staff at the practice. They will complete notes as much as possible between observations. Similarly, for the time and motion study, they will complete observations as unobtrusively as possible. Mapping will occur during non-clinic hours and will therefore place no burden on the practice.

Semi-structured interviews and focus groups will be scheduled at the convenience of the respondents. The location of interviews will be chosen at convenience of respondents. Every effort will be made to ensure the interviews and focus groups entail minimal burden.

6. Consequences if Information Collected Less Frequently

This is a one-time data collection.

7. Special Circumstances

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.

8. Federal Register Notice and Outside Consultations

8.a. Federal Register Notice

As required by 5 CFR 1320.8(d), notice was published in the Federal Register on January 28^th, 2013 for 60 days, and again on April 17^th, 2013 for 30 days. No comments were received. The Notice is included as Attachment F.

8.b. Outside Consultations

AHRQ consulted with its research contractor, Billings Clinic, in developing the study protocol. The research plan and data collection instruments were also shared with David Hunt from the Office of the National Coordinator for Health IT for review.

9. Payments/Gifts to Respondents

Respondents directly participating in the study will each be provided with $25 in appreciation for their time. This is consistent with gifts provided to clinical staff under past projects conducted by AHRQ’s contractor, which typically range from $25 to $50 per participant to ensure adequate participation rates.

10. Assurance of Confidentiality

Individuals and organizations will be assured of the confidentiality of their replies under Sections 944(c) and 308(d) of the Public Health Service Act[42 U.S.C. 299c-3(c) and 42 U.S.C. 242m(d)]. They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.

All respondent involvement will be voluntary. Informed consent will be obtained from each respondent from each organization prior to participation. Respondents will be informed that: (1) the project team will not share their name, their organization’s name, or copies of the interview notes with anyone outside of the research team; and (2) respondent comments may be included in reports, but will not be attributed to specific individuals or organizations. All of this information is included in a Consent Form (Attachment E).

All research data collected in this project will be de-identified prior to analysis and dissemination. A unique code will be assigned to each participant in order to link the data collected from different sources. The mapping information will only be accessible to the Project Director, Dr. Kai Zheng, and will be stored separately from the other research data. All respondents will be notified of these procedures before being observed or interviewed and will be asked to sign the Consent Form (Attachment E).

11. Questions of a Sensitive Nature

This research protocol does not include any questions of a sensitive nature.

12. Estimates of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated annual burden hours for participation in this study. The semi-structured interview will be completed by 60 respondents across the 6 clinics (10 per practice) and requires one hour. Sixty (60) clinic staff members will be asked to participate in the focus group across all 6 clinics (10 per practice). The focus group requires no more than 45 minutes. The total annual burden is estimated to be 105 hours.

Exhibit 2 shows the estimated annual cost burden associated with the respondents' time to participate in this research. The total annual burden is estimated to be $5,505.

Exhibit 1. Estimated annualized burden hours

Form Name	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Semi-Structured Interview Guide	60	1	1	60
Focus Group Guide	60	1	45/60	45
Total	120	na	na	105

 

Exhibit 2. Estimated annualized cost burden

Form Name	Number of respondents	Total burden hours	Average hourly wage rate*	Total cost burden
Semi-Structured Interview Guide	60	60	$55	$3,300
Focus Group Guide	60	45	$49	$2,205
Total	120	105	na	$5,505

*Based upon the mean of the average wages, National Compensation Survey: Occupational wages in the United States July 2010, U.S. Department of Labor, Bureau of Labor Statistics, http://www.bls.gov/ncs/ocs/sp/nctb1477.pdf (accessed September, 2012). For the semi-structured interviews, hourly wage is an average including 2 physicians or surgeons ($85.67), 1 registered nurse ($32.42), 2 non-physician providers (measured here as physician assistants, $43.44), and 1 senior administrator (measured here as “Medical and health services managers,” $42.28). For focus groups, 3.34 physicians or surgeons ($85.67), 1.66 non-physician providers (measured here as physician assistants, $43.44), 3.34 registered nurses ($32.42), and 1.66 medical assistants ($14.46).

13. Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.

14. Estimates of Annualized Cost to the Government

The total cost of this study is $799,014 over a 36-month time period from June 1, 2012 through May 31, 2015 for an annualized cost of $266,338. Exhibit 3 provides a breakdown of the estimated total and average annual costs by category.

Exhibit 3. Estimated Total and Annualized Cost

Cost Component	Total Cost	Annualized Cost
Project Development	$ 135,759	$ 45,253
Data Collection Activities	$ 177,460	$ 59,153
Data Processing and Analysis	$ 239,426	$ 79,809
Publication of Results	$ 51,779	$ 17,260
Project Management	$ 67,729	$ 22,576
Overhead	$ 126,861	$ 42,287
Total	$ 799,014	$ 266,338

15. Changes in Hour Burden

This is a new collection of information.

16. Time Schedule, Publication and Analysis Plans

The anticipated schedule is shown in Exhibit 4. Once clearance from the Office of Management and Budget is obtained, AHRQ will begin identifying appropriate respondents and scheduling and conducting data collection. Because the project entails gathering data at three points in time (before, during and after health IT implementation), a period of twenty-one (21) months is planned for data collection.

Study findings will be made publicly available in a final report available for download from the AHRQ National Resource Center for Health IT Web site.

Exhibit 4. Anticipated Schedule

Activity	Estimated timeline following OMB clearance
Conduct Research Study and Gather Data	Months 1-21
Conduct Data Analysis	Months 6-25
Prepare Final Report	Months 25-29
Prepare Presentation	Months 28-29

Analysis plans.

Data collection will employ a qualitative and quantitative approach consisting of mapping and remapping of study practices, staff observation, before-after time motion study, extraction of electronic information, semi-structured interview, and focus group. The design of data collection and analysis are informed by the Workflow Elements Model (WEM) described in Unertl et al. (2010)¹³ and definitions of workflow proposed by Ellis¹⁹ and included in the scholarly literature on computer-supported cooperative work (CSCW). As described by Karsh “CSCW research focuses on how people collaborate and how technology can mediate that collaboration effectively.”²⁰

The qualitative data to be collected will be analyzed using a constant comparison approach²¹ and operationalized in the following three steps: (1) theme formation; (2) theme matching along themes and patterns observed in the data; and (3) theme comparison across practice sites.²² These processes will be facilitated by the NVivo qualitative data analysis software (QSR International, Doncaster, Australia).

The quantitative workflow data collected for this study will be analyzed using three methods proposed and empirically validated in Zheng et al. (2010)¹⁵: workflow fragmentation assessments, pattern recognition, and data visualization. These methods are designed to uncover regularities undetectable by other existing approaches.¹ Computerized algorithms implementing these methods are available in an automated analytical tool called Clinical Workflow Analysis Tool (CWAT) that the research team developed previously (a demonstration version can be found at http://sitemaker.umich.edu/workflow).

17. Exemption for Display of Expiration Date

AHRQ does not seek this exemption.

Attachments:

Attachment A: Observation Guide

Attachment B: Screenshot of the Time and Motion Data Collection Tool

Attachment C: Semi-Structured Interview Guide

Attachment D: Focus Group Guide

Attachment E: Consent Form

Attachment F: Federal Register Notice

References

1. Girosi F, Meili R, Scoville R. Extrapolating Evidence of Health Information Technology Savings and Costs. Santa Monica, CA: RAND Corp.; 2005.

2. Institute of Medicine (U.S.). Crossing the quality chasm: a new health system for the 21st century. Washington D.C.: National Academy Press; 2001.

3. Jones SS, Adams JL, Schneider EC, Ringel JS, McGlynn EA. Electronic health record adoption and quality improvement in US hospitals. Am J Manag Care. Dec 2010;16(12 Suppl HIT):SP64-71.

4. Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med. Jul 9 2007;167(13):1400-1405.

5. Romano MJ, Stafford RS. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Arch Intern Med. May 23 2011;171(10):897-903.

6. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. Sep-Oct 2006;13(5):547-556.

7. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. Dec 2005;116(6):1506-1512.

8. Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. Mar 9 2005;293(10):1197-1203.

9. Bloomrosen M, Starren J, Lorenzi NM, Ash JS, Patel VL, Shortliffe EH. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. J Am Med Inform Assoc. Jan-Feb 2011;18(1):82-90.

10. Sittig DF, Ash JS, Zhang J, Osheroff JA, Shabot MM. Lessons from "Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system". Pediatrics. Aug 2006;118(2):797-801.

11. Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. Jul-Aug 2007;14(4):415-423.

12. Carayon P, Karsh B. Incorporating Health Information Technology Into Workflow Redesign. In: Agency for Healthcare Research and Quality, ed. Rockville 2010.

13. Unertl KM, Novak LL, Johnson KB, Lorenzi NM. Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review. J Am Med Inform Assoc. May-Jun 2010;17(3):265-273.

14. Zheng K, Hanauer DA, Padman R, et al. Handling anticipated exceptions in clinical care: investigating clinician use of exit strategies in an electronic health records system. J Am Med Inform Assoc. Nov-Dec 2011;18(6):883-889.

15. Zheng K, Haftel HM, Hirschl RB, O'Reilly M, Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. J Am Med Inform Assoc. Jul-Aug 2010;17(4):454-461.

16. Bohnsack KJ, Parker DP, Zheng K. Quantifying temporal documentation patterns in clinician use of AHLTA-the DoD's ambulatory electronic health record. AMIA Annu Symp Proc. 2009;2009:50-54.

17. Lanham HJ, Leykum LK, McDaniel RR, Jr. Same organization, same electronic health records (EHRs) system, different use: exploring the linkage between practice member communication patterns and EHR use patterns in an ambulatory care setting. J Am Med Inform Assoc. May-Jun 2012;19(3):382-391.

18. Zheng K, Padman R, Johnson MP, Diamond HS. An interface-driven analysis of user interactions with an electronic health records system. J Am Med Inform Assoc. Mar-Apr 2009;16(2):228-237.

19. Ellis C. Workflow Technology. In: Beaudouin-Lafon M, ed. Computer Supported Cooperative Work. Chichester: John Wiley & Sons; 1999:29–54.

20. Karsh B-T. Clinical practice improvement and redesign: how change in workflow can be supported by clinical decision support. AHRQ Publication No. 09-0054-EF. Rockville, Maryland: Agency for Healthcare Research and Quality; June 2009.

21. Strauss A, Corbin J. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 2nd ed. Thousand Oaks, CA: Sage Publications, Inc; 1998.

22. Yin R. Case Study Research: Design and Method. Thousand Oaks, CA: Sage; 2003.