SS Part B -- Applying Novel Methods to Better Understand the Relationship Between HIT 4-17-2013

SUPPORTING STATEMENT

Part B

Applying Novel Methods to Better Understand the Relationship between Health IT and Ambulatory Care Workflow Redesign

March 6^th, 2013

Agency for Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 6

3. Methods to Maximize Response Rates 8

4. Tests of Procedures 8

5. Statistical Consultants 8

B. Collections of Information Employing Statistical Methods

1. Respondent universe and sampling methods

For this study, a purposive sampling strategy will be used to identify 6 small and medium-sized ambulatory care practices implementing a patient-centered medical home (PCMH). These 6 practices will be drawn from two large health care organizations that have agreed to participate in this research; the Billings Clinic in Billings, Montana and Cabin Creek Health Systems, in Dawes, West Virginia. Practices will be chosen based on their plans to implement a new health IT system as part of their practice redesign effort. While study findings cannot be generalized, findings will generate insights into understanding how health IT implementation alters clinical workflow.

Both primary and specialty care practices will be part of the study in order to increase heterogeneity of the study participants. Including specialty clinics will broaden the understanding of how health IT can impact workflow practice, because specialty clinics may follow different workflow processes from primary care clinics and may encounter unique challenges in relation to health IT implementation.

Exhibit 5 shows the data collection activities involving human subjects. Sample sizes for each data collection activity are indicated

Exhibit 5. Research Activities Involving Human Subjects

Research activity	In relation to health IT implementation phases	Sample size	Data collection methods
Staff Observations	Pre, During, Post	60	Observation notes recorded by observers. The Observation Guide is provided in Attachment A.
Before–After Time and Motion Study	Pre, Post	54	Quantitative time and motion data collected by RAs via a tablet computer (a screenshot is provided in Attachment B).
Semi-Structured Interviews	Post	60	Audio recordings (and subsequent transcripts) of interview conversations. The Semi-Structured Interview Guide is provided in Attachment C.
Focus Groups	Post	60	Audio recordings (and subsequent transcripts) of panel discussions during member checking sessions. The Focus Group Guide is provided as Attachment D.
Total Respondent Universe	Pre, During, Post	60

Staff Observations

All clinic staff members at the six practices will be asked to participate and a total of 60 individuals are expected to be observed in this component of the study. Each observer will, in general, observe two clinicians/staff per day. Each session shadowing the same individual will last at most two hours. It may conclude sooner if the volume of observation data is deemed sufficient, i.e. the number of incidents of interest has diminished to such a level that the observer is confident additional data collection would not likely lead to additional insights, otherwise referred to as “theoretical saturation”.⁵

Before–After Time and Motion Study

A purposive total sample of up to 9 clinicians and other clinic personnel from each of the 6 study practices will be invited to participate in the Before-After Time and Motion study (T&M study), for a total of up to 54 participants across the six study practices. Currently, there is no gold standard with respect to the optimal sample size for T&M studies. Therefore, the strategy is to include the maximum number of subjects, and the maximum number of observation hours. This strategy is based on a general assumption that a larger sample size would lead to more comprehensive and more generalizable results. Compared to the T&M studies identified in a literature review,^6,7 the sample size of this project will be the largest in terms of observation hours, and among the largest in terms of number of subjects.

Subjects will be selected to achieve a balanced sample representing key clinical roles and different levels of clinical experience and computer literacy. The T&M study will enlist almost every primary care provider at the study practices. To identify the remaining participants (other types of clinicians and clinic personnel), study investigators will use a theoretical sampling approach.⁸ First, recruitment will be based on clinical roles to ensure that most major types of health care workers present in a typical ambulatory primary care clinic are included. Then, among the prospective participants, selection will be based on tenure at the study practice and ability to provide a diverse perspective on health IT implementation and clinical workflow. The selection process will be informed by results obtained from Staff Observations preceding each stage of T&M data collection, and assisted by site coordinators and the managerial team at each study site. Of the 54 subjects for the T&M study, it is estimated approximately 36 will be clinicians and 18 will be other types of clinic personnel.

Semi-Structured Interviews

In each participating organization, study investigators will use purposive sampling to recruit up to 30 clinicians and 30 other clinic personnel, including management executives (e.g., COO, CIO) and IT professionals from each organization (up to 10 per study practice) as subjects of the Semi-Structured Interviews. The investigator team will carefully identify interview candidates based on their potential to provide diverse perspectives on the impact of health IT implementation on workflow. Using this sampling approach is appropriate for studying less mature phenomena and when studying the average case is likely to produce less rich information.⁸

Focus Groups

All members of the health care team in a given practice will be invited to participate in Focus Groups. This will be included in the initial study orientation meeting at each practice. It is expected that up to 10 subjects will participate at each study site. Focus groups of up to 10 individuals provide a sufficient sample size to generate meaningful conversations and elicit useful feedback.^9-12

2. Information Collection Procedures

Staff Observations

Two research assistants will observe a total of 60 individual practice members during normal work activities, recording data such as the clinical tasks they perform, the types of information they need to complete these tasks, and the extent to which they interact with others (including patients). Observations will be conducted in all practice areas, including nursing stations, patient reception areas, support staff work areas, and employee break rooms.^1-4

The two-hour observation period can be shortened if saturation is achieved before the end of the projected 2-hour window. In this case, the observer will either (1) begin observing the next subject scheduled for the day, or (2) opportunistically approach individuals previously observed, with whom the data saturation had not been achieved, to conduct additional observations.

Research assistants will take field notes during the staff observations using the semi-structured Observation Guide (see Attachment A). The field notes will be subsequently typed into a computer to facilitate data analyses. The research assistants will be instructed not to record any information that may directly or indirectly identify a study participant.

Before–After Time and Motion Study

Research assistants will record the T&M data using an iPad application (a screenshot is in Attachment B) following well-established data collection guidelines.^3,6 Key data elements will include the clinician being observed (represented by a Study ID), date, clinical tasks, the starting and ending time of each task, and the location where the task takes place.

Semi-Structured Interviews and Focus Groups

Research assistants will also conduct in-person Semi-Structured Interviews with clinicians, clinic personnel, and management executives (see Attachment C). Following preliminary data analysis, the research assistants will conduct focus groups (see Attachment D). Focus Groups will be conducted with the health care teams at the study practices to ensure the research findings, as well as the interpretation of the findings, accurately reflect the health care teams’ practices and their experiences using health IT. The Semi-Structured Interviews and Focus Groups will be audio taped and subsequently transcribed for qualitative data analyses. All identifying information disclosed during the interviews and focus groups will be removed from the transcripts. The tapes will be destroyed after transcription.

The research team will make every effort to remove identifying information from the research data collected. Meta-data that may contain partial identifying information, such as name of the study practice and participants’ clinical roles, will be password-protected and stored separately from the research data.

3. Methods to Maximize Response Rates

While a non-probabilistic sample approach will be used as described above, strategies will be employed to ensure the desired sample size or data saturation is reached for each data collection activity.

Both organizations have agreed to participate in this research study, and have offered their support in recruiting participants. Letters of support were received from physician leaders in all potential practices. Maximal participation will be achieved due to organizational support and the fact that the data collection is minimally intrusive. The staff observations, time and motion study, and extraction of clinical data do not require active participation from clinic staff. Participants in the semi-structured interviews and focus groups will be provided a $25 gift card in appreciation for their time and contributions. AHRQ, therefore, foresees optimal participation.

4. Tests of Procedures

Protocols from the before-after time and motion study, semi-structured interviews, and focus groups will each be pre-tested by two project site coordinators and 7 clinicians or staff to be identified from non-study sites at participating organizations, for a total of no more than 9 individuals per protocol. It is expected that only minor refinements will be made to the draft protocols. Any changes will be submitted with the final OMB package for OMB review.

5. Statistical Consultants

This mixed-methods study will employ qualitative and quantitative methods for data collection and analysis. Project co-investigator Dr. Holly Lanham is an expert on qualitative methods^9-12 and will lead all qualitative project activities including Staff Observations, Semi-Structured Interviews, and Focus Groups.

The principal investigator Dr. Kai Zheng has significant experience and specialized expertise in conducting workflow analysis, and in related areas including human–computer interaction, computer supported cooperative work, technology acceptance, organization studies, and socio-technical investigations.^7,9,13-22 Dr. Zheng has considerable experience conducting time and motion studies and will lead data collection and analysis activities for the quantitative components of the study. The quantitative workflow data will be analyzed using three methods proposed and empirically validated in Zheng et al. (2010)⁷: workflow fragmentation assessments; pattern recognition; and data visualization.

AHRQ’s contractor, Billings Clinic, will be responsible for overseeing the recruitment of participants, conducting all of the data collection, and analyzing and reporting the findings. The principal investigator is Dr. Kai Zheng and project director is Dr. Elizabeth Ciemins. Dr. Zheng can be reached by phone at (412) 708-2202 or by email at [email protected]. Dr. Ciemins can be reached by phone at (406) 238-5724 and by email at [email protected].

References:

1. Wetterneck T, Lapin J, Krueger D, Holman G, Beasley J, Karsh B. Development of a Primary Care Physician Task List to Evaluate Clinic Visit Workflow. BMJ Quality & Safety. 2012;21(1):47-53.

2. Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entry: effects on physicians' time utilization in ambulatory primary care internal medicine practices. J Am Med Inform Assoc. Jul-Aug 2001;8(4):361-371.

3. Pizziferri L, Kittler AF, Volk LA, et al. Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study. J Biomed Inform. Jun 2005;38(3):176-188.

4. Agency for Healthcare Research and Quality. Time and Motion Study Tool: Ambulatory Practice (TMS-AP). http://healthit.ahrq.gov/portal/server.pt/gateway/PTARGS_0_1248_216071_0_0_18/AHRQ%20NRC%20Time-Motion%20Study%20Tool%20Guide.pdf. Accessed January 12, 2012.

5. Patton M. Qualitative Research & Evaluation Methods. 3rd ed. Thousand Oaks, CA: SAGE Publications; 2002.

6. Zheng K, Guo MH, Hanauer DA. Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. J Am Med Inform Assoc. Sep-Oct 2011;18(5):704-710.

7. Zheng K, Haftel HM, Hirschl RB, O'Reilly M, Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. J Am Med Inform Assoc. Jul-Aug 2010;17(4):454-461.

8. Eisenhardt K. Building theories from case study research. Academy of Management Review. 1989(14):532-550.

9. Lanham HJ, Leykum LK, McDaniel RR, Jr. Same organization, same electronic health records (EHRs) system, different use: exploring the linkage between practice member communication patterns and EHR use patterns in an ambulatory care setting. J Am Med Inform Assoc. Aug 24 2011.

10. Lanham HJ, McDaniel RR. An exploration of heterogeneity in electronic medical record use: Information technology use as emergent and driven by values and expertise. Paper presented at: Twenty-Ninth International Conference on Information Systems2008.

11. Lanham HJ, McDaniel RR. Are we putting the cart before the horse? A microcosm of intended and unintended consequences of electronic medical records implementation. Paper presented at: Fifteenth America's Conference on Information Systems 2009.

12. Lanham HJ, McDaniel RR, Jr., Crabtree BF, et al. How improving practice relationships among clinicians and nonclinicians can improve quality in primary care. Jt Comm J Qual Patient Saf. Sep 2009;35(9):457-466.

13. Zhou X, Zheng K, Ackerman MS, Hanauer DA. Cooperative documentation: The patient problem list as a nexus in electronic health records (in press). Paper presented at: ACM 2012 Conference on Computer Supported Cooperative Work 2012.

14. Zheng K, Fear K, Chaffee BW, et al. Development and validation of a survey instrument for assessing prescribers perception of computerized drug-drug interaction alerts. J Am Med Inform Assoc. Dec 2011;18 Suppl 1:i51-i61.

15. Dennehy P, White MP, Hamilton A, et al. A partnership model for implementing electronic health records in resource-limited primary care settings: experiences from two nurse-managed health centers. J Am Med Inform Assoc. Nov-Dec 2011;18(6):820-826.

16. Zhou X, Ackerman MS, K Z. CPOE workarounds, boundary objects, and assemblages. Paper presented at: 29th ACM Conference on Human Factors in Computing Systems 2011.

17. Zheng K, Padman R, Krackhardt D, Johnson MP, Diamond HS. Social networks and physician adoption of electronic health records: insights from an empirical study. J Am Med Inform Assoc. May-Jun 2010;17(3):328-336.

18. Zhou X, Ackerman MS, Zheng K. Doctors and psychosocial information: Records and reuse in inpatient care. Paper presented at: 28th ACM Conference on Human Factors in Computing Systems 2010.

19. Zhou X, Ackerman MS, Zheng K. I just don’t know why it’s gone: Maintaining informal information use in inpatient care. Paper presented at: 27th ACM Conference on Human Factors in Computing Systems 2009.

20. Bohnsack KJ, Parker DP, Zheng K. Quantifying temporal documentation patterns in clinician use of AHLTA-the DoD's ambulatory electronic health record. AMIA Annu Symp Proc. 2009;2009:50-54.

21. Zheng K, Hanauer DA, Padman R, et al. Handling anticipated exceptions in clinical care: investigating clinician use of exit strategies in an electronic health records system. J Am Med Inform Assoc. Nov-Dec 2011;18(6):883-889.

22. Zheng K, Padman R, Johnson MP, Diamond HS. An interface-driven analysis of user interactions with an electronic health records system. J Am Med Inform Assoc. Mar-Apr 2009;16(2):228-237.