Coding Manual v1.0
Version Date: 1/08
CODING MANUAL
DRAFT v 1.0
IMPROVING QUALITY THROUGH HEALTH IT: TESTING THE FEASIBILITY AND ASSESSING THE IMPACT OF USING EXISTING HEALTH IT INFRASTRUCTURE FOR BETTER CARE DELIVERY
(HHSA29020060022, TASK ORDER #3 (HIT))
(Adapted with permission from Bryan Weiner, Ph.D.,
University of North Carolina-Chapel Hill)
Instructions
Background
Coding involves breaking interview transcripts into discrete “text units” and interpreting their meaning vis-à-vis deductively or inductively derived constructs. “Text units” are simply chunks of text that seem to have meaning or coherence. With ATLAS.ti, you can make text units any size we want: a word, a phrase, a line, a sentence, a paragraph, or an entire document. In general, though, larger text units are more interpretable than smaller text units. One-line text units, for example, often lack enough of the conversational context to render their meaning unambiguous. In this project, we will set the default text unit size as a sentence.
Text units that illustrate or inform a theoretical construct are tagged or labeled with a “code” that represents that construct. Codes are simply labels (representing constructs) that have defined parameters indicating when and when not to apply them.
To ensure consistency in coding, the investigators conducting this study have developed a coding manual that defines each code conceptually, outlines the decision rules for when to apply the code and when not to apply it, provides examples of appropriate and inappropriate uses of the code, and tracks any revisions that you make to the code’s definition or decision rules as the analysis proceeds.
To create the code book (which you have in your hand), we have used the study’s conceptual framework to generate a “starting list” of codes, which we plan to supplement with new codes as coding and analysis proceeds.
This code book is a “living document.” As we apply the codes to the interview transcripts, questions will arise about the meanings of the codes, the differences between codes, and the decision rules about when to apply codes. These questions will prompt discussion which, in turn, will prompt revisions and refinements of the code book. Definitions will get sharpened, new codes will get added, decision rules will be modified, and examples will get changed.
Directions for Coding
We will instruct you in the mechanics of using ATLAS.ti to attach codes and memos to text units.
Take an “inclusive” approach to coding. If you have any doubt about applying a code, your bias should be in favor of applying the code. If the data for the code are oblique—for example, if the meaning of a given text unit depends on a text unit elsewhere in the interview—then you should attach a memo to the given text unit with a note to the investigator about the decision to apply the code. The memo should reference the other text unit in terms of its ATLAS.ti location number (i.e., document and line number).
You should also attach memos to text units where you considered applying a code, but chose not to do so. Explain which code you chose not to apply and why you chose not to apply it.
As indicated in the decision rules below, code both positive and negative instances of a code. For example, use implementation climate to code both positive and negative statements about climate.
In the first cycle of coding, we will be coding at a broad or general level. In later cycles we will code at a more refined or micro level. For example, we will begin by coding for implementation policies and practices. Later, we will code for specific types of policies and practices (e.g., staffing changes, information systems).
If you feel that a new code is needed (i.e., an “emergent” code), please create the new code, code the text units in question, and alert the investigator.
ORGANIZATIONAL READINESS FOR CHANGE
Organizational Readiness for Change (ORC) refers to the extent to which targeted organizational members (especially the implementers and intended users) are psychologically and behaviorally prepared to make the changes in organizational policies and practices that are necessary to put the innovation into practice and to support innovation use [1].
ORC is a two-dimensional construct reflecting organizational members’ collective commitment and collective efficacy to implement an organizational change. Change commitment refers to organizational members’ shared resolve to pursue courses of action that will lead to successful change implementation. We emphasize shared resolve because implementing complex innovations involves collective action by many people, each of whom contributes something to the implementation effort. Change efficacy refers to organizational members’ shared beliefs in their conjoint capabilities to organize and execute the courses of action required to implement change successfully [2]. These shared beliefs result from organizational members’ exposure to external and internal stimuli, including past performance in implementing change [3, 4].
Comment:
Implementing complex innovations usually entails making a host of inter-related changes in organizational structures and activities. Organizational readiness to make changes such as these is reflected in the level of change commitment and change efficacy. We posit that organizations that exhibit high readiness are more likely than those that exhibit low readiness for change to initiate the changes in organizational structures, policies, and practices that are necessary to support innovation use, and to do so more effectively.
Use When:
This code applies only to the “pre-implementation phase”—that is, the period before the organization implements the information system (or laboratory information system) (i.e., the health information technology, or the HIT). Once implementation begins, this code is no longer pertinent.
Interview participants comment on the level of commitment that specific groups (e.g., nurses, managers, or physicians) or organizational members as a whole (i.e., “everyone”) had during the pre-implementation phase for implementing the HIT. Use the code regardless of whether the level of commitment was high or low, or whether the commitment was widely shared or limited to certain groups. Look for words like “motivated,” “supported” “excited,” “reluctant,” “skeptical,” ”open,” etc.
Interview participants comment on the level of confidence that specific groups (e.g., nurses, managers, or physicians) or organizational members as a whole (i.e., everyone) had during the pre-implementation phase that they could successfully implement the HIT—that they could mobilize the resources, take the actions necessary, and make adjustments along the way. Look for words like “can,” “could,” “confident,” “sure,” and “certain” (as well as their antonyms).
Do Not Use:
Interview participants comment on how things went (are going) after the pre-implementation phase. Consider using codes like management support or implementation climate.
Interview participants mention outcome expectancies: what might or might not occur if they successfully perform the action. Efficacy focuses on the question: Can I (or we) do this? Outcome expectancy focuses on the question: If I (or we) do this, what will happen?
Interview participants talk only about their own personal commitment or sense of efficacy about their role in the implementation process. If they do not reference collective (i.e., group or organizational) commitment or efficacy, then code the statements as personal readiness for change.
Interview participants make general statements about having an innovative or change-oriented culture. ORC is change-specific. An organization can have high ORC for certain types of change, and at the same time low ORC for other types of change. Code general statements like this as innovation-values fit. We consider such statements as indicators of innovation fit with organizational values.
Examples of Correct Use:
“People here were excited to implement an HIT.” (Code ORC because the statement refers to collective commitment/motivation to engage in the courses of action necessary to achieve desired goal.)
“The physicians were very interested in using the new HIT, but the nurses were not.” (Code ORC because the statement suggests a lack of shared commitment to implementing an HIT.)
“We were concerned about whether we could do it.” (Code ORC because the statement refers to confidence in performing actions necessary to achieve desired goal.)
Examples of Incorrect Use:
“I really wanted to see it happen.” (Do not code ORC unless this sense of commitment was shared by a specific group or organizational members as a whole. Code this as personal readiness.)
“Our practice has always been innovative. We’re always looking for new ideas.” (Do not code ORC because this statement is general, not change-specific. Code innovation-values fit.)
“We didn’t think the reimbursement would cover the true costs of the HIT.” (Do not code ORC because this statement does not refer to action capabilities. Code resources.)
MANAGEMENT SUPPORT (Leadership Domain, etc.)
Management support refers to managers’ shared resolve to pursue courses of action that promote the successful implementation of the innovation.
Comment:
Management support shapes the organizational context of implementation by providing resources to support implementation (allocating), clarifying and shaping the meaning of the innovation (communicating), and providing legitimacy to the innovation (championing)[5]. Management support is often critical because implementation is resource intensive. Financial, material, and human resources are more likely to be forthcoming if management supports the implementation effort. In addition, managers’ symbolic actions also contribute to successful implementation by signaling management’s commitment to the innovation and by convincing implementers and users to expend the effort required to put the innovation into practice [6, 7]. Finally, managers’ support is often critical for overcoming resistance and resolving intra-organizational disputes concerning changes in resource allocations, organizational routines, and lines of authority.
Use When:
Interview participants refer specifically to support for the EHR or EHRS among community health center leaders.
Interview participants mention management’s provision or non-provision of financial, material, or human resources to support implementation
Interview participants mention management’s verbal expressions of support (or lack of expressions of support) for the innovation, including statements about the innovation’s importance to the organization.
Interview participants mention management’s efforts to overcome resistance or otherwise alter the intra-organizational political situation regarding the innovation.
This code may overlap with IPP if the interview participant mentions management support in connection with deployment (or lack of deployment) of an implementation policy or practice.
This code may also overlap with ORC when interview participants comment on management support during the pre-implementation phase.
Examples of Correct Use:
We’ve had tremendous support from the health center management. The administrator values what we’re doing.” (Code management support.)
“We got along great with the vendor, but then they put us under the information services. We’ve had some real problems with them.” [Code management support.]
“The health center leadership gives us a half-time IT specialist.” [Code management support.]
Examples of Incorrect Use:
We requested and received an extra half-time IS person through another grant. She will focus exclusively on implementation. [Code IPP, not management support, since this resource allocation comes from external funding, not from management.]
PROCESSES AND ROLES (Processes and Roles, IT Tools, Communication of Results Domains, etc.)
Processes and Roles (PRs) refer to the current processes for ordering lab tests and reviewing results, and the roles of clinicians and administrators in following and modifying those processes and roles to support innovation use [8]
Comment:
PRs are the means by which an organization carries out the work of ordering laboratory tests and reviewing lab results. Some PRs are temporary processes and roles that intentionally or naturally disappear when the organization reaches desired levels of innovation use. Others remain in place long after the implementation phase in order to support and reinforce continued innovation use.
Organizations can make use of a rich inventory of PRs including current work flow, clinical roles, administrative roles, specifics about IT tools (e.g., systems, alerts, decision support), and communication of results to both clinicians and patients.
Use When:
Interview participants mention specific processes or roles describing how laboratory ordering and results reporting occurs that is associated with implementation of the EHR or the EHRS.
Current processes in lab ordering and results review
Roles and responsibilities of clinicians and administrators in refining processes and roles
New decision-making policies about processes or roles (e.g., new authority)
Current communication processes for orders (e.g., between health center and reference lab; between reference lab and health center; results communication to physicians, patients, and involved physicians in other settings)
Communication process for alerts and any changes (e.g., use of alerts within the HIT system)
Interview participants mention that a specific process or role is missing or needed (e.g., opportunities to improve the processes, change roles)
Interview participants mention a change in processes and roles that had an unintended effect on EHR/EHRS implementation and performance, or a way to work around a problem with current processes or roles.
Do Not Use:
Interview participants mention processes or roles that originate outside the community health center (e.g., from the reference labs). Code these policies and practices as External Actions.
Examples of Correct Use:
“Our health center administrator oversees everything.” (Code PR.)
“We really need a better way to track patients who have abnormal tests, and then feed that information back to the physicians.” (Code PR.)
Examples of Incorrect Use:
“The administrator left. The new administrator doesn’t support use of the EHR/EHRS.” (Code management support, not IPP.)
“We’ve got too much to do to use the chronic disease management forms. I mean, it’s just not worth it.” (Code implementation climate, not IPP…unless they talk about specific incentives or disincentives.)
IMPLEMENTATION POLICIES AND PRACTICES (IT Training, Chronic Disease Management Forms Domains, etc.)
Implementation Policies and Practices (IPPs) refer to the plans, practices, structures, and strategies that an organization employs to put the innovation into place to support innovation use [8]
Comment:
IPPs are the means by which an organization assimilates an innovation in order to achieve an acceptable level of operational, cultural, and strategic fit. Implementation policies and practices are the means by which an organization assimilates an innovation in order to achieve an acceptable level of operational, cultural, and strategic fit. The assimilation process, as others have noted, entails a mutual adaptation of the innovation and the organization [9, 10]. Some IPPs are temporary measures that intentionally or naturally disappear when the organization reaches desired levels of innovation use. Others remain in place long after the implementation phase in order to support and reinforce continued innovation use.
Organizations can make use of a rich inventory of IPPs including training and technical support, rewards or incentives, persuasive communications, end-user participation in decision-making, workflow or workload changes, alterations in staffing levels or mix, new reporting relationships, and/or documentation, monitoring or enforcement policies/procedures [8]. In general, the more IPPs, the better; yet, some high-quality IPPs may compensate for the absence or low quality of other polices and practices. Also, organizations can achieve the same level of implementation with differing mixes of policies and practices [8].
Use When:
Interview participants mention specific policies or practices intended to support the implementation of the EHR or the EHRS.
New decision-making policies or practices (e.g., new committees, roles, or authority)
Training and education (e.g., training ijn use of new IT tools, etc.)
Rewards or incentives (e.g., recognition, praise, monetary and non-monetary reward)
Persuasive communication (e.g., administrator or clinician exhorts colleagues in staff meetings)
Workflow or workload changes (e.g., use of chronic disease management forms)
New reporting relationships (e.g., new IT personnel and who they report to)
Changes in staffing levels or mix (e.g., hiring new individuals, redistributing work roles)
New documentation, monitoring, or enforcement procedures (e.g., tracking systems)
Use the code regardless of whether the described policy or practice was actually used or merely considered but postponed or rejected.
Interview participants mention that a specific policy or practice is missing or needed.
Interview participants mention either focusing on certain kinds of features of the system or otherwise adapting system attributes (e.g., advocating for changes in the EHR or EHRS system).
Do Not Use:
Interview participants mention policies or practices that originate outside the community health center (e.g., from HRSA). Code these policies and practices as External Actions.
Interview participants mention a change in policy or practice that had an unintended effect on EHR/EHRS implementation and performance. These changes are important, but they are not IPPs. Consider the possibility of coding these Implementation Climate.
Examples of Correct Use:
“We hired a great health center administrator.” (Code IPP.)
“We really need a better way to track patients who have abnormal tests, and then feed that information back to the physicians.” (Code IPP.)
Examples of Incorrect Use:
“The administrator left. The new administrator doesn’t support use of the EHR/EHRS.” (Code management support, not IPP.)
“We’ve got too much to do to use the chronic disease management forms. I mean, it’s just not worth it.” (Code implementation climate, not IPP…unless they talk about specific incentives or disincentives.)
IMPLEMENTATION CLIMATE
Implementation Climate refers to organizational members’ shared perceptions of implementation policies and practices in terms of their meaning and significance for innovation use [8]. (James & Jones, 1974; James & Snells, 1981)
Comment:
Implementation climate is a gestalt based on organizational members’ shared information about, discussions of, and experiences with the organization’s implementation policies and practices (IPPs) [8]. Through their actions, interactions, and discussions, organizational members make sense of the organization’s IPPs. The meaning and significance that they ascribe to these IPPs enable them to interpret current events, predict possible outcomes, and gauge the appropriateness of their own subsequent actions (Jones and James, 1979). Of particular interest here is whether organizational members (especially the implementers and intended users) get the sense from the organization’s IPPs that innovation use is expected, supported, and rewarded.
Climate is based on sense-making (interpretations). Norms might inform or shape organizational members’ interpretations, but norms are not synonymous with climate (or interpretations). For example, a strong norm to distrust management might prompt intended users to interpret in a negative light various changes in work roles intended to support innovation use (“They’re trying to take control of our work or pay us less”).
Organizations can create a strong implementation climate by making use of IPPs that enhance organizational members’ means, motives, and opportunities for innovation use. Like organizational readiness for change, implementation climate is innovation-specific. This specificity differentiates implementation climate from more general constructs like organizational climate or organizational culture. An organization may exhibit a positive climate in the workplace or a culture that values novelty and change, yet still exhibit a poor implementation climate for a specific innovation.
Use When:
Interview participants comment on the extent to which a particular IPP supports (or does not support) innovation use (i.e., engaging in those activities necessary to use the HIT). Support could take the form of enhancing knowledge and skills (means), encouraging effort (motives), or creating opportunities or removing barriers for innovation use (opportunities). Use of the implementation climate code will often, but not always, overlap with the use of the IPP code.
Interview participants mention that engaging in HIT-related activities is something that is expected, supported, and rewarded—even if they do not link this perception to a particular IPP. We would prefer that interview participants tie these perceptions to specific IPPs, but we don’t want to neglect to code “floating” perceptions (i.e., those not tied to specific IPPs).
Interview participants comment on the extent to which specific groups (e.g., nurses, managers, or physicians) or organizational members as a whole (i.e., “everyone”) share the perception that a particular IPP supports (or does not support) innovation use. Such perceptions might be widely shared, somewhat shared, nor not shared at all.
Do Not Use:
Interview participants talk only about their personal perceptions of IPPs and do not comment at all on whether those perceptions are shared by specific groups or organizational members as a whole. Use the psychological climate code instead.
Interview participants mention internal motivating factors as opposed to external motivating factors. Use the innovation-values fit code instead. Implementation climate is about people’s perceptions of their work environment—especially those aspects of their work environment pertaining to the innovation.
Interview participants focus on management’s support or lack of support for the HIT. Use the management support code instead.
Examples of Correct Use:
“I don’t think the physicians think about it much. They just don’t think about it when they’re seeing patients.” (Code implementation climate. If they didn’t think about because they didn’t care, then that might be innovation-value fit. But, in this example, they do care; they just have trouble remembering at the time of service delivery. This suggests that some critical IPPs are missing.)
“The nurses know what they are supposed to do….” (Code implementation climate. The nurses know what is expected of them. A better answer would indicate that they know what to do because they have clear understanding of the HIT and data required.)
Examples of Incorrect Use:
“The hospital CEO left. The new CEO doesn’t support HIT.” (Code management support, not implementation climate.)
“We keep track of nurses’ activity and we take that into account in annual performance reviews.” (Code IPP, not implementation climate. There is no mention of what the nurses think in terms of whether innovation use is rewarded, supported, or expected.)
IMPLEMENTATION EFFECTIVENESS
Implementation effectiveness refers to the consistency, quality, and appropriateness of innovation use within an organization [8, 11, 12].
Comment: Implementation effectiveness is an organization-level construct [8]. Generally speaking, the stronger an organization’s implementation climate, the greater the likelihood that targeted employees as a group will exhibit consistent, high-quality, appropriate innovation use.
Use When:
Interview participants comment on:
HIT usage and performance (collective, not individual, performance)
Number of physicians using HIT
Clinician productivity (not individual nurses, but nurses as a group)
Yield or efficiency of HIT-supported processes
Reduction in duplicate lab tests
Other metrics indicating effectiveness of HIT
Use to code both positive and negative statements about HIT usage and performance.
Do Not Use:
Interview participants comment on individual nurse or physician performance, not organizational (collective) performance.
Interview participants mention individual or organizational benefits of the HIT. Use the code innovation effectiveness instead.
Examples of Correct Use:
“We were able to reduce duplicate tests by about half. Our usage of the HIT was great.” (Code implementation effectiveness.)
“We’ve used the HIT regularly, but haven’t seen any changes in metrics.” (Code implementation effectiveness.)
“Dr. ____ is our star. He uses the HIT all the time.” (Code implementation effectiveness. Although this comment focuses on one physician’s performance, it signals that he or she is contributing disproportionately to collective performance. Because collective performance is implicitly referenced in the quote, code it implementation effectiveness.)
Examples of Incorrect Use:
“I haven’t used the HIT at all.” (This is about personal, not collective performance.)
INNOVATION-VALUES FIT
Innovational-Values Fit refers to the extent to which targeted employees perceive that innovation use will foster the fulfillment of their values [8, 11, 13, 14]. Values are concepts or beliefs that (a) pertain to desirable end-states or behaviors, (b) transcend specific situations, and (c) guide the selection and evaluation of behavior and events [15].
Comment:
While individual differences exist in values, we are more interested in values that are either shared by all organizational members (organizational values) or by groups of organizational members (group values). When an organization adopts an innovation, targeted organizational members form judgments about the extent to which an innovation is congruent with their values. A good fit exists when targeted organizational members regard the innovation as congruent with their high-intensity values. A poor fit exists when targeted organizational members view the innovation as incongruent with their high-intensity values. A neutral fit exists when targeted organizational members see the innovation as either moderately congruent with their high-intensity values or moderately incongruent with their low-intensity values.
Use When:
Interview participants comment on the fit (or lack of fit) that specific groups (e.g., nurses, managers, or other clinicians) or organizational members collectively perceive between the HIT and the values that they hold. For example, implementing and using an HIT might or might not be compatible with the following values:
Autonomy/flexibility/discretion/control over one’s work processes
Innovation/novelty/state-of-the-art/experimental/leader in the field
Evidence-based/scientific
Community-oriented/community benefit
Interview participants mention the importance that specific groups or organizational members as a whole ascribe to the abovementioned values. Whereas the first decision rule emphasizes fit, this decision rule emphasizes intensity, or the amount of feeling attached to a particular value.
Do Not Use:
Interview participants talk about personal values-fit rather than group or organizational values-fit. That is, they talk about themselves as individuals and do not reference groups within the practice or the organization as a whole.
Interview participants talk about operational fit (e.g., fit with workflow). Code these statements as innovation-task fit.
Interview participants talk about the fit of the HIT vis-à-vis the organization’s mission. Do not code these statements as innovation-values fit unless you get the sense that certain groups or organizational members as a whole believe in the mission (i.e., hold those values dearly).
Interview participants comment on the benefits or outcomes that result from using HIT. Consider coding these statements as innovation effectiveness. People can value the benefits or outcomes that result from the innovation (e.g., greater resources) but not necessarily value the innovation itself. There are many ways to monitor data, for example.
Examples of Correct Use:
“Our health center is committed to advancing the science. We might not have designed the HIT, but we want to stay at the forefront of the science and the evidence base.” (Code innovation-values fit.)
“Our health center is very community oriented. We are here for the community. Using this HIT increases access to state-of-the-art care in our community.” (Code innovation-values fit and innovation effectiveness.)
“The nurses strongly believe that we should be doing more in terms of helping patients manage their symptoms. The HIT alerts have really raised the physicians’ awareness of patients’ quality of life.” (Code innovation-values fit and innovation effectiveness.)
“Clinic management sees HIT as a drain, a cost.” (Code innovation-values fit.)
Examples of Incorrect Use:
“The CEO really feels this is important. He’s totally committed.” (This statement is more about individual values-fit than it is about group or organizational values-fit)
“The nurses feel like it’s important, you know. The nursing director is always stressing how important this is for the health center and the community.” (Code IPP and implementation climate, not innovation-values fit. The statement reflects the use of persuasive communication to create a positive climate for innovation use. The statement does not indicate whether innovation use—i.e., HIT use—promotes the fulfillment of nurses’ values).
RESOURCE AVAILABILITY
Resource availability refers to the accessibility of financial, material, or human assets that can be used to support initial and ongoing innovation use.
Comment:
It takes resources to initiate and maintain implementation policies and practices. The most important resources that are needed for HIT are (1) systems, (2) systems support and training, and (3) money. These resources must not only be available, but accessible. That is, they must be readily available in order to support implementation.
Use When:
Interview participants comment on:
The training available to the community health center for HIT use
Financial and non-financial assistance provided to support HIT
The amount and accessibility of external funding for HIT
Use to code both positive and negative statements about resource availability
Do Not Use:
Interview participants mention staffing issues (e.g., inexperience, turnover, short-staffing). Use the other barriers code instead.
Interview participants mention the general “support” provided by hospital (cancer service line) leaders. Code these statements as management support if no specific form of assistance is mentioned.
Examples of Correct Use:
“The health center gave us really nice printers to support lab report production. We don’t have to cover that.” (Code resource availability.)
“You have to find time to learn the system. That is very time-consuming.” (Code resource availability due to mention of no money for time of training, time away from clinical care.)
Examples of Incorrect Use:
“We lost one of our lab technicians. He left the area.” (Code personnel issues, not resource availability)
The health center’s been very supportive. They really value what we’re doing here.” (Code management support, not resource availability.)
RIVAL ACTIVITIES
Rival activities are events or actions that compete with the innovation for attention, resources, or both.
Comment:
Rival activities can absorb resources that might otherwise be available for implementation policies and practices. They can command management attention, drawing off management support for innovation implementation. Finally, they can create for targeted organizational members doubt, confusion, or conflict over organizational priorities, thus resulting in weaker implementation climate.
Use When:
Interview participants comment about competing priorities in the health center. Different initiatives represent a potential rival for HIT implementation and usage. Code statements that indicate either the presence or the absence of competition between HIT implementation and other activities.
Interview participants mention some organization-level, market-level, or sector-level event that draws away attention from HIT implementation (e.g., personnel incident or organizational merger.)
Do Not Use:
Interview participants mention some event or activity that they feel is important, but do not link that event to HIT implementation or performance.
Examples of Correct Use:
“Using HIT actually helps our efficiency. It gives us resources we would not otherwise have.” (Code rival activities and resource availability.)
“She’s been doing more and more systems support across the health center so her availability to support this HIT implementation has dropped.” (Code rival activities.)
“The two medical groups in town merged last year and that flooded us with patients.” (Code rival activities.)
Examples of Incorrect Use:
“Well Medicare cut reimbursement for laboratory services, but that didn’t affect our work with this HIT.” (This is an important event, but the comment does not suggest that it affected the HIT. Code other barriers if they though that the Medicare cuts affected HIT.)
INNOVATION EFFECTIVENESS
Innovation effectiveness refers to the benefits an organization realizes from an innovation [8].
Comment:
Innovation effectiveness is an organization-level construct, not an individual-level or group-level construct [8]. For community health centers, these benefits could include better patient care, higher employee morale, lower physician/nurse turnover, higher physician/nurse retention, or enhanced public image. Implementation effectiveness is a pre-requisite for innovation effectiveness. However, if the “program logic” of the innovation itself is faulty (e.g., if use of EHRs/EHRS does not really have quality-enhancing effects), or if during implementation the innovation has been adapted in ways that undercut its efficacy, no amount of consistent, high-quality, and appropriate use will generate the benefits anticipated by innovation adoption.
Use When:
Interview participant mention the presence or absence of measurable organizational benefits have resulted from use of the EHR/EHRS. Possible organizational benefits include:
Better patient care
Higher employee morale
Lower physician/nurse turnover
Higher physician/nurse retention
Enhanced public image
Improved practice (or hospital) revenues
Reduced duplication of laboratory test ordering
Better coordination of care
Faster communication of laboratory results
Improved ability to address abnormal lab results
Interview participants mention organizational benefits they perceive have or have not resulted from adoption of the EHR/EHRS. Note that this might overlap with innovation values-fit, especially if interview participants mention organization-level benefits.
Do Not Use:
Interview participants mention personal or individual benefits resulting from use of the EHR or EHRS.
Interview participants mention achieving HIT usage goals or targets. Use the implementation effectiveness code instead.
Examples of Correct Use:
“I believe that patient care has gotten better as a result of HIT use.” (Code innovation effectiveness.)
“We’ve been able to attract some top-flight clinicians because we have that HIT.” (Code innovation effectiveness.)
“It costs us money to use the HIT. Reimbursement doesn’t cover the full cost.” (Code innovation effectiveness.)
Examples of Incorrect Use:
“I feel like my own practice has gotten better as a result of using the HIT.” (This statement refers to personally realized benefits rather than organizationally realized benefits.)
“This is the third year in a row that we’ve improved our outcomes.” (Code implementation effectiveness, not innovation effectiveness.)
INNOVATION-TASK FIT
Innovation-Task Fit refers to the extent to which the innovation is compatible with task demands, work processes, and organizational capabilities.
Comment:
Implementation typically involves a process of mutually adapting the innovation and the organization to achieve a reasonable degree of operational, cultural, and strategic fit. Health centers can use a wide array of implementation policies and practices to adapt the organization to the innovation. However, health centers have less ability to alter the attributes of the innovations (HITs) to fit the organization’s task performance capabilities (e.g., patient populations served, workflow, etc.). This is because the decision about HIT may be centralized. Thus, even if a health center organization builds a strong implementation climate, implementation effectiveness (efficiency) will suffer if the HIT’s design characteristics not fit the community health center organization’s task performance capabilities.
Use When:
Interview participant make general comments about the ease or difficulty of using HIT.
Interview participants identify specific HIT attributes that make usage difficult (or easy). Examples include data collection forms, laboratory reporting requirements.
Interview participants identify specific organizational features that make HIT use easy or difficult. Examples include workload, workflow, staffing levels, or staffing mix.
Do Not Use:
Interview participants mention intrinsic motivating factors. Use the innovation-values fit code instead.
Interview participants talk about whether using HIT is rewarded, supported, or expected. Use the implementation climate code instead.
Interview participants mention specific organizational changes that they implemented, but do not explicitly reference some challenge or obstacle that these changes were intended to address. We expect that community health centers will use implementation policies and practices to increase innovation-task fit, but we do not want to infer innovation-task fit from comments that do not directly mention issues of fit.
Examples of Correct Use:
“The physicians don’t have the time to talk to patients about lab results.” (Code innovation-task fit because this comment indicates a poor fit with workload or workflow.)
“We weren’t putting enough patients on symptom management because the physicians didn’t know what symptoms the patients were experiencing. They were focused on treatment, not symptoms. So, we put a symptom management checklist in place through the HIT and patient satisfaction jumped.” (Code innovation-task fit, IPP, and implementation effectiveness.)
Examples of Incorrect Use:
“The nurses don’t feel that it’s that important.” (Code implementation climate, not innovation-task fit.)
“The physicians really believe in HIT. They see it as a great way to improve patient care.” (Code innovation-values fit, not innovation-task fit.)
INNOVATION CHAMPION
Innovation champion refers to a charismatic individual who throws his/her weight behind the innovation, thus, overcoming the indifference or resistance that a new idea often provokes in an organization.
Use When:
The champion's role is explicit. Interview participants identify someone who made a difference in implementation, particularly where they have made a personal investment in the innovation, e.g., putting personal prestige on the line.
Champions are likely to be the Administrator the Medical Director, some really influential clinician, or some really influential administrator. Use the code to capture descriptions of someone who goes “above and beyond” the call of duty on behalf of HIT implementation.
Do Not Use When:
Interview participants refer to support for the HIT among health center leaders. Use the management support code instead. When their advocacy of HIT implementation and use appears to be wholly related to their positional responsibilities."
Examples of Correct Use:
“Dr. ____ [the Medical Director] is the key to the whole thing. If a physician is giving me static, I just call up Dr. ___ and he personally gets on the phone and talks to the physician. He’s always there for us.” (Code innovation champion.)
“Dr. ___ [a urologist] somehow convinced all of the referring physicians in town use the HIT remotely.” (Code innovation champion.)
Examples of Incorrect Use:
“The CEO has always been a big supporter of the HIT.” (Code management support, not innovation champion.)
“The physicians really believe in HIT. They see it as a great way to improve patient care.” (Code innovation-values fit, not innovation champion.)
EXTERNAL ACTIONS
External actions refer to the policies and practices that the external groups put into place that affect HIT implementation effectiveness.
Comment: Community health centers are embedded in their communities, and influenced by a variety of external stakeholders. This code is designed to capture these “external” influences.
Use When:
Interview participants mention an external policy or practice that positively or negatively affects HIT implementation or performance.
Interview participants mention an external group policy or practice that positively or negatively affects HIT implementation or performance.
Interview participants mention an external group policy or practice that is missing or needed that could affect HIT implementation or performance.
Do Not Use:
Interview participants mention actions taken on the part of health center management. Use the management support code instead.
Interview participants mention policies or practices that the health center puts into place that affect HIT implementation or performance. Use the IPP code instead.
Examples of Correct Use:
“Medicaid has got to raise the reimbursement rate for using HIT. We can’t keep subsidizing this.” (Code External actions.)
“Some of the groups now let us enter our data using the Internet. That’s taken a huge load off the nurses. It took a while to learn, though.” (Code External actions.)
Examples of Incorrect Use:
“We really need a better way to track which clinicians are using the HIT, and then feed that information back to them.” (Code IPP not External actions.)
“The health center director gave us an extra half-time nurse when we started implementing the HIT last year.” (Code management support not External actions.)
PERSONAL READINESS FOR CHANGE
Personal Readiness for Change refers to the extent to which an individual is psychologically and behaviorally prepared to make the changes in organizational policies and practices that are necessary to put the innovation into practice and to support innovation use [1].
Like ORC, personal readiness is a two-dimensional construct reflecting individuals’ commitment and efficacy to implement an organizational change. Commitment refers to an individual’s resolve to pursue courses of action that will lead to successful change implementation. Efficacy refers to an individual’s confidence in his or her personal capabilities to organize and execute the courses of action required to implement change successfully.
Comment:
Readiness for change is a multi-level construct. Organizational readiness for change (ORC) captures collective readiness at the group or organizational level. Personal readiness for change captures individuals’ personal readiness.
Personal readiness for change is a “control” code. We are primarily interested in ORC, but we want to capture statement of personal readiness for change because this construct might represent a plausible alternative explanation for implementation effectiveness.
Use When:
This code applies only to the “pre-implementation phase”—that is, the period before the organization implements the HIT. Once implementation begins, this code is no longer pertinent.
Interview participants comment on their own personal level of commitment during the pre-implementation phase. Use the code regardless of whether the level of commitment was high or low. Look for words like “motivated,” “supported” “excited,” “reluctant,” “skeptical,” ”open,” etc.
Interview participants comment on their personal level of confidence during the pre-implementation phase that they could carry out those tasks necessary to successfully integrate the HIT within their organizational role. Look for words like “can,” “could,” “confident,” “sure,” and “certain” (as well as their antonyms).
Do Not Use:
Interview participants comment on how things went (are going) after the pre-implementation phase. Consider using codes like psychological climate.
Do Not Use When:
Interview participants talk about collective commitment or collective capabilities. Use the ORC code instead.
Examples of Correct Use:
“I didn’t think I could use an information technology.” (Code personal readiness for change if comment refers to beliefs held in the pre-implementation phase.)
“I wanted to see this happen.” (Code personal readiness for change, especially if interview participant gives no indication there were other organizational members who shared his or her commitment.)
Examples of Incorrect Use:
“I wasn’t sure that we could meet those goals for using the HIT.” (Code ORC, if statement refers to a belief held during the pre-implementation phase, because the interview participant refers to “we.”)
“I don’t think it’s a priority.” (Code psychological climate because the interview participant does not indicate whether there are other organizational members who share this perception.)
“Since we started using chronic disease management forms, I’ve changed how I interact with patients.” (Code implementation policies and practices.)
PSYCHOLOGICAL CLIMATE
Psychological Climate, as the term is used here, refers to individual organizational members own perceptions of implementation policies and practices in terms of their meaning and significance for innovation use [8]. (James & Jones, 1974; James & Snells, 1981)
Comment:
Psychological climate refers to the perceptual and experiential components of a reciprocal interaction between the organizational environment and the employee (Michela, Lukaszwski, & Allegrante, 1995). It has been conceptualized as a construct ‘‘comprising an individual’s psychologically meaningful representations of proximal organizational structures, processes and events’’ and as ‘‘a means of explaining an individual’s motivational and affective reactions to change’’ (Parker et al., 2003, p. 390).
Climate is a multi-level construct. Psychological climate refers to individual perceptions of the way things are done around here. It concerns the perceptions of individual employees as to what is expected, rewarded, and supported in the organization. When individual employees share the same perception of the work environment, then an organizational climate is said to exist.
In this project, psychological climate is a “control variable.” The construct does not figure in our conceptual model. Our interest in coding for it is to exclude psychological climate as a plausible alternative explanation for implementation climate.
Use When:
Interview participants talk only about their personal perceptions of IPPs and do not comment at all on whether those perceptions are shared by specific groups or organizational members as a whole.
Do Not Use:
Interview participants comment on the extent to which specific groups (e.g., nurses, managers, or physicians) or organizational members as a whole (i.e., “everyone”) share the perception that a particular IPP supports (or does not support) innovation use. Such perceptions might be widely shared, somewhat shared, nor not shared at all.
Examples of Correct Use:
“Yes, I feel that I am expected to use the HIT. I know how when I am expected to use it.” (Code psychological climate because there is no reference here to anyone other than the interview participant.)
Examples of Incorrect Use:
“The nurses know what they are supposed to do…when they have to use the HIT.” (Code implementation climate, not psychological climate because there is a reference here to a specific group. The nurses know what is expected of them. A better answer would indicate that they know what to do because they have clear HIT goals and routinely receive performance information.)
“I didn’t think I would have any problem using the HIT.” (Code personal readiness for change if the interview participant was commenting on the pre-implementation phase.)
OTHER BARRIERS
Other Barriers refers to challenges or obstacles that affect HIT implementation and performance that do not fit the codes listed above.
Comment:
Other Barriers is a “residual” code. Use it to capture challenges or obstacles that seem important but do not fit the more analytically useful codes above.
Use When:
Interview participants mention a challenge or obstacle, but no other code seems to apply.
Examples could include staffing challenges (e.g., turnover), patient population challenges (e.g., distrust), health care payment challenges (e.g., Medicare cuts), and regulatory challenges (e.g., adverse event reporting).
Do Not Use:
Another code applies.
Examples of Correct Use:
“We’re getting killed trying to keep up with all the adverse event reporting requirements.” (Code other barriers.)
“We just lost a key physician. He moved out of state.” (Code other barriers.)
Examples of Incorrect Use:
“The nurses don’t feel that it’s that important [to use HIT]. There’s no one breathing down their necks to do it.” (Code implementation climate, not other barriers.)
“It costs us money to use the HIT. Reimbursement doesn’t cover the full cost.” (Code innovation effectiveness and External actions, not other barriers.)
“We need better training.” (Code IPP, not other barriers.)
REFERENCES
1. Armenakis, A.A., S.G. Harris, and K.W. Mossholder, Creating Readiness for Organizational Change. Human Relations, 1993. 46(6): p. 681-703.
2. Bandura, A., Self-efficacy : the exercise of control. 1997, New York: W.H. Freeman. ix, 604.
3. Lindsley, D.H., D.J. Brass, and J.B. Thomas, Efficacy-Performance Spirals - a Multilevel Perspective. Academy of Management Review, 1995. 20(3): p. 645-678.
4. Zaccaro, S.J., et al., Collective Efficacy, in Self-Efficacy, Adaptation, and Adjustment: Theory, Application, and Research, J.E. Maddux, Editor. 1995, Plenum Press: New York. p. 305-330.
5. Sharma, R. and P. Yetton, The contingent effects of management support and task interdependence on successful information systems implementation. Mis Quarterly, 2003. 27(4): p. 533-555.
6. Leonard-Barton, D. and I. Deschamps, Managerial Influence in the Implementation of New Technology. Management Science, 1988. 34(10): p. 1252-1265.
7. Purvis, R.L., V. Sambamurthy, and R.W. Zmud, The assimilation of knowledge platforms in organizations: An empirical investigation. Organization Science, 2001. 12(2): p. 117-135.
8. Klein, K.J. and J.S. Sorra, The Challenge of Implementation. Academy of Management Review, 1996. 21(4): p. 1055-1080.
9. Leonard-Barton, D., Implementation as Mutual Adaptation of Technology and Organization. Research Policy, 1988. 17(5): p. 251-267.
10. Leonard-Barton, D. and W.A. Kraus, Implementing New Technology. Harvard Business Review, 1985. 63(6): p. 102-110.
11. Holahan, P.J., et al., Implementing Computer Technology: A Multi-Organizational Test of Klein and Sorra's Model. Journal of Engineering and Technology Management, 2004. 21: p. 31-50.
12. Klein, K.J., A.B. Conn, and J.S. Sorra, Implementing Computerized Technology. Journal of Applied Psychology, 2001. 86(5): p. 811-824.
13. Leonard-Barton, D., Implementation as Mutual Adaptation of Technology and Organization. Research Policy, 1988. 17: p. 251-267.
14. Leonard-Barton, D., Implementation Characteristics of Organizational Innovations. Communication Research, 1988. 15: p. 603-631.
15. Schwartz, S.H. and W. Bilsky, Toward a Theory of the Universal Content and Structure of Values - Extensions and Cross-Cultural Replications. Journal of Personality and Social Psychology, 1990. 58(5): p. 878-891.
Draft: 1/08
| File Type | application/msword |
| File Title | Version Date: 07/01/07 |
| Author | Preferred Customer |
| Last Modified By | jyonek |
| File Modified | 2008-01-25 |
| File Created | 2008-01-25 |