Methods for ?User Experience and Satisfaction with the Virtual Lifetime Electronic Record (VLER) Health Data Exchange

D raft – 08/16/2011

Methods for “User Experience and Satisfaction with the Virtual Lifetime Electronic Record (VLER) Health Data Exchange”

Provider Experience and Satisfaction

Overview

The goal of VLER Health data exchange is to improve quality and continuity of care for patients, and achievement of this goal is predicated on acceptance and use of VLER Health by providers from all participating organizations. Given the importance of provider usage to the success of VLER Health, VA leadership has identified provider satisfaction as an official criterion in the decision to deploy VLER Health at the national level. VA is also interested in the satisfaction of PSPs as they treat a shared patient population. This qualitative analysis will assess provider satisfaction, the usability and usefulness of the VLER system, as well as the effects of training and programs designed to motivate use, such as participation remuneration. Real-life examples, anecdotes, and stories of how VLER has impacted the care of Veterans will be captured through this qualitative component of the analysis.

Questions of Interest:

1. Are providers aware of VLER Health and what it can provide?

2. Are providers satisfied with the usability, including learnability and ease of use, of the VLER technical system?

3. What do providers perceive as the potential and actual impacts of VLER Health on their clinical practice?

4. What are the facilitators and barriers to use?

5. What are providers’ perceptions of participation remuneration programs to increase use?

Certain aspects of VLER Health program, such as provider training and the design of the user interface, are particularly relevant to the questions of interest and were considered when developing the qualitative guides and the analysis plan.

Education and Training

VLER Private Sector Partners PSPs are facilitating use of VLER Health by conducting training sessions with end-users. Training sessions include a review of the goals of VLER, a summary of participating organizations, and step-by-step lessons in how to operate the system. Training also includes a discussion of the potential benefits of VLER Health to both the patients and the providers.

Table 1. Measurement of Provider Satisfaction

Performance Measures	Reporting Frequency
Number of positive/negative experiences based on “a composite qualitative measure that includes usefulness, usability, etc. (CONOPS, v.1.04)** Westat will review each provider’s responses across relevant domains and assign a summary measure of positive or negative experience**	Once at end of measurement period**
Qualitative (i.e., narrative) data by sub-domain	Once at end of measurement period (interviews);
6-month measurement period is October 1, 2011 to March 31, 2012 ** “Go/No Go” measure

Data Sources and Collection

In-depth provider interviews will be conducted to capture both a breadth and depth of information on providers’ experiences with VLER Health. Semi-structured interviews will be administered during the October 2011 to March 2012 measurement period. Interviews will cover provider awareness of VLER, perceived usefulness and usability, and facilitators and barriers to use.

Feedback from PSPs will be collected through in-depth telephone interviews across all pilot sites, using the semi-structured VLER Health Provider Interview Guide. Interviewers will have the opportunity to ask clarifying and/or probing questions. For example, probes could be used to clarify whether the provider is referring to a patient that is shared with VA or with the DoD. Two members of the Westat staff will participate in each interview. One staff member will conduct the interview, and the second staff member will take detailed notes.

Up to ten VA PSP providers (e.g., physicians, nurses, laboratory managers, etc.) will be interviewed per VHC. In-depth interviews will be conducted over a six-month period from October 2011 to March 2012 at the following eleven VLER pilot sites:

• Hampton, Virginia

• Richmond, Virginia

• San Diego, California

• Indianapolis, Indiana

• Spokane, Washington

• Puget Sound, Washington

• Rural Utah

• Asheville, North Carolina

• Charleston, South Carolina

• Minneapolis, Minnesota

• Buffalo, New York

Providers will be eligible to participate if they had a scheduled clinical encounter with a VLER patient between October 2011 to March 2012. The eligible providers will be identified using local appointment lists. In addition, the feasibility of identifying some providers who provided unscheduled urgent or emergent care to VLER patients will also be explored. Westat staff will contact eligible providers by email to schedule interviews. The interviews will be 10-15 minutes in duration. Participation is voluntary, and participants can choose to end the interview at any time or decline to answer a specific question.

Analysis and Reporting

Notes from the interviews will be analyzed to identify common responses, themes and insightful vignettes. Descriptive statistics of the respondent population will be prepared. Results will be compared across organizations, across VHCs, and by provider characteristics (e.g., specialty, years practicing), as appropriate. Interview findings will be reported as part of the interim report in July 2012.

An overview rating of the in-depth interviews will be used to inform a specific Go/No Go criterion related to the percentage of providers who had a positive experience with VLER. A rating of positive or negative will be given to each completed interview based on the respondent’s answers to a small number of core questions.

Example core questions for overview rating of provider interviews

• Do you agree or disagree with the following statements

• Overall, I am satisfied with the way VLER works. (Please explain why.)

• The data that are available through VLER are useful to me. (Please explain why.)

• It was a good use of my time to look up a patient’s information using VLER. (Please explain why.)

• How has (or how do you think) VLER will affect you as a care provider? Are these effects benefits or drawbacks?

• Have you experienced any situations in which VLER was valuable? (Please describe.)

Overview ratings will be reported with contextual information from the qualitative analysis to provide a richer understanding to Go/No-Go decision makers.

Veteran Experience and Satisfaction

Overview

VLER Health data exchange is designed to improve the health and healthcare of Veterans, and that value can only be derived if Veterans choose to participate in VLER Health data exchange. The Veteran experience during initial recruitment and ongoing use has a direct effect on their willingness to participate in VLER Health as well as their long-term perceptions about their health and the healthcare system that supports them. Further, the Veteran experience with VLER Health will be used to derive measures that are criteria for the Go/No Go decision for VLER Health. Real-life examples, anecdotes, and stories of how VLER has impacted the care of participating Veterans will be captured through this qualitative component of the analysis.

Questions of Interest:

1. What are Veterans’ perceptions of their experiences with VLER Health?

2. How positive or negative are their experiences?

3. Are Veterans satisfied with the VLER initiative?

4. What impact do Veterans perceive VLER Health has had on their care?

Veterans are not direct users of VLER Health but, instead, are participants of the VLER and NwHIN initiatives. As such, their experience with VLER Health is derived from the interactions with the authorization forms, staff members, and care personnel who help implement the program. The sum of this experience will comprise a Veteran’s overall level of satisfaction with VLER.

Education and Training

The VLER Program Office shares a standard set of educational materials through a variety of channels, including mail, the VA website, or on-site at VA facilities. Veterans can also speak directly with VLER program staff while they are on-site, or they can call a toll-free number to reach telephone support. The educational materials describe VLER in terms of the local healthcare that organizations are participating, the types of data that are being exchanged, and the individuals who are allowed to access VLER (e.g., healthcare providers). The materials also describe potential benefits to the patients, such as improved care and fewer repeated tests. These educational materials are anticipated to increase Veterans’ general understanding of the VLER program. An assessment of Veterans’ level of awareness and understanding of VLER may identify expectations and perceived benefits which in turn may impact whether their experiences are positive or negative and their level of satisfaction. Furthermore, this information may uncover the need to increase and improve education and information about VLER.

Authorization Forms

To participate in VLER, Veterans must complete a “Request for and Authorization to Release Protected Health Information to Nationwide Health Information Network”. This form has been certified as HIPAA-compliant. It specifies exactly what types of protected health information will be released, to whom, and over what duration of time. Failure to complete this form correctly will result in the omission of a Veteran from the VLER data exchange. VA stakeholders have indicated that it would be helpful to understand if the authorization form is user-friendly and clear to Veterans.

Experience and Activation

Understanding the types of experience that Veterans have had and expect from the VLER program is not only important in terms of informing the Go/No Go decision, but is also important for understanding whether it increases the Veterans’ level of activation, that is their control of and engagement in their care.

Table 2. Measurement of Veteran Experience and Satisfaction

Performance Measures	Reporting Frequency
A positive experience based on a composite qualitative measure that includes usefulness, usability, etc.**	Once at end of measurement period**
Qualitative data within the domains of: Awareness of VLER Satisfaction with the VLER Program Value/Benefit Experience / Activation	Once at end of measurement period (interviews);
6-month measurement period is October 1, 2011 to March 31, 2012 **Go/No Go measure

Data Sources and Collection

In-depth interviews will be conducted to capture both a breadth and depth of information on Veteran experiences with VLER. Interviews will be administered first during the October 2011 to March 2012 measurement period. Interviews will cover Veteran awareness of VLER, usability of the authorization forms, perceived value and benefit, and overall satisfaction with the program.

Interviews

A semi-structured interview guide has been developed to assess Veterans’ perceptions and experiences while participating in the VLER program. The interviews will be conducted by phone. Two Westat staff will participate in each interview. One staff member will conduct the interview, and the second staff member will take detailed notes.

A convenience sample of approximately ten participants per VHC will be interviewed depending upon interest and availability. Eligible participants will be identified using the “Tracker Spreadsheet”, which is used by VA staff to track VLER authorization (opt-in). Local appointment lists will be used to identify Veterans whose health information may have been retrieved during a patient encounter. Veterans who receive care at the following eleven sites may be interviewed.

• Hampton, Virginia

• Richmond, Virginia

• San Diego, California

• Indianapolis, Indiana

• Spokane, Washington

• Puget Sound, Washington

• Rural Utah

• Asheville, North Carolina

• Charleston, South Carolina

• Minneapolis, Minnesota

• Buffalo, New York

Analysis and Reporting

Veteran interviews will be analyzed by general content analysis, coding of patient responses, and abstraction of common themes and response categories. These themes and commonalities will be compared across pilot sites and compared against different strategies for recruitment and authorization, level of provider participation, and degree to which patient data is communicated within a community. Characteristics of the population interviewed will be produced, and, where appropriate, described by VHC (e.g., outreach and authorization methods) and Veteran characteristics (e.g., age, gender). Findings from the Veteran interviews will be reported as part of the interim report in July 2012.

Specific to Go/No Go criteria established within the CONOPS⁽¹⁾,Veteran responses will be analyzed to assess whether the majority of Veterans interviewed indicated a positive experience with the VLER program. This information will then be reported to assist in informing the Go/No Go decision. An overview rating of positive or negative will be given to each interview based on the respondent’s answers to five core questions.

Example core questions for overview rating of Veteran in-depth interviews

• How easy or difficult was it to understand the educational materials about VLER?

• How easy or difficult was it to fill out the authorization forms provided to you?

• In comparison to physically carrying around your medical records, do you think that VLER will make it easier, harder, or about the same to get information from one provider to another?

• Do you think VLER would benefit Veterans generally?

• Overall, how satisfied are you with VLER?

Overview ratings will be reported with contextual information from the qualitative analysis to provide a richer understanding to Go/No Go decision makers.

4