Responses to OMB 6-10-2009

Regenstrief Institute 06/10/09

Re: 73 FR 74721 Page 7

Supporting Statement Part A

* As this is a case study, will AHRQ be generalizing the results beyond Indiana? What are the characteristics of the Indiana HIE that make it similar and dissimilar from other HIEs? Does the INPC represent a "best case scenario" of sorts, in the sense that it is established and already includes larger provider groups?

As this is a case study, AHRQ understands that this study will have limited generalizability but will still be useful for many HIEs. INPC represents a best case scenario in that it is established and includes a large percentage of providers in the Indianapolis area. It is similar to other HIEs in that it has had to develop governance structures, data sharing agreements, make technology decisions, determine data types for exchange, etc., like any HIE has to do. It is also similar to other HIEs in that it started the data exchange among the large hospitals in the market, and that it has relied on various funding streams (user fees, public and private grant and contract funds) to sustain itself. It is dissimilar in that it has been operating for over 10 years and has a stable technical and operational platform. As a successful, long established HIE, Indiana may provide useful lessons for other HIEs

* What percent of small hospitals, small physician practices, and large physician practices already participate in the INPC?

Approximately 15% of small hospitals, 4% of small physician practices, and 17% of large physician practices in Indiana already participate in the INPC.

* What other studies have been conducted about barriers to HIE participation in other states? Do the other HIEs also have trouble attracting small hospitals, small physician practices, and large physician practices, or do they also have trouble attracting some of the larger providers (e.g. hospitals, Medicaid program, etc.)?

There have been some papers written about attracting stakeholders to participate in HIEs. One study looked specifically at overcoming barriers to stakeholder participation. That study was based on discussions with potential stakeholders of four HIEs, and did not involve a systematic survey of non-participating stakeholders. (Grossman, J., Kushner, K., November, E., "Creating Sustainable Local Health Information Exchanges: Can Barriers to Stakeholder Participation be Overcome?" Research Brief No. 2, February 2008. http://www.hschange.org/CONTENT/970/#ib3 accessed March 2009). Anecdotally, and according to the Grossman report, other HIEs also have trouble attracting small hospitals, small and large physician practices.

Supporting Statement Part B

* How did AHRQ determine that 20 small hospitals, 40 small physician practices, and 40 large physician practices were the right sample sizes for this study?

We identified these three key stakeholder groups. This list was determined as being proportional to the list of all such organizations in the State of Indiana within each of these three groups. We selected these sample sizes to provide geographic distribution across the regions of the state and to provide enough variation.

* What will AHRQ do if a 75% response rate is not obtained? Will there be non-response bias analysis conducted? If so, what type of analysis will be performed?

We will continue to call additional potential participants until we obtain a 75% response rate. We have every expectation, based on our prior experience providing healthcare expertise and assisting in the development and implementation of healthcare solutions in the State of Indiana, that we can obtain the stated level of participation. We will monitor the response and response patterns carefully, and should we obtain response under 70 percent, we will examine the patterns of response and include that information in publications. Given that this is a case study, with limited generalizability to begin with, a formal, highly structured non-response analysis is not planned.

* What percentage of the respondents will already be participating in an HIE, what percentage will have been contacted about INPC but not participating, and what percentage will be neither participating in INPC and never have been contacted?

In this case study we will learn what percentage of respondents are already participating in an HIE as the survey is conducted; what percentage have been contacted about INPC but are not participating; and, what percentage is neither currently participating and never connected about INPC.

OMB: Because the goal is to identify the barriers to participation and solicit feedback on how to overcome those barriers, it seems better to focus on those who are not yet participating. We see little utility in surveying respondents that are already participating. Since you should know who is already participating, we would recommend excluding them from your sample. As mentioned previously, what sets this study apart from previous study is that you are looking systematically at non-users, not users. So please focus on non-users.

We agree that it is better to focus on those who are not yet participating in health information exchange. Because we hope to identify actual, as well as potential, solutions to the barriers we identify we believe there is benefit in including a minority of respondents who may be participating in the INPC or other health information exchanges in order to elicit how they overcame these barriers. We have modified the protocol to ensure that at least 75% of respondents will be non-participants.

* Information collection procedures:

• Please send a copy of the "communication packet" to OMB.

The “communication” includes: (a) an HIE description and definition; (b) description of the INPC, its organization mission, overall direction, and other relevant background information; and (c) purpose for the contact, estimated time required to complete the web-based questionnaire and a link to the questionnaire and is included in the Attachment C screener document and Attachment E telephone interview. This information was erroneously labeled as a communication “packet.”

• Is there some reason why the "communication packet" isn't the first encounter with the study that the participant receives? Why is the first interaction a screening interview? In order to agree to participate in the study, it seems like the participants will need the information contained in the "communication packet," which presumably also includes the information regarding the confidentiality of their responses.

The communication isn’t included in the first encounter with the study because in the first encounter we learn the name of the appropriate individual to complete the survey and the respondent’s email address. The initial contact is to explain the survey process and identify the most appropriate individual to participate in the survey.

OMB: what is the “script” that the caller is supposed to use during this initial call? Attachment C is a list of bullet points that describe what the study is. It does not mention anything about what the caller is actually going to say in order to determine who the right person is at the organization to complete the survey.

We have revised the attachment to include a specific script that we will use to introduce the project and to identify the key informants

• On page 4, the supporting statement says that the follow-up interview will "determine the steps necessary to overcome the barriers to HIE identified in the web-based questionnaire." However, it seems like the questions used in the interview only repeat the questions used in the survey. Please clarify how the interview questions will help AHRQ determine what steps are necessary to overcome the barriers that are not already solicited in the survey.

The web-based questionnaire includes structured responses whereas the telephone survey provides an opportunity to follow-up to the web-based questionnaire with more detailed information and responses to open-ended questions. Some of the telephone survey questions are based on the responses in the survey, so less time is taken up during the phone interview.

OMB: It is still unclear how the interview questions will allow AHRQ to “determine the steps necessary to overcome the barriers to HIE identified in the web-based questionnaire.” Please elaborate.

We have restructured the telephone interview questions to directly elicit from the interviewee the potential or actual ways that the barriers may be overcome. The interviewer will explore internal and external changes that might allow an organization to overcome the barriers the interviewee identifies.

• Given that the follow-up interview largely duplicates the web-survey, it seems the information collection procedure could be improved by merging several of the encounters. This would also improve response rates. OMB would suggest a procedure like this: 1) the participants receive the "communication packet" which contains more in depth information about the study than the screening interview document currently does (attachment C). 2) those who agree to participate will be interviewed over the phone, to collect the responses to the survey questions as well as to probe more in-depth about those survey questions. Step 2 would effectively merge the survey and the follow-up interview in one step. Since the survey contains a long list of response categories, a copy of these questions and response categories could be included in the "communication packet" so that the participant can read along as the phone interview is occurring.

Because we are approaching three different types of organizations, the first step in our collection procedure is to learn the name and contact information, at a particular organization, of the most appropriate individual to complete the questionnaire, and participate in the telephone interview. This is most likely not the first person to take our initial call. We don’t know who the respondent will be and whether they will agree to participate. If we mailed the “communication” without agreement to participate, we anticipate the survey request may be lost in the workflow and we will not be contacted.

We considered the work flow in each of the types of organizations that will be contacted about this survey and determined this method to provide the best chance at obtaining a 75% response rate There is not an HIE specialist at each hospital, and practice site, therefore an initial phone call to identify the individual will be necessary. The second interview is to follow up with more detail to the structured responses provided in the web survey.

OMB: It seems to us that the web survey and the phone interview are largely duplicative of each other. As such, we cannot approve this study as it is currently designed. We agree that the purpose of the interview should be to follow up with more detail on the responses to the web survey, but the interview schedule as currently designed will not elicit that detail. By contrast, many of the interview questions are yes/no or multiple choice questions, which are more appropriate for a web survey.

If AHRQ is interested in yes/no or multiple choice responses, we would recommend folding the questions from the interview into the web survey. Alternatively, the interview should be redesigned to elicit information that is not currently being obtained in the web survey. And the focus of the interview should be on eliciting feedback from the non-participants, not only on the barriers they face but on how those barriers might be overcome.

We have combined the web survey and telephone interview to eliminate duplication and reduce the amount to time required from the interviewee.

Instruments

* Attachment C (initial screening interview) is too cursory. What will the screeners actually be telling the participants? Isn't there some kind of script? How are they going to disclose confidentiality provisions, for example? More detail should be provided.

A script has been developed for the screening interview. This script is attached. A confidentiality statement has been added to Attachment C screener document.

* Web survey

• If participants receive the "communication packet," won't they all know what HIE and INPC are? (first 2 questions on the web survey)

The web survey has been changed to reflect this concern.

OMB: The web survey begins by defining what HIEs are and what the INPC is. Therefore, wouldn’t you expect that everyone will answer “yes” to the first 2 questions?

Also, if the concern is that participants may THINK they know what HIEs are when actually they confuse it with EMRs, then how would the questions—as they are currently posed—elicit the response you are looking for? People may mistakenly believe that HIEs are EMRs and, if asked question #1, they will simply say “yes,” they are familiar with the term HIE.

If the concern is to make sure that respondents understand the distinction between HIEs and EMRs, then the appropriate question to ask is, “Are you familiar with the distinction between HIEs and EMRs?” It may even be better to simply tell the respondent what the distinction is.

In addition to adding a brief distinguishing sentence in the overview materials, we explicitly ask the interviewer to probe to be sure that the interviewee understands the distinction between HIE and EHRs.

• OMB would recommend including the response categories suggested by AHIMA in their 1/7/09 comment.

The response categories suggested have been included in the revised web survey that is attached.

• Question 3biii is worded ambiguously. For example, it is not clear whether "knowledge of available HIEs" is the barrier in question or whether it is LACK of knowledge. What does "engage staff to participate" mean? Is the barrier here the staff or the energy required by the administrator to engage the staff? What does "reliability" mean? Reliability of what? And are you looking for lack of reliability as the barrier? Please clarify the response categories.

To provide clarification, the web-based survey has been revised to indicate the barriers as a lack of awareness; lack of knowledge; difficulty to engage staff due to lack of openness to change; difficulty to engage staff due to resource limitations; and lack of confidence in reliability of HIE technology.

* Follow-up interview

• What is the content in the box on page 1. Is this going to be read to the participant?

The content in the box on page 1 is the communication “packet.” The respondent will receive this information in the email communication, and the information is provided on the phone interview as a reminder to the interviewer to review the information with the respondent, if necessary.

• What is the rationale for what is in blue text and what is in black text? How is the interviewer deciding when to ask the questions in blue? Some of the blue questions seem important: why are they only being asked some times?

The black text depicts the questions to be asked of the respondents, the blue text is text provided to the interviewers for consistency in gathering the data during the interview.

• How is the interviewer supposed to know what the "high points" are?
  
  

The high points will be dependent upon whether the respondents participate in an HIE. The high points are in the script: HIE participation (past, current, or future), previous experiences, and barriers identified in web survey responses.

• What does "Decision making" and "Selection process" mean? Please use full sentences for your questions. Please also explain the utility of these questions.

To provide better explanation, we made revisions in this document to use full sentences for these questions. “Decision making” and “selection process” mean “do we” or “don’t we,” and to what extent.

• Page 3: Since the focus is on smaller provider groups, shouldn't there be a question that asks how HIEs can be more "user friendly" for smaller groups rather than asking whether the HIE better fits larger or smaller organizations?

Our questions are structured in this manner because in many cases, a system could be user-friendly, but not fit within the technological infrastructure of the group.

OMB: it is not clear what this means. Our understanding is that AHRQ is particularly interested in what the barriers to participation are for small provider groups. However, there aren’t too many questions that probe what it is about HIEs that make it unattractive to small provider groups. So we think the utility of this collection could be increased if more of the questions probed this issue.

The survey and interview guide have been updated to address this concern. The focus has now been placed on the barriers and how to overcome them.

• Page 3: Why is the question "would you make modifications to the HIE that you participate in?" only asked of those people who did not report financial limitations? What does "current participating services" mean?

Revisions were made to provide clarification of this question.

OMB: it is still unclear. Are you limited the respondents to this question to only those people who had no financial limitations? That is how it currently reads. Also, if you are asking people to consider what they would want if they had no financial limitations, then why would they have to take into consideration “only the services that can be accessed through your licensing?” Those two statements seem contradictory.

This question has been removed as we feel it may not be reflective of the overall theme in overcoming barriers to implementation of HIE.

• Page 4: Why is the question "would you like other types of services to be offered from a HIE" only asked of those people who said there were no financial limitations or services with no access? What does "services with no access" mean?

Revisions were made to provide clarification of this question.

OMB: it is still unclear. Are you limited the respondents to this question to only those people who had no financial limitations? That is how it currently reads. Also, “include all services that are available” is not clear. What does this mean? What informatioz are you trying to get at?

This question has been removed as we feel it may not be reflective of the overall theme in overcoming barriers to implementation of HIE.

• Page 5: This section seems to be the most important because they cover those providers who do not currently participate in the HIE. However, the questions only ask what they perceive to be the benefits of HIE. It also basically repeats the question about barriers from the web-survey. Should there be more probing questions for this section about what the barriers are and how they might be overcome? Isn't this the goal of the study?

We added additional potential barriers to those listed in this section, and instructions to the interviewer for follow-up questions that will provide an opportunity for more dialogue about barriers perceived and real.

OMB: we do not see where the follow-up questions are. The only follow-up questions seem to be “get list and then prioritize top 3.” However, this could be done through a web survey, so it is unclear why you are asking these types of questions in a phone interview.

The only other follow-up question asks the respondent to “guess” at what other providers perceive to be barriers. Why are you asking providers to get into other people’s heads? Is that a reliable way to assess what the barriers are?

Since this is the most critical group of respondents—i.e. the non-users—there should be more follow-up questions and questions that are appropriate for an interview (i.e. questions that require the interview format to elicit the needed questions, rather than questions that can be answered through a survey).

The survey has been updated per your request/suggestion.

Initial Questions Initial Responses

OMB Questions Follow Responses