Form #1 Form #1 Value Grant Questionnaire

SURVEY OF VALUE GRANTEES

Form Approved
OMB No. 0935-XXXX
Exp. Date XX/XX/20XX

Public reporting burden for this collection of information is estimated to average 30 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-XXXX) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.

The purpose of this survey is to gain a better understanding of your experience as an R01 grantee for the AHRQ-sponsored Funding Opportunity Announcement, Demonstrating the Value of Health Information Technology RFA (HS-04-012). The primary purpose of this RFA was to fund projects that would increase the knowledge and understanding of the value of health information technology (IT) to improve patient safety and quality of care.

The RFA specified four major areas of value:

• Clinical, including medical errors, effectiveness, and CDS systems.

• Organizational, including access to health care and coordination of care.

• Financial, including costs and productivity.

• Other, including patient satisfaction, transparency, readiness for health IT adoption, etc., and the five long-term goals of the THQIT initiative listed in the Introduction.

This survey is about your participation in and results of the value grant described in the final report located here: [Click for Final Report].

The survey contains eight brief sections, as follows:

Section I: Project Partnership

Section II: Project Findings

Section III: Lessons Learned and Dissemination of Findings

Section IV: Adoption or Changes in Use of Health IT among Participating Organizations

Section V: Focus On AHRQ Priority Populations

Section VI: Continuation of Health IT Work/Partnerships

Section VII: Sustainability/Expansion of Health IT

Section VIII: Other Benefits

Public reporting burden for this collection of information is estimated to average 30 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-XXXX) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.

Public reporting burden for this collection of information is estimated to average 30 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-XXXX) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.

SECTION I: PROJECT PARTNERSHIP

1. What was your role in the Transforming Healthcare Quality through Information Technology (THQIT) value grant?

Check all that apply.

¹ Principal Investigator

² Project Director or Project Coordinator

³ Other individual directly involved with the value grant

⁴ An administrator at a participating organization during the grant period

⁵ A provider at a participating organization during the grant period

⁶ I did not work at a participating organization during the grant period

⁷ Other (please specify): _____________________

2. Our records indicate that you partnered with (collaborated with) the following organizations for this study:

[org1]

[org2]

[org3]

[org4]

[orgX]

3. Please indicate any corrections to the partnered organizations here:

4. Below is a list of types of organizations that focus on the delivery of patient care.

In Column A, please indicate the types of partner organizations that were actively involved throughout the course of this project.

In Column B, for partner organization types that were involved in the THQIT value grant please indicate the total number of unique organizations involved.

If involved organizations have multiple sites, please count the sites that were part of the health IT project that was being studied. For example, suppose a physician group with 9 small-to-medium practice sites partnered with a hospital system with 2 hospitals and a university. If the value study was focused on health IT that would affect both hospitals and all 9 sites, count them all below. If the value study was focused on a subset of the practice sites and/or one of the two hospitals, count the ones on which it was focused.

Type of Organizations	Column A Partner for THQIT Value Grant	Column B Number of Organizations Involved
Organizations Focused on Delivering Patient Care
Critical access hospitals	1
Other rural hospitals with <100 beds	1
Other non-rural hospitals with <100 beds	1
Rural hospitals with 100 or more beds	1
Non-rural hospitals with 100 or more beds	1
Private physician practices with <5 physicians	1
Private physician practices with 5-24 physicians	1
Private physician practices with 25 or more physicians	1
Long-term care organizations	1
Home health care organizations	1
Pharmacies	1
Emergency medical service agencies	1
Mental Health Centers or Behavioral Health Facilities	1
Federally Qualified Health Centers	1
Other safety net clinics	1
Other point-of-care organizations* (please indicate below)
_______________________________________	1
__________________________________________	1

*By point-of-care organizations, we mean organizations that focus primarily on patient care delivery.

5. Below is a list of types of organizations that DO NOT deliver patient care or that the delivery of patient care is NOT their main focus.

In Column A, please indicate the types of partner organizations that were actively involved throughout the course of this project.

In Column B, for partner organization types that were involved in the THQIT value grant please indicate the total number of unique organizations involved.

Type of Organizations	Column A Partner for THQIT Value Grant	Column B Number Of Organizations Involved
Universities (units that do not provide patient care)	1
Other research-focused organizations	1
Social service agencies	1
Schools	1
Health departments	1
Health IT consulting firms	1
Other consulting firms	1
Health IT vendors	1
Professional associations	1
Other organizations that do not deliver patient care (please indicate)
_______________________________________	1
__________________________________________	1

Programmer note: If Q5h=1, go to Q6. Else, Q7.

6. You indicated that other consulting firms were partners on the THQIT value grant. Please specify the type of consulting firms in the space provided.

7. After the value grant ended, to what extent did partner organizations work together to pursue a new health care improvement or research activity (which may or may not have focused on health IT to improve quality)?

¹ All organizations continued to work together

² Some organizations continued to work together

⁰ No organizations continued to work together

^-1 I don’t know if any organizations continued to work together

Programmer Note: If Q above=0 or -1, go to Q10.

8. Thinking about the new activity that partner organizations pursued, did it include an evaluation component?

¹ Yes

⁰ No Go to programmer note before Q10.

9. Thinking about evaluation component, did it include internal researchers, external researchers, or both?

¹ Internal researchers

² External researchers

³ Both internal and external researchers

Programmer note: If Q7=2 or 0, then 10. Else go to Q12.

10. What type(s) of partner organizations stopped working together after the grant project?

Check all that apply.

¹ Patient care delivery organization(s)

² Research organization(s)

⁰ Other type

Programmer note: If Q10 includes 1, go to Q11. Else go to Q12.

11. Please indicate why some or all point-of-care partner organizations did not continue to work together after the value grant ended.

Check all reasons that apply to one or more of the partner organizations.

¹ Financial constraints of a partner organization

² Lack of senior leadership endorsement (support) for health IT at a partner organization

³ Lack of clinicians’ endorsement (support) at a partner organization

⁴ Differences in readiness to use health IT among partner organizations

⁵ Competition for patients and/or revenue among partner organizations

⁶ Divergence in the types of health IT, health IT vendor, or functions used across partners

⁷ Disagreements on the implementation plan

⁸ Issues with data sharing or data use agreements

⁹ Other (please specify): _____________________

12. The February 2009 passage of the American Recovery and Reinvestment Act (ARRA)/ Health Information Technology for Economic and Clinical Health (HITECH) may have changed the ways organizations work together. Did the extent to which you and your partners worked together change because of ARRA/HITECH? 

HITECH is Title XIII of ARRA, also known as “The Stimulus Law”(ARRA Public Law 111-5). Among other things, it contains incentives designed to accelerate the adoption of electronic health records, beginning in 2011.  

¹ We collaborated more after ARRA/HITECH

² We collaborated less after ARRA/HITECH

³ We collaborated at about the same level before and after ARRA/HITECH

^-1 Don’t know

SECTION II: PROJECT FINDINGS

13. The following is a list of outcomes that you may have measured to examine the results of the grant project.

In Column A, please indicate if the outcome was measured by the end of the project.

In Column B, please indicate if the outcome was measured after the end of the project.

In Column C, for the most recent measurement of each outcome, please indicate if it was improved, unchanged, or worsened.

Outcomes	Column A Was this outcome measured by the end of the project?		Column B Was this outcome measured after the end of the project?			Column C Most Recently Measured Outcome was….
	Yes	No	Yes	No	Don’t Know	Improved	Unchanged	Worsened	Don’t Know
Medical errors	1	0	1	0	-1	1	2	3	-1
Other patient safety outcomes	1	0	1	0	-1	1	2	3	-1
Clinical outcomes for patients	1	0	1	0	-1	1	2	3	-1
Population level outcomes	1	0	1	0	-1	1	2	3	-1
Timeliness of care	1	0	1	0	-1	1	2	3	-1
Access to care	1	0	1	0	-1	1	2	3	-1
Patient satisfaction with care	1	0	1	0	-1	1	2	3	-1
Costs or other economic outcomes	1	0	1	0	-1	1	2	3	-1
Efficiency of health care processes	1	0	1	0	-1	1	2	3	-1
Quality of health care processes	1	0	1	0	-1	1	2	3	-1
Other (please specify below)
______________	1	0	1	0	-1	1	2	3	-1
______________	1	0	1	0	-1	1	2	3	-1

14. Are you confident in your knowledge of whether your organization’s value grant partners measured project outcomes after the grant period?

¹ Yes, I am confident in my knowledge.

² No, I am not confident in my knowledge

³ I am not knowledgeable about whether they measured outcomes or not

15. Are there plans to further measure the outcomes of the project?

¹ Yes

⁰ No Go to Question 17.

16. Which outcomes will be further measured?

Check all that apply

¹ Medical errors

² Other patient safety outcomes

³ Clinical outcomes for patients

⁴ Population level outcomes

⁵ Timeliness of care

⁶ Access to care

⁷ Patient satisfaction with care

⁸ Costs or other economic outcomes

⁹ Efficiency of health care processes

¹⁰ Quality of health care processes

¹¹ Other (please specify):

17. Where you able to execute the analytical plan specified in your grant proposal?

¹ Implemented in total Go to Question 19.

² Implemented in part Go to Question 19.

⁰ Unable to implement

18. Which of the following factors impeded you from executing the analytical plan specified in your grant proposal?

Select all that apply.

¹ Implementation took longer than expected and time ran out for conducting the analyses

² Implementation took longer than expected and there were insufficient funds to provide support of an adequately trained evaluation expert (e.g., biostatistician) to conduct the research.

³ Lack of quality, uniformity, or ability to integrate appropriate data elements

⁴ Other (please specify): ______________________

Programmer note: Go to Question 24.

19. Which analytical method(s) did you employ to evaluate your project?

¹ Quantitative analyses only

² Qualitative analyses only Go to Question 22.

³ Combination of quantitative and qualitative analyses

20. In your quantitative analytical plan and subsequent analysis, did you consider sample size, power, and minimum detectable difference?

¹ Yes

• Go to Question 22. 10.

⁰ No

^-1 Don’t know

21. Were your sample size, power, and minimum detectable difference sufficient to reasonably detect the effects of interest with 80 percent power?

¹ Yes

⁰ No

^-1 Don’t know

22. Was there sufficient time between implementation of the health IT and observation to reasonably observe effects of interest?

¹ Yes

⁰ No

^-1 Don’t know

23. Could elements of the pre-implementation (planning) stage and/or the implementation stage of this health IT study have been strengthened and thereby contributed to better outcomes of the study?

¹ Yes

⁰ No

^-1 Don’t know

24. The following is a list of planning and implementation issues that could impact the success of the project and subsequent improvement in outcomes.

In the table below, in Column A, please indicate if each of the following was a strength or weakness or had no impact with regard to your project outcomes.

In Column B, for each aspect that was a weakness, please indicate the impact of the weakness.

While some items may have had both pros and cons for the project, please decide on balance whether the item was a strength or weakness (choose one response only from column A).

Implementation / Planning Aspect	Column A			Column B Impact of Weakness
	Strength	Weakness	No Impact
Continuity of leadership	1	2	0	Drop down Prevented implementation or forced discontinuation Limited project benefits Did not limit project benefits
Continuity of project staff (do not include AHRQ staff)	1	2	0
Level of trust among partners	1	2	0
Level of participation across partners	1	2	0
Physician champions	1	2	0
Enthusiasm for the project among physicians	1	2	0
Administrative leadership support	1	2	0
Thorough selection process for the health IT	1	2	0
Planned budget for health IT	1	2	0
Plans for data privacy and security	1	2	0
Fit of health IT with workflows	1	2	0
Reliability of the software	1	2	0
Adequacy of hardware and system capacity to run software optimally	1	2	0
Fit of new health IT with pre-existing system	1	2	0
Availability of knowledgeable health IT staff	1	2	0
Health IT vendor support	1	2	0
Usability of the health IT	1	2	0
Usefulness of the health IT to patient care	1	2	0
Business case for the health IT and availability of funds	1	2	0
Cost of ongoing maintenance	1	2	0
Interoperability with other providers’ health IT systems	1	2	0
Tradition of teamwork prior to implementation	1	2	0
Degree of clinicians’ or staff’s comfort with computers prior to implementation	1	2	0
Geographic distance between partners	1	2	0
Sufficiency of personnel with needed knowledge, skills, abilities	1	2	0
Infrastructure foundation for health IT	1	2	0
Other strengths or weaknesses (please specify below)
	1	2	0
	1	2	0

25. Were concerns raised during project implementation about information security or patient confidentiality?

Check all that apply.

¹ Yes – information security

² Yes – patient confidentiality

⁰ No Go to Question 27.

26. Did these concerns prevent implementation, force discontinuation, or limit project benefits?

¹ Prevented implementation

² Forced discontinuation

³ Limited project benefits

⁴ Did not limit project benefits

SECTION III: LESSONS LEARNED AND DISSEMINATION OF FINDINGS

27. Which two sources best identify where you disseminated your lessons learned about planning and implementation?

Please choose up to two.

¹ Peer-reviewed manuscript ⁴ Final report

² In-house documents ⁵ Work under review

³ Other grey literature

28. Please indicate how the source that best illustrates lessons that were informed by this project can be accessed (provide web link, if available)?

Click here if you do not know: (Go to Question 28a)

Programmer note: If Q28 is answered, go to Q29.

28a. Who would be the key contact to identify how key sources can be accessed?

Name:

Title:

Organization:

Phone: __ __ __ - __ __ __ - __ __ __ __

Email: @ .

29. Is there a key contact who understands lessons learned with greater insight than what is provided in available documents?

¹ Yes

⁰ No Go to Question 32.

30. Are you the key contact who could articulate the “lessons learned”?

Go to Question 32.

¹ Yes

⁰ No

31. Who would be the key contact for lessons learned?

Name:

Title:

Organization:

Phone: __ __ __ - __ __ __ - __ __ __ __

Email: @ .

SECTION IV: ADOPTION OR CHANGES IN USE OF HEALTH IT AMONG PARTICIPATING ORGANIZATIONS

32. In Column A, for each of the following types of partner organizations, please indicate if new health IT was purchased as a result of participating in the value study.

In Column B, for partner organizations where new health IT was purchased, please indicate the total number of organizations where this occurred.

Type of Organizations	Column A Purchase occurred (by any)	Column B Number of Organizations where Purchase Occurred
Organizations Focused on Delivering Patient Care
Critical access hospitals	1
Other rural hospitals with <100 beds	1
Other non-rural hospitals with <100 beds	1
Rural hospitals with 100 or more beds	1
Non-rural hospitals with 100 or more beds	1
Private physician practices with <5 physicians	1
Private physician practices with 5-24 physicians	1
Private physician practices with 25 or more physicians	1
Long-term care organizations	1
Home health care organizations	1
Emergency medical service agencies	1
Mental Health Centers or Behavioral Health Facilities	1
Federally Qualified Health Centers	1
Other safety net clinics	1
Other (please specify below)
_______________________________________	1
__________________________________________	1

33. In Column A, for each of the following types of partner organizations, please indicate if increased use of existing health IT was a result of participating in the value study.

In Column B, for partner organizations where there was increased use of health IT, please indicate the total number of organizations where this occurred.

Type of Organizations	Column A Increased Use (by any)	Column B Number of Organizations with Increased Use of Health IT
Organizations Focused on Delivering Patient Care
Critical access hospitals	1
Other rural hospitals with <100 beds	1
Other non-rural hospitals with <100 beds	1
Rural hospitals with 100 or more beds	1
Non-rural hospitals with 100 or more beds	1
Private physician practices with <5 physicians	1
Private physician practices with 5-24 physicians	1
Private physician practices with 25 or more physicians	1
Long-term care organizations	1
Home health care organizations	1
Emergency medical service agencies	1
Mental Health Centers or Behavioral Health Facilities	1
Federally Qualified Health Centers	1
Other safety net clinics	1
Other (please specify below)
_______________________________________	1
__________________________________________	1

SECTION V: FOCUS ON AHRQ PRIORITY POPULATIONS

34. To what extent were AHRQ priority populations involved in the health IT that was studied?

AHRQ priority populations are as follows: low-income populations, minority populations, women, children, the elderly, individuals with special health care needs, individuals needing end-of-life care, rural populations, and inner city populations.

¹ AHRQ priority populations were included in the target population proportionate to their distribution in the general population, but this research project did NOT focus on an AHRQ priority population

² AHRQ priority populations were included in the target population proportionate to their distribution in the general population, and at least one of the AHRQ priority populations was a focus of the study

³ At least one of the AHRQ priority populations was a focus for this project

⁴ The extent to which AHRQ priority populations were involved is unknown

PROGRAMMER: IF Q 34=2 OR 3, GO TO Q35. ELSE, GO TO Q37.

35. Which priority populations were targeted with your health IT value grant?

Check all that apply.

¹ Low income populations

² Minority populations

³ Women

⁴ Children

⁵ The elderly

⁶ Individuals with special health care needs

⁷ Individuals needing end-of-life care

⁸ Rural populations

⁹ Inner city populations

36. What if any impact do you believe the health IT you studied had on the quality of care (including access) of the following AHRQ priority populations served by the partner organizations?

	Positive Impact	Likely No Impact	Negative Impact	Not a focus of project
Low income populations	1	2	3	-1
Minority populations	1	2	3	-1
Women	1	2	3	-1
Children	1	2	3	-1
The elderly	1	2	3	-1
Individuals with special health care needs	1	2	3	-1
Individuals needing end-of-life care	1	2	3	-1
Rural populations	1	2	3	-1
Inner city populations	1	2	3	-1

SECTION VI: CONTINUATION OF HEALTH IT WORK/PARTNERSHIPS

37. Did the value study lead to the study team and participating organizations/sites pursuing funding opportunities for additional research or health IT projects (include any proposed projects that were not funded)?

Go to Question 40.

¹ Yes

⁰ No

38. How many additional research or health IT projects were pursued?

[dropdown from 1 to 5 or more]

39. Thinking about the first health IT project pursued after the THQIT grant, please list the health IT project name and funding source(s) (include any proposed projects that were not funded). If the project was funded, please include the name of the PI as well as the grant/contract number, if known.

Include research projects or other health IT projects. For grant proposals sponsored by Department of Health and Human Services, please indicate Principal Investigator’s name and grant/contract number.

Research of Health IT Project Name:

Funding source pursued:

Check all that apply.

¹ Internal funding

² AHRQ funding

³ Funding from the Office of the National Coordinator for Health Information Technology

⁴ Funding from the Health Resources and Services Administration (HRSA)

⁵ Other federal funding (please specify):

⁶ State or local public funding

⁷ Private foundation(s)

⁸ Other funding (please specify): ______________________________

Was this project funded?

¹ Yes

⁰ No

^-1 Don’t know

Principal Investigator’s name:

Grant/contract number:

Programmer note: Loop through above based on reply to Q38. Max loop=5.

SECTION VI: SUSTAINABILITY/EXPANSION OF HEALTH IT

40. To what extent is the health IT implemented during the project (or its successor product) being used currently?

Check all that apply.

¹ More health IT functions have been added

² More organizations are using the health IT

³ More clinicians or staff within the original partner organizations are using the health IT

⁴ Use during grant period has been sustained (neither increased nor decreased)

⁵ Decreased use

⁶ Not currently used

Programmer note:

If Q40 includes 1, 2, 3, or 4, go to Q42.

If Q40 includes 6, go to Q41 and then Q43 (skip Q42).

Else, go to Q43.

41. The following are some reasons that organizations may not use health IT. Please indicate whether any of these are reasons why the organizations you studied are not currently using the health IT intervention studied in the project.

Check all that apply.

¹ High cost of health IT product

² Concerns about the legality of donating a system to an associated physician practice

³ Disruption in clinical care during implementation

⁴ Concern about lack of future health IT vendor support

⁵ Other (please specify):__________________

42. Please indicate the reasons for sustained use of the types of health IT technologies studied in the project?

Check all that apply.

¹ No reason to terminate

² Adequate clinician support

³ Resulted in demonstrated benefits

⁴ Resources to support health IT are provided by project partner organizations

⁵ Resources to support health IT are provided from new partner organizations

⁶ Subsequent grant or contract funding received

⁷ Revenue from usage fee for use of health IT

⁸ Productive relationship with health IT vendor

⁹ Other (please specify): ___________________

43. Did the findings from this project lead to or influence implementation or expansion of health IT to organizations not involved in the study?

¹ Yes

Go to Question 45.

⁰ No

^-1 Don’t know

44. To what extent did the findings from this project lead to or influence implementation or expansion of health IT to organizations not involved in the study?

In Column A, for each of the following type of organizations, please indicate if implementation or expansion of health IT occurred as a likely result of findings from the value study.

In Column B, for organizations where new implementation or expansion of health IT occurred as a likely result of the findings of the value study, please indicate the total number of organizations where this occurred.

Type of Organizations	Column A Increased Use (any)			Column B Number of Organizations Increasing Use Likely Due to Study Findings
	Yes	No	Don’t Know
Organizations Focused on Delivering Patient Care
Critical access hospitals	1	0	-1
Other rural hospitals with <100 beds	1	0	-1
Other non-rural hospitals with <100 beds	1	0	-1
Rural hospitals with 100 or more beds	1	0	-1
Non-rural hospitals with 100 or more beds	1	0	-1
Private physician practices with <5 physicians	1	0	-1
Private physician practices with 5-24 physicians	1	0	-1
Private physician practices with 25 or more physicians	1	0	-1
Long-term care organizations	1	0	-1
Home health care organizations	1	0	-1
Emergency medical service agencies	1	0	-1
Mental Health Centers or Behavioral Health Facilities	1	0	-1
Federally Qualified Health Centers	1	0	-1
Other safety net clinics	1	0	-1
Other (please specify below )
_______________________________________	1		-1
__________________________________________	1		-1

SECTION VII: OTHER BENEFITS

45. Did the value study have any benefits for the organizations or clinicians or patients that participated, beyond those described in your final study report and other publications?

¹ Yes

Go to Question 47.

⁰ No

^-1 Don’t know

46. Please describe those benefits in the space provided.

47. The Value Grant Funding Opportunity Announcement supported up to $500,000 per year for a project period up to three years and a maximum of $1,500,000. If AHRQ were to fund similar value projects in the future, based on your experience what funding amount and study period would you recommend?

Funding Amount Recommended:

¹ $500,000

² $1,000,000

³ $1,500,000

⁴ $2,000,000

⁵ Other (please indicate):

Recommended Project Period:

¹ 12 months

² 24 months

³ 36 months

⁴ 48 months

⁵ Other (please indicate):

48. In general, how would you rate the importance of health care organizations and researchers being able to pursue a planning grant (e.g., one-year, $200,000) as a means to prepare them for future large-scale research on health IT or healthcare improvements topics?   

Not very important								Very important
1	2	3	4	5	6	7	8	9	10

49. In   your opinion, how likely is it that a planning grant project involving these partners and research topics would have had a significant positive impact on the success of this project to achieve its desired aims in the initial project period (up to three years) provided?

¹ Very likely

² Somewhat likely

³ Somewhat unlikely

⁴ Very unlikely

⁵ Don’t know

50. If you would like to add comments to AHRQ about the THQIT program, please do so in the space provided.

THANK YOU FOR COMPLETING

THE SURVEY OF VALUE GRANTEES!

F.39