Form 1 Attachment A – Approach Document—Future of MONAHRQ® Surv

Proposed Approach

Future of MONAHRQ^® Survey 2014:
Current and Prospective Host Users

Purpose and Scope

Summary of Future of MONAHRQ Survey 2014

• Voluntary customer service feedback mechanism

• 10-15 minutes

• 15 questions

• 14,700 recipients: current and prospective MONAHRQ users

• Maximum of three email contacts: one invitation, two reminders

• No incentives

• Descriptive statistical analysis

MONAHRQ is innovative, free, website-generating software from the Agency for Healthcare Research and Quality (AHRQ). This desktop software enables any organization to create a website quickly and easily with understandable, evidence-based health care performance reports for providers, consumers, and others for use in improving health care quality.

This document suggests a survey approach to gather customer service feedback to improve the MONAHRQ software. The Future of MONAHRQ Survey 2014 will require OMB-approval. The 10-15 minute survey would be voluntary and the results used internally within AHRQ and the MONAHRQ team to determine whether and how future versions of MONAHRQ will meet the needs current and prospective Host Users. Descriptive statistical analysis will be done on the data and help guide the prioritization of features, functionality, resources, and materials for future versions of MONAHRQ.

Audience

Organizations that download MONAHRQ and generate reports to help improve health care are referred to in this Guide as “Host Users.” The Future of MONAHRQ Survey 2014 will be accessible to current and prospective Host Users. Examples of Host Users include:

• State agencies

• Public health departments

• Hospital associations, hospital systems, and individual hospitals

• Multi-stakeholder alliances and coalitions

• Quality Improvement Organizations (QIOs)

• Health plans

To maximize value from using an OMB-approved survey, we will disseminate it to the following groups:

• 14,461+ members of the GovDelivery listserv

• 47 Healthcare Cost and Utilization Project (HCUP) partners

• 16 Aligning Forces for Quality (AF4Q) communities

• 17 recipients of Office of the National Coordinator for Health IT (ONC) Beacon grants

• 17 recipients of AHRQ R21 Building the Science of Public Reporting research grantees

• 31 members of the Network for Regional Healthcare Improvement

• ~50 agency staff from state agencies with responsibility for public reporting.

Some organizations belong to several of these groups, and thus it may not be necessary to disseminate a link to the survey beyond the GovDelivery list; limiting dissemination to GovDelivery may also eliminate some duplication. One drawback to disseminating the survey to a listserv versus individual emails means inability to control the number of times any one person can complete the survey or who has access to the survey. However, this is of minimal concern. Given the size of this audience as well as the level of engagement of the MONAHRQ listserv and partner organizations, incentives will not be provided.

Approach

To gain an understanding of the diverse needs and perspectives of the current and prospective Host User audiences, the Pantheon Team, will develop a 10-15 minute, 14-question OMB-approved web-based survey. Based on prior survey development and discussions with AHRQ, specific topic areas to be addressed by this survey include:

• Current uses of MONAHRQ (including version used, target End User audiences, private versus public reporting)

• Ranking of current MONAHRQ features

• Barriers to using MONAHRQ

• Suggested improvements to MONAHRQ

• Prioritization of future MONAHRQ capabilities

• Future topics for MONAHRQ webinars

In addition, survey respondents will be given the opportunity to provide contact information if they would like to participate in a follow-up interview about their comments.

This secure, web-based survey will contain 15 questions, some of which will be programmed to automatically skip survey recipients for whom the question(s) is not applicable. AIR will use SurveyMonkey to create and manage responses to this survey. AIR recommends sending the link to the online survey out from an AHRQ email address, namely, the MONAHRQ email address, to survey recipients. Next, two reminder emails—one week after the initial invitation and a final reminder two days before the survey closes—may motivate additional responses. The survey will remain open for a minimum of 12 business days.

Reporting

At the conclusion of the response period, AIR will clean, analyze, and compile the collected survey data into a draft report for AHRQ review. We will use descriptive statistics to describe the results of the data. Although we cannot anticipate the response rate for this audience, if more than 100 respondents reply to the survey, more in-depth statistical analysis and comparisons may be possible, for example by organization/stakeholder type. To protect the privacy of participants, results will only be reported in aggregate and names of individual respondents and their organizations will not be associated with results.

Activities and timelines

The timeline for planning activities, survey creation, and response periods are defined below. The timeline assumes a 90-day review period for OMB.

May 2014: Pantheon Team submits approach, text for email invitation, survey, and OMB generic clearance form to AHRQ for review.

June 2014: Pantheon Team finalizes survey materials based on AHRQ review.

July 2014: AIR collects email addresses and creates survey recipient list.

September 2014: OMB approval received (approximate date)

AHRQ/MONAHRQ sends survey invitation and link to survey recipients.

Survey response period opens.

First reminder email sent to survey recipients.

Second reminder email sent to survey recipients.

Survey ends.

October 2014: AIR cleans and analyzes data, prepares draft report for Pantheon and AHRQ review.

November 2014: AIR finalizes survey report based on AHRQ review.

Detailed Draft Timeline

The detailed timeline below is a screenshot from the Dissemination and Outreach Work Plan.

Draft data collection materials

E-mail Invitation Text

GovDelivery listerv and additional participants will be sent an e-mail from AHRQ, soliciting interest and participation in the survey. Sample text can be found below:

Because of your work in health care improvement, you are invited to participate in the Future of MONAHRQ Survey. The purpose of the survey is to gather input on current and potential uses for MONAHRQ. Your feedback will be used to help identify the most useful ways to improve MONAHRQ, such as adding new features.

MONAHRQ® is innovative, free, website-generating software from the Agency for Healthcare Research and Quality. This desktop software enables any organization to create a public or private website quickly and easily. The websites provide understandable, evidence-based health care reports for consumers, providers, policy makers, and others for use in improving transparency and the safety, quality and affordability of health care.

This survey should take no more than 15 minutes to complete. Please plan to complete the entire survey in one session, as you cannot stop the survey and begin again at a later time. The survey is available for you to complete anytime between now and September 5, 2014.

To take the survey, click the link below:

%URL%

Participation in this survey is voluntary. You can decide not to complete the survey or to stop completing the survey at any time. We will only share your responses in aggregate.

The survey is being administered by the American Institutes for Research (AIR), a subcontractor of Pantheon, on behalf of the Agency for Healthcare Research and Quality. If you have any concerns or questions about your rights as a participant in this survey, contact AIR’s Institutional Review Board (which is responsible for the protection of participants) at [email protected] or toll free at 1-800-634-0797. For questions, concerns, or follow-up on the survey contact the MONAHRQ Team at [email protected] or 919-918-4507.

Sincerely,

The MONAHRQ Team

Unsubscribe:<optout link>

E-mail Reminder 1

We want to hear from you! Last week, you should have received an email about the Future of MONAHRQ Survey. From now until September 5, 2014, you have the chance to provide your input to improve MONAHRQ.* If you have already taken the survey, thank you. We received your responses and there is no need to repeat the survey.

To take the survey, click the link below:

%URL%

Participation in this survey is voluntary. You can decide not to complete the survey or to stop completing the survey at any time. We will only share your responses in aggregate.

The survey is being administered by the American Institutes for Research (AIR), a subcontractor of Pantheon, on behalf of the Agency for Healthcare Research and Quality. If you have any concerns or questions about your rights as a participant in this survey, contact AIR’s Institutional Review Board (which is responsible for the protection of participants) at [email protected] or toll free at 1-800-634-0797. For questions, concerns, or follow-up on the survey contact the MONAHRQ Team at [email protected] or 919-918-4507.

Sincerely,

The MONAHRQ Team

Unsubscribe:<optout link>

*MONAHRQ® is innovative, free, website-generating software from the Agency for Healthcare Research and Quality. This desktop software enables any organization to create a public or private website quickly and easily.

E-mail Reminder 2 (final)

If you have already taken the Future of MONAHRQ Survey, thank you. We received your responses and there is no need to repeat the survey. You still have 2 days, until September 5, 2014 left to share your input to improve MONAHRQ.*

To take the survey, click the link below:

%URL%

Participation in this survey is voluntary. You can decide not to complete the survey or to stop completing the survey at any time. We will only share your responses in aggregate.

The survey is being administered by the American Institutes for Research (AIR), a subcontractor of Pantheon, on behalf of the Agency for Healthcare Research and Quality. If you have any concerns or questions about your rights as a participant in this survey, contact AIR’s Institutional Review Board (which is responsible for the protection of participants) at [email protected] or toll free at 1-800-634-0797. For questions, concerns, or follow-up on the survey contact the MONAHRQ Team at [email protected] or 919-918-4507.

Sincerely,

The MONAHRQ Team

Unsubscribe:<optout link>

* MONAHRQ® is innovative, free, website-generating software from the Agency for Healthcare Research and Quality. This desktop software enables any organization to create a public or private website quickly and easily.

Survey Introduction

Thank you for choosing to complete the Future of MONAHRQ Survey. We value your feedback.

The survey will take about 15 minutes to complete. When you are ready to answer the survey questions, scroll to the bottom of the page and click the next button.

As part of this survey, we will ask you to describe your current level of involvement with MONAHRQ and your satisfaction with the functions it offers. We will also give you the opportunity to offer suggestions for potential new features and uses for MONAHRQ.

Participation in this survey is voluntary. You can decide not to complete the survey or to stop completing the survey at any time. We will not use your name or organization in connection with anything you say without permission from you to do so.

The survey is being administered by the American Institutes for Research (AIR), a subcontractor of Pantheon, on behalf of the Agency for Healthcare Research and Quality. If you have any concerns or questions about your rights as a participant in this survey, contact AIR’s Institutional Review Board (which is responsible for the protection of participants) at [email protected] or toll free at 1-800-634-0797. For questions, concerns, or follow-up on the survey contact the MONAHRQ Team at [email protected] or 919-918-4507. To begin the survey questions, please click on the “Next” button below. At the end of the survey, please click “Done” on the final page. Your responses will not be saved if you stop the survey before clicking “Done.”

Form Approved
OMB No. 0935-0179
Exp. Date 7/31/2014

Survey questions

1. How, if at all, have you used the MONAHRQ software? (Please select all that apply)

1. I have downloaded the MONAHRQ software [If yes, go to Q1a]

2. I have used the MONAHRQ software to generate a website [If yes, go to Q1a]

3. I have attended events related to MONAHRQ

4. I have not used the MONAHRQ software [Skip to Question 4]

5. Other (Please specify): ___________________________

2. What is the most recent version of the MONAHRQ software that you have downloaded?

1. MONAHRQ 5.0

2. MONAHRQ 4.1

3. MONAHRQ 4.0

4. MONAHRQ 3.0

5. MONAHRQ 2.0

6. Don’t know/not sure

7. Other (Please specify): ___________________________

3. How are you currently using MONAHRQ? (Please select all that apply.)

1. To build a public reporting site

2. To create a private reporting site

3. To run internal analytics

4. Not applicable

5. Other (Please specify): ___________________________

4. Who is your target audience for a current or future website to share comparative health care reports? (Please select all that apply.)

1. Consumers/Patients

2. Providers (including hospitals, physicians/physician group practices, nursing homes, etc.)

3. Policymakers

4. Researchers/Analysts

5. Purchasers/ Payers/Employers

6. Other (Please specify): ___________________________

Public reporting burden for this collection of information is estimated to average 15 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-0179) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.

5. What data does your organization collect? (Please select all that apply.)

1. Hospital (inpatient) data

2. Clinic (outpatient) data

3. Emergency department data

4. Claims data

5. Clinical data (lab results, tobacco use, etc.)

6. Patient survey data

7. Not applicable

8. Other (Please specify): ___________________________

6. What data is your organization currently reporting either publicly or privately? (Please select all that apply.)

1. Hospital (inpatient) data

2. Clinic (outpatient) data

3. Emergency department data

4. Claims data

5. Clinical data (lab results, tobacco use, etc.)

6. Patient survey data

7. Not applicable/None

8. Other (Please specify): ___________________________

7. Please rank the following features of MONAHRQ in order of importance (1= most important; 6 = least important) to your organization.

1. Evidence-based – uses measures, data sources, and reporting techniques that are based on research

2. Quick – generates websites and reports in hours or days, not months

3. Easy – designed to be intuitive so no training is needed to use it

4. Fully supported with documentation, technical assistance, and software updates

5. Flexible – allows customization to ensure that the websites and reports are relevant

6. Free – provided by the Agency for Healthcare Research and Quality to anyone who wants to produce health care websites and reports

8. What has prevented you from using MONAHRQ? (Please select all that apply)

   1. I was unaware of the availability of MONAHRQ.

   2. My organization is not interested in producing comparative health care reports.

   3. My organization uses a different website service or tool to produce comparative health care reports.

   4. I experienced technical difficulties while using MONAHRQ.

   5. MONAHRQ did not meet my organization’s needs for producing comparative health care reports.

   6. Not applicable

   7. Other (Please specify): ___________________________

9. How could MONAHRQ better meet your needs for producing comparative health care reports?

   1. Expand compatibility of MONAHRQ to work with more hospital related data sources.

   2. Expand compatibility of MONAHRQ with new types of data sources, beyond hospitals.

   3. Provide hands-on consulting during installation and use of MONAHRQ.

   4. Allow for greater customization of websites and reports.

   5. Other (Please specify): ___________________________

10. Please select the five most important capabilities that should be implemented within MONAHRQ in the future.

1. Create a guide to showcase use of evidence-based research design within MONAHRQ

2. Add new data source: nursing home

3. Add new data source: home health

4. Add new data source: physician or ambulatory care quality

5. Add new data source: maternity care

6. Add new data source: pediatrics

7. Add new report: presentation of cost-related data and quality information together

8. Add ability to import custom data

9. Increase customization options for websites

10. Increase customization options for reports

11. Add the ability to show data across multiple years or trending

12. Add new ways to search by providers (by type, individual name, location, etc.)

13. Add the ability for website users to share reports via social media (e.g., Facebook, twitter, LinkedIn)

14. Other (Please specify): ___________________________

11. What topics would you like to see addressed on future MONAHRQ webinars? (Please select all that apply.)

1. Showcase of current and future functions in MONAHRQ

2. Learn about others’ experiences using MONAHRQ

3. Training on advanced features of MONAHRQ

4. Increasing awareness and use of your MONAHRQ-generated public reporting website

5. Other (Please specify): ___________________________

12. Do you track web-analytics or monitor your (public or private) reporting website’s traffic?

1. Yes

2. No

13. Which of the following best describes the type of organization you work for?

1. State agency

2. Public health departments

3. Hospital associations

4. Hospital system or individual hospitals

5. Multi-stakeholder alliance or coalition

6. Quality Improvement Organization (QIO)

7. Health plan

8. Other (Please specify): ___________________________

14. Do you have any other comments or suggestions for us?

___________________________

15. Would you be willing to speak with a representative from the MONAHRQ Team about your comments? If so, please provide your contact information below.

Full Name: ___________________________

Organization: ___________________________

Email Address: ___________________________

Confirm Email: ___________________________

Address: ___________________________

Phone: ___________________________

Public reporting burden for this collection of information is estimated to average 15 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-0179) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.

Updated: 6/16/14 14