Supporting Statement Part B EHC combined 5-29-09

SUPPORTING STATEMENT

Part B

Eisenberg Center Customer Satisfaction Survey for the

Effective Health Care Program

May 29, 2009

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 3

3. Methods to Maximize Response Rates 4

4. Tests of Procedures 4

5. Statistical Consultants 4

B. Collections of Information Employing Statistical Methods

1. Respondent universe and sampling methods

A significant portion of the proposed information collection is for surveys to collect feedback about Consumer, Clinician, and Policymaker Summary Guides and an Osteoporosis Decision Aid. These are not statistical surveys and are not intended to provide results that can be generalized to all users of the Summary Guides and Decision Aid. It is intended only to provide feedback that can be used in the development and refinement of the Summary Guides and Decision Aids.

Similarly, data collected on the Glossary of Terms will be used only for purposes of determining if the information provided in the Glossary is helpful to users and how the definitions and/or related explanatory information might be expanded, revised, or modified to be of greater value. There will be no complex statistical analyses of the data gathered through the online questionnaire.

Data from the follow-up with participants in CME activities will be analyzed using descriptive methods only. Once the CME offerings have been established as a regular feature of the EHC Web site, more rigorous and structured assessments (e.g., randomized controlled trials involve chart audits or other confirmatory processes) may be undertaken. However, in the early stages of assessment of CME impact, only basic descriptive methods will be employed to analyze data from participants who voluntarily provide follow-up data.

2. Information Collection Procedures

To determine how well the Summary Guides, Decision Aids, and Glossary, meet the needs for information and how useful they are perceived to be, voluntary feedback surveys will be administered using on-line questionnaires as described in the Supporting Statement Part A. These questionnaires will be used to ascertain perceptions of the understandability and usefulness of the Summary Guides, Decision Aids, and Glossary directly from those who use them. An invitation to provide feedback and a link to the on-line questionnaire will appear at the beginning and at the end of each Summary Guide and at the end of each Decision Aid and Glossary Entry. All questions will have close-ended answer choices. However, respondents also will be given an opportunity to describe in their own words the value and adequacy of the products to them. The feedback data can be sorted and tabulated by respondent characteristics, but it will not represent a random survey of users. Nevertheless, the information can be useful in shaping future products to meet the needs of the target audiences more effectively.

As soon as the online feedback surveys are implemented, we will begin monitoring and reviewing responses on a weekly basis. Data on the Summary Guides will be collected for no more than one year, and until the targeted numbers of people have responded. Data gathering concerning the Glossary is expected to continue indefinitely, providing opportunities for Glossary users to continue to inform the process and indicate new terms that should be added.

Data collection for the CME follow-up is managed through direct e-mails to professionals who have requested CME credit. This practice will be maintained with the offerings provided through the EHC Web site. Only individuals who have already provided contact information in conjunction with provision of CME credit are contacted, and their decision to provide follow-up information is totally voluntary.

Findings will be made available to AHRQ managers in monthly, quarterly and annual reports. A final summary report will be submitted to AHRQ within 6 weeks of the end of data collection.

3. Methods to Maximize Response Rates

Invitations to “Tell Us What You Think” will appear at the beginning and end of the web-based Summary Guides and at the ends of Glossary entries. Invitations to provide feedback on what women think of the decision aid will appear at the end of the decision aid. Respondents will be informed that each survey takes about 5 minutes to complete and that their responses are anonymous and cannot be used to identify them. To allow respondents to the summary guide feedback surveys to see their completion status as they move through the questionnaire, a progress bar will be displayed on each page of the survey. The decision aid survey screens will be numbered “Page 1 of 4”, “Page 2 of 4”, etc so that respondents can gauge their progress. This progress information will reduce the number of people who abandon the surveys before they complete them. No progress bar will be used with the questionnaires related to the Glossary, which is quite brief. The accuracy and reliability of the information to be obtained is adequate for the intended uses.

Experience with the CME follow-up survey has indicated that no progress bar is required to encourage use. The professionals who complete the follow-up survey will be familiar with the format and length of the survey, having completed a similar survey immediately after the online CME activity. The follow-up survey, administered approximately 2 months following participation in the CME activity, provides an opportunity to assess topical knowledge retention. It also provides participants in a CME activity the opportunity to indicate if they have implemented any changes in their clinical practice based on knowledge acquired through their participation in the CME activity.

4. Tests of Procedures

Time needed to complete the surveys and the skip sequences was tested by center staff. There will be no outside tests of procedures.

5. Statistical Consultants

There will be no use of statistical consultants.

4