CME CE SupportingStatement_PartB_FINAL

SUPPORTING STATEMENT

Part B

AHRQ Continuing Education for Comparative Effectiveness Research 6-Month Survey

July 2014

Agency for Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 3

3. Methods to Maximize Response Rates 3

4. Tests of Procedures 4

5. Statistical Consultants 4

B. Collections of Information Employing Statistical Methods

1. Respondent universe and sampling methods

The evaluation framework for the Continuing Education for Comparative Effectiveness Research modules will involve three points of evaluation: a pre-module assessment to determine baseline knowledge around the topics covered in the module; a post-module assessment to collect (a) information around the learners’ immediate reactions to the module and its content and (b) a post-test factual knowledge assessment to determine any change from baseline; and a 6-month followup survey that will be administered to learners to complete the module. The survey will be completed by individuals who elect to take the modules rather than identified through systematic statistical sampling. The target population of this study consists of health care professionals, including nurses, nurse practitioners, physicians, physician assistants, pharmacists, health education specialists, case managers, dieticians, psychologists, medical assistants, social workers, and other allied health professionals.

The anticipated number of people who will complete each module is 2,000 individuals. Of that number, 10% are anticipated to complete the 6-month followup survey, making for a total of 200 respondents per module. With 22 modules, this comes out to a total of 4,400 anticipated participants who will complete this survey.

2. Information Collection Procedures

As noted in the response to item 1, the 6-month follow-up survey will be administered to all individuals who complete the learning modules. After completing the post-module assessment immediately after they complete the module, learners will be informed that a web-based survey will be conducted in 6 months’ time as follow-up. At the 6-month mark, they will receive an email asking them to take a survey. The message will also communicate:

• the reason for the information collection,

• the information collection method,

• the burden estimate,

• the nature of the response (voluntary),

• the nature and extent of confidentiality, and

• the OMB control number.

Participants will be reminded up to a total of four times following the initial email to help increase response rate.

3. Methods to Maximize Response Rates

Response and retention rates are of the utmost importance to the AHRQ. Deloitte has extensive understanding of the impact of survey length, frequency of email reminders, accuracy of email contact information, and periodicity of conducting the survey on survey response rates and non-response: upon receiving a sample that contains email addresses, Deloitte creates a unique ID and password for every email address (respondent). An email invitation is then created for each respondent. Inside the invitation is a special one-time use link that only the respondent that received the email can use. Once the link is used, it expires. Once the survey has begun, Deloitte can monitor the progress of the survey. The survey keeps track of who started the survey, who completed the survey, and who did not finish or start at all. Typically, participants are given one week to take the survey before a reminder email is sent. Deloitte will check the survey for anyone who did not start or did not complete the survey and send them a reminder email. The reminder email contains the same link and added verbiage that kindly reminds them to complete the survey. Only people marked as not complete or not started receive a reminder email. Anyone who started but did not complete the survey will start where they left off. Depending on the amount of time for fieldwork, multiple reminder emails can be sent.

Deloitte will leverage this understanding to maximize response rates and reduce non-response in several ways, including keeping the overall length of the CME/CE Learning Module survey to 10 questions for minimally burdensome data gathering. All survey responses will be collected via an online web-based survey interface. Each digitally signed email will include a URL to the web-based survey and a text explanation of the study to help encourage learners to participate.

4. Tests of Procedures

The data collection instrument will be pre-tested by the evaluation team during instrument development activities. We will identify a cohort of physicians and clinicians internal to Deloitte and conduct cognitive testing on the instrument with them. The instrument will be approved by the AHRQ Task Order officer for the project prior to the full launch of the survey. The draft questionnaire will be tested to ensure that respondents can properly understand the questions and that the response options are comprehensive. The testing will validate question wording, response options, and total administration time fielding to the survey’s target populations.

5. Statistical Consultants

Karen Crotty, PhD, MPH

Director

Health Services Research

Hayes Inc.

[email protected]

+1 267 498 7934

Patrick Koeppl, PhD

Deloitte Consulting, LLC.

[email protected]

+1 412 402 5983