Supporting Statement Part B Grants Evaluation 6-24-2008

SUPPORTING STATEMENT

Part B

Evaluation of the Effectiveness of the Agency for Healthcare Research and Quality’s (AHRQ) Grant-Supported Research on Healthcare Costs, Productivity, and Market Forces

March 17, 2008

Revised June 24, 2008

Agency of Healthcare Research and Quality (AHRQ)

Table of Contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent Universe and Sampling Methods 3

2. Information Collection Procedures 3

3. Methods to Maximize Response Rates 4

4. Tests of Procedures 4

5. Statistical Consultants 4

B. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

Since the universe of relevant grants is not sufficiently large to allow for sampling, the entire universe of grantees will be covered by the data collection and AHRQ has contact information for each grantee. This data collection has not been conducted previously; however the estimated response rate is at least 80 percent. Given the saliency of the collection to the grantees, we are optimistic that that estimate will be exceeded. The subject matter and purpose of the survey is highly salient to the universe of grantees. After conducting preliminary qualitative in-depth interviews with nine of the grantees, MPR found that these grantees felt the subject matter was important and were willing to participate and provide information about how they have developed and disseminated the findings from their research. For these reasons, achieving a high response rate to this survey is expected.

2. Information Collection Procedures

This is a one-time data collection. Data collection will be done using a self-administered web-based survey. The questionnaire for this survey was developed by MPR, AHRQ’s contractor, working together with AHRQ and receiving input from a technical expert panel on questions of interest to this evaluation.

The web-based survey will be implemented by MPR. It will allow respondents to log in to their survey via the Internet, using a password protected, secure website hosted at MPR, and complete the survey at their own pace and schedule. Since some questions ask for information about where their research results have been disseminated, respondents may need time to search for this information. The web survey will allow the respondent to return and complete the survey at a later time if they prefer. MPR will oversee the administration of the survey and respond to requests for information. The data collection instrument is included as Attachment A. Follow-up interviews will be conducted with a small number of interviewees. Because of the open ended nature of these interviews, no schedule is attached, however, the burden estimate included the expected time of the followup interview. These follow-up interviews will focus on how research has been used.

An advance letter from AHRQ on its letterhead will be mailed to all 149 grantees explaining the purpose of the survey, requesting their participation, and describing confidentiality. The advance letter underscores that AHRQ wants to improve its processes and to learn how findings from its supported research are disseminated. The letter also notifies the grantees that they will receive an email from MPR with a web-link to the survey. The advance letter and the email are included as Attachments B and C, respectively.

3. Methods to Maximize Response Rates

To maximize responses by the grantees, MPR will administer the questionnaire as a self-administered web-based survey. Using this mode will allow respondents to complete the survey at their own pace and schedule. MPR will also make a hardcopy of the survey available if grantees prefer to complete the survey on paper. MPR will also telephone grantees that have not completed their survey after two weeks to remind sample members about the survey and to check if there are other accommodations that MPR can make to facilitate completion of the survey.

4. Test of Survey Procedures

The questionnaire was pretested with fewer than 10 respondents to learn about problems respondents might experience in providing the requested information and to make appropriate changes to the questionnaire. Pretest responses and comments to the questionnaire were collected by mail to emulate the self-administration that will be used for the survey. MPR staff followed up with pretest respondents by telephone to learn their reactions and determine how to improve language. The pretest also established the average interview length, currently estimated at 120 minutes per survey. The results of the pretest were used to revise the questionnaire.

5. Statistical Consultants

Since this data collection will not involve a statistical design, no statisticians were consulted on the design of this project.

References

Krissik, Tara, Tim Lake, and Marsha Gold. “Evaluation of the Effectiveness of AHRQ’s Grant-Supported Research on Health Care Costs, Productivity, and Market Forces: Interim Report.” Washington, DC: Mathematica Policy Research, Inc., November 2007.

5