December 11 2008 memo

 

1.  Please provide a copy of the report (or a link to it) with the results from the baseline survey.

A recently published report is attached

2.  Please provide the completed sample and response rate from the baseline survey by strata. What combination of original strata are being used in the current survey design?

        Response:  With a starting sample of 2,050 hospitals, 1,652 hospitals completed the baseline survey, for an 81% response rate.  The current survey design uses exactly the same strata that we used in the original survey. 

3.  It is not clear why "refreshing" the sample in one stratum as a fallback based on response rates makes sense. If you want to refresh the sample then the sample design should be modified to reflect that.

        Response:  Our sampling approach is based on establishing a longitudinal sample within stratum, which will consist of a subset of the hospitals within each stratum that responded to both the original survey and this survey.  To obtain this longitudinal sample, we will do a random draw from the original responding sample, within stratum, to reach a total follow-up starting sample of 1,200 that is representative of the population of hospitals within each stratum.  Because we will be surveying hospitals that already completed the original survey, and we had a high response rate for that survey, we expect that this original sample will suffice to achieve a representative completed sample within each stratum.  The response rates for the original survey did not vary much across strata, and few strata had low response rates.  Therefore, we expect to obtain the planned number of completed surveys within each stratum from this original starting sample.

Only if we do not get the intended number of completed surveys in any given stratum will we supplement the original sample.  The first source of supplemented sample will be the remaining hospitals in the relevant stratum(a) that completed the original survey.  Only after that source is exhausted would we turn to drawing new sample hospitals for a stratum that had not completed the original survey.  It is not likely that we will need to take this step to "refresh" with new hospitals, and if we do, it probably will be a small number of hospitals.  We include the refresher step in our sampling plan only as a last resort, in case we encounter a problem with response rates.  We note that any analyses that work with newly sampled hospitals (i.e., not in the sample that completed the first survey) could only be cross-sectional comparisons instead of longitudinal analyses of changes for a cohort of hospitals.

We are taking care to establish a solid sample size for this second AERS survey, anticipating that AHRQ will field this survey again in the future and will want to continue the approach of longitudinal analyses with a cohort of hospitals.  We had the advantage of a large sample from the original survey that is allowing us to stay within that sample for the second survey.  This is not likely to be the case for subsequent surveys, so it may be necessary to refresh each of the subsequent survey samples with new hospitals right from the start, as suggested by OMB. 

4.  Is the survey being sent directly to the person who was the risk manager on the baseline survey or is it addressed to "Risk Manager?"

       Response:  The survey will be sent to the risk manager who completed the baseline survey or current risk manger.  Example:  John Smith or Current Risk Manager

5.  Please provide more information on the results of the multivariate logistic regression model used for nonresponse weighting in the baseline survey.

        Response:  Using our three strata categories of hospital characteristics (bed size categories, ownership categories, and accreditation status, we established a set of strata that were the full combination of the levels established for these three categories (i.e., a three-dimensional crossing of the category levels).  For each stratum, we calculated the non-response weight as the number of hospitals selected for the sample divided by the number that responded, which is the reciprocal of the response rate.  This is the equivalent to estimating a logistic regression with the set of strata variables as predictor variables and response/nonresponse as the dependent variable.

6.  Please explain how this study relates to other adverse event/medical error/patient safety initiatives AHRQ is conducting (e.g. MEADERS)

Response.  The MEADERS project involved a test of a reporting system for medication events in ambulatory care setting.  The National Hospital Adverse Event Reporting Survey is for the hospital setting. The MEADERS project is now completed and there are no immediate plans for its deployment beyond the initial test of the system which is now complete.  The adverse event survey is for hospitals and does not cover ambulatory care facilities.  The other patient safety initiate that does relate to the Hospital Adverse Event Reporting Survey is the Hospital Survey on Patent Safety Culture (HSOPS).  These two surveys were originally design as companion surveys one looking at event report from a hospital perspective and the culture survey looking at staff perception of culture including items about event reporting.  We have examined the data from the Adverse Event Survey and HSOPS from the Comparative Benchmarking Database for year 2007, 2008, and soon to be released 2009 databases and will track staff responses to items related to event reporting with the results of the National Survey on Adverse.  The relationship between HSOPS and the adverse event survey data will be extremely useful to the Agency and to those interested in patient safety.

7.  How will this follow-up study relate to the PSO rule and data collected as part of that effort? Has AHRQ given thought to including questions in this package that would help guide the development of standardized reporting elements for purposes of PSOs, which could increase their utility? Has AHRQ considered conducting this study after the PSO rule has been finalized so that questions could be included in the questionnaire to assess progress with implementing PSOs?

Response:  The follow-up study is specifically designed to obtain a second baseline of the current state of hospital event reporting prior to the implementation of the PSO rule, which will become effective on January 21, 2009.  Since PSOs are just beginning to be established, there will be limited impact on hospital reporting practices in the immediate future.  A separate process has been used to develop Common Formats including standardized reporting elements for reporting to PSOs by a federal agency workgroup.  AHRQ’s common formats are being reviewed for comment by the National Quality Forum (NQF) to assist the Agency with revisions and updates.  This process is separate from this survey.  The data generated by National Hospital Adverse Event Reporting Survey will be useful in allowing the future assessment of the impact of PSOs. Having a trend of two points in time will generate a more complete picture of hospital event reporting practices prior to PSOs becoming fully operational nationwide.  A third administration of the National Hospital Adverse Event Reporting Survey is anticipated in order to fully assess the impact of PSOs on hospital adverse event reporting practices.