0471 ss Part B FINALREWRITE 3-24-11

ADVERSE EVENT PROGRAM FOR MEDICAL DEVICES

(Medical Product Safety Network (MEDSUN))

0910-0471

SUPPORTING STATEMENT

2. Statistical Methods:
   
   

While the MedSun sample is not based on statistical methodology, it is a loosely representative sample of sizes and types of hospitals in the country. The following is a description for how the sample is drawn and maintained.

1. Respondent Universe and Sampling Methods

The sample size is 350 sites, with the potential respondent universe including 6,000 hospitals, 17,000 nursing homes, and approximately 20,000 “other” types of health facilities (outpatient treatment centers, outpatient diagnostic centers, emergency health services, and home health services). For purposes of continuing this as a program, FDA believes that 350 sites (of greater than 100 beds) is the number we can continue to accommodate in the program. This number may even need to be decreased if the budget is cut for this program. The number in the sample is based on the fact that we have achieved a roughly representative sample of small, medium, and large hospitals across the country, and that we are currently receiving more reports than the staff can accommodate. Constant training and encouragement goes on to continue the hospitals’ reporting into the program; it is very labor intensive. Hospitals in the program have the expectation that each report will receive individual attention, which the staff currently struggles to achieve due to the number of reports coming in from this successful program.

Therefore, the program cannot grow passed the current number due to cost and time restraints. Ultimately, while we strive to keep the sample roughly representative as to size of facilities spread throughout the 5 census regions of the country, the final sample may best be described as a convenience sample, because in addition to our loosely representative sample, we also include sites which ask to be part of the program.

While most sites are good reporters, we find that each year we must cycle out, and replace, about 50 sites which have not reported well during the previous year. Thus, continued recruitment and orientation goes on; it is not a static sample. To replace the sites we remove from the program, we use the American Hospital Sample Frame (AHA) to select a hospital similar to the one we are rolling out and that is in the same census region.

Since denominator data is unknown for the number and types of incidents that occur with each of the medical devices in the United States, FDA will not use the data collected to make national estimates of devices problems, use problems, etc. In considering the selection of new facilities to participate in the pilot (to replace the 50 that are rolled out each year), it is important to understand that the main purpose of the network is not to make precise statistical estimates of frequency of occurrence of adverse medical device events. The purpose of this postmarket surveillance is to obtain signals that problems are occurring and to learn as much as possible about the incidents.

Therefore, in identifying replacement facilities in the same census region and the same size as the site being removed, we also take into consideration how willing a potential new recruit may be to work closely with FDA to continue the program. The greatest participation, and thus the best signals, is obtained from facilities which have organizational structures and cultures that are willing to share information with the outside world in the interest of increased patient safety. Additionally, most of the current larger hospitals have other types of adjunct facilities as part of the organization – such as long-term care facilities (similar to nursing homes), outpatient treatment centers, outpatient diagnostic centers, and home health agencies. The larger institutions are more apt to have experience with diverse medical devices and, given the volume of patients that is admitted each year, are also more likely to have adverse events occur with those medical devices.

Enrollment into the program is voluntary.

2. Procedures for the Collection of Information

After selecting potential candidates based on the criteria described, above, replacement facilities are invited to participate in the program. If they agree to participate, they sign a Memorandum of Participation which describes the reporting procedure they will be expected to complete (send adverse event reports via the web – both mandatory and voluntary reports), participate in training in how to participate prior to submitting adverse event reports, and protect their passwords. The facilities will report incidents of adverse events via the Internet-based reporting system to FDA. The facility will answer the questions that currently appear on the 3500A form as well as some additional questions (cleared by OMB). FDA analysts will review the reports for completeness and will follow up with the facility as needed to ensure the report is a complete as possible and to more fully understand why the event may have occurred.

3. Methods to Maximize Response Rates and Deal with Non-Response

Every effort will be made to maximize responses. However, in postmarket surveillance that is based on numerator, rather than denominator, data, obtaining important signals that problems may be occurring is more critical than obtaining large numbers of reports.

Facilities will continue to use the Internet-based system to report adverse events. FDA's MedSun postmarket medical device adverse event reporting is considered ‘enhanced’ surveillance, rather than the ‘passive’ surveillance of our other reporting systems. Since reporting first requires individuals working within a facility to recognize that an adverse event was related to the use of a medical device and then to report that incident through the appropriate channels within that facility, FDA supplies many tools to the hospitals to train staff. However, while FDA will train each facility in the mechanisms of reporting, facilities will still only forward reports if the ‘lead’ reporter with the MedSun system password learns about a medical device adverse event from staff, or learns of the potential for an adverse event to occur. It is impossible to predict how many adverse events will occur per year at each facility during the pilot. Our current average is 8 reports per year per site, but we are striving for an average of 15 reports per year.

If a facility sends less than 2 reports in the first 6-months of entering the pilot, the FDA analyst calls the facility contact person to ask how the system seems to be working, if the facility needs more training in the mechanics of working in the pilot project, etc. This gentle probing is to remind the facility contact person that the success of the project depends on each facility forwarding all medical device related adverse events. But, if the site continues to be a poor performing, it is cycled out of the program and replaced.

Facilities will voluntarily fill in answers to data fields not currently included in the 3500A form (OMB approved questions). To encourage the facilities to fill in these data elements, the FDA MedSun staff calls the facility’s contact person to ascertain the reason the items were not filled in. The responses given by the contact person will aid FDA in determining if the questions are too difficult for the facility to find in a timely manner, or if the facility is not comfortable, for whatever reason, in providing the answers. This information can then be reflected in changes brought forth in the next 3-year OMB Justification cycle.

4. Test of Procedures or Methods to be Undertaken

The need for user facilities to report adverse events associated with medical devices is currently mandated by law, so the fundamental mechanics of how to work with a reporting program is well-known to facilities. Before MedSun, all reporting had been a paper system, so this pilot was the first test of a new electronic version. Now that we are evolving into the full program, our methods are well-tested and yielding excellent results.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting

and/or Analyzing Data

CODA, the original contractor tasked with helping FDA develop and implement the MedSun pilot, assisted FDA in determining the design of the data collection. CODA has been sold to Social and Scientific Systems, which now assists FDA in the pilot design.

Social and Scientific Systems

8757 Georgia Ave.

12^th Floor

Silver Spring, MD 20910 301-268-0265 (Tina Powell – MedSun Lead for SSS)

FDA collects and analyzes the data.