sentinel ss part b 2-17-2010

Evaluation of Potential Sources for the Sentinel Initiative

0910-[NEW]

SUPPORTING STATEMENT

B. No advanced statistical analysis is required. However, the basic analytical approach is described below.

Booz Allen Hamilton will import completed survey responses into a single database for evaluation and reporting.  Summary counts and descriptive statistics will be reported for questions with distinct response options or numeric entry. 

For example, for questions with distinct response options, counts of respondents for each response option will be listed along with the total number of respondents for the given question. In addition, a percentage will be reported where the numerator is the number of respondents who chose the given response option and the denominator is the total number of respondents for the given question.

Using the summary counts and descriptive statistics described above, as well as high-level information provided in the respondents’ open ended questions, Booz Allen Hamilton will evaluate the data sources and/or data environments focusing on the following topics:

1. Size of population, degree of capture, and duration of longitudinal follow-up across different patient care settings and payment systems;

2. Structure and coding, including consistency with widely recognized standards;

3. Completeness, timeliness, and accessibility of data, including estimated times from service delivery to accessibility of data via queries;

4. Level of detail that would be available to examine temporal relationships between product administration and associated adverse events/outcomes; and

5. Potential limitations if used for post-market product safety surveillance.

The summary counts, descriptive statistics, and any key findings from the evaluation topics listed above, will be included in a final project report.

1. Respondent Universe and Sampling Methods

The FDA identified 246 health care organizations possessing electronic health care data that may contain be useful for active medical product surveillance. These organizations were identified by Booz Allen Hamilton in collaboration with FDA as potential data sources that would be useful to FDA in the Sentinel System. Not all organizations have expressed interest in being part of the Initiative. These organizations are grouped into the following types:

Organization Type	Count
Commercial Data Sets	9
Consumer Facing Applications	8
Federal Agencies	8
Health Information Exchanges	42
Health Plans and Health Plan Data Sources	18
Lab Data	2
Large Hospital Systems	31
Large Multi-Specialty Medical Groups and Academic Medical Centers	21
Medical Societies and Other Large Practice Data Sets	7
Pharmacy and PBM	24
Registries	61
Research Networks and Collaboratives	9
Other	6
Grand Total	246

FDA plans to send the survey to the identified health care organizations. In addition, the survey will be accessible on the FDA’s website. No additional sampling will be preformed. The data from all surveys will be compiled for the FDA’s review.

2. Procedures for the Collection of Information

A one-time survey will be used to collect information from potentially participating data sources and/or environments. The data we are seeking will describe the characteristics of the data available, not personally identifiable information. The findings will help FDA plan for this proposed system and for future work related to the Sentinel Initiative.

This survey will collect information on the scope, content, structure, quality, and timeliness of data; patient population(s), duration of follow up and capture of care across all settings; availability, experience and interest of investigators with knowledge of the data in using it for postmarket product safety surveillance as well as plans for further data source enhancements; availability, experience and interest of investigators with knowledge of the data in participating in a distributed data system; and barriers that exist to including each data source in the Sentinel Initiative.

As a first step, the Agency has let a series of contracts to inform the developing initiative. The contract, Evaluation of Potential Data Sources for the Sentinel Initiative, will require collection of information from potential data sources to better understand the characteristics of the data contained within their databases.

This project will require a thorough analysis of each data source’s: (1) utility for post-market surveillance of FDA-regulated drugs, biologics (excluding blood and tissue products), and devices; (2) scope, content, structure, quality, and timeliness of data; (3) patient population(s), duration of follow up and capture of care across all settings; (4) availability, experience and interest of investigators with knowledge of the data in using it for post-market product safety surveillance as well as plans for further data source enhancements; (5) availability, experience and interest of investigators with knowledge of the data in participating in a distributed data network; (6) potential privacy and security concerns related to the use of electronic health data; and (7) existing barriers that should be considered which may impact a data sources' ability to participate in the Sentinel Initiative

The contractor, Booz Allen Hamilton, that was awarded this contract has proposed developing an assessment tool to send to prospective data sources to voluntarily complete to characterize their data. This completed tool would be submitted to the contractor for analysis and recommendation to fulfill the requirements outlined in the statement of work.

3. Methods to Maximize Response Rates and Deal with Non-response

The Agency consulted with other Federal Agencies to optimize this survey tool. Comments were received and incorporated in the final survey from the National Institutes of Health and the Centers for Disease Control and Prevention.

In addition, the survey tool was beta tested with 9 private organizations and other Federal Agencies to determine its ease of use and understanding of questions. Comments received were incorporated in the final survey.

4. Test of Procedures or Methods to be Undertaken

The survey tool was beta tested with 9 private organizations and other Federal Agencies to determine its ease of use and understanding of questions. Comments received were incorporated in the final survey.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

None

INSTRUCTIONS AND BACKGROUND TO ACCOMPANY SURVEY

As part of its Sentinel Initiative, FDA has contracted with Booz Allen Hamilton to identify and characterize potential data sources that may participate in the Sentinel System. The FDA is conducting a survey to better understand the characteristics of data sources. We are asking data owners to complete the attached survey.

Background on Sentinel:

In May 2008, the Secretary of Health and Human Services and the FDA Commissioner announced the Sentinel Initiative. The Sentinel Initiative is a long-term effort by FDA to create a national electronic system for monitoring product safety.  Once developed and implemented, the Sentinel System is intended to augment FDA’s existing postmarket (primarily passive) safety surveillance systems and to enable FDA to actively gather information about postmarket safety and performance of FDA-regulated products.

As currently envisioned, the Sentinel System will enable FDA to capitalize on the capabilities of multiple, existing automated healthcare data systems (e.g. electronic health record systems, administrative claims databases, registries, others) to strengthen FDA’s current safety surveillance capabilities. The Sentinel System will enable queries of disparate data sources quickly and securely for relevant product safety information. Data will continue to be managed by its owners, and only data of organizations who agree to participate in the Sentinel System will be included. Queries would be sent to the appropriate participating data owners, who in turn would, in accordance with existing privacy and security safeguards, evaluate their data and send summary results for Agency review.

For additional background on the Initiative, please see: http://www.fda.gov/Safety/FDAsSentinelInitiative/default.htm.

Instructions:

Please complete the attached Excel survey and return your responses to Dorothy Stam at Booz Allen Hamilton by <insert date>. Dorothy’s email address is [email protected]. She can also be reached by telephone at 206.652.3018 if you have questions.

Please note there are three tabs to complete in the Excel survey.

Who should complete the survey?

All automated healthcare data owners are encouraged to complete the survey. The FDA is particularly interested in automated healthcare data sources that have the technical capability to link exposure of a FDA-regulated product to a health outcome.

What if an organization has more than one source of data?

If your organization has more than one source of data FDA would like your organization to complete a separate survey for each source of data.

What if an organization has limited resources to respond to the survey?

Although FDA encourages each organization to complete the entire survey, partially completed surveys will also be accepted.

What if an organization has additional supporting information to share with FDA?

In addition to collecting survey responses, FDA is interested in collecting any additional supporting information about a given data source that might help FDA better understand the data source, especially as it relates to its potential participation in the Sentinel System.

Information of Interest	Rationale/Importance
Describe capacity for data holder to validate signals via medical chart review	Help FDA determine if the signals that are detected and strengthened in database evaluations can be validated in source documentation as having actually occurred.
Describe strengths and potential limitations of the data source for postmarket surveillance	Help FDA better understand the performance characteristics of the data source from those most familiar with the data
Describe experience in safety signal detection	Assist FDA in better understanding capabilities of potential Sentinel System data sources
Describe available epidemiologic expertise, data management, and programming resources	Inform FDA on the available personnel capable of carrying out evaluations in the potential data source
Provide plans for future data source enhancements	Inform FDA about anticipated changes in data characteristics  and structure  as they relate to the current status of the data source reflected in the survey response
Describe existing policies and/or processes and agreements that are ordinarily put in place to enable the use of the data for public health purposes	Inform FDA of policy issues that will need to be addressed during the development of the Sentinel System
Describe interest in participating in a distributed data system for postmarket safety surveillance	Assist FDA in better understanding the scope of potential participants in the Sentinel System
Provide letters of support and curriculum vitae from internal and external researchers that describe their experience working with the data source that highlight current uses of the data	Assist FDA in better understanding how these uses relate to potential capabilities and functionalities envisioned within the Sentinel System

Who will see the responses to the survey?

Booz Allen Hamilton has been contracted by the FDA to conduct an evaluation of possible data sources for use in the Sentinel Initiative. They will receive responses and will employ the following procedures to protect confidentiality of information submitted by respondents.

• Respondent information will be accessed on “need to know” basis. Only those individuals responsible for results compilation and analysis will have access to this information

• All contractors with “need to know” access will have signed Non Public Information agreements.

• Respondent Microsoft Excel files will be password protected upon receipt.

• Respondent information will be stored on Booz Allen computers that are protected by Safeboot Encryption as well as Microsoft Windows encryption

• Any paper copies of respondent information will be stored in locked cabinets

• Upon contract close-out, all respondent information will be provided to FDA and contract will erase and destroy all responses.

Booz Allen Hamilton, as a part of the deliverable for this contract, will complete a report detailing the findings from this survey which will be made publicly available on FDA’s Sentinel Initiative website. Any information included in any survey responses that is deemed proprietary in nature and is therefore not releaseable in accordance with the Freedom of Information Act will be remain confidential in accordance with the Act.

Who should an organization contact if they have questions about the survey?

If questions arise related to the survey, please contact Dorothy Stam at Booz Allen Hamilton. Dorothy’s email address is [email protected]. She can also be reached by telephone at 206.652.3018.