sentinel ss part a 2-17-2010

Evaluation of Potential Sources for the Sentinel Initiative

0910-[NEW]

SUPPORTING STATEMENT

A. Justification

1. Circumstances Making the Collection of Information Necessary

The Food and Drug Administration (FDA) is requesting OMB approval of the following information collection through a voluntary survey as part of the Sentinel Initiative.

In the fall of 2007, Congress passed the Food and Drug Administration Amendment Act (FDAAA), mandating FDA to establish an active surveillance system for monitoring drugs, using electronic data from healthcare information holders. The Sentinel Initiative is the FDA’s response to that mandate. Its goal is to build and implement a new active surveillance system that will eventually be used to monitor all FDA-regulated products.

Section 905 of FDAAA calls for the Health and Human Services Secretary to develop methods to obtain access to disparate data sources and to establish a postmarket risk identification and analysis system to link and analyze healthcare data from multiple sources. The law sets a goal of access to data from 25 million patients by July 1, 2010, and 100 million patients by July 1, 2012. The law also requires FDA to work closely with partners from public, academic, and private entities.

As currently envisioned, the Sentinel Initiative will enable the Agency to capitalize on the capabilities of multiple, existing automated healthcare data systems (e.g. electronic health record systems, administrative claims databases, registries) to augment the Agency’s current surveillance capabilities. The Sentinel Initiative will enable queries of disparate data sources quickly and securely for relevant product safety information. Data will continue to be managed by its owners, and only data of organizations who agree to participate in this system will be included. Questions would be sent to appropriate, participating data sources, who in turn would, in accordance with existing privacy and security safeguards, evaluate their data and send results for Agency review. The Agency plans for all Personal Health Information (PHI) to remain with data owners in their secure environment. Those who manage and maintain the data will conduct analyses, and will transmit summary results. Therefore, there will be no transmission or disclosure of PHI. All use of PHI will be in accordance with applicable privacy regulations.

The success of this Initiative will depend largely on the content, quality, searchability, and responsiveness of participating data sources and/or data environments. It is essential that FDA understand the strengths and limitations of potential data sources that might be included in the Sentinel Initiative.

2. Purpose and Use of the Information Collection

A one-time survey will be used to collect information from potentially participating data sources and/or environments. The data we are seeking will describe the characteristics of the data available, not personally identifiable information. The findings will help FDA plan for this proposed system and for future work related to the Sentinel Initiative.

This survey will collect information on the scope, content, structure, quality, and timeliness of data; patient population(s), duration of follow up and capture of care across all settings; availability, experience and interest of investigators with knowledge of the data in using it for postmarket product safety surveillance as well as plans for further data source enhancements; availability, experience and interest of investigators with knowledge of the data in participating in a distributed data system; and barriers that exist to including each data source in the Sentinel Initiative.

As a first step, the Agency has let a series of contracts to inform the developing initiative. The contract, Evaluation of Potential Data Sources for the Sentinel Initiative, will require collection of information from potential data sources to better understand the characteristics of the data contained within their databases.

This project will require a thorough analysis of each data source’s: (1) utility for post-market surveillance of FDA-regulated drugs, biologics (excluding blood and tissue products), and devices; (2) scope, content, structure, quality, and timeliness of data; (3) patient population(s), duration of follow up and capture of care across all settings; (4) availability, experience and interest of investigators with knowledge of the data in using it for post-market product safety surveillance as well as plans for further data source enhancements; (5) availability, experience and interest of investigators with knowledge of the data in participating in a distributed data network; (6) potential privacy and security concerns related to the use of electronic health data; and (7) barriers that exist to including each data source in the Sentinel Initiative.

The contractor, Booz Allen Hamilton, that was awarded this contract has proposed developing an assessment tool to send to prospective data sources to voluntarily complete to characterize their data. This completed tool would be submitted to the contractor for analysis and recommendation to fulfill the requirements outlined in the statement of work.

3. Use of Improved Information Technology and Burden Reduction

The assessment tool has been developed using Microsoft Excel. Respondents are being asked to submit their responses electronically using the assessment tool.

4. Efforts to Identify Duplication and Use of Similar Information

FDA is the only agency that requests this information. There is no similar kind of information available from any other source.

5. Impact on Small Businesses or Other Small Entities

This survey will not have any impact on small businesses or other small entities.

6. Consequences of Collecting the Information Less Frequently

This collection of information through a survey is a one-time collection.

There are no technical or legal obstacles to reducing the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This voluntary survey is a one-time collection of information to understand the characteristics encompassed in the electronic data of the respondent.

Respondents may consider some of the detailed information regarding their data sources to be proprietary. The contractor, Booz Allen Hamilton, will employ the procedures described in section 10 of this document to protect the information submitted by respondents.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with 5 CFR 1320.8(d), FDA published a 60-day notice for public comment in the FEDERAL REGISTER of 03/09/2009 (Vol. 74 No. 44 FR page number 10053). One comment was received. This comment did not address (1) Whether the proposed collection of information is necessary for the proper performance of FDA's functions, including whether the information will have practical utility; (2) the accuracy of FDA's estimate of the burden of the proposed collection of information, including the validity of the methodology and assumptions used; (3) ways to enhance the quality, utility, and clarity of the information to be collected; or (4) ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques, when appropriate, and other forms of information technology. It did however note that this data collection is an important first step in beginning to assess how to address the complex issues associated with using federated data to drive the Sentinel Initiative.

The Agency consulted with other Federal Agencies to optimize this survey tool. Comments were received and incorporated in the final survey from the National Institutes of Health and the Centers for Disease Control and Prevention.

In addition, the survey tool was beta tested with 9 private organizations and other Federal Agencies to determine its ease of use and understanding of questions. Comments received were incorporated in the final survey.

9. Explanation of Any Payment or Gift to Respondents

FDA has not provided and has no intention to provide any payments or gifts to respondents.

10. Assurance of Confidentiality Provided to Respondents

Respondents may consider detailed information regarding their data sources to be proprietary. The contractor, Booz Allen Hamilton, will employ the following procedures to protect the information submitted by respondents.

1. Respondent information will be accessed on “need to know” basis. Only those individuals responsible for results compilation and analysis will have access to this information

2. All contractors with “need to know” access will have signed Non Public Information agreements (FDA Form 3398).

3. Respondent Microsoft Excel files will be password protected upon receipt.

4. Respondent information will be stored on Booz Allen computers that are protected by Safeboot Encryption as well as Microsoft Windows encryption

5. Any paper copies of respondent information will be stored in locked cabinets

6. Upon contract close-out, all respondent information will be provided to FDA and contract will erase and destroy all responses.

Booz Allen Hamilton, as a part of the deliverable for this contract, will complete a report detailing the findings from this survey which will be made publicly available on FDA’s Sentinel Initiative website. Any information included in any survey responses that is deemed proprietary in nature and is therefore not releaseable in accordance with the Freedom of Information Act will be remain confidential in accordance with the Act.

11. Justification for Sensitive Questions

Respondents may consider detailed information regarding their data sources to be proprietary. The contractor, Booz Allen Hamilton, will employ the procedures described in section 10 of this document to protect the information submitted by respondents.

12. Estimates of Annualized Burden Hours and Costs

The estimated annual burden for this information collection is 4,375 hours.

Table 1.--Estimated Annual Reporting Burden1
Activity	No. of Respondents	Annual Frequency per Response	Total Annual Responses	Hours per Response	Total Hours
Data Source and/or Environment Survey	250	1	250	17.5	4,375

¹ There are no capital costs or operating and maintenance costs associated with this collection of information.

FDA estimates that approximately 250 respondents will participate in this voluntary survey. These respondents will consist mostly of other Federal agencies, health plan data sources, health information exchanges, large multi-specialty medical groups and academic medical centers, large hospital systems, pharmacies, medical societies, consumer-oriented websites, commercial data sets, research networks, lab data, and registries.

Each respondent will extend approximately 17.5 hours to complete 1 survey for a total of 6,125 hours (250 x 1 x 17.5 = 4,375).

The estimated cost burden for each organization is $2,957.50 based on 17.5 hours of effort at a rate of $169 hour for analysis by a data manager (17.5 x $169.00 = $2,957.50).

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There are no capital costs or operating and maintenance costs associated with this collection.

14. Annualized Cost to the Federal Government

This one-time survey is being carried out to by Booz Allen Hamilton in response to a contract that was awarded by FDA on September 19, 2008 for $96,532.00.

15. Explanation for Program Changes or Adjustments

This is a one time collection of information as a result of FDAAA, which mandates FDA to establish an active surveillance system for monitoring drugs, using electronic data from healthcare information holders. The Sentinel Initiative is the FDA’s response to that mandate. Its goal is to build and implement a new active surveillance system that will eventually be used to monitor all FDA-regulated products.

16. Plans for Tabulation and Publication and Project Time Schedule

There are no tabulated results to publish for this information collection.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

FDA is not seeking approval to exempt display of the expiration date for OMB approval.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Not applicable.

B.

No advanced statistical analysis is required. However, the basic analytical approach is described below.

Booz Allen Hamilton will import completed survey responses into a single database for evaluation and reporting.  Summary counts and descriptive statistics will be reported for questions with distinct response options or numeric entry. 

For example, for questions with distinct response options, counts of respondents for each response option will be listed along with the total number of respondents for the given question. In addition, a percentage will be reported where the numerator is the number of respondents who chose the given response option and the denominator is the total number of respondents for the given question.

Using the summary counts and descriptive statistics described above, as well as high-level information provided in the respondents’ open ended questions, Booz Allen Hamilton will evaluate the data sources and/or data environments focusing on the following topics:

1. Size of population, degree of capture, and duration of longitudinal follow-up across different patient care settings and payment systems;

2. Structure and coding, including consistency with widely recognized standards;

3. Completeness, timeliness, and accessibility of data, including estimated times from service delivery to accessibility of data via queries;

4. Level of detail that would be available to examine temporal relationships between product administration and associated adverse events/outcomes; and

5. Potential limitations if used for post-market product safety surveillance.

The summary counts, descriptive statistics, and any key findings from the evaluation topics listed above, will be included in a final project report.

1. Respondent Universe and Sampling Methods

The FDA identified 246 health care organizations possessing electronic health care data that may contain be useful for active medical product surveillance. These organizations were identified by Booz Allen Hamilton in collaboration with FDA as potential data sources that would be useful to FDA in the Sentinel System. Not all organizations have expressed interest in being part of the Initiative. These organizations are grouped into the following types:

Organization Type	Count
Commercial Data Sets	9
Consumer Facing Applications	8
Federal Agencies	8
Health Information Exchanges	42
Health Plans and Health Plan Data Sources	18
Lab Data	2
Large Hospital Systems	31
Large Multi-Specialty Medical Groups and Academic Medical Centers	21
Medical Societies and Other Large Practice Data Sets	7
Pharmacy and PBM	24
Registries	61
Research Networks and Collaboratives	9
Other	6
Grand Total	246

FDA plans to send the survey to the identified health care organizations. In addition, the survey will be accessible on the FDA’s website. No additional sampling will be preformed. The data from all surveys will be compiled for the FDA’s review.

2. Procedures for the Collection of Information

A one-time survey will be used to collect information from potentially participating data sources and/or environments. The data we are seeking will describe the characteristics of the data available, not personally identifiable information. The findings will help FDA plan for this proposed system and for future work related to the Sentinel Initiative.

This survey will collect information on the scope, content, structure, quality, and timeliness of data; patient population(s), duration of follow up and capture of care across all settings; availability, experience and interest of investigators with knowledge of the data in using it for postmarket product safety surveillance as well as plans for further data source enhancements; availability, experience and interest of investigators with knowledge of the data in participating in a distributed data system; and barriers that exist to including each data source in the Sentinel Initiative.

As a first step, the Agency has let a series of contracts to inform the developing initiative. The contract, Evaluation of Potential Data Sources for the Sentinel Initiative, will require collection of information from potential data sources to better understand the characteristics of the data contained within their databases.

This project will require a thorough analysis of each data source’s: (1) utility for post-market surveillance of FDA-regulated drugs, biologics (excluding blood and tissue products), and devices; (2) scope, content, structure, quality, and timeliness of data; (3) patient population(s), duration of follow up and capture of care across all settings; (4) availability, experience and interest of investigators with knowledge of the data in using it for post-market product safety surveillance as well as plans for further data source enhancements; (5) availability, experience and interest of investigators with knowledge of the data in participating in a distributed data network; (6) potential privacy and security concerns related to the use of electronic health data; and (7) existing barriers that should be considered which may impact a data sources' ability to participate in the Sentinel Initiative

The contractor, Booz Allen Hamilton, that was awarded this contract has proposed developing an assessment tool to send to prospective data sources to voluntarily complete to characterize their data. This completed tool would be submitted to the contractor for analysis and recommendation to fulfill the requirements outlined in the statement of work.

3. Methods to Maximize Response Rates and Deal with Non-response

The Agency consulted with other Federal Agencies to optimize this survey tool. Comments were received and incorporated in the final survey from the National Institutes of Health and the Centers for Disease Control and Prevention.

In addition, the survey tool was beta tested with 9 private organizations and other Federal Agencies to determine its ease of use and understanding of questions. Comments received were incorporated in the final survey.

4. Test of Procedures or Methods to be Undertaken

The survey tool was beta tested with 9 private organizations and other Federal Agencies to determine its ease of use and understanding of questions. Comments received were incorporated in the final survey.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

None

INSTRUCTIONS AND BACKGROUND TO ACCOMPANY SURVEY

As part of its Sentinel Initiative, FDA has contracted with Booz Allen Hamilton to identify and characterize potential data sources that may participate in the Sentinel System. The FDA is conducting a survey to better understand the characteristics of data sources. We are asking data owners to complete the attached survey.

Background on Sentinel:

In May 2008, the Secretary of Health and Human Services and the FDA Commissioner announced the Sentinel Initiative. The Sentinel Initiative is a long-term effort by FDA to create a national electronic system for monitoring product safety.  Once developed and implemented, the Sentinel System is intended to augment FDA’s existing postmarket (primarily passive) safety surveillance systems and to enable FDA to actively gather information about postmarket safety and performance of FDA-regulated products.

As currently envisioned, the Sentinel System will enable FDA to capitalize on the capabilities of multiple, existing automated healthcare data systems (e.g. electronic health record systems, administrative claims databases, registries, others) to strengthen FDA’s current safety surveillance capabilities. The Sentinel System will enable queries of disparate data sources quickly and securely for relevant product safety information. Data will continue to be managed by its owners, and only data of organizations who agree to participate in the Sentinel System will be included. Queries would be sent to the appropriate participating data owners, who in turn would, in accordance with existing privacy and security safeguards, evaluate their data and send summary results for Agency review.

For additional background on the Initiative, please see: http://www.fda.gov/Safety/FDAsSentinelInitiative/default.htm.

Instructions:

Please complete the attached Excel survey and return your responses to Dorothy Stam at Booz Allen Hamilton by <insert date>. Dorothy’s email address is [email protected]. She can also be reached by telephone at 206.652.3018 if you have questions.

Please note there are three tabs to complete in the Excel survey.

Who should complete the survey?

All automated healthcare data owners are encouraged to complete the survey. The FDA is particularly interested in automated healthcare data sources that have the technical capability to link exposure of a FDA-regulated product to a health outcome.

What if an organization has more than one source of data?

If your organization has more than one source of data FDA would like your organization to complete a separate survey for each source of data.

What if an organization has limited resources to respond to the survey?

Although FDA encourages each organization to complete the entire survey, partially completed surveys will also be accepted.

What if an organization has additional supporting information to share with FDA?

In addition to collecting survey responses, FDA is interested in collecting any additional supporting information about a given data source that might help FDA better understand the data source, especially as it relates to its potential participation in the Sentinel System.

Information of Interest	Rationale/Importance
Describe capacity for data holder to validate signals via medical chart review	Help FDA determine if the signals that are detected and strengthened in database evaluations can be validated in source documentation as having actually occurred.
Describe strengths and potential limitations of the data source for postmarket surveillance	Help FDA better understand the performance characteristics of the data source from those most familiar with the data
Describe experience in safety signal detection	Assist FDA in better understanding capabilities of potential Sentinel System data sources
Describe available epidemiologic expertise, data management, and programming resources	Inform FDA on the available personnel capable of carrying out evaluations in the potential data source
Provide plans for future data source enhancements	Inform FDA about anticipated changes in data characteristics  and structure  as they relate to the current status of the data source reflected in the survey response
Describe existing policies and/or processes and agreements that are ordinarily put in place to enable the use of the data for public health purposes	Inform FDA of policy issues that will need to be addressed during the development of the Sentinel System
Describe interest in participating in a distributed data system for postmarket safety surveillance	Assist FDA in better understanding the scope of potential participants in the Sentinel System
Provide letters of support and curriculum vitae from internal and external researchers that describe their experience working with the data source that highlight current uses of the data	Assist FDA in better understanding how these uses relate to potential capabilities and functionalities envisioned within the Sentinel System

Who will see the responses to the survey?

Booz Allen Hamilton has been contracted by the FDA to conduct an evaluation of possible data sources for use in the Sentinel Initiative. They will receive responses and will employ the following procedures to protect confidentiality of information submitted by respondents.

• Respondent information will be accessed on “need to know” basis. Only those individuals responsible for results compilation and analysis will have access to this information

• All contractors with “need to know” access will have signed Non Public Information agreements.

• Respondent Microsoft Excel files will be password protected upon receipt.

• Respondent information will be stored on Booz Allen computers that are protected by Safeboot Encryption as well as Microsoft Windows encryption

• Any paper copies of respondent information will be stored in locked cabinets

• Upon contract close-out, all respondent information will be provided to FDA and contract will erase and destroy all responses.

Booz Allen Hamilton, as a part of the deliverable for this contract, will complete a report detailing the findings from this survey which will be made publicly available on FDA’s Sentinel Initiative website. Any information included in any survey responses that is deemed proprietary in nature and is therefore not releaseable in accordance with the Freedom of Information Act will be remain confidential in accordance with the Act.

Who should an organization contact if they have questions about the survey?

If questions arise related to the survey, please contact Dorothy Stam at Booz Allen Hamilton. Dorothy’s email address is [email protected]. She can also be reached by telephone at 206.652.3018.