CLIA Waiver Guidance

Guidance for Industry and FDA Staff:

Recommendations for Clinical Laboratory Improvement Amendments of 1988 (CLIA) Waiver Applications for Manufacturers of In Vitro Diagnostic Devices

Document issued on: January 30, 2008

The draft of this guidance document was released on September 7, 2005.

For questions regarding this document contact Carol Benson, 301-796-5459 or by email at [email protected]

OMB control number: 0910-0598

Expiration Date: XX/XX/XXXX

See additional PRA statement in Section VIII of this guidance

U.S. Department of Health and Human Services

Food and Drug Administration

Center for Devices and Radiological Health

Office of In Vitro Diagnostic Device Evaluation and Safety

_Preface

Public Comment

Written comments and suggestions may be submitted at any time for Agency consideration to the

Division of Dockets Management, Food and Drug Administration, 5630 Fishers Lane, Room

1061, (HFA-305), Rockville, MD, 20852. Alternatively, electronic comments may be submitted to http://www.fda.gov/dockets/ecomments. When submitting comments, please refer to Docket No. 2008D-0031. Comments may not be acted upon by the Agency until the document is next revised or updated.

Additional Copies

Additional copies are available from the Internet http://www.fda.gov/cdrh/oivd/guidance/1171.pdf. You may also send an email request to dsmi[email protected].gov to receive an electronic copy of the guidance, or send a FAX request to 240-276-3151 to receive a hard copy. Please use the

document number 1171 to identify the guidance you are requesting.

TABLE OF CONTENTS

I. INTRODUCTION............................................................................................................. 5

The Least Burdensome Approach ............................................................................................... 7

II. COMPONENTS OF A CLIA WAIVER APPLICATION............................................ 7

III. DEMONSTRATING "SIMPLE".................................................................................... 8

IV. DEMONSTRATING "INSIGNIFICANT RISK OF AN ERRONEOUS RESULT" – Failure Alerts and Fail-Safe Mechanisms....................................................................... 9

A. Tier 1: Risk Analysis and Flex Studies.......................................................................... 11

B. Tier 2: Fail-Safe and Failure Alert Mechanisms.......................................................... 13

1. Points to consider for designing fail-safe and failure alert mechanisms........ 13

2. External control materials ................................................................................. 14

3. Additional points concerning control materials............................................... 15

C. Validating Fail-Safe and Failure Alert Mechanisms, including External Control

Procedures ............................................................................................................................. 15

V. DEMONSTRATING INSIGNIFICANT RISK OF AN ERRONEOUS RESULT – “ACCURACY” ............................................................................................................... 16

A. Clinical Study Sites and Participants ............................................................................ 17

1. Testing sites.......................................................................................................... 17

2. Clinical study participants ................................................................................. 17

3. Clinical study reports ......................................................................................... 19

B. Clinical Studies for Tests with Quantitative Results.................................................... 19

1. Selection of the Comparative Method (CM) .................................................... 20

2. Establishing Allowable Total Error Zones and Zones for Erroneous Results

............................................................................................................................... 20

3. Clinical Study Design.......................................................................................... 22

C. Clinical Studies for Tests with Qualitative Results...................................................... 26

1. Selection of the Comparative Method (CM) .................................................... 26

2. Clinical Study Design.......................................................................................... 27

3. Performance Criteria for Qualitative Tests ..................................................... 31

VI. LABELING FOR WAIVED DEVICES ....................................................................... 32

A. Quick Reference Instructions (QRI) ............................................................................. 33

B. Quality Control (QC) Labeling Recommendations ..................................................... 33

C. Educational Information ................................................................................................ 34

VII. SAFEGUARDS FOR WAIVED TESTS....................................................................... 35

VIII. Paperwork Reduction Act of 1995................................................................................. 35

REFERENCES............................................................................................................................ 36

Appendix A: Statistical Notes .................................................................................................... 38

Appendix B: Labeling................................................................................................................. 40

Appendix C: Definition of Terms as Used in this Document ................................................. 42

_{Guidance for Industry and FDA Staff:}

Recommendations for Clinical Laboratory Improvement Amendments of 1988 (CLIA) Waiver Applications for Manufacturers of In Vitro Diagnostic Devices

	This guidance represents the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for, or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance.

I. INTRODUCTION

The Secretary of Health and Human Services has delegated to FDA the authority to determine whether particular tests are "simple" and have "an insignificant risk of an erroneous result" under CLIA and thus eligible for waiver categorization (69 FR 22849, April 29, 2004). The Centers

for Medicare & and Medicaid Services (CMS) is responsible for oversight of clinical laboratories, which includes issuing waiver certificates. CLIA requires that clinical laboratories obtain a certificate before accepting materials derived from the human body for laboratory tests.

42 U.S.C. § 263a(b). Laboratories that perform only tests that are "simple" and that have an

"insignificant risk of an erroneous result" may obtain a certificate of waiver. 42 U.S.C.

§ 263a(d)(2).

CLIA, 42 U.S.C. § 263a(d)(3) Examinations and Procedures, as modified by the Food and Drug Administration Modernization Act (FDAMA), reads as follows regarding tests that may be performed by laboratories with a Certificate of Waiver:

The examinations and procedures [that may be performed by a laboratory with a Certificate of Waiver]… are laboratory examinations and procedures that have been approved by the Food and Drug Administration for home use or that, as determined by the Secretary, are simple laboratory examinations and procedures that have an insignificant risk of an erroneous result, including those that -- (A) employ methodologies that are so simple and accurate as to render the likelihood of erroneous results by the user negligible, or (B) the

Secretary has determined pose no unreasonable risk of harm to the patient if performed incorrectly.

This guidance describes recommendations for device manufacturers seeking to submit information through a CLIA waiver application to FDA to support a determination whether the device meets CLIA statutory criteria for waiver.

In this document, FDA (we) recommends an approach for you to use to demonstrate that your device is simple and has an insignificant risk of an erroneous result. As part of demonstrating the latter, we recommend studies you can conduct to demonstrate the test is "accurate." While we recommend you adopt the approach we have outlined in this guidance for waiver applications, you may use another approach that you believe would be appropriate for your device’s waiver application if it meets the CLIA statutory requirements. (See Least Burdensome Approach, below.)

We based the recommendations in this document on our interpretation of the law, experience with CLIA complexity determinations, and interactions with stakeholders. Interactions included an open public workshop on August 14-15, 2000, a proposal presented by AdvaMed (Advanced Medical Technology Association) at the September 2003 Clinical Laboratory Improvement Advisory Committee (CLIAC) meeting, and recommendations proposed by CLIAC during the February 2004 meeting. In addition, we considered the comments received in response to a 2001 draft guidance and the 2005 draft guidance document, and incorporated revisions based on these comments as appropriate. ¹

Some of the changes reflected in this document (as well as in the 2005 draft guidance document) from the earlier 2001 draft guidance document entitled “Guidance for CLIA 1988 Criteria for Waiver,” include the following:

• Greater emphasis on scientifically-based flex studies² and validation and/or verification studies, linked to the risk assessment for each device.

• Recognition that reference methods may not be available for every device type. (However, devices should be traceable to true reference methods of known accuracy, when such methods are available.)

• Additional emphasis on use of quality control procedures.

• Greater emphasis on intended users (which may include medical assistants, nurses or doctors, and lay people, as appropriate) during studies testing the device.

• Updated study recommendations with emphasis on use of patient specimens, in an intended use environment, over time.

¹ The draft document of September 7, 2005, entitled “Recommendations for Clinical Laboratory Improvement Amendments of 1988 Waiver Applications” (70 FR 53231) replaced the draft “Guidance for Clinical Laboratory Improvement Amendments of 1998 (CLIA) Criteria for Waiver,” March 1, 2001 (66 FR 12939)).

² For the definition of this term and other technical terms, as they are used in this document, see

Appendix C.

This document does not address test systems cleared or approved by FDA for over-the-counter or prescription home use since these automatically qualify for CLIA waiver. 42 U.S.C. 263a(d)(3). This guidance document also does not address use of the Office of In Vitro Diagnostic Device Evaluation and Safety (OIVD)’s replacement reagent and instrument family policy³ for waived devices; that policy does not currently apply to CLIA waiver applications.

The draft of this document was issued September 7, 2005. We have also issued a draft guidance entitled "Guidance for Administrative Procedures for CLIA Categorization," www.fda.gov/cdrh/ode/guidance/1143.html. In it, we propose recommendations to device manufacturers on FDA’s administrative procedures for CLIA categorization.

FDA's guidance documents, including this guidance, do not establish legally enforceable responsibilities. Instead, guidances describe the Agency's current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidances means that something is suggested or recommended, but not required.

The Least Burdensome Approach

We believe we should consider the least burdensome approach in all areas of medical device regulation. This guidance reflects our careful review of the relevant scientific and legal requirements and what we believe is the least burdensome way for you to comply with those requirements. However, if you believe that an alternative approach would be less burdensome, please contact us so we can consider your point of view. You may send your written comments to the contact person listed in the preface to this guidance or to the CDRH Ombudsman. Comprehensive information on CDRH's Ombudsman, including ways to contact him, can be found on the Internet at http://www.fda.gov/cdrh/ombudsman/.

II. COMPONENTS OF A CLIA WAIVER APPLICATION

This guidance discusses the following components that we recommend you include in a CLIA

waiver application:

• A description of your device that demonstrates it is simple to use. (Section III.)

• The results of risk analysis including the identification of potential sources of error for your device. (Section IV.)

• The results of flex studies demonstrating insensitivity of the test system to environmental and usage variations under conditions of stress. (Section IV.)

• The results of risk evaluation and control including a description of (1) measures you have implemented to mitigate the risk of errors, and (2) validation and/or verification studies demonstrating the ability of failure alert, fail-safe mechanisms, and other control

³ For a description of this policy, see Guidance for Industry and FDA Staff; Replacement

Reagent and Instrument Family Policy, http://www.fda.gov/cdrh/oivd/guidance/950.html

measures that you have incorporated into your device to mitigate the risk of errors, even under conditions of stress. (Section IV.)

• A description of the design and results of clinical studies you conducted to demonstrate that the device has an insignificant risk of erroneous result in the hands of the intended user (hereinafter operator). (Section V.)

• Proposed labeling with instructions for use consistent with a device that is “simple.” (Section VI.)

III. DEMONSTRATING "SIMPLE"

CLIA requires that tests performed by laboratories with a Certificate of Waiver be "simple." 42

U.S.C. 263a(d)(2), (3). We recommend that, as a first step in the process of deciding whether your device could be a candidate for waiver, you determine whether your device is simple.

Under the approach recommended in this guidance, FDA believes that a simple test should have characteristics such as the following:

• Is a fully automated instrument or a unitized or self-contained test.

• Uses direct unprocessed specimens, such as capillary blood (fingerstick), venous whole blood, nasal swabs, throat swabs, or urine.

• Needs only basic, non-technique-dependent specimen manipulation, including any for decontamination.

• Needs only basic, non-technique-dependent reagent manipulation, such as “mix reagent

A and reagent B.”

• Needs no operator intervention during the analysis steps.

• Needs no technical or specialized training with respect to troubleshooting or interpretation of multiple or complex error codes.

• Needs no electronic or mechanical maintenance beyond simple tasks, e.g., changing a battery or power cord.

• Produces results that require no operator calibration, interpretation, or calculation.

• Produces results that are easy to determine, such as ‘positive’ or ‘negative,’ a direct readout of numerical values, the clear presence or absence of a line, or obvious color gradations.

• Provides instructions in the package insert for obtaining and shipping specimens for confirmation testing in cases where such testing is clinically advisable.

• Has test performance comparable to a traceable reference method as demonstrated by studies in which intended operators⁴ perform the test. If a reference method is not

⁴ In this guidance, intended operator refers to a test operator with limited or no training or hands- on experience in conducting laboratory testing. Laboratory professional refers to a person who

available for a test you are proposing for waiver, please contact OIVD to discuss your proposed plan prior to submitting your application.

• Contains a quick reference instruction sheet that is written at no higher than a 7^th grade reading level.

We believe a test that is simple should not have the following characteristics:

• Sample manipulation is required to perform the assay. (For example, tests that use plasma or serum are not considered simple.) Sample manipulation includes processes such as centrifugation, complex mixing steps, or evaluation of the sample by the operator for conditions such as hemolysis or lipemia.

• Measurement of an analyte could be affected by conditions such as sample turbidity or cell lysis.

After you consider whether your device is “simple” based on the items listed above, it may be helpful for you to contact OIVD for feedback on this issue prior to conducting clinical studies to support waiver. In your waiver application, you should describe features of your device that address the issues listed above. Whenever possible (for example, if your test system consists of a unitized device), you should include sample(s) of the device with your waiver application to aid FDA in its determination of whether your device is “simple.” You may also schedule a meeting to bring your device to FDA to aid FDA in making this determination.

IV. DEMONSTRATING "INSIGNIFICANT RISK OF AN ERRONEOUS RESULT" – Failure Alerts and Fail-Safe Mechanisms

Generally, the risk of an erroneous result should be far less for waived tests than non-waived tests. You should demonstrate in your CLIA waiver application that (1) the test system design is robust, e.g., insensitive to environmental and usage variation, and (2) that all known sources of error are effectively controlled. In general, flex studies should be used to demonstrate robust design while risk management should be used to demonstrate identification and effective control of error sources, although the two are not mutually exclusive.

Most risk control measures should be fail-safe measures or failure alert mechanisms. Appropriate fail-safe mechanisms and failure alert mechanisms help assure that a test has “an insignificant risk of an erroneous result.” Examples of fail-safe mechanisms are lock-out functions to ensure that a test system does not provide a result when test conditions are inappropriate, such when there is a component malfunction or operator error. Other examples are measures within the system to prevent operator error, such as guides or channels that prevent improper strip placement. We recommend that test system design incorporate fail-safe mechanisms whenever it is technically practicable.

meets the qualifications to perform moderate or high complexity testing, such as a medical technologist (MT) or medical laboratory technician (MLT).

If fail-safe mechanisms are not technically practicable for some risks, failure alert mechanisms should be used. Failure alert mechanisms notify the operator of any test system malfunction or problem. They may include measures such as external controls, internal procedural controls, or electronic controls. Devices with such mechanisms allow the operator to correct the error, or put the operator on notice that the results will be unreliable due to the error. For example, in cases where the result exceeds the reportable range (e.g., extremely high or low glucose result) and the result is a critical value, the device should give a message such as "out of range high" or "out of range low."

We recommend a two-tiered approach, outlined below, to demonstrate that your device is robust and has appropriate and effective risk control measures to ensure insignificant risk of an erroneous result.

Tier 1: Risk Analysis and Flex Studies. You should conduct a systematic and comprehensive risk analysis that identifies all potential sources of error, including test system failures and operator errors, and identifies which of these errors can lead to a risk of a hazardous situation. We recommend that the “Operator error/Human factors” examples on pages 12-13 be used as an analytical aid to complement the risk analysis method(s) you use.

You should conduct flex studies, i.e., studies that stress the operational limits of your test system. Flex studies should be used to validate the insensitivity of the test system to variation under

stress conditions. Where appropriate, flex studies should also be used to verify and/or validate

the effectiveness of control measures at operational limits. [See also Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices (http://www.fda.gov/cdrh/ode/guidance/337.html) for further discussion.]

In your waiver application you should include:

• The risk analysis results which serve as a basis for the tabular reporting of risk management results. (See Tier 2.)

• A summary of the design and results of your flex studies.

• Conclusions you draw from the flex studies.

Tier 2: Fail-Safe and Failure Alert Mechanisms. Once you have identified the potential sources of error, you should identify the control measures, including fail-safe and failure alert mechanisms that will reduce risks for these sources of error. When the control measures have been implemented, you should (1) verify that each control measure has been properly implemented, and (2) verify and/or validate the effectiveness of each control measure.

We recommend that this risk management information be presented in tabular form in your waiver application. It should include the following information for each risk for each potential source of error:

• Identification of each risk and the potential source of error that causes it.

• Identification and physical description of the risk control measure or combination of

measures used to reduce risk to an acceptable level. This includes fail-safe mechanisms, failure alert mechanisms, external controls as well as any other controls used or that you recommend the operator use for your device. It also includes a description of the manner in which the control measure(s) either reduces the probability of occurrence of the error, mitigates the effect of the error, or both.

• Objective evidence verifying that each control measure or combination of measures has been implemented, including a description of the method of verification.

• Objective evidence from testing, confirming the effectiveness of fail-safe and/or failure- alert mechanisms in preventing and/or mitigating the effects of false results. The evidence and results should also support your recommended control procedures and frequencies. Any limitations of fail-safe and failure alert mechanisms, including all internal and external controls, should be described.

A. Tier 1: Risk Analysis and Flex Studies

As noted above, you should identify all potential sources of error by conducting a systematic and comprehensive risk analysis. This analysis should be part of your risk management process consisting of risk analysis, evaluation, and control. FDA recognizes the process of the International Standard ISO 14971, Medical Devices - Application of Risk Management to Medical Devices. http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfStandards/Detail.CFM?STANDARD I DENTIFICATION_NO=5188. This guidance is consistent with that process and uses the same risk management terminology. In general, the standard and its annexes can be used to obtain more detailed information about risk management concepts and practices⁵ .

Based on the results of the risk analysis and identification of potential problems with sensitivity to environmental or usage variation, you should conduct flex studies. Flex studies are designed to challenge the system under conditions of stress to identify potential device deficiencies, including failures, and determine the robustness of the test system. Examples are shown in Table 1.

In your analysis, you should consider multiple skill levels of users, as well as potential instrument and reagent problems. The following websites contain additional information to consider concerning human factors that may affect test performance: http://www.fda.gov/cdrh/humanfactors/index.html, http://www.fda.gov/cdrh/humanfactors/resource-manufac.html#2.

Examples of potential sources of error to consider for the risk analysis and flex studies are listed below. See also CLSI EP-18A for examples (See [1]). You should consider each of these potential sources of error, as applicable to your device, and also consider any other potential system failures that may be specific to your device.

Operator error/ Human factors

⁵ However, it may not always be appropriate to justify that risks are acceptable based solely on the “As Low As Reasonably Practicable" principle described in Annex D of ISO 14971: 2007.

• Use of incorrect specimen type.

• Incorrect application of the specimen to the device (e.g., incorrect placement, incorrect volume).

• Incorrect handling of reagents including those in self-contained unitized test devices.

• Incorrect placement of device (e.g., non-level surface).

• Incorrect placement of reagents, including strips, or other components that contain reagents.

• Use of incorrect reagents (for example, reagents that are not specific for the particular device or lot or generic reagents).

• Incorrect order of reagent application.

• Use of incorrect amount of reagent.

• Incorrect timing of procedures (e.g., specimen application, running the test, or reading results).

• Incorrect reading of test results.

• Incorrect reading due to color blindness.

Specimen integrity and handling

• Error in specimen collection.

• Use of inappropriate anticoagulant.

• Clotted specimen.

• Error in specimen handling.

• Incorrect specimen transport and/or storage.

• Presence of interfering substances.

• Presence of bubbles in the specimen.

Reagent integrity (Reagent viability)

• Use of improperly stored reagents.

• Use of outdated reagents.

• Use of improperly mixed reagents.

• Use of contaminated reagents.

Hardware, software, and electronics integrity

• Power failure.

• Power fluctuation.

• Incorrect voltage.

• Repeated plugging and unplugging of the device.

• Hardware failure.

• Software failure (see Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices http://www.fda.gov/cdrh/ode/guidance/337.pdf)

• Electronic failure.

• Physical trauma to unit.

Stability of calibration and internal controls

• Factors that affect calibrator and calibration stability, including determination of calibration stability over time and after power failures.

• Factors that may interfere with calibration.

Environmental factors

• Impact of key environmental factors (heat, humidity, barometric pressure changes, altitude (if applicable), sunlight, surface angle, device movement, etc.) on reagents, specimens, and test results.

• Impact of key environmental factors (including electrical or electromagnetic interference) on instruments, if appropriate.

B. Tier 2: Fail-Safe and Failure Alert Mechanisms

1. Points to consider for designing fail-safe and failure alert mechanisms

We recommend that you consider including the items on the following list, as appropriate. You should consider incorporating fail-safe mechanisms when possible.

• Lock-out functions that do not allow output of results if controls or system checks are not successfully completed.

• Lock-out functions that do not allow output if expired reagents are used.

• Lock-out functions that do not allow output of results if the device was mishandled (e.g., dropped) and the device detects damage during internal electronic system checks.

• Physical features to ensure correct placement of components, such as strips or cartridges.

• Monitors of environmental conditions (e.g., indicator desiccants) incorporated into the test system or the kit container to alert the user to environmental conditions that are outside of the recommended storage conditions.

• Battery checks.

• Internal procedural controls to flag procedural problems such as improper sample flow, incorrect use of components, or improper addition of specimen. (However, procedural controls generally provide limited problem detection and, by themselves, are generally not sufficient to serve as a failure alert mechanism.)

• Internal non-procedural controls (e.g., for checking the integrity of the reagents).

• Controls to check that electronic features of the device are within specifications.

• External control material.

When designing controls, you should consider the unique features of the test system and link control procedures to the robustness of the assay, as determined by your flex studies. The controls you devise to mitigate the risks you identify may be based on procedures typical for laboratory-based methodologies (e.g., testing external materials at two levels

at a time interval of once per shift or on each day of testing) or may be a combination of features, such as those listed above, that ensure complete system quality monitoring. When designing packaging for your device, you should also consider that the number of tests per kit should depend on the stability of the reagents or the robustness of your test as demonstrated through testing.

When appropriate, you should incorporate capabilities into the test system software that allow for data retention, identification of outliers, and trend detection in order to alert the user to the occurrence of random or systematic errors.

Procedural controls, which are typically internal, are desirable for waived devices. However, these types of controls generally do not replace external controls especially because they often only control for adequate volume. Your flex studies and validation and/or verification studies should evaluate the sensitivity of internal control reagents to all applicable test system errors. The total quality control (QC) system (including all

control procedures and internal checks) should control for all aspects of test performance, including electronic aspects and integrity of reagents.

We do not recommend training as a sole means of mitigating potential sources of harm. Aspects of the device design that are controlled and maintained by the manufacturer can potentially be considered as mitigations.

2. External control materials

Whenever feasible, you should include external control materials in the test kit. External control materials for waived tests should be ready to use or employ only very simple preparation steps, e.g., breaking a vial in order to mix liquid and dry components of the control material. Reconstitution steps should not require pipetting by the user. For both quantitative and qualitative tests, the levels of the control materials should correspond to the medical decision level(s) relevant to the indications for use for your

test. More than one level may be needed in order to ensure accuracy for quantitative tests. For qualitative tests, you should ensure that control materials include those with concentrations sufficiently close to the cutoff to provide adequate assessment of test performance for patient samples near the clinical cutoff.

You should alert operators about control procedures and the availability of control materials and integrate instructions for external control testing within the test procedure instructions (package insert and Quick Reference Instructions), in order to increase the

likelihood that operators will perform QC correctly. In the test instructions, you should specify minimum frequency for running controls and include recommended levels of control materials that correspond to medical decision levels. The labeling should indicate in bold why external controls are important and the consequences of not performing all QC procedures.

In addition, when control materials are not included in the test kit, you should also recommend, in the package insert and Quick Reference Instructions, the use of specific control material(s) that will ensure optimal verification of the test system performance. Providing or recommending external control materials may not be critical in those limited cases where sufficient fail-safe mechanisms are in place for the entire system. Although we are currently unaware of any such systems, should you develop one, we recommend that you explain, in your waiver application, your rationale for omitting these control materials.

You should describe, in your application, how you established the QC limits and how you demonstrated that the chosen limits provide adequate assessment of test performance

with patient samples. For quantitative tests, you should consider the precision of the test system, as well as the total allowable error for the particular analyte. Ranges that are too broad may be incapable of reliably detecting unacceptable levels of imprecision or bias.

Control materials should mimic performance of patient samples as closely as possible. When the matrix of the material differs from that of the specimen, you should determine and describe in your application how these differences may affect or limit the information provided by the control result. You can accomplish this by testing control materials in parallel with actual patient samples of similar known values and comparing the results of the control material and patient samples with respect to precision or bias observed. You should account for matrix effects when setting the limits for control material to be used with your test.

3. Additional points concerning control materials

If you did not previously submit information addressing the items below in your premarket submission, you should provide them in your waiver application:

• Opened and unopened control material stability data. You should include acceptance criteria and results. The term "unopened" refers to shelf-life stability whereas "opened" refers to reconstituted conditions, or other conditions after the vial is initially opened by the user.

• Lot-to-lot reproducibility, conducted on at least three consecutive lots of control material.

C. Validating Fail-Safe and Failure Alert Mechanisms, including

External Control Procedures

You should conduct studies that validate all fail-safe and failure alert mechanisms (including any procedures you recommend that use external control materials) that address all the

causes of test errors that you identified in your risk analysis. These studies should be conducted under conditions that stress the device in order to demonstrate how fail-safe and failure alert mechanisms respond to such conditions. You should describe your validation and/or verification studies and the results of these studies in your waiver application and indicate how the results support the ability of fail-safe and failure alert mechanisms to detect and mitigate test errors. You should include a description of how your recommendations for external control materials and procedures (including frequency) are supported by your validation and/or verification studies and confirmed by the clinical studies described in Section V, below.

Table 1 - Examples of approaches to flex studies and control validation studies under conditions of stress

POTENTIAL SOURCE OF ERROR	EXAMPLES OF FLEX STUDIES	EXAMPLES OF VALIDATION STUDIES
Operational storage is 2-4° C. What happens when the kit is stored improperly?	Environmental studies included storing the kit at 0°, 2°, 10°, 25°, and 37° C. Studies showed that when frozen, or stored at 25° C for over 3 days, the device failed.	Studies to validate that fail- safe mechanisms, or failure alerts, including external control procedures, alert the operator to frozen conditions or storage at 25° C for more than 3 days.
Procedure is to add 3 drops. What happens when an improper number of drops are added to the test procedure?	Flex studies consist of adding 1, 2, 3, 4, 5, and 6 drops and observing when incorrect results are obtained. Studies show that <2 drops or >5 drops give erroneous results.	Studies to validate that fail- safe mechanisms, or failure alerts, including control procedures, alert the operator of an error when <2 drops or > 5 drops are added.

V. DEMONSTRATING INSIGNIFICANT RISK OF AN ERRONEOUS RESULT – “ACCURACY”

In this guidance document, we use the term “accurate” to refer to those tests that are comparable to tests whose results of measurements are traceable to designated references of higher order, usually national or international standards (see [2 and 3]). To demonstrate that your device is “accurate” in the hands of the intended operator, we recommend that you perform prospective clinical studies of the device proposed for waiver using patient samples⁶ collected in the intended testing environment. In this way, the studies will demonstrate, as closely as possible, how the device performs in the hands of intended operators under the conditions of intended

use.

⁶ Spiked samples may also be appropriate for a portion of the study; See Section V.B.3.a, below.

In this section, we describe the study designs we recommend for a quantitative test and a qualitative test. A quantitative test is a test that gives results expressing a numerical amount or level of an analyte in a specimen; a qualitative test is a test that provides only two responses (i.e., positive/negative or yes/no) (see [4]). If your test does not fit the paradigm of a quantitative, numerical test or a qualitative, two-response test, we recommend that you consult with OIVD.

You should evaluate test performance in a setting designed to replicate, as closely as possible, the actual intended clinical use setting. Therefore, the clinical study design should include the following:

• Intended testing sites.

• Intended operators. We encourage you to enroll operators with the least amount of training that might be encountered at the types of sites for which this device is intended.

• Intended sample type and matrix.

• Testing over time, as in the typical intended use setting.

A. Clinical Study Sites and Participants

1. Testing sites

You should conduct the clinical study to support CLIA waiver at a minimum of three intended use sites at different demographic locations (e.g., outpatient clinic, physician’s office). In your CLIA waiver application, you should present a brief description of each site, including its name, address, and the date the study was performed. If there were sites that were included at the beginning, but then did not complete the study, you should provide a brief explanation for why those sites did not complete the study.

2. Clinical study participants a. Operators

(1) Intended operators

You should ensure that the operator study participants enrolled represent anticipated operators of the device you propose for CLIA waiver. We recommend that you record and tabulate the education (including experience and training) and the occupation of each operator to demonstrate that these participants meet the definition of intended operators and include this in your CLIA waiver application. In addition, for each study site, we ask you to report the same information on other operators that were available at the testing site but that were not chosen to participate.

(2) Instructions for use

You should provide the operators who participate in the study with the labeling and training materials that will be provided with the test kit in the actual intended use settings when the test is marketed. This includes the proposed package insert and/or Quick Reference Instructions (see Section VI.C. Educational Information). These labeling materials should include all control procedures you plan to

recommend to device users when the device is marketed. Study participants should receive no additional instructions (e.g., written or verbal training, coaching, or prompting) beyond the materials included with the test system. Likewise, study participants should have no opportunity to discuss the test with other participants or otherwise coach or observe each other. Study participants

may call an 800 number help-line if such a service is to be provided for the device when it is marketed. You should include, in your waiver application, the instructions you provided to test operators participating in the study.

(3) Universal precautions

Your clinical study must comply with all pertinent laws and regulations including Occupational Health and Safety Administration (OSHA) regulations pertaining to biological hazards ("universal precautions"), 29 CFR 1910.1030.

(4) Operator questionnaire

You should develop an operator questionnaire to be filled out by all test operators participating in the study. This questionnaire should be designed to help assess whether the participants understood how to use the device correctly. It is important that the questionnaire be given to test operators after the completion of the clinical study, so the questions do not bias the participants during the study. Some questions may ask operators to indicate agreement on a 1-5 scale (1=strongly disagree; 5=strongly agree). The following are examples:

• The instructions were easy to follow.

• It was easy to apply the sample correctly.

• It was easy to see and understand the test results (e.g., appearance of the line, change of color).

• The control line was always distinct and easy to read.

• The instructions clearly explain what to do if a test result does not appear or is invalid.

• I needed help from someone the first time I ran the test.

We recommend that, as part of the questionnaire, you show various possible test results and control results that are positive, negative, and invalid and ask the operator to read these results. You may wish to present these questions as true/false or multiple choice questions.

You should also strongly encourage general comments by the study operators. We recommend that you include your survey questions and results with your CLIA application.

b. Subjects (Patients)

You should ensure that subjects from whom you will obtain specimens for the clinical

study meet inclusion and exclusion criteria corresponding to the intended use population of the test. Once a subject has met appropriate inclusion criteria, he/she should be informed of the study and invited to participate in the CLIA waiver study.

You should follow applicable regulations for patient privacy, including informed consent and Health Insurance Portability and Accountability Act (HIPAA) requirements. 45 CFR Part 46; P.L. 104-191. See 21 CFR Parts 50 and 56 for regulations regarding protection of human subjects for studies that support marketing applications.

c. Financial disclosure

If clinical investigators are involved in the clinical study, we ask that you include a Financial Disclosure Statement with your waiver application. For information on financial disclosure statements, we recommend you consult, "Guidance for Industry: Financial Disclosure by Clinical Investigators," http://www.fda.gov/oc/guidance/financialdis.html and 21 CFR Part 54, Financial Disclosure by Clinical Investigators.

3. Clinical study reports

You should report results of the clinical study intended to support CLIA waiver by each intended site and overall, if appropriate. Reports should include the following:

• Protocol description.

• Number of subjects (i.e., patients) studied.

• Procedures for subject inclusion and exclusion.

• Description of the subject population.

• Description of how specimens were collected and stored.

• Masking techniques.

• Discontinuations.

• Complaints, device failures, and replacements.

• Any invalid results and how these were handled.

• Information about QC procedures that were performed by intended users.

• Pertinent tabulations.

• Annotated line listings of results (including electronic versions).

• Clear descriptions and presentations of the statistical analyses.

You should not remove "outliers." You should provide an explanation for data that are incomplete or missing.

B. Clinical Studies for Tests with Quantitative Results

The clinical studies to support CLIA waiver should compare results obtained with the device proposed for CLIA Waiver (Waiver Method, or WM) to results obtained by a Comparative Method (CM). The CM for the clinical studies for quantitative tests should be performed in a laboratory setting by laboratory professionals. The topics outlined in this section, and in Section C are discussed in greater detail in references [2-16]. We recommend that you consult these references.

1. Selection of the Comparative Method (CM)

The following Comparative Methods are presented in the order in which we recommend you select them if available. Thus, we recommend you use a CM of type A (Reference Method (RM)) if it is available. If it is not available, we recommend you select a CM of type B (best available traceable method) and provide adequate justification for selection of this method in your waiver application. If neither a CM of type A nor a CM of type B are available, we strongly recommend that you consult with OIVD to discuss potential comparison with another well-documented method.

• CM of Type A (Reference Method): This is a method that has been thoroughly investigated. It has been shown to yield values having trueness and precision of measurement such that the RM can be used to assess the trueness of other methods for the same quantity or for assigning values to reference materials (for details, see Harmonized Terminology Database at the CLSI website: www.clsi.org).⁷

• CM of Type B (Traceable Method): This is a method traceable to references of higher order. A reference of higher order can be a certified reference material, a reference measurement procedure, or a network of reference laboratories [2]. The traceable method is a method in which the results of measurement can be related to stated references, usually national or international standards, through an unbroken chain of comparisons in which measurement uncertainties have been documented

at every step in the procedure. For details on traceability, see [2] and [3]. You should provide information supporting traceability, including a description of reference materials or methods used to establish the traceability of the calibrators. You should provide the values and uncertainties for each of those reference materials or methods along with any other relevant information concerning traceability (for details, see [3]).

We recommend that you describe the clinical performance of the CM used in your study in your CLIA waiver application and the sources you used for this description (e.g., published literature, labeling). The measurement range of the CM should be at least as wide as the measurement range of the WM.

2. Establishing Allowable Total Error Zones and Zones for Erroneous

Results

You should establish performance criteria for your device before you begin the clinical study in order to objectively evaluate the WM. You should establish criteria for the following zones and demonstrate that your device meets these criteria.

• The Allowable Total Error (ATE) zones.

Values of the WM that fall within the ATE zones are values that can be tolerated

⁷ FDA recommends you consult information on this website as of the date of issuance of this final guidance document.

without invalidating the medical usefulness of the WM. Typically at least 95% of the sample results should fall within the established ATE zone (see Figure 1, below).

The differences between test values obtained with the CM and test values obtained with the WM can be described through total analytical error (for information on total analytical error, see Section 3.b(2), below.)

• The Zones of Limits for Erroneous Results (LER).

Patient results inside the LER zone pose a risk to patient safety. When WM values fall inside the LER zones, potential harm can occur to the patients if these results are utilized in medical decision-making. Therefore, your device should not have WM results in the LER zones.

Usually, boundaries of the ATE and LER are expressed as percentages of CM values for higher values and as a flat limit for low values. A hypothetical example of ATE and LER zones is illustrated below:

Allowable Total Error ( ≥ 95% of samples in study)

Limits for Erroneous Results (0% of samples in study)

Figure 1. Example of ATE and LER zones

Several sources of information can be used to establish criteria for ATE and LER zones for the analyte your device measures. OIVD will review your criteria for ATE and LER and determine whether they are defined appropriately to meet the waiver requirements of CLIA (42 U.S.C. 263a).

For analytes that have existing performance limits for professional use (e.g., those listed in the CLIA regulations (see 42 CFR Part 493 Subpart I, sections 909-959 http://www.phppo.cdc.gov/CLIA/regs/subpart_i.aspx#493.909)), these limits should be

used to define the boundaries of the ATE zone (but the limits should not be expressed as a multiple of standard deviation units). The limits can be used in the following way. Let a value %R be the allowable percent of deviation from the target value described in the CLIA regulations cited above. For example, the value of %R equals 7% for hemoglobin. Once the value of %R is defined, the zone of ATE is bounded by CM ± %R * CM. If, when values of the CM and WM are both low, one can tolerate a larger value of %R * CM without invalidating the medical usefulness of the WM results, then the zone of ATE for these low values can be established as CM ± D, where D is a fixed number. Thus, the ATE zone can be defined as CM ± D for low CM values (i.e., below a defined threshold value) and CM ± %R * CM, for high CM values that exceed the threshold. For example, for the analyte lithium, the ATE zone can be defined as CM ± 0.3 nmol/L if CM < 1.5 nmol/L, and the ATE zone can be defined as CM ± 20% * CM if CM ≥ 1.5 nmol/L.

For analytes not listed in the CLIA regulations 42 CFR Part 493 Subpart I sections 909-

959 (and for those listed but expressed as a multiple of standard deviation units), other criteria may be acceptable, but you should consult with OIVD. For example, the ATE and LER zones could be based on the ATE and LER zones from the same analyte used by professional laboratory operators. Other criteria could be based on medical decision- making, consideration of intra-individual variations (for example, see [4]), needs for accuracy of the samples within the reference intervals, or other approaches as applicable

to the particular analyte You should provide the literature and appropriate documentation to support the ATE and LER you establish. WMs that have a high imprecision or a systematic bias may have a greater challenge meeting acceptance criteria and may not be suitable for CLIA waiver.

We recommend that you include the ATE and LER zones on the scatter plots in your statistical analyses (WM vs. CM).

3. Clinical Study Design

a. Specimen Collection and Sample Preparation

Exact number of samples and operators needed for a CLIA waiver study may vary depending on performance requirements of a particular test. However, FDA recognizes that pre-defining all relevant parameters for performance (i.e. slope, intercept, precision) may be very difficult. We suggest manufacturers consider using the CLSI document on calculation of total error to demonstrate performance at high, middle, and low levels. This would usually include a minimum of 360 patient

samples from the intended use population in the clinical study. The specimens should represent the intended use specimens (i.e., intended use population as well as

specimen matrix) consistent with the clinical practice of the intended use sites. Samples should span the measuring range of the WM. The samples should be obtained at a minimum of 3 independent testing sites that are representative of both the intended patient population and the intended operators. Between 1 and 3 intended operators per site should be selected, and at least 9 operators should participate. For example, you could use 6 sites with 1 or 2 operators per site to reach the minimum of

9 operators. The patient samples should be as equally distributed among the operators as possible. The samples should be collected from consecutive patients

over an appropriate period of time (this period may depend on the specific clinical

site, the prevalence of disease, or other factors). We suggest a 1 month period may be useful and recommend not less than 2 weeks. The goal is to assess how well the device you propose for CLIA waiver works in practice, recognizing that operators

may have other duties. "

You should ensure that the 360 patient samples span the measuring range of the device and adequately represent all possible values of the CM test consistent with the measuring range of the WM (See [6 and 7]). You should divide samples into low, medium and high medically relevant intervals (according to the CM values) with approximately the same number of patients in each. One hundred and twenty observations are recommended within each interval to estimate the total analytical error within each of these intervals (See [6]). A suggestion concerning the distribution of CM values in your data may be found for some analytes in Table 1 of [7].

We believe that actual patient specimens provide the best assessment of a device proposed for CLIA waiver in the hands of intended operators when available. However, in some cases this might be impractical (e.g., due to low prevalence or where some concentration levels may be rare). Samples might be distributed insufficiently to challenge the measuring range, or to generate enough values near medical decision points. In such cases, it may be appropriate to supplement up to 120 actual patient specimens with banked samples. If banked samples are not available, individual spiked or diluted patient samples should be used (spiked, diluted, or otherwise contrived samples used in the study should be individual samples, i.e., they should not be aliquots from a single pool). In some cases, use of otherwise contrived matrix-specific specimens may also be appropriate. The matrix of any of these alternative specimens should be the same as that specified by the intended use of the device. You should describe how you prepared and determined the assigned value for any alternative specimens you used.

Each sample should be split in two parts. If the sample cannot be split into parts, then a second sample from the same patient should be collected within a suitable time interval. (In this case, if the order in which the samples are collected impacts the results of testing, we strongly encourage you to consult OIVD concerning your

clinical study design.) One part (or sample) should be tested by the intended operator as a study participant using the WM, and the other part (or sample) should be tested by a laboratory professional using the CM. Since the calculations of total analytical

error of the WM include the imprecision of the CM as part of the total analytical error of the WM, it is recommended that you obtain replicate measurements with the CM if the random error (expressed as a standard deviation or a coefficient of variation (CV)) of the CM is more than one third of the random error of the WM (see [5]). You

should then replace individual CM results with the average value of the CM in all subsequent calculations.

b. Statistical Analyses

You should calculate the Total Analytical Error of the WM (based on analyses of differences) (see [6]) and compare the WM results with the CM results by an appropriate regression analysis (see [7]).

(1) Descriptive statistics

You should provide the following information based on each site separately, as well as a combined analysis over all sites:

• Descriptive statistics for both the WM and CM results, including the number of results, mean, standard deviation (SD), minimum, median, and maximum, and side by side box-and-whiskers plots for both the WM and CM values (see [8]).

• Scatter plot of the results, where the WM results are on the Y-axis and the CM results are on the X-axis. Both axes should have the same scale, and the line of identity (y=x) should be presented. The same scale on the axes should be applied to the data from each site.

(2) Total Analytical Error

For purposes of this guidance, we use the term "total analytical error" as an interval that contains a specified proportion (specifically 95%), of the distribution of differences between the WM and CM values. Total analytical error may also be expressed in terms of relative differences. Relative differences are defined as:

the WM result minus the CM result, divided by the CM result. (For details on the total analytical error, see [6].) We recommend that you provide the following information (combined over all sites) concerning total analytical error:

• For every sample tested by the WM and CM, you should calculate a difference (WM result minus CM result). In a similar way, for every sample, you should also calculate a relative difference: WM result minus CM result, divided by CM result.

• You should provide a difference plot with the CM values on the X-axis and differences between WM and CM values on the Y-axis. You should also provide another plot with the CM values on the X-axis and relative differences on the Y-axis. (For details see [6].)

• You should divide the measuring range into three medically relevant intervals (low, medium, and high), as described earlier, in which each interval contains approximately the same number of points (about 120). (For details, see [6 and 7].) These ranges will be used for further analyses (see bullet, below).

• For each interval (low, medium, and high), you should calculate total analytical errors based on analyses of differences or relative differences (whichever is appropriate) for 95% of the differences. Usually,

calculation of relative differences is more appropriate for intervals containing high values and calculation of differences is more appropriate for the intervals with low values. You should also provide a histogram of the relative differences (percentages) or actual differences for each interval. You should calculate both the mean and standard deviation of these differences. You should identify the 2.5th and 97.5th percentiles derived from the calculation of the total error for 95% of the differences.

(3) Regression Analysis

• You should provide results of an appropriate regression analysis for each site separately as well as a combined analysis over all sites. The regression method you choose should account for the random errors associated with the WM and CM. You should present results for the combined data as well as separately for each site. A Deming regression

procedure or one of several similar methods may be appropriate (see [7, 9, and 10]). Since the methods may vary in their assumptions, you should include some justification of the choice of procedure in the application. If the random error of the CM is negligible compared to the random error of the WM, and the standard deviation of the random error of the WM is roughly constant over the range of CM values, then ordinary least-square regression analysis is also appropriate (see [7]).

• You should provide the 95% confidence intervals of the slope and intercept from the regression above. We recommend that appropriate data on slope and intercept be provided for each site, and as combined data. You should calculate the systematic bias at medically important points. (For details, see Section 6.1 in [7].)

• Alternative statistical approaches are welcome. We recommend they be discussed with FDA prior to initiating a study

Please note that correlation alone is often inappropriately used to assess agreement between the WM and CM, and we caution you against such an approach.

If one of the medically important points of the WM in the hands of intended operators includes the Limit of Blank and/or the Limit of Detection, then some additional calculations for the samples with very low level of analyte may be needed for appropriate comparison of the Limits of Blank and Limits of Detection of the WM in the hands of intended operators to the CM (see [11]).

(4) Performance Criteria for Quantitative Tests

For the zone of the ATE, you should report:

• The percentage of the observations that fall within the ATE zone for the low, medium, and high ranges of the CM. These percentages should approach 95% for each range of the CM.

• The percentage of the observations over the entire measuring range that fall within the ATE zone, with a lower one-sided 95% confidence bound. This lower bound should exceed 92%. This is because, for a sample consisting of 360 observations with 95% of observations (342 out of 360) falling in the ATE zone, the lower one-sided 95% score confidence bound is above 92%. For details about confidence intervals, see [12-15] and Statistical Notes in Appendix A.

For the zone of the LER, you should report:

• The percentage of the observations that fall within the zone for the low, medium, and high ranges of the CM.

• The percentage of the observations over the entire measuring range that

fall within the zone, which has an upper one-sided 95% confidence bound. This upper bound should be below 1%. This is because, for a sample consisting of 360 observations and zero observations (0 out of 360) falling in the LER zone, the upper one-sided 95% confidence bound is below 1%.

C. Clinical Studies for Tests with Qualitative Results

There are two parts to the clinical study design for qualitative data. The first part (discussed in Section C2a, below) is a method comparison and is similar to the study design discussed above for quantitative results. The objective in this part is to compare the CM and WM in terms of performance. The second part (discussed in Section C2b, below) is designed to identify problems that may occur when the analyte is near the cutoff region of the CM, where a cutoff is defined as the threshold at which the device differentiates a positive from a negative outcome. As in the case of a test with a quantitative outcome, the CM for the CLIA waiver clinical studies for qualitative tests should be performed in a laboratory setting by a laboratory professional.

1. Selection of the Comparative Method (CM)

The following CMs are presented in the order in which we recommend you select them if available. Thus, we recommend you use a CM of type A if available. If a CM of type A is not available, then we recommend that you use a CM of type B. If neither a CM of

type A, nor a CM of type B is available, then we recommend you use the best available traceable method (CM of type C) and, similarly, for CMs of types D and type E. If you choose a comparative method other than type A or B, you should provide justification for this in your application. If you are considering use of a CM of types C, D, E, or another well-documented method, we strongly recommend that you contact OIVD prior to conducting the clinical study in support of a CLIA Waiver application.

• CM of type A: A quantitative reference method such as those outlined in the section on quantitative tests with the appropriate cutoff value for the positive and negative results.

• CM of type B: A qualitative reference method (for examples of qualitative reference methods, consult with OIVD).

• CM of type C: A quantitative method with measurement results traceable to higher-order references such as that outlined in the section on quantitative tests with the appropriate cutoff value for the positive and negative results.

• CM of type D: A qualitative method with measurement results traceable to higher-order references.

• CM of type E: A qualitative method that was tested by reference specimen panels (e.g., panels of samples prepared by well recognized institutions, such as WHO, CDC, NIST). We recommend that you contact OIVD for input concerning your choice of specimen panels.

In any case, you should provide a description of the specific CM, including 1) intended use population/target population (those individuals for whom the test (CM) is intended),

2) condition of interest/target condition (a particular disease, a disease stage, health status, or any other identifiable condition or characteristic of interest within a subject), 3) reference standard (gold standard) for establishing the presence or absence of the condition or characteristic of interest, and 4) diagnostic accuracy of the test (as diagnostic sensitivity and specificity, positive and negative predictive values with prevalence of condition of interest) (for details, see [5]) with sources for this information (published literature, labeling and others).

For the CMs of type C and D, you should provide information supporting traceability, e.g., a description of references that were used to establish the traceability of the calibrators with values and uncertainties on each of those references and other relevant information concerning traceability (for details, see [3]).

2. Clinical Study Design

a. General Approach for Method Comparison

As in the study for tests with quantitative results, the samples should be obtained at a minimum of 3 intended use sites that are representative of both the intended patient population and the intended operators of the WM. Between 1 and 3 intended operators at each site should be selected. However, a minimum of 9 operators should participate. Thus, you could use 6 sites with 1 or 2 operators per site to reach the minimum of 9 operators. The patient samples should be approximately equally distributed among the operators. The samples should be collected from consecutive patients over an appropriate period of time. (This period may depend on the specific clinical site, the prevalence of disease, or other factors.) The goal is to assess how well the device proposed for CLIA waiver works in practice, recognizing that operators may have other duties. A one month period for the clinical study to support CLIA waiver may be useful, especially when the prevalence rate is low, and we recommend not less than 2 weeks. The goal should be to have a minimum of 120 samples positive by CM and a minimum of 120 samples negative by CM. However, the overall sample size may well exceed 240 since one cannot anticipate the CM outcome for each patient’s data at the time of sample collection. Each operator

should observe a minimum of 5 positive and 5 negative samples. However, we stress that the samples should be masked with respect to the CM outcomes. If some sites have low prevalence rates, or if the overall availability of positive samples for novel

or rare diseases is low, different study designs and corresponding statistical analyses can be considered. We recommend that you contact OIVD for feedback prior to conducting a study, especially if it is based on an alternative design.

Tests for low prevalence diseases can present challenges in terms of obtaining at least

120 samples positive by the CM. Some approaches to solving these challenges include expanding the duration of the CLIA waiver study, or including a site with higher disease prevalence among the study subjects. In some cases, the seasonal variations in the prevalence of disease should be taken into account for the study design. We believe that actual patient specimens provide the best assessment of a device in the hands of intended operators. However, in some situations, such as when a disease has low prevalence, you may supplement prospective patient samples with neat (unaltered), banked samples. If neat, banked patient samples are not available, it may be acceptable to supplement the unaltered patient samples with alternative samples such as individual spiked or diluted patient samples. In some cases, other types of contrived samples may also be applicable. (Spiked, diluted, or otherwise contrived samples used in the study should be individual samples, i.e., they should

not be aliquots from a single pool). In any of these cases, the sample matrix should

be the same as that of the intended use patient samples. No more than one third of the positive and no more than one third of the negative samples (when positive and negative are determined by the CM) should be banked or artificially contrived. In most cases, banked or contrived samples will be used when needed to ensure that samples are distributed across the assay range. Therefore, in most cases, these should include near-cutoff samples. In your CLIA waiver application, you should describe the preparation and origin of these samples and justify the need for using them.

Each sample should be split in two parts: one part should be tested by the intended operator, employing the WM; the other part should be tested by a laboratory professional employing the CM. This is generally feasible if there is sufficient sample available that can be split for comparisons between the two methods. If the

sample cannot be split into parts, then a second sample from the same patient within a suitable time interval should be obtained. If the order in which the samples are collected impacts the results of testing, we encourage you to consult OIVD

concerning your study design.

Statistical analysis of method comparison results for qualitative tests:

For the positive and negative samples, as determined by CM, you should calculate positive and negative percent agreement estimates along with lower two-sided 95% confidence bounds for each. You should perform calculations for each site individually and, if appropriate, for all sites combined. (See [12-15] and Statistical Notes concerning analysis of percentages and the formulas for the score confidence bounds in Appendix A.)

b. Determining Device Performance with Analyte Concentrations

Near the Cutoff

We recommend the following study design for determining performance with samples containing analyte concentrations near the cutoff. This study design and the principles behind it are described in detail in User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline. CLSI document EP12-A [5].

The objective of this study should be to demonstrate that the interpretation of positive and negative (or presence or absence) for the analyte is essentially the same for the WM and the CM near the cutoff. In order to determine the WM performance with analyte concentrations near the cutoff, you should prepare two types of sample material: one should correspond to a sample that is a weak positive using the CM and the other should correspond to a sample that is a weak negative using the CM.

(1) Preparation of Weak Positive and Weak Negative Samples (as measured by the CM)

In concept, for a theoretically large number of the individual tests, a sample with an analyte concentration at the cutoff value of the CM produces results above the cutoff 50% of the time (and results below the cutoff 50% of the time). It follows that for a given number of tests performed on a sample with a concentration larger than the cutoff concentration, positive results would be expected more than 50%

of the time. In this guidance we refer to an analyte concentration at which repeated tests are 95% positive, as a weak positive concentration (or “C₉₅” concentration). We refer to a sample concentration below the cutoff at which repeated tests are 95% negative (or 5% positive) as a weak negative concentration (or “C₅” concentration). Note that if the limit of blank (LoB) is used as a cutoff, then the concentration C₉₅ is a limit of detection (LoD) (see [11]).

If the CM is a quantitative method with the appropriate cutoff value, then the weak positive (C₉₅) and weak negative (C₅) concentrations can be evaluated from precision studies of the CM for the concentrations near the cutoff (see Statistical Notes in Appendix A). If the CM is a qualitative method (for example, it requires professional operator interpretation(s) in detecting the presence or absence of

color in a solution or on a test strip), then the weak positive and weak negative concentrations of CM can be estimated through dilution studies. For these dilution studies, you should prepare a series of samples (patient samples or sample pools) with concentrations slightly greater than and slightly less than the cutoff. Additional samples that range from strongly positive to samples that do not have analyte present should also be included. You should prepare multiple samples at each concentration in the cutoff series and ensure that the samples are masked to the professional operators who will participate in the study. The professional operators of the CM should perform the test according to the instructions for use. The intent is to find analyte concentrations where the indicator (a colored line or solution) is observed 95% (weak positive) or 5% (weak negative) of the time by the professional operators of the CM. We

recommend that you plot the data as indicated in the graph below: Percent of Positive Results vs. Concentration by the CM (or Dilution Factors) and find the concentrations (or dilution factors) which correspond to 95% and 5% of the positive results by model fitting. We recommend that you contact OIVD prior to conducting the dilution studies for determination of weak positive (C₉₅) and weak negative (C₅) concentrations as measured by the CM if there are concerns about accepted dilution factors.

_Cutoff

Weak Negative

(C₅)

Weak Positive

(C₉₅)

Figure 2. Example of Percent Positive Results by the CM for the Concentration

Near the Cutoff

(2) Performance of the WM with Weak Positive and Weak Negative

Concentrations

For each sample type, you may need to pool patient samples or dilute a specimen in order to create a sufficient amount of material for 60 aliquots. The samples used should mimic the type of samples intended for use with the WM as closely

as is feasible. In your waiver application, you should provide information on how you prepared each of these samples. You should select 3 intended use sites for

this part of the study.

• Prepare 60 aliquots of one sample with the weak positive concentration, defined to be a concentration at which the professional operators of the CM obtained positive results 95%-99% of the time.

• Prepare 60 aliquots of a weak negative sample, defined to be a concentration at which the professional operators of the CM obtained correct results 95-99% of the time. For some tests, when the cutoff value is set in such a way that only samples with no analyte produce negative results 95%-99% of the time (the cutoff of the CM is the limit of blank), the prepared aliquots should not have any analyte.

• These 60 weak positive aliquots and 60 weak negative aliquots should be approximately equally distributed across all operators with 20 samples of each type at each of the three participating sites. You should ensure that the labels on the aliquots are masked as to their true designation, and that each aliquot has a unique label. You should also ensure that samples are tested in random order during the study.

_{(3) Statistical Analysis for Results of Analyte Concentrations near the Cutoff:}

• For each of the two types of aliquots (weak positive and weak negative), you should calculate the percent of positive results for the weak positive sample and the percent of negative results for the weak negative sample with the two-sided 95% confidence intervals for both. In the waiver application, you should provide these results for each site and overall.

• We recommend that you compare the percents of positive results for the weak positive sample among the three sites (20 measurements per site) by using a Fisher-Freeman-Halton test (a generalization of Fisher’s exact test, see [16]). You should conduct a similar test for the weak negative results.

3. Performance Criteria for Qualitative Tests

For the first parts of the study design, described in Section C2a, above:

The observed positive and negative percent agreements between the test proposed for waiver and the CM should be 95% or greater; but in all cases the score lower confidence bound for the two-sided 95% lower confidence bound should be equal to or greater than 89% (see [12-15] and Statistical notes in Appendix A).

In some cases, a higher percent agreement and a higher value for the lower confidence bound may be needed to reasonably assure that the WM is “accurate.” In some cases (for example, when the CM is a reference method for the analyte), a lower percent agreement and a lower value for the lower confidence bound may be medically acceptable with sufficient risk/benefit justification.

We also recommend that you report any invalid results obtained. This can be summarized using a table or graph that captures the number of times a sample and/or a specimen needed to be tested in order to obtain a valid result.

For the second parts of the study design, described in Section C2b, above:

The percent of positive results for the 60 aliquots of the prepared, weak positive samples should be close to 95%. That is, approximately 57 out of 60 of these samples should yield positive results. The percent negative results for the prepared weak negative samples should also be approximately 95%. Any differences found between the CM and WM performances near the cutoff should be investigated to determine what percentage of the intended use population would be affected by this difference. The differences in percent of positive results among the three sites for the weak positive sample and differences in percent of negative results among the three sites

for the weak negative sample should not be clinically or statistically significant at

α=0.05.

VI. LABELING FOR WAIVED DEVICES

In order to ensure that the device is "simple" and has "an insignificant risk of an erroneous result," your labeling should contain instructions for accurately running the test and reporting results and be written at a level appropriate for the intended operator, which for a "simple" test should be at a 7^th grade reading level or lower. It may be helpful for you to refer to the following documents. They are available on the Internet as shown:

• "Write it Right," http://www.fda.gov/cdrh/dsma/897.pdf

• "Medical Device Use-Safety: Incorporating Human Factors Engineering into Risk

Management," http://www.fda.gov/cdrh/humfac/1497.html

• "Guidance on Medical Device Patient Labeling, Final Guidance for Industry and FDA Reviewers" http://www.fda.gov/cdrh/ohip/guidance/1128.html

You should include your proposed labeling, including Quick Reference Instructions, package insert, and outer labels in your waiver application. (Note that labeling for in vitro diagnostic devices must meet all applicable labeling requirements as stated in 21 CFR 809.10(b).)

In addition, the package insert for waived test systems should include additional information appropriate for untrained operators, including⁸ :

• Facilities performing testing must have a CLIA Certificate of Waiver. 42 USC

263a(c)(2). Also, the labeling should identify your test system as waived and note that all applicable state and local laws must be met.

• A statement that laboratories eligible for a Certificate of Waiver must follow the test system instructions, including use with only the waived specimen type(s), instructions for limitations/intended use, and performance of QC testing as a failure-alert mechanism. (42

CFR 493.15(e).) You should state that any modification to the test or the manufacturer’s instructions will result in the test being classified as high complexity.

⁸ See also Appendix B for more detailed recommendations.

• Instructions for QC integrated with procedural instructions for performing the test in both the package insert and the Quick Reference Instructions.

• Results of clinical studies that supported test waiver (in consultation with FDA). The performance information in your labeling can be finalized after study results are reviewed and the test is determined to be waived by FDA.

A. Quick Reference Instructions (QRI)

You should include Quick Reference Instructions (QRI) in your application. Preferably, the QRI should be laminated and attached to the test system. These instructions should be clear, easy to understand, in a readable font of 12 or greater, and, when possible, include pictures. You should write instructions at no higher than a 7^th grade reading level. For recommendations on what to include in the QRI, see Appendix B. You should include all the items in that appendix that are applicable to your test system as well as any other appropriate information specific to your test system.

B. Quality Control (QC) Labeling Recommendations

Instructions should clearly explain why QC is needed and emphasize the value of external control testing at regular intervals for ensuring operator competency, and reagent and instrument (when appropriate) integrity. Instructions on how to perform control procedures using external controls are always recommended and are critical if you are using them as a failure alert to help ensure “insignificant risk of an erroneous result.” Instructions relating to procedures used for QC should be integrated within the instructions for performing the test and should include the following:

• Step by step information on how to test control material, including testing frequency and concentration of materials.

• How to read results of control procedures.

• How to determine if results are invalid (for example, for tests with an internal procedural control line, the test is not valid if the line is not present).

• Actions to take when control results are out of range or invalid. For example, the directions should direct the user to call technical assistance when control results are out of range or invalid and state that the results should not be reported.

• The limitations on all control mechanisms, including procedural controls, which you identified during the risk assessment. For example if your procedural controls only test that a liquid was applied, this limitation needs to be communicated to the user.

Your explanations of QC systems should include a description of what is being measured by all elements of both internal and external quality controls for a particular test system. To aid in addressing QC problems, you should provide a toll-free telephone number for technical assistance. We recommend that QC instructions take into account information obtained during the clinical studies (Section V), as well as results of flex studies and validation and/or verification studies under conditions of stress (Section IV).

You should include discussions of benefits and limitations of the various device controls. For example, for a unitized test, the following may be appropriate and could be indicated in bold in the labeling for emphasis:

When you run test (xyz), you should always see an extra line (the control line) in addition to the test line. This extra line lets you know that you added the correct sample volume. Good laboratory practice recommends that you also use additional positive and negative control materials that are not built in to the test. (They are external controls.) You can order external controls from (abc). External controls can monitor test features such as whether test reagents are working properly or whether the test was performed correctly. If any of the controls do not perform as expected, do not report patient results. Review the instructions to see if the test was performed correctly, and then repeat the test. If the controls still do not give the expected results, contact technical assistance before testing patient samples.

Examples of possible minimum frequency recommendations for running external controls are listed below. You should base your specific recommendations on data from your studies.

• Each new lot.

• Each new shipment of materials even if it is the same lot previously received.

• Each new operator (i.e., operator who has not performed the test recently).

• Monthly, as a check on continued storage conditions.

• When problems (storage, operator, instrument, or other) are suspected or identified.

• If otherwise required by your laboratory’s standard QC procedures.

C. Educational Information

As part of an overall plan to ensure that the likelihood of erroneous results by the user is negligible, manufacturers should ensure that laboratories can read and understand the labeling, including test performance and limitations (as reflected by the analytical and clinical studies). We encourage you to consider innovative mechanisms to provide technical assistance to laboratories and to ensure they understand the directions in the labeling, e.g., a downloadable version of the test procedure with computer animation showing the correct steps for performing the test. We also recommend that manufacturers assist laboratories performing waived tests in becoming better educated on proper laboratory techniques. For example, we recommend that you participate in the development and promotion of good laboratory practice guidelines by developing training and education programs for the end operator. We also encourage you to incorporate proficiency testing, when feasible, and to promote laboratory participation in proficiency programs. In addition, we recommend that you include good laboratory practice information in the package insert, in accessory

educational or technical material, and through the development of formal educational training programs. We recommend that you provide information on the following topics to operators:

• Importance of retaining a current version of the package insert with the latest revision date.

• Importance of following the test instructions in the sequence given in the instructions.

• Need for proper operator training and retraining in order to maintain competency.

• Need for users to follow all instructions related to storage, preparation, and expiration dating in order to maintain adequate test performance.

• Importance of documenting results and maintaining records as needed for proper performance of the test and patient management.

• General purpose of quality control and value of using quality control within a broader system of quality assurance.

• Common errors – errors that are likely to occur. For example, incorrect timing that may be mitigated by use of a mechanical timer with an alarm.

We recommend that you consult CDC documents as a good source of information regarding good laboratory practices for waived testing sites, survey findings from testing sites holding a Certificate of Waiver under the Clinical Laboratory Improvement Amendments of 1988, and Recommendations for Promoting Quality Testing. (see [17])

VII. SAFEGUARDS FOR WAIVED TESTS

1. FDA is also recommending that manufacturers of waived tests put a brief description of the MedWatch medical products reporting program along with the MedWatch phone number (1-800-FDA-1088), and fax numbers (1-800-FDA-0178), and website (www.fda.gov/medwatch) in the package insert. You may also describe how the MedWatch program works, which failures should be reported to both the company and FDA, and when failures should be reported to ensure proper tracking and reporting of waived testing issues.

2. Manufacturers of devices must maintain and implement medical device reporting procedures as required by 21 CFR 803.17 and must establish and maintain medical

device report (MDR) event files as required by 21 CFR 803.18. See also 21 CFR 803.20,

21 CFR Part 803, Subpart E.

3. Manufacturers of devices must submit MDRs of individual adverse events as required by

21 CFR 803.10(c).

VIII. Paperwork Reduction Act of 1995

This guidance contains information collection provisions that are subject to review by the

Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995 (44

U.S.C. 3501-3520).

The time required to complete this information collection is estimated to average 4,000 hours per response, including the time to review instructions, search existing data sources, gather the data needed, and complete and review the information collection. Send comments regarding this burden estimate or suggestions for reducing this burden to:

Department of Health and Human Services

Food and Drug Administration

Office of Chief Information Officer

Paperwork Reduction Act (PRA) Staff

[email protected]

This guidance also refers to previously approved collections of information found in FDA regulations. The collections of information in 21 CFR part 803 have been approved under OMB control number 0910-0437; and the collections of information in 21 CFR part 809 have been approved under OMB control number 0910-0485.

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this information collection is 0910-0598 (expires XX/XX/XXXX).

REFERENCES

1. CLSI (former NCCLS). Quality Management for Unit-Use Testing; Approved Guideline.

CLSI document EP18-A (ISBN 1-56238-481-3). CLSI, 940 West Valley Road, Suite

1400, Wayne, Pennsylvania 19087-1898 USA, 2002.

2. ISO 17511: 2003 (E) In vitro diagnostic medical devices – measurement of quantities in biological samples – metrological traceability of values assigned to calibrators and control materials.

3. CLSI (former NCCLS). Metrological Traceability and Its Implementation; A Report.

CLSI document X5-R [ISBN 1-56238-598-4]. CLSI, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2006.

4. Fraser,C.G. Biological Variation: from Principles to Practice, AACC Press, 2001

5. CLSI (former NCCLS). User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline. CLSI document EP12-A [ISBN 1-56238-468-6]. CLSI, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2002.

6. CLSI (former NCCLS). Estimation of Total Analytical Error for Clinical Laboratory

Methods; Approved Guideline. CLSI document EP21-A (ISBN 1-56238-502-X). CLSI,

940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2003.

7. CLSI (former NCCLS). Method Comparison and Bias Estimation Using Patient Samples; Approved Guideline – Second Edition. CLSI document EP9-A2 (ISBN 1-56238-000-0). CLSI, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2000.

8. Fisher LD, Van Belle G. Biostatistics. A Methodology for the Health Sciences 1993, John

Wiley & Sons, Inc.

9. Linnet K. Evaluation of the linear relationship between the measurements of two methods with proportional errors. Statistics in Medicine 1990; 9:1463-1473.

10. Martin RL. General Deming Regression for Estimating Systematic Bias and Its

Confidence Intervals in Method-Comparison Studies. Clinical Chemistry 2000;

46(1):100-104.

11. CLSI (former NCCLS). Protocols for Determination of Limits of Detection and Limits of Quantitation; Approved Guideline. CLSI document EP17-A (ISBN 1-56238-551-8). CLSI, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2004.

12. Clopper CJ, Pearson E. The use of confidence or fiducial limits illustrated in the case of

_{Binomial. Biometrika 1934;26:404-413.}

13. Altman D.A., Machin D., Bryant T.N., Gardner M.J. eds. Statistics with Confidence. 2^nd

ed. British Medical Journal; 2000.

14. Agresti, A. and Coull, B.A. Approximate is better than “exact” for interval estimation of binomial proportions. The American Statistician 1988;52:119-126.

15. Newcombe, R.G. Two-sided confidence intervals for the single proportion: comparison of seven methods. Statistics in Medicine 1998; 17: 857-872.

16. Agresti, A. Categorical Data Analysis. 1990, John Wiley & Sons, Inc.

17. Centers for Disease Control and Prevention. Good laboratory practices for waived testing sites; survey findings from testing sites holding a Certificate of Waiver under the Clinical Laboratory Improvement Amendments of 1988 and Recommendations for Promoting Quality Testing. MMWR 2005; 54 (No.RR-13):1-25.

Appendix A: Statistical Notes

1) The following are additional recommendations for performing statistical analyses of percentages or proportions.

The 95% confidence interval for positive percent agreement and 95% confidence interval for negative percent agreement yield a rectangular region that contains both population parameters with confidence level 90% (≈ 0.95 x 0.95). A 95% confidence region for both population positive percent agreement and negative percent agreement is

constructed from 97.5% (=√0.95) univariate confidence intervals.

Confidence limits for positive percent agreement and negative percent agreement can be calculated using formulas for calculating a confidence interval for a binomial proportion. There are several different methods available, and they can be obtained from many software packages or can be obtained from published tables. We suggest that either a score method described by Altman, et al. (see [13]) and Agresti and Coull (see [14]) or a Clopper-Pearson Method (see [12]) be used. An advantage with the score method is that it has better statistical properties (see [15]) and it can be calculated directly. Score confidence bounds tend to yield narrower confidence intervals than Clopper-Pearson confidence intervals, resulting in a larger lower confidence bound. So with n=120 samples and 114/120=95% agreement, the score lower confidence bound is 89.5% whereas the Clopper-Pearson lower confidence bound is 89.4%. In this document we have illustrated the reporting of confidence intervals using the score approach. For convenience we provide the formulas for the score confidence interval for a percentage

here. Note that the lower bound of a two-sided 95% score confidence interval is the same as the lower bound of a one-sided 97.5% score confidence interval.

A 2-sided 95% score confidence interval for positive percent agreement or negative percent agreement is calculated as:

[100 % (Q₁ − Q₂ ) / Q₃ , 100 % (Q₁ + Q₂ ) / Q₃ ] ,

where the quantities Q₁, Q₂, and Q₃ are computed from the data using the formulas below.

_{For positive percent agreement:}

1

Q = 2 A + 1.96² = 2 A + 3.84

^Q₂ ^{= 1.96}

1.96² + 4 AC /( A + C ) = 1.96

3.84 + 4 AC /( A + C )

3

Q = 2( A + C + 1.96² ) = 2( A + C ) + 7.68

where A is the number of samples that are positive using both the WM and the CM and C

is the number of samples that are negative via the WM but positive via the CM.

_{For negative percent agreement, similar equations are needed:}

1

Q = 2D + 1.96 ² = 2 D + 3.84

^Q₂ ^{= 1.96}

1.96 ² + 4 BD /( B + D ) = 1.96

3.84 + 4 BD /( B + D )

3

Q = 2( B + D + 1.96 ² ) = 2( B + D ) + 7.68

D is the number of samples that are negative via the CM and WM whereas B is the number of samples that are negative via the CM and positive via the WM.

In the formulas above, 1.96 is the 97.5^th quantile from the standard normal distribution

that corresponds to 95% two-sided confidence. The 97.5% lower confidence bound is the smaller of the two values obtained using the 95% two-sided confidence interval formulas above.

In Section C2 we refer to calculation of a one-sided 95% lower confidence bound for the percent of observations that fall within the ATE or LER. In that instance, one should use

1.645 in place of 1.96 in the formulas above.

2) Evaluation of C₅ and C₉₅ using the CM from precision studies:

If the standard deviations (SD) in the precision studies of the CM for concentrations

around the cutoff value are almost constant, then

C₉₅ = cutoff +1.645 x SD and

^C₅ ^{= cutoff – 1.645 x SD.}

For example, if the cutoff value =1.00 and the SD around the cutoff is approximately

0.10, then C₉₅ is approximately 1.16 and C₅ is approximately 0.84. In other words, a sample with actual concentration of 1.16 produces positive results (above 1.00) approximately 95% of the time and a sample with actual concentration of 0.84 produces negative results (below 1.00) approximately 95% of the time.

If the coefficient of variation (CV) in the precision studies of the CM for concentrations around the cutoff value are almost constant, then C₉₅ = cutoff + 1.645 x CV x C₉₅ and C₅

^{= cutoff – 1.645 x CV x C}₅^{. From here,}

C₉₅ = cutoff / (1 – 1.645 x CV) and

^C₅ ^{= cutoff / (1 + 1.645 x CV).}

For example, if the cutoff value =1.00 and %CV around the cutoff is approximately 10% (i.e., CV=0.10), then C₉₅ is approximately 1.20 and C₅ is approximately 0.86.

Appendix B: Labeling

_{SPECIFIC LABELING RECOMMENDATIONS FOR WAIVED DEVICES}

QUICK REFERENCE INSTRUCTIONS (QRI) WITH PICTURES AND DIAGRAMS should generally contain the following and any other sections appropriate for your specific device. The QRI should be written at no higher than a 7^th grade reading level:

The name of the test and a statement that labs with a Certificate of Waiver may use it.

A statement clearly listing the specimen type, e.g., this test is only waived for urine specimens.

A statement that users should read the complete test procedure, including recommended QC

procedures, before performing the test and that they should refer to the package insert for more complete information. If appropriate, a statement that users should perform control procedures before performing the test.

A statement that laboratories with a Certificate of Waiver must follow the manufacturer's

instructions for performing the test. 42 CFR 493.15(e)(1)

Step-by-step test instructions. Include, as appropriate: physical environmental

specifications/conditions for test performance; specifications for specimen collection, handling, storage, and preservation; preparation of reagents and control materials; storage of reagents and control materials; and calibration procedures. Utilize diagrams and flowcharts to illustrate how to run the test when helpful.

Step-by-step instructions for all control procedures, including frequencies, action to be taken if

control results are out of range or invalid or if other failure alert or fail safe mechanisms are activated.

Interpretation of results, including diagrams on how to read and assess validity of test results and

control results.

A warning addressing color blindness when waived tests use color-coded reagents and/or

endpoints.

Safety considerations for test operation that particularly apply to untrained users.

Critical maintenance, such as cleaning (including safety considerations).

Telephone number to contact manufacturer for technical assistance or troubleshooting the test system. Direct the user to call for assistance when the device or the control materials do not

work as specified by the manufacturer.

PACKAGE INSERT - Considerations for waived tests (in addition to any other requirements specified in 21 CFR 809.10 and any other considerations specific for your

device type)

Identification of the test as CLIA waived, a statement that a Certificate of Waiver is required to perform the test in a waived setting, and information on how users can obtain a certificate.

A statement that laboratories with a Certificate of Waiver must follow the manufacturer's

instructions for performing the test. 42 CFR 493.15(e)(1).

Test operation safety considerations that particularly apply to untrained users.

The physical environmental specifications/conditions for performing the test.

A warning addressing color blindness when waived tests use color-coded reagents and/or

endpoints.

Step-by-step operating instructions for performing the test that are integrated with instructions

for all control procedures.

Action to be taken when no test result is obtained or when the result is out of the reportable

range.

Study results demonstrating how the test compares to a known method that is traceable to a

reference method if applicable.

A brief description and summary of the results from the waiver studies.

When appropriate, warnings about clinical errors that can occur even when the test result is

analytically correct.

Instructions indicating when and how additional testing should be done (e.g., in cases where

results should be confirmed by a reference procedure performed by an appropriately certified laboratory).

Any other limitations, restrictions, and special considerations for your test system.

Appropriate QC recommendations or requirements (see below, “Quality Control Labeling

Recommendations”).

Information on reporting test system problems to the manufacturer and/or FDA. You should

include statements encouraging users to contact you and/or FDA so that you can track and account for device problems. (www.fda.gov/medwatch)

Manufacturer contact information (phone number and the party to contact with a valid email

address if available).

Quality Control Labeling Recommendations

Step by step information on how to test control materials.

Frequency for testing control materials.

How to read control results and procedural controls.

Actions to take when control results are out of range or invalid.

Limitations of the device’s controls that were identified during the risk assessment.

_{Appendix C: Definition of Terms as Used in this Document} ⁹

“Accurate”	In this guidance document we use the term “accurate” tests (in quotes) to refer to those tests that are comparable to a traceable method, for which the results of measurements can be related to stated references. We write the term “accurate” in quotes to denote that it is used in the context of CLIA and this guidance. The definition of accuracy may be different in other contexts, including those provided in other scientific standards. (For example, see CLSI Harmonized Terminology Database).
Allowable total error (ATE)	In this document, the allowable total error is the limit for the differences between the WM and the CM or the WM and the RM that can be tolerated without invalidating the medical usefulness of the test. These differences can be expressed as percents of the CM values for high values and as a concentration difference for low values. Typically at least 95% of the observed differences should fall within the established ATE zone.
Control material	Material used to verify performance characteristics of a medical device.
Control measure	Protective measure implemented for reducing risks. (Fail-safe mechanisms and failure alerts are control measures.)
Control procedures	Operational techniques and activities at the point of use to monitor the performance of the device and fulfill the laboratory’s requirements for quality. Any single control procedure might monitor all or part of the measurement procedure, ranging from the collection of sample to reporting the result of the measurement.
External control material	Control material that is not built into the device. Typically this is in a similar matrix as the intended use specimen and is processed using the same procedures as patient specimens. External control materials for waived tests should be ready to use or employ only very simple preparation steps, e.g., breaking a vial in order to mix liquid and dry components of the control material.
Fail-safe mechanisms	Mechanisms to ensure that a test system does not provide a result when test conditions are inappropriate or when the result is based on faulty test functioning. Also, measures that prevent improper operation of the device

⁹ Also see Harmonized Terminology Database at the CLSI website: http://www.clsi.org/AM/Template.cfm?Section=Harmonized_Terminology_Database, for further details and more general use of the terms

	(for example guides or channels that prevent improper strip placement).
Failure alert mechanisms	Mechanisms that notify the operator of any test system malfunction or problem. Failure alert mechanisms ideally include built-in controls or checks. Procedures that use external control material can also be considered failure alert mechanisms.
Flex studies	Studies performed using the device under conditions of operational stress. These studies are performed to identify sources of error as part of the risk assessment.
Hazards (for IVD’s)	Potential source of harm (to a patient or test operator). For IVD’s, hazards for patients are generally incorrect patient results or operator injuries.
Intended operator (user)	In this guidance, intended operator (user) refers to a test operator with limited or no training or hands-on experience in conducting laboratory testing (e.g., medical assistant, nurse, doctor, or an individual with no medical training).
Internal control	A control, or system check, built into the device system, i.e., the user does not need to use additional reagents to perform that particular control process.
Laboratory Professional	A person who meets the qualifications to perform moderate or high complexity testing, such as a medical technologist (MT) or medical laboratory technician (MLT).
Limit of Blank (LoB)	The highest measurement result that is likely to be observed (with a stated probability) for a blank sample;
Limit of Detection (LoD)	The lowest amount of analyte in a sample that can be detected with (stated) probability, although perhaps not quantified as an exact value.
Limits of erroneous results (LER)	In this guidance the limits of erroneous results are limits for the differences between the WM and the CM or the WM and the RM. Results inside these limits pose a risk to patient safety.
Matrix	The totality of components within a patient specimen, controls, or calibrator other than the analyte.
Matrix effects	The influence of a property of the sample, other than the analyte, on the device’s performance characteristics. Examples: (a) The test values for a particular analyte in whole blood collected via a venous sample may differ from those for a fingerstick. (b) Control material in a matrix different from that of the specimen should be tested to ensure that test performance is the same as that for the specimen.
Procedural control	Controls or indicators to monitor whether specific aspects of the procedure were performed correctly. Often, procedural controls are in the form of

	control lines on a single use cassette or dip devices and indicate whether sufficient sample was applied. Procedural controls generally do not serve as a control for the entire test system.
Quality control (QC)	The entire set of procedures and system checks designed to monitor the test method and the results to assure acceptable test system performance.
Quick Reference Instructions	A short (usually one or two page) version of the test instructions, preferably laminated, that can be posted. It contains instructions needed on a frequent basis and directs operators to the package insert for topics such as performance characteristics, long term maintenance instructions, troubleshooting, and other more detailed instructions.
Reference material	A preparation of the analyte whose concentration and purity are sufficiently well-established and well-recognized for the material to be suitable for calibration or value assignment.
Reference method (RM)	A method which has been thoroughly investigated and has been shown to yield values having trueness and precision of measurement such that the method can be used to assess the trueness of other methods for the same analyte or for assigning values to reference materials.
Risk analysis	Systematic use of available information to identify hazards and estimate risk.
Risk control	Process through which decisions are reached and protective measures are implemented for reducing risks to or ensuring specified levels.
Risk evaluation	Judgment of whether acceptable risk has been achieved based on risk analysis.
Risk Management	Systematic application of management policies, procedures, and practices to analyze, evaluate, and control risk (includes risk analysis, evaluation, and control).
Source of error	A component of the device, measurement method, or operator practice that can lead to device failure and, thus, create a risk for patients, operators, or other individuals.
Total analytical error	The combination of errors from all sources, both systematic and random. It is often expressed in terms of an interval that contains a specified proportion (e.g., 95%) of the distribution of observed differences between the WM and CM values (for the CM of types A and B). Sometimes relative differences (e.g., ((WM-CM)/(CM)) are used instead of differences.

Trueness	The closeness of agreement between the average values obtained from a large series of test results and an accepted reference value. The measure of trueness is usually expressed in terms of bias. It is also often referred to as systematic bias.
Unitized test device	A self-contained test device to which a specimen is added directly and in which all steps of the testing process occur. A unitized device is used for a single test and must be discarded after testing.