CLIA Guidance

CLIA Guidance.pdf

CLIA Waiver Applications

CLIA Guidance

OMB: 0910-0598

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0910-0598 can be found here:

Document [pdf]

Download: pdf | pdf

The draft of this guidance document was released on September 7, 2005.
For questions regarding this document contact Carol Benson, 240-276-0396 or by email at
[email protected]
OMB control number: 0910-0598
Expiration Date: 06/30/2013

See additional PRA statement in Section VIII of this guidance

U.S. Department of Health and Human Services
Food and Drug Administration
Center for Devices and Radiological Health
Office of In Vitro Diagnostic Device Evaluation and Safety

Contains Nonbinding Recommendations

Preface
Public Comment
Written comments and suggestions may be submitted at any time for Agency consideration to the
Division of Dockets Management, Food and Drug Administration, 5630 Fishers Lane, Room
1061, (HFA-305), Rockville, MD, 20852. Alternatively, electronic comments may be submitted
to http://www.fda.gov/dockets/ecomments. When submitting comments, please refer to Docket
No. 2008D-0031. Comments may not be acted upon by the Agency until the document is next
revised or updated.

Additional Copies
Additional copies are available from the Internet http://www.fda.gov/cdrh/oivd/guidance/1171.pdf.
You may also send an email request to [email protected] to receive an electronic copy of the
guidance, or send a FAX request to 240-276-3151 to receive a hard copy. Please use the
document number 1171 to identify the guidance you are requesting.

Contains Nonbinding Recommendations

TABLE OF CONTENTS
I.

INTRODUCTION............................................................................................................. 5

The Least Burdensome Approach ............................................................................................... 7
II.

COMPONENTS OF A CLIA WAIVER APPLICATION............................................ 7

III.

DEMONSTRATING "SIMPLE".................................................................................... 8

IV.

DEMONSTRATING "INSIGNIFICANT RISK OF AN ERRONEOUS RESULT" –
Failure Alerts and Fail-Safe Mechanisms....................................................................... 9
A. Tier 1: Risk Analysis and Flex Studies.......................................................................... 11
B. Tier 2: Fail-Safe and Failure Alert Mechanisms.......................................................... 13
1.
Points to consider for designing fail-safe and failure alert mechanisms........ 13
2.
External control materials ................................................................................. 14
3.
Additional points concerning control materials............................................... 15
C. Validating Fail-Safe and Failure Alert Mechanisms, including External Control
Procedures ............................................................................................................................. 15

DEMONSTRATING INSIGNIFICANT RISK OF AN ERRONEOUS RESULT –
“ACCURACY” ............................................................................................................... 16
A. Clinical Study Sites and Participants ............................................................................ 17
1.
Testing sites.......................................................................................................... 17
2.
Clinical study participants ................................................................................. 17
3.
Clinical study reports ......................................................................................... 19
B. Clinical Studies for Tests with Quantitative Results.................................................... 19
1.
Selection of the Comparative Method (CM) .................................................... 20
2.
Establishing Allowable Total Error Zones and Zones for Erroneous Results
............................................................................................................................... 20
3.
Clinical Study Design.......................................................................................... 22
C. Clinical Studies for Tests with Qualitative Results...................................................... 26
1.
Selection of the Comparative Method (CM) .................................................... 26
2.
Clinical Study Design.......................................................................................... 27
3.
Performance Criteria for Qualitative Tests ..................................................... 31

VI.

LABELING FOR WAIVED DEVICES ....................................................................... 32
A. Quick Reference Instructions (QRI) ............................................................................. 33
B. Quality Control (QC) Labeling Recommendations ..................................................... 33
C. Educational Information ................................................................................................ 34

Contains Nonbinding Recommendations
VII.

SAFEGUARDS FOR WAIVED TESTS....................................................................... 35

VIII. Paperwork Reduction Act of 1995................................................................................. 35
REFERENCES............................................................................................................................ 36
Appendix A: Statistical Notes .................................................................................................... 38
Appendix B: Labeling................................................................................................................. 40
Appendix C: Definition of Terms as Used in this Document ................................................. 42

Contains Nonbinding Recommendations

Guidance for Industry and FDA Staff:
Recommendations for Clinical Laboratory
Improvement Amendments of 1988 (CLIA)
Waiver Applications for Manufacturers of
In Vitro Diagnostic Devices
This guidance represents the Food and Drug Administration's (FDA's) current thinking on
this topic. It does not create or confer any rights for, or on any person and does not operate to
bind FDA or the public. You can use an alternative approach if the approach satisfies the
requirements of the applicable statutes and regulations. If you want to discuss an alternative
approach, contact the FDA staff responsible for implementing this guidance. If you cannot
identify the appropriate FDA staff, call the appropriate number listed on the title page of this
guidance.

INTRODUCTION

The Secretary of Health and Human Services has delegated to FDA the authority to determine
whether particular tests are "simple" and have "an insignificant risk of an erroneous result" under
CLIA and thus eligible for waiver categorization (69 FR 22849, April 29, 2004). The Centers
for Medicare & and Medicaid Services (CMS) is responsible for oversight of clinical
laboratories, which includes issuing waiver certificates. CLIA requires that clinical laboratories
obtain a certificate before accepting materials derived from the human body for laboratory tests.
42 U.S.C. § 263a(b). Laboratories that perform only tests that are "simple" and that have an
"insignificant risk of an erroneous result" may obtain a certificate of waiver. 42 U.S.C.
§ 263a(d)(2).
CLIA, 42 U.S.C. § 263a(d)(3) Examinations and Procedures, as modified by the Food and Drug
Administration Modernization Act (FDAMA), reads as follows regarding tests that may be
performed by laboratories with a Certificate of Waiver:
The examinations and procedures [that may be performed by a laboratory with a Certificate
of Waiver]… are laboratory examinations and procedures that have been approved by the
Food and Drug Administration for home use or that, as determined by the Secretary, are
simple laboratory examinations and procedures that have an insignificant risk of an
erroneous result, including those that -- (A) employ methodologies that are so simple and
accurate as to render the likelihood of erroneous results by the user negligible, or (B) the

Contains Nonbinding Recommendations
Secretary has determined pose no unreasonable risk of harm to the patient if performed
incorrectly.
This guidance describes recommendations for device manufacturers seeking to submit
information through a CLIA waiver application to FDA to support a determination whether the
device meets CLIA statutory criteria for waiver.
In this document, FDA (we) recommends an approach for you to use to demonstrate that your
device is simple and has an insignificant risk of an erroneous result. As part of demonstrating
the latter, we recommend studies you can conduct to demonstrate the test is "accurate." While
we recommend you adopt the approach we have outlined in this guidance for waiver
applications, you may use another approach that you believe would be appropriate for your
device’s waiver application if it meets the CLIA statutory requirements. (See Least
Burdensome Approach, below.)
We based the recommendations in this document on our interpretation of the law, experience with
CLIA complexity determinations, and interactions with stakeholders. Interactions included an
open public workshop on August 14-15, 2000, a proposal presented by AdvaMed (Advanced
Medical Technology Association) at the September 2003 Clinical Laboratory Improvement
Advisory Committee (CLIAC) meeting, and recommendations proposed by CLIAC during the
February 2004 meeting. In addition, we considered the comments received in response to a 2001
draft guidance and the 2005 draft guidance document, and incorporated revisions based on these
comments as appropriate. 1
Some of the changes reflected in this document (as well as in the 2005 draft guidance document)
from the earlier 2001 draft guidance document entitled “Guidance for CLIA 1988 Criteria for
Waiver,” include the following:
•

Greater emphasis on scientifically-based flex studies 2 and validation and/or verification
studies, linked to the risk assessment for each device.

•

Recognition that reference methods may not be available for every device type.
(However, devices should be traceable to true reference methods of known accuracy,
when such methods are available.)

•

Additional emphasis on use of quality control procedures.

•

Greater emphasis on intended users (which may include medical assistants, nurses or
doctors, and lay people, as appropriate) during studies testing the device.

•

Updated study recommendations with emphasis on use of patient specimens, in an
intended use environment, over time.

The draft document of September 7, 2005, entitled “Recommendations for Clinical Laboratory
Improvement Amendments of 1988 Waiver Applications” (70 FR 53231) replaced the draft
“Guidance for Clinical Laboratory Improvement Amendments of 1998 (CLIA) Criteria for
Waiver,” March 1, 2001 (66 FR 12939)).
2
For the definition of this term and other technical terms, as they are used in this document, see
Appendix C.
6

Contains Nonbinding Recommendations
This document does not address test systems cleared or approved by FDA for over-the-counter or
prescription home use since these automatically qualify for CLIA waiver. 42 U.S.C. 263a(d)(3).
This guidance document also does not address use of the Office of In Vitro Diagnostic Device
Evaluation and Safety (OIVD)’s replacement reagent and instrument family policy3 for waived
devices; that policy does not currently apply to CLIA waiver applications.
The draft of this document was issued September 7, 2005. We have also issued a draft guidance
entitled "Guidance for Administrative Procedures for CLIA Categorization,"
www.fda.gov/cdrh/ode/guidance/1143.html. In it, we propose recommendations to device
manufacturers on FDA’s administrative procedures for CLIA categorization.
FDA's guidance documents, including this guidance, do not establish legally enforceable
responsibilities. Instead, guidances describe the Agency's current thinking on a topic and should
be viewed only as recommendations, unless specific regulatory or statutory requirements are
cited. The use of the word should in Agency guidances means that something is suggested or
recommended, but not required.

The Least Burdensome Approach
We believe we should consider the least burdensome approach in all areas of medical device
regulation. This guidance reflects our careful review of the relevant scientific and legal
requirements and what we believe is the least burdensome way for you to comply with those
requirements. However, if you believe that an alternative approach would be less burdensome,
please contact us so we can consider your point of view. You may send your written comments
to the contact person listed in the preface to this guidance or to the CDRH Ombudsman.
Comprehensive information on CDRH's Ombudsman, including ways to contact him, can be
found on the Internet at http://www.fda.gov/cdrh/ombudsman/.

II. COMPONENTS OF A CLIA WAIVER APPLICATION
This guidance discusses the following components that we recommend you include in a CLIA
waiver application:
•

A description of your device that demonstrates it is simple to use. (Section III.)

•

The results of risk analysis including the identification of potential sources of error for
your device. (Section IV.)

•

The results of flex studies demonstrating insensitivity of the test system to environmental
and usage variations under conditions of stress. (Section IV.)

•

The results of risk evaluation and control including a description of (1) measures you
have implemented to mitigate the risk of errors, and (2) validation and/or verification
studies demonstrating the ability of failure alert, fail-safe mechanisms, and other control

For a description of this policy, see Guidance for Industry and FDA Staff; Replacement
Reagent and Instrument Family Policy, http://www.fda.gov/cdrh/oivd/guidance/950.html
7

Contains Nonbinding Recommendations
measures that you have incorporated into your device to mitigate the risk of errors, even
under conditions of stress. (Section IV.)
•

A description of the design and results of clinical studies you conducted to demonstrate
that the device has an insignificant risk of erroneous result in the hands of the intended
user (hereinafter operator). (Section V.)

•

Proposed labeling with instructions for use consistent with a device that is “simple.”
(Section VI.)

III. DEMONSTRATING "SIMPLE"
CLIA requires that tests performed by laboratories with a Certificate of Waiver be "simple." 42
U.S.C. 263a(d)(2), (3). We recommend that, as a first step in the process of deciding whether
your device could be a candidate for waiver, you determine whether your device is simple.
Under the approach recommended in this guidance, FDA believes that a simple test should have
characteristics such as the following:
•

Is a fully automated instrument or a unitized or self-contained test.

•

Uses direct unprocessed specimens, such as capillary blood (fingerstick), venous whole
blood, nasal swabs, throat swabs, or urine.

•

Needs only basic, non-technique-dependent specimen manipulation, including any for
decontamination.

•

Needs only basic, non-technique-dependent reagent manipulation, such as “mix reagent
A and reagent B.”

•

Needs no operator intervention during the analysis steps.

•

Needs no technical or specialized training with respect to troubleshooting or
interpretation of multiple or complex error codes.

•

Needs no electronic or mechanical maintenance beyond simple tasks, e.g., changing a
battery or power cord.

•

Produces results that require no operator calibration, interpretation, or calculation.

•

Produces results that are easy to determine, such as ‘positive’ or ‘negative,’ a direct
readout of numerical values, the clear presence or absence of a line, or obvious color
gradations.

•

Provides instructions in the package insert for obtaining and shipping specimens for
confirmation testing in cases where such testing is clinically advisable.

•

Has test performance comparable to a traceable reference method as demonstrated by
studies in which intended operators 4 perform the test. If a reference method is not

In this guidance, intended operator refers to a test operator with limited or no training or handson experience in conducting laboratory testing. Laboratory professional refers to a person who
8

Contains Nonbinding Recommendations
available for a test you are proposing for waiver, please contact OIVD to discuss your
proposed plan prior to submitting your application.
•

Contains a quick reference instruction sheet that is written at no higher than a 7th grade
reading level.

We believe a test that is simple should not have the following characteristics:
•

Sample manipulation is required to perform the assay. (For example, tests that use plasma
or serum are not considered simple.) Sample manipulation includes processes such as
centrifugation, complex mixing steps, or evaluation of the sample by the operator for
conditions such as hemolysis or lipemia.

•

Measurement of an analyte could be affected by conditions such as sample turbidity or
cell lysis.

After you consider whether your device is “simple” based on the items listed above, it may be
helpful for you to contact OIVD for feedback on this issue prior to conducting clinical studies to
support waiver. In your waiver application, you should describe features of your device that
address the issues listed above. Whenever possible (for example, if your test system consists of a
unitized device), you should include sample(s) of the device with your waiver application to aid
FDA in its determination of whether your device is “simple.” You may also schedule a meeting to
bring your device to FDA to aid FDA in making this determination.

IV. DEMONSTRATING "INSIGNIFICANT RISK OF AN
ERRONEOUS RESULT" – Failure Alerts and Fail-Safe
Mechanisms
Generally, the risk of an erroneous result should be far less for waived tests than non-waived
tests. You should demonstrate in your CLIA waiver application that (1) the test system design is
robust, e.g., insensitive to environmental and usage variation, and (2) that all known sources of
error are effectively controlled. In general, flex studies should be used to demonstrate robust
design while risk management should be used to demonstrate identification and effective control
of error sources, although the two are not mutually exclusive.
Most risk control measures should be fail-safe measures or failure alert mechanisms. Appropriate
fail-safe mechanisms and failure alert mechanisms help assure that a test has “an insignificant
risk of an erroneous result.” Examples of fail-safe mechanisms are lock-out functions to ensure
that a test system does not provide a result when test conditions are inappropriate, such when
there is a component malfunction or operator error. Other examples are measures within the
system to prevent operator error, such as guides or channels that prevent improper strip
placement. We recommend that test system design incorporate fail-safe mechanisms whenever it
is technically practicable.
meets the qualifications to perform moderate or high complexity testing, such as a medical
technologist (MT) or medical laboratory technician (MLT).

Contains Nonbinding Recommendations

If fail-safe mechanisms are not technically practicable for some risks, failure alert mechanisms
should be used. Failure alert mechanisms notify the operator of any test system malfunction or
problem. They may include measures such as external controls, internal procedural controls, or
electronic controls. Devices with such mechanisms allow the operator to correct the error, or put
the operator on notice that the results will be unreliable due to the error. For example, in cases
where the result exceeds the reportable range (e.g., extremely high or low glucose result) and the
result is a critical value, the device should give a message such as "out of range high" or "out of
range low."
We recommend a two-tiered approach, outlined below, to demonstrate that your device is robust
and has appropriate and effective risk control measures to ensure insignificant risk of an
erroneous result.
Tier 1: Risk Analysis and Flex Studies. You should conduct a systematic and comprehensive
risk analysis that identifies all potential sources of error, including test system failures and
operator errors, and identifies which of these errors can lead to a risk of a hazardous situation.
We recommend that the “Operator error/Human factors” examples on pages 12-13 be used as an
analytical aid to complement the risk analysis method(s) you use.
You should conduct flex studies, i.e., studies that stress the operational limits of your test system.
Flex studies should be used to validate the insensitivity of the test system to variation under
stress conditions. Where appropriate, flex studies should also be used to verify and/or validate
the effectiveness of control measures at operational limits. [See also Guidance for the Content of
Premarket Submissions for Software Contained in Medical Devices
(http://www.fda.gov/cdrh/ode/guidance/337.html) for further discussion.]
In your waiver application you should include:
•
•
•

The risk analysis results which serve as a basis for the tabular reporting of risk
management results. (See Tier 2.)
A summary of the design and results of your flex studies.
Conclusions you draw from the flex studies.

Tier 2: Fail-Safe and Failure Alert Mechanisms. Once you have identified the potential
sources of error, you should identify the control measures, including fail-safe and failure alert
mechanisms that will reduce risks for these sources of error. When the control measures have
been implemented, you should (1) verify that each control measure has been properly
implemented, and (2) verify and/or validate the effectiveness of each control measure.
We recommend that this risk management information be presented in tabular form in your
waiver application. It should include the following information for each risk for each potential
source of error:
•

Identification of each risk and the potential source of error that causes it.

•

Identification and physical description of the risk control measure or combination of
10

Contains Nonbinding Recommendations
measures used to reduce risk to an acceptable level. This includes fail-safe mechanisms,
failure alert mechanisms, external controls as well as any other controls used or that you
recommend the operator use for your device. It also includes a description of the manner
in which the control measure(s) either reduces the probability of occurrence of the error,
mitigates the effect of the error, or both.
•

Objective evidence verifying that each control measure or combination of measures has
been implemented, including a description of the method of verification.

•

Objective evidence from testing, confirming the effectiveness of fail-safe and/or failurealert mechanisms in preventing and/or mitigating the effects of false results. The
evidence and results should also support your recommended control procedures and
frequencies. Any limitations of fail-safe and failure alert mechanisms, including all
internal and external controls, should be described.

A. Tier 1: Risk Analysis and Flex Studies
As noted above, you should identify all potential sources of error by conducting a systematic
and comprehensive risk analysis. This analysis should be part of your risk management
process consisting of risk analysis, evaluation, and control. FDA recognizes the process of the
International Standard ISO 14971, Medical Devices - Application of Risk Management to
Medical Devices.
http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfStandards/Detail.CFM?STANDARD I
DENTIFICATION_NO=5188. This guidance is consistent with that process and uses the
same risk management terminology. In general, the standard and its annexes can be used to
obtain more detailed information about risk management concepts and practices 5 .
Based on the results of the risk analysis and identification of potential problems with
sensitivity to environmental or usage variation, you should conduct flex studies. Flex studies
are designed to challenge the system under conditions of stress to identify potential device
deficiencies, including failures, and determine the robustness of the test system. Examples are
shown in Table 1.
In your analysis, you should consider multiple skill levels of users, as well as potential
instrument and reagent problems. The following websites contain additional information to
consider concerning human factors that may affect test performance:
http://www.fda.gov/cdrh/humanfactors/index.html,
http://www.fda.gov/cdrh/humanfactors/resource-manufac.html#2.
Examples of potential sources of error to consider for the risk analysis and flex studies are
listed below. See also CLSI EP-18A for examples (See [1]). You should consider each of
these potential sources of error, as applicable to your device, and also consider any other
potential system failures that may be specific to your device.
Operator error/ Human factors
5

However, it may not always be appropriate to justify that risks are acceptable based solely on
the “As Low As Reasonably Practicable" principle described in Annex D of ISO 14971: 2007.

Contains Nonbinding Recommendations
•

Use of incorrect specimen type.

•

Incorrect application of the specimen to the device (e.g., incorrect placement,
incorrect volume).

•

Incorrect handling of reagents including those in self-contained unitized test devices.

•

Incorrect placement of device (e.g., non-level surface).

•

Incorrect placement of reagents, including strips, or other components that contain
reagents.

•

Use of incorrect reagents (for example, reagents that are not specific for the particular
device or lot or generic reagents).

•

Incorrect order of reagent application.

•

Use of incorrect amount of reagent.

•

Incorrect timing of procedures (e.g., specimen application, running the test, or reading
results).

•

Incorrect reading of test results.

•

Incorrect reading due to color blindness.

Specimen integrity and handling
•

Error in specimen collection.

•

Use of inappropriate anticoagulant.

•

Clotted specimen.

•

Error in specimen handling.

•

Incorrect specimen transport and/or storage.

•

Presence of interfering substances.

•

Presence of bubbles in the specimen.

Reagent integrity (Reagent viability)
•

Use of improperly stored reagents.

•

Use of outdated reagents.

•

Use of improperly mixed reagents.

•

Use of contaminated reagents.

Hardware, software, and electronics integrity
•

Power failure.

•

Power fluctuation.

•

Incorrect voltage.

•

Repeated plugging and unplugging of the device.

•

Hardware failure.

Contains Nonbinding Recommendations
•

Software failure (see Guidance for the Content of Premarket Submissions for
Software Contained in Medical Devices
http://www.fda.gov/cdrh/ode/guidance/337.pdf)

•

Electronic failure.

•

Physical trauma to unit.

Stability of calibration and internal controls
•

Factors that affect calibrator and calibration stability, including determination of
calibration stability over time and after power failures.

•

Factors that may interfere with calibration.

Environmental factors
•

Impact of key environmental factors (heat, humidity, barometric pressure changes,
altitude (if applicable), sunlight, surface angle, device movement, etc.) on reagents,
specimens, and test results.

•

Impact of key environmental factors (including electrical or electromagnetic
interference) on instruments, if appropriate.

B. Tier 2: Fail-Safe and Failure Alert Mechanisms
1. Points to consider for designing fail-safe and failure alert
mechanisms
We recommend that you consider including the items on the following list, as
appropriate. You should consider incorporating fail-safe mechanisms when possible.
•

Lock-out functions that do not allow output of results if controls or system checks
are not successfully completed.

•

Lock-out functions that do not allow output if expired reagents are used.

•

Lock-out functions that do not allow output of results if the device was
mishandled (e.g., dropped) and the device detects damage during internal
electronic system checks.

•

Physical features to ensure correct placement of components, such as strips or
cartridges.

•

Monitors of environmental conditions (e.g., indicator desiccants) incorporated
into the test system or the kit container to alert the user to environmental
conditions that are outside of the recommended storage conditions.

•

Battery checks.

•

Internal procedural controls to flag procedural problems such as improper sample
flow, incorrect use of components, or improper addition of specimen. (However,
procedural controls generally provide limited problem detection and, by
themselves, are generally not sufficient to serve as a failure alert mechanism.)

Contains Nonbinding Recommendations
•

Internal non-procedural controls (e.g., for checking the integrity of the reagents).

•

Controls to check that electronic features of the device are within specifications.

•

External control material.

When designing controls, you should consider the unique features of the test system and
link control procedures to the robustness of the assay, as determined by your flex studies.
The controls you devise to mitigate the risks you identify may be based on procedures
typical for laboratory-based methodologies (e.g., testing external materials at two levels
at a time interval of once per shift or on each day of testing) or may be a combination of
features, such as those listed above, that ensure complete system quality monitoring. When
designing packaging for your device, you should also consider that the number of tests per
kit should depend on the stability of the reagents or the robustness of your test as
demonstrated through testing.
When appropriate, you should incorporate capabilities into the test system software that
allow for data retention, identification of outliers, and trend detection in order to alert the
user to the occurrence of random or systematic errors.
Procedural controls, which are typically internal, are desirable for waived devices.
However, these types of controls generally do not replace external controls especially
because they often only control for adequate volume. Your flex studies and validation
and/or verification studies should evaluate the sensitivity of internal control reagents to
all applicable test system errors. The total quality control (QC) system (including all
control procedures and internal checks) should control for all aspects of test performance,
including electronic aspects and integrity of reagents.
We do not recommend training as a sole means of mitigating potential sources of harm.
Aspects of the device design that are controlled and maintained by the manufacturer can
potentially be considered as mitigations.

2. External control materials
Whenever feasible, you should include external control materials in the test kit. External
control materials for waived tests should be ready to use or employ only very simple
preparation steps, e.g., breaking a vial in order to mix liquid and dry components of the
control material. Reconstitution steps should not require pipetting by the user. For both
quantitative and qualitative tests, the levels of the control materials should correspond to
the medical decision level(s) relevant to the indications for use for your
test. More than one level may be needed in order to ensure accuracy for quantitative
tests. For qualitative tests, you should ensure that control materials include those with
concentrations sufficiently close to the cutoff to provide adequate assessment of test
performance for patient samples near the clinical cutoff.
You should alert operators about control procedures and the availability of control
materials and integrate instructions for external control testing within the test procedure
instructions (package insert and Quick Reference Instructions), in order to increase the

Contains Nonbinding Recommendations
likelihood that operators will perform QC correctly. In the test instructions, you should
specify minimum frequency for running controls and include recommended levels of
control materials that correspond to medical decision levels. The labeling should indicate
in bold why external controls are important and the consequences of not performing all
QC procedures.
In addition, when control materials are not included in the test kit, you should also
recommend, in the package insert and Quick Reference Instructions, the use of specific
control material(s) that will ensure optimal verification of the test system performance.
Providing or recommending external control materials may not be critical in those limited
cases where sufficient fail-safe mechanisms are in place for the entire system. Although
we are currently unaware of any such systems, should you develop one, we recommend
that you explain, in your waiver application, your rationale for omitting these control
materials.
You should describe, in your application, how you established the QC limits and how you
demonstrated that the chosen limits provide adequate assessment of test performance
with patient samples. For quantitative tests, you should consider the precision of the test
system, as well as the total allowable error for the particular analyte. Ranges that are too
broad may be incapable of reliably detecting unacceptable levels of imprecision or bias.
Control materials should mimic performance of patient samples as closely as possible.
When the matrix of the material differs from that of the specimen, you should determine
and describe in your application how these differences may affect or limit the information
provided by the control result. You can accomplish this by testing control materials in
parallel with actual patient samples of similar known values and comparing the results of
the control material and patient samples with respect to precision or bias observed. You
should account for matrix effects when setting the limits for control material to be used
with your test.

3. Additional points concerning control materials
If you did not previously submit information addressing the items below in your
premarket submission, you should provide them in your waiver application:
• Opened and unopened control material stability data. You should include

acceptance criteria and results. The term "unopened" refers to shelf-life stability
whereas "opened" refers to reconstituted conditions, or other conditions after the
vial is initially opened by the user.
• Lot-to-lot reproducibility, conducted on at least three consecutive lots of control

material.

C. Validating Fail-Safe and Failure Alert Mechanisms, including
External Control Procedures
You should conduct studies that validate all fail-safe and failure alert mechanisms (including
any procedures you recommend that use external control materials) that address all the

Contains Nonbinding Recommendations
causes of test errors that you identified in your risk analysis. These studies should be
conducted under conditions that stress the device in order to demonstrate how fail-safe and
failure alert mechanisms respond to such conditions. You should describe your validation
and/or verification studies and the results of these studies in your waiver application and
indicate how the results support the ability of fail-safe and failure alert mechanisms to detect
and mitigate test errors. You should include a description of how your recommendations for
external control materials and procedures (including frequency) are supported by your
validation and/or verification studies and confirmed by the clinical studies described in
Section V, below.

Table 1 - Examples of approaches to flex studies and control validation studies
under conditions of stress
POTENTIAL SOURCE
OF ERROR

EXAMPLES OF FLEX
STUDIES

EXAMPLES OF
VALIDATION STUDIES

Operational storage is 2-4°
C.

Environmental studies
included storing the kit at 0°,
2°, 10°, 25°, and
37° C.
Studies showed that when
frozen, or stored at 25° C for
over 3 days, the device failed.

Studies to validate that failsafe mechanisms, or failure
alerts, including external
control procedures, alert the
operator to frozen conditions
or storage at 25° C for more
than 3 days.

What happens when the kit
is stored improperly?

Procedure is to add 3 drops. Flex studies consist of adding
1, 2, 3, 4, 5, and 6 drops and
What happens when an
observing when incorrect
improper number of drops
results are obtained.
are added to the test
Studies show that <2 drops or
procedure?
>5 drops give erroneous
results.

Studies to validate that failsafe mechanisms, or failure
alerts, including control
procedures, alert the operator
of an error when <2 drops or >
5 drops are added.

V. DEMONSTRATING INSIGNIFICANT RISK OF AN
ERRONEOUS RESULT – “ACCURACY”
In this guidance document, we use the term “accurate” to refer to those tests that are comparable
to tests whose results of measurements are traceable to designated references of higher order,
usually national or international standards (see [2 and 3]). To demonstrate that your device is
“accurate” in the hands of the intended operator, we recommend that you perform prospective
clinical studies of the device proposed for waiver using patient samples 6 collected in the intended
testing environment. In this way, the studies will demonstrate, as closely as possible, how the
device performs in the hands of intended operators under the conditions of intended
use.

Spiked samples may also be appropriate for a portion of the study; See Section V.B.3.a, below.
16

Contains Nonbinding Recommendations
In this section, we describe the study designs we recommend for a quantitative test and a
qualitative test. A quantitative test is a test that gives results expressing a numerical amount or
level of an analyte in a specimen; a qualitative test is a test that provides only two responses (i.e.,
positive/negative or yes/no) (see [4]). If your test does not fit the paradigm of a quantitative,
numerical test or a qualitative, two-response test, we recommend that you consult with OIVD.
You should evaluate test performance in a setting designed to replicate, as closely as possible,
the actual intended clinical use setting. Therefore, the clinical study design should include the
following:
•

Intended testing sites.

•

Intended operators. We encourage you to enroll operators with the least amount of
training that might be encountered at the types of sites for which this device is intended.

•

Intended sample type and matrix.

•

Testing over time, as in the typical intended use setting.

A. Clinical Study Sites and Participants
1. Testing sites
You should conduct the clinical study to support CLIA waiver at a minimum of three
intended use sites at different demographic locations (e.g., outpatient clinic, physician’s
office). In your CLIA waiver application, you should present a brief description of each
site, including its name, address, and the date the study was performed. If there were sites
that were included at the beginning, but then did not complete the study, you should
provide a brief explanation for why those sites did not complete the study.

2. Clinical study participants
a. Operators
(1) Intended operators
You should ensure that the operator study participants enrolled represent
anticipated operators of the device you propose for CLIA waiver. We
recommend that you record and tabulate the education (including experience and
training) and the occupation of each operator to demonstrate that these
participants meet the definition of intended operators and include this in your
CLIA waiver application. In addition, for each study site, we ask you to report
the same information on other operators that were available at the testing site but
that were not chosen to participate.
(2) Instructions for use
You should provide the operators who participate in the study with the labeling
and training materials that will be provided with the test kit in the actual intended
use settings when the test is marketed. This includes the proposed package insert
and/or Quick Reference Instructions (see Section VI.C. Educational Information).
These labeling materials should include all control procedures you plan to

Contains Nonbinding Recommendations
recommend to device users when the device is marketed. Study participants
should receive no additional instructions (e.g., written or verbal training,
coaching, or prompting) beyond the materials included with the test system.
Likewise, study participants should have no opportunity to discuss the test with
other participants or otherwise coach or observe each other. Study participants
may call an 800 number help-line if such a service is to be provided for the device
when it is marketed. You should include, in your waiver application, the
instructions you provided to test operators participating in the study.
(3) Universal precautions
Your clinical study must comply with all pertinent laws and regulations including
Occupational Health and Safety Administration (OSHA) regulations pertaining to
biological hazards ("universal precautions"), 29 CFR 1910.1030.
(4) Operator questionnaire
You should develop an operator questionnaire to be filled out by all test operators
participating in the study. This questionnaire should be designed to help assess
whether the participants understood how to use the device correctly. It is
important that the questionnaire be given to test operators after the completion of
the clinical study, so the questions do not bias the participants during the study.
Some questions may ask operators to indicate agreement on a 1-5 scale
(1=strongly disagree; 5=strongly agree). The following are examples:
•
•
•
•
•
•

The instructions were easy to follow.
It was easy to apply the sample correctly.
It was easy to see and understand the test results (e.g., appearance of the
line, change of color).
The control line was always distinct and easy to read.
The instructions clearly explain what to do if a test result does not appear
or is invalid.
I needed help from someone the first time I ran the test.

We recommend that, as part of the questionnaire, you show various possible test
results and control results that are positive, negative, and invalid and ask the
operator to read these results. You may wish to present these questions as
true/false or multiple choice questions.
You should also strongly encourage general comments by the study operators.
We recommend that you include your survey questions and results with your
CLIA application.

b. Subjects (Patients)
You should ensure that subjects from whom you will obtain specimens for the clinical
study meet inclusion and exclusion criteria corresponding to the intended use
population of the test. Once a subject has met appropriate inclusion criteria, he/she
should be informed of the study and invited to participate in the CLIA waiver study.

Contains Nonbinding Recommendations
You should follow applicable regulations for patient privacy, including informed
consent and Health Insurance Portability and Accountability Act (HIPAA)
requirements. 45 CFR Part 46; P.L. 104-191. See 21 CFR Parts 50 and 56 for
regulations regarding protection of human subjects for studies that support marketing
applications.

c. Financial disclosure
If clinical investigators are involved in the clinical study, we ask that you include a
Financial Disclosure Statement with your waiver application. For information on
financial disclosure statements, we recommend you consult, "Guidance for Industry:
Financial Disclosure by Clinical Investigators,"
http://www.fda.gov/oc/guidance/financialdis.html and 21 CFR Part 54, Financial
Disclosure by Clinical Investigators.

3. Clinical study reports
You should report results of the clinical study intended to support CLIA waiver by each
intended site and overall, if appropriate. Reports should include the following:
•
•
•
•
•
•
•
•
•
•
•
•
•

Protocol description.
Number of subjects (i.e., patients) studied.
Procedures for subject inclusion and exclusion.
Description of the subject population.
Description of how specimens were collected and stored.
Masking techniques.
Discontinuations.
Complaints, device failures, and replacements.
Any invalid results and how these were handled.
Information about QC procedures that were performed by intended users.
Pertinent tabulations.
Annotated line listings of results (including electronic versions).
Clear descriptions and presentations of the statistical analyses.

You should not remove "outliers." You should provide an explanation for data that are
incomplete or missing.

B. Clinical Studies for Tests with Quantitative Results
The clinical studies to support CLIA waiver should compare results obtained with the device
proposed for CLIA Waiver (Waiver Method, or WM) to results obtained by a Comparative
Method (CM). The CM for the clinical studies for quantitative tests should be performed in a
laboratory setting by laboratory professionals. The topics outlined in this section, and in
Section C are discussed in greater detail in references [2-16]. We recommend that you consult
these references.

Contains Nonbinding Recommendations

1. Selection of the Comparative Method (CM)
The following Comparative Methods are presented in the order in which we recommend
you select them if available. Thus, we recommend you use a CM of type A (Reference
Method (RM)) if it is available. If it is not available, we recommend you select a CM of
type B (best available traceable method) and provide adequate justification for selection
of this method in your waiver application. If neither a CM of type A nor a CM of type B
are available, we strongly recommend that you consult with OIVD to discuss potential
comparison with another well-documented method.
•

CM of Type A (Reference Method): This is a method that has been thoroughly
investigated. It has been shown to yield values having trueness and precision of
measurement such that the RM can be used to assess the trueness of other methods
for the same quantity or for assigning values to reference materials (for details, see
Harmonized Terminology Database at the CLSI website: www.clsi.org).7

•

CM of Type B (Traceable Method): This is a method traceable to references of
higher order. A reference of higher order can be a certified reference material, a
reference measurement procedure, or a network of reference laboratories [2]. The
traceable method is a method in which the results of measurement can be related to
stated references, usually national or international standards, through an unbroken
chain of comparisons in which measurement uncertainties have been documented
at every step in the procedure. For details on traceability, see [2] and [3]. You
should provide information supporting traceability, including a description of
reference materials or methods used to establish the traceability of the calibrators.
You should provide the values and uncertainties for each of those reference
materials or methods along with any other relevant information concerning
traceability (for details, see [3]).

We recommend that you describe the clinical performance of the CM used in your study
in your CLIA waiver application and the sources you used for this description (e.g.,
published literature, labeling). The measurement range of the CM should be at least as
wide as the measurement range of the WM.

2. Establishing Allowable Total Error Zones and Zones for Erroneous
Results
You should establish performance criteria for your device before you begin the clinical
study in order to objectively evaluate the WM. You should establish criteria for the
following zones and demonstrate that your device meets these criteria.
The Allowable Total Error (ATE) zones.
Values of the WM that fall within the ATE zones are values that can be tolerated

•

FDA recommends you consult information on this website as of the date of issuance of this
final guidance document.

Contains Nonbinding Recommendations
without invalidating the medical usefulness of the WM. Typically at least 95% of the
sample results should fall within the established ATE zone (see Figure 1, below).
The differences between test values obtained with the CM and test values obtained
with the WM can be described through total analytical error (for information on total
analytical error, see Section 3.b(2), below.)
The Zones of Limits for Erroneous Results (LER).
Patient results inside the LER zone pose a risk to patient safety. When WM values
fall inside the LER zones, potential harm can occur to the patients if these results are
utilized in medical decision-making. Therefore, your device should not have WM
results in the LER zones.

•

Usually, boundaries of the ATE and LER are expressed as percentages of CM values for
higher values and as a flat limit for low values. A hypothetical example of ATE and LER
zones is illustrated below:

Allowable Total
Error ( ≥ 95% of
samples in study)

Limits for Erroneous
Results (0% of
samples in study)

Figure 1. Example of ATE and LER zones
Several sources of information can be used to establish criteria for ATE and LER zones
for the analyte your device measures. OIVD will review your criteria for ATE and LER
and determine whether they are defined appropriately to meet the waiver requirements of
CLIA (42 U.S.C. 263a).
For analytes that have existing performance limits for professional use (e.g., those listed
in the CLIA regulations (see 42 CFR Part 493 Subpart I, sections 909-959
http://www.phppo.cdc.gov/CLIA/regs/subpart_i.aspx#493.909)), these limits should be

Contains Nonbinding Recommendations
used to define the boundaries of the ATE zone (but the limits should not be expressed as a
multiple of standard deviation units). The limits can be used in the following way. Let a
value %R be the allowable percent of deviation from the target value described in the
CLIA regulations cited above. For example, the value of %R equals 7% for hemoglobin.
Once the value of %R is defined, the zone of ATE is bounded by CM ± %R * CM. If,
when values of the CM and WM are both low, one can tolerate a larger value of %R *
CM without invalidating the medical usefulness of the WM results, then the zone of ATE
for these low values can be established as CM ± D, where D is a fixed number. Thus, the
ATE zone can be defined as CM ± D for low CM values (i.e., below a defined threshold
value) and CM ± %R * CM, for high CM values that exceed the threshold. For example,
for the analyte lithium, the ATE zone can be defined as CM ± 0.3 nmol/L if CM < 1.5
nmol/L, and the ATE zone can be defined as CM ± 20% * CM if CM ≥ 1.5 nmol/L.
For analytes not listed in the CLIA regulations 42 CFR Part 493 Subpart I sections 909959 (and for those listed but expressed as a multiple of standard deviation units), other
criteria may be acceptable, but you should consult with OIVD. For example, the ATE
and LER zones could be based on the ATE and LER zones from the same analyte used
by professional laboratory operators. Other criteria could be based on medical decisionmaking, consideration of intra-individual variations (for example, see [4]), needs for
accuracy of the samples within the reference intervals, or other approaches as applicable
to the particular analyte You should provide the literature and appropriate documentation
to support the ATE and LER you establish. WMs that have a high imprecision or a
systematic bias may have a greater challenge meeting acceptance criteria and may not be
suitable for CLIA waiver.
We recommend that you include the ATE and LER zones on the scatter plots in your
statistical analyses (WM vs. CM).

3. Clinical Study Design
a. Specimen Collection and Sample Preparation
Exact number of samples and operators needed for a CLIA waiver study may vary
depending on performance requirements of a particular test. However, FDA
recognizes that pre-defining all relevant parameters for performance (i.e. slope,
intercept, precision) may be very difficult. We suggest manufacturers consider using
the CLSI document on calculation of total error to demonstrate performance at high,
middle, and low levels. This would usually include a minimum of 360 patient
samples from the intended use population in the clinical study. The specimens should
represent the intended use specimens (i.e., intended use population as well as
specimen matrix) consistent with the clinical practice of the intended use sites.
Samples should span the measuring range of the WM. The samples should be
obtained at a minimum of 3 independent testing sites that are representative of both the
intended patient population and the intended operators. Between 1 and 3 intended
operators per site should be selected, and at least 9 operators should participate. For
example, you could use 6 sites with 1 or 2 operators per site to reach the minimum of
9 operators. The patient samples should be as equally distributed among the
operators as possible. The samples should be collected from consecutive patients
22

Contains Nonbinding Recommendations
over an appropriate period of time (this period may depend on the specific clinical
site, the prevalence of disease, or other factors). We suggest a 1 month period may be
useful and recommend not less than 2 weeks. The goal is to assess how well the
device you propose for CLIA waiver works in practice, recognizing that operators
may have other duties. "
You should ensure that the 360 patient samples span the measuring range of the device
and adequately represent all possible values of the CM test consistent with the
measuring range of the WM (See [6 and 7]). You should divide samples into low,
medium and high medically relevant intervals (according to the CM values) with
approximately the same number of patients in each. One hundred and twenty
observations are recommended within each interval to estimate the total analytical
error within each of these intervals (See [6]). A suggestion concerning the distribution
of CM values in your data may be found for some analytes in Table 1 of [7].
We believe that actual patient specimens provide the best assessment of a device
proposed for CLIA waiver in the hands of intended operators when available.
However, in some cases this might be impractical (e.g., due to low prevalence or where
some concentration levels may be rare). Samples might be distributed insufficiently to
challenge the measuring range, or to generate enough values near medical decision
points. In such cases, it may be appropriate to supplement up to 120 actual patient
specimens with banked samples. If banked samples are not available, individual
spiked or diluted patient samples should be used (spiked, diluted, or otherwise
contrived samples used in the study should be individual samples, i.e., they should not
be aliquots from a single pool). In some cases, use of otherwise contrived matrixspecific specimens may also be appropriate. The matrix of any of these alternative
specimens should be the same as that specified by the intended use of the device. You
should describe how you prepared and determined the assigned value for any
alternative specimens you used.
Each sample should be split in two parts. If the sample cannot be split into parts, then
a second sample from the same patient should be collected within a suitable time
interval. (In this case, if the order in which the samples are collected impacts the
results of testing, we strongly encourage you to consult OIVD concerning your
clinical study design.) One part (or sample) should be tested by the intended operator
as a study participant using the WM, and the other part (or sample) should be tested by
a laboratory professional using the CM. Since the calculations of total analytical
error of the WM include the imprecision of the CM as part of the total analytical error
of the WM, it is recommended that you obtain replicate measurements with the CM if
the random error (expressed as a standard deviation or a coefficient of variation (CV))
of the CM is more than one third of the random error of the WM (see [5]). You
should then replace individual CM results with the average value of the CM in all
subsequent calculations.

Contains Nonbinding Recommendations

b. Statistical Analyses
You should calculate the Total Analytical Error of the WM (based on analyses of
differences) (see [6]) and compare the WM results with the CM results by an
appropriate regression analysis (see [7]).

(1) Descriptive statistics
You should provide the following information based on each site separately, as
well as a combined analysis over all sites:
• Descriptive statistics for both the WM and CM results, including the
number of results, mean, standard deviation (SD), minimum, median, and
maximum, and side by side box-and-whiskers plots for both the WM and
CM values (see [8]).
•

Scatter plot of the results, where the WM results are on the Y-axis and the
CM results are on the X-axis. Both axes should have the same scale, and
the line of identity (y=x) should be presented. The same scale on the axes
should be applied to the data from each site.

(2) Total Analytical Error
For purposes of this guidance, we use the term "total analytical error" as an
interval that contains a specified proportion (specifically 95%), of the distribution
of differences between the WM and CM values. Total analytical error may also be
expressed in terms of relative differences. Relative differences are defined as:
the WM result minus the CM result, divided by the CM result. (For details on the
total analytical error, see [6].) We recommend that you provide the following
information (combined over all sites) concerning total analytical error:
•

For every sample tested by the WM and CM, you should calculate a
difference (WM result minus CM result). In a similar way, for every
sample, you should also calculate a relative difference: WM result minus
CM result, divided by CM result.

•

You should provide a difference plot with the CM values on the X-axis
and differences between WM and CM values on the Y-axis. You should
also provide another plot with the CM values on the X-axis and relative
differences on the Y-axis. (For details see [6].)

•

You should divide the measuring range into three medically relevant
intervals (low, medium, and high), as described earlier, in which each
interval contains approximately the same number of points (about 120).
(For details, see [6 and 7].) These ranges will be used for further analyses
(see bullet, below).

•

For each interval (low, medium, and high), you should calculate total
analytical errors based on analyses of differences or relative differences
(whichever is appropriate) for 95% of the differences. Usually,

Contains Nonbinding Recommendations
calculation of relative differences is more appropriate for intervals
containing high values and calculation of differences is more appropriate
for the intervals with low values. You should also provide a histogram of
the relative differences (percentages) or actual differences for each
interval. You should calculate both the mean and standard deviation of
these differences. You should identify the 2.5th and 97.5th percentiles
derived from the calculation of the total error for 95% of the differences.

(3) Regression Analysis
•

You should provide results of an appropriate regression analysis for each
site separately as well as a combined analysis over all sites. The
regression method you choose should account for the random errors
associated with the WM and CM. You should present results for the
combined data as well as separately for each site. A Deming regression
procedure or one of several similar methods may be appropriate (see [7, 9,
and 10]). Since the methods may vary in their assumptions, you should
include some justification of the choice of procedure in the application. If
the random error of the CM is negligible compared to the random error of
the WM, and the standard deviation of the random error of the WM is
roughly constant over the range of CM values, then ordinary least-square
regression analysis is also appropriate (see [7]).

•

You should provide the 95% confidence intervals of the slope and intercept
from the regression above. We recommend that appropriate data on slope
and intercept be provided for each site, and as combined data. You should
calculate the systematic bias at medically important points. (For details,
see Section 6.1 in [7].)

•

Alternative statistical approaches are welcome. We recommend they be
discussed with FDA prior to initiating a study

Please note that correlation alone is often inappropriately used to assess
agreement between the WM and CM, and we caution you against such an
approach.
If one of the medically important points of the WM in the hands of intended
operators includes the Limit of Blank and/or the Limit of Detection, then some
additional calculations for the samples with very low level of analyte may be
needed for appropriate comparison of the Limits of Blank and Limits of Detection
of the WM in the hands of intended operators to the CM (see [11]).

(4) Performance Criteria for Quantitative Tests
For the zone of the ATE, you should report:
• The percentage of the observations that fall within the ATE zone for the
low, medium, and high ranges of the CM. These percentages should
approach 95% for each range of the CM.

Contains Nonbinding Recommendations
•

The percentage of the observations over the entire measuring range that
fall within the ATE zone, with a lower one-sided 95% confidence bound.
This lower bound should exceed 92%. This is because, for a sample
consisting of 360 observations with 95% of observations (342 out of 360)
falling in the ATE zone, the lower one-sided 95% score confidence bound
is above 92%. For details about confidence intervals, see [12-15] and
Statistical Notes in Appendix A.

For the zone of the LER, you should report:
• The percentage of the observations that fall within the zone for the low,
medium, and high ranges of the CM.
•

The percentage of the observations over the entire measuring range that
fall within the zone, which has an upper one-sided 95% confidence bound.
This upper bound should be below 1%. This is because, for a sample
consisting of 360 observations and zero observations (0 out of 360) falling
in the LER zone, the upper one-sided 95% confidence bound is below 1%.

C. Clinical Studies for Tests with Qualitative Results
There are two parts to the clinical study design for qualitative data. The first part (discussed in
Section C2a, below) is a method comparison and is similar to the study design discussed
above for quantitative results. The objective in this part is to compare the CM and WM in
terms of performance. The second part (discussed in Section C2b, below) is designed to
identify problems that may occur when the analyte is near the cutoff region of the CM, where
a cutoff is defined as the threshold at which the device differentiates a positive from a negative
outcome. As in the case of a test with a quantitative outcome, the CM for the CLIA waiver
clinical studies for qualitative tests should be performed in a laboratory setting by a laboratory
professional.

1. Selection of the Comparative Method (CM)
The following CMs are presented in the order in which we recommend you select them if
available. Thus, we recommend you use a CM of type A if available. If a CM of type A
is not available, then we recommend that you use a CM of type B. If neither a CM of
type A, nor a CM of type B is available, then we recommend you use the best available
traceable method (CM of type C) and, similarly, for CMs of types D and type E. If you
choose a comparative method other than type A or B, you should provide justification for
this in your application. If you are considering use of a CM of types C, D, E, or another
well-documented method, we strongly recommend that you contact OIVD prior to
conducting the clinical study in support of a CLIA Waiver application.
•

CM of type A: A quantitative reference method such as those outlined in the
section on quantitative tests with the appropriate cutoff value for the positive and
negative results.

•

CM of type B: A qualitative reference method (for examples of qualitative
reference methods, consult with OIVD).

Contains Nonbinding Recommendations
•

CM of type C: A quantitative method with measurement results traceable to
higher-order references such as that outlined in the section on quantitative tests
with the appropriate cutoff value for the positive and negative results.

•

CM of type D: A qualitative method with measurement results traceable to
higher-order references.

•

CM of type E: A qualitative method that was tested by reference specimen panels
(e.g., panels of samples prepared by well recognized institutions, such as WHO,
CDC, NIST). We recommend that you contact OIVD for input concerning your
choice of specimen panels.

In any case, you should provide a description of the specific CM, including 1) intended
use population/target population (those individuals for whom the test (CM) is intended),
2) condition of interest/target condition (a particular disease, a disease stage, health status,
or any other identifiable condition or characteristic of interest within a subject), 3)
reference standard (gold standard) for establishing the presence or absence of the condition
or characteristic of interest, and 4) diagnostic accuracy of the test (as diagnostic sensitivity
and specificity, positive and negative predictive values with prevalence of condition of
interest) (for details, see [5]) with sources for this information (published literature,
labeling and others).
For the CMs of type C and D, you should provide information supporting traceability,
e.g., a description of references that were used to establish the traceability of the
calibrators with values and uncertainties on each of those references and other relevant
information concerning traceability (for details, see [3]).

2. Clinical Study Design
a. General Approach for Method Comparison
As in the study for tests with quantitative results, the samples should be obtained at a
minimum of 3 intended use sites that are representative of both the intended patient
population and the intended operators of the WM. Between 1 and 3 intended operators
at each site should be selected. However, a minimum of 9 operators should
participate. Thus, you could use 6 sites with 1 or 2 operators per site to reach the
minimum of 9 operators. The patient samples should be approximately equally
distributed among the operators. The samples should be collected from consecutive
patients over an appropriate period of time. (This period may depend on the specific
clinical site, the prevalence of disease, or other factors.) The goal is to assess how
well the device proposed for CLIA waiver works in practice, recognizing that
operators may have other duties. A one month period for the clinical study to support
CLIA waiver may be useful, especially when the prevalence rate is low, and we
recommend not less than 2 weeks. The goal should be to have a minimum of 120
samples positive by CM and a minimum of 120 samples negative by CM. However,
the overall sample size may well exceed 240 since one cannot anticipate the CM
outcome for each patient’s data at the time of sample collection. Each operator
should observe a minimum of 5 positive and 5 negative samples. However, we stress
that the samples should be masked with respect to the CM outcomes. If some sites
have low prevalence rates, or if the overall availability of positive samples for novel
27

Contains Nonbinding Recommendations
or rare diseases is low, different study designs and corresponding statistical analyses
can be considered. We recommend that you contact OIVD for feedback prior to
conducting a study, especially if it is based on an alternative design.
Tests for low prevalence diseases can present challenges in terms of obtaining at least
120 samples positive by the CM. Some approaches to solving these challenges include
expanding the duration of the CLIA waiver study, or including a site with higher
disease prevalence among the study subjects. In some cases, the seasonal variations in
the prevalence of disease should be taken into account for the study design. We
believe that actual patient specimens provide the best assessment of a device in the
hands of intended operators. However, in some situations, such as when a disease has
low prevalence, you may supplement prospective patient samples with neat
(unaltered), banked samples. If neat, banked patient samples are not available, it may
be acceptable to supplement the unaltered patient samples with alternative samples
such as individual spiked or diluted patient samples. In some cases, other types of
contrived samples may also be applicable. (Spiked, diluted, or otherwise contrived
samples used in the study should be individual samples, i.e., they should
not be aliquots from a single pool). In any of these cases, the sample matrix should
be the same as that of the intended use patient samples. No more than one third of the
positive and no more than one third of the negative samples (when positive and
negative are determined by the CM) should be banked or artificially contrived. In
most cases, banked or contrived samples will be used when needed to ensure that
samples are distributed across the assay range. Therefore, in most cases, these should
include near-cutoff samples. In your CLIA waiver application, you should describe
the preparation and origin of these samples and justify the need for using them.
Each sample should be split in two parts: one part should be tested by the intended
operator, employing the WM; the other part should be tested by a laboratory
professional employing the CM. This is generally feasible if there is sufficient
sample available that can be split for comparisons between the two methods. If the
sample cannot be split into parts, then a second sample from the same patient within a
suitable time interval should be obtained. If the order in which the samples are
collected impacts the results of testing, we encourage you to consult OIVD
concerning your study design.
Statistical analysis of method comparison results for qualitative tests:
For the positive and negative samples, as determined by CM, you should calculate
positive and negative percent agreement estimates along with lower two-sided 95%
confidence bounds for each. You should perform calculations for each site
individually and, if appropriate, for all sites combined. (See [12-15] and Statistical
Notes concerning analysis of percentages and the formulas for the score confidence
bounds in Appendix A.)

Contains Nonbinding Recommendations

b. Determining Device Performance with Analyte Concentrations
Near the Cutoff
We recommend the following study design for determining performance with samples
containing analyte concentrations near the cutoff. This study design and the
principles behind it are described in detail in User Protocol for Evaluation of
Qualitative Test Performance; Approved Guideline. CLSI document EP12-A [5].
The objective of this study should be to demonstrate that the interpretation of positive
and negative (or presence or absence) for the analyte is essentially the same for the
WM and the CM near the cutoff. In order to determine the WM performance with
analyte concentrations near the cutoff, you should prepare two types of sample
material: one should correspond to a sample that is a weak positive using the CM and
the other should correspond to a sample that is a weak negative using the CM.
(1) Preparation of Weak Positive and Weak Negative Samples (as measured
by the CM)
In concept, for a theoretically large number of the individual tests, a sample with an
analyte concentration at the cutoff value of the CM produces results above the
cutoff 50% of the time (and results below the cutoff 50% of the time). It follows
that for a given number of tests performed on a sample with a concentration larger
than the cutoff concentration, positive results would be expected more than 50%
of the time. In this guidance we refer to an analyte concentration at which
repeated tests are 95% positive, as a weak positive concentration (or “C95”
concentration). We refer to a sample concentration below the cutoff at which
repeated tests are 95% negative (or 5% positive) as a weak negative concentration
(or “C5” concentration). Note that if the limit of blank (LoB) is used as a cutoff,
then the concentration C95 is a limit of detection (LoD) (see [11]).
If the CM is a quantitative method with the appropriate cutoff value, then the weak
positive (C95) and weak negative (C5) concentrations can be evaluated from
precision studies of the CM for the concentrations near the cutoff (see Statistical
Notes in Appendix A). If the CM is a qualitative method (for example, it requires
professional operator interpretation(s) in detecting the presence or absence of
color in a solution or on a test strip), then the weak positive and weak negative
concentrations of CM can be estimated through dilution studies. For these
dilution studies, you should prepare a series of samples (patient samples or
sample pools) with concentrations slightly greater than and slightly less than the
cutoff. Additional samples that range from strongly positive to samples that do
not have analyte present should also be included. You should prepare multiple
samples at each concentration in the cutoff series and ensure that the samples are
masked to the professional operators who will participate in the study. The
professional operators of the CM should perform the test according to the
instructions for use. The intent is to find analyte concentrations where the
indicator (a colored line or solution) is observed 95% (weak positive) or 5%
(weak negative) of the time by the professional operators of the CM. We

Contains Nonbinding Recommendations
recommend that you plot the data as indicated in the graph below: Percent of
Positive Results vs. Concentration by the CM (or Dilution Factors) and find the
concentrations (or dilution factors) which correspond to 95% and 5% of the
positive results by model fitting. We recommend that you contact OIVD prior to
conducting the dilution studies for determination of weak positive (C95) and weak
negative (C5) concentrations as measured by the CM if there are concerns about
accepted dilution factors.

Cutoff

Weak Negative
(C5)

Weak Positive
(C95)

Figure 2. Example of Percent Positive Results by the CM for the Concentration
Near the Cutoff
(2) Performance of the WM with Weak Positive and Weak Negative
Concentrations
For each sample type, you may need to pool patient samples or dilute a specimen
in order to create a sufficient amount of material for 60 aliquots. The samples
used should mimic the type of samples intended for use with the WM as closely
as is feasible. In your waiver application, you should provide information on how
you prepared each of these samples. You should select 3 intended use sites for
this part of the study.

Contains Nonbinding Recommendations
•

Prepare 60 aliquots of one sample with the weak positive concentration,
defined to be a concentration at which the professional operators of the
CM obtained positive results 95%-99% of the time.

•

Prepare 60 aliquots of a weak negative sample, defined to be a
concentration at which the professional operators of the CM obtained
correct results 95-99% of the time. For some tests, when the cutoff value
is set in such a way that only samples with no analyte produce negative
results 95%-99% of the time (the cutoff of the CM is the limit of blank),
the prepared aliquots should not have any analyte.

•

These 60 weak positive aliquots and 60 weak negative aliquots should be
approximately equally distributed across all operators with 20 samples of
each type at each of the three participating sites. You should ensure that
the labels on the aliquots are masked as to their true designation, and that
each aliquot has a unique label. You should also ensure that samples are
tested in random order during the study.

(3) Statistical Analysis for Results of Analyte Concentrations near the Cutoff:
•

For each of the two types of aliquots (weak positive and weak negative),
you should calculate the percent of positive results for the weak positive
sample and the percent of negative results for the weak negative sample
with the two-sided 95% confidence intervals for both. In the waiver
application, you should provide these results for each site and overall.

•

We recommend that you compare the percents of positive results for the
weak positive sample among the three sites (20 measurements per site) by
using a Fisher-Freeman-Halton test (a generalization of Fisher’s exact test,
see [16]). You should conduct a similar test for the weak negative results.

3. Performance Criteria for Qualitative Tests
For the first parts of the study design, described in Section C2a, above:
The observed positive and negative percent agreements between the test proposed for
waiver and the CM should be 95% or greater; but in all cases the score lower
confidence bound for the two-sided 95% lower confidence bound should be equal to
or greater than 89% (see [12-15] and Statistical notes in Appendix A).
In some cases, a higher percent agreement and a higher value for the lower
confidence bound may be needed to reasonably assure that the WM is “accurate.”
In some cases (for example, when the CM is a reference method for the analyte), a
lower percent agreement and a lower value for the lower confidence bound may be
medically acceptable with sufficient risk/benefit justification.
We also recommend that you report any invalid results obtained. This can be summarized
using a table or graph that captures the number of times a sample and/or a specimen
needed to be tested in order to obtain a valid result.

Contains Nonbinding Recommendations

For the second parts of the study design, described in Section C2b, above:
The percent of positive results for the 60 aliquots of the prepared, weak positive
samples should be close to 95%. That is, approximately 57 out of 60 of these samples
should yield positive results. The percent negative results for the prepared weak
negative samples should also be approximately 95%. Any differences found between
the CM and WM performances near the cutoff should be investigated to determine
what percentage of the intended use population would be affected by this difference.
The differences in percent of positive results among the three sites for the weak
positive sample and differences in percent of negative results among the three sites
for the weak negative sample should not be clinically or statistically significant at
α=0.05.

VI. LABELING FOR WAIVED DEVICES
In order to ensure that the device is "simple" and has "an insignificant risk of an erroneous result,"
your labeling should contain instructions for accurately running the test and reporting results and
be written at a level appropriate for the intended operator, which for a "simple" test should be at a
7th grade reading level or lower. It may be helpful for you to refer to the following documents.
They are available on the Internet as shown:
• "Write it Right," http://www.fda.gov/cdrh/dsma/897.pdf
•

"Medical Device Use-Safety: Incorporating Human Factors Engineering into Risk
Management," http://www.fda.gov/cdrh/humfac/1497.html

•

"Guidance on Medical Device Patient Labeling, Final Guidance for Industry and FDA
Reviewers" http://www.fda.gov/cdrh/ohip/guidance/1128.html

You should include your proposed labeling, including Quick Reference Instructions, package
insert, and outer labels in your waiver application. (Note that labeling for in vitro diagnostic
devices must meet all applicable labeling requirements as stated in 21 CFR 809.10(b).)
In addition, the package insert for waived test systems should include additional information
appropriate for untrained operators, including 8:

•

Facilities performing testing must have a CLIA Certificate of Waiver. 42 USC
263a(c)(2). Also, the labeling should identify your test system as waived and note that all
applicable state and local laws must be met.

•

A statement that laboratories eligible for a Certificate of Waiver must follow the test
system instructions, including use with only the waived specimen type(s), instructions for
limitations/intended use, and performance of QC testing as a failure-alert mechanism. (42
CFR 493.15(e).) You should state that any modification to the test or the manufacturer’s
instructions will result in the test being classified as high complexity.

See also Appendix B for more detailed recommendations.
32

Contains Nonbinding Recommendations
•

Instructions for QC integrated with procedural instructions for performing the test in both
the package insert and the Quick Reference Instructions.

•

Results of clinical studies that supported test waiver (in consultation with FDA). The
performance information in your labeling can be finalized after study results are reviewed
and the test is determined to be waived by FDA.

A. Quick Reference Instructions (QRI)
You should include Quick Reference Instructions (QRI) in your application. Preferably, the
QRI should be laminated and attached to the test system. These instructions should be clear,
easy to understand, in a readable font of 12 or greater, and, when possible, include pictures.
You should write instructions at no higher than a 7th grade reading level. For
recommendations on what to include in the QRI, see Appendix B. You should include all the
items in that appendix that are applicable to your test system as well as any other appropriate
information specific to your test system.

B. Quality Control (QC) Labeling Recommendations
Instructions should clearly explain why QC is needed and emphasize the value of external
control testing at regular intervals for ensuring operator competency, and reagent and
instrument (when appropriate) integrity. Instructions on how to perform control procedures
using external controls are always recommended and are critical if you are using them as a
failure alert to help ensure “insignificant risk of an erroneous result.” Instructions relating to
procedures used for QC should be integrated within the instructions for performing the test
and should include the following:
•

Step by step information on how to test control material, including testing frequency
and concentration of materials.

•

How to read results of control procedures.

•

How to determine if results are invalid (for example, for tests with an internal
procedural control line, the test is not valid if the line is not present).

•

Actions to take when control results are out of range or invalid. For example, the
directions should direct the user to call technical assistance when control results are
out of range or invalid and state that the results should not be reported.

•

The limitations on all control mechanisms, including procedural controls, which you
identified during the risk assessment. For example if your procedural controls only
test that a liquid was applied, this limitation needs to be communicated to the user.

Your explanations of QC systems should include a description of what is being measured by
all elements of both internal and external quality controls for a particular test system. To aid
in addressing QC problems, you should provide a toll-free telephone number for technical
assistance. We recommend that QC instructions take into account information obtained
during the clinical studies (Section V), as well as results of flex studies and validation and/or
verification studies under conditions of stress (Section IV).

Contains Nonbinding Recommendations

You should include discussions of benefits and limitations of the various device controls.
For example, for a unitized test, the following may be appropriate and could be indicated in
bold in the labeling for emphasis:
When you run test (xyz), you should always see an extra line (the control line)
in addition to the test line. This extra line lets you know that you added the
correct sample volume. Good laboratory practice recommends that you also
use additional positive and negative control materials that are not built in to the
test. (They are external controls.) You can order external controls from (abc).
External controls can monitor test features such as whether test reagents are
working properly or whether the test was performed correctly. If any of the
controls do not perform as expected, do not report patient results. Review the
instructions to see if the test was performed correctly, and then repeat the test.
If the controls still do not give the expected results, contact technical
assistance before testing patient samples.
Examples of possible minimum frequency recommendations for running external controls are
listed below. You should base your specific recommendations on data from your studies.
•

Each new lot.

•

Each new shipment of materials even if it is the same lot previously received.

•

Each new operator (i.e., operator who has not performed the test recently).

•

Monthly, as a check on continued storage conditions.

•

When problems (storage, operator, instrument, or other) are suspected or identified.

•

If otherwise required by your laboratory’s standard QC procedures.

C. Educational Information
As part of an overall plan to ensure that the likelihood of erroneous results by the user is
negligible, manufacturers should ensure that laboratories can read and understand the
labeling, including test performance and limitations (as reflected by the analytical and clinical
studies). We encourage you to consider innovative mechanisms to provide technical
assistance to laboratories and to ensure they understand the directions in the labeling, e.g., a
downloadable version of the test procedure with computer animation showing the correct
steps for performing the test. We also recommend that manufacturers assist laboratories
performing waived tests in becoming better educated on proper laboratory techniques. For
example, we recommend that you participate in the development and promotion of good
laboratory practice guidelines by developing training and education programs for the end
operator. We also encourage you to incorporate proficiency testing, when feasible, and to
promote laboratory participation in proficiency programs. In addition, we recommend that
you include good laboratory practice information in the package insert, in accessory
educational or technical material, and through the development of formal educational training
programs. We recommend that you provide information on the following topics to operators:

Contains Nonbinding Recommendations
•

Importance of retaining a current version of the package insert with the latest revision
date.

•

Importance of following the test instructions in the sequence given in the instructions.

•

Need for proper operator training and retraining in order to maintain competency.

•

Need for users to follow all instructions related to storage, preparation, and expiration
dating in order to maintain adequate test performance.

•

Importance of documenting results and maintaining records as needed for proper
performance of the test and patient management.

•

General purpose of quality control and value of using quality control within a broader
system of quality assurance.

•

Common errors – errors that are likely to occur. For example, incorrect timing that
may be mitigated by use of a mechanical timer with an alarm.

We recommend that you consult CDC documents as a good source of information regarding
good laboratory practices for waived testing sites, survey findings from testing sites holding a
Certificate of Waiver under the Clinical Laboratory Improvement Amendments of 1988, and
Recommendations for Promoting Quality Testing. (see [17])

VII. SAFEGUARDS FOR WAIVED TESTS
1. FDA is also recommending that manufacturers of waived tests put a brief description of
the MedWatch medical products reporting program along with the MedWatch phone
number (1-800-FDA-1088), and fax numbers (1-800-FDA-0178), and website
(www.fda.gov/medwatch) in the package insert. You may also describe how the
MedWatch program works, which failures should be reported to both the company and
FDA, and when failures should be reported to ensure proper tracking and reporting of
waived testing issues.
2. Manufacturers of devices must maintain and implement medical device reporting
procedures as required by 21 CFR 803.17 and must establish and maintain medical
device report (MDR) event files as required by 21 CFR 803.18. See also 21 CFR 803.20,
21 CFR Part 803, Subpart E.
3. Manufacturers of devices must submit MDRs of individual adverse events as required by
21 CFR 803.10(c).

VIII. Paperwork Reduction Act of 1995
This guidance contains information collection provisions that are subject to review by the
Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995 (44
U.S.C. 3501-3520).
The time required to complete this information collection is estimated to average 780 hours
per response, including the time to review instructions, search existing data sources, gather

Contains Nonbinding Recommendations
the data needed, and complete and review the information collection. Send comments
regarding this burden estimate or suggestions for reducing this burden to:
Carol Benson,
Center for Devices and Radiological Health (HFZ-440),
Food and Drug Administration,
2098 Gaither Road,,
Rockville, MD 20850.
This guidance also refers to previously approved collections of information found in FDA
regulations. The collections of information in 21 CFR part 803 have been approved under OMB
control number 0910-0437; and the collections of information in 21 CFR part 809 have been
approved under OMB control number 0910-0485.
An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number
for this information collection is 0910-0598 (expires February 28, 2010).

REFERENCES
1. CLSI (former NCCLS). Quality Management for Unit-Use Testing; Approved Guideline.
CLSI document EP18-A (ISBN 1-56238-481-3). CLSI, 940 West Valley Road, Suite
1400, Wayne, Pennsylvania 19087-1898 USA, 2002.
2. ISO 17511: 2003 (E) In vitro diagnostic medical devices – measurement of quantities in
biological samples – metrological traceability of values assigned to calibrators and
control materials.
3. CLSI (former NCCLS). Metrological Traceability and Its Implementation; A Report.
CLSI document X5-R [ISBN 1-56238-598-4]. CLSI, 940 West Valley Road, Suite 1400,
Wayne, Pennsylvania 19087-1898 USA, 2006.
4. Fraser,C.G. Biological Variation: from Principles to Practice, AACC Press, 2001
5. CLSI (former NCCLS). User Protocol for Evaluation of Qualitative Test Performance;
Approved Guideline. CLSI document EP12-A [ISBN 1-56238-468-6]. CLSI, 940 West
Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2002.
6. CLSI (former NCCLS). Estimation of Total Analytical Error for Clinical Laboratory
Methods; Approved Guideline. CLSI document EP21-A (ISBN 1-56238-502-X). CLSI,
940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2003.
7. CLSI (former NCCLS). Method Comparison and Bias Estimation Using Patient Samples;
Approved Guideline – Second Edition. CLSI document EP9-A2 (ISBN 1-56238-000-0).
CLSI, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2000.
8. Fisher LD, Van Belle G. Biostatistics. A Methodology for the Health Sciences 1993, John
Wiley & Sons, Inc.
9. Linnet K. Evaluation of the linear relationship between the measurements of two methods
with proportional errors. Statistics in Medicine 1990; 9:1463-1473.

Contains Nonbinding Recommendations
10. Martin RL. General Deming Regression for Estimating Systematic Bias and Its
Confidence Intervals in Method-Comparison Studies. Clinical Chemistry 2000;
46(1):100-104.
11. CLSI (former NCCLS). Protocols for Determination of Limits of Detection and Limits of
Quantitation; Approved Guideline. CLSI document EP17-A (ISBN 1-56238-551-8).
CLSI, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2004.
12. Clopper CJ, Pearson E. The use of confidence or fiducial limits illustrated in the case of
Binomial. Biometrika 1934;26:404-413.
13. Altman D.A., Machin D., Bryant T.N., Gardner M.J. eds. Statistics with Confidence. 2nd
ed. British Medical Journal; 2000.
14. Agresti, A. and Coull, B.A. Approximate is better than “exact” for interval estimation of
binomial proportions. The American Statistician 1988;52:119-126.
15. Newcombe, R.G. Two-sided confidence intervals for the single proportion: comparison
of seven methods. Statistics in Medicine 1998; 17: 857-872.
16. Agresti, A. Categorical Data Analysis. 1990, John Wiley & Sons, Inc.
17. Centers for Disease Control and Prevention. Good laboratory practices for waived testing
sites; survey findings from testing sites holding a Certificate of Waiver under the Clinical
Laboratory Improvement Amendments of 1988 and Recommendations for Promoting
Quality Testing. MMWR 2005; 54 (No.RR-13):1-25.

Contains Nonbinding Recommendations

Appendix A: Statistical Notes
1) The following are additional recommendations for performing statistical analyses of
percentages or proportions.
The 95% confidence interval for positive percent agreement and 95% confidence interval
for negative percent agreement yield a rectangular region that contains both population
parameters with confidence level 90% (≈ 0.95 x 0.95). A 95% confidence region for both
population positive percent agreement and negative percent agreement is
constructed from 97.5% (=√0.95) univariate confidence intervals.
Confidence limits for positive percent agreement and negative percent agreement can be
calculated using formulas for calculating a confidence interval for a binomial proportion.
There are several different methods available, and they can be obtained from many
software packages or can be obtained from published tables. We suggest that either a
score method described by Altman, et al. (see [13]) and Agresti and Coull (see [14]) or a
Clopper-Pearson Method (see [12]) be used. An advantage with the score method is that it
has better statistical properties (see [15]) and it can be calculated directly. Score
confidence bounds tend to yield narrower confidence intervals than Clopper-Pearson
confidence intervals, resulting in a larger lower confidence bound. So with n=120 samples
and 114/120=95% agreement, the score lower confidence bound is 89.5% whereas the
Clopper-Pearson lower confidence bound is 89.4%. In this document we have illustrated
the reporting of confidence intervals using the score approach. For convenience we
provide the formulas for the score confidence interval for a percentage
here. Note that the lower bound of a two-sided 95% score confidence interval is the same
as the lower bound of a one-sided 97.5% score confidence interval.
A 2-sided 95% score confidence interval for positive percent agreement or negative
percent agreement is calculated as:

[100 % (Q1 − Q2 ) / Q3 , 100 % (Q1 + Q2 ) / Q3 ] ,
where the quantities Q1, Q2, and Q3 are computed from the data using the formulas
below.
For positive percent agreement:
Q1 = 2 A + 1.96 2 = 2 A + 3.84
2
Q2 = 1.96 1.96 + 4 AC /( A + C ) = 1.96 3.84 + 4 AC /( A + C )

Q3 = 2( A + C + 1.96 2 ) = 2( A + C ) + 7.68

where A is the number of samples that are positive using both the WM and the CM and C
is the number of samples that are negative via the WM but positive via the CM.
For negative percent agreement, similar equations are needed:
Q1 = 2D + 1.96 2 = 2 D + 3.84

Contains Nonbinding Recommendations
Q2 = 1.96 1.96 2 + 4 BD /( B + D ) = 1.96 3.84 + 4 BD /( B + D )
Q 3 = 2( B + D + 1.96 2 ) = 2( B + D ) + 7.68

D is the number of samples that are negative via the CM and WM whereas B is the
number of samples that are negative via the CM and positive via the WM.
In the formulas above, 1.96 is the 97.5th quantile from the standard normal distribution
that corresponds to 95% two-sided confidence. The 97.5% lower confidence bound is the
smaller of the two values obtained using the 95% two-sided confidence interval formulas
above.
In Section C2 we refer to calculation of a one-sided 95% lower confidence bound for the
percent of observations that fall within the ATE or LER. In that instance, one should use
1.645 in place of 1.96 in the formulas above.
2) Evaluation of C5 and C95 using the CM from precision studies:
If the standard deviations (SD) in the precision studies of the CM for concentrations
around the cutoff value are almost constant, then
C95 = cutoff +1.645 x SD and
C5 = cutoff – 1.645 x SD.
For example, if the cutoff value =1.00 and the SD around the cutoff is approximately
0.10, then C95 is approximately 1.16 and C5 is approximately 0.84. In other words, a
sample with actual concentration of 1.16 produces positive results (above 1.00)
approximately 95% of the time and a sample with actual concentration of 0.84 produces
negative results (below 1.00) approximately 95% of the time.
If the coefficient of variation (CV) in the precision studies of the CM for concentrations
around the cutoff value are almost constant, then C95 = cutoff + 1.645 x CV x C95 and C5
= cutoff – 1.645 x CV x C5. From here,
C95 = cutoff / (1 – 1.645 x CV) and
C5 = cutoff / (1 + 1.645 x CV).
For example, if the cutoff value =1.00 and %CV around the cutoff is approximately 10%
(i.e., CV=0.10), then C95 is approximately 1.20 and C5 is approximately 0.86.

Contains Nonbinding Recommendations

Appendix B: Labeling
SPECIFIC LABELING RECOMMENDATIONS FOR WAIVED DEVICES
QUICK REFERENCE INSTRUCTIONS (QRI) WITH PICTURES AND DIAGRAMS
should generally contain the following and any other sections appropriate for your specific
device. The QRI should be written at no higher than a 7th grade reading level:
The name of the test and a statement that labs with a Certificate of Waiver may use it.
A statement clearly listing the specimen type, e.g., this test is only waived for urine specimens.
A statement that users should read the complete test procedure, including recommended QC
procedures, before performing the test and that they should refer to the package insert for more
complete information. If appropriate, a statement that users should perform control procedures
before performing the test.
A statement that laboratories with a Certificate of Waiver must follow the manufacturer's
instructions for performing the test. 42 CFR 493.15(e)(1)
Step-by-step test instructions. Include, as appropriate: physical environmental
specifications/conditions for test performance; specifications for specimen collection, handling,
storage, and preservation; preparation of reagents and control materials; storage of reagents and
control materials; and calibration procedures. Utilize diagrams and flowcharts to illustrate how
to run the test when helpful.
Step-by-step instructions for all control procedures, including frequencies, action to be taken if
control results are out of range or invalid or if other failure alert or fail safe mechanisms are
activated.
Interpretation of results, including diagrams on how to read and assess validity of test results and
control results.
A warning addressing color blindness when waived tests use color-coded reagents and/or
endpoints.
Safety considerations for test operation that particularly apply to untrained users.
Critical maintenance, such as cleaning (including safety considerations).
Telephone number to contact manufacturer for technical assistance or troubleshooting the test
system. Direct the user to call for assistance when the device or the control materials do not
work as specified by the manufacturer.
PACKAGE INSERT - Considerations for waived tests (in addition to any other
requirements specified in 21 CFR 809.10 and any other considerations specific for your
device type)
Identification of the test as CLIA waived, a statement that a Certificate of Waiver is required to
perform the test in a waived setting, and information on how users can obtain a certificate.
A statement that laboratories with a Certificate of Waiver must follow the manufacturer's
instructions for performing the test. 42 CFR 493.15(e)(1).
Test operation safety considerations that particularly apply to untrained users.
The physical environmental specifications/conditions for performing the test.
A warning addressing color blindness when waived tests use color-coded reagents and/or
endpoints.
Step-by-step operating instructions for performing the test that are integrated with instructions

Contains Nonbinding Recommendations
for all control procedures.
Action to be taken when no test result is obtained or when the result is out of the reportable
range.
Study results demonstrating how the test compares to a known method that is traceable to a
reference method if applicable.
A brief description and summary of the results from the waiver studies.
When appropriate, warnings about clinical errors that can occur even when the test result is
analytically correct.
Instructions indicating when and how additional testing should be done (e.g., in cases where
results should be confirmed by a reference procedure performed by an appropriately certified
laboratory).
Any other limitations, restrictions, and special considerations for your test system.
Appropriate QC recommendations or requirements (see below, “Quality Control Labeling
Recommendations”).
Information on reporting test system problems to the manufacturer and/or FDA. You should
include statements encouraging users to contact you and/or FDA so that you can track and
account for device problems. (www.fda.gov/medwatch)
Manufacturer contact information (phone number and the party to contact with a valid email
address if available).
Quality Control Labeling Recommendations
Step by step information on how to test control materials.
Frequency for testing control materials.
How to read control results and procedural controls.
Actions to take when control results are out of range or invalid.
Limitations of the device’s controls that were identified during the risk assessment.

Contains Nonbinding Recommendations

Appendix C: Definition of Terms as Used in this Document 9
“Accurate”

In this guidance document we use the term “accurate” tests (in quotes) to
refer to those tests that are comparable to a traceable method, for which the
results of measurements can be related to stated references. We write the
term “accurate” in quotes to denote that it is used in the context of CLIA and
this guidance. The definition of accuracy may be different in other contexts,
including those provided in other scientific standards. (For example, see
CLSI Harmonized Terminology Database).

Allowable total
error (ATE)

In this document, the allowable total error is the limit for the differences
between the WM and the CM or the WM and the RM that can be tolerated
without invalidating the medical usefulness of the test. These differences can
be expressed as percents of the CM values for high values and as a
concentration difference for low values.
Typically at least 95% of the observed differences should fall within the
established ATE zone.

Control material

Material used to verify performance characteristics of a medical device.

Control measure

Protective measure implemented for reducing risks. (Fail-safe mechanisms
and failure alerts are control measures.)

Control
procedures

Operational techniques and activities at the point of use to monitor the
performance of the device and fulfill the laboratory’s requirements for
quality. Any single control procedure might monitor all or part of the
measurement procedure, ranging from the collection of sample to reporting
the result of the measurement.

External control
material

Control material that is not built into the device. Typically this is in a similar
matrix as the intended use specimen and is processed using the same
procedures as patient specimens. External control materials for waived tests
should be ready to use or employ only very simple preparation steps, e.g.,
breaking a vial in order to mix liquid and dry components of the control
material.

Fail-safe
mechanisms

Mechanisms to ensure that a test system does not provide a result when test
conditions are inappropriate or when the result is based on faulty test
functioning. Also, measures that prevent improper operation of the device

Also see Harmonized Terminology Database at the CLSI website:
http://www.clsi.org/AM/Template.cfm?Section=Harmonized_Terminology_Database, for further
details and more general use of the terms

Contains Nonbinding Recommendations

Failure alert
mechanisms

(for example guides or channels that prevent improper strip placement).
Mechanisms that notify the operator of any test system malfunction or
problem. Failure alert mechanisms ideally include built-in controls or
checks. Procedures that use external control material can also be considered
failure alert mechanisms.

Flex studies

Studies performed using the device under conditions of operational stress.
These studies are performed to identify sources of error as part of the risk
assessment.

Hazards (for
IVD’s)

Potential source of harm (to a patient or test operator). For IVD’s, hazards
for patients are generally incorrect patient results or operator injuries.

Intended
operator (user)

In this guidance, intended operator (user) refers to a test operator with limited
or no training or hands-on experience in conducting laboratory testing (e.g.,
medical assistant, nurse, doctor, or an individual with no medical training).

Internal control

A control, or system check, built into the device system, i.e., the user does not
need to use additional reagents to perform that particular control process.

Laboratory
Professional

A person who meets the qualifications to perform moderate or high
complexity testing, such as a medical technologist (MT) or medical
laboratory technician (MLT).

Limit of Blank
(LoB)

The highest measurement result that is likely to be observed (with a stated
probability) for a blank sample;

Limit of
Detection (LoD)

The lowest amount of analyte in a sample that can be detected with (stated)
probability, although perhaps not quantified as an exact value.

Limits of
erroneous results
(LER)

In this guidance the limits of erroneous results are limits for the differences
between the WM and the CM or the WM and the RM. Results inside these
limits pose a risk to patient safety.

Matrix

The totality of components within a patient specimen, controls, or calibrator
other than the analyte.

Matrix effects

The influence of a property of the sample, other than the analyte, on the
device’s performance characteristics.
Examples: (a) The test values for a particular analyte in whole blood
collected via a venous sample may differ from those for a fingerstick. (b)
Control material in a matrix different from that of the specimen should be
tested to ensure that test performance is the same as that for the specimen.

Procedural
control

Controls or indicators to monitor whether specific aspects of the procedure
were performed correctly. Often, procedural controls are in the form of

Contains Nonbinding Recommendations
control lines on a single use cassette or dip devices and indicate whether
sufficient sample was applied. Procedural controls generally do not serve as
a control for the entire test system.
Quality control
(QC)

The entire set of procedures and system checks designed to monitor the test
method and the results to assure acceptable test system performance.

Quick Reference
Instructions

A short (usually one or two page) version of the test instructions, preferably
laminated, that can be posted. It contains instructions needed on a frequent
basis and directs operators to the package insert for topics such as
performance characteristics, long term maintenance instructions,
troubleshooting, and other more detailed instructions.

Reference
material

A preparation of the analyte whose concentration and purity are sufficiently
well-established and well-recognized for the material to be suitable for
calibration or value assignment.

Reference
method (RM)

A method which has been thoroughly investigated and has been shown to
yield values having trueness and precision of measurement such that the
method can be used to assess the trueness of other methods for the same
analyte or for assigning values to reference materials.

Risk analysis

Systematic use of available information to identify hazards and estimate risk.

Risk control

Process through which decisions are reached and protective measures are
implemented for reducing risks to or ensuring specified levels.

Risk evaluation

Judgment of whether acceptable risk has been achieved based on risk
analysis.

Risk
Management

Systematic application of management policies, procedures, and practices to
analyze, evaluate, and control risk (includes risk analysis, evaluation, and
control).

Source of error

A component of the device, measurement method, or operator practice that
can lead to device failure and, thus, create a risk for patients, operators, or
other individuals.

Total analytical
error

The combination of errors from all sources, both systematic and random.
It is often expressed in terms of an interval that contains a specified proportion
(e.g., 95%) of the distribution of observed differences between the WM and
CM values (for the CM of types A and B). Sometimes relative differences
(e.g., ((WM-CM)/(CM)) are used instead of differences.

Contains Nonbinding Recommendations
Trueness

The closeness of agreement between the average values obtained from a large
series of test results and an accepted reference value. The measure of
trueness is usually expressed in terms of bias. It is also often referred to as
systematic bias.

Unitized test
device

A self-contained test device to which a specimen is added directly and in
which all steps of the testing process occur. A unitized device is used for a
single test and must be discarded after testing.

File Type	application/pdf
File Title	Guidance for Clinical Laboratory Improvement Amendments of 1988 (CLIA) Criteria for Waiver; Draft Guidance for Industry and FD
Author	atd
File Modified	2013-04-18
File Created	2013-04-18