Finding the truth in laboratory testing: Commutability, traceability, and uncertainty of measurement

CONTINUING EDUCATION

To earn CEUs, visit www.mlo-online.com under the CE Tests tab.

LEARNING OBJECTIVES

1. Explain why it is important to identify the trueness of analytical results.
2. Define and discuss commutability, as it relates to measurement results.
3. Define and discuss traceability, as it relates to measurement error.
4. Define and discuss measurement uncertainty, as it relates to test performance.

Truth, trueness, true value: these terms have become part of the clinical laboratory lexicon over the past decade. We seek to know the “true” value of an analyte (measurand) in a patient sample. We strive to determine the “trueness” of our measurement systems by participating in PT/EQA. We test control materials alongside patient samples to verify that the test system is “truly” operating within specification. This leads to the question of whether the processed materials we use, such as control materials, PT/EQA samples, reference materials, and so on, “truly” reflect what is happening with patient samples. Part of the answer to this question lies in the topics of commutability, traceability, and measurement uncertainty.

The three topics are especially relevant at the moment, as there are national and international committees working to provide direction and guidance to clinical laboratories. At the same time, laboratory professionals may not understand when commutability of quality control materials should be evaluated, whether control material needs to be traceable for daily operations, or how uncertainty of measurement is relevant to reporting patient results.

This article provides an overview of the importance of each of these topics in the clinical laboratory. “Truth” in laboratory testing is central to contemporary discussions concerning commutability, traceability, and measurement uncertainty.

COMMUTABILITY

The question of what is the appropriate relationship between control performance and patient test results has been around since the introduction of antigen-antibody diagnostic methods in the 1990s. Laboratories have been perplexed by changes in control performance when implementing a new reagent kit lot that are not manifested in patient samples. This situation occurs primarily because control materials are processed materials. During the creation of control materials in a clinical laboratory or a manufacturer’s laboratory, patient samples (patient pools), base material (matrix of serum, plasma, urine, body fluid, or other material), or analyte (measurand) may be processed to achieve desirable effects, such as target values or stability for long-term use.

The Clinical and Laboratory Standards Institute (CLSI) defines a processed sample as “a sample that is prepared to be used to mimic one obtained from a patient. It is considered a processed sample if it has been modified in any way that causes it to be different from one obtained from a patient—for example, freezing, lyophilization, adding non-endogenous substances or stabilizers.”¹ Therefore, processed samples include control materials, EQA/PT samples, calibrators, certified reference materials, trueness controls, and altered patient samples.

These processes may include spiking measurands to achieve desired concentrations, using measurands not of human origin due to cost or a unique molecular structure, using recombinant measurands, adding preservatives and stabilizers to assure expected performance, or undergoing extreme heat and pressure to create lyophilized materials. Furthermore, some matrices are not human, but instead are contrived, albumin-based, or occasionally bovine.

Each of these processes can cause some degree of unwanted effects during testing; these are called matrix effects. When matrix effects are of a magnitude that is not seen in the patient samples, the material is considered to be “non-commutable.” But, is this a problem?

The general definition of commutability is “equivalence.” There are two similar definitions found in ISO standards ISO 15194² and ISO 17511³: “property of a given reference material, demonstrated by the closeness of agreement between the relation among the measurement results for a stated quantity in this material, obtained according to two measurement procedures, and the relation obtained among the measurement results for other specified materials” (ISO 15194); and “closeness of agreement between the mathematical relationship of the measurement results obtained by two measurement procedures for a stated quantity in a given material and the mathematical relationship obtained for the quantity in routine samples” (ISO 17511).

When two “different” samples are examined by the same method or when one sample is examined by two different methods and the results are the same within a specified tolerance, the materials are considered equivalent or commutable. Establishing equivalency or commutability (that is, identifying non-commutability) is a mathematical/statistical exercise and has value for determining whether control materials may be used to verify performance after a reagent lot change or whether patient samples are required. But there is one other facet of commutability relevant to control materials that must not be overlooked.

The primary purpose of control materials is to verify that the test system is performing within established specifications based on past performance of the control material. Control materials are not and never have been intended to verify the “trueness” of patient test results—only whether the test system is working within specification or not. The quality control process (regardless of whether the control material is quantitative or qualitative) is basically a pass/fail system. In this respect, commutability should only be an issue of whether to recalculate the performance specifications or focus on test system failure when a change in test system performance is verified; that is, control materials do not have to be statistically commutable to perform their intended purpose.

So, when do laboratory leaders suggest testing processed samples for commutability? Common instances when commutability experiments need to be performed on processed samples may include (but are not limited to) the following:

verifying that control materials can be used to verify the presence (or absence) of reagent lot-to-lot performance changes
determining when statistical process control ranges need to be adjusted after a reagent lot change
developing assurance by providers that EQA/PT samples are equivalent in performance to patient samples, particularly when results of testing EQA/PT samples are used to determine outlier performance or to de-certify the use of a method
verifying claims of “trueness” for manufactured materials (e.g., control materials).

Consider the following example: The laboratory has a new reagent kit lot. Before putting the kit into use, the laboratory needs to verify that the test system performs within specification after the new kit is put into use as it did before. Historically, laboratories have used control materials to verify performance before and after a kit or reagent lot change. Current laboratory best practices dictate that the laboratory should first run experiments before any need arises to verify whether the control materials are suitable for this purpose or exhibit matrix effects to the extent that would invalidate their use for such verifications: that is, a commutability study. If control materials are found to be fit for this purpose (commutable), they can be used to verify performance subsequent to a reagent lot change; if not, then patient samples must be used. CLSI EP14 examines the topic of commutability and explains the evaluation process with worked examples.

TRACEABILITY

For years, leaders in the laboratory profession have sought to achieve harmonization of diagnostic testing. The goal has been to achieve equivalent results for a single test among varying testing methods. The first approach was to create and implement an international regulatory standard (ISO) that would require calibrations of diagnostic devices to trace back to a recognized international reference measurement procedure. The standard for diagnostic tests is ISO 17511, and a companion standard for enzyme calibration is ISO 18153.⁴Essentially, these two standards are based on a series of calibrator manufacturing steps called a “calibration cascade.” Briefly, the cascade is represented in Figure 1. (Please refer to ISO 17511 for more detailed versions of the cascade.)

Figure 1. Representation of a manufacturer’s calibration cascade

There are three things that are important to know about ISO 17511 and ISO 18153.

First, the standard is intended for manufacturers or laboratories developing their own tests (LDTs) and applies only to the manufacture of calibrators and control materials that are intended for “trueness” determinations.
Second, the manufacturing of daily use control materials, such as those commonly in use in most laboratories, are exempt from the requirements of these two standards. It is recognized that these precision controls are not required to meet the requirements for traceability because they have no direct bearing on the production of patient test results. They simply provide an assessment of the system’s continued performance. Furthermore, since control materials are treated like patient samples, the result of the control material is traceable through the calibration cascade just like the patient test result.
Third, as illustrated, the measurement uncertainty of the calibrator increases with each manufacturing iteration reflecting the introduction of analytical variables contributing to error along the cascade.

A trueness control material is defined by ISO 17511 as “reference material that is used to assess the bias of measurement of a measuring system.” These materials are manufactured under a higher performance standard than other materials such as daily use controls, and they typically exhibit very little variability; for example, they have low imprecision and very low measurement uncertainty, which one would expect of a material manufactured under tightly controlled conditions. Consequently, when control manufacturers claim that their daily use control materials are trueness controls (that is, intended to measure trueness), they should supply the appropriate certificate of traceability and trueness that includes the reference method or material. If their material is not equivalent to a reference material in its manufacturing or its performance characteristics, then it is unlikely to be a trueness control material.

In practice, laboratories are likely to be primarily concerned about traceability only when selecting a new commercial test or test method or when developing their own tests or calibrators.

MEASUREMENT UNCERTAINTY

One might say that measurement uncertainty is the new kid on the block. It first gained the attention of clinical laboratories and accreditors with the 2003 publication of the first international laboratory practice standard, ISO 15189.⁵ This standard, and later versions in 2007 and 2012, require laboratories accredited to the standard to calculate their own measurement uncertainties and have this data available for each quantitative test performed in the laboratory. Attribute tests (for example; positive, negative) must also have measurement uncertainty calculated if the method has a quantitation step, such as an absorbance value for determining a cutoff. Unfortunately, there still is no companion standard from ISO that describes for laboratories the process and formulae to calculate the measurement uncertainty statistic, nor is there a publication of measurement uncertainty benchmarks for comparison purposes. As a quality initiative, laboratories that do maintain a record of calculated uncertainties should review those calculations at regular intervals to assure their continued relevance.

So, what value does measurement uncertainty provide? Measurement uncertainty is a range expressed by a percent (and sometimes a unit of measure) within which the “true value” of the measurand is expected to be. It can be helpful to a clinician when a laboratory test result is at or near an actionable value. The statistic can be useful in making decisions about health status (normal/abnormal), physiology (treatment effective/not effective), or comparison of two results on the same patient (Is the difference due to physiology, treatment, or an artifact of the test performance?). Measurement uncertainty can also be useful when a result is questioned and a repeat test is ordered and performed: Is the repeat value within or outside expectation? A repeat within the measurement uncertainty would go far in validating the original result reported.

One approach for calculating uncertainty originates within the science of metrology and is referred to as the GUM⁶ (Guide to Uncertainty of Measurement) approach. It is extremely detailed, uses advanced statistical tools, and requires examination of every possible contributor to variability—for example, the uncertainty associated with the fill of anticoagulant in collection tubes. Many in the industry regard this approach as onerous and unwarranted for medical laboratory testing, and the approach has been met with stiff resistance. However, if a laboratory develops tests or modifies a test, the GUM must be followed.

An alternate approach has been suggested and is being developed under the auspices of the ISO organization in Technical Committee 212. This standard is still in development as of this writing, and is expected to be based on information and statistics easily developed or obtained by the laboratory. Laboratories should also be aware that CLSI has published a guidance, C51,⁷ that examines a modified GUM procedure and an alternate approach using laboratory data to calculate uncertainty.

“To thine own self be true”

So said Polonius in Shakespeare’s Hamlet, and that advice might have relevance to this brief discussion of truth in laboratory testing. The laboratory must first and foremost be “true” to its stated goals and objectives by protecting its credibility with appropriate quality systems, using materials and devices with proven quality, and having a rigorous program of self-assessment and self-awareness. Only then can the laboratory assure the well-being of the patients it serves.

REFERENCES

EP14-A3. Evaluation of Commutability of Processed Samples; Approved Guideline – Third Edition. Clinical and Laboratory Standards Institute (CLSI). August 2014.
ISO 17511. In vitro diagnostic medical devices – Measurement of quantities in biological samples – Metrological traceability of values assigned to calibrators and control materials. 2003.
ISO 15194. In vitro diagnostic medical devices – Measurement of quantities in samples of biological origin – Requirements for certified reference materials and the content of supporting documentation.
ISO 18153. In vitro diagnostic medical devices – Measurement of quantities in biological samples – Metrological traceability of values for catalytic concentrations of enzymes assigned to calibrators and control materials, 2003.
ISO 15189. 2012 Medical laboratories – Requirements for quality and competence. 2012.
Evaluation of measurement data – Guide to the expression of uncertainty in measurement, BIPM, JCGM 100:2008, GUM 1995 with minor corrections. 2008.
C51. Expression of Measurement Uncertainty in Laboratory Medicine – Approved Guideline, Clinical and Laboratory Standards Institute (CLSI. 2012).

Greg Cooper, CLS, CQA, MHA, is a California-licensed clinical laboratory scientist, holds a Masters degree in Healthcare Administration, and recently retired after more than 30 years of service in the clinical laboratory and diagnostics industries. He is principal owner of W. Gregory Cooper LLC, an enterprise advising laboratories and diagnostics manufacturers on medical laboratory quality systems and requirements.