In today’s clinical lab environment, tests being utilized for the detection of SARS-CoV-2 antibodies are not only new to a lab but are also new for the manufacturers, the regulators, and the lab industry as a whole. Common sense and QC (quality control) experts tell us that our newest or weakest tests or instruments are where we need to concentrate on quality control. As such, antibody tests for SARS-CoV-2 certainly qualify for extra attention.
Serology tests as qualitative assays
One novel aspect for some labs is that these are qualitative tests. The most common definition of a qualitative test is one that produces binary results, often positive/negative. A better definition is that a qualitative test has only one medical decision point, and that medical decision point is the cutoff.
Also, the anti-SARS-CoV-2 test is a serology test, an assay that has been carefully designed and manufactured to provide one simple output – often a signal-to-cutoff (s/co) ratio or a color that indicates the presence of antibodies to this specific virus. The secret behind the simplicity is that humans can make lots of different antibodies to the same virus if the virus has many different proteins associated with it. Depending on its design, a qualitative serology test for anti-SARS-CoV-2 may be looking for antibodies to some piece of the virus’s spike glycoprotein (S) or its nucleocapsid phosphoprotein (N), IgG, IgM antibodies, either or both, to either or both of these antigens, all with one s/co result.
Quality control for qualitative tests
Controls for qualitative tests monitor the accuracy and precision at the cutoff or the medical decision point. But what does that mean exactly? The Clinical Laboratory Standards Institute (CLSI) document User Protocol for Evaluation of Qualitative Test Performance1 calls the cutoff C50, because a sample with a s/co response of 1.000 should test positive 50 percent of the time and negative the other 50 percent, with normal imprecision or variability in the response. As you move away from C50 toward a higher s/co, the percentage of positive results increases along with the analyte concentration, until 100 percent of results are positive. The opposite is true as you move away from C50 toward a lower (<1.0) s/co.
The boundaries in Figure 1A have been set as C5 to C95. C5 is the point at which only 5 percent of samples with a mean s/co of 1.00 would still yield a positive result. C95 is the point at which 95 percent of samples with a mean s/co of 1.00 would yield a positive result.
Precision is measured as the distance along the x-axis that describes the run-to-run variability or random error in the cutoff.
Figure 1B illustrates imprecision in two methods. The solid red line represents a more precise method, where the C5 to C95 range is narrower, and thus the variability at C50 is less. The method represented by the broken blue line has a wider C5 to C95 range, describing a greater variability around the cutoff.
Independent controls
The use of independent controls (aka third-party or external controls) is considered best practice for serology tests, even those that come with manufacturer supplied controls.2 Kit controls are most commonly designed to monitor bias and precision in a specific kit lot. One major added value of an independent control is that it is designed to work across kit lots. Thus, if there is a significant shift in the cutoff with a kit lot change, or a significant increase in random error, a kit control may not see it, but an independent control should.
If the test chosen produces a numeric result (often a s/co), the independent controls can be charted to show bias as a change in the mean of sequential control observations, and a change in precision as increased or decreased amplitude of the ‘spikes’ in the chart.
Figure 23 describes a recent issue with a seemingly minor reagent change that resulted in decreased sensitivity in a commercial anti-HCV assay.
The independent control shifted when the lot with the changed reagent was introduced. A lab using an independent control noted the shift, which persisted in replacement lots, and an investigation with some naturally occurring and formulated low titer samples confirmed decreased sensitivity (false negatives) in the affected lots. The manufacturer’s internal investigation found the problem and resolved it.4
Conclusion
Either a mean shift, or an amplitude change in an independent control, signals a change in the sensitivity/specificity of the test. As with quantitative assays, consistent variability within the established control limits signals an assay that is working as intended.
Recently posted CDC Guidance5recommends the use of a highly specific SARS-CoV-2 antibody test, because coronaviruses are common in the environment, and therefore, antibodies to coronaviruses are common too. The inevitable trade-off between sensitivity and specificity comes into play here. A highly specific test will (almost) only detect antibodies to SARS-CoV-2 and will produce very few false positives. High specificity comes at the cost of lower sensitivity – some low positives will be missed, i.e., they will be resulted as negatives (false negatives). But low positives will be relatively unimportant for clinical or surveillance purposes for SARS-CoV-2 antibodies.
Use of independent low positive and negative controls, along with the manufacturer’s recommended quality control practices, will help laboratorians become more familiar with, and hopefully more confident in, novel coronavirus antibody tests, and ultimately help lab personnel detect errors.
References:
- Figures 1A and 1B are reprinted with permission from the Clinical and Laboratory Standards Insitute from: CLSI. User protocol for evaluation of qualitative test performance; approved guideline—second edition. CLSI document EP12-A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2008.
- Public Health England. Quality assurance in the diagnostic virology and serology laboratory. In: Standards Unit Microbiology Services. Colindale, UK: Public Health England, 2015
- NRL. https://www.nrlquality.org.au/. Accessed July 31, 2020.
- Dimech W, Vincini G, Cabuang L, Wieringa M. Does a change in quality control results influence the sensitivity of an anti-HCV test? Clin Chem Lab Med 2020 Jul 28; 58(8): 1372-1380, doi: 10.1515/cclm-2020-0031.
- Interim guidelines for COVID-19 antibody testing in clinical and public health settings. Centers for Disease Control and Prevention (CDC). https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antibody-tests-guidelines.html. Accessed May 27, 2020.
