|
|
||||||||
Enzymes and Protein Markers |
1
Bayer Corporation, Business Group Diagnostics, Tarrytown, NY 10591.
2
Department of Pathology, Hartford Hospital, Hartford, CT
06102.
3
Johns Hopkins University, Baltimore, MD 21287.
4
M.D. Anderson Cancer Center, Houston, TX 77030.
5
Memorial Sloan-Kettering Cancer Center, New York, NY
10021.
a Address correspondence to this author at: The Institute for Diagnostic Research, Inc., 23 Business Park Dr., Brandford, CT 06405. Fax 203-315-4002; e-mail carol.cheli{at}diabetesdisc.com.
| Abstract |
|---|
|
|
|---|
4%.
Lot-to-lot reproducibility for 26 different lots of reagents and
calibrators manufactured over a 2-year period was demonstrated (CV,
1.1%). Results for the Bayer Immuno 1 assay correlated well with the
Biomira TRUQUANT® BR(TM) 27.29 and Centocor® CA 15-3 RIAs
(r
0.94). The upper limit of the reference interval for
the Bayer Immuno 1 assay was 35.9 kilounits/L (35.9 units/mL); values
were similar for all methods. Longitudinal monitoring of healthy women
yielded assay values with an average CV of 11% and 21% for the Bayer
Immuno 1 and Biomira assays, respectively. The Bayer Immuno 1 assay
demonstrated the analytical features, intermethod correlation, and
long-term performance characteristics that are essential for
longitudinal monitoring of breast cancer patients. | Introduction |
|---|
|
|
|---|
Serum MUC 1 mucin bears epitopes that are recognized by two murine monoclonal antibodies, 115D8, generated against milk fat globulin membranes, and DF3, generated against a membrane extract from human metastatic breast carcinoma (11)(12). Several commercially available immunoassays use both of these monoclonal antibodies for quantifying CA 15-3 in serum; the antibody 115D8 is used as the capture antibody, whereas DF3 serves as the detection antibody. The first CA 15-3 assays were manual and used radioisotopic detection. Later, nonisotopic immunoenzymatic assays became available. Today, immunoassay automation can improve the precision of replicate results and increase the speed of generating results compared with nonautomated methods (13).
The clinical usefulness of longitudinal monitoring of individual patient serum CA 15-3 concentrations has been hampered by the analytical variability of CA 15-3 tests (14)(15)(16). At present, it is difficult for clinicians to distinguish between the variations in results due to tumor progression or regression from variations due to biological and analytical variability when monitoring their patients sequentially (15)(16). When monitoring a patient longitudinally, small increases or decreases in tumor marker concentration can be indicative of early recurrence of disease or response to therapy, respectively. Poor assay precision can mask these changes and confound the interpretation of disease status and, therefore, lead to poor medical decision-making. Suggestions to improve the long-term analytical performance of CA 15-3 immunoassays include: establishment of analytical reliability; comparability of CA 15-3 assay determinations between laboratories; comparability of lot-to-lot reagent performance; and automation of analytical procedures (14)(15)(16).
We have developed a fully automated random access immunoassay for the quantitative measurement of serum CA 15-3 on the Bayer Immuno 1(TM) immunoanalyzer. The aim of our studies was to evaluate the analytical and clinical performance of the Bayer Immuno 1 CA 15-3 assay. As part of this evaluation, we examined serial variability of the CA 15-3 assay values in healthy women, an important criterion often overlooked in the evaluation of serial tumor marker assays (14)(15). Our aim was to perform a realistic assessment of the variability of the method for monitoring longitudinal serial samples. Here we report our multicenter evaluation of the Bayer Immuno 1 CA 15-3 assay. We compared the Bayer Immuno 1 assay with two FDA-approved manual RIAs: the Biomira TRUQUANT® BR(TM) RIA (Biomira, Inc., Edmonton, Alberta, Canada) for the quantitation of serum CA 27.29, and the Centocor® CA 15-3(TM) RIA (Centocor, Malvern, PA) for the quantitation of serum CA 15-3. These assays have been reported to measure the same antigen encoded by the MUC 1 gene (17).
| Materials and Methods |
|---|
|
|
|---|
The Centocor CA 15-3 RIA system is a solid-phase immunoassay in which plastic beads coated with monoclonal antibody 115D8 form the solid phase and the radiolabeled monoclonal antibody DF3 is the detector. In the Biomira assay, polystyrene tubes are coated with human CA 27.29 antigen. The CA 27.29 antigen present in the sample competes with the solid-phase antigen for binding of the radiolabeled B27.29 tracer antibody. Tests were performed according to manufacturers' instructions.
calibrators and controls
The CA 15-3 assay controls used for the performance studies
included two concentrations of Bio-Rad Lyphochek Tumor Marker Controls
(BioRad Labs) and the Medical Decision Pool. The Medical Decision Pool
is a serum-based control material manufactured at Bayer Corporation and
used internally for routine quality control. This is a pool of
healthy human serum to which antigen is added to a mean target
concentration of 31 kilounits/L. The Bayer Immuno 1 CA 15-3 assay uses
six calibrators, 0, 12.5, 25, 50, 100, and 200 kilounits/L of antigen
added to a 60 g/L solution of bovine serum albumin. The antigen used in
the Bayer Immuno 1 assay calibrators was obtained from Centocor, Inc.
specimen and patient selection
Human serum samples used for the correlation analysis and the
determination of cutoff values were obtained from BioClinical Partners,
Inc., and the in-house specimen collection at Bayer Corporation
(Tarrytown, NY). There were ~500 serum samples from patients with
healthy, malignant, and benign conditions used in these studies. Ages
of the subjects ranged from 18 to 92 years (median age, 51). The
samples from patients with malignant conditions examined included
breast cancer (n = 150), lung cancer (n = 35), ovarian cancer
(n = 35), and colorectal cancer (n = 35). Additionally, we
analyzed sera from a total of 199 healthy subjects (100 premenopausal
and 99 postmenopausal) and 46 patients with benign breast disease.
The majority of patient specimens for longitudinal monitoring of healthy individuals were obtained at the investigational sites or supplied by Western States Plasma Co., Inc. and the Bayer Corp. specimen bank. Both premenopausal (n = 21) and postmenopausal (n = 24) women were enrolled in the healthy subject longitudinal study. Ages of the subjects ranged from 22 to 68 years.
Healthy subjects included individuals with no fever or infections, who met the criteria for blood bank donation and had no known current or previous malignancy, and were not taking prescription medication at the time of sample collection. Healthy subjects were screened for their state of health by questionnaire, and patients with benign and malignant diseases were screened by physical examination. Preanalytical conditions of serum samples included age of specimen no greater than 10 years and storage within 24 h of collection at a temperature no higher than -20 °C. The serum samples used for the study were collected under a protocol reviewed and approved by the Institutional Review Board at each investigational site.
ca 15-3 performance protocol
The Bayer Immuno 1 CA 15-3 Assay was assessed for imprecision and
lower limit of detection at four participating evaluation sites. These
sites include the M. D. Anderson Cancer Center (Houston, TX), Johns
Hopkins University (Baltimore, Maryland), Memorial Sloan-Kettering
Cancer Center (New York, NY), and Hartford Hospital (Hartford, CT).
Variability of longitudinal monitoring in healthy women was assessed at
three of these sites by using the Bayer Immuno 1 CA 15-3 assay and the
Biomira RIA. The remainder of the nonclinical performance evaluation
was conducted at Bayer Corporation. For the correlation analysis and
determination of the upper limit of the reference interval, serum
samples were tested at Bayer Corporation using the Bayer Immuno 1 CA
15-3 assay and the Centocor CA 15-3 RIA and at Dianon Systems, Inc.
(Stratford, CT) using the Biomira RIA.
lot-to-lot variability
To evaluate reproducibility in the manufacturing process of
successive lots of reagents and calibrators, 26 lots of reagents and
calibrators were used to assay one lot of the Medical Decision Pool.
Assays were performed over a 2-year period in the manufacturing
facility of Bayer Diagnostics (Bridgend, Wales). Mean assay values were
determined from 20 replicate analyses per lot of reagent or calibrator,
and the CV of the means was calculated.
imprecision
Imprecision of the CA 15-3 assay was evaluated by analysis of
Bio-Rad Tumor Marker Controls (two concentrations), the CA 15-3 Medical
Decision Pool, and three lots of the Bayer Immuno 1 CA 15-3 calibrators
(levels 16). Samples were run in duplicate in 20 independent runs
over 10 days by each of the four evaluation sites. At each site, three
lots of Bayer Immuno 1 reagents were used. Site-to-site imprecision
pooled across reagent lots and total imprecision pooled across sites
and reagent lots were determined. Data were analyzed for variance
components using the SAS (Version 6.07) NESTED procedure to give
statistical estimates of within-run, run-to-run imprecision in the same
laboratory, and total imprecision across laboratories.
lower limit of detection
The lower limit of detection of the assay was evaluated by
determination of the minimum detectable concentration of CA 15-3 assay
values that can be statistically distinguished from the concentration
of the lowest calibrator, as calculated from a typical calibration
curve. The minimum detectable concentration was calculated from the
pooled within-run standard deviations of the zero calibrator response
rates (mA/min). The rates were pooled over three calibrator lots for
each of three reagent lots tested at each of the four evaluation sites.
The overall minimum detectable concentration was calculated as the mean
Bayer Immuno 1 CA 15-3 assay concentration corresponding to two pooled
within-run SDs for each of the 12 reagent lot/site combinations.
linearity
To validate assay linearity over the dynamic range of the assay
(0200 kilounits/L), dilutions of four pools of human serum samples
were analyzed over the entire calibration range. Each pool containing a
high CA 15-3 assay concentration (180200 kilounits/L) was diluted
with the first calibrator (0 kilounit/L). Final concentrations
representing 100%, 75%, 50%, 25%, 10%, and 0% of each high-sample
pool were assayed with two lots of Bayer Immuno 1 reagents on two
instruments. Test samples were assayed in triplicate. The data
generated were subjected to regression analysis. Linearity was
evaluated by comparing dilution-corrected values to the concentration
of the undiluted sample.
high-dose hook effect
Very high concentrations of analyte in a sample can saturate both
capture and tag antibodies, which decreases the assay signal, resulting
in a falsely low result. To determine whether highly increased
concentrations of CA 15-3 in patient serum cause a "hook effect", a
pure preparation of antigen was diluted in the zero calibrator. Final
concentrations ranged from 7 to 28 500 kilounits/L. Test samples were
assayed in triplicate. The samples were tested in the Bayer Immuno 1 CA
15-3 assay with two lots of reagents. The assay response for the test
samples was reported as the observed reaction rate vs the nominal CA
15-3 assay concentration.
correlation
To examine the agreement of the Bayer Immuno 1 CA 15-3 assay with
the Centocor and Biomira RIAs, patient sample CA 15-3 or CA 27.29 assay
values generated by the individual methods were compared by correlation
analysis. Correlation studies were performed by using a panel of 500
serum specimens from healthy volunteers and from patients with
malignant and benign diseases. The evaluation was conducted over an
8-week period. Patient results were obtained from the mean of duplicate
determinations for the Biomira CA 27.29 RIA and the Centocor CA 15-3
RIA and from singlicate determinations by using the Bayer Immuno 1 CA
15-3 assay. CA 15-3 and CA 27.29 assay values generated from the three
methods were compared by the linear least squares and Passing Bablok
regression statistics.
upper limit of reference interval
The upper limit of the reference interval was determined for 199
healthy women by the Bayer Immuno 1 CA 15-3 assay and the Centocor and
Biomira RIAs. Calculations of the mean, median, and range of CA 15-3 or
CA 27.29 assay values were determined. The reference interval of CA
15-3 or CA 27.29 assay values was calculated by using a one-tailed
nonparametric statistic as the assay value, which corresponds to the
97.5th percentile of all the measured values.
longitudinal variability of ca 15-3 or ca 27.29 assay values in
healthy women
We prospectively studied total variability (biological and
analytical) with the Bayer Immuno 1 CA 15-3 assay and the Biomira RIA
in six samples from each of 15 women at each of three evaluation sites.
Samples were obtained approximately once a month for 6 months.
Specimens from any one individual were assayed by using one of four
lots of Bayer Immuno 1 CA 15-3 reagent. Approximately equal numbers of
patient specimens were assayed with each lot. The mean and CV for six
serial assay determinations for both methods were calculated for each
of the 45 healthy individuals entered into the study.
t-tests for statistical significance between the two
methods, individual means, grand means, individual variances, and
average CV were calculated. The mean CV was calculated as 100 times the
square root of the mean variance divided by the grand mean. The mean CA
15-3 and CA 27.29 longitudinal values for each patient were compared by
the linear least squares and Passing Bablok linear regression
statistics.
| Results |
|---|
|
|
|---|
|
Imprecision data for two levels of Bio-Rad controls, Medical Decision
Pool, and calibrators pooled across three reagent lots and four
evaluation sites are presented in Table 1
. Pooled within-run CVs were 1.3%3.4%. Total pooled CVs over
the four evaluation sites and three reagent lots were 3.0%4.0%.
|
In this study, a mean lower limit of detection of 0.13 kilounit/L was observed for the CA 15-3 assay. This value was determined based on multiple determinations (n = 1336) of the Level 1 calibrator (0 kilounit/L) at four evaluation sites using three lots of Immuno 1 calibrators and reagents. The detection limit supports a claim of 0.2 kilounit/L for the Bayer Immuno 1 CA 15-3 assay.
We examined linearity of this new method by analyzing five serial dilutions of four serum sample pools diluted with Bayer Immuno 1 CA 15-3 zero calibrator. The clinical samples diluted linearly as determined by linear regression analysis. The regression analysis yielded slopes ranging from 0.9938 to 1.01 (r = 0.99).
No hook effect was seen in samples containing 7 to 28 500 kilounits/L of antigen (data not shown).
Fig. 2
shows the distribution of 199 healthy patients and their Bayer
Immuno 1, Biomira, and Centocor RIA values. The numbers of assay values
obtained by the three methods at each assay concentration range were
similar. The detailed results of the range analysis are presented in
Table 2
. The 97.5th percentile normal reference range
cutoff for the Bayer Immuno 1 assay was 35.9 kilounits/L, 31.3
kilounits/L for the Biomira RIA, and 35.6 kilounits/L for the Centocor
RIA.
|
|
Table 3
shows the correlation coefficients and the regression
parameters for 500 samples over the range of the calibration curve
analyzed by the Bayer Immuno 1 assay with the Biomira and Centocor
RIAs. Graphs of the correlation between methods for the patient samples
are shown in Fig. 3
. Test results obtained with the Bayer Immuno 1 CA 15-3 assay
were similar to results obtained with the comparison methods.
|
|
The means and CVs for six serial measurements for each of 45 healthy
women assayed using the Biomira RIA and the Immuno 1 CA 15-3 assay are
summarized in Fig. 4
. The CV determined for the Bayer Immuno 1 assay for each of the
45 individuals was plotted against the corresponding CV determined from
the Biomira RIA assay. The Biomira assay values were more variable than
the assay values obtained with the Bayer Immuno 1 assay. The average CV
was 11% for the Bayer Immuno 1 CA 15-3 assay and was 21% for the
Biomira assay. In addition to the comparison of serial variability by
using these two assays, we also evaluated the mean CA 15-3 assay values
for both assays. The mean of six serial measurements using the Bayer
Immuno 1 CA 15-3 assay for each individual was plotted against the
mean values obtained with the Biomira RIA and results are presented as
an inset in Fig. 4
. Linear regression analysis yielded: Immuno 1 =
1.27 x Biomira-5.56, Sy
x = 3.37,
r = 0.93. The difference between the individual means
as well as the grand means for the two methods were not statistically
significant (P = 0.22 and 0.67, respectively). However, the
difference between the variances of the methods was significant
(P = 0.0001), as was the difference in the average CV
(P = 0.0001).
|
| Discussion |
|---|
|
|
|---|
The Bayer Immuno 1 CA 15-3 automated method demonstrates advantages for the longitudinal monitoring of breast cancer patients. The imprecision across multiple sites and multiple lots of reagents and calibrators was 3%4%. Imprecision (CV) with manual methods reportedly is >10% (13), whereas the CV of the Abbott CA 15-3 IMx automated method was 4.1%5.8% in a multicenter evaluation (19). The Centocor and Biomira assays have reported imprecision data across multiple sites ranging from 8.4%12.4% and 6.0%15%, respectively (manufacturers product labeling). These findings indicate the improved performance of automation over manual methods and demonstrate that the testing of patient samples at different sites using the Bayer Immuno 1 CA 15-3 assay will produce comparable results.
Lot-to-lot variation is an important source of analytical variation
within a method, but it is seldom quantified
(16)(20). During the manufacturing of reagents
and calibrators, considerable variation between lots can occur. van
Dalen (14) emphasized the importance of quality control and
standardization in the manufacture of immunoassay reagents and made the
recommendation that manufacturers report lot-to-lot variation for their
tumor marker assays (14)(20). Our analysis from
tests using multiple lots of Bayer Immuno 1 reagents and calibrators
performed over a 2-year duration (Fig. 1
) document the precision (and
quality control in the manufacturing process) of the Bayer Immuno 1 CA
15-3 assay.
The lower limit of detection of 0.13 kilounit/L appears acceptable because serum concentrations of MUC 1 mucin <3 kilounits/L are rare and of no known clinical importance. The linearity and freedom of the assay from a high-dose "hook effect" are important for monitoring because high values of the marker are to be expected, and dilutions are required. It has been reported that manual assays are less reliable in this respect and often give dilution values that are too high, possibly reflecting the higher imprecision of these tests (13).
Both the correlation analysis and the determination of the upper limit of the reference interval demonstrate agreement of patient sample assay values of the Bayer Immuno 1 CA 15-3 assay with the Biomira and Centocor RIAs. The differences noted in the y-intercept and the slopes of the regression lines may be due to differences in the statistical method used and may reflect the sensitivity of ordinary least squares regression to outlier results (21).
We found significant (P = 0.0001) discrepancy in the mean CV for serial measurements of healthy patient samples between the Biomira (manual) and automated methods. Variability during sequential testing is a combination of biological and analytical variability. In our study, the biological variability remained constant for both assays, because the same serum samples were analyzed. The improved analytical precision of the Bayer Immuno 1 CA 15-3 assay renders it attractive for serial monitoring. Moreover, the study demonstrates a realistic assessment of the method by analyzing its performance when monitoring clinical samples longitudinally over time, the clinical situation where the assay is most useful.
For CA15-3 immunoassays to provide reliable data in serial monitoring of patients, a high degree of long-term analytical performance of an assay must be maintained (15)(22). Variation in assay values because of poor analytical performance can mask small changes in assay values due to disease progression and compromise the clinical utility of the CA 15-3 assay. Long-term analytical precision of an assay will provide a more accurate interpretation of longitudinal results, including early detection of recurrence of disease, increasing the lead time for therapeutic intervention and a more accurate determination of therapeutic response.
Despite the strong linear relationship between the Biomira and Bayer Immuno 1 methods, we observed a bias in the slope of the regression line. This indicated that the two methods yield results that are related but not interchangeable. It has been reported that although there is good method correlation between assays in the quantitation of MUC 1 gene products, it is inappropriate to exchange patient results from one method to another (13)(20). A recent study has indicated that different antibodies used to detect the MUC 1 mucin differ in their affinity and epitope specificity for the mucin and, therefore, produce inexchangeable results (23). This may explain the difference observed in our study between the Biomira 27.29 assay and the Bayer Immuno 1 CA 15-3 assay. Other studies have demonstrated that even when using the same antibodies, the many different test systems available yield different assay results (14)(19)(24). International standardization has been proposed that may improve this situation (14)(24), but at present, it is prudent that only one test system be used during the follow-up of a given patient.
| Acknowledgments |
|---|
| References |
|---|
|
|
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |