|
|
||||||||
Articles |
1
Department of Clinical Biochemistry AKH and Skejby Hospital, Aarhus University Hospital, DK-8000 Aarhus C, Denmark.
2
Department of Pharmacology, University of Bergen, 5021
Bergen, Norway.
3
Axis Biochemicals, 0510 Oslo, Norway.
4
Department of Biochemistry, Trinity College, University
of Dublin, Dublin 2, Ireland.
5
Department of Pharmacology, University of Oxford, Oxford
OX1 3QT, United Kingdom.
6
Department of Pediatric Research, University of Oslo,
0316 Oslo 3, Norway.
a Address correspondence to this author at: Department of Clinical Biochemistry, AKH, Aarhus University Hospital, Norrebrogade 44, DK-8000 Aarhus C, Denmark. Fax 45-8949-3060; e-mail ene{at}post9.tele.dk
| Abstract |
|---|
|
|
|---|
Methods: Two immunological methods for measurement of plasma total homocysteine (P-tHcy), the fluorescence polarization immunoassay (FPIA) and the enzyme immunoassay (EIA), were compared with two comparison methods, HPLC and gas chromatographymass spectrometry (GC-MS). All laboratories performed the following procedures: (a) familiarization; (b) determination of linearity and precision by analyzing five plasma samples with interrelated concentrations for 20 days; (c) correlation using patients samples; and (d) assessment of long-term performance.
Results: Both immunological methods were linear for P-tHcy between 5 and 45 µmol/L. The intralaboratory imprecision (CV) was <5% for FPIA and <9% for EIA used with a sample processor. The bias was -2% to 3% for FPIA and 24% for EIA used with a sample processor.
Conclusions: The immunological methods provide results with little bias compared with HPLC and GC-MS. The imprecision of the assays must be considered in the context of their intended use(s).
| Introduction |
|---|
|
|
|---|
New methods for measurement of plasma total homocysteine (P-tHcy)1 currently are under evaluation. Until recently, measurement of P-tHcy required time-consuming and complex methods such as HPLC and gas chromatographymass spectroscopy (GC-MS) (1)(2), but now an immunological method for the determination of P-tHcy has been developed and subsequently used in both a fluorescence polarization immunoassay (FPIA) format in conjunction with the IMx® analyzer (3) and in an enzyme immunoassay (EIA) format (4). These assays are simple and may be superior to the conventional chromatographic assays for routine use.
The introduction of these novel tests for P-tHcy illustrates a typical situation in clinical chemistry today. The assays were developed for commercial use, and the producer provided extensive documentation on specificity, analytical range, reproducibility, accuracy, and linearity. However, systematic data on the performance in individual routine laboratories of various sizes and experience are lacking, not only for the P-tHcy assays but for nearly all tests introduced into routine laboratory practice.
This report describes a strategy to evaluate novel P-tHcy assays. The study is part of a European Union-funded demonstration project involving six centers in four countries. The objective was to evaluate the performance of these assays in a range of settings reflecting their future use. The strategy involved the following procedures: (a) familiarization; (b) assessment of linearity and imprecision; (c) assessment of correlation with the comparison method; and (d) long-term performance using samples from patients.
| Materials and Methods |
|---|
|
|
|---|
|
Samples
Samples for testing linearity and imprecision.
Two pools of
human plasma with no exogenous tHcy were prepared, with one
containing 45 µmol/L P-tHcy [S1; close to the upper limit for the
immunoassay (4)] and the other containing 5 µmol/L P-tHcy
[S5; close to the lower reference limit of an adult population
(5)]. S1 and S5 were then mixed so that samples S2, S3, and
S4 contained 50%, 25%, and 12.5% S1 and 50%, 75%, and 87.5% S5,
respectively. The tHcy concentrations of S2, S3, and S4 were calculated
from the fractional composition of S1 and S5. Aliquots of each S sample
were frozen at -20 °C. In each laboratory, samples S1S5 were
thawed and further aliquoted for the analysis of each sample on three
occasions daily (FPIA) or twice daily as duplicates (EIA) for 20
consecutive working days.
Patients samples used in the correlation analysis.
Plasma
samples from 57 individuals covering a P-tHcy concentration range of
434 µmol/L were collected from individuals screened at the
University of Bergen clinical chemistry laboratory and stored at
-20 °C until analysis.
Control samples.
Internal control samples at three
concentrations (~8, 12, and 25 µmol/L) were supplied by the
manufacturer (AXIS) and used to accept or reject the individual
runs, according to prespecified rules (6). External quality
assessment samples were supplied by the Danish Institute for External
Quality Assurance for Hospital Laboratories and analyzed on six
occasions to test the long-term performance at each center. All
samples, with the exception of the external assessment samples, were
shipped to each of the participating laboratories on dry ice.
Laboratories
A total of six laboratories in four countries participated in the
demonstration project.
Laboratory 1.
Laboratory 1 developed and carried out the
premarket testing of the immunoassays. The laboratory participated in
the protocol with the FPIA and the EIA using automatic sample
processing. The inclusion of this laboratory permitted a comparison of
the performance of the assay producer with the routine laboratories.
Laboratory 2.
Laboratory 2 was a research laboratory with more
than 15 years of experience with P-tHcy determination and serves as a
reference laboratory for P-tHcy determinations. This laboratory
developed a method in 1985 that formed the basis for the novel
immunoassays (7). The fully automated HPLC method, developed
in 1989 and modified in 1993, has been widely used in several large
clinical and epidemiological studies on P-tHcy
(8)(9)(10). For this project, this laboratory measured
P-tHcy with four methods, EIA, FPIA, a HPLC technique based on
monobromobimane derivatization and fluorescence detection
(11), and a GC-MS method involving ethylchloroformate
derivatization as described by Husek (12). The EIA method
was run using an automated sample processor.
Laboratory 3.
Laboratory 3 had performed P-tHcy determination
with HPLC based on 7-fluorobenzofurazane-4-sulfonic
acid derivatization and fluorescence detection
(13)(14) for ~5 years. This laboratory
measured P-tHcy using both HPLC and FPIA.
Laboratories 4 and 5.
Laboratories 4 and 5 were laboratories
with no previous experience in the determination of P-tHcy, and both
measured P-tHcy using the EIA with manual sample processing.
Laboratory 6.
Laboratory 6 included two laboratory units. One
was responsible for the GC-MS comparison method
(15)(16) and had almost 10 years of experience
with P-tHcy determination in clinical practice as well as in research.
The other unit was comparable to a routine clinical chemistry
laboratory introducing P-tHcy determination (the FPIA variant) for the
first time.
Study design
The following steps were performed.
Familiarization.
After installation of the equipment, the
technicians received practical training. This involved performing a run
that included calibration and analysis of internal controls using a
specified protocol. The acceptance of the internal controls was based
on the limits supplied by the manufacturer.
Linearity and imprecision.
Samples S1S5 were run in random
order on three occasions per day together with the three internal
control samples for the FPIA or twice daily in duplicate for the EIA on
20 consecutive working days. Runs were accepted if the internal
controls were within the limits stipulated by the manufacturer. The
mean P-tHcy concentrations and SDs for the internal controls were
calculated and used for acceptance or rejection in the subsequent runs.
Correlation analyses.
The 57 samples from patients were
analyzed once, and runs were accepted based on values obtained for the
internal controls.
Long-term performance and external quality assessment.
Over a
12-month period, six pairs of samples from an external quality
assessment scheme were run within 2 weeks of receipt.
One laboratory continued the FPIA assay on a routine basis to test its practicability. The assay was run once a week by seven different technicians over a 5-month period. The number of rejected runs was recorded, and the imprecision was calculated based on the values obtained for the internal controls.
Statistical methods
The within-day, between-day, intralaboratory, between-laboratory,
and interlaboratory variances were calculated using nested analysis of
variance. The interlaboratory variance is the variation between results
obtained from samples run in various laboratories. The intralaboratory
variance is the variation for results obtained from samples run over
several days in the same laboratory. The within-day variance describes
the variation for results obtained from samples assayed in the same
run. The between-day variance was calculated by subtracting the
within-day variance from the intralaboratory variance. Similarly, the
between-laboratory variance was calculated by subtracting the
intralaboratory variance from the interlaboratory variance. Bias was
calculated from the results obtained for the S samples and for the
patients samples. Bias was defined as the difference between the test
method result and the comparison method result divided by the
comparison method result. The results from the 57 patients samples
obtained by the different methods were analyzed by linear regression
and according to the procedure of Bland and Altman
(17)(18). Results obtained by GC-MS (laboratory
6) were used as the comparison method or "gold standard", and this
laboratory unit was not involved in running the novel immunological
assays.
| Results |
|---|
|
|
|---|
Familiarization
The six laboratories were familiarized with the immunological
methods within a few weeks of installation of the equipment.
Assessment of linearity and precision
Based on the results of the analysis of five samples (S1S5) with
interrelated concentrations, the linearity and imprecision of the
methods were assessed and compared with similar data for the comparison
methods. In general, all methods showed linearity throughout the 545
µmol/L P-tHcy range (Fig. 1
). A statistically significant deviation from linearity was
observed for the FPIA run at laboratory 3 because of results obtained
for sample S2.
|
The intralaboratory variation for the various assays is shown in Table 2
. The intralaboratory imprecision (CV) was <5% for
FPIA, <9% for the automated EIA, and <13% for the manual EIA. The
interlaboratory imprecision for the FPIA was 35% for the range of
values studied and was comparable to values obtained with the
comparison methods. The interlaboratory imprecision for the EIA was
somewhat higher: 69% when a sample processor was used, and 1017%
when the assay was performed by manual pipetting.
|
The proportions of the variance derived from the within-day,
between-day, and between-laboratory variances were calculated using
nested analyses of the variance (17); the results are shown
in Fig. 2
. For the FPIA, the within-day variance accounted for 2763%
of the variance and the between-day variation accounted for most of the
remaining variance. The between-laboratory contribution was marginal. A
similar pattern was obtained with the EIA performed with a sample
processor, whereas the between-laboratory component was considerably
larger for EIA carried out by manual pipetting.
|
The bias relative to the comparison method is shown in Table 3
. Based on the samples, the FPIA showed a negative bias of
~2%, whereas one of the laboratories showed a small positive bias
for the EIA. The most significant bias was observed between the two
GC-MS methods, and this was most likely attributable to differences in
the calibrators used in each laboratory.
|
Correlation analyses
Analysis of patients samples throughout the range of P-tHcy
values between 4 and 34 µmol/L allowed a comparison of results
obtained with the novel immunological methods with the GC-MS method
used as gold standard (Fig. 3
). Exclusion of one outlier (34 µmol/L) did not alter the
overall results of the regression analysis (data not shown). Regression
analysis based on all 57 samples showed no significant deviation from a
slope of 1 and an intercept of 0 for three of the four laboratories
performing FPIA and three of the four laboratories performing EIA. The
deviations were significant for the remaining two laboratories
(laboratories 3 and 5), showing slopes varying 811% from a slope of
1 and intercepts deviating -0.5 and 2 µmol/L for the FPIA and EIA
assays, respectively. By comparison, similar data for the two HPLC
methods (laboratories 2 and 3) showed 9% and 11% deviation from a
slope of 1, and -0.8 and 0.9 µmol/L deviation from an intercept of
0. The other GC-MS method (laboratory 2) showed a deviation of 4% from
a slope of 1 and no deviation for the intercept. Laboratory 3 obtained
a similar deviation for the FPIA and HPLC. This deviation is unlikely
to reflect calibration because all immunological assays used the same
calibrators, whereas the HPLC used local calibrators.
|
Bias was calculated using the patients samples, and the results are
shown in Table 3
. One of the manual EIA assays showed a positive bias
of 9%, but none of the other assays had a bias exceeding 4%.
Long-term performance
All laboratories participated in an external quality assessment
program (19). Over a 12-month period, each laboratory
returned results for 12 samples. The results obtained with the FPIA
method and three of the comparison methods were all within the 90th
percentile. Four of the 48 results obtained with the EIA method carried
out in three laboratories were outside the 90th percentile. In
addition, four of the results obtained with the comparison method
(HPLC; laboratory 3) were outside the 90th percentile.
The performance of the FPIA run by seven technicians over a 5-month period was excellent. Only 1 of the 60 runs required repeat measurement because of internal controls being outside the accepted limits, and the CV (obtained for the internal controls covering the range 725 µmol/L) was <3%.
| Discussion |
|---|
|
|
|---|
Our strategy for the initial testing of a new method involved familiarization; estimation of linearity, imprecision, and bias; assessment of correlation with comparison methods; and evaluation of long-term performance. Our protocol included two additional strategies compared with recommended guidelines for the implementation of a new methodology into the routine laboratory (20)(21). We compared the performance in different laboratories and used human samples with interrelated concentrations. Assessment of performance using samples with interrelated concentrations allows a systematic evaluation of the new methods that is independent of the comparison method (22). This is of particular importance when the new assay is superior to the currently used methodology or when no comparison method is available.
Our evaluation of the immunological assays for P-tHcy confirms and expands the results obtained from previous studies (23)(24)(25)(26)(27). The practicability and low imprecision of the FPIA method throughout the range of values tested suggest that it is suitable for routine use in laboratory practice. The accuracy of the FPIA method makes it feasible to directly compare values obtained in different laboratories. This is an important issue both in clinical studies and in routine clinical chemistry. The EIA has the advantage of a high throughput and low sample volume requirement, but the imprecision is higher than that of the FPIA, especially when combined with manual sample handling. These features make the EIA format suitable mainly for screening purposes, where detection of a substantial change in P-tHcy is required. The results obtained by the immunological methods compared well with results obtained with the comparison method, and no systematic bias of significant magnitude was observed.
In conclusion, when implementing novel assays in routine clinical practice, it is important to evaluate these assays in the relevant laboratory settings and take account of laboratory performance as well as assay performance. This project illustrates an approach that could be used when introducing other novel assays for routine clinical practice. Manufacturers of novel assays should have the responsibility to evaluate their performance in this way before their introduction for routine clinical practice.
| Acknowledgments |
|---|
| Footnotes |
|---|
| References |
|---|
|
|
|---|
The following articles in journals at HighWire Press have cited this article:
![]() |
A. D. Dangour, E. Breeze, R. Clarke, P. S. Shetty, R. Uauy, and A. E. Fletcher Plasma Homocysteine, but Not Folate or Vitamin B-12, Predicts Mortality in Older People in the United Kingdom J. Nutr., June 1, 2008; 138(6): 1121 - 1128. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. N. Rowan, H. O. Dickinson, S. Stephens, C. Ballard, R. Kalaria, and R. Anne Kenny Homocysteine and post-stroke cognitive decline Age Ageing, May 1, 2007; 36(3): 339 - 343. [Full Text] [PDF] |
||||
![]() |
A. L. Morkbak, A.-M. Hvas, Z. Lloyd-Wright, T. A.B. Sanders, O. Bleie, H. Refsum, O. K. Nygaard, and E. Nexo Effect of Vitamin B12 Treatment on Haptocorrin Clin. Chem., June 1, 2006; 52(6): 1104 - 1111. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. M Devlin, R. Clarke, J. Birks, J. G. Evans, and C. H Halsted Interactions among polymorphisms in folate-metabolizing genes and serum total homocysteine concentrations in a healthy elderly population Am. J. Clinical Nutrition, March 1, 2006; 83(3): 708 - 713. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. Refsum, C. Johnston, A. B. Guttormsen, and E. Nexo Holotranscobalamin and Total Transcobalamin in Human Plasma: Determination, Determinants, and Reference Values in Healthy Adults Clin. Chem., January 1, 2006; 52(1): 129 - 137. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Windelberg, O. Arseth, G. Kvalheim, and P. M. Ueland Automated Assay for the Determination of Methylmalonic Acid, Total Homocysteine, and Related Amino Acids in Human Serum or Plasma by Means of Methylchloroformate Derivatization and Gas Chromatography-Mass Spectrometry Clin. Chem., November 1, 2005; 51(11): 2103 - 2109. [Abstract] [Full Text] [PDF] |
||||
![]() |
E.-C. Chan, P.-Y. Chang, T.-L. Wu, and J. T. Wu Enzymatic Assay of Homocysteine on Microtiter Plates or a TECAN Analyzer Using Crude Lysate Containing Recombinant Methionine {gamma}-Lyase Ann. Clin. Lab. Sci., April 1, 2005; 35(2): 155 - 160. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. Refsum, A. W. Grindflek, P. M. Ueland, A. Fredriksen, K. Meyer, A. Ulvik, A. B. Guttormsen, O. E. Iversen, J. Schneede, and B. F. Kase Screening for Serum Total Homocysteine in Newborn Children Clin. Chem., October 1, 2004; 50(10): 1769 - 1784. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. A.D.M. van Osch, E. Hogervorst, M. Combrinck, and A. D. Smith Low thyroid-stimulating hormone as an independent risk factor for Alzheimer disease Neurology, June 8, 2004; 62(11): 1967 - 1971. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Arndt, B. Guessregen, A. Hohl, and B. Heicke Total Plasma Homocysteine Measured by Liquid Chromatography-Tandem Mass Spectrometry with Use of 96-Well Plates Clin. Chem., April 1, 2004; 50(4): 755 - 757. [Full Text] [PDF] |
||||
![]() |
H. Refsum, A. D. Smith, P. M. Ueland, E. Nexo, R. Clarke, J. McPartlin, C. Johnston, F. Engbaek, J. Schneede, C. McPartlin, et al. Facts and Recommendations about Total Homocysteine Determinations: An Expert Opinion Clin. Chem., January 1, 2004; 50(1): 3 - 32. [Abstract] [Full Text] [PDF] |
||||
![]() |
A.-L. B. Monsen, H. Refsum, T. Markestad, and P. M. Ueland Cobalamin Status and Its Biochemical Markers Methylmalonic Acid and Homocysteine in Different Age Groups from 4 Days to 19 Years Clin. Chem., December 1, 2003; 49(12): 2067 - 2075. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Clarke, H. Refsum, J. Birks, J. G. Evans, C. Johnston, P. Sherliker, P. M Ueland, J. Schneede, J. McPartlin, E. Nexo, et al. Screening for vitamin B-12 and folate deficiency in older persons Am. J. Clinical Nutrition, May 1, 2003; 77(5): 1241 - 1247. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. M. Blanck, B. A. Bowman, G. R. Cooper, G. L. Myers, and D. T. Miller Laboratory Issues: Use of Nutritional Biomarkers J. Nutr., March 1, 2003; 133(3): 888S - 894. [Abstract] [Full Text] [PDF] |
||||
![]() |
N.D. Prins, T. den Heijer, A. Hofman, P.J. Koudstaal, J. Jolles, R. Clarke, and M.M.B. Breteler Homocysteine and cognitive function in the elderly: The Rotterdam Scan Study Neurology, November 12, 2002; 59(9): 1375 - 1380. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Q. Hanson, J. H. Eckfeldt, K. Schwichtenberg, O. Aras, and M. Y. Tsai Interlaboratory Variation of Plasma Total Homocysteine Measurements: Results of Three Successive Homocysteine Proficiency Testing Surveys Clin. Chem., September 1, 2002; 48(9): 1539 - 1545. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. S. Sachdev, M. Valenzuela, X. L. Wang, J. C.L. Looi, and H. Brodaty Relationship between plasma homocysteine levels and brain atrophy in healthy elderly individuals Neurology, May 28, 2002; 58(10): 1539 - 1541. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. Hogervorst, H. M. Ribeiro, A. Molyneux, M. Budge, and A. D. Smith Plasma Homocysteine Levels, Cerebrovascular Risk Factors, and Cerebral White Matter Changes (Leukoaraiosis) in Patients With Alzheimer Disease Arch Neurol, May 1, 2002; 59(5): 787 - 793. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. L. Duarte, X. L. Wang, and D. E. L. Wilcken Effects of Anticoagulant and Time of Plasma Separation on Measurement of Homocysteine Clin. Chem., April 1, 2002; 48(4): 665 - 668. [Full Text] [PDF] |
||||
![]() |
D. M Kerins, M. J Koury, A. Capdevila, S. Rana, and C. Wagner Plasma S-adenosylhomocysteine is a more sensitive indicator of cardiovascular disease than plasma homocysteine Am. J. Clinical Nutrition, December 1, 2001; 74(6): 723 - 729. [Abstract] [Full Text] |
||||
![]() |
A.-L. B. Monsen, P. M. Ueland, S. E. Vollset, A. B. Guttormsen, T. Markestad, E. Solheim, and H. Refsum Determinants of Cobalamin Status in Newborns Pediatrics, September 1, 2001; 108(3): 624 - 630. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. C. McKinley, J.J. Strain, J. McPartlin, J. M. Scott, and H. McNulty Plasma Homocysteine Is Not Subject to Seasonal Variation Clin. Chem., August 1, 2001; 47(8): 1430 - 1436. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Clarke Prevention of vitamin B-12 deficiency in old age Am. J. Clinical Nutrition, February 1, 2001; 73(2): 151 - 152. [Full Text] [PDF] |
||||
![]() |
J. C. Chambers, P. M. Ueland, M. Wright, C. J. Dore, H. Refsum, and J. S. Kooner Investigation of Relationship Between Reduced, Oxidized, and Protein-Bound Homocysteine and Vascular Endothelial Function in Healthy Human Subjects Circ. Res., July 20, 2001; 89(2): 187 - 192. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||