| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 Department of Pharmacology, Therapeutics & Toxicology, University of Wales College of Medicine, Cardiff CF14 4XN, United Kingdom.
2 Department of Clinical Biochemistry, University Hospital Aintree, Liverpool L9 7AL, United Kingdom.
3 18 London Road, Burgess Hill RH15 8QX, United Kingdom.
4 Pharmacy Department and Department of Medicine & Therapeutics, Western Infirmary, North Glasgow Hospitals University NHS Trust, Glasgow G11 6NT, United Kingdom.
5 Department of Bacteriology, Southern General University Hospitals NHS Trust, Glasgow G51 4TF, United Kingdom.
6 Department of Pathology, Maudsley Hospital, London SE5 8AZ, United Kingdom.
7 Department of Chemical Pathology, Trafford General Hospital, Manchester M41 5SL, United Kingdom.
8 TICTAC Communications Ltd., St. George’s Hospital Medical School, London SW17 0RE, United Kingdom.
9 Department of Pathology, Princess Royal Hospital NHS Trust, Telford TF6 6TF, United Kingdom.
aAuthor for correspondence. Fax 44-29-2074-8316; e-mail wilsonjf{at}cardiff.ac.uk.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Methods: We distributed lyophilized, serum-based materials containing low, medium, and high weighed-in concentrations of the drugs. The 297 participating laboratories received the materials on two occasions, 7 months apart. Expected concentrations were determined by gas chromatography or HPLC methods in five laboratories using serum-based NIST reference materials as calibrators.
Results: In general, bias was consistent across concentrations for a method but often differed in magnitude for different drugs. Bias ranged from -1.9% to 8.6% for phenytoin, -2.7% to 3.1% for phenobarbital, -2.7% to 0.5% for primidone, -8.6% to 0.3% for carbamazepine, -5.6% to 2.0% for ethosuximide, and -7.2% to 0.1% for valproate. Intralaboratory sources of imprecision significantly exceeded interlaboratory sources for many drug/method combinations. The mean CVs for intra- and interlaboratory errors for the different drugs were 6.3–7.8% and 3.3–4.2%, respectively.
Conclusions: For these long-established and relatively high-concentration analytes, the closed analytical platforms generally performed no better than open systems or chromatography, where use of calibrators prepared in house predominated. To improve the accuracy of measurements, work is required principally by the manufacturers of immunoassays to ensure minimal calibration error and to eliminate batch-to-batch variability of reagents. Individual laboratories should concentrate on minimizing dispensing errors.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
28 countries, that provides a robust comparison of the accuracy and precision of current analytical techniques available for the routine clinical measurement of anticonvulsants. |
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
.
|
secondary standards
Three secondary standards, prepared by Cardiff Bioanalytical Services Ltd. (CBSL; Cardiff, United Kingdom), contained the same six analytes as the primary standards in low, medium, and high concentration mixtures. Lyophilized samples (2-mL) were prepared by adding weighed-in concentrations of drugs to 1.7 L of human serum (Scipac Ltd.). Blank samples containing no added drug were similarly prepared for analysis. The CV of dispensing by weighing was 0.09%.
transfer of primary values to secondary standards
Five reference laboratories that used chromatographic methods were selected from among UKNEQAS participants. These laboratories were volunteers from the eight participants with the smallest root-mean-square difference in laboratory measurements vs their method subgroup consensus mean for the six study analytes in 48 routine EQA samples distributed during 1999.
After the main laboratory survey, the five reference laboratories were provided with frozen aliquots of the primary reference materials, which had been reconstituted by CBSL by addition of a weighed quantity of water. The reference laboratories measured all six drugs simultaneously in both primary and secondary materials by either HPLC or gas chromatographic methods. All laboratories used assay calibrators prepared in house.
laboratory survey
The three secondary standard samples were distributed for measurement to UKNEQAS participants in August 1999 and again in February 2000. Laboratories reported the measured drug concentration, the analytical method, and the type of calibrator used for calculating each measurement.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
All reference laboratories reported zero or not detectable drug concentrations in the blank samples supplied with the primary standards and in blank samples prepared by CBSL. On average, the reference laboratories underestimated the primary values by 2.4–5.2%. Two laboratories reported significantly low values compared with the reference values (P <0.05, t-test) for four or five analytes, two other laboratories reported significantly low values for one analyte each, and the fifth laboratory reported significantly high values for one analyte. However, simultaneous analysis of both primary and secondary standard data expressed as percentage differences from the sample means showed no significant interaction between a laboratory and standard set (P >0.05, ANOVA). Thus, the pattern of between-laboratory differences in bias was comparable for both primary and secondary standards, allowing an unbiased transfer of the primary standard values to the secondary material.
Reference values for each drug were assigned to the secondary material by correcting the measured secondary standard values by the mean percentage bias in primary standard measurements calculated across all three concentrations of the primary standard. The percentage transformation eliminated significant between-sample differences in bias (P = 0.81, two-way ANOVA). The use of a common concentration-independent correction was superior to separate corrections at the three drug concentrations. It reduced by a factor of 12 the variance of differences between assigned secondary reference and weighed-in values for the secondary reference samples (P >0.05, ANOVA). The drug concentrations assigned to the secondary standards and their precision are listed in Table 2
.
|
eqa laboratory survey data
Drug measurements by 14 different methods were reported for some or all analytes by a total of 297 laboratories (60% in the United Kingdom, 30% in Europe, 10% outside Europe). Users of commercial immunoassays used the calibrators supplied by the manufacturers almost exclusively. Of 35 laboratories reporting measurements by HPLC, 26 prepared their own calibrators and 9 used calibrators from 8 different external suppliers.
Data selection.
Data for each sample were first screened to reject outliers >3 SD from the sample mean by the robust method of Healy (4). Data were then selected for a drug where a laboratory had reported nonrejected measurements on both occasions for a sample by the same method and where there was a total of >20 measurements for the method. The numbers of measurements and those rejected or not meeting the selection criteria are displayed in Table 3
. The most common reasons for excluding measurements were that the laboratory reported on only one of the two separate distributions of samples or because they used different analytical methods on the two occasions.
|
Accuracy.
Drug measurements expressed as the percentage difference from the reference value were analyzed separately for each analyte by three-way ANOVA between method, drug concentration, and sample distribution. Significant differences between methods and concentrations were identified by the Student–Newman–Keuls test (P <0.05) and significant differences from the reference value by t-test (P <0.05).
No deterioration in the secondary standard was apparent over the 7 months of the study, there being no significant differences between sample distributions (P >0.05).
Significant between-method differences in accuracy were detected for all drugs. The mean (SE) percentage bias at the three different drug concentrations are displayed in Fig. 1
, ordered by mean overall method bias. Methods not significantly different are indicated by solid lines at the bottom of each panel. Sixty-three percent of all drug/method combinations differed significantly from the reference value. The methods differing significantly are indicated in Fig. 1
by a star after the method name. Mean values for a method ranged from -8.6% (carbamazepine/Vitros assay) to +8.6% (phenytoin/Bayer turbidimetric assay) with a grand mean of -0.8% across the whole study. HPLC, when undertaken by the general population of scheme participants, significantly underestimated concentrations of four drugs by 2.3–2.8%.
|
Significant concentration-related differences in accuracy within a method were detected in 34% of cases (P <0.05). The Abbott TDx differed for phenytoin, phenobarbital, primidone, and ethosuximide, and the AxSYM analyzer differed for phenytoin, phenobarbital, and carbamazepine. The Vitros and the Bayer turbidimetric assays differed for phenytoin, and the CEDIA and Beckman turbidimetric assays differed for both phenobarbital and valproate. The Syva Emit differed for primidone.
Precision.
The intra- and interlaboratory components of within-method variance were estimated by fitting a two-way ANOVA between laboratory and concentration to the data for each method with the two sample distributions as replicates (5). Because the analysis was undertaken separately for each method, the variances calculated excluded differences in accuracy among methods identified in the previous section and were not additive to produce total variance. Significant differences (P <0.05) between intra- and interlaboratory variances were identified by the Bartlett test followed by the Levy test (6).
Intralaboratory sources of variation exceeded interlaboratory sources in all cases except one (Fig. 2
). The difference reached significance in 44% of drug/method combinations. These cases are indicated in Fig. 2
with a star after the method name. The mean intra-/interlaboratory variance ratio was 1.8. Between-method differences were significant for three drugs. The only comparisons that achieved statistical significance involved methods with variances at the extreme ends of the range. For phenytoin, the smallest intralaboratory variance, for the Abbott AxSYM, differed from the largest values, seen for the Bayer turbidimetric, Syva Emit, and Vitros methods. The two smallest interlaboratory variances for phenytoin, for the Abbott AxSYM and Bayer turbidimetric methods, differed significantly from up to five methods with the largest values. For carbamazepine, the intralaboratory variance of the Syva Emit was significantly greater than that for the two Abbott methods, whereas the interlaboratory variance of the CEDIA method was significantly greater than that for the Abbott AxSYM, Roche fluorescence polarization immunoassay (FPIA), and HPLC. For valproate, the only differences were in interlaboratory variance. The largest variance for the Beckman turbidimetric method differed from those for HPLC and the Abbott methods.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
There were significant differences in the magnitude of the bias at different drug concentrations for approximately one-third of drug/method combinations. In some cases, the pattern was of increased bias in the low-concentration sample, whereas in others, bias changed sign at different concentrations. These effects imply some nonlinearity in performance of the immunoassays concerned with respect to drug concentration and would prevent simple correction for bias across concentrations.
There were significant differences in accuracy among methods for all drugs. No one method performed consistently across all analytes. For example, the popular Abbott TDx FPIA gave high results for phenytoin but low results for carbamazepine and valproate. The chromatographic methods were generally members of the statistical group that was closest to the reference values. Significant differences from the reference values were nevertheless produced by chromatographic methods. These differences can be attributed to heterogeneity in the calibrators used by the laboratories performing chromatographic assays. Nevertheless, chromatographic methods were able to outperform many immunoassays in terms of accuracy, despite many laboratories choosing to prepare calibrators in house.
Within-method imprecision was partitioned into intra- and interlaboratory sources. Almost universally, intralaboratory errors exceeded those from interlaboratory sources, which is in agreement with observations from the College of American Pathologists drug survey (10). The magnitude of intralaboratory imprecision was generally comparable among methods, although the Abbott AxSYM analyzer was shown to be statistically superior to some other methods in two cases. Interlaboratory variation within methods varied more between methods than intralaboratory variance. It was not notably less for analyzer/reagent groups from a single supplier compared with open analyzer systems or with chromatography, which might have been predicted to be more heterogeneous.
The scale of both inaccuracy and imprecision quantified by this study was surprising because it represents the current "state of the art" for a series of long-established assays working at relatively high micromolar concentration. Worse may be expected for drugs that are monitored in the nanomolar range or for which a commercial calibration material is not available. In comparison with a range of analytical goals for drug assays (9), the performance of these methods will not necessarily meet clinical requirements. The closed analytical platforms were not superior to open systems or to chromatography, where preparation of in-house calibrators predominated. It is therefore recommended that there should be attempts to improve performance by all concerned. Work is required principally from manufacturers of the available immunoassays to ensure minimal calibration error. That the scale of intralaboratory errors generally exceeds interlaboratory errors might seem to suggest that work by individual laboratories will be worthwhile either on internal quality control or to improve the accuracy of dispensing procedures. However, comparison of intralaboratory errors from simultaneous assay of replicates with those calculated from replicates measured 4 months apart (10) has demonstrated that long-term variance is more than twice that over the short term. In the present study, intralaboratory errors were those occurring over a 7-month period. Factors that may contribute to this variability will include changes in batches of reagents and calibrators supplied by the reagent manufacturers to a laboratory, and drift and maintenance of instruments. It is possible, therefore, that a significant proportion of intralaboratory variance may be attributable to factors outside the control of the laboratory (10).
Laboratories have a responsibility to achieve the best consensus accuracy and minimum imprecision. Where they can influence such matters this should be done. However, our study indicates that their potential actions are limited and that variances are probably affected by changes in services received from suppliers. There is a need for the suppliers of drug assay products to consider their whole system variance over time and to take steps to minimize this. Setting agreed targets with users may be one way of deriving professionally agreeable targets. EQA schemes can stimulate these improvements and provide a cost-effective means to monitor progress by tracing results from routine laboratories to international standards.
|
Acknowledgments |
|---|
|
Footnotes |
|---|
1 Nonstandard abbreviations: EQA, external quality assessment; UKNEQAS, United Kingdom National External Quality Assessment Scheme; CBSL, Cardiff Bioanalytical Services Ltd.; and FPIA, fluorescence polarization immunoassay. 
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  J. F. Wilson and K. Barnett Variation with Time in Components of Variance for Measurements of Therapeutic Drugs Clin. Chem., December 1, 2005; 51(12): 2385 - 2387. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
), medium (
), and high (
) drug concentrations.
indicates a significant (P <0.05) method difference from the reference value. Thick solid lines span methods not significantly different (P >0.05). GC, gas chromatography. TDx and AxSYM are from Abbott Laboratories, Diagnostics Division; Bayer chemiluminescence and turbidimetric assays are from Bayer; Beckman turbidimetric assay is from Beckman Coulter Inc.; CEDIA assay is from Microgenics Corp.; Roche FPIA is from F. Hoffmann-La Roche Ltd., Diagnostics Division; Syva Emit is from Dade Behring; Vitros is from Ortho-Clinical Diagnostics.
