| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Articles |
1 Department of Clinical Biochemistry, St. Bartholomew’s and The Royal London School of Medicine & Dentistry, Turner Street, London E1 2AD, United Kingdom. Fax 44-20-7377-1544; e-mail c.p.price{at}mds.qmw.ac.uk
 |
Abstract |
|---|
 Top Abstract IntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
|
Introduction |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
Interestingly, to date evidence-based medicine appears to have had limited impact in the sphere of laboratory medicine. Furthermore, there are some data to suggest that adherence to criteria for the use of robust evidence in scientific papers on the use of diagnostic tests is poor (5). Laboratory medicine also provides some of the more overt examples of practice lacking a good foundation of evidence—perhaps the best examples being the variations seen in testing strategies between different hospitals for the same clinical presentations (6)(7). It is therefore hardly surprising that there are ardent critics of laboratory medicine and a considerable body of literature devoted to the inappropriate use of diagnostic tests (8)(9)(10). Perhaps the greatest challenge to laboratory medicine is the suggestion that diagnostic tests are not perceived to have a major impact on patient outcomes (11). Whereas most would consider this an extremely misguided viewpoint, it does indicate the degree of ignorance or misunderstanding that surrounds the value of diagnostic tests and poses one of the major challenges for today’s laboratory professionals.
|
Definitions and Concepts |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
Diagnostic tests
It has always been recognized that the use of a diagnostic test is
an intervention (13). A diagnostic test should be requested
only when a question is being posed and when there is evidence that the
result will provide an answer to the question. There are several
reasons why a physician will order a diagnostic test (14);
however, the nature of the question being asked and the decision to be
made often will depend on the clinical setting in which the patient is
found.
Outcomes research
Outcomes research, an important facet of evidence-based laboratory
medicine, has been the subject of debate
(13)(15)(16) over the past three
decades. An outcome can be defined as the result of an intervention and
may be a health outcome or an economic outcome
(17)(18). This definition of an outcome may, in
light of the limited perception of the contribution made by a
diagnostic test (11), be a contributing factor to the
paucity of good evidence on the effectiveness or benefit of diagnostic
procedures in laboratory medicine.
The expectations for outcomes may be different for healthcare providers and patients (16). The patient is interested in receiving prompt and effective treatment, the relief of symptoms, and improvement in the quality of life. The service provider will also focus on the delivery of effective care, with the promptness of delivery varying between different countries but within the framework of optimum use of resources and minimization of long-term costs. Recognizing that any outcome will be a cascade of many synergistic decisions and actions, it may be more appropriate to focus greater attention on the use of diagnostic tests in the individual elements of the decision-making process. This may help to tease out the key constraints to delivering the desired patient outcomes while highlighting the value of the diagnostic test. In a recent editorial, Scott (19) stressed the importance of identifying a measurable outcome linked with the diagnostic procedure in question [point-of-care testing (POCT) in this case] (20).
Decision-making
Thus, the outcome of a diagnostic test can be considered as any
part of the decision-making process that leads to an improved outcome.
This approach was first described by Fryback and Thornbury
(21), who set out the elements of clinical decision-making
in relation to the efficacy of diagnostic tests. Thus, in terms of
clinical benefit a test may improve the diagnostic process
and/or the therapeutic strategy, and thus the overall health outcome.
The outcome of a diagnostic test may be an operational or an economic
benefit. Thus, in the example of the patient with chest pain, the first
"questions" or decisions to be made relate to the recognition of
cardiac pain and the urgency for referral. These early decisions are
appropriate because we know that early intervention improves overall
patient outcome.
Health technology assessment
Health technology assessment is a tool that examines
systematically the consequences of the application of health technology
to support decision-making in policy and practice. Technology
encompasses drugs, devices, and procedures together with the
organizational and support systems, therefore including diagnostic
tests and differing modalities of delivery. However, the assessment
focuses on the way in which the test or device is used rather than on
whether it "works"; the latter is assumed and is the remit of
another evaluation process (e.g., the Medical Devices Agency in the
United Kingdom). The key principles of health technology assessment are
outlined in Table 1
.
|
Health technology assessment can be seen as the means by which evidence is developed to support decision-making, and it embraces much of the discipline of health services related or applied research. It may require original or primary research on the one hand, or a systematic review of established literature.
|
Context of the Evidence-based Culture |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
. All of these pressures translate into a common currency of
cost pressures. It is worth exploring some of these pressures in a
little more detail, focusing on issues related to implementation of new
technology, decision-making, quality, and cost.
|
There is no doubt that healthcare professionals are expected to deal with an increasing burden of knowledge as well as having access to increasing amounts of new technology, both diagnostics and interventions. Earlier commentators have pointed to the numbers of scientific journals published each year and the limited time given to updating of knowledge (23). One of the strongest arguments for maintaining a health technology assessment program is the recognition that the process of evaluation and implementation of new technology is cumbersome, with long delays between clinical effectiveness being established and adoption into routine practice (24). There are examples in the literature, e.g., the implementation of thrombolytic therapy (25), that also demonstrate the importance of maintaining a continuous awareness of developments in the field to avoid unnecessary delays in the implementation of an effective new technology. It might also be argued that a lack of awareness of current literature leads to wastage of resources on additional evaluation projects when the answer is known. There are also examples in the literature where a new technology has been implemented ("technology creep") and subsequently considered to be ineffective or to lead to unnecessary interventions (26).
Quality in clinical practice has risen on the political agenda (22)(27), and the evidence-based culture has begun to play a greater role in policy-making on a global scale (28). The practical implications of the quality agenda have been a stronger focus on training and assessment of competence (29), on continuing education for maintenance of competence (30), and the introduction of clinical guidelines (31).
The rising cost of healthcare is also a topic high on the agenda of many governments and which is thought to contribute to the reduction in the number of hospital beds and the downsizing of laboratories (22)(32). It is also undoubtedly a stimulus for the literature on the appropriateness of laboratory testing and the demands for proper evaluation of new technology as a prerequisite for introduction into the healthcare system. However, it is also true that resource management is a major problem in the overall context of healthcare delivery, with little evidence that value for money plays a part in determining the investment in laboratory medicine for a given patient episode or disease state. For example, an examination of the economic impact of introducing a molecular-based test for Chlamydia and the influence on disease prevalence illustrates the complexity of resource allocation while demonstrating the central role of good evidence in decision-making (33).
|
How to Practice |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
, which illustrates this as being a continuous process. There
are many publications that provide guidelines on evaluation of the
effectiveness of interventions (35)(36),
although there are few that deal specifically with diagnostic tests.
However, guidelines for the systematic review of diagnostic tests help
by providing information on the critical appraisal of evidence
(37)(38)(39)(40).
|
Define the question
The starting point is the identification of the clinical question
that is being asked; the ability, with the aid of the test result, to
make a decision is therefore the outcome. There are many examples to
illustrate this point, but it is not always recognized that a test can
be used to make a "rule in" or "rule out" decision. Much of the
literature in laboratory medicine has focused on "rule in"
decisions, but "rule out" can often be an important step in a
decision-making cascade. Silverstein and Boland (41) have
suggested that when medical care costs are analyzed, the focus is
directed to "high cost decisions"; they give the example of the
decision to admit a patient. This may be a decision faced regularly by
the primary care physician and may be addressed through a "rule in"
or "rule out" question, depending on the actual diagnostic
performance of the test. However, few diagnostic tests have been
evaluated for the effectiveness in a "rule out" decision strategy.
Examples where this approach has been proposed include urine testing
for leukocyte esterase and nitrite, which can be effective in "ruling
out" urinary tract infection but not "ruling in"; such a test can
then be used to determine what samples are sent to a central laboratory
for culture—with important operational and economic implications
(42). Similarly it has been suggested that myoglobin could
be used to "rule out" myocardial infarction, but not to "rule
in" (43); in the past myoglobin has not been considered a
useful marker of myocardial damage because of its lack of tissue
specificity. This also illustrates the point that scientific reasoning
may not always yield the correct interpretation in relation to clinical
outcome—a point illustrated in the case of therapeutic interventions
(44). The clear identification of a question can be
particularly important when comparing the potential of a new test.
Thus, the fact that serum cystatin C demonstrates a correlation with a
reference clearance test superior to that of serum creatinine in itself
does not prove that the test will offer a clinical benefit
(45).
Hierarchy of evidence
Evidence on the performance of a diagnostic test can be considered
in a hierarchy, all of the elements of which are important to making a
decision (Fig. 2
). This approach was first proposed for a diagnostic test by
Fryback and Thornbury (21) and applied to diagnostic
radiology (41).
|
Technical performance
The foundation of any evidence is technical performance, and this
can have an important bearing on diagnostic performance. In addition to
precision, accuracy, analytical range, and interferences, other
pre-analytical factors such as biological variation and sample
stability can influence the utility of a test. In general terms,
laboratory professionals are extremely good at validating technical
performance, although pre-analytical factors are less commonly
documented (46). Pre-analytical factors can limit the
benefits of a test in routine practice, e.g., the biological variation
in the markers of bone resorption.
Diagnostic performance
Diagnostic performance provides an assessment of the test in terms
of the objective for using the test, namely the sensitivity, which
defines the proportion of people who are correctly identified by the
test as having the disease, and the specificity, which defines the
proportion of people who are correctly identified by the test as not
having the disease. Although these are parameters of any test
irrespective of the population on which the test is used, the
significance of the test is also determined by the prevalence of the
condition in the population being studied. It has been suggested by
Irwig et al. (38) and Moore (23) that the
likelihood ratio combined with the pretest probability is a clearer way
of identifying post test probability and thereby integrating the
relevant information into a clinical decisions pathway. Batstone
(47) has suggested that the number needed to diagnose (NND),
derived from 1/[sensitivity - (1 - specificity)],
provides a useful comparison between tests and helps to encompass the
financial implications in decision-making.
Clinical benefit
However, it is the clinical impact or benefit of the test and the
contribution to decision-making that provide the greatest challenge;
the majority of evidence available in the literature on the use of
diagnostic tests deals with technical and diagnostic performance. For
example, Hobbs et al. (48), in a systematic review on POCT
in general practice (primary care), found that few papers addressed
clinical impact, the majority focusing on technical performance.
The clinical impact can be divided into the effect that use of a test
or procedure will have (a) on the diagnostic strategy, i.e.,
compared with the use of other tests, in improving diagnostic
performance; (b) on the therapeutic strategy, i.e., use of
therapies, optimization of therapy, avoidance of harm, and so forth;
and (c) on the health outcome. Thus, one can evaluate
the impact of the detection of microalbuminuria in terms of
(a) earlier detection of diabetic nephropathy and
(b) better management of diabetes and co-morbid conditions,
e.g., hypertension, with a view to (c) reducing the rate of
development of renal failure. Some more examples of clinical impact are
given in Table 3
.
|
Operational benefit
The use of a diagnostic test may have an operational, as distinct
from a clinical, impact; this often is considered an economic impact,
which it may be because use of resources always resolves down to
economic considerations. However, identification of an operational
question may help to determine the optimal organizational aspects of a
care strategy, e.g., disposition of staff and use of beds, as part of a
wider economic analysis. Operational benefits may include reduced
length of hospital stay, reduced staff time utilization, reduced
utilization of estate (facilities), and reduced utilization of other
resources. It may be important to focus very specifically on the
decisions that can follow from the identification of such a benefit,
e.g., length of stay in relation to bed requirement. The
decision-making will probably address two issues, namely bed
utilization and clinical risk associated with early discharge. Rainey
(49) considered length of stay as a medical outcome;
however, this "definition" lacks the clarity of the decisions that
need to be made.
Economic benefit
The economic impact of the use of a diagnostic test and the
broader issues of cost-benefit analysis are poorly understood tools in
decision-making in healthcare. However, this is extremely important
when the new test or procedure is more expensive than the existing
modality of testing, as often is the case. It is also a wasted
opportunity if the test brings real benefits to the patient and the
healthcare organization. In these considerations, it is important not
to focus solely on the test but on the complete patient episode or
outcome (19)(50), determining where the
investment is required and the gain achieved.
|
Evidence |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
|
Systematic Review |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
and are mainly concerned with handling of large amounts of
data, minimizing the effects of bias introduced into individual
studies, and increasing confidence in the result.
|
A systematic review of current evidence through the use of meta-analysis can identify methodological deficiencies as well as provide an up-to-date systematic review (51). The summary ROC curve can be used to combine studies, illustrating differences that may exist and determining their potential level of significance (52). Good examples of the use of summary ROC curve analysis for two clinical decisions are given in the reports by Guyatt et al. (53) and Olatidoye et al. (54). The work of Olatidoye et al. (54) demonstrates that the variability in diagnostic performance can be large and that this can present a dilemma to the evaluator. It also illustrates the need for multiple studies. Variability in performance may reflect differences in design, choice of patients, and clinical setting, all of which can lead to significant bias in the results (5)(55).
Quality of evidence
Given that evidence informs a decision to be made, then it will be
robust only if the evidence is of good quality. There have been many
publications describing the quality of evidence that is acceptable
(5)(40)(56)(57)(58)(59) and identifying ways
in which evidence can be gathered free of bias. In broad terms, bias
can be introduced into evidence either from the choice of study
population, the design of the experimental work, or the way in which
the results are reported. The key priorities in designing a study to
generate good quality evidence are summarized in Table 5
.
|
The selection of the patient population for study has a critical bearing on the results produced and should be relevant to the question being asked. Thus the diagnostic accuracy of a test will be overestimated in a group of patients already known to have the disease when compared with a group of healthy individuals (59). Bias may also be introduced if the selection of patients for a trial is not randomized or if some patients are excluded from the trial; the latter may be the very patients on whom the quality of the decision hinges and may differentiate one test from another. Lijmer et al. (55) in a study of bias in 11 meta-analyses on 218 studies of diagnostic tests found that the largest effect on diagnostic accuracy occurred in case-control studies, suggesting that the mild cases, which are difficult to diagnose, were excluded, thereby causing overestimates of the sensitivity as well as specificity.
When assessing the diagnostic performance of a test, it is important to use a robust reference method. The reference method should be used in both the test and the control populations; this requirement cannot always be met, sometimes for ethical or cost reasons. However, it is recognized that failing to apply verification can lead to bias in the results. In some situations, to overcome this problem different reference methods have been applied to the test and control populations; again this can introduce bias, and it generated the second largest effect in the study by Lijmer et al. (55). As an example, in a study on the use of C-reactive protein (CRP) in the diagnosis of appendicitis, surgery and pathology were used as the reference for patients with high CRP, whereas verification was limited to clinical follow-up in the patients with low CRP concentrations (60).
It has also been suggested that blinding the evaluator of the new method to the results of the reference method will also improve the validity of the result. However, in the study by Lijmer et al. (55), it had little impact on the diagnostic accuracy of a test; conversely, in situations where the reference procedure is imperfect this effect may be increased. Valenstein (61), however, urged that rather than use an imperfect reference standard, one should focus on a more practical and measurable outcome; he used the example of the imperfect means of predicting the condition of a patient’s myocardium, urging the use of a measurable outcome such as response to therapy or death in hospital. An imperfect reference procedure raises the prospect of interoperator variation, which can also contribute to bias (62). A good example is the diagnosis of myocardial infarction, where the evaluation of new biochemical markers introduces the bias associated with the use of other markers in the diagnostic triad but also recognizes the fact that there is interoperator variation in the ultimate reference procedure—the autopsy (63).
It is evident from the literature that study design related to outcomes has been investigated more in relation to the use of a pharmaceutical intervention than a diagnostic intervention. Thus, Moore and Fingerova (64) have identified other characteristics of study design that will introduce bias to the results. The ideal approach, however, is identical whether it is a diagnostic or a therapeutic intervention and whether it is an assessment of diagnostic performance or outcome, and that is the use of prospective blind comparison of test and reference procedure in a consecutive series of patients from a relevant clinical population (60).
Studies designed to assess clinical, operational, and/or economic outcomes require a clear definition of the outcome measure. In certain clinical studies, this may require the use of a surrogate because the true outcome can only be assessed over several decades; examples include the use of hemoglobin (Hb)A1C as a surrogate for normoglycemic control and the DEXA scan for normal bone mineral density. An alternate outcome may be the "avoidance of disease" as was used in the Diabetes Control and Complications Trial (65); the approach currently underpins our assessment of wellness. It is also important to be aware of confounding factors that might also influence the outcome measure; thus in the study by Kendall et al. (66) on the use of POCT in the emergency room, the "time to result" was reduced, but the "length of stay" in the emergency room (potentially a very relevant outcome measure) was not influenced. The demonstration of reduced time to result, however, is still valid and points to another step in the process as being the impediment to reducing the length of stay.
It is also clear that publication of results also introduces bias; there is a greater tendency to publication of positive findings. Easterbrook et al. (67) showed in a study of a large number of submissions to a local ethics committee that eventual publication was more prevalent in studies where there were significant findings. Dickersin et al. (68) also found that there was an association between significant results and publication, the bias originating with the authors rather than journal editors. Chalmers et al. (69) have defined three stages of publication: (a) organizing and undertaking of the research; (b) acceptance or rejection of a manuscript depending on the presentation of positive or negative findings; and (c) the bias that may result from interpretation, reviews, and meta-analyses. The authors also make recommendations on how publication bias can be minimized.
|
Application |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
Clinical audit
The establishment of a new practice, as well as established
practice, should always be subject to regular audit (72);
this underpins the commitment to maintenance of good practice, one of
the important tenets of clinical governance. The audit will assess
whether the new technology has been implemented satisfactorily and
whether the outcomes found bear out the findings of the original
research. The outcome of the audit may identify the need to modify
practice or may lead to the identification of a new research question.
Experience has shown that auditing established practice as, for
example, a means of controlling demand for a laboratory service, can
identify unmet clinical needs as well as abuse and inappropriate use of
laboratory services. Thus, evidence-based laboratory medicine in all
its facets is the foundation of a continuous quality improvement
program (Fig. 3
).
|
|
Implications of an Evidence-based Culture |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
Training and maintaining performance
It has been suggested that the principles of evidence-based
medicine will provide a better foundation for training because it
focuses more on the evidence to support the type of decision-making
required in clinical practice (73). It is also clear that
performance deteriorates with age (74) and that traditional
means of continuing education are less effective than an evidence-based
approach (75). Thus, through participation in activities to
maintain the evidence database and participation in audit activities,
professional performance can be maintained and kept up to date.
Research and development agenda
The continuous process of practice review, the generation of new
knowledge, and the availability of new technology combine to generate a
powerful development agenda. The use of evidence-based practice
guidelines will ensure that an effective program can be developed to
meet patients’ needs in a timely and cost-effective fashion.
Decision-making
To establish the role of laboratory medicine in clinical
decision-making, it is important to develop the type of evidence that
focuses on these decisions. This requires an explicit recognition that
medical research can be delineated into that which creates basic
knowledge and that which is associated with the application of that
knowledge. Applied research itself can then be differentiated into the
application of knowledge in the development of diagnostic procedures
and therapeutic interventions and that which focuses on the use of such
innovation in decision-making (76). Evidence can then
support decision-making in relation to diagnosis and therapeutic
intervention, together with operational issues, including the
appropriate utilization of resources.
Value for money
The increasing cost of healthcare is one of the major pressures
affecting both purchasers and providers of care. It is also evident
that on the one hand, there is a perception that the cost of laboratory
medicine is high, whereas on the other there is limited perception of
true value for money—both in terms of the true cost and the framework
in which value can be judged. The most obvious is the focus on the
debate between the cost and value of POCT; invariably, and not
unexpectedly, the cost of a point-of-care test is greater than its
central laboratory counterpart, with the clinical and operational
benefits accruing to other sections of the provider system
(77). Thus, the value will only be appreciated outside of
the laboratory, thereby requiring a wider perspective or review of
value than the confines of the laboratory service.
Quality
An alternative description of evidence-based medicine is a
commitment to life-long problem-based learning (73). A
commitment to the activities outlined in Fig. 4
working within a framework as outlined in Fig. 1
will ensure that a
high quality of service is maintained; these attributes are embodied in
the principles of laboratory accreditation (78),
professional self-regulation (79), and ultimately clinical
governance (27), representing commitment to provision of the
highest quality of service to the patient.
|
|
Conclusions |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
|
References |
|---|
|
TopAbstractIntroductionDefinitions and ConceptsContext of the Evidence-based...How to PracticeEvidenceSystematic ReviewApplicationImplications of an Evidence...ConclusionsReferences |
|---|
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  X. Bossuyt, K. Verweire, and N. Blanckaert Laboratory Medicine: Challenges and Opportunities Clin. Chem., October 1, 2007; 53(10): 1730 - 1733. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Rej Clinical Chemistry through Clinical Chemistry: A Journal Timeline Clin. Chem., December 1, 2004; 50(12): 2415 - 2458. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
B. Lumbreras-Lacarra, J. M. Ramos-Rincon, and I. Hernandez-Aguado Methodology in Diagnostic Laboratory Test Research in Clinical Chemistry and Clinical Chemistry and Laboratory Medicine Clin. Chem., March 1, 2004; 50(3): 530 - 536. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Sturgeon Practice Guidelines for Tumor Marker Use in the Clinic Clin. Chem., August 1, 2002; 48(8): 1151 - 1159. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
