Clinical Chemistry Email Content Delivery
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Clinical Chemistry 48: 55-60, 2002;
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit an electronic Letter to
the Editor about this paper
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (11)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by van Wijk, M. A.M.
Right arrow Articles by van Bemmel, J. H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by van Wijk, M. A.M.
Right arrow Articles by van Bemmel, J. H.
Related Collections
Right arrow Laboratory Management
Right arrow Evidence Based Laboratory Medicine and Test Utilization
(Clinical Chemistry. 2002;48:55-60.)
© 2002 American Association for Clinical Chemistry, Inc.

Compliance of General Practitioners with a Guideline-based Decision Support System for Ordering Blood Tests

Marc A.M. van Wijk1a, Johan van der Lei1, Mees Mosseveld1, Arthur M. Bohnen2 and Jan H. van Bemmel1

1 Institute of Medical Informatics and
2 Department of General Practice, Faculty of Medicine and Health Sciences, Erasmus University Rotterdam, 3000 DR Rotterdam, The Netherlands.

aAddress correspondence to this author at: Institute of Medical Informatics, Faculty of Medicine and Health Sciences, ErasmusUniversity Rotterdam, P.O. Box 1738, 3000 DR Rotterdam, The Netherlands. Fax 31-10-408-9447; e-mail wijk{at}mi.fgg.eur.nl.


   Abstract
Top
 Abstract
Introduction
 BloodLink
 Materials and Methods
 Results
 Discussion
 References
 
Background: Guidelines are viewed as a mechanism for disseminating a rapidly increasing body of knowledge. We determined the compliance of Dutch general practitioners with the recommendations for blood test ordering as defined in the guidelines of the Dutch College of General Practitioners.

Methods: We performed an audit of guideline compliance over a 12-month period (March 1996 through February 1997). In an observational study, a guideline-based decision support system for blood test ordering, BloodLink, was integrated with the electronic patient records of 31 general practitioners practicing in 23 practices (16 solo). BloodLink followed the guidelines of the Dutch College of General Practitioners. We determined compliance by comparing the recommendations for test ordering with the test(s) actually ordered. Compliance was expressed as the percentage of order forms that followed the recommendations for test ordering.

Results: Of 12 668 orders generated, 9091 (71%) used the decision-support software rather than the paper order forms. Twelve indications accounted for >80% of the 7346 order forms that selected a testing indication in BloodLink. The most frequently used indication for test ordering was "vague complaints" (2209 order forms; 30.1%). Of the 7346 order forms, 39% were compliant. The most frequent type of noncompliance was the addition of tests. Six of the 12 tests most frequently added to the order forms were supported by revisions of guidelines that occurred within 3 years after the intervention period.

Conclusions: In general practice, noncompliance with guidelines is predominantly caused by adding tests. We conclude that noncompliance with a guideline seems to be partly caused by practitioners applying new medical insight before it is incorporated in a revision of that guideline.


   Introduction
Top
 Abstract
 Introduction
BloodLink
 Materials and Methods
 Results
 Discussion
 References
 
Dutch investigators report that a lack of general practitioners’ knowledge concerning the indications for blood tests leads to inappropriate use of diagnostic tests (1). Investigators argue that improving the quality of blood test ordering deserves attention (2)(3).

To deal with the rapidly expanding body of medical knowledge, guidelines are increasingly viewed as a mechanism for distributing knowledge to practitioners (4)(5). In The Netherlands, the Dutch College of General Practitioners issues guidelines for the general practitioner. The procedure for creating a guideline consists of four stages (6). The first stage involves the selection of topics for new guidelines by an independent advisory board. In the second stage, a small taskforce, consisting of four to eight general practitioners with special interest in and expertise on the topic of that guideline, prepares a draft. This draft is based on a review of the available literature, current medical practice, and consensus in the taskforce with respect to appropriate medical practice in primary care. In the third stage, a random sample of general practitioners and specialists reviews this draft. The final stage involves authorization of the guideline by a board consisting of leading general practitioners. After authorization, the guideline is published in the journal of the Dutch College of General Practitioners. After publication, the guidelines are revised at regular intervals.

Even when authoritative guidelines are available, changing the behavior of physicians is difficult (7)(8)(9)(10). Investigators acknowledge that the dissemination and implementation of guidelines constitute an important research area that must be addressed (11)(12)(13). Dissemination of guidelines alone is not enough; it needs to be combined with an appropriate implementation strategy (2)(14), including an educational strategy (15). Some authors argue that the use of electronic patient records will provide new opportunities for decision support (11)(16)(17); integration of decision support facilities with the electronic patient record may provide a natural way to integrate that support in day-to-day practice (18)(19)(20). Recently, Shiffman et al. (21) analyzed the functionality and effectiveness of computer-based guideline implementation and reported that guideline adherence improved in 14 of 18 systems in which it was measured.

In The Netherlands, the majority of general practitioners have replaced their traditional paper-based medical records with electronic patient records to record medical data during patient encounters (22). We have built a test-ordering module, BloodLink, which is integrated with the electronic patient record to support the general practitioner in requesting blood tests. Our objective was to improve adherence to the guidelines by implementing a test-ordering module that provides the general practitioner with recommendations for test ordering based on these guidelines. In a randomized trial, BloodLink led to a significant reduction of the number of tests ordered by the general practitioners (23).

The impact of BloodLink on the volume of tests ordered is not necessarily an indication of the degree of compliance to the guidelines. In this study, we determined the compliance of Dutch general practitioners with the recommendations for blood test ordering as defined in the guidelines of the Dutch College of General Practitioners. We will first, however, briefly describe BloodLink.


   BloodLink
Top
 Abstract
 Introduction
 BloodLink
Materials and Methods
 Results
 Discussion
 References
 
By January 1996, the Dutch College of General Practitioners had published 54 guidelines covering most clinical situations in primary care. We analyzed the version of each guideline available in January 1996 (24). If the guideline contained a reference to a blood test, we determined (a) the clinical situation in which the test should be performed (the indication) and (b) the tests that should be performed in that situation (the advised tests).

Of the 54 guidelines, 23 guidelines addressed blood tests and allowed us to identify the indications for those tests (24). We distinguished five categories of indications for the following clinical situations: (a) the use of blood tests to support or refute a provisional diagnosis; (b) the use of blood tests to investigate the underlying pathology that could be causing the already diagnosed disease; (c) monitoring the course of a disease; (d) the use of blood tests to select appropriate treatment; and (e) monitoring side effects of drugs.

BloodLink, which is based on these five categories of indications and covers 66 different indications, contains a total of 37 different tests (24).

The general practitioner, using the electronic patient record, can activate BloodLink to order blood tests. BloodLink first provides an overview of the available guidelines. The general practitioner selects the appropriate guideline. BloodLink queries the general practitioner about the reasons for requesting the tests until an indication is identified, e.g., the guideline "blood tests and liver disease" mentions 10 different indications, and the module asks for the disease involved (e.g., hepatitis B) and the indication (e.g., monitoring the course of hepatitis B). Optional help texts are available that explain the choices and provide the relevant sections of the guidelines. After the indication has been identified, the system proposes the relevant tests. BloodLink does not cover all possible indications for blood tests in primary care. To deal with these situations, the general practitioner can select the heading "other indication" and type the initial letter of the desired test. BloodLink prints a hardcopy order form for the laboratory. In addition, BloodLink logs the physician’s use of the system and stores the order in the electronic patient record.

Guideline implementation occurs in the context of conflicting pressures for clinical autonomy and professional standardization and quality improvement (25)(26)(27). Although the guidelines are explicit but crude summaries of both "state of the art" evidence-based medicine and implicit skills, they should be used not to dictate practice but to inform clinical judgment. Moreover, patients have multiple problems and often present with nonspecific symptoms. The indication-oriented test protocols of BloodLink, therefore, are only a recommendation to the physician. The general practitioner makes the decision of protocol adherence. At any time, the practitioner may adapt the test ordering to the patient-specific clinical situation by adding or removing tests from the list proposed by BloodLink.


   Materials and Methods
Top
 Abstract
 Introduction
 BloodLink
 Materials and Methods
Results
 Discussion
 References
 
The study was an audit of guideline compliance over a 12-month period and was part of a larger study reported previously (23). In August and September 1995, the general practitioners in the region of Delft (a total of 94 practitioners in 64 practices) were asked to participate in a trial in which we studied test-ordering behavior (23). Of these 64 practices, 44 agreed to participate. Of these 44 practices, 24 practices involving 32 practitioners were assigned to use the BloodLink decision support system based on the guidelines of the Dutch College of General Practitioners. During a 3-month period, the general practitioners were allowed to use BloodLink in their practices. After this period, the general practitioners were asked whether they wished to start the study. One practitioner did not want to proceed, stating that he did not like the software. During the study period, the participating general practitioners did not receive any feedback. The median enrolled population per practice was 3205, the median average age of population was 37.6 years, and the median percentage of females was 49.9%.

The study period was March 1996 through February 1997. We logged the use of the BloodLink software in the general practitioners’ offices by recording the use of the software in the patients’ medical records. For test ordering, the physicians were not forced to use the BloodLink software; paper order forms remained available during the entire intervention period.

For all indications, we counted the frequency of use and the number of tests added to or removed from the proposed test panel per practice. Protocol compliance was measured by comparing the BloodLink test recommendations with the tests actually ordered per indication. If the physician did not change the recommendations of BloodLink, we labeled the order form as compliant. If the physician did modify the recommendations, we labeled the order form as noncompliant. The noncompliant order forms were subsequently classified into one of three categories. If the physician only added tests, we classified the order form as noncompliant by addition of tests. If the physician only removed tests, we classified the order form as noncompliant by removal of tests. If the physician both removed and added tests, we classified the order form as noncompliant by both addition and removal of tests. The main outcome measure was the percentage of order forms compliant with the recommendations for test ordering of the guidelines of the Dutch College of General Practitioners.

In October 1999, we analyzed which guidelines involving test ordering had been updated by the Dutch College of General Practitioners after the end of the intervention period (i.e., after February 28, 1997). To assess whether noncompliance was related to these changes in the guidelines, we determined whether the updated guidelines included indications that recommended the three most frequently added tests.


   Results
Top
 Abstract
 Introduction
 BloodLink
 Materials and Methods
 Results
Discussion
 References
 
For test ordering, the general practitioner had the choice to use either the BloodLink software or a paper form. Of the 12 668 order forms generated, 9091 (71%) used the decision-support software. For 7346 of these 9091 order forms, general practitioners selected an indication listed in BloodLink, with a total of 66 different indications. As shown in Table 1 , 12 indications accounted for >80% of the 7346 order forms; the most frequently used indication was "vague complaints", which accounted for 2209 of the 7346 order forms (30.1%). When the general practitioner could not find the indication, BloodLink allowed the general practitioner to enter the indication and select tests. Of the 9091 forms, the general practitioner entered an indication and selected tests in 1745 order forms; these 1745 order forms were excluded from further analysis.


View this table:
[in this window]
[in a new window]
  		Table 1. Most frequently used indications.

After selecting the indication, the general practitioner could change the recommendations of BloodLink. Of the 7346 order forms, 2874 (39.1%) were compliant with the recommendations of BloodLink. The number of compliant order forms per practice ranged from 10.2% to 60.9%. The remaining 4472 order forms were noncompliant. Of the 7346 order forms, 262 (3.6%; range, 0–11.3%) were noncompliant by removal of tests, 3337 (45.4%; range, 18.1–81.0%) were noncompliant by addition of tests, and 873 (11.9%; range, 2.5–33.6%) were noncompliant by both removal and addition of tests. When removing tests, the general practitioner removed, on average, 1.4 tests. When adding tests, the general practitioner added on average 2.9 tests. Table 2 shows, per indication, the percentages of compliant order forms, noncompliant order forms by removal of tests, noncompliant order forms by addition of tests, and noncompliant order forms by both removal and addition of tests.


View this table:
[in this window]
[in a new window]
  		Table 2. Percentage of order forms per indication that were compliant, noncompliant by removal of tests, noncompliant by addition of tests, and noncompliant by both removal and addition of tests.

To assess whether noncompliance was related to pending revision of guidelines, we analyzed which guidelines were revised after the intervention period. We found that a total of 13 of 54 guidelines were revised after the intervention period. Four of these revised guidelines showed changed recommendations for test ordering: hypercholesterolemia, diabetes mellitus, sore throat, and hypertension. In the revised guideline, the indications to perform tests changed and recommendations for test ordering were updated. In addition, new indications were introduced. All revisions in the recommendations for test ordering involved adding more tests when compared with the recommendations in the previous guidelines. Compared with the previous guideline, the revised guideline for hypertension added potassium to the diagnostic workup of new patients; the revised guideline for hypercholesterolemia added, for certain patients (e.g., patients who smoke or suffer from hypertension), the investigation of HDL-cholesterol and glucose to screen for additional risk factors; the revised guideline for sore throat added a white blood cell count when leukemia or agranulocytosis was suspected; and the revised guideline for diabetes mellitus added hemoglobin A1c for monitoring treatment and HDL-cholesterol and triglycerides for all patients.

In Table 3 , we compare the three most frequently added tests in the noncompliant order forms with the recommendations of the updated guidelines for hypercholesterolemia, diabetes mellitus, sore throat, and hypertension. In total, the general practitioners generated 2295 order forms by selecting an indication from these four guidelines. The general practitioners used indications from the guideline for hypercholesterolemia a total of 1037 times. Of these 1037 order forms, 398 (38.4%) were compliant with the guidelines that were available during the intervention period, whereas 639 (61.6%) were noncompliant. Of the 639 noncompliant order forms, 4 (0.6%) were noncompliant by removal of tests, 608 (95.2%) were noncompliant by addition of tests, and 27 (4.2%) were noncompliant by both removal and addition of tests. When adding tests, the general practitioners added, on average, 3.0 tests. The most frequently added tests were for triglycerides, HDL-cholesterol, and glucose; of the 639 noncompliant order forms, 509 (79.7%) order forms involved adding triglycerides, 506 (79.2%) involved adding HDL-cholesterol, and 186 (29.1%) involved adding glucose.


View this table:
[in this window]
[in a new window]
  		Table 3. Most frequently added tests in the noncompliant order forms compared with recommendations of the updated guideline.

The general practitioners used indications from the guideline for hypertension a total of 666 times. Of these 666 order forms, 343 (51.5%) were compliant and 323 (48.5%) were noncompliant. Of the 323 noncompliant order forms, 26 (8.1%) were noncompliant by removal of tests, 256 (79.3%) were noncompliant by addition of tests, and 41 (12.7%) were noncompliant by both removal and addition of tests. When adding tests, the general practitioners added, on average, 2.8 tests. The most frequently added tests were for potassium, hemoglobin, and triglycerides; of the 323 noncompliant order forms, 108 (33.4%) order forms involved adding potassium, 94 (29.1%) involved adding hemoglobin, and 87 (26.9%) involved adding triglycerides.

The general practitioners used indications from the guideline for diabetes mellitus a total of 332 times. Of these 332 order forms, 106 (31.9%) were compliant and 226 (68.1%) were noncompliant. Of the 226 noncompliant order forms, 4 (1.8%) were noncompliant by removal of tests, 195 (86.3%) were noncompliant by addition of tests, and 27 (12.0%) were noncompliant by both the removal and addition of tests. When adding tests, the general practitioners added, on average, 2.5 tests. The most frequently added tests were for hemoglobin A1c, triglycerides, and HDL-cholesterol; of the 226 noncompliant order forms, 161 (71.2%) order forms involved adding hemoglobin A1c, 45 (19.9%) involved adding triglycerides, and 43 (19.0%) involved adding HDL-cholesterol.

The general practitioners used indications from the guideline for sore throat a total of 260 times. Of these 260 order forms, 78 (30.0%) were compliant and 182 (70.0%) order forms were noncompliant. Of the 182 noncompliant order forms, 4 (2.2%) were noncompliant by removal of tests, 159 (87.4%) were noncompliant by addition of tests, and 19 (10.4%) were noncompliant by both removal and addition of tests. When adding tests, the general practitioner added, on average, 2.8 tests. The most frequently added tests were for erythrocyte sedimentation rate, hemoglobin, and glucose; of the 182 noncompliant order forms, 129 (70.9%) order forms involved adding erythrocyte sedimentation rate, 88 (48.4%) involved adding hemoglobin, and 45 (24.7%) involved adding glucose.


   Discussion
Top
 Abstract
 Introduction
 BloodLink
 Materials and Methods
 Results
 Discussion
References
 
In The Netherlands, the ordering of blood tests has been the subject of numerous studies that attempted to reduce the number of blood tests ordered by general practitioners (1)(28)(29)(30)(31)(32)(33)(34). In a previous study, we reported that introducing indication-oriented test panels based on the guidelines of the Dutch College of General Practitioners reduced the number of tests ordered (23). The observed reduction of the number of tests requested by the physician, however, does not necessarily mean that the physicians adhered to the protocols. In this study, therefore, we focused on protocol adherence.

Although BloodLink supports 66 indications, 12 indications accounted for >80% of the general practitioners’ use of the system. The most frequently used indication was "vague complaints", reflecting the morbidity in primary care. Primary care is characterized by its role in diagnosing undifferentiated problems, often presenting minor illness, or early stages of diseases. This pattern of morbidity differs significantly from the morbidity in secondary care. The fact that vague complaints was the most common indication for test ordering is not surprising and is consistent with other studies in Dutch primary care (34)(35).

Our study shows that removing tests from the recommendations of the guidelines is rare compared with adding tests. General practitioners seem to use the guideline-based test recommendations as a minimum for test ordering in daily practice. Although the introduction of BloodLink produced a clear reduction of the number of tests ordered (23), this study shows that complete compliance to the guidelines would have produced a still larger reduction.

In our study, the addition of tests was the main cause of noncompliance. Noncompliance, however, does not necessarily indicate poor medical practice. A possible reason for adding test is the fact that general practitioners may have multiple indications for tests ordering. A patient presenting with an acute problem may also have a chronic disease (e.g., diabetes mellitus) that requires periodic monitoring. The blood tests required for the acute problem may be combined with the tests required for the monitoring of the chronic disease in a single test order.

Our study shows another reason for adding tests: anticipating pending changes in the guidelines. The Dutch College of General Practitioners revises the guidelines periodically. Many of the modifications that caused the order form to be noncompliant were supported by subsequent revision of guidelines. For example, HDL-cholesterol, which was added in 79.2% of the noncompliant order forms, is included in the recommendations of the revised guideline for hypercholesterolemia. Likewise, hemoglobin A1c, which was added in 71.2% of the noncompliant order forms, is included in the recommendations of the revised guideline for diabetes mellitus. Apparently, general practitioners are aware of the new medical evidence and anticipate pending changes in the guidelines. Evidence-based medicine requires that guidelines be revised in the light of the available randomized clinical trials. New trials first appear in medical journals and are read by physicians. The evidence may subsequently lead to revision of guidelines. Adoption of recent insight into daily practice, however, can precede dissemination of revised guidelines. When physicians’ adherence to guidelines is being audited, the dynamic nature of guideline development must be taken into account. Ignoring this dynamic nature of guidelines could lead to penalties for physicians who apply the most recent knowledge.


   References
Top
 Abstract
 Introduction
 BloodLink
 Materials and Methods
 Results
 Discussion
 References
 

  1. Zaat JO, Van Eijk JT, Bonte HA. Laboratory test form design influences test ordering by general practitioners in the Netherlands. Med Care 1992;30:189-198.[ISI][Medline] [Order article via Infotrieve]
  2. Van Walraven C, Goel V, Chan B. Effect of population-based interventions on laboratory utilization: a time-series analysis. JAMA 1998;280:2028-2033.[Abstract/Free Full Text]
  3. Van Walraven C, Naylor CD. Do we know what inappropriate laboratory utilization is? A systematic review of laboratory clinical audits. JAMA 1998;280:550-558.[Abstract/Free Full Text]
  4. Wong ET, Lincoln TL. Ready! Fire! ... Aim. JAMA 1983;250:2510-2513.[Abstract]
  5. Kelly JT. Role of clinical practice guidelines and clinical profiling in facilitating optimal laboratory use. Clin Chem 1995;41:1234-1236.[Abstract/Free Full Text]
  6. Thomas S. Standard setting in the Netherlands: impact of the human factor on guideline development. Br J Gen Pract 1994;44:242-243.[ISI][Medline] [Order article via Infotrieve]
  7. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ 1995;153:1423-1431.[Abstract]
  8. Grol R. Development of guidelines for general practice care. Br J Gen Pract 1993;43:146-151.[ISI][Medline] [Order article via Infotrieve]
  9. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993;342:1317-1322.[ISI][Medline] [Order article via Infotrieve]
  10. Grol R. National standard setting for quality of care in general practice: attitudes of general practitioners and response to a set of standards. Br J Gen Pract 1990;40:361-364.[ISI][Medline] [Order article via Infotrieve]
  11. Elson RB, Connelly DP. Computerized patient records in primary care. Their role in mediating guideline-driven behavior change. Arch Fam Med 1995;4:698-705.[Abstract/Free Full Text]
  12. Shiffman RN, Brandt CA, Liaw Y, Corb GJ. A design model for computer-based guideline implementation based on information management services. J Am Med Inform Assoc 1999;6:99-103.[Abstract/Free Full Text]
  13. Lomas J. Words without action? The production, dissemination and impact of consensus recommendations. Annu Rev Public Health 1991;12:41-65.[ISI][Medline] [Order article via Infotrieve]
  14. Forrest D, Hoskins A, Hussey R. Clinical guidelines and their implementation. Postgrad Med 1996;72:19-22.[Abstract]
  15. Onion CW, Bartzokas CA. Changing attitudes to infection management in primary care: a controlled trial of active versus passive guideline implementation strategies. Fam Pract 1998;15:99-104.[Abstract/Free Full Text]
  16. Tierney WM, Overhage JM, Takesue BY, Harris LE, Murray MD, Vargo DL, et al. Computerizing guidelines to improve care and patient outcomes: the example of heart failure. J Am Med Inform Assoc 1995;2:316-322.[Abstract/Free Full Text]
  17. Henry SB, Douglas K, Galzagorry G, Lahey A, Holzemer WL. A template-based approach to support utilization of clinical practice guidelines within a electronic record. J Am Med Inform Assoc 1998;5:237-244.[Abstract/Free Full Text]
  18. Hunt DL, Haynes RB, Hannah SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998;15:1339-1346.
  19. Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinicians performance and patient outcome. Ann Intern Med 1994;11:497-499.
  20. Smith BJ, McNeely MDD. The influence of an expert system for test ordering and interpretation on laboratory investigations. Clin Chem 1999;45:1168-1175.[Abstract/Free Full Text]
  21. Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc 1999;6:104-114.[Abstract/Free Full Text]
  22. Van der Lei J, Duisterhout JS, Westerhof HP, Van der Does E, Cromme PVM, Boon WM, et al. The introduction of computer-based patient records in the Netherlands. Ann Intern Med 1993;119:1036-1041.[Abstract/Free Full Text]
  23. van Wijk MAM, Van der Lei J, Mosseveld M, Bohnen AM, van Bemmel JH. Assessment of decision support for blood test ordering in primary care. A randomized trial. Ann Intern Med 2001;134:274-281.[Abstract/Free Full Text]
  24. Van Wijk MA, Bohnen AM, Van der Lei J. Analysis of the practice guidelines of the Dutch College of General Practitioners with respect to the use of blood tests. J Am Med Inform Assoc 1999;6:322-331.[Abstract/Free Full Text]
  25. Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H. Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ 1998;317:858-861.[Abstract/Free Full Text]
  26. Langley C, Faulkner A, Watkins C, Gray S, Harvey I. Use of guidelines in primary care-practitioners’ perspectives. Fam Pract 1998;15:105-111.[Abstract/Free Full Text]
  27. Grol R, Wensing M. Implementation of quality assurance and medical audit: general practitioners’ perceived obstacles and requirements. Br J Gen Pract 1995;45:548-552.[ISI][Medline] [Order article via Infotrieve]
  28. Bugter-Maessen AM, Winkens RA, Grol RP, Knottnerus JA, Kester AD, Beusmans GH, et al. Factors predicting differences among general practitioners in test ordering behaviour and in the response to feedback on test requests. Fam Pract 1996;13:254-258.[Abstract/Free Full Text]
  29. Geldrop WJ, Lucassen PLBJ, Smithuis LOMJ. Een probleemgeorienteerd aanvraagformulier voor laboratoriumonderzoek. Huisarts Wet 1992;35:192-197.
  30. Winkens RAG, Pop P, Grol RPTM. Effect of feedback on test ordering behavior of general practitioners. BMJ 1992;304:1093-1097.
  31. Winkens RA, Pop P, Bugter-Maessen AM, Grol RP, Kester AD, Beusmans GH, Knottnerus JA. Randomised controlled trial of routine individual feedback to improve rationality and reduce numbers of test requests. Lancet 1995;25:498-502.
  32. Winkens RAG, Pop P, Grol RP, Bugter-Maessen AM, Kester AD, Beusmans GH, et al. Effects of routine individual feedback over nine years on general practitioners’ requests for tests. BMJ 1996;24:490.
  33. Kluijt I, Zaat JOM, Van der Velden J, van Eijk JTM, Schellevis FG. Voor een prikje: Huisarts en bloedonderzoek. Huisarts Wet 1991;34:67-71.
  34. Dijksterhuis PH, Van Boven C, Lamberts H. Waarom aanvullend onderzoek?. De functie van het aanvullend onderzoek door de huisarts Huisarts Wet 1994;37:467-472.
  35. Foets M, Van der Velden J, De Bakker DH. Dutch National Study of General Practice 1992:228 NIVEL Utrecht. .



The following articles in journals at HighWire Press have cited this article:


Home page
 Arch Intern MedHome page
J. Meyerovitch, P. Rotman-Pikielny, M. Sherf, E. Battat, Y. Levy, and M. I. Surks
Serum Thyrotropin Measurements in the Community: Five-Year Follow-up in a Large Network of Primary Care Physicians
Arch Intern Med, July 23, 2007; 167(14): 1533 - 1538.
[Abstract] [Full Text] [PDF]

Home page
 Clin. Chem.Home page
M. J. Poley, K. I. Edelenbos, M. Mosseveld, M. A.M. van Wijk, D. H. de Bakker, J. van der Lei, and M. P.M.H. Rutten-van Molken
Cost Consequences of Implementing an Electronic Decision Support System for Ordering Laboratory Tests in Primary Care: Evidence from a Controlled Prospective Study in The Netherlands
Clin. Chem., February 1, 2007; 53(2): 213 - 219.
[Abstract] [Full Text] [PDF]

Home page
 Int J Qual Health CareHome page
R. Bindels, A. Hasman, M. Derickx, J. W. J. van Wersch, and R. A. G. Winkens
User satisfaction with a real-time automated feedback system for general practitioners: a quantitative and qualitative study
Int. J. Qual. Health Care, December 1, 2003; 15(6): 501 - 508.
[Abstract] [Full Text] [PDF]

Home page
 BMJHome page
J. D Emery, I. N Purves, R. Beaumont, T. Fahey, L. Mitchell, F. Sullivan, A. Montgomery, P. Gregor, J. T van Wyk, P. W Moorman, et al.
Effect of computerised evidence based guidelines
BMJ, February 15, 2003; 326(7385): 394 - 394.
[Full Text]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit an electronic Letter to
the Editor about this paper
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (11)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by van Wijk, M. A.M.
Right arrow Articles by van Bemmel, J. H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by van Wijk, M. A.M.
Right arrow Articles by van Bemmel, J. H.
Related Collections
Right arrow Laboratory Management
Right arrow Evidence Based Laboratory Medicine and Test Utilization

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS