Current issues of ACP Journal Club are published in Annals of Internal Medicine


Quality improvement

Computer-based clinical decision support systems: a review

ACP J Club. 1994 May-June;120:86. doi:10.7326/ACPJC-1994-120-3-086


Source Citation

Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research. Ann Intern Med. 1994 Jan 15;120:135-42.


Abstract

Objective

To review the effects of computer-based clinical decision support systems (CDSSs) on clinical performance and patient outcomes.

Data sources

Studies were identified using the MEDLINE, EMBASE (Excerpta Medica), SCISEARCH, and INSPEC (International Information Service for the Physics and Engineering Communities) databases searching from 1974 to February 1992. Textbooks, previously published reviews, and conference proceedings in the areas of artificial intelligence and computer applications in medicine were also reviewed. Reference lists of relevant articles were scanned, and authors and experts in medical informatics were asked to identify additional studies.

Study selection

Studies were selected for review if the population of interest was composed of clinicians; the intervention was a computer-based CDSS evaluated in a clinical setting; the main outcomes were clinician performance, process of care, or patient outcomes; and the study design included contemporaneous controls, comparing patient care with a CDSS to patient care without one.

Data extraction

Two reviewers independently rated the selected studies for quality. Data concerning setting, clinicians and patients, interventions, and outcomes were extracted from each article and were verified. Separate summaries were prepared for physician performance and patient outcomes. Within these 2 categories, studies were classified according to the primary purpose of the CDSS: drug dose determination, diagnosis, or quality assurance.

Main results

793 citations were examined, and 28 controlled trials met the selection criteria. Evidence of improved physician performance existed in 3 of 4 studies of computerized aids for determining the dose for toxic drugs, 1 of 5 studies evaluating the effects of computerized decision aids for diagnosis, 4 of 6 studies of CDSSs designed to enhance the quality of preventive care, and 7 of 9 studies assessing the effect of CDSSs on the quality of care for active medical problems. A beneficial effect on patient outcomes was noted in 3 of 10 studies, including 1 study of computer-assisted diagnosis and 2 studies assessing the effect of CDSSs on the quality of active medical care.

Conclusions

The success of computer-based clinical decision support systems in improving physician performance was variable and depended on the task being studied. Few systems improved patient outcomes.

Sources of funding: Brian C. Decker Health Information Research Fund and the Medical Research Council of Canada.

For article reprint: Dr. R.B. Haynes, McMaster University Health Sciences Centre, Room 3H7, 1200 Main Street West, Hamilton, Ontario, L8N 3Z5, Canada. FAX 905-546-0401.


Commentary

Many have predicted that computers and artificial intelligence systems will revolutionize our approach to diagnosis and therapy, yet computers have made only minor inroads into direct clinical decision-making. In this carefully done critical appraisal of the clinical decision-support literature, Johnston and colleagues found only 28 reports of decision-support systems that had been adequately tested in clinical trials. Of these, only 15 showed improvement in physician performance. Decision-support systems that provided assistance with dosing toxic drugs and reminders for preventive care and appropriate test ordering were successful more often. Computer-aided diagnosis systems were successful less often.

The methods used in this paper are a model of how to do a critical review. The authors followed explicit and careful methods to ensure complete coverage of the literature, to appropriately select relevant citations, and to assess the methodologic rigor of each study. At each step, they used 2 independent raters to minimize any bias in selection or rating of the papers to be included. The authors were fair in their conclusions from the reviewed literature and were appropriately careful about not over-generalizing.

The most important conclusion is that the success of CDSSs is both task- and system-dependent. An elegant computer system does not guarantee improved physician performance or better patient outcome. Nevertheless, some striking successes have been reported (1), such as the decision-support system for inpatient ordering at the Indiana University School of Medicine that reduces average patient charges by $887 per admission. Rapid advances in computer technology and human interface design hold considerable promise for useful and effective CDSSs.

Thomas G. Tape, MD
University of Nebraska Omaha, Nebraska, USA