Current issues of ACP Journal Club are published in Annals of Internal Medicine


Letters

Criteria for the evaluation of diagnostic tests

ACP J Club. 1994 July-Aug;121:27. doi:10.7326/ACPJC-1994-121-1-027



To the Editor

ACP Journal Club has criteria for selecting studies of diagnostic tests (1-3). It is, however, necessary to evaluate studies on diagnostic measures according to more detailed criteria. A working group of the German Society for Medical Documentation, Informatics and Statistics (GMDS) has proposed 4 clearly defined phases for the evaluation of diagnostic tests (4, 5). In phase 1, technical and methodologic investigations, “the fundamental principles for the application and applicability of the diagnostic test are investigated” (5). In phase 2, estimation of the test's sensitivity and specificity in selected patients, the distribution of test results is determined in healthy controls and diseased persons. In phase 3, controlled diagnostic studies provide an undistorted estimation of the test parameters in a clinical setting in daily routine use in contrast to phase 2 studies in which the disease status is already established. In addition to sensitivity and specificity, the prevalence and the predictive values are determined. Phase 4 investigations are analogous to the study of the effectiveness of therapies: “The effectiveness of a diagnostic measure should be assessed as a basis of a continuing risk-benefit-evaluation” (5).

According to these guidelines, ACP Journal Club' s criterion of “clearly identified comparison groups, at least one of which is free of the disorder or derangement of interest” (1), relates to phase 2 studies designed for the estimation of sensitivity and specificity in selected patients. ACP Journal Club's criterion of a mutually blind interpretation of the diagnostic standard and the test to be evaluated (1) is required for both phase 2 and phase 3 studies. An objective modus for the diagnosis (“gold standard”) is required for both phase 3 studies (4, 5) and Journal Club articles (1).

It is recommended to the editors of ACP Journal Club to classify the reviewed papers according to the proposed guidelines (4, 5) in order to rate the diagnostic studies more precisely and more informatively. This serves the purpose of avoiding inappropriate recommendations for the application of diagnostic tests in clinical routine, unless they have been evaluated according to an established method. It is also necessary to follow the guidelines published in the “Memorandum for the Evaluation of Diagnostic Measures” to avoid placing patients in ethically unacceptable study situations (4, 5).

Hartmut Tillil, MD
Heinrich-Heine-University
Düsseldorf, Germany

In response

Dr. Tillil and his working group colleagues have provided a valuable outline for the staged evaluation of diagnostic tests. Researchers should follow these guidelines, and clinical readers can and should apply them in their appraisal of reports of diagnostic test assessments, including the ones that appear in ACP Journal Club.

The guidelines are ahead of their time, however, because the evaluation of diagnostic tests at present seldom reaches phase 3 or 4. On the basis of our criteria for study selection, which set a lower cutoff at phase 2 of the working group guidelines, only about 11% of the articles we abstracted in 1993 were about diagnostic tests. Phase 3 studies are few and far between and phase 4 studies are almost never done. Thus, we abstract the “best available” studies in ACP Journal Club, recognizing that few of these are definitive.

From our perspective, Dr. Tillil's letter is most useful to readers as a reminder that ACP Journal Club starts the critical appraisal process but commentators and readers must complete it. To assist with this process, when we send diagnostic test articles out for comment we will provide the commentator with a copy of Dr. Tillil's letter, as an aid to providing comments about the clinical strengths and weaknesses of the findings. Readers also may wish to refer to more detailed criteria for the appraisal of diagnostic tests (6, 7).

The Editor


References

1. Purpose and Procedure. ACP J Club. 1991 Jul/Aug;A10-11. (Ann Intern Med. vol 115, suppl 1).

2. Guyatt GH.Readers' guide for articles evaluating diagnostic tests. ACP J Club. 1991 Sept/Oct;A16. (Ann Intern Med. vol 115, suppl 2).

3. Haynes RB.Change in criteria for studies of diagnostic tests. ACP J Club. 1991 Nov-Dec;A13. (Ann Intern Med. vol 115, suppl 2).

4. Arbeitsgruppe “Methoden der Prognose und Entscheidungsfindung.” Memorandum zur evaluierung diagnostischer ma?nahmen. In: Købberling J, Tramisch HJ, Windeler J, eds. Deutschen Gesellschaft für Medizinische Dokumentation, Informatik und Statistik e.V. (GMDS). Schriftenreihe der Deutschen Gesellschaft für medizinische Dokumentation, Informatik und Statistik (e.V., vol 10, Wagner G, ed.). New York: Schattauer;1989:1-16.

5. Working Group on Methods for Prognosis and Decision Making. Memorandum for the evaluation of diagnostic measures. J Clin Chem Clin Biochem. 1990;28:873-9.

1. Jaeschke R, Guyatt GH, Sackett DL, for the Evidence-based Medicine Working Group. Users' guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? JAMA. 1994;271:389-91.

2. Jaeschke R, Guyatt GH, Sackett DL, for the Evidence-based Medicine Working Group. Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? JAMA. 1994;271:703-7.