Scientific evidence and expert opinion in ACP Journal Club: commentary on commentaries
ACP J Club. 1994 Jan-Feb;120:A11. doi:10.7326/ACPJC-1994-120-1-A11
Related Content in the Archives
• Letter: Scientific evidence and expert opinion in ACP Journal Club: commentary on commentaries
Each ACP Journal Club article abstract is accompanied by a commentary from an expert in the field. Using common definitions, commentaries in ACP Journal Club are a combination of descriptive and explanatory statements of observation (about a study) tempered by opinions that may be interpreted as criticism. The commentator's tasks are to place the findings into the context of related research, identify any important scientific strengths or weaknesses of the study, and indicate the clinical application of the findings.
All abstracts in ACP Journal Club are derived from studies or reviews that meet explicit criteria for scientific merit. Commentators are selected, however, not by explicit criteria, but by their self-stated expertise, the editors' knowledge of them, and the editors' ratings of their performance on any previous commentaries they have done for ACP Journal Club.
Recently, authors of articles and commentators have engaged more often in prepublication debate. Since ACP Journal Club began, we have sent the commentary to the article author for information before publication. The first 17 issues of ACP Journal Club contained 506 abstracts, and in 143 instances (28%) correspondence was received from authors about either suggested changes to the abstract (n = 34; 6.7%) or disagreement with the contents of the commentary (n = 106; 21%). Beginning with the issue of November/December, 1991, we routinely returned authors' suggestions to commentators for possible revision of their commentaries. Although the proportion of commentaries prompting complaints from article authors remained steady (17%) just after the introduction of this policy, lately the extent of prepublication discussion has increased dramatically. For the 2 issues published just before the preparation of this editorial, disagreements between primary authors and commentators occurred for 22 of 60 abstracts (37%). These recent data may represent a chance occurrence or portend a potentially alarming trend.
What are the responsibilities of the editors and implications for readers when article authors and commentators have differences of opinion? At one extreme, unreserved support for the commentator's viewpoint suggests preference for the expert's opinion over the author's expert interpretation of the original findings. On the other hand, support for the article author (who may be biased about the merits and implications of her or his own research) could be interpreted as favoring the author's expert opinion over that of the commentator and also may reduce the willingness of commentators to contribute in the future. Editorial negotiation to resolve differences has been effective in most situations, but important points of scientific debate that occur on the telephone and facsimile cannot easily be represented on the printed page, especially considering our 1-page-per-article format. Worse still, if a disagreement occurs and readers themselves tend to give more weight to the commentators' opinions, this would be contrary to one of ACP Journal Club's guiding tenets: Let the facts speak for themselves.
In the ACP Journal Club editorial that introduced the purpose of commentaries (1), we stated that “readers may have different views of the value of commentaries than we [the editors] do.” To clarify the journal's perspective, the editorial presented the purpose of commentaries and the methods of selection of commentators and their tasks. In view of the recent trends presented here, we offer to extend our views on the role of commentaries and to further describe our approach for dealing with author-commentator disagreements.
We strongly support the practice of evidence-based medicine (2) and the empowerment of clinicians (and other readers) as users of the literature through critical appraisal of the relevance and scientific quality of original data from clinical research. We also believe that commentators provide a valuable service for readers by examining the complete article, applying their own expertise and experience, and helping readers to put scientific findings into the broader clinical and scientific context (as is often done by the authors in the Discussion section of their original published papers).
We do not attempt to stifle legitimate disagreements about the interpretation of evidence through backroom negotiations, but we do resolve some disagreements on matters of opinion behind the scenes and readers should be aware of this. We generally view the behind-the-scenes debates of authors and commentators as a healthy process that results in the publication of what authors and commentators agree on, or what they agree to disagree on, in which case we try to give the author the last word.
We also wish to emphasize to readers that commentators are often expressing their opinions about the clinical implications and appropriate applications of a study's findings. These opinions should not be given undue weight by readers, who may have better opinions themselves or who may want to wait for stronger evidence.
George P. Browman, MD, MSc
1. The commentaries in ACP Journal Club: Can you help out? Haynes RB. ACP J Club. 1994 Mar-April;120:A12.