**Editorial**

# Numbers needed to treat derived from meta-analysis: A word of caution

PDF*ACP J Club*. 2003 Mar-Apr;138:A11. doi:10.7326/ACPJC-2003-138-2-A11

*Related Content in the Archives*

• Letter: Numbers needed to treat derived from meta-analysis: a word of caution

In clinical trials, treatment effects from binary outcomes, such as “alive” or “dead,”
can be presented in various ways (e.g., relative risk reduction [RRR] and absolute
risk reduction [ARR]) (1, 10). (See Glossary for definitions and calculations). Alternatively, the *number needed to treat* (NNT) is an expression of the number of patients who need to be treated to prevent
one additional adverse event (2-4). Mathematically, the NNT equals the reciprocal of the ARR. Many journals now report
results from clinical trials using the NNT, along with 95% confidence intervals (CIs)
(5).

Since its introduction (3), a debate has ensued whether reporting NNTs from meta-analyses is misleading (4, 6, 7). *ACP Journal Club* has devoted attention to this debate, and methodologic questions have been raised
in various review discussions and commentaries (8, 9). The objective of this editorial is to provide *ACP Journal Club* readers with caveats and suggestions when applying NNTs from a meta-analysis to an
individual patient. We highlight 2 problems. First, NNTs from a meta-analysis are
subject to variation in risk differences among the studies included in the meta-analysis
as well as in baseline risks. Second, applying NNTs to an individual requires adjustment
for their baseline risk.

NNTs derived from meta-analyses are affected by variations in risk differences among the studies, as well as baseline event rates in control groups of randomized controlled trials (1, 2). Summary estimates of NNTs assume constant risk differences between trials, a problematic assumption because of inevitable variation in baseline event rates between trials, differences in outcomes considered, effects of secular trends on disease risk, and differences in clinical setting as well as duration of follow-up (i.e., time horizon) (2, 10). In primary prevention of chronic disease, such as cardiovascular disease, the effect of time trends will become noticeable.

Several approaches have been introduced to derive or present NNTs from summary risk
estimates of meta-analysis. *ACP Journal Club* readers have become familiar with standard summary presentations of meta-analytic
research that provides NNTs based on pooled risk differences. However, in meta-analysis,
the assumption underlying the pooling of any effect estimate is that measures are
reasonably homogeneous across trials and that any variation can be attributed to random
chance. In the case of the NNT, this means that absolute risk differences should be
constant across studies. Growing empirical evidence suggests that relative risks and
odds ratios in most instances provide more homogeneous estimates than absolute risk
differences (1). For this reason, clinicians should use NNTs that are derived from pooled estimates
of relative risks instead of absolute risk differences.

Application of NNTs from meta-analyses to individual patients also requires attention to baseline risks, subject to the following conditions: knowledge of the baseline risk of the study patients and the estimated risk of the individual patient to whom the NNT will be applied, and information on the time horizon of the study (e.g., the median follow-up time).

We illustrate these points using the meta-analysis of randomized controlled trials
investigating the effect of n-3 polyunsaturated fatty acids on clinical endpoints
in patients with coronary heart disease as an example (11, 12). The meta-analysis found a 19% (CI 10% to 27%) RRR of overall mortality favoring
the use of n-3 polyunsaturated fatty acids. In the *ACP Journal Club* structured summary of this meta-analysis, the NNT derived from a pooled risk difference,
and based on an average baseline risk of 9.5%, is 73 (CI 49 to 147) (11). However, this estimate does not reflect the high variability of baseline risk of
the included trials. As discussed by the authors of the original meta-analysis (12), the baseline risk of included trials ranged from 2% to 22%. Based on the pooled
relative risk estimate, the corresponding NNT for an average follow-up of 18 months
is 263 (CI 185 to 500) for a baseline risk of 2% and 24 (CI 17 to 45) for a baseline
risk of 22%. This example illustrates that if risk differences and baseline event
rates in meta-analysis vary considerably across trials, the NNT derived from pooled
absolute risk differences is not very informative and may be misleading if used to
assess the benefit of a given intervention for an individual patient. Only if the
baseline risk does not vary considerably across trials may the NNT derived from pooled
risk differences be applied to determine the potential benefit for an individual patient.

Cook and Sackett (4) and Straus (13) have proposed a simple method to apply estimates of RRRs from single trials or a meta-analysis to individual baseline risk. They suggest comparing the patient’s baseline risk with that of a typical patient in the published trial. If the baseline risk of an individual patient is a factor f compared with the baseline risk of a typical study patient and the relative risk is constant, the ARR for the patient is scaled according to the same factor f. The estimated NNT corresponding to patients at the revised baseline risk is therefore simply the study NNT multiplied by f. Thus, in our example, if a high-risk patient is judged to have baseline risk 11 times higher than that of a low-risk patient, then f = 11 and the corresponding NNT is 24 × 11 = 264. Confidence intervals can also easily be obtained by multiplying the limits of the corresponding confidence interval from the original study with the factor f. However, as we have mentioned, this method should not be used with NNTs that are derived from pooled estimates of absolute risk differences. For example, if a clinician used an NNT of 73 based on absolute risk differences and a weighted control event rate of 9.5% to assess the benefit of n-3 polyunsaturated fatty acids for a patient with a baseline risk of 2%, the resulting NNT would be 347 (compared with 263 based on relative risk estimates). Therefore, clinicians should be wary when interpreting NNTs derived from pooled risk differences of a meta-analysis if baseline risks or absolute risk differences vary across trials. In this case, it makes no sense to use an NNT for the “average” single trial patient or the “average” meta-analysis patient.

### Acknowledgments

We thank Matthias Egger and Brian Haynes for their critical review and contributions to this paper.

**Arthur Marx, MD, MPH**

Basel Institute for Clinical Epidemiology

Basel, Switzerland**Heiner C. Bucher, MD, MPH**

Basel Institute for Clinical Epidemiology

Basel, Switzerland

## References

1. **Deeks JJ, Altman DG.** Effect measures for meta-analysis of trials with binary outcomes. In: Egger M, Davey
Smith G, Altman DG, eds. Systematic Reviews in Health Care. 2d ed. London: BMJ Publishing
Group; 2001:313-35.

2. **Ebrahim S.** Numbers needed to treat derived from meta-analyses: pitfalls and cautions. In: Egger
M, Davey Smith G, Altman DG, eds. Systematic Reviews in Health Care. 2d ed. London:
BMJ Publishing Group;2001:386-99.

3. **Laupacis A, Sackett DL, Roberts RS.** An assessment of clinically useful measures of the consequences of treatment. N Engl
J Med. 1988;318:1728-33. [PubMed ID: 3374545]

4. **Cook RJ, Sackett DL.** The number needed to treat: a clinically useful measure of treatment effect. BMJ.
1995;310:452-4. [PubMed ID: 7873954]

5. **Altman DG.** Confidence intervals for the number needed to treat. BMJ. 1998;317:1309-12. [PubMed ID: 9804726]

6. **Smeeth L, Haines A, Ebrahim S.** Numbers needed to treat derived from meta-analyses–sometimes informative, usually
misleading. BMJ. 1999;318:1548-51. [PubMed ID: 10356018]

7. **Lubsen J, Hoes A, Grobbee D.** Implications of trial results: the potentially misleading notions of number needed
to treat and average duration of life gained. Lancet. 2000;356:1757-9. [PubMed ID: 11095272]

8. **Guyatt GH, Cook DJ, Jaeschke R.** How should clinicians use the results of randomized trials? [Editorial]. ACP J Club.
1995;122:A12-3. [PubMed ID: 7704477]

9. **Glasziou P, Guyatt GH, Dans AL, et al.**Applying the results of trials and systematic reviews to individual patients [Editorial].
ACP J Club. 1998;129:A15-6. [PubMed ID: 9825009]

10. **McAlister FA.** Commentary: relative treatment effects are consistent across the spectrum of underlying
risks…usually. Int J Epidemiol. 2002;31:76-7. [PubMed ID: 11914298]

11. **Simon JA.**Review: n-3 polyunsaturated fatty acids reduce fatal MI, sudden death, and overall
mortality in coronary heart disease. ACP J Club. 2002;137:49. [PubMed ID: 12207427]

12. **Bucher HC, Hengstler P, Schindler C, Meier G.** N-3 polyunsaturated fatty acids in coronary heart disease: a meta-analysis of randomized
controlled trials. Am J Med. 2002;112:298-304. [PubMed ID: 11893369]

13. **Straus SE.** Individualizing treatment decisions. The likelihood of being helped or harmed. Eval
Health Prof. 2002;25:210-24. [PubMed ID: 12026754]