Current issues of ACP Journal Club are published in Annals of Internal Medicine


Cost-effectiveness analysis: Are the outputs worth the inputs?

ACP J Club. 1996 Jan-Feb;124:A12. doi:10.7326/ACPJC-1996-124-1-A12

Related Content in the Archives
• Editorial: Economic evaluation of health care interventions: an economist's perspectiveCost-effectiveness analysis: Are the outputs worth the inputs?

Thomas Carlyle (1795 to 1881) once tagged economists as the “respectable professors of the dismal science.” Dismal it may be, but in health care, economic analysis is here to stay. Policymakers, prepayment plan managers, and hospital administrators already make informal use of cost-effectiveness criteria to assess new technologies and clinical strategies. The governments in both Australia and Canada have approved guidelines for the economic evaluation of new drugs and devices. Meanwhile, in America, the voluntary approach prevails: Journal editors and leading academics have issued conflict-of-interest and other guidelines for adherents to the cost-effectiveness religion, lest they be accused of doctrinal departures or, worse still, the secret worship of Mammon.

The rush to embrace cost-effectiveness analysis is understandable. For many nations, public sector indebtedness has created an affordability crisis in health care. Open-ended fee-for-service payment plans are being pushed aside. Instead, global budget caps in publicly administered or privately managed care systems are used to contain overall levels of expenditure. A tight budget in a health care system, however, is like a tight budget in a household: It underscores the severity of the trade-offs that must be made when weighing alternative uses of the same resources. Do we renovate the basement or buy a family membership in a local gymnasium? Do we extend the hours of operation of the hospital's CAT scanner or endorse the routine use of tissue plasminogen activator (tPA) rather than streptokinase for our patients with acute myocardial infarction?

Enter the health economists, holding aloft tables of various interventions and their comparative cost-effectiveness ratios. The health care planner or clinician-qua-administrator somehow is supposed to examine these ratios and decide on the shopping basket of services that will maximize the health outcomes of patients in a local hospital, entire populations of patients in managed care chains, or even national health systems.

There are 7 good reasons why clinicians should maintain a healthy skepticism about the results of cost-effectiveness analysis and the usefulness of these results in purchasing and planning decisions.

1. Cost-effectiveness analysis is seldom about cost saving or cost minimization. Consider how such an analysis unfolds in comparing new drug N to old drug O. We determine how many extra years of life are gained by using drug N and the associated costs. The extra costs and life-years associated with the use of drug N are then divided to create a marginal cost-effectiveness ratio—the cost per life-year gained. Weights called “utilities” are commonly added to adjust the life-years for associated variations in quality of life, on a scale that rates perfect health as 1.0 and death as 0. The result becomes a cost-utility analysis with results expressed as costs per quality-adjusted life-year, or QALY.

Each of the economic analyses published in the November/December and current issue of ACP Journal Club falls into this genre, and only 1 projects better outcomes and reduced costs (1). Thus, the methods either predispose to spiraling expenditures, or, under conditions of capped budgets, demand reallocation of resources away from the interventions that are least efficient. This assumes, however, that all existing interventions have been evaluated in the same rigorous fashion as the new intervention. Obviously, that assumption is unrealistic.

2. The costs in cost-effectiveness analyses may not be reliable or generalizable. In particular, long-term costs are seldom understood with precision. Short-term costs can vary from hospital to hospital, state to state, country to country, and year to year. Moreover, the economics (and ethics) of the analysis are heavily influenced by the perspective taken. Is it more appropriate to use actual production costs or charges? Let us say we are taking a societal perspective wherein the productivity losses of premature death are incorporated in some fashion, often by extrapolating from the national average of wages and salaries for a person of that age. Are we then prepared to devalue retirees because they are no longer in the work force? Also, the economic costs in one sector may be a gain in another sector. Health economists tend to set these aside as transfers. But are we as physicians or citizens really indifferent to the distributional effect of innovation and resource reallocation in delivering medical care?

3. The long-term survival benefits (i.e., life-years gained) with new treatments are seldom defined at the point when hard decisions must be made about their adoption. Thus, educated guess-work is the norm. For example, the excellent article by Hamilton and colleagues (2) used Framingham data to estimate the life-expectancy benefits from treating persons with asymptomatic dyslipidemias with HMG-CoA reductase inhibitors. Unfortunately, no trial has confirmed that these gains are achievable. Mark and colleagues (3) examined the cost-effectiveness of tPA compared with streptokinase for acute myocardial infarction, incorporated 1-year survival data from the GUSTO trial, and used complex mathematical models to project the very long-term survival of the 1-year survivors. As a co-author, I believe these estimates are reasonable, but they are educated guesses.

4. The utilities used to “adjust” the life-years gained are variable, and the methods used to derive them are debatable. Some economists claim that the “standard gamble” is the reference method to determine how patients value their health status. With this method, a patient is offered 2 alternatives: continued existence in her current health state or to take a gamble on obtaining perfect health where the penalty for losing is death. A mind game ensues, wherein the chances of death or perfect health are shuffled about to calibrate the patient's strength of preference for his or her health state under these conditions of artificial uncertainty. This construct deliberately fuses health perceptions and attitudes about death, has limited test-retest reliability, and is further confused by the fact that perfect health may be sought by patients because of comorbid symptoms that are unrelated to the target condition affected by the treatment of interest. Moreover, health states change over time, and there are intermethodologic, interindividual, and intercultural variabilities in treatment preferences and health perceptions of patients. No clear basis exists for deciding whether to use volunteers, nominated representatives of the general public, or persons with the disease of interest to derive utility values.

5. In light of the foregoing, one appreciates that cost-utility analysis puts 3 uncertain quantities together in a single unstable ratio. In the numerator are the costs, which vary from place to place and year to year. In the denominator are the life-expectancy gains, which are usually estimates, and the utility weights, which have their own biases and sampling variation. (Indeed, the very concept of an average utility is something of an ethical contradiction in terms because health perceptions and risk attitudes are highly individualized.) Small wonder that the range of possible values for any cost-effectiveness ratio is very wide. The authors of each cost-effectiveness analysis that has been abstracted in ACP Journal Club have tried to make some allowance for uncertainty with sensitivity analyses (i.e., assumptions about some inputs are varied to determine how much the results change). None, in my view, has done an ideal job of exploring the effects of simultaneously varying the 3 key inputs in any analysis.

6. Most adherents to the religion of cost-effectiveness are utilitarians: Their goal is “utility maximization,” or the greatest good for the greatest number. But other strategies (e.g., “satisficing,” wherein one settles for a modest gain to avoid the risk for large losses) may be equally rational in some clinical and policy contexts (4). Moreover, distributional issues are hidden in every analysis. Consider 2 preventive programs that last 6 years. On average, the yield from 1 is an extra 15 weeks free of heart disease. The other gives 5% of participants an extra 2 to 6 years free of heart disease, 10% up to 2 years free of heart disease, and 85% no gain. Patients clearly prefer the latter, although it is arithmetically equivalent to the former (5). A corollary is that treatments yielding quality-of-life improvements for large numbers of persons may end up with cost-effectiveness ratios that are much better than treatments that are usually ineffective but occasionally save lives. As was evident when Oregon used such calculations in its decision to stop funding transplants for Medicaid recipients, this is perilous territory (6).

7. Because many treatments have not been rigorously appraised with cost-effectiveness analysis, the above-noted issue remains of how to interpret and apply the results of any analysis. Consider a shopping expedition: You buy varying numbers and types of grocery items according to a set of felt needs and preferences. Some items may go from the shelves to the cart and back again as you move through the aisles and recognize the superior utility of alternative purchases. Decisions for individual utility maximization cascade into the aggregate relations among price, supply, and demand. The mechanisms in health care purchasing, however, are much clumsier. Entire programs sometimes must be financed en bloc; downsizing them may incur sunk costs, diseconomies of scale, or quality problems.

To avoid a situation where every intervention or program must be benchmarked against every alternative, a potential approach is to simply set rough guidelines for what is and what is not an acceptable cost-effectiveness ratio. A common benchmark in this respect is hemodialysis, which costs about U.S. $40 000 to $50 000 per year of life sustained. Unfortunately, the cost-effectiveness and cost-utility ratios for most treatments are so unstable that the pass/fail grade of an intervention for any specific threshold may be strongly dependent on the assumptions used in the analysis (7).

Last, the Oregon experience belies the concept that cost-effectiveness analysis offers a straightforward mechanism for rational allocation of health care resources. Some doubtless will claim that cost-effectiveness analysis did not fail in Oregon (6, 8) and that Oregon's approach to Medicaid coverage is a model for the industrialized world. Others will happily note that Oregon's policy-makers frequently (and sensibly) departed from strict axioms of cost-utility analysis in their decisions. Still others may wonder about the legitimacy and generalizability of a process whereby well-insured middle-class administrators and professionals used debatable data to decide what benefits should be denied to the underclass in a nation that spends far more per capita on health care than any other on earth.

Given the economic imperative in modern health care, cost-effectiveness and cost-utility analyses will continue to dot the clinical literature. Many analyses will be frustratingly dependent on assumptions and guesswork, but even where incomplete data leave uncertainties about the conclusions, some directions for future research may be inferred. Above all, economic evaluation often helps clarify the trade-offs involved in purchasing decisions and policy formulation. The clinician-reader, however, might want to recall an imaginary society called “Erewhon” created by the Victorian social critic Samuel Butler (1835 to 1902). When confronted with fashionable ideas from one expert or another, the citizens of Erewhon were “easily led by the nose and quick to offer up common sense at the shrine of logic …” (9). In contrast, when using the results of cost-effectiveness analyses, physicians should temper their interpretations with common sense, leavened by compassion and a sense of justice.

David Naylor, MD
Institute for Clinical Evaluative SciencesUniversity of Toronto
North York, Ontario, Canada


1. Interferon-a2b might prolong life and save costs in HBeAg-positive chronic hepatitis B. ACP J Club. 1995 Nov-Dec;123:85.

2. Cost-effectiveness of lovastatin varied by age, sex, and nonlipid CHD risk factors. ACP J Club. 1995;123:84.

3. Costs and survival were higher for acute MI treated with tPA compared with streptokinase. ACP J Club. 1996;123:25.

4. Deber RB, Goel V. Using explicit decision ethics in decision analysis: when is it not rational to maximize expected utility? Med Decis Making. 1990;10:181-94.

5. Hux JE, Naylor CD. Communicating the benefits of chronic preventive therapy: does the format of efficacy data determine patients' acceptance of treatment? Med Decis Making. 1995;15:152-7.

6. Hadorn DC. Setting health care priorities in Oregon. Cost-effectiveness meets the rule of rescue. JAMA. 1991;265:2218-5.

7. Naylor CD, Williams JI, Basinski A, Goel V. Technology assessment and cost-effectiveness analysis: misguided guidelines? Can Med Assoc J. 1993;148:921-4.

8. Eddy DM. Oregon's methods. Did cost-effectiveness analysis fail? JAMA. 1991;266:2135-41.

9. Butler S. Erewhon. New York: Lancer; 1968:298.