In February of 2005, The American Psychiatric Association (APA), National Alliance for the Mentally Ill (NAMI), and the National Mental Health Association (NMHA) released a statement publicly criticising the use of evidence-based research reviews for policy decisions affecting patients. Further, they argued against the use of evidence-based standards to guide reimbursement for healthcare services. We reviewed the statement and are concerned that many of the arguments are incorrect or misleading. In this note, we offer our responses by reproducing quotes from the APA-NAMI-NMHA statement and commenting on the merits of the arguments.

CLINICAL JUDGMENT AS EVIDENCE

The statement suggests that clinical experience should be given great weight in evidence-based decisions. The statement argues:

“… true evidence-based approaches marry all available and appropriate scientific research with clinical experience to ensure treatments lead to the best possible outcomes.”

We agree that clinical expertise is essential to discern which treatments supported by scientific evidence best match the needs and preferences of particular patients. In the best and most widely quoted definition of evidence-based medicine, Eddy states “Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values.”¹ However, clinical judgment is known to vary² even among experts. Dr Eddy reports widely divergent estimates of breast cancer implant rupture rates by 58 experts (around 50% of experts estimate a 10% or lower implant rupture rate and 20% estimate a 75% or higher rupture rate with the rest scattered in between); he has reported similar findings for experts’ judgment in other health conditions.

One of the reasons that evidence-based guidelines have come to replace strictly clinical judgment is that there is substantial evidence for regional variation in the application of medical care.^3–,⁵ For example, Medicare spends nearly twice as much for each Southern California recipient in comparison to each recipient in New Mexico.⁴ However, we have no evidence that residents in high expense areas obtain better health outcomes than those in low expense areas.^6,7 A substantial literature documents differences between Boston, Massachusetts and New Haven, Connecticut. Medicare spends $1.64 per recipient in Boston for each dollar it spends in New Haven. Yet, on every known health indicator, those living in New Haven do at least as well as those in Boston.⁴ In fact, new evidence suggests that those living in high expense areas actually do somewhat worse than those living in areas that receive less medical care.⁶ The major factor that drives variation is the supply of providers. In areas where there is an excessive number of healthcare providers, more tests are done, more services are delivered, and costs are higher. However, the benefit appears to be to the providers—not to the patients.⁷ Evidence-based medicine is designed to create guidelines so that healthcare is delivered in a more uniform way. When decisions about healthcare are guided by systematic evidence, patients get the benefit of the best scientifically supported treatments.

EVIDENCE-BASED MEDICINE AND SAVING MONEY

The statement plays on the common public misperception that the motivation for evidence-based medicine is to deprive patients of the best treatments in order to save money. Specifically, the statement says:

“using a narrow definition of evidence-base simply to rationalize budgetary objectives not only threatens patient health, but will ultimately cost taxpayers more … As policy-makers weigh all of these elements, quality of care should be the primary factor driving policy decisions”

It is a common misconception that evidence-based medicine is a thinly veiled attempt to ration healthcare. Although this reasoning evokes strong emotional responses, it has little merit because most advocates for evidence-based medicine do not favour cutting budgets for healthcare. They do acknowledge that healthcare resources are limited, and they favour using those limited resources wisely.² If budgets for healthcare are fixed, evidence-based medicine uses systematic methods and empirical evidence to determine how to gain the most help for the most people. In asking, “What do we know and how certain are we that we know it?”, evidence-based medicine seeks to inform rational decision making at all levels of healthcare, from practice to policy. Healthcare based on advocacy rather than evidence seems more likely to result in expenditures for services that may not benefit patients. For example, screening people for diseases for which early treatment offers no benefit over usual diagnosis and care may use large portions of an available budget with little health benefit. When this happens, fewer dollars remain to support programmes with good evidence for patient benefit. Ultimately, cost effectiveness and evidence-based medicine are not about saving money. Their rationale is to improve quality care and save lives by using resources wisely.

Evidence-based medicine advocates consider all forms of evidence, while emphasising critical appraisal of the best available research. Not all kinds of studies are given equal weight. Standards of evidence have evolved to place greater emphasis on randomised controlled clinical trials.⁸ The highest level of evidence comes from systematic reviews of several randomised trials. However, we don’t always have treatment effectiveness evidence from well conducted randomised clinical trials. In such cases we must rely more heavily on other forms of credible research evidence. When randomised trials are not available, other trial designs are considered as the best available evidence to judge some policy and practice relevant questions.⁹

Evidence-based medicine does not diminish the expertise of healthcare providers. In all instances, regardless of the kind of research evidence being applied, the clinical experience of well trained physicians and psychologists is essential in formulating practice relevant research (evidence generation), evidence reviews, guidelines for practice, and policies.¹⁰ At a minimum, in applying these evidence resources to practice, practitioners need to draw upon their clinical experience and knowledge of the patient to determine which evidence is relevant and how it applies to each particular individual. Sometimes, when no systematic research has been done, clinical experience with individual cases is all that’s available. Yet, substantial numbers of documented cases show that well reasoned, well intended intuitions are often incorrect when empirically evaluated.¹¹ Modern definitions of quality care increasingly link quality to applications of evidence supported treatments.¹² Consequently, it behoves practitioners to join the dialogue and develop methods to demonstrate evidence-based decision making in the care of individual patients.

NEW DRUGS ARE NOT NECESSARILY BETTER THAN OLDER, CHEAPER PROJECTS

It is often assumed that newer more expensive drugs must be better than older and less expensive options. The statement embraces this belief, stating:

“… scientific evidence does not exist to determine whether older, less expensive medications are as effective as newer, costlier ones; however, clinical practice has shown that newer medications have fewer harmful side effects.”

New expensive medications may be better; they may be of equal value; or they even may be of lesser value. Research comparing the older and newer medications provides the only fair and valid way to answer such questions. It bears noting that it is especially difficult to know the harms associated with new medications, since serious risks can go unrecognised even after a drug has been extensively prescribed for many years. Such was the case for the diet drugs fenfluramine (Pondimin) and dexfenfluramine (Redux), which had been prescribed for more than a decade before being found to cause heart valve defects.¹³

For efficacy as well as safety, there are numerous examples of more expensive medications that are no better than older cheaper ones. The proton pump inhibitors provide an excellent example. Expensive Nexium (esomeprazole magnesium) for example, is the chemical equivalent isomer of over-the-counter Prilosec (omeprazole). Evidence-based reviews for the treatment of high blood pressure demonstrate that older, inexpensive diuretics are actually more effective than some of the newer more expensive other classes of drugs.¹⁴ Massive evidence from the Food and Drug Administration (FDA)¹⁵ suggests that there are very few differences among drugs within a particular class (anti-hypertensives, selective serotonin reuptake inhibitors (SSRIs), etc), yet the cost for products in each category can vary dramatically. The burden of proof that there is a net benefit for the additional money when using new expensive drugs should rest with the manufacturers.

Having many possible treatments does not mean that all options are good. The statement suggests that having more treatment options automatically yields higher quality care.

“APA, NAMI and NMHA caution against relying exclusively on research reviews or misusing the label “evidence-based” to cut costs by limiting treatment options. Without a complete range of treatments and services valued by consumers and clinicians, people with mental illnesses will suffer needlessly and will be forced to rely on costlier and less effective care in other areas such as emergency rooms and justice systems.”

We agree that quality of care should be the primary force influencing healthcare decisions. At issue here is the definition of quality and the suggestion that physician autonomy inevitably leads to improved quality. Most emerging definitions of quality healthcare emphasise adherence to evidence-based guidelines.^12,16 Even before the availability of of high quality systematic reviews or guidelines, evidence-based practice means providing access and coverage for treatments that existing research has demonstrated to achieve better outcomes for patients. It comes as little surprise that several studies have shown that compliance to evidence-based standards results in better patient health status.¹⁷ Contrary to the interpretation that provider choice automatically improves clinical outcomes, we would assert that it is high quality collaborative care that achieves that end.¹⁸ We suggest that it is sometimes a healthcare professional’s responsibility to advise a patient, “As your healthcare provider I advise against using that treatment because its safety and efficacy are not well established by evidence.”

CONCLUSION

In summary, the APA, NAMI, and NMHA statement is presented as though it advocates for patients. The statement essentially argues that coverage should routinely be provided for new, expensive treatments before knowing their risks and benefits relative to less expensive older alternatives. It also suggests that practitioners’ judgement about an individual patient’s treatment should be unquestioned. The likely consequence of this approach is arbitrary practice variability resulting in higher healthcare costs and poorer quality healthcare. The mission of evidence-based practice is to optimise the chances that patients receive treatments that are most likely to enhance their health. The consequences of complete provider autonomy are that healthcare costs escalate with no clear connection to patient benefit. And, as we are seeing, when costs go up, employers are more likely to leave their employees uninsured. In other words, one consequence of provider autonomy is that the number of people with no healthcare increases and the average health of a community declines.¹⁹ Thus, instead of advocating for patients, the true beneficiaries of the APA-NAMI-NMHA statement proposal are more likely to be those who market healthcare, rather than the patients who consume it.