Is there something better than Evidence Based Medicine out there

Steve Simon

2010-09-20

Someone asked me about a claim made on an interesting blog post on the Science Based Medicine blog. The writer of this post, Kimball Atwood, claims that Science Based Medicine (SBM), that tries to draw a distinction between that practice and Evidence Based Medicine (EBM). SBM is better because “EBM, in a nutshell, ignores prior probability (unless there is no other available evidence and falls for the p-value fallacy; SBM does not.” Here’s what I wrote.

No. The gist of the science based medicine blog appears to be that we should not encourage research into medical therapies that have no plausible scientific mechanism. That’s quite a different message, in my opinion, that the message promoted by the p-value fallacy article by Goodman.

I view EBM as a tripod of best available evidence, physician insight and experience, and individual patient preferences. Take away any leg and the tripod falls.

A plausible scientific mechanism is certainly something that can and should be incorporated into physician insight and experience.

Everybody seems to criticize EBM for an exclusive reliance on randomized clinical trials (RCTs). The blog uses the term “methodolatry” in this context. A group of nurses who advocate a post-modern philosophical approach to medical care also criticized EBM and used an even stronger term, micro-fascism, to describe the tendency of EBM to rely exclusively on RCTs.

But I have not seen any serious evidence of EBM relying exclusively on RCTs. That’s certainly not what David Sackett was proposing in the 1996 BMJ editorial “Evidence based medicine: what it is and what it isn’t”. Trish Greenhalgh elaborates on quite clearly in her book “How to Read a Paper: The Basics of Evidence Based Medicine” that EBM is much more than relying on the best clinical trial. There is, perhaps, too great a tendency for EBM proponents to rely on checklists, but that is an understandable and forgivable excess.

So I think that this criticism of EBM is putting up a “straw man” to knock down. No thoughtful practitioner of EBM, to my knowledge, has suggested that EBM ignore scientific mechanisms.

I would argue further that it is a form of methodolatry to insist on a plausible scientific mechanism as a pre-requisite for ANY research for a medical intervention. It should be a strong consideration, but we need to remember that many medical discoveries preceded the identification of a plausible scientific mechanism.

There is also societal value to carefully test interventions that are widely used in society, even when those interventions have no plausible mechanism. These interventions are not wasteful of resources if they end up providing evidence that a widely adopted intervention is useless. This research will be taken more seriously by some than just a rant of “where’s the mechanism” (though there will always be some who will not accept evidence from the RCTs either).

As to whether we should use p-values or Bayes factors (as Goodman recommends), that is an entirely separate issue. In my general perception, researchers will informally adopt prior distributions and demands a higher standard of proof for interventions that have no plausible mechanisms. If we forced people to specify priors, it would help, but we’d not solve the problem because different people have different interpretations of the word “plausible” in the term “plausible scientific mechanism.” Thus, their prior distributions would be equally diverse leading to the same problems with deciding what interventions merit further investigation.

The blog raises important issues and comments intelligently on them. I disagree with the need to distinguish between SBM and EBM. Maybe we should distinguish between EBM and PIEBM (Poorly Implemented Evidence Based Medicine).

You can find an earlier version of this page on my original website.