More on the top ten studies in EBM

Steve Simon


[StATS]: More on the top ten studies in EBM (February 28, 2005)

I may never get this paper done, but the effort is still worthwhile in that I am learning a lot. I rediscovered a wonderful web page developed by Ben Djulbegovic, Randomized trials that changed medical practice, which along with Non-randomized trials that changed medical practice lists research studies that have changed how we practice medicine.

The latter page lists several criteria by which you can judge when a randomized trial would not be needed.

  1. The Oxford/Sackett “All or none” criterion. When a disease has 100% morbidity/mortality prior to the use of an intervention and the intervention now allows some fraction of patients to survive, then a randomized trial is not necessary. Alternately, when the morbidity/mortality of a disease drops to 0% after a new therapy is introduced, no randomized trial is necessary.
  2. The Italian/Nino Cartabellotta crieria: When a disease has a very high level of morbidity/mortality, a large reduction in risk when the therapy is applied, very few or rare side effects, a convincing biological mechanism, and no alternative treatment is available, then randomized trials are unnecessary.
  3. Nick Black’s criteria: These can be summarized in four words: unnecessary, inappropriate, impossible, or inadequate. When the effect is so dramatic that you can rule out any confounding variables, so that a randomized trial to balance out these covariates is unnecessary. Or the outcome is very rare and follow-up time is very long so that a randomized trial is inappropriate. Or strong clinician preference, ethical constraints, or legal obstacles make a randomized trial impossible. Or for the intervention being considered, the low external validity of randomized trials makes their use inadequate.

I’m not sure of the proper source for the first set of conditions, but it is similar advice I had heard in an article by Sir Michale Rawlins in Pharamfocus, though with a caution that the population being studied had to be homogenous and there had to be a plausible mechanistic explanation for the therapy.

Nino Cartabellotta and the Italian Group on Evidence Based Medicine (GIMBE) have several publications in Italian that might be the source of the second set of conditions.

I suspect that the last set of conditions comes from

but I have not seen the full article.

A few highlights from the Randomized trials that changed medical practice are:

Historical firsts. The very first published example of randomization in a medical trial appears in

and showed that streptomycin was superior to bed rest. Actually, a randomized trial testing the efficacy of immunization against whooping cough was started first, but did not get reported until 1951.

Two historical perspectives on the streptomycin trial appear in a special 1998 theme issue of the BMJ celebrating 50 years of progress in randomized trials:

Folic acid supplementation in pregnant women to prevent spina bifida. The Djulbegovic page cites

This was a large trial where 2471 pregnant women received a multivitamin supplement including folic acid, and 2391 pregnant women received a trace-element supplement (copper, manganese, zinc and vitamin C). In the latter group, six cases of neural-tube defects were found, while none were found in the folic acid group. Surprisingly<U+FFFD> this was not mentioned in an MMWR report published in 1992 which instead cites

probably because it was published in English, as well as

Interestingly, a brief news report in Nature in 1982 has the following abstract:

Controversy has erupted in Britain over the ethical implications of an impending clinical trial to investigate the role of folic acid and other multivitamin supplements in the prevention of neural tube defects. The study population will consist of women who had previously conceived a spina bifida child and who are now planning another pregnancy. Some critics, believing there is already sufficient evidence of a folic acid link to spina bifida, object to withholding folic acid from one of the high-risk control groups. Concern has also been expressed about the extent to which participants will be briefed. [Medline]

So the concept of equipoise is not a new one.

Placebo surgery trials. Two widely cited early randomized double blind trials for surgery are

These trials examined a treatment for angina that involved cutting an artery that leads away from the heart to encourage more blood flow to the heart itself through the narrowed coronary arteries. While anecdotal evidence for this procedure accumulated in the 1950s, these two randomized double blind trials showed that the surgery was ineffective.

These two studies are frequently cited as rationale for use of placebos and blinding. See, for example,

Some additional placebo surgery trials include

Again, these trials showed no additional effectiveness of the therapies over a placebo surgery. A placebo surgery trial is highly controversial, because patients in the placebo arm undergo all of the risks of surgery, (side effects of anesthesia, increased chance for infection, side effects associated with subsequent antibiotic use, etc.).

CAST trials. Three drugs, encainide, flecainide, and moricizine, were thought to prevent heart attacks by suppressing asymptomatic ventricular arrhythmias (actually, ventricular premature depolarization or VPD), a condition associated with arrhythmic death. A large scale randomized trial, however, showed that rather than reducing your risk of arrhythmic death, these drugs actually increased the risk of death. In 1989, the researchers ended the two arms of the study associated with encainide and flecainide early because of a higher rate of arrhythmia deaths, of nonfatal cardiac arrests, and a higher rate of overall mortality. The study was then redesigned to compare only the third drug, moricizine, to placebo, with some changes in the entry criteria to enroll only the more seriously ill patients. This redesigned study, CAST-II, was also ended early, because of excessive cardiac deaths during the first two weeks of drug exposure, and a futility analysis. The futility analysis showed that the current data held out little hope that additional data would accumulate to the point where there would be evidence of long term survival.

Why did this happen. One possible explanation is that only a few people with VPD will die from the condition, limiting the effectiveness of these drugs, but all those who take these drugs are exposed to potential side effects. The CAST trials illustrate the important principle that a change in a surrogate outcome (reduction in VPD) does not always translate into a reduction in mortality.

You can also browse for pages similar to this one at Category: Teaching resources.

You can find an earlier version of this page on my website.