I am working with a new group looking at early childhood interventions, and they want my advice about a variety of issues related to Evidence-Based Medicine (EBM). Why would a Statistician provide advice about EBM? Well, I can’t claim that I know everything, but it might help to elaborate on how I got interested in this topic.
One of my Statistics teachers at the University of Iowa described Statisticians as “guardians of the scientific method.” I do not totally agree with that sentiment. I don’t want to be a guardian of anything, much less something as important as the scientific method. Nevertheless, the phrase “guardians of the scientific method” does emphasize the centrality of Statisticians to almost every research endeavor.
I got interested in EBM from almost my first day at Children’s Mercy Hospital (I started in 1996). This was mostly because I was invited to a bunch of different journal clubs. The organizers wanted someone who could explain those really complicated statistical analyses.
Fair enough. I always like examining how data is analyzed. But the realization came early to me that explaining the data analysis was the least important part of my job. Issues about how the study was conducted–things like was randomization used, did they blind the treatments–were far more important than how the data was analyzed. After all, the fanciest analysis in the world won’t salvage a poorly designed study. The converse is also true. If the study is designed well, then any reasonable analysis will probably be fine.
I wrote about EBM on my website (www.pmean.com) and a series of invited editorials for the Journal of Andrology. These I eventually consolidated into a book, Statistical Evidence in Medical Trials.
EBM is a very broad field. One definition of EBM divides it into the five A’s:
- Ask an well formulated question.
- Acquire the evidence associated with that question.
- Appraise the quality of the evidence.
- Apply the evidence in your workplace.
- Assess whether the application had an impact.
My efforts in EBM focus mostly on the “appraise” step. The quality of the evidence, as I mentioned above, is determined mostly with how the data was collected. As a Statistician, I know a lot about how to collect data.
As a Statistician, I can’t pretend that I know as much as the medical professional about the problem being addressed using EBM. As a relative outsider, though, I can provide several complementary strengths to the subject matter experts.
First, I tend to think about issues very abstractly. I decompose complex multiple outcomes into the individual components and assign those components abstract names like Y1, Y2, … I do a similar decomposition of a complex intervention and assign abstract names like X1, X2, …
Once the problem is reduced to Y’s and X’s, it is easy to identify multiple approaches, especially approaches that may have been developed in different disciplines.
Second, I don’t come in with a large number of pre-existing biases. I was working with a dermatologist on a problem and she complained that so many doctors have a phobia about the use of steroids. Steroids are indeed potent drugs with significant side effects. But you can’t let the potency and side effects scare you away from any use of the drugs. As an outsider, I don’t have any idea about steroids, so I am going to be more even handed in appraising the evidence for and against the use of steroids.
Third, I have to ask lots of questions to understand the problem being addressed by EBM, and the process of asking questions helps the subject matter experts to more clearly define their problem. This can, at times, force the experts to rethink the rationale for their approach.
Now I would be the first to admit that taking an abstract approach can have serious problems. If I take an abstract perspective, it always has to be brought back to the special environment of the problem being addressed.