[StATS]:(2006-07-10) **When can a conservative trust a liberal information source (and vice versa)<U+FFFD> **
I have a brother-in-law who loves to debate politics and religion. He always takes an aggressively conservative stand (I'm a flaming liberal, but try not to mention politics too much on this weblog). Often he will cite a "liberal" source, such as the New York Times to support his arguments, and although he does not trust most of what is published in these liberal sources, he will still cite them when they make a point in favor of a conservative viewpoint. His rationale is when a liberal source cites data supporting a conservative cause, they only do it grudgingly and because the facts are too overwhelming to ignore.
I suppose I do the same thing myself, but with the politics reversed. But this is a dangerous approach to take for several reasons.
- there are several dimensions to political beliefs--libertarians, for example, have tendencies of both liberals (on drug laws) and conservatives (on tax policy).
- sometimes extreme liberals and extreme conservatives have the same motive for exaggerating a problem, since it helps generate publicity and/or political contributions.
- sometimes facts are unreliable for reasons other than politics.
Another problem with this approach is that it encourages cherry picking, that is, the selective reporting of data that supports your cause. If you read the New York Times for the supposedly rare tidbit of conservative data and ignore all the other data, you end up just mindlessly reinforcing your own perspective on life.
Information sources in science
The purpose of this weblog is not to debate politics, but to discuss research methodology, but I think the same thing applies in scientific circles. Critics of evolution, for example, will frequently take quotes out of context that seems to sow doubt about the theory of evolution. For example, a researcher might start out a paper with a statement like "The Cambrian explosion presents the most challenging problem for the theory of evolution"
The goal of this statement is to point out why research on the Cambrian explosion is especially valuable, but a quote like this is paraded around by the Intelligent Design community as proof that evolutionary biologists know that evolution is a weak theory. There may be 100 quotes stating the exact opposite, but the one quote is the only one that matters.
It is as if all these scientists were caught in a rare moment of honesty admitting that evolutionary theory really is full of holes. It does not matter that others in the same specialty would counter the critiques, that others would suggest other reasons for the problems noted, consistent with (or even demanded by) evolutionary theory. Nor does it even matter that the scientists quoted do not themselves see their criticisms as falsifying evolution as the creationists do. www.ncseweb.org/resources/articles/8828_issue_14_volume_4_number_4__6_23_2003.asp
Critics of evidence based medicine make the same mistake. They are fond of citing a statistic that less than half, less than 25%, or less than 15% (the number varies) of all medical practices are based on solid research. While there is indeed some data to suggest that a large fraction of medicine is not based on solid research, there is more data to suggest the opposite.
- The evidence for evidence-based medicine. R. Imrie, D. W. Ramey. Complement Ther Med 2000: 8(2); 123-6. [Medline] (Evidence, Resources)
Some additional discussion of this is on the web at
There's a closely related concept called confirmation bias, which is the tendency to selectively remember evidence that supports your hypothesis, but to ignore or forget or disregard evidence that refutes your hypothesis.
Joel Best, author of two very good books about the use and abuse of statistics,
- Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists. Joel Best (2001) Berkeley, California: University of California Press. [BookFinder4U link]
- More Damned Lies and Statistics. Joel Best (2004) Berkeley and Los Angeles, CA: University of California Press. [BookFinder4U link]
commented in the second book about this problem
People like examples of an opponent's bad statistics, but they don't care to have their own numbers criticized because, they worry, people might get the wrong idea: criticizing my statistics might lead someone to question my larger argument, so let's focus on the other guy's errors and downplay mine.
This is not to say that you shouldn't consider the source as you evaluate a fact. There is empirical evidence that conflicts of interest, especially financial conflicts of interest, tend to lead to distortions of the facts. I discuss conflict of interest in several places on my weblog:
- Stats: What constitutes a conflict of interest? (May 4, 2006)
- Stats: More on conflicts of interest (March 23, 2005)
- Stats: Conflict of interest (July 20, 2004)
- Stats: Conflict of Interest (February 25, 2004)
<U+FFFD>Just be careful that you don't overreact.
Intellectual conflict of interest
Of special interest, is the so-called "intellectual conflict of interest." A researcher was barred from an FDA review panel because he had made some public comments that were critical of a certain drug and manufacturer.
Their rationale, I suspect, was to eliminate anyone who had "already made up their minds about the problem" but researchers who had made public comments defending the drug company were not similarly barred. There were also researchers with financial conflicts of interest on the same review panel, but these researchers received waivers.
A similar claim is that researchers who work for a federal agency or who receive grants from federal agencies have a vested interest in exaggerating the size of the problem that they are researching in order to justify more government money for this area. This is an easy charge to make and one that looks very attractive on its face, but there is no empirical evidence to support this allegation.
So how should a conservative approach a liberal resource, and vice versa? Here are some general guidelines.
- Read the whole article. Don't stop when you've found the fact that supports your viewpoint. Are there any data, statistics, or facts in the same article that might contradict the fact that you are trying to use? Are you taking the facts out of context. Were the data collected to make a different point than the one you are trying to make?
- Play a what if game. What if the fact that I am using ended up supporting the exact opposite conclusion. What criticisms might I tend to consider in this case? Make a regular habit of reading and critiquing viewpoints that you disagree with. Note the critiques that you use regularly in this exercise and apply them with equal vigor to the viewpoints that you agree with.
- Don't pretend that you understand the motivations of the writer. You don't. No one is able to look into the soul of another person and discern their true intent. If you criticize a piece of data, do so based on overt and objective standards.
But don't be too open-minded
Most people who are avid liberals or conservatives don't fall into this trap, but it is worth noting that you can sometimes be too lax and uncritical of competing ideas. While you should pay serious attention to ideas that disagree with your world view, you don't want to pretend that all arguments for varying perspectives are equally valid. In most situations, one side can marshal far stronger arguments for their position than the other side can. It's not that they are better debaters, but merely a recognition that there is more evidence and supporting facts on one side.
It is a mistake to pretend that there is always a symmetry to the two sides of an argument. The facts supporting a causal link between vaccines and autism are very weak, and most of the data supports the opposite view. So don't be too open-minded about opposing arguments.
As the Baptist minister Carlyle Marney wisely noted, A window stuck open is as useless as a window stuck closed. In either case, you've lost the use of the window. www.brucewoolley.com/TherapeuticsReport/1998/Oct98.html
This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Conflict of interest.
interest](../category/ConflictOfInterest.html). for pages similar to this one at [Category: Conflict of with general help resources. You can also browse Children's Mercy Hospital website. Need more information? I have a page reproducing it here as a service, as it is no longer available on the Hospital. Although I do not hold the copyright for this material, I am This page was written by Steve Simon while working at Children's Mercy