[StATS]: A nonspecific diagnostic test (October 13, 2005)
Someone posted a note on the IRBFORUM complaining that the IRB was preventing him/her from volunteering for a test. This company has a very promising “non-intrusive Universal Early Cancer Detection test” but the company could not take any volunteers before getting IRB approval. There were a lot of good comments, but one thing that caught my eye in a later email was the following claim:
Besides, the test itself is non-intrusive: they will take 2 micro liter samples of blood and within 3 hours we get our results on any type of 1_week_or_older cancer.
This claims illustrates a very important point: research claims which lack specificity also lack credibility. A treatment that claims to cure everything probably cures nothing.
Specificity was one of the nine conditions that Sir Bradford Hill developed in 1965 to argue when the evidence was persuasive enough to establish a causal link. There are a few limited exceptions (aspirin is helpful for a wide range of conditions) but the principle has stood the test of time.
Let’s look again at the original claim. This diagnostic test can detect ANY type of cancer. That’s certainly possible, but extremely unlikely. The ways that breast cancer, melanoma, leukemia, prostate cancer, lung cancer, bladder cancer, etc. progress are quite different, and it extremely unlikely that a diagnostic test could find the single biomarker that was common to all these types of cancer, because that biomarker probably does not exist.
The claim is even more non-specific than that, though. The test, apparently, can detect cancer that is only ONE WEEK OLD. That is a rather amazing claim if you think about it. Most one week old tumors are microscopic, I suspect, but this test is claimed to be sensitive enough to detect even these tumors.
There are other technical issues worth discussing, such as the possibility that this test will confuse benign and malignant tumors, but the bottom line is that a diagnostic test that claims to detect any type of tumor, even one week old tumors, is probably worthless.
Now, I have no problem with letting these scientists test their product (carefully and ethically, of course). If they are right and I am wrong, then they can quote my email and laugh at me when they claim their Nobel Prize. But based on past experience, claims that are overly broad usually represent either delusional thinking or the use of highly biased research samples.
You can find an earlier version of this page on my website.