Perfect isn’t quite good enough

Steve Simon

2006-12-12

Someone wanted me to double check their calculations for Fisher’s Exact test. If the control group, 3 out of 10 patients experienced an unfortunate outcome. In the treatment group none did (out of 6). You would think that a perfect result in the treatment group would be compelling, but the one-sided p-value for Fisher’s Exact test is 0.21.

That calculation is a bit disappointing, but perhaps not too surprising. The rule of three states that when you observe zero events in a sample of n patients, then 3/n is an approximate upper 95% confidence interval. So with 0/6, that upper limit would be 3/6 or 50% which is well above the control rate of 30%.<U+FFFD> So perfection just isn’t good enough when you have such small sample sizes.

Related web pages: