[StATS]: Post hoc comparisons (created 2006-03-15)
Note: It’s been a while since I last used the “Ask Professor Mean” format on these web pages, and I can’t think of a good reason why I abandoned that format. I just saw a question on the Epidemiology email discussion group that<U+FFFD> I could adapt to the Ask Professor Mean format.
Dear Professor Mean, I need to run multiple comparisons among all possible pairs of means following an analysis of variance test. What is the best approach? Tukey? Scheffe? Bonferroni?
Entire books have been written about this topic:
- Simultaneous Statistical Inference Second Edition. Rupert G. Miller (1981) New York: Springer-Verlag. [BookFinder4U link]
- Multiple Comparisons Theory and Methods. Jason C. Hsu (1996) London: Chapman & Hall. [BookFinder4U link]
so even Professor Mean is incapable of providing a good summary. Furthermore, there is no consensus in the research community about the best approach. So plan on having someone point out the error in the approach you choose, no matter what your choice. There are two procedures, however, that are ESPECIALLY LIKELY to draw criticism: Duncan, because it is too liberal, and Scheffe because it is too conservative. Most other procedures work reasonably well and it may not be worth your time and trouble to try to distinguish among the choices.
I like Tukey because it is easy to understand and reasonably safe. But Tukey is going to encounter problems with seriously unbalanced data and/or seriously unequal variation from one group to another.
Further reading:
- Stats: Step-down procedures for multiple comparisons (June 16, 2005)
- Stats: Unequal sample sizes (November 24, 2004)
- Stats: Multiple degree of freedom tests (September 22, 2004)
- Stats: Guidelines for ANOVA models
- Stats: Bonferroni correction
This page was written by Steve Simon while working at Children’s Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children’s Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Analysis of variance.
variance](../category/AnalysisOfVariance.html). for pages similar to this one at [Category: Analysis of with general help resources. You can also browse Children’s Mercy Hospital website. Need more information? I have a page reproducing it here as a service, as it is no longer available on the Hospital. Although I do not hold the copyright for this material, I am This page was written by Steve Simon while working at Children’s Mercy