A commonly occurring question is what to do when you have a small population that you are sampling and your sample effectively constitutes all or almost all of the population. For example, I got the following email today.
I conduct pro bono surveys for schools in my area. For my real job we use some form of sampling but with the schools we often survey the entire population (i.e. all/nearly all the 50 to 100 teachers, all/nearly all the 200 to 1000 students). How does one test for differences in this case?
There are two schools of thought on this. The first school argues that confidence intervals and p-values are irrelevant. You have the entire population, so you have zero sampling error.
The second school argues that you often want to extrapolate to a conceptual larger population, such as all teachers in the present and future years.
Except in some very narrowly drawn circumstances, I tend to follow the second school. For such a conceptual population, confidence intervals and p-values are still meaningful.
An earlier weblog entry
Enumerative and Analytic Studies
discusses this concept in further detail.
You can find an earlier version of this page on my original website.