I got an interesting question about an application in information theory of a statistical distribution called the Type I extreme value distribution. This distribution, also known as the Gumbel distribution, is useful for modeling the maximum or minimum of a large number of variables.
Reliability statistics often consider the maximum or minimum value as an important quantity to measure. For example, if you have a machine with a large number of components, and the machine fails when any one of the components fails, then the lifetime of the machine is the minimum of the individual component failure times. Insurance companies are also interested in extreme values as they represent the worst case scenarios. A 100 year flood level also represents an extreme value distribution.
There are two other distributions used to model extreme values (not surprisingly they are called the Type II and Type III extreme value distributions). Which distribution you use depends on factors like whether the individual component distribution has finite moments and/or a bounded tail. I’m not an expert on extreme value problems or reliability statistics, so I looked up a few good resources.
Extreme Value Distributions. Annis C. Accessed on 2006-01-09.
[Excerpt] The average of n samples taken from any distribution with finite mean and variance will have a normal distribution for large n. This is the CLT. The largest member of a sample of size n has a LEV, Type I largest extreme value, also called Gumbel, distribution, regardless of the parent population, IF the parent has an unbounded tail that decreases at least as fast as an exponential function, and has finite moments (as does the normal, for example). www.statisticalengineering.com/extreme_value.htm
Extreme value distributions. Tobias P, NIST/SEMATECH e-Handbook of Statistical Methods. Accessed on 2006-01-09.
[Excerpt] We have already referred to Extreme Value Distributions when describing the uses of the Weibull distribution. Extreme value distributions are the limiting distributions for the minimum or the maximum of a very large collection of random observations from the same arbitrary distribution. Gumbel (1958) showed that for any well-behaved initial distribution (i.e., F(x) is continuous and has an inverse), only a few models are needed, depending on whether you are interested in the maximum or the minimum, and also if the observations are bounded above or below. www.itl.nist.gov/div898/handbook/apr/section1/apr163.htm
Extreme Value Type I Distribution. Filliben JJ, Heckert A, NIST/SEMATECH e-Handbook of Statistical Methods. Accessed on 2006-01-09.
[Excerpt] The extreme value type I distribution has two forms. One is based on the smallest extreme and the other is based on the largest extreme. We call these the minimum and maximum cases, respectively. Formulas and plots for both cases are given. The extreme value type I distribution is also referred to as the Gumbel distribution. www.itl.nist.gov/div898/handbook/eda/section3/eda366g.htm
Extreme value theory. Wikipedia. Accessed on 2006-01-09.
[Excerpt] Extreme value theory is a branch of statistics dealing with the extreme deviations from the median of probability distributions. The general theory sets out to assess the type of probability distributions generated by processes. Extreme value theory is important for assessing risk for highly unusual events, such as 100-year floods. en.wikipedia.org/wiki/Extreme_value_theory
A rather unusual approach to extreme values has been advocated by Benoit Mandelbrot, probably the most famous name in the area of fractal geometry.
Q&A with Benoit Mandelbrot. Wright CM, National Association of Real Estate Investment Trusts. Accessed on 2006-01-09.
[Excerpt] Forget Euclidean geometry with its smooth lines and planes. Now comes Benoit Mandelbrot, the inventor of fractal geometry, who recently wrote an entertaining and challenging book, “The (mis)Behavior of Markets,” in which he argues that his study of roughness, already applied to topography, meteorology, the compression of computer files, and many other fields, will rewrite the canon on finance. Portfolio asked the Yale University mathematics professor, among other things, how real estate prices look under the fractal microscope. www.nareit.com/portfoliomag/05mayjun/capital.shtml
Mandelbrot’s Extremism [PDF]. Beirlant J, Schoutens W, Segers J, Published December 6, 2004. Accessed on 2006-01-09.
[Abstract] In the sixties Mandelbrot already showed that extreme price swings are more likely than some of us think or incorporate in our models. A modern toolbox for analyzing such rare events can be found in the field of extreme value theory. At the core of extreme value theory lies the modelling of maxima over large blocks of observations and of excesses over high thresholds. The general validity of these models makes them suitable for out-of-sample extrapolation. By way of illustration we assess the likeliness of the crash of the Dow Jones on October 19, 1987, a loss that was more than twice as large as on any other single day from 1954 until 2004. www.kuleuven.ac.be/ucs/research/reports/2004/report2004_08.pdf
The work by Mandelbrot may not be useful in the context of the original question, but it is still a fascinating and very active area of research.