Skip to content

Comment on The Normal Distribution: A derivation from basic principles [pdf]parent

Comments

While this is a nice little derivation, the reason the normal distribution is important (and common) is the Central limit theorem[1].

This loosely means that if you have n samples that are independently drawn from the same distribution, their sum approaches a normal distribution for large n. The generality comes from the fact that this happens (almost) independently of distribution the samples come from.

[1]https://en.wikipedia.org/wiki/Central_limit_theorem

This is often given as the reason in the textbooks, but it actually is a sloppy argument, for there is no reason to presuppose sums of independent variables (they do not have to be drawn from the same distribution in some variants of the CLT) are so ubiquitous in nature. A more convincing reason is given by:

http://en.wikipedia.org/wiki/Maximum_entropy_probability_dis...

Well, I think textbooks often give the maxent justification as well, but probably neither is strictly superior to the other.

Often, where a measurement is perturbed by a litany of physically unlinked disturbances (e.g., antenna noise), the CLT argument is compelling. In a case like antenna or front-end electronics noise, justifying the maxent argument would require buying in to an almost mystical belief by comparison.

Interestingly, the two seem to have a deep connection related to thermodynamics, e.g. http://en.wikipedia.org/wiki/Thermal_fluctuations, where you can see both as expansions of a configuration count up to second order in the exponent.

This is manifest in the detectable remnants of the Big Bang, which are a particular realization of a Gaussian random field that was present due to quantum fluctuations of a primordial system. It's funny to think of the universe as a draw from a random number generator.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.