I'm a bit confused. What is this notion of "pure randomness" that this article and many comments seem to be eluding to? Perhaps I'm just not using the same definitions of terms, but I thought a uniform distribution is still considered random.
> What is this notion of "pure randomness" that this article and many comments seem to be eluding [sic] to?
First, s/eluding/alluding/
Second, although the topic is complex, one test of randomness is that an ideal compression method, one able to find and exploit any repetitive pattern, cannot compress a random sequence.
Third, the term "entropy" as used in information theory is tied to randomness, as explained here:
A quote: "The entropy rate for a [fair coin] toss is one bit per toss. However, if the coin is not fair, then the uncertainty, and hence the entropy rate, is lower."
I think you're fudging what is supposed to be uniform here. In your example, the unigrams (i.e. single digits) may be uniform, but the n-grams for n > 2 are not.
The glow worms are still distributed randomly in the mathematical sense, but knowing the position of one tells you something about the likely positions of others, so they are not independent.
Common parlance doesn't do a great job at talking about the features of random distributions, but when people say "purely random," they often seem to mean "uniformly distributed and independent." Both pictures have uniformly distributed points, but the glow worms are not independent.
The distributions are describing different aspects. Let's take the WoW loot example. A monster could drop 50 different objects with equal probability. Kill it a few million times you'll get a uniform distribution of kills against item type.
Now if we take that same kill data and instead ask how many times do I get item type 2 in 100 kills you'll see a Poisson distribution. If you didn't see a Poisson distribution when asking that question then the events were probably not independent of each other, and hence weren't actually random (e.g. The monster could just drop the 50 items types in order).
Comments
I'm a bit confused. What is this notion of "pure randomness" that this article and many comments seem to be eluding to? Perhaps I'm just not using the same definitions of terms, but I thought a uniform distribution is still considered random.
> Perhaps I'm just not using the same definitions of terms, but I thought a uniform distribution is still considered random.
No, a uniform distribution is not evidence of randomness. Consider the digits 0 - 9 repeated endlessly:
0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789 ...
Uniformly distributed? Yes. Random? No.
> What is this notion of "pure randomness" that this article and many comments seem to be eluding [sic] to?
First, s/eluding/alluding/
Second, although the topic is complex, one test of randomness is that an ideal compression method, one able to find and exploit any repetitive pattern, cannot compress a random sequence.
Third, the term "entropy" as used in information theory is tied to randomness, as explained here:
http://en.wikipedia.org/wiki/Entropy_(information_theory)
A quote: "The entropy rate for a [fair coin] toss is one bit per toss. However, if the coin is not fair, then the uncertainty, and hence the entropy rate, is lower."
Based on that, high entropy -> high randomness.
Not to oversimplify a complex topic.
>Uniformly distributed? Yes. Random? No.
I think you're fudging what is supposed to be uniform here. In your example, the unigrams (i.e. single digits) may be uniform, but the n-grams for n > 2 are not.
Actually, the n-grams are periodic and repetitive also. They certainly aren't random.
They may be periodic and repetitive, but they aren't uniform. For example, there are lots of "01"s and no "02"s.
Could you edit that long line somehow? It makes reading harder in my phone.
The concept you're looking for is "independence"
http://en.wikipedia.org/wiki/Independence_(probability_theor...
The glow worms are still distributed randomly in the mathematical sense, but knowing the position of one tells you something about the likely positions of others, so they are not independent.
Common parlance doesn't do a great job at talking about the features of random distributions, but when people say "purely random," they often seem to mean "uniformly distributed and independent." Both pictures have uniformly distributed points, but the glow worms are not independent.
The distributions are describing different aspects. Let's take the WoW loot example. A monster could drop 50 different objects with equal probability. Kill it a few million times you'll get a uniform distribution of kills against item type.
Now if we take that same kill data and instead ask how many times do I get item type 2 in 100 kills you'll see a Poisson distribution. If you didn't see a Poisson distribution when asking that question then the events were probably not independent of each other, and hence weren't actually random (e.g. The monster could just drop the 50 items types in order).