Male heights have a normal distribution centered at 5' 10" with a standard deviation of 3 inches.
The probability of an 8 foot tall man is on the order of 1 in 1,000,000,000,000,000,000.
And yet there are men taller than 8'. The record is 8' 11".
Information is quantified in terms of surprise. If everything in a message is expected, then it contains no information, no matter how long it is.
#informationtheory
Bayes theorem says the order in which you learn about two things doesn't matter.
P(A) P(B | A) = P(B) P(A | B)
You could first learn about A, and then learn about B given A, or you could first learn about B, then learn about A given B.
It’s not very useful to say heights are not normally distributed. Nothing is exactly normally distributed.
It’s more useful to say how well the normal model fits, by what criteria, and under what conditions.