It is definitional, which I mean in the strictest mathematical sense: the information content of a result is directly derived from how “unexpected” it is.

A result which conveys 2 bits of information should occur with 25% expected probability. Because that’s what we mean by “two bits of information.”