I haven't yet told you how to formally calculate
Shannon information content; I've only
told you intuitively that it's the amount
of surprise that a receiver has when
receiving a message from a message
source. So intuitively, we can say that
the message source, the one-year-old who always
says "Da", gives you absolutely no surprise, wherereas the second
one-year-old who says "Dada Mama Baba" gives you
some surprise; you don't know which of those three choices
the one-year-old going to say next. So, this one has higher information content
And similarly, the message source that reports the fair coin
has only two messages, heads or tails, whereas the message source
that reports the fair die has six possible messages, so it's less predictable and has more
information content.