1
00:00:00,501 --> 00:00:04,923
I'm going to show you three models that we've developed to illustrate the
2
00:00:04,923 --> 00:00:08,644
ideas that we've been learning about Shannon information content.
3
00:00:08,644 --> 00:00:16,020
The first is our Shannon Information Content of Coin Flips. So, to set up, we
4
00:00:16,020 --> 00:00:22,961
see our coin here, and we can either flip a fair coin or flip a biased coin, with
5
00:00:22,961 --> 00:00:28,438
some probability of heads. So I flip a fair coin, the coin flips, it gives me
6
00:00:28,438 --> 00:00:32,526
heads, and it keeps track of the number of heads and the number of tails.
7
00:00:32,526 --> 00:00:40,915
I can flip it any number of times, and it's just flipping at random, and after
8
00:00:40,915 --> 00:00:46,337
I get some collection of these things, I can then calculate the information
9
00:00:46,337 --> 00:00:52,538
content and it shows me what information content it's gotten so far.
10
00:00:52,538 --> 00:00:59,701
So, even though this is a fair coin, we see that we've still gotten five tails
11
00:00:59,701 --> 00:01:04,322
and only two heads, because we've only done seven flip, so we haven't
12
00:01:04,322 --> 00:01:08,157
yet gotten enough statistics to really see that heads and tails each have
13
00:01:08,157 --> 00:01:12,415
50/50 chance of coming up. We can also set our biased coin to any
14
00:01:12,415 --> 00:01:20,861
probability we want of heads, and then flip our biased coin and see how
15
00:01:20,861 --> 00:01:23,906
that affects our information content.
16
00:01:23,906 --> 00:01:28,468
The second information content model is one in which you can measure
17
00:01:28,468 --> 00:01:34,158
the information content of a text, just like we showed briefly in the previous
18
00:01:34,158 --> 00:01:43,252
video. So, I have copied a text from an online site, which gives the
19
00:01:43,252 --> 00:01:51,172
entire "To be or not to be" speach from Hamlet, and if I click on "go", this
20
00:01:51,172 --> 00:01:55,323
shows me how many words there were and the frequencies of the different
21
00:01:55,323 --> 00:01:59,277
words, so you can see these various frequencies, it only shows me the
22
00:01:59,277 --> 00:02:04,866
top, small number of frequencies, but here it's showing me the frequency
23
00:02:04,866 --> 00:02:09,951
distribution of these words, and the information content. So you can play
24
00:02:09,951 --> 00:02:16,317
with this to see what is the measured information content of various texts
25
00:02:16,317 --> 00:02:19,828
that you can paste in here, and we'll have some exercises on this in
26
00:02:19,828 --> 00:02:21,662
the homework.
27
00:02:21,662 --> 00:02:26,933
Our final model looks at the information content in the symbolic dynamics
28
00:02:26,933 --> 00:02:30,935
of the logistic map. Let me show you what I mean by that.
29
00:02:30,935 --> 00:02:36,263
So you might remember our logistic map, from our earlier unit on dynamics
30
00:02:36,263 --> 00:02:45,301
and chaos, and I can do my---set my R to 3.51 and my x_0 of point 2,
31
00:02:45,301 --> 00:02:52,533
and we get a periodic attractor, and what this is doing is it's calculating
32
00:02:52,533 --> 00:02:56,929
symbolic dynamics, which means that I can set threshold---I've set a
33
00:02:56,929 --> 00:03:03,609
threshold of .5 here---every time this dot goes above .5 on the y-axis, the
34
00:03:03,609 --> 00:03:09,860
system outputs a 1, every time it goes below .5 the system outputs a 0.
35
00:03:09,860 --> 00:03:15,349
Now, I can look at the information content in that set of messages that
36
00:03:15,349 --> 00:03:19,414
consist of 1's and 0's, and you can think of the message source as the
37
00:03:19,414 --> 00:03:24,575
logistic map at a given value of R. And now, this shows us the information
38
00:03:24,575 --> 00:03:29,105
content of that message source, given these symbolic dynamics.
39
00:03:29,105 --> 00:03:33,667
So this is another model in which you'll get to do some experiments on
40
00:03:33,667 --> 00:03:39,354
in the homework. So the homework is optional, but I really urge you to do
41
00:03:39,354 --> 00:03:43,264
at least the beginner level part, because that will give you the opportunity
42
00:03:43,264 --> 00:03:46,865
to do some experiments with these various models, which I think will give
43
00:03:46,865 --> 00:03:50,027
you a much better handle on the ideas I've been talking about, relating
44
00:03:50,027 --> 00:03:54,866
to information and Shannon information content.