Hello, everybody.
In this unit I'll cover the notion of information
and its relations to characterizing order and disorder in complex systems.
We'll start by discussing the notion of entropy,
which is a particular way of characterizing disorder.
Then we'll try to make sense of the second law of thermodynamics,
and we'll look at the famous paradox called Maxwell's Demon,
which links physics to information.
Then we'll touch on the field of statistical mechanics,
which will give us the foundation for quantifying the notion of information.
We'll talk about Claude Shannon's formulation of information
and discuss its limits in quantifying information processing in complex systems.
Finally, we'll hear from Jim Crutchfield,
a physicist at University of California and the Santa Fe Institute
about his work in a prime notion from information theory to understanding complex systems.
Let's get started.
In Unit 1 we saw several examples of complex systems in which self organization appears.
For example, we saw how ants are able to collectively produce,
organize structures such as bridges made up of their own bodies,
how termites organize themselves to build intricate nest structures,
how neurons become organized into functional units of cognition,
how the immune system's trillions of cells organized to defend the body against intruders,
and we saw how individuals organize themselves
into ecologies,
social networks,
cities,
and complex economies.
In Units 2 and 3 we briefly cover the topic of
dynamics and fractals
and how the iterations of simple rules can lead to both simple and complex behaviors,
how it can lead to organization and chaos.
But another key to the phenomenon of self organization is the concept of information.
In order to understand the phenomenon of self organization
we need to understand how information is represented,
communicated and processed in complex systems.
As said by the physicist Murray Gell-Mann,
"Although complex systems differ widely in their physical attributes,
they resemble one another in the way that they handle information.
That common feature is perhaps the best starting point for exploring how they operate."
In this unit, we'll look at some ways in which information can be quantified.
Historically the mathematical structure of information starts with
the laws of thermodynamics. The first law simply says that
in an isolated system, energy is conserved.
An isolated system is one in which
no energy can be added from outside the system
or can escape outside the system.
So what do we mean by energy exactly?
Energy is defined in physics as a system's potential to do work.
Work has a technical meaning here.
But you can think of it in a colloquial way
that is, work is getting stuff done.
So very informally,
energy is the potential of the system to get things done.
Energy can take on different forms,
and it can be transformed from one form into another.
Here are a few examples.
Energy from sunlight can be transformed into chemical energy in plants via photosynthesis.
Electrical energy can be transformed into thermal energy.
That's how your electric heaters or stoves work.
Chemical energy from food or fuel can be transformed into mechanical or kinetic energy
that we use to move via our muscles or vehicles.
Now according to the first law, in an isolated system, well,
energy can be transformed among different kinds.
the total amount of energy in the system always remains the same.
That was meant by "energy is conserved."
Now, the second law of thermodynamics says that
in an isolated system, entropy always increases until it reaches a maximum value.
So what do we mean by entropy?
Well, whenever energy is transformed from one source to another,
there is nearly always a loss of energy that can't be used for work.
This is sometimes called heat loss.
The technical term for this is entropy.
For example, suppose you're pushing a bicycle up the steep hill.
Your stored energy from calories is being transformed into mechanical
or kinetic energy that is the motion of the bike and your body.
But there is a price to pay.
This transformation also results in heat loss from your body.
That is energy that can't be used for work.
In other words, energy that doesn't contribute to moving the bike and you up the hill.
This is entropy produced by the transfer of energy.
Entropy can be thought of as a measure of disorder in a system.
The second law of thermodynamics then says that
disorder in a system always increases until it reaches its maximum value.
Let's illustrate this by a simple example.
I'll use one of the NetLogo library models.
It's called GasLab Two Gas.
So, let's go to Chemistry and Physics,
and GasLab,
and this one called GasLab Two Gas.
OK, so I open it up.
We look at setup.
What this does is going to illustrate the behavior of two rooms with gases in them.
And gas is just a collection of moving particles.
So each one of these circles is a particle like a molecule.
And this is going to simulate the physics of a gas.
So I'm going to show you this model
with the number of molecules on the right hand side
being equal to the number of molecules on the left hand side.
Cyan is the blue color here. Magenta is the purple color here.
The initial speed will be 30 for the right and 10 for the left.
I'll have them each have mass 8.
OK, this will make it a lot simpler.
OK, we'll do setup.
And now what happens is these molecules start moving.
I've set the initial speed of these blue ones to be faster than
the initial speed of these red ones or purple ones.
If we look at their speeds you can see that
the blue ones have a higher speed, the red ones have a lower speed, and so on.
So this is an ordered system
because we have all the reddish particles over here and all the bluish particles over here.
And they are not mixing.
But what if we allow them to mix by clicking on Open up here.
Let's see what happens.
And I'll speed it up for a little bit.
When they collide, they can lose energy and thus slow down.
And you can see as we allow these molecules to just
move according to the law of physics in this frictionless gas,
very quickly they start mixing the red and the blue particles to average speeds,
their average energies become equal.
And the whole system becomes very disordered.
So we start off, if I stop this.
We start off from a very highly ordered, low entropy system,
and if we allow them to mix,
the system very quickly becomes a disordered, high entropy system.
Now these things are just moving around kind of bouncing off each other,
and conceivably by some strange chance,
all the blue ones could come back over here onto the right side,
all the red ones could come back over here onto the left side.
But that's just incredibly unlikely.
And that's the idea of the second law of thermodynamics, which just says
in a system like this the system will become more disordered,
it will not become more ordered
unless someone puts some work into making it more ordered.
The second law of thermodynamics has some profound implications.
First, systems are naturally disordered.
That is, they can't become organized without the input of some work.
This implies that perpetual motion machines are not possible.
A perpetual motion machine is one that might create and feed back energy to itself
and therefore always be in motion without the input of additional energy from the outside.
That would be a machine in which there is absolutely no heat loss.
And the second law of thermodynamics implies this is impossible.
And third, time has a direction.
That is the direction of increasing entropy.
We can see this in a striking example
where we look at the movie of a glass breaking played backwards.
Let's play that again.
This is something you can look at and realize very quickly
that is a movie played in reverse.
And the reason that you know that is because you know that by the laws of physics
things like broken glasses don't just by themselves magically get repaired and put back together.
You have to put intense work into the broken glass to repair it
even though it's very easy to break a glass.
So we have a sense of which direction time flows in by looking at these kinds of movies.
This is a fundamental result of the second law of thermodynamics,
which in some sense gives meaning to the notion of time.