A principle called the second law of thermodynamics derived from the works of people such as Carnot,
Clausius, Maxwell, Boltzman and Szilard, is a fundamental concept in science and physics.
It describes how energy and everything related to energy such as heat and work evolve over
time and interact with the environment, how heat and energy is transferred from one side
to another and how it behaves in closed and open environments.
To help us understand this principle and its connections to the concept of information,
let's perform what is called a thought experiment, that is, an experiment that one can run in
one's own mind to reach some sort of logical conclusion.
In this case is going to rather be illustrated by an actual computer simulation.
So let's imagine for an instant that we have a room with gas.
Gas is made of particles so particles are free in the room only constrained by the room
walls.
Particles are moving around bouncing against each other.
Here is an illustration of 100 particles shown in black inside a square of size 100 x 100
pixels.
For illustration purposes, the room can be divided in 100 x 100 cells each of the same
size, making 10 thousand cells that can contain one of the 100 particles.
To describe any particular state of the room at any given time we would have to provide
the bit value of those 10 thousand particles.
Alternatively, we can simply give the 100 coordinates where particles are located.
If the room were empty, however, it would just suffice to say that the room is empty
and nothing else, that would characterize the state of the room in a very simple description,
but if the particles are distributed randomly then one would need to provide the exact coordinate
for each particle and no simpler description would be possible in order to reproduce the
same state of the room.
If you wanted to describe this room you would need to provide the exact location of each
particle.
But there was an idea in quantum physics to partially circumvent the need to study such
detailed descriptions that are impossible to get in practice, and even in principle,
according to some aspects of quantum mechanics when we are considering very small particles.
So, statistical mechanics makes qualitative assessments instead of providing the exact
locations.
It would characterize the room statistically, in this case, for example, as typically 'disordered'.
What the second law of thermodynamics would then tell is that most configurations will
look disordered and those ordered will be unlikely to appear by chance and, if they
did, they would be unstable and quickly evolve into disordered configurations.
The room would then be said to be in a high Entropy configuration.
Such qualitative description is statistical in nature.
The idea is that if you plot the distribution of the particles across all the possible places
in which they could be, the distribution would look the same for most random configurations
and thus anything with such distribution would be characterized as being in the same state
of disorder.
So this is the way in which the distribution of the particles in our room looks like when
dividing the room into smaller square pieces of equal size, so when looking at the room
in 100 places no particular place seems to have any particular feature meaning that particles
look mostly distributed equally, the same when dividing the room into 20, 10 and only
5 places to look at as if we were looking at the corners and centre to see if there
is some arrangement of the particles to notice, but basically everything looks the same in
any direction.
Imagine that you were able to place the same number of particles into a smaller room:
What would happen is that the same number of particles have now fewer places to be,
what is called less degrees of freedom or less cells in the square to be or move into.
So because there is less room for the particles in the new room it can be said that the room
has a lower Entropy compared to the larger one for the same number of particles but different
volume or area.
If one reduced the number of particles it would also lead to less Entropy because you
would need to describe less particles, so Entropy is a function of at least 3 elements:
the number of particles, the size of the space and the number of places in which the particles
can be.
Imagine you pushed all particles into a smaller and smaller place in the corner of the room:
Eventually the particles would have no place to move and particles would accumulate next
to each other in an increasingly simpler configuration.
Then the room would be said to be in a low Entropy state because the configuration is
not typical, there are only a few cases among all possible in which the statistical distribution
looks like the one distributing particles in the corner of the room.
This is how the distribution plots similar to the ones we made for the random configuration
would look like in this case: Only moving one particle outside that corner
would make Entropy to increase.
So what does it tell us the second law of thermodynamics?
It tell us that if we looked at a room with some gas particles, the chances to find the
room in a disarranged or disordered state, that is, of high Entropy, has much greater
chances because there are many of those configurations that distribute particles in a random-looking
distribution compared to a particular arrangement of low Entropy like the one with all the particles
in the corner.
In other words, there are many more possible disordered arrangements than ordered ones,
and low Entropy configurations will tend to produce higher Entropy configurations if left
unattended and no work is performed to place them in a single non random-looking configuration.
So it is the case that to reach a special configuration one has to invest work or apply
energy, to push all the particles to the corner.
Moreover, if the particles were moving at constant speed, by pushing them into a corner
they would bounce with each other with higher frequency producing heat that, when released,
would naturally cool down hence establishing a strong connection with heat and temperature.
This is why this law or guiding principle is so important, because it connects most
if not all the fundamental quantities in physics.
But it is also all about information as we saw before, as it involves knowledge about
the position of the particles and ways to steer them to one side or the other.
So we have connected all these measures with information, heat, disorder, Entropy and temperature.
So the second law of thermodynamics tells us that, in general, because particles will
tend to maximal Entropy, they will also dissipate heat over time instead of gaining it, and
they will eventually reach thermodynamic equilibrium by matching the environment's temperature,
just as it happens with hot drinks eventually reaching room temperature.
This also means that if you disconnect your fridge, it will reach room temperature soon
because of the second law of thermodynamics even though there are chances that it can
remain cold or even colder, but those chances are so incredibly small that it will almost
never happen.
It is right then to think of the second law of thermodynamics as a law based on probability
and deeply connected to information about the specific and overall configuration of
a system.
The traditional way in which the second law of thermodynamics, for a closed system like
our room, is formally written like this.
The increase of Entropy represented by S over time t will remain or increase but not decrease.