Let's now look at two prototypical examples of applications of Markov Chains
What I've drawn here is something called A Directed Graph
In a directed graph we have nodes. In this case there are 3 nodes/possible outcomes
What we're describing with this graph are transitional probabilities between various
outcomes for an economy
You'll notice, in a directed graph, there are arrows pointing from one state
or one outcome to another
For instance we have the outcome of our economy in experiencing normal growth,
mild recession or severe resession
In between each of these outcomes we have an arrow again directed
towards another outcome
Attached to each of those arrows is a weight or a probability measure
For example if I wanted to determine, based on this analysis, the probability
that if our economy is in a normal growth state it will maintain normal growth
That probability is quiet high, 97,1%
On the other hand the probability that we transition from a normal growth
at any given year to a mild recession is 0,029% and so forth
There are aren't arrows between each of the nodes,
in witch case there would be probability zero
Because a mild recession represents something of an intermediate outcome
there is no direct arrow between normal growth and severe recession and vice versa
That probability is zero - first we have to devolve into mild recession
before we hit a severe recession
These numbers are actually based on recent statistics related to the U.S. economy
They weren't totally plucked out of thin air in other words
Based on this directed graph, I would like to define my stochastic matrix: "P"
for this Markov Chain
Then, given some initial state of our economy, we can then make a prediction
We can in other words forecast bouts of our economic future
Here we have the associated stochastic matrix
we call it P
That once again reflects the transitional probabilities as shown
in our directed graph above
For the numbers in column 1 we can think of interpreting them
as transitional probabilities for our economy if we're currently
in a natural growth state
The probability in other words, when we're in natural growth
if you look at the directed graph to maintain a natural growth is 0,971
and we see that number here
On the other hand, to transition probability from natural growth
to mild recession by natural growth in this column
Mild recession in the second row that probability is 0,29 is kind of
moving around the graph
Let's say if we're in severe recession what's the transitional probability
to maintain severe recession? That probability is 0,492
That corresponds with the number in the lower right
If we're in this current state what's the probability we then transition?
In other words, if we stay in severe recession we have 0,492
As I mentioned before, there are a couple of zero probabilities
to go from one extreme case to another.
So if we're in severe recession the probability of then go
into a normal growth state is zero. There is no arrow from that node
of severe recession to normal growth.
Now that we have our stochastic matrix for this particular economic model,
let's go ahead and use it in the context of a Markov Chain to then
make some predictions
Let's go ahead and calculate a Markov Chain for this example of
economic forecasting, again recalling that here is our stochastic matrix = P
representing the various transitional probabilities
In the first computation here I'm just going to be very optimistic in terms
of where we start. So our X- knot, our initial state-vector, I'll make;
1, 0, 0, where 1 = 100%, we're "all in" for normal growth,
and 0, 0, is for the mild, and severe recession types
We start of with a best case scenario, what does this Markov Chain predict
for future economic outcomes?
Remember then; By the "Markov condition", as we called it, each subsequent state
in our chain depends only on the previous state
The way we transition from state to state is that we multiply by the matrix P
Let's go ahead and do that computation and find a prediction
for a probability vector for year 1
Then we'll multiply our initial state vector 1, 0, 0 by these
stochastic matrix P on the left That will then give us our state vector
for the following year let's call it year one
Notice though this calculation is quiet trivial, I'm essentially doing a
dot product when I multiply matrices together.
I'm dotting each row with 1, 0, 0 So really I pluck out and retain
the first column from my stochastic matrix How do we interpret that state vector?
Well, after our first year the outlook is as follows;
We have a 97% chance of being in once again normal growth mode
and a 0,29% chance of being in a mild recession
and a zero % chance for severe recession
That's well and good, but I'd really like, at this point, to pear deeper into my
crystal ball. In other words I'd like to make a projection about a future outcome
Let's say ten to twenty years down the road using this particular model
The way I do that with the Markov Chain is I iterate the process of
matrix multiplication
To get, let's say, state vector two after year two, I multiply by P
But you'll notice with each subsequent computation to get the next state vector
in my chain or sequence, I just multiply by P
So a nice shortcut is this; if I want to know for instance,
The K-P state vector, that is my economic outlook in this case, after "k" years
All I do is exponentiate, in other words take a power of my stochastic matrix,
the k-power in particular, multiply it by the initial state vector.
Then I get my prediction for the forecast for year k
Once again we see a really nice application here of
taking powers of a matrix
Now we can efficiently make predictions about future outcomes
using this particular mathematical model
For instance what is the predicted status ten years down the road?
What is our economic health and wellbeing going to look like?
Take my stochastic matrix, raise it simply to the tenth power,
Then I multiply it by the initial state vector 1, 0, 0 again
That is the predicted outcome So at that point we've got 85%
chance of transitioning to a normal growth
Notice; creeping up here are the incresed probabilities of
the recessions respectively
And then, one more time, if I wanted then predict further on down the road
20 years, let's say I raise my stochastic to the 20th power,
Multiplying again by the initial state vector
and I get these following probabilistic outcomes
So if you're curious you might wonder; what if we had started with a different
initial state vector, a different X-knot in other words
This was a very optimistic outlook indeed Can we be a little more cynical and see
what then the model sort of conjures up for us
Well sure, for instance if I assume, and maybe it's a fair assumption honestly
in this day and age, if I assume that maybe I have a uniform probability vector
to begin with, in other words, all outcomes are equally likely
as my initial state vector, so if times are not so good economically
we say 1/3, 1/3, 1/3 for those components Then what happens when I turn the crank
with the Markov Chain and forecast 20 years down the road,
in other words I raise my stochastic matrix to the 20th power,
multiplying by this different initial state vector
And I get the following outcome, which isn't drastically different
but you'll notice a little bit more dire indeed
Ok, so let's just summarize our findings between those two cases
In one case we had this initial state vector that was representing
our cheerily optimistic economic outlook.
We used our Markov Chain to make a prediction 20 years down the road
What is our state of affairs then?
On the other hand, in the second case we started with a vastly different
initial state vector, a uniform probability vector
once again we used our same Markov Chain to make a prediction about the outlook
once again 20 years down the road, now again, even though our initial vectors
were quite different, the final state vectors, predicted by the models
in both cases were not entirely dissimilar
A good question to ask at this juncture is whether or not my Markov Chain
depends heavily on my initial state vector?
To what extent does tweaking that vector change the outcome in the long run
of our Markov chain? We're going to answer that question momentarily
And a closely related conceptual question which is of ultimate importance
for Markov Chains is to ask the following; If I project way into the future into my
sequence of state vectors defined by this Markov Chain, does that sequence
ever settle down? In other words, to put that in mathematical terms is;
Do my vectors converge to a particular value?
If they do by the way, that vector is called a steady state vector for
the Markov Chain. That means that my dynamic process has converged
to a particular value. In other words; It's settled down!