This segment is about noise -
something that you can never really avoid
in real-world situations -
kind of like the HVAC system
in my office,
which is producing a high-frequency hiss,
which gets picked up
by the nice microphone,
but not so much by the headset.
The segment is also about filtering -
the act of removing - noise from data.
Now, in order to filter noise
out of data,
you need to have a way to discriminate
between signal and noise.
In most traditional applications,
that discriminating factor is where
the signal is in the frequency spectrum.
Most commonly, we assume
that high frequency noise -
like my HVAC system - is bad,
and low frequency noise -
like my voice, which is much lower
frequency than that - is good.
If that assumption is true,
you can use something called
"a low-pass filter"
to get rid of the noise.
A low-pass filter
is a circuit or an algorithm
that lets through
the low frequency stuff
and doesn't let through
the high frequency stuff.
But, if the signal and the noise
are all mixed together,
all the way across
the frequency spectrum,
there's no way you can draw
a discriminating line
anywhere in that spectrum and say
that the stuff over here is good,
and the stuff over here is bad.
And then, you can't use
traditional filtering technology.
If you did, you would be
"throwing the baby out
with the bathwater,"
to use an old expression -
throwing away meaningful components,
the signal along with the noise.
And that's a bad idea.
Here's a paper that makes that point
quite emphatically.
Okay - so how to reduce noise if you
can't use the traditional weaponry.
There are a number of ways.
I will talk about two:
one that takes advantage of the geometry
of stable and unstable manifolds,
and another that takes advantage
of topological properties.
A noisy measurement is like a point in
state space with a noise ball around it.
If there were no noise,
you'd measure the point right in
the middle where the blue point is.
But, because of the noise you might see
that point any place in this ball.
Now, think about what the stable
and unstable manifolds
will do to a ball like this.
As time evolves, that ball will stretch
along the unstable manifold,
and compress along the stable manifold.
You saw this on a unit test.
That can be turned to advantage.
Here's the idea.
You have three successive
noisy measurements
of a trajectory on a chaotic attractor,
at three successive points in time.
The central points are
the true state of the system.
If there were no noise,
you'd measure the point right there.
But, because of the noise,
you might see the point
any place in the noise balls.
Since this is a chaotic attractor,
it has at least one unstable manifold.
And, since it's a chaotic attractor,
it has at least one stable manifold.
And, if you have a way
to move time forwards and backwards -
that is, to make forecasts
and hindcasts -
you can leverage the effects of these
manifolds to reduce the noise.
If you evolve the green measurement -
the left one -
forwards to the point in time
where the blue one was performed,
it will effectively stretch out
the green noise ball
along the unstable manifold and
compress it along the stable manifold.
So, the green noise ball,
if we evolve it forward in time
from here to here, will look like this.
Same idea with projecting
the black noise ball,
at the third point backwards in time,
to the same time point
as the blue ball was measured.
It will look like this -
because in backwards time, it will
stretch out along the stable manifold
and compress
along the unstable manifold.
If you did the evolution right,
all three of those should really be
at the same point in state space,
so the true value will be somewhere
in this overlap region.
And, notice that that overlap region
is a lot smaller than
the original noise balls.
This scheme is due to
Farmer and Sidorowich.
Here's the version of that drawing
from their paper.
The operation that's schematized
by these arrows
is effected by some sort of model
of the system dynamics -
for example, fourth-order Runge-Kutta
on the ODE, if you have it -
or some sort of prediction model,
which we'll get to in the last unit.
Here's a summary of the algorithm -
a more compact version of
the explanation I just gave you -
so that you can refer back to it
in the course slides.
This last bullet here goes
back to the picture.
This works if you have stable
and unstable manifolds that cross -
that is, they're transverse.
It works best if they're at 90 degrees,
just because that will squash
the noise ball the most.
Here are some results from their paper.
This is a standard numerical experiment.
They took a trajectory from a
known system - HÃ©non in this case -
they added some noise,
and they saw if their scheme
could get rid of that noise.
And it did.
What this figure is actually plotting
is the amount of noise - here -
at each point along
the length of the trajectory,
before and after their
filtering operation was performed.
And, you may not be able to read this.
This is one part in a thousand
of noise that they added.
This is one part
in ten to the thirteenth noise
that was left
after their filtering scheme.
So they reduce the noise
by 10 orders of magnitude,
which is pretty good.
Again, what's going on here
is a use of the geometry of the dynamics
to reduce noise.
There are other properties
of chaotic attractors
that we can also use to reduce noise,
like the topology.
I'm not going to talk about this
in any depth,
but the basic idea here is
that chaotic attractors are connected,
and connected sets don't have
any isolated points -
which are kind of what they sound like.
Therefore, if you find
isolated points in your trajectory,
using what are called
"computational topology techniques,"
those are probably noise effects -
and can safely be thrown away.
There's a reference here
at the bottom of the slide.
If you're interested in it,
you can get that paper off my web page.
There are other topological properties
besides connectedness
that you can leverage to get rid
of noise, including continuity.
This is related to that
there's only one downhill direction
from any point idea.
If two points that are nearby
in state space
end up really far apart
at the next time click,
then there may have been
different dynamics
operating on that region of state space
here at different times.
And that can be used to detect
regime shifts - like bifurcations -
and to separate signal from noise.
There's a citation down here,
if you're interested.