We've talked about a lot of theory and algorithms
over the past 7 units in this course. In this
unit, and the one that follows, we're going to
get our hands dirty with data from dynamical systems;
time series data. And that's the kind of data
it makes sense to think about if you are interested
in dynamics - how things change - a series of
measurements taken over the course of time
of a dynamical system. Most commonly, you
have a single series of measurements from a system,
that is, a table of xs and ts where the xs are scalars
and the time stamps of the measurements are
monotonically increasing. Sometimes you'll
have more than one measurement. We'll get back
to that later. Time series data is extremely common
and very important in any number of applications
in science, medicine, engineering, economics, etc.
The stock markets are time series data, so is an EKG,
so is a bird call like the plot at the bottom of this
picture. There are lots of ways to analyze time series
data. A large and important class of techniques is
called spectral analysis. These are ways to figure out
what frequencies are present in a signal. When you see
a sound board at a concert, you're seeing some of that.
These bars on the left here are telling you how much
of the signal's power is in each of a number of
different frequency bands. The sliders on these
sound board kinds of devices or graphic equalizers,
the things on the right, allow the sound engineer to
amplify or surpress signals in those different bands
In science and math, we generally plot the
frequency spectrum a little bit differently. Like this.
We don't bin the signal's frequencies like the
graphic equalizer does where each of those bars
on the left-hand side here reflects the power
in a range of frequencies like 100-200 hz.
A graphic equalizer would add up all the power in
that range and use that value to figure out how
many bars to display. This kind of curve doesn't do that.
It shows you how much power there is at each
frequency. If you played a pure tone like a single
key on a piano, you'd see a single spike on a graph
like this, neglecting all the overtones in the
instrument, that is. A piano doesn't really play
a pure tone. If you played a chord, you'd see something
like this. Of course, if your little finger weren't strong,
you might see something like this instead because
there is less power in that high note. Spectral analysis
techniques like fast Fourier Transforms, or FFTs,
let you analyze the frequency content of a time
series and produce graphs like that. That class
of technique takes a signal and pulls it apart into
building blocks. If it's an FFT, those are sine
waves of different frequencies and those
building blocks put back together make up the
signal. Technically though, that only works
if the signal is periodic and the system is linear,
otherwise, the mathematical requirements for
the procedure, for example, superposition, don't
work. Those techniques are also problematic if
the signal is non-stationary, for example if someone
is playing different notes at different times on the
piano. Since that's a pretty common situation,
people have come up with all sorts of extensions
to handle it. These include wavelets which you
may have heard of, and spectrograms, which
people use to think about bird calls among
other things. Spectrograms are essentially
power vs frequency plots turned on their side
and tracked over time. Here is a spectrogram
of a bird call.
Bird noises
A challenge here is the balance between how
much signal you need to compute the spectrum
and how fast the signal is changing. If, for example,
you need this long of a chunk to do a spectral analysis,
but the bird switches songs more rapidly like
it's doing in here, than the spectrogram will
not be accurate. Despite these limitations,
spectral analysis techniques can actually be
very useful in practice. However, they're rarely
used in the field of nonlinear dynamics, so I won't
talk about them much in this course.
There are also all sorts of statistics and
correlations that you can compute from time
series data, beginning with the mean and the
variance. But both of those are lumped things.
They take a whole bunch of information about a
signal and squash it into a single number.
There are lots of different signals that have the same
means and the same variances that look very different
dynamically. There are more sophisticated
statistical analysis techniques that will come up a bit
in this unit and the next one, but be aware that
they all do the same kind of thing. They do some
sort of statistical calculation across the entire
time series and give you one number back.
If you do that, you are effectively assuming,
or hoping, that your time series is stationary
with respect to that statistic. We'll get more into
that in the last segment of this unit. By the way,
there are also lots of ways to forecast time series
data, many of which can also get tripped up by
non-linearity and non-stationarity if they are
linear time and variant techniques. You may have
heard of things with acronyms like ARMA -
autoregressive moving-average models. We'll
talk about prediction in the last unit of this course.
Okay, going back to dynamical systems, when you
have time series data from a dynamical system,
your measurement may be a point of the state
variables, but that's not always possible. In a
complicated dynamical system, you may not know
what all the state variables are, and even if you
did know what they were, it might be hard to
measure them. And even if you can measure them,
you may not be able to do that without affecting
the dynamics of the system that you are studying.
The theory and algorithms we've used so far
in this course make the utopian assumption that
we know all the state variables, what they are,
and what their values are. But reality is very
different. You generally have a measurement of
one quantity that some function of some number
of the state variables. In the data that I measured
from my driven pendulum, for instance, I had
a sensor that measured the rotation of the bob,
but that sensor had a precision of 0.7 degrees and
a range of 0-360 degrees, so that data is both
quantized and wrapped in the sense that 361
degrees appears like 1 degree. So this is not
an exact measurement of the angle, theta,
of the pendulum, and even if the measurement
function - that is, the operation performed by
the device that you are using to gather the data -
really does measure a state variable of the system,
that data is a projection of the full state space.
On the left here I'm showing the Lorenz attractor in
3 dimensions. x, y, and z are the state variables.
In the middle is an x,z-projection of that. That is,
y is gone - I only measured x and z, so the attractor
gets smashed down flat onto the plane.
On the right-hand side, is what the attractor looks
like if I only measure z - that is, if I don't measure
x or don't measure y. The attractor gets further
smashed down onto a line. In all of these cases,
the projection that is effected by you not measuring
all of the state variables causes trajectories that
don't cross in reality to look like they are crossing.
If you look in the middle of the 2-D projection
at the center of the slide, you see lots of trajectory
crosses. That's a problem for 2 reasons.
First, in the kinds of systems that we're covering in
this course, there can only be one downhill
direction at any point in state space. Now, in
non-autonomous systems where the dynamical
landscape can change with time, that need not be
true, but we are not going there in this course.
Projections cause trajectories to violate that
requirement - that is, to cross - and if sensors
cause projections, that's a problem.
The second reason that crossings aren't okay goes
back to that notion of topology. I'll come back to this
in the third segment of this unit, but just for now,
topology is the fundamental mathematics of shape
and projections destroy that fundamental shape
because they create those false crossings.
This object, for instance, has a single hole. But
you could imagine if you looked at it from here,
and squashed it flat onto this plane, it might
look like this. And this squashed flat object
has 2 holes. One here and one there because of
this extra crossing. Again, if the action of the
sensor is causing this, that's a problem because
the sensor is altering the fundamental shape
of the object that we want to study - that is,
the trajectory from the dynamical system,
which should have one hole, but because we've
projected it, has 2.
One of the holy grail problems in control theory is
what's called the observer problem. To deduce the
internal state variables of a system and their
values from the outputs of the system. Imagine,
for example, trying to deduce the internal
state of a traffic light controller at a busy intersection
simply from observing the colors and timing
of the lights. That involves undoing a projection, which
you might think is impossible. The next segment is
about a quite amazing way to get around this
which is called delay coordinate embedding.