We've talked about a lot of theory and algorithms over the past 7 units in this course. In this unit, and the one that follows, we're going to get our hands dirty with data from dynamical systems; time series data. And that's the kind of data it makes sense to think about if you are interested in dynamics - how things change - a series of measurements taken over the course of time of a dynamical system. Most commonly, you have a single series of measurements from a system, that is, a table of xs and ts where the xs are scalars and the time stamps of the measurements are monotonically increasing. Sometimes you'll have more than one measurement. We'll get back to that later. Time series data is extremely common and very important in any number of applications in science, medicine, engineering, economics, etc. The stock markets are time series data, so is an EKG, so is a bird call like the plot at the bottom of this picture. There are lots of ways to analyze time series data. A large and important class of techniques is called spectral analysis. These are ways to figure out what frequencies are present in a signal. When you see a sound board at a concert, you're seeing some of that. These bars on the left here are telling you how much of the signal's power is in each of a number of different frequency bands. The sliders on these sound board kinds of devices or graphic equalizers, the things on the right, allow the sound engineer to amplify or surpress signals in those different bands In science and math, we generally plot the frequency spectrum a little bit differently. Like this. We don't bin the signal's frequencies like the graphic equalizer does where each of those bars on the left-hand side here reflects the power in a range of frequencies like 100-200 hz. A graphic equalizer would add up all the power in that range and use that value to figure out how many bars to display. This kind of curve doesn't do that. It shows you how much power there is at each frequency. If you played a pure tone like a single key on a piano, you'd see a single spike on a graph like this, neglecting all the overtones in the instrument, that is. A piano doesn't really play a pure tone. If you played a chord, you'd see something like this. Of course, if your little finger weren't strong, you might see something like this instead because there is less power in that high note. Spectral analysis techniques like fast Fourier Transforms, or FFTs, let you analyze the frequency content of a time series and produce graphs like that. That class of technique takes a signal and pulls it apart into building blocks. If it's an FFT, those are sine waves of different frequencies and those building blocks put back together make up the signal. Technically though, that only works if the signal is periodic and the system is linear, otherwise, the mathematical requirements for the procedure, for example, superposition, don't work. Those techniques are also problematic if the signal is non-stationary, for example if someone is playing different notes at different times on the piano. Since that's a pretty common situation, people have come up with all sorts of extensions to handle it. These include wavelets which you may have heard of, and spectrograms, which people use to think about bird calls among other things. Spectrograms are essentially power vs frequency plots turned on their side and tracked over time. Here is a spectrogram of a bird call. Bird noises A challenge here is the balance between how much signal you need to compute the spectrum and how fast the signal is changing. If, for example, you need this long of a chunk to do a spectral analysis, but the bird switches songs more rapidly like it's doing in here, than the spectrogram will not be accurate. Despite these limitations, spectral analysis techniques can actually be very useful in practice. However, they're rarely used in the field of nonlinear dynamics, so I won't talk about them much in this course. There are also all sorts of statistics and correlations that you can compute from time series data, beginning with the mean and the variance. But both of those are lumped things. They take a whole bunch of information about a signal and squash it into a single number. There are lots of different signals that have the same means and the same variances that look very different dynamically. There are more sophisticated statistical analysis techniques that will come up a bit in this unit and the next one, but be aware that they all do the same kind of thing. They do some sort of statistical calculation across the entire time series and give you one number back. If you do that, you are effectively assuming, or hoping, that your time series is stationary with respect to that statistic. We'll get more into that in the last segment of this unit. By the way, there are also lots of ways to forecast time series data, many of which can also get tripped up by non-linearity and non-stationarity if they are linear time and variant techniques. You may have heard of things with acronyms like ARMA - autoregressive moving-average models. We'll talk about prediction in the last unit of this course. Okay, going back to dynamical systems, when you have time series data from a dynamical system, your measurement may be a point of the state variables, but that's not always possible. In a complicated dynamical system, you may not know what all the state variables are, and even if you did know what they were, it might be hard to measure them. And even if you can measure them, you may not be able to do that without affecting the dynamics of the system that you are studying. The theory and algorithms we've used so far in this course make the utopian assumption that we know all the state variables, what they are, and what their values are. But reality is very different. You generally have a measurement of one quantity that some function of some number of the state variables. In the data that I measured from my driven pendulum, for instance, I had a sensor that measured the rotation of the bob, but that sensor had a precision of 0.7 degrees and a range of 0-360 degrees, so that data is both quantized and wrapped in the sense that 361 degrees appears like 1 degree. So this is not an exact measurement of the angle, theta, of the pendulum, and even if the measurement function - that is, the operation performed by the device that you are using to gather the data - really does measure a state variable of the system, that data is a projection of the full state space. On the left here I'm showing the Lorenz attractor in 3 dimensions. x, y, and z are the state variables. In the middle is an x,z-projection of that. That is, y is gone - I only measured x and z, so the attractor gets smashed down flat onto the plane. On the right-hand side, is what the attractor looks like if I only measure z - that is, if I don't measure x or don't measure y. The attractor gets further smashed down onto a line. In all of these cases, the projection that is effected by you not measuring all of the state variables causes trajectories that don't cross in reality to look like they are crossing. If you look in the middle of the 2-D projection at the center of the slide, you see lots of trajectory crosses. That's a problem for 2 reasons. First, in the kinds of systems that we're covering in this course, there can only be one downhill direction at any point in state space. Now, in non-autonomous systems where the dynamical landscape can change with time, that need not be true, but we are not going there in this course. Projections cause trajectories to violate that requirement - that is, to cross - and if sensors cause projections, that's a problem. The second reason that crossings aren't okay goes back to that notion of topology. I'll come back to this in the third segment of this unit, but just for now, topology is the fundamental mathematics of shape and projections destroy that fundamental shape because they create those false crossings. This object, for instance, has a single hole. But you could imagine if you looked at it from here, and squashed it flat onto this plane, it might look like this. And this squashed flat object has 2 holes. One here and one there because of this extra crossing. Again, if the action of the sensor is causing this, that's a problem because the sensor is altering the fundamental shape of the object that we want to study - that is, the trajectory from the dynamical system, which should have one hole, but because we've projected it, has 2. One of the holy grail problems in control theory is what's called the observer problem. To deduce the internal state variables of a system and their values from the outputs of the system. Imagine, for example, trying to deduce the internal state of a traffic light controller at a busy intersection simply from observing the colors and timing of the lights. That involves undoing a projection, which you might think is impossible. The next segment is about a quite amazing way to get around this which is called delay coordinate embedding.