This segment is about noise - something that you can never really avoid in real-world situations - kind of like the HVAC system in my office, which is producing a high-frequency hiss, which gets picked up by the nice microphone, but not so much by the headset. The segment is also about filtering - the act of removing - noise from data. Now, in order to filter noise out of data, you need to have a way to discriminate between signal and noise. In most traditional applications, that discriminating factor is where the signal is in the frequency spectrum. Most commonly, we assume that high frequency noise - like my HVAC system - is bad, and low frequency noise - like my voice, which is much lower frequency than that - is good. If that assumption is true, you can use something called "a low-pass filter" to get rid of the noise. A low-pass filter is a circuit or an algorithm that lets through the low frequency stuff and doesn't let through the high frequency stuff. But, if the signal and the noise are all mixed together, all the way across the frequency spectrum, there's no way you can draw a discriminating line anywhere in that spectrum and say that the stuff over here is good, and the stuff over here is bad. And then, you can't use traditional filtering technology. If you did, you would be "throwing the baby out with the bathwater," to use an old expression - throwing away meaningful components, the signal along with the noise. And that's a bad idea. Here's a paper that makes that point quite emphatically. Okay - so how to reduce noise if you can't use the traditional weaponry. There are a number of ways. I will talk about two: one that takes advantage of the geometry of stable and unstable manifolds, and another that takes advantage of topological properties. A noisy measurement is like a point in state space with a noise ball around it. If there were no noise, you'd measure the point right in the middle where the blue point is. But, because of the noise you might see that point any place in this ball. Now, think about what the stable and unstable manifolds will do to a ball like this. As time evolves, that ball will stretch along the unstable manifold, and compress along the stable manifold. You saw this on a unit test. That can be turned to advantage. Here's the idea. You have three successive noisy measurements of a trajectory on a chaotic attractor, at three successive points in time. The central points are the true state of the system. If there were no noise, you'd measure the point right there. But, because of the noise, you might see the point any place in the noise balls. Since this is a chaotic attractor, it has at least one unstable manifold. And, since it's a chaotic attractor, it has at least one stable manifold. And, if you have a way to move time forwards and backwards - that is, to make forecasts and hindcasts - you can leverage the effects of these manifolds to reduce the noise. If you evolve the green measurement - the left one - forwards to the point in time where the blue one was performed, it will effectively stretch out the green noise ball along the unstable manifold and compress it along the stable manifold. So, the green noise ball, if we evolve it forward in time from here to here, will look like this. Same idea with projecting the black noise ball, at the third point backwards in time, to the same time point as the blue ball was measured. It will look like this - because in backwards time, it will stretch out along the stable manifold and compress along the unstable manifold. If you did the evolution right, all three of those should really be at the same point in state space, so the true value will be somewhere in this overlap region. And, notice that that overlap region is a lot smaller than the original noise balls. This scheme is due to Farmer and Sidorowich. Here's the version of that drawing from their paper. The operation that's schematized by these arrows is effected by some sort of model of the system dynamics - for example, fourth-order Runge-Kutta on the ODE, if you have it - or some sort of prediction model, which we'll get to in the last unit. Here's a summary of the algorithm - a more compact version of the explanation I just gave you - so that you can refer back to it in the course slides. This last bullet here goes back to the picture. This works if you have stable and unstable manifolds that cross - that is, they're transverse. It works best if they're at 90 degrees, just because that will squash the noise ball the most. Here are some results from their paper. This is a standard numerical experiment. They took a trajectory from a known system - Hénon in this case - they added some noise, and they saw if their scheme could get rid of that noise. And it did. What this figure is actually plotting is the amount of noise - here - at each point along the length of the trajectory, before and after their filtering operation was performed. And, you may not be able to read this. This is one part in a thousand of noise that they added. This is one part in ten to the thirteenth noise that was left after their filtering scheme. So they reduce the noise by 10 orders of magnitude, which is pretty good. Again, what's going on here is a use of the geometry of the dynamics to reduce noise. There are other properties of chaotic attractors that we can also use to reduce noise, like the topology. I'm not going to talk about this in any depth, but the basic idea here is that chaotic attractors are connected, and connected sets don't have any isolated points - which are kind of what they sound like. Therefore, if you find isolated points in your trajectory, using what are called "computational topology techniques," those are probably noise effects - and can safely be thrown away. There's a reference here at the bottom of the slide. If you're interested in it, you can get that paper off my web page. There are other topological properties besides connectedness that you can leverage to get rid of noise, including continuity. This is related to that there's only one downhill direction from any point idea. If two points that are nearby in state space end up really far apart at the next time click, then there may have been different dynamics operating on that region of state space here at different times. And that can be used to detect regime shifts - like bifurcations - and to separate signal from noise. There's a citation down here, if you're interested.