1 99:59:59,999 --> 99:59:59,999 [Logo] Complexity Explorer, Santa Fe Institute 2 99:59:59,999 --> 99:59:59,999 In this last segment I am going to talk about something completely different, 3 99:59:59,999 --> 99:59:59,999 the use of chaos in music and dance. 4 99:59:59,999 --> 99:59:59,999 This dates back to work by Diana Dabby 5 99:59:59,999 --> 99:59:59,999 who was a PHD student of Steven Strogatz at MIT, 6 99:59:59,999 --> 99:59:59,999 and also a Carnegie Hall-level concert pianist. 7 99:59:59,999 --> 99:59:59,999 The reason I found out about this 8 99:59:59,999 --> 99:59:59,999 was because I got Diana's paper to review. 9 99:59:59,999 --> 99:59:59,999 Some of you may not know about this, 10 99:59:59,999 --> 99:59:59,999 the way you get a paper published in the scientific literature is, 11 99:59:59,999 --> 99:59:59,999 you send it in, the editor asks 12 99:59:59,999 --> 99:59:59,999 other people who can evaluate your work for a review, 13 99:59:59,999 --> 99:59:59,999 and if the reviews are positive the paper gets in. 14 99:59:59,999 --> 99:59:59,999 I have reviewed lots and lots of papers over my career 15 99:59:59,999 --> 99:59:59,999 but this was the first and only one 16 99:59:59,999 --> 99:59:59,999 that came with a cassette tape. 17 99:59:59,999 --> 99:59:59,999 Remember those? Maybe not... 18 99:59:59,999 --> 99:59:59,999 Diana's idea was to take a piece of music 19 99:59:59,999 --> 99:59:59,999 and consider it as a pitch sequence, 20 99:59:59,999 --> 99:59:59,999 just a sequence of labeled notes. 21 99:59:59,999 --> 99:59:59,999 The piece above is C, E, G, C, E, G and so on and so forth 22 99:59:59,999 --> 99:59:59,999 And then she took a chaotic attractor, 23 99:59:59,999 --> 99:59:59,999 didn't matter which one, 24 99:59:59,999 --> 99:59:59,999 and she generated a trajectory on that 25 99:59:59,999 --> 99:59:59,999 chaotic attractor with the points spaced 26 99:59:59,999 --> 99:59:59,999 the same distance that the notes were 27 99:59:59,999 --> 99:59:59,999 spaced in the original piece. 28 99:59:59,999 --> 99:59:59,999 Now neglect the fact that different notes 29 99:59:59,999 --> 99:59:59,999 can be different lengths, 30 99:59:59,999 --> 99:59:59,999 we can talk about that on the forum if you would like. 31 99:59:59,999 --> 99:59:59,999 Then what she did, was take the first point on the trajectory 32 99:59:59,999 --> 99:59:59,999 and she labeled it with the first note in the piece, 33 99:59:59,999 --> 99:59:59,999 and then she took the second point in the trajectory, 34 99:59:59,999 --> 99:59:59,999 and labeled it with the second note in the piece, and so on and so forth, 35 99:59:59,999 --> 99:59:59,999 until she had the pitch sequence wrapped 36 99:59:59,999 --> 99:59:59,999 all the way around the chaotic attractor. 37 99:59:59,999 --> 99:59:59,999 That established a mapping between 38 99:59:59,999 --> 99:59:59,999 the flow of the dynamics and 39 99:59:59,999 --> 99:59:59,999 the flow of the musical piece, 40 99:59:59,999 --> 99:59:59,999 and she used that mapping to generate 41 99:59:59,999 --> 99:59:59,999 variations on that musical piece. 42 99:59:59,999 --> 99:59:59,999 Now, if you generate a trajectory from 43 99:59:59,999 --> 99:59:59,999 exactly the same initial condition, and 44 99:59:59,999 --> 99:59:59,999 you had some sort of device that could 45 99:59:59,999 --> 99:59:59,999 recognise which little green square it was 46 99:59:59,999 --> 99:59:59,999 and play the appropriate note, 47 99:59:59,999 --> 99:59:59,999 you would get back exactly the original piece, but 48 99:59:59,999 --> 99:59:59,999 if you chose some other point on the 49 99:59:59,999 --> 99:59:59,999 chaotic attractor, generated a trajectory, 50 99:59:59,999 --> 99:59:59,999 and used the mapping to play the notes 51 99:59:59,999 --> 99:59:59,999 that that trajectory followed, 52 99:59:59,999 --> 99:59:59,999 you would get a variation. 53 99:59:59,999 --> 99:59:59,999 So let me play you some of these examples. 54 99:59:59,999 --> 99:59:59,999 Here's the original piece... 55 99:59:59,999 --> 99:59:59,999 (music plays) 56 99:59:59,999 --> 99:59:59,999 As many of you know, that is the 57 99:59:59,999 --> 99:59:59,999 Prelude in C Major from the first book of Bach's Well Tempered Clavier 58 99:59:59,999 --> 99:59:59,999 Now if Diana wrapped that piece, 59 99:59:59,999 --> 99:59:59,999 the pitch sequences in that piece, 60 99:59:59,999 --> 99:59:59,999 around a chaotic attractor, the Lorenz attractor, 61 99:59:59,999 --> 99:59:59,999 and used that mapping to generate 62 99:59:59,999 --> 99:59:59,999 a chaotic variation, this is what you would hear... 63 99:59:59,999 --> 99:59:59,999 (music plays) 64 99:59:59,999 --> 99:59:59,999 And just like a variation is supposed to do, 65 99:59:59,999 --> 99:59:59,999 it sounds like the original in some sense, 66 99:59:59,999 --> 99:59:59,999 and yet is different. 67 99:59:59,999 --> 99:59:59,999 I just love this work, and I talk about it 68 99:59:59,999 --> 99:59:59,999 every year in my University of Colorado 69 99:59:59,999 --> 99:59:59,999 version of this class. 70 99:59:59,999 --> 99:59:59,999 After one of those classes, 71 99:59:59,999 --> 99:59:59,999 one of the students, Josh Stewart, 72 99:59:59,999 --> 99:59:59,999 came up after class and said 73 99:59:59,999 --> 99:59:59,999 "I wonder if we could do this for dance?" 74 99:59:59,999 --> 99:59:59,999 That is, instead of music in, music out 75 99:59:59,999 --> 99:59:59,999 how about dance in, dance out? 76 99:59:59,999 --> 99:59:59,999 So here's the idea, very much like Diana Dabby's, 77 99:59:59,999 --> 99:59:59,999 Imagine that you have 300 dance moves, 78 99:59:59,999 --> 99:59:59,999 maybe they are key frames in an animation, or something like that, 79 99:59:59,999 --> 99:59:59,999 or you have taken a picture of a dancer 80 99:59:59,999 --> 99:59:59,999 every tenth of a second, 81 99:59:59,999 --> 99:59:59,999 then you take a trajectory on a chaotic attractor, 82 99:59:59,999 --> 99:59:59,999 and you evenly space in time, not in space, but in time, 83 99:59:59,999 --> 99:59:59,999 you evenly space 300 points around that attractor, 84 99:59:59,999 --> 99:59:59,999 then you generate a tiling of that attractor, 85 99:59:59,999 --> 99:59:59,999 Josh used a Voronoi diagram, 86 99:59:59,999 --> 99:59:59,999 such that each point is in the centre 87 99:59:59,999 --> 99:59:59,999 of one of these cells. 88 99:59:59,999 --> 99:59:59,999 Now you can imagine that the original 89 99:59:59,999 --> 99:59:59,999 trajectory as it goes around, 90 99:59:59,999 --> 99:59:59,999 lighting up those cells in sequence, 91 99:59:59,999 --> 99:59:59,999 that's called the cell itinerary of these dynamics, 92 99:59:59,999 --> 99:59:59,999 so this might be the first cell, 93 99:59:59,999 --> 99:59:59,999 and the second cell, and the third cell, 94 99:59:59,999 --> 99:59:59,999 and the fourth cell, and so on and so forth. 95 99:59:59,999 --> 99:59:59,999 Then what you do, is that you look at the dance, 96 99:59:59,999 --> 99:59:59,999 and you take the first dance move 97 99:59:59,999 --> 99:59:59,999 and you put it in the first cell, 98 99:59:59,999 --> 99:59:59,999 and then you take the second dance move, 99 99:59:59,999 --> 99:59:59,999 and you put it in the second cell 100 99:59:59,999 --> 99:59:59,999 and so on and so forth, 101 99:59:59,999 --> 99:59:59,999 and you end up with a mapping that looks like this, 102 99:59:59,999 --> 99:59:59,999 and then you use this much in the same way 103 99:59:59,999 --> 99:59:59,999 as Diana Dabby used her scheme. 104 99:59:59,999 --> 99:59:59,999 If you started at the exact same initial condition 105 99:59:59,999 --> 99:59:59,999 and generated a trajectory and 106 99:59:59,999 --> 99:59:59,999 played the dance moves for every cell it hit, 107 99:59:59,999 --> 99:59:59,999 you would get back the original dance piece, 108 99:59:59,999 --> 99:59:59,999 but if you started from some-place else 109 99:59:59,999 --> 99:59:59,999 on the same attractor, you would get a 110 99:59:59,999 --> 99:59:59,999 chaotic variation of that dance. 111 99:59:59,999 --> 99:59:59,999 Here is a demonstration of that scheme in action... 112 99:59:59,999 --> 99:59:59,999 This is the dance we used as a demonstration case, 113 99:59:59,999 --> 99:59:59,999 not as nice a Bach. 114 99:59:59,999 --> 99:59:59,999 Here is a chaotic variation of that dance, 115 99:59:59,999 --> 99:59:59,999 generated using the Lorenz equations. 116 99:59:59,999 --> 99:59:59,999 You can see it looks like 117 99:59:59,999 --> 99:59:59,999 the original in some sense, 118 99:59:59,999 --> 99:59:59,999 but it departs from that original... 119 99:59:59,999 --> 99:59:59,999 There is nothing special about the Lorenz equations, 120 99:59:59,999 --> 99:59:59,999 you can do this with the Rossler equations too, 121 99:59:59,999 --> 99:59:59,999 and you get similar effects. 122 99:59:59,999 --> 99:59:59,999 By the way, the first time I talked 123 99:59:59,999 --> 99:59:59,999 about this work at a conference 124 99:59:59,999 --> 99:59:59,999 and I showed this demo, 125 99:59:59,999 --> 99:59:59,999 a guy piped up from the second row 126 99:59:59,999 --> 99:59:59,999 and said "it looks like Al Gore doing the 'Macarena'". 127 99:59:59,999 --> 99:59:59,999 OK, what's going on here? 128 99:59:59,999 --> 99:59:59,999 129 99:59:59,999 --> 99:59:59,999 The original trajectory lit up those 130 99:59:59,999 --> 99:59:59,999 cells in a certain order. 131 99:59:59,999 --> 99:59:59,999 The variation trajectory 132 99:59:59,999 --> 99:59:59,999 lights them up in a different order. 133 99:59:59,999 --> 99:59:59,999 So what this ends up doing, 134 99:59:59,999 --> 99:59:59,999 is taking chunks out of 135 99:59:59,999 --> 99:59:59,999 different parts of the dance 136 99:59:59,999 --> 99:59:59,999 and splicing them together 137 99:59:59,999 --> 99:59:59,999 in a different order. 138 99:59:59,999 --> 99:59:59,999 Here's a demonstration that will make that visible. 139 99:59:59,999 --> 99:59:59,999 This is a medley. There is a Macarena... ballet jump... and a kenpo karate kata... 140 99:59:59,999 --> 99:59:59,999 And when I show you a chaotic variation of that medley, 141 99:59:59,999 --> 99:59:59,999 you will be able to see different chunks of the different parts of the dance 142 99:59:59,999 --> 99:59:59,999 pasted together in a different order. 143 99:59:59,999 --> 99:59:59,999 There's some Makarena... goes in and out of karate, more Makarena... there's karate 144 99:59:59,999 --> 99:59:59,999 Some place in here there's a full copy of the ballet jump... there it is. 145 99:59:59,999 --> 99:59:59,999 Anyway, you get the idea. 146 99:59:59,999 --> 99:59:59,999 If we simply take the same dance moves and shuffle them randomly, 147 99:59:59,999 --> 99:59:59,999 this is what it looks like... any continuity you see here 148 99:59:59,999 --> 99:59:59,999 is because the animation software is connecting the dots, 149 99:59:59,999 --> 99:59:59,999 it is interpolating between the different key frames, 150 99:59:59,999 --> 99:59:59,999 but you can see it looks kind of like it's having an epileptic seizure. 151 99:59:59,999 --> 99:59:59,999 There's not the same kind of structure in this movement. 152 99:59:59,999 --> 99:59:59,999 I've had lots of students playing with this over the years. 153 99:59:59,999 --> 99:59:59,999 It doesn't just apply to sequences of movement snapshots, 154 99:59:59,999 --> 99:59:59,999 or sequences of musical notes, it also works with words. 155 99:59:59,999 --> 99:59:59,999 Here is a chaotic variation on a piece of Alice in Wonderland. 156 99:59:59,999 --> 99:59:59,999 Now as I said, it's taking chunks out of different regions of the dance 157 99:59:59,999 --> 99:59:59,999 and sticking them together in a different order. 158 99:59:59,999 --> 99:59:59,999 This is something that happens in lots of different kinds of music. 159 99:59:59,999 --> 99:59:59,999 It's also been done in dance by a modern choreographer named Merce Cunningham, 160 99:59:59,999 --> 99:59:59,999 who would chunk up a dance into phrases 161 99:59:59,999 --> 99:59:59,999 and throw the I Ching to determine the order in which the dancers 162 99:59:59,999 --> 99:59:59,999 would execute those phrases. 163 99:59:59,999 --> 99:59:59,999 And the dancers, and the critics and the audiences, 164 99:59:59,999 --> 99:59:59,999 all hated it because there are potentially abrupt transitions at the chunk boundaries. 165 99:59:59,999 --> 99:59:59,999 Those abrupt transitions arise because the two moves on either side of 166 99:59:59,999 --> 99:59:59,999 those chunk boundaries, may be very very far apart in - kind of - body space. 167 99:59:59,999 --> 99:59:59,999 You can really see that in this animation... 168 99:59:59,999 --> 99:59:59,999 This is again the original, before the chaotic variation, 169 99:59:59,999 --> 99:59:59,999 of a short ballet adagio composed by a colleague of mine... 170 99:59:59,999 --> 99:59:59,999 Now, what does it look like if we generate a chaotic variation on that? 171 99:59:59,999 --> 99:59:59,999 You can see the smooth movement on the - ouch... ouch... ouch - 172 99:59:59,999 --> 99:59:59,999 so you can really see the transitions here. 173 99:59:59,999 --> 99:59:59,999 This set us off on a primrose path, because I wanted to interpolate, 174 99:59:59,999 --> 99:59:59,999 that is to smooth those gaps. 175 99:59:59,999 --> 99:59:59,999 Not just in a way that was faithful to the tendons and muscles in the body, 176 99:59:59,999 --> 99:59:59,999 but also that was faithful to the style of the movement genre. 177 99:59:59,999 --> 99:59:59,999 So what we did, was we took a corpus, 178 99:59:59,999 --> 99:59:59,999 a whole bunch of examples of a certain kind of dance, 179 99:59:59,999 --> 99:59:59,999 and we looked a the individual joints in the body 180 99:59:59,999 --> 99:59:59,999 so like the wrist, and the elbow, and the knee, 181 99:59:59,999 --> 99:59:59,999 and we built a directed graph to capture how each of those joints moved. 182 99:59:59,999 --> 99:59:59,999 Once you do that, you look at the graph for say the right shoulder 183 99:59:59,999 --> 99:59:59,999 and you look for the shoulder position in the initial state 184 99:59:59,999 --> 99:59:59,999 and then you look at that same graph and you look for the shoulder position in the final state. 185 99:59:59,999 --> 99:59:59,999 So getting the shoulder from here to here can be accomplished simply by looking 186 99:59:59,999 --> 99:59:59,999 for something like the shortest path through that graph. 187 99:59:59,999 --> 99:59:59,999 Now it gets a little bit more complicated than that 188 99:59:59,999 --> 99:59:59,999 because you actually have 44 joints in your body 189 99:59:59,999 --> 99:59:59,999 and you have to do that in parallel and they all have to be the same path length 190 99:59:59,999 --> 99:59:59,999 because otherwise all the different joints in the body will depart from the initial 191 99:59:59,999 --> 99:59:59,999 condition at the right time, 192 99:59:59,999 --> 99:59:59,999 but they will all arrive at the final condition at different times, 193 99:59:59,999 --> 99:59:59,999 and that doesn't look good. 194 99:59:59,999 --> 99:59:59,999 The other thing that made this hard is the graphs really are 195 99:59:59,999 --> 99:59:59,999 a lot more complicated than that one I just showed you. 196 99:59:59,999 --> 99:59:59,999 So here is an interpolation task that we gave this programme. 197 99:59:59,999 --> 99:59:59,999 Get me from here to here in a manner that is consistent with the observed 198 99:59:59,999 --> 99:59:59,999 movement patterns in a corpus of ballet. 199 99:59:59,999 --> 99:59:59,999 Here is what the programme produced... 200 99:59:59,999 --> 99:59:59,999 So computer-generated dance, using machine learning techniques 201 99:59:59,999 --> 99:59:59,999 on a ballet corpus. 202 99:59:59,999 --> 99:59:59,999 This work didn't just suck me down the primrose path of machine learning, 203 99:59:59,999 --> 99:59:59,999 it also got me working with dancers. 204 99:59:59,999 --> 99:59:59,999 And that has culminated in a piece that has been performed in a number of cities.