in this section we thought we would talk about an advanced concept and that is ways that machine learning has helped ways that machine learning has helped improve traditional algorithms and in particular this stack 20 cargo algorithm is work that I have done in my research so let's imagine that we want to find something called the expected value of a function that is if in many systems they operate under different conditions and we want to know how the system behaves on average and one way to find out this average thing is to do something that's called a Monte Carlo procedure and if you remember from way back during the go section we talked about Monte Carlo tree search and the Monte Carlo is the same idea here So let's say we have a die and we want to figure out what number we expect to get on average weighting each number according to its value so what we could do is we could roll the dice a whole bunch of times maybe the first time we get a two and then we get a six and then we get another two and another three and we keep rolling it and maybe you find that average value is three point four now the true average value of a die is three point five so we didn't get the exact number but we did get pretty close now this is also used for other systems so for example let's say we have an airplane and we want to know the average drag on this airplane now airplanes are really finely tuned to behave well under a variety of conditions so maybe when we flew our airplane four times we found drag values if say eight point one six eight point one for another eight point one four and then maybe an eight point one five finding an average value of around eight point one four now you could imagine firstly that it's really expensive to fly an airplane and so we want to find this expected value this expected performance using as few flights as possible because each flight is really expensive the other really key idea here to note though is that with that die example we found that with a couple of rolls we had a pretty large error while on the airplane example after a couple of flights our error was only really small and that's because the die numbers vary all the way from 1 to 6 while with this airplane performance example there are much close to each other eight point one six eight point one four so a key concept Monte Carlo is that the more your simulations can vary the larger error you can have in how bad you are predicting the expected value so how this works is on an aircraft we have a lot of sensors that measure what the conditions were when the flight was taking place so we don't just know the value of the drag for example maybe we also know the Mach number I wish the airplane was flying at that is how fast the airplane was flying relative to the speed of sound and so what we have is this input-output relationship where the input is how fast we are flying and the output is the drag of the airplane and again we want to know the average drag over a bunch of flights so for example we know how much fuel the airplane needs and so on the right here we see a plot where the x-axis has the Mach number that we're flying at the y-axis is what is the value of that drag and say the blue line is the true value of drag aught that Mach number now this isn't actually how an airplane behaves so don't go ahead and use this for your designs but it does give us this intuition where now what we can do is take that input Mach number data and learn what the drag is as a function of that input data now maybe we come up with this dotted green line that's in the image now we see it's not an exact match it's not a perfect fit for how much drag there is but we see it's doing a pretty good job even though we've only had a small number of samples okay and so now the key concept is what we can do is we can take those two lines and subtract them from one another and so now we get this new what we will call a regularized surface that is where we have the true behavior what when we're removing the effects that we can learn really well and so what we have now is this red line below that's the difference between our the true estimate and our machine learned value and now we can Monte Carlo this object here and if you remember Monte Carlo has a really large error when the numbers changed by a lot but by bringing in this machine learned surface now the function is much more smooth the wiggles aren't as big and so our expected error is going to go down by a lot just because we've reduced what we would call the variance of the function how many times the function Wiggles around and so if you'll pardon the one technical slide the x-axis here shows how many samples we have that is how many times did we fly our airplane and the y-axis has what about our estimate given that number of samples on average so in order to generate this plot I kind of ran this idealized procedure a large number of times to see how well worked now what we see is that all of for all of these lines the error goes down as we get more samples right that makes sense the more data points we have the better we're going to do but what this also tells us is the Green Line was if we just used Monte Carlo we only took two average values like we did with a dye the red line shows what if we had just used the machine learning data so we had taken our samples fit machine learning and saw the average performance of the machine learning and then the blue line shows this procedure that I told you where we combine these two things together and have this different Monte Carlo surface and that's the blue line where we've combined the two and we see that for all numbers of samples so no matter how many data points we had the stack Monte Carlo estimate has a lower error so what we saw here is that we took Monte Carlo a technique that was invented during the Manhattan Project in the 40s but by using machine learning we were able to augment this very simple idea and decrease the error and the state for example decreasing the number of airplane flights we would need to get a good enough estimate of the drag so go back a couple of lectures back to computers playing the game of Go we can actually see that this kind of idea is related to the alphago program and how it was able to be so effective and so an alphago they had this Monte Carlo tree search where they played out random games to measure a probability of winning that's not the only thing they did they augment adapt probability of winning with what we called the value function which estimated how good a given position was and so by combining these two ideas they regularize the Monte Carlo and made the estimate of winning much more accurate than otherwise so this concludes our tutorial thank you for joining our to me and I we hope you've gained a good high level overview of machine learning is and are now well to learn more one great way to learn more is to look at the supplementary material where we've included a lot more information about the topics we've discussed we hope that you've enjoyed these lessons and we hope that you're eager to learn more thank you