Lecture 15 Fall 2025: Backpropagation transcript

All right, welcome back everybody. So today we are going to talk about an example of using stochastic gradient descent which we talked about on Tuesday and specifically using stochastic gradient descent to train a neural network or back propagation. So to start off, let's recall stochastic gradient descent. And today's lecture is a little bit notationheavy. Um, but I've already posted the lecture notes online. So if you do want to download them to follow along, you're more than welcome to.

So what we did with stochastic gradient descent at the last lecture is we said we had a loss function with respect to our parameters w that we could decompose or break down as the sum or the average of individual loss functions like this where each term here would correspond to the loss associated for instance with a single training sample I. And so when we do stochastic gradient descent at iteration um T, we would select a sample it say uniformly at random or based on some random shuffling at every epic and then we would update our weights.

So our weight vector estimate for iteration t + 1 would be our weight at time t minus a step size of length towo times a step in the direction of the negative gradient of this loss just for the teeth sample evaluated at our current estimate wt. So at each iteration what we're doing is we are selecting a new sample I subt and then we are updating our parameters or our weight vector um using the gradient of only the loss associated with that if sample.

So when we talk about training a neural network and running back propagation, this is really at the heart of it. And everything we're going to be talking about today is how to figure out what these gradients are and how to calculate them efficiently in the context of a neural network. So to start off, we're going to start with a very simple neural network. And I'm going to draw it like this. So I'm going to have these circles corresponding to nodes in my network.

And these on the left are going to be my input nodes. So what I'm inputting are say features. So I would have x i1, x i2, x i3, all the way down to x i p if I've got p different feature vectors. And then for this last node, I'm just going to input a one. And I'll talk a little bit more about that in a second. And then I'm going to have an output node here. And I draw this like this. So I've got all of these lines here from each input node to my output node.

And associated with each one of those lines is a weight. So here I will have W1, W2, W3, all the way to WP. And then the weight on the one I'm going to call B for a bias. So I've got weights and a bias. And then inside this node here I will have a sigma and the output will be a yhat i. If I feed in feature vector x i. So when I write this diagram this corresponds to a very simple equation. This means yi is equal to this function sigma applied to my feature vector x i inner product with the vector of weights w1 through wp plus this bias b um and and that's it.

So this diagram here is just a graphical rep representation of this very simple model. So without the sigma, this is just exactly the linear model that we've been working with all quarter. So the only thing that's really changed now is that we've got this sigma function. So first of all, just to kind of simplify our discussion here, we said with stochastic gradient descent at each iteration, we'll choose one sample. So I'm going to just stop writing the eyes just to keep things a little bit simpler.

So, we're just going to say um yhat is sigma xrpose w plus b. Um and the sample i is going to be implied for the rest of this lecture. And I'm also right now going to call this thing inside the sigma z just to give us a little bit of shorthand that'll help things out. And sigma is called an activation function. Okay, so there's many common examples of activation functions. Of course, the one that we already have experience with is sigma of Z is equal to Z.

So, we just really don't do anything. Um, and then in our our model is simply Yhat is XRPOSE W + B, which is what we've been doing all quarter long. And so then we would call this a linear activation. Okay. So that's maybe the least interesting example, but just um I mention it because I want you to realize that this is something that we've at least looked at a special case of all along here. Okay. But let's look at some other activation functions that are in common use.

So, so first of all, I just want to say sigma of Z is going to be the activation function just like I've said. And then I'm going to use the notation sigma prime Z to correspond to the derivative of sigma with respect to Z. And that's something we are also going to need. So when we talk about examples, one is the RLU activation and RLU stands for rectified linear unit. So here sigma of z corresponds to simply um z if z is positive and zero otherwise or in other words it's going to be equal to the maximum of z and zero.

And if we think about what the derivative of this is, this isn't exactly differentiable everywhere, right? When when Z is equal, so if we were to plot this function as a function of Z and we were to look at ReLU of Z, then it looks like zero and then linear. So right here at zero, it's not differentiable, but everywhere else it is. And so we're going to still use the notation sigma prime z. And so the derivative is equal to zero whenever z is less than zero.

It's equal to 1 when z is greater than zero. And right at zero we can use any value we want between 0 and one. Um and we call this a a um subgradient. Um but in terms of practical usage, this is what we're going to use for um sigma prime of z. Okay. So this is extremely commonly used um in part because the gradients are very easy. Almost everywhere the gradient is um the the values that we're going to be computing are ones and zeros.

So that can make some computation very simple. Another common example is tan h where I just want to make sure I get this right. Sigma of z is equal to e to the positive z uh minus e to the z divided by e to the positive z + e to the z. Um this is one that's especially widely used in um recurrent neural networks and in um large language models. And here sigma prime z the derivative is equal to 1us sigma z^ 2. And then finally we're going to look at the logistic activation function where sigma z is equal to 1 over 1 + e to the negative z.

And for this the derivative sigma z prime is equal to um sigma z time 1 minus sigma z. Okay. So we are going to talk about strategies for optimizing the weights w and the bias b in this very simple neural network. And what we'll see is that obviously sigma and its derivative are going to show up. Now we are going to walk through this in detail for the logistic activation function but all the principles that we are going to use translate easily to these and other activation functions.

I just wanted to highlight some of the ones that are the most common here. Are there any questions so far? Great. All right. So, oh okay. So, we are going to consider squared air loss just to work this out in detail. So in particular, our LI of W and B is going to equal Yi minus um sigma of X I transpose W plus B quantity squared. So this is the loss associated with the E sample when we've got weights W and bias B. And so what I'm going to do is just introduce a little notation that's going to help us with computing gradients here.

So inside the sigma I'm going to call this ZI. And then after computing the sigma I'm going to call this Yi hat. Right? So if I were to feed an XI into my network, then if we look back at our little graphical picture over here, the way I think about this is what's being fed into this node here where the sigma is drawn inside is zi is this inner product of x i and w plus b. And then what's coming out is yi hat which is just zi fed into this activation function sigma.

So zi is going in and it's just a linear function of our weights and the inputs and yihat is coming out and in general that's a nonlinear function of all these inputs. Yeah. Xi, that's a single observation. >> Um, X subi is a vector of the features for the E sample. Sorry, I did say a few minutes ago that I wasn't always going to use the I. Um, and now I'm taking that back. Um I'm I yeah I'm going to try to be as clear and careful as I can but I also don't want us to get like totally buried in notation if we can avoid it or were there any other follow-ups to that or okay great okay so like I said we are going to do stochastic gradient descent and so we need to calculate two different things we need the gradient of our loss with respect respect to this weight vector W and we need the derivative of this loss LI with respect to the bias B.

And we're going to do them separately. But really once we've figured out the gradient with respect to W, it's trivial to figure out what the gra uh derivative with respect to B is. So it'll be very very straightforward. And to do this, we're going to use the chain rule. So remember that the gradient with respect to a vector w this is going to be uh a vector where I've got d l i dw1 d l i dw2 all the way to d l i dw. And so what we're going to do then is we are going to think about the one one element of this gradient or of this vector here.

And I'm going to give myself plenty of room. So I'm going to think about the derivative of my loss with respect to w uh or well with respect to keep my notation yeah with respect to wj. So i training sample j weight which is the weight on the j feature within that sample. Okay. So in order to do this we are going to use the chain rule like I said. So first we're going to compute the derivative of this loss li with respect to the error or I'm sorry with respect to the output yi hat.

Then we're going to compute the derivative of yihat with respect to zi everything going into that nonlinear activation function. And then finally the derivative of zi with respect to this weight wj of interest. So we've got three different derivatives that we need to calculate here. Um and then once we have them we multiply them together. So it's just a straightforward application of the chain rule. Okay. So let's look at them one by one.

So first we are going to have the derivative of li with respect to yi hat. And so remember that our loss here is squared error. So this is the derivative of yi - yihat^ squar with respect to yi and what y i hat and what is that derivative? Yes. >> Yes, that's right. So, this is 2 * yi - yi. Exactly right. Next, we are going to look at the derivative of yihat with respect to zi. So remember that yihat is simply this nonlinear activation function sigma applied to zi and now we're taking the derivative of that with respect to zi.

So this is where sigma prime is going to play a role. And so this is simply sigma prime of zi. This is why I emphasized that when I was introducing the different activation functions and in the special case of logistic um activation function which is what we're going to work out in detail here. This is simply sigma of zi * 1 minus sigma of zi. Okay, so we've worked out the second factor in this chain rule expression.

And then finally we need to have the derivative of zi with respect to wj. And remember that zi was simply x i transpose w plus b. So we want the derivative of this with respect to wj. And just to be really kind of pedantic and slow about this, this is equal to the sum from uh I'll just say k = 1 to p of x i k w k + b. And we're taking the derivative of that with respect to wj. And so if I compute that derivative, what do I get?

Okay, so I've got a sum of different terms here. Which ones corres which ones depend on WJ? >> Yeah. >> Exactly. Where K equals J? Nothing else depends on WJ. So when K is equal to J, then we just have that the derivative is X I J. Okay, so we've gone through and we have computed all of the different derivatives that we need in order to compute this gradient. And so putting all the pieces together, what we have is that this derivative of the loss for the i sample with respect to wj is 2 yi hat minus yi times sigma z i 1us sigma zi Okay.

Time xigj. Now the thing that I want to point out is that [clears throat] in this expression I have the xig and then I have everything else here in red and green. This is a scalar and it is going to um be independent of J. Not not statistically independent. It just there's no J in that expression. So, as I'm thinking about computing my gradient vector, the gradient or the derivatives for all the different WJs, every single element in that gradient vector is going to have this first part be exactly the same.

The only thing that changes as I change J is the Xig. And so, I'm going to give this first part a name. I'm going to call this W um delta I. So, I've got a delta here that is indexed by whatever sample I'm using. I um and just to be totally clear, this delta I does depend on whatever my current estimate of W is. It's just the same regardless of whether I'm taking the derivative with respect to WJ or W K or something else.

And so now I can put all of the pieces together and I can say the gradient of li with respect to w evaluated at wt um is going to be equal to this delta i here that we just derived times and now I'm going to have a vector vector where the j element is xig j and so that vector is going to simply be my sample feature vector x i. So the gradient of my loss with respect to the weight w is simply this scalar delta i times my feature vector x i.

So very straightforward to compute. Any questions so far? Okay, so now let's bring it home. We also need the derivative of li with respect to b. And so this is going to have some components related to what we just did with derivatives with respect to w. We're going to have the derivative of li with respect to y i hat and then the derivative of y i hat with respect to zi and then the derivative of zi with respect to b.

And this should make you feel really happy because we've already done 2/3 of the work. We know exactly what the derivative of li with respect to yi is. We know y i hat is and we know the derivative of y i hat with respect to zi. And in fact, we just gave this thing a name. We called this delta i. So now in order to compute the derivative with respect to the bias, we only have to do one new thing. We have to compute the derivative just give it a new color here. the derivative of zi with respect to b.

And so here the derivative of xirpose w + b with respect to b. And what is that derivative? >> One. And so the derivative of li with respect to b is simply delta i * 1. So once I've computed this delta i for our new sample i, then from there we can immediately compute the derivative with respect to the bias and the gradient with respect to the weights. And so we then have our stochastic gradient descent updates. So at iteration t we are going to choose a sample i subt.

And then we are going to say that our weight vector for the new iteration t + 1 is going to be equal to the weight vector that we are currently at minus our step size tow times this gradient of l i subt evaluated at wt. And what we just derived is that this is equal to w t minus toao delta i times the vector x i. And our new bias B at time t + 1 is going to be B at iteration t minus toao * the derivative of l i t with respect to b which we just derived would be bt minus toao delta i.

Are there any questions? Yes. >> So I finding some notations the loss function like there is like a one / n before the sum of all the features >> um you're right last time I did write this one overn um I guess I did at the beginning of today's lecture as well um that can be handy for some convergence proofs but ultimately we're not really concerned with the raw value of the loss we're just trying to find the weights that make the loss as small as possible.

So if we multiply that loss by a scalar constant 1 / n or if we don't it doesn't really change the way that we um or or what the minimizing weights are. Um and so in that sense it's irrelevant. Um whether or not you multiply by 1 / n of course will determine what good values of toao are. And so when we try to prove what's a good value of tao like I did in last week uh on Tuesday's lecture notes then that 1 overn can play a role.

But in terms of what these update equations look like we're just taking the whether or not you've got the 1 /n might change this by a scalar factor or the tow. >> Yes. >> Oh yeah. Um okay so what I was doing here is I was saying we've computed the derivative of li with respect to wj. What we really want is the gradient of li with respect to the vector w. And so that gradient like we drew out on the far right board is simply the vector of these derivatives for w1, w2, etc.

And so I'm taking this expression here and I'm saying, okay, now if I stack this up for all the different J's, what would that be? And so I'm saying, well, it's going to be the vector X I times the scalar factor, which is simply this delta I. Yeah. >> Yes. >> Is it confirmed the LI are just these individual loss functions that make up the loss function? >> Exactly right. So just like we did we said at the very beginning we're thinking about our overall loss as being a um sum of all the individual losses across all the individual samples.

So for instance with squared error loss we are taking the squared error across each of the training samples and then adding them up. Yeah, >> I want to ask question about the ones behind like this X or XJI because like if we have X I to be the ice column like shouldn't it be the second entry >> what we're doing is um just like we did u at the beginning of the course with le squares where XI would be my E training sample so we're in the setting where um we are given training pairs x i yi where the x i's are in p dimensions and the y i's are just some scalar values and i goes from 1 to n.

So we are not saying the x i is a column of a particular matrix here. We're just saying x i is the i feature vector for the this feature vector for the I sample. >> Okay. So xig refers to the entry of >> yes x is the j entry of the training sample. has no relationship with the metrics, >> right? So, um, the way we did this back at the beginning of the quarter was we said, um, that a feature matrix X would be X1 transpose, X2 transpose all the way to XN transpose.

And this matrix would be N by P. But we are not using that matrix in this lecture because in this lecture we are focusing on stochastic gradient descent where we only handle one feature vector or one training sample at a time. So we're not loading in this whole matrix in part because this matrix is really enormous for modern context where n could be in the billions for instance. Okay. So now what we're going to do is work with a much bigger and deeper neural network.

What we are going to do now is take those same simple ideas from our very simple neural network and see how they manifest in a much wider and deeper network and show what the gradient descent updates look like in that context. So we start off just like before. We have our training features XI1, XI2, XI3 all the way to XIP. And then we are also going to have a node where we just input a one. And now after that, instead of mapping directly to an output, we are going to have what's called hidden nodes or a hidden layer.

And so these hidden nodes are going to be taking inputs just like our output did in our simple network from the input nodes. And so these lines here are playing exactly the same role that they played when we had our very simple example. And our biases are going to point to these nodes. And so we are going to call the inputs to these nodes Z's. So the node inputs in this case is going to be a vector. It's going to have subscript I to indicate that it's for training sample I and superscript one because this is our first hidden layer.

And we can talk about individual elements like ZI11, ZI12 I sorry yeah ZI13 etc. The outputs are going to be the result of applying our activation function sigma to the node inputs. So we will call the output vector a I with superscript one and that is simply sigma of zi with superscript one where when I'm applying this activation function sigma to a vector I just mean that I'm going to apply it separately to each element of the vector as before each of these lines is going to correspond to a weight.

So here we are going to have a matrix of weights. So if we think about the inputs to the different nodes, what I'm saying here is that ZI1, this vector being input to each of the different nodes can be written as a weight matrix W1 times whatever is coming from the previous node. And just for giving us some ourselves some general um notation, I'm going to call a 0 simply my feature or a i z a i z as being my feature vector x i.

And so I'm going to apply this weight matrix to the what's coming out of the previous layer. So I will have a I zero here and then I will add to it a vector of biases P. This is a vector now because we've got multiple different biases, one for each of the different nodes. I'm going to use a superscript one to indicate that it's the biases associated with this first layer. So in our diagram this is this first um bias would be B11 second one B12 third one B13 etc all the way to B1 and then the number of these that we are going to have is going to be M sub one I'm going to have M sub one initial internal nodes here and similarly the white lines correspond to elements ments of this weight matrix W.

So this first line is W1 with subscript 1 one because I am going from node one to node one in the next layer. This line right here would have corresponds to weight w with a superscript one and then subscript 21 because it is going into node 2 from node one. And so with this pattern of assigning a weight to each one of these lines, we can represent that whole collection of weights as a matrix that we multiply to the outputs of the nodes from the previous layer.

We add in the biases and that gives us the inputs to these nodes. And then like I said before, we are going to have a i1 correspond simply to applying this nonlinear activation function sigma to z i1 vector. And now we are going to repeat this pattern. So for our next hidden layer, we are going to have a collection of hidden nodes and a bias node. And so within here, we are going to have a series of weights that I will represent using lines just like we have before.

And I am going to use red for my biases. And so what this is representing is taking the outputs from this first hidden layer A I1 and applying a new matrix of weights W with superscript 2 and adding in another vector of biases with superscript 2 corresponding to these new biases. And the result of that is what we are calling Z I with superscript 2. And what's coming out is going to be a I with superscript 2, which is this nonlinear activation sigma applied to ZI2.

And so again, we can label the different lines in this network or the uh weights. So the biases would be B superscript 2 1 all the way to B superscript 2 and then I'm going to use capital M2 to correspond to the number of different nodes in this second hidden layer. And similarly I'm going to have here weight matrix 2 superscript 2 and then element 1 one because I am going into node one from node one and this line here would be w with superscript 2 and I'd be looking at element 21 because I'm going into node two from node one.

And I would keep repeating this pattern for as many layers as I want. So here's my final hidden layer. And then finally, I'm going to have my outputs. I'm going to make this a little bit more general than we had before. I'm going to have more than one output. So this will be y i1 all the way to y iq where I've got two different outputs. We use multiple outputs here. For instance, if I wanted to um think about my feature vectors x as corresponding to say pixels in an image, then my outputs could be um predictions of whether or not certain objects are in the image.

So my first one could be whether or not there's a dog. Second could be whether or not there's a cat. Third, whether or not there's a house. So I can have multiple different predictions that I want to make for a single um feature vector. And so we're trying to predict all of those. And just like we did with all of the other layers, we are using the results from the or the outputs of the hidden layer before and feeding those into these output nodes.

And so what we're saying here is that Z capital L I for my final layer of outputs is going to be equal to W capital L times the output of my penultimate layer or my last hidden layer. So a I capital L minus one plus my biases B capital L and the result or the output is going to be Yi hat. This is now a vector now. And this is going to be exactly equivalent to being a capital L I or in other words this sigma let's do it like this a capital L I or my nonlinear activation function sigma applied to Z I L.

So at each layer I'm doing the same thing. I'm taking the output of the previous set of nodes, previous layer, multiplying it times a weight matrix and adding a bias vector. Then I feed all of that through my nonlinear activation function and that gives me the output of my new hidden layer. And I repeat this all the way along to the end. Okay. So within this framework, I want to be crystal clear. When I write W L for layer L J K, this corresponds to the weight that we apply on the output of the K node at layer L -1 is a lowercase L to the J node at layer L.

So then we are going to let ML be equal to the number of hidden nodes at layer L. And so now within this framework at each layer we are going to have Z I L. And this is the set of inputs to the nodes at layer L. And so this is a vector of dimension ML. We also have a i l and these are the outputs of the nodes at layer L and this is also a vector of dimension ML. We also have all of our biases. So BL this vector is equal to the biases at layer L.

This is a vector of dimension ML. And our full weight matrix for layer L is going to be a matrix of dimension ML by ML -1 because I am mapping from the M L minus1 nodes at the previous layer to the ML nodes at my new layer. And at every stage here, I'm going to have exactly the same update equations. So Z for layer L and training sample I is going to correspond to taking WL and multiplying it by the vector of outputs from the previous layer A I L + L minus one and adding to that my biases B L and then I am going to have A I L is equal to simply my nonlinear activation sigma applied to ZIL.

And so now what we need to do is to put take all of this together and compute the gradients of the losses with respect to our unknowns. All of the BLS across all of the layers and all of the WLs across all of the layers. And once we have those gradients, then we can apply stochastic gradient descent to train the network. Or in other words, to use our training samples to learn the full collection of biases and weights.

So in particular, where we're headed right now is computing the gradient of the loss with respect to the E training sample with respect to our weight vector or weight matrix at layer L. Just like we did before, we're going to do this element by element. So we're going to compute the derivative of li with respect to this WL matrix J kith element. Now just like we did before we are going to apply the chain rule. So this is the derivative of the loss for the E training sample with respect to Z I J L.

So I'm looking at the input into the ELF layer for the E sample in the J node. And then I'm going to multiply this by the derivative Z L I J with respect to W J K L. So when we remember our update equation, so let's just write it here. Z I L was w L * A I L -1 + B L. So as I vary this weight matrix WL, it's causing variation in the ZILS, the inputs into the next layer of the network. And so we are breaking down the derivative of the loss with respect to this weight into the derivative of the loss with respect to those inputs and then the derivative of those inputs with respect to the weight matrix.

So we're going to look at these factors one at a time. So first let's look at the derivative of li with respect to z l iig j. And again we are going to use the chain rule. We are going to write this as the derivative dl i with respect to d a i j l times the derivative of a i j l with respect to z i j l. Now we are going to tandle each factor in this separately. So for our first factor where we take the derivative with respect to a we are going to have d l i d a i j l.

And so if we think about how the loss varies as the output of the node J at the elf layer varies, we have to think about what that output is affecting. So if we go back over to our drawing over here, if we think about what happens for instance as we vary this weight, as we vary that weight, it impacts the corresponding Z obviously the input to this node. It impacts the component of A that's output to this node. And as that varies, that's going to vary the inputs to all the nodes in the next layer.

And so as we compute this derivative over here of the loss with respect to a lig, what we need to do is compute the sum over all of the nodes in the next layer. So, I'm going to sum from k = 1 to m l + 1 because I'm looking at the next layer. And I'm going to have the derivative of LI with respect to Z I K L + 1 times D Z I K at layer L + 1 D A I J Well, so I'm summing over all of the different nodes in the next layer which are affected by changes in the output of my current layer.

A I JL. Okay. So, at this point, you might start to feel a little bit worried because now we have to do this over and over for each subsequent layer. And it feels like we're just going to get an exploding number of terms. But in fact, we're going to start seeing some patterns that are going to help us a lot. So we are going to give this derivative of li with respect to z lig a name. And in particular we are going to call this delta superscript l for the elf layer i j.

And now as we look at [snorts] our new expression here, we see that we've got a very similar term here. Here we are looking at the derivative of li with respect to one of the z's at the next layer. So we can put in here delta l + 1 i k. So this is going to help us simplify things quite a bit. So first of all we are going to write this expression um in terms we're going to simplify this expression. Before we do let's think about this derivative right here.

So let's go off to the side to calculate that. We want to calculate the derivative of Z I K L + one with respect to A I J from layer L. And we're going to recall again this basic equation that we keep using. So we've got here the derivative of W L +1 a L I vector plus B L + 1 and then we are just taking the kith element of that matrix vector operation and we are computing the derivative of that with respect to a l i j and so we can think about this as a big sum up at the top.

So in particular in our square brackets here we have the sum for J prime all the different nodes J prime and we're going to have W L +1 and then the K J prime element times A I J prime for the L layer plus B L +1 K and we're taking the derivative of that with respect to i aigi j at the elf layer. And now that we've written it out long form like this, it's clear that in this big sum, there's only one term that varies with a l i j and that corresponds to the term where j prime is equal to j.

And so what we are left with is w l + 1 k j. And so that is what we are going to put right over here in this expression w l + 1 k j. Okay. Okay, so putting these pieces together, we have here that this is equal to the sum from k = 1 to m l + 1 of delta i k for the l +1 layer times w l +1 kj. And now as we see that we're summing over K's here, we see that we're computing a product that looks a lot like an inner product.

And so in fact what this corresponds to is computing this matrix W l +1 transpose times a vector delta sub i l + 1 where I just take all the delta i k's and I stack them into a vector just like we did in our simple network example. So I take that matrix vector product and I simply keep the J element of it. So I've got this very simple expression in terms of my weight matrix and in terms of this delta vector for my derivative here that's going to feed into this part of my chain rule.

Next, I need to compute the derivative of a JL with respect to Z AJL. And so, just recall that from our master equations, AI for layer L was simply equal to sigma for applied to ZI at layer L like that. And so if we come over here and we say well what is the derivative of aig j at layer l with respect to zig j at layer l. We simply get sigma prime at zigj at layer l. And in the case of logistic activation functions, this would simply be sigma zigj for layer L time 1 minus sigma zigj at layer L.

Okay. So now we can go back over here and derive an expression for this delta LIG by taking by by multiplying these two factors from the chain rule that we've already derived. So in particular what we are going to get is W L +1 transpose time delta I for layer L + one and we just keep the J element of that and we multiply that by sigma prime Z I J for layer L. So now we have this nice simple expression for our delta um LIG.

Now I just want to be really explicit here that we do this for layers L less than capital L. So before our final layer because when we're at our final layer there is no w capital L+1. So this doesn't really hold up for our final layer and we need to derive the derivative for our final layer separately. So if we were to take delta I J for layer capital L. Well, just like before, we can use the chain rule and we can write this as d l i with respect to a l i j time d a l i j with respect to z l i j.

So that part doesn't change. The only part that changes is how we compute the first factor in yellow here. The derivative I got this right. The derivative of the loss for the ice sample with respect to a i j capital l. But remember that a capital L corresponds to our predicted output. So this is the derivative of the loss with respect to yi hat and the j output. And just like we did with our simple network, we have a very simple expression here.

So if we have squared error loss, this L subi is going to be the sum from uh J prime = 1 to Q of Y I J hat J prime hat minus Y J prime squared. So I'm just looking at the sum of my squared errors across all of my different um uh outputs and I want to compute the derivative of this with respect to yig j hat. So within this sum the only term that depends on y hat i j occurs when j prime is equal to j. And so what we get is simply 2 y minus yig j.

This second factor d a l i j with respect to d z l iig is exactly the same as before. So we can use our green equations just like we did before. And so in the special case where capital L is equal to little L then this expression for delta JL is simply the product of 2 Y hat - Y times sigma prime of Z capital L I Okay, so now we have expressions for this delta JL for L equal the final layer and on the previous board L for all the previous layers.

Okay, so going back here, we said ultimately we wanted to compute the derivative of the loss with respect to the JK element of the weight matrix at layer L. We did this chain rule and we had two factors. The derivative of LI with respect to the inputs to layer L, I J uh J node for sample I. And now we have left this derivative of Z li with respect to the element of the weight matrix that we're interested in. So that's the next thing we are going to calculate here.

So coming back over to this board, we now want to compute the derivative of Z LIG with respect to W L JK. Okay, so now we need to compute this derivative. Things are going to get a little bit easier and less tedious from here. And in particular, we are going to use this master equation again and remind ourselves that when we think about Zigj L, what we have is WL L* A L -1 um for sample I plus B L and then we are going to take the J element of that vector.

And now what we want to do is compute the derivative of that with respect to W l J K. So if we think about what this numerator is, we can write it out more slowly as being the sum from K prime equals 1 to the number of um nodes ML of WL J K prime time A LUS1 for sample I entry ent K prime plus B L entry J and now we want the derivative of that with respect to WL JK. So within this sum the only term that depends or varies with WLJ K occurs when K prime is equal to K.

And so what we are left with is simply a L minus one I K prime um I K. Okay. So now we just need to put the pieces together. So in particular we wanted to compute the derivative of LI with respect to WL JK. And what we derived through all these calculations is that this is going to be delta L I J times a L -1 I K. And so now we can stack all of the pieces together in order to get the gradient of LI with respect to the whole weight matrix L.

So let's just think about dimensions for a second. This should have the same size as WL. So it should be M L by M LUS1. Now if we were to form the vector delta L I by stacking together all the elements across all the different J's then this would be a vector of length ML because we've got J is indexing over all the hidden nodes at layer L and there's ML of them. And similarly, if we were to form the vector a i l minus one by stacking together all the elements across all the different k's where k is indexing all the hidden nodes at layer l minus one.

That's going to be a vector of length m l minus one. And so we can write this gradient here as simply being the outer product of the vector deltal i and a i l minus one. So I'm going to just write that as a i l minus one transpose. and we'll have the right dimensions and the JK entry of this will correspond exactly to what we had written over here and what we derived and we have on the other boards expressions for what this delta li is.

Okay, so now we have this very nice simple expression for the gradient of LI with respect to the weight matrix at layer L. And we can do this for each of the different layers. Okay, before we get to the full back propagation algorithm, we need to calculate the derivatives and the gradients with respect to the biases. I think we're all a little bit tired now, but in fact, we've done almost all of the hard calculations and we'll be able to reuse some things here.

So, we want the derivative of the loss for the E sample with respect to the bias at the elf layer. And I am just going to refer to the J element of the bias here. Okay. So, to calculate that, we're going to again use the chain rule. We've got the derivative of L I with respect to Z L I J times the derivative of Z L I J with respect to DB LJ. Okay, so this first factor here is something that we've seen before. This is just exactly delta J at layer L.

So, we've already done at least half the work and all that's left is to figure out the derivative of Z Lig with respect to B L J. So, just like before what we are going to do is we are going to remember that Zig JL corresponds to W L A L -1 I + B L and then we just take the J entry and we want to know the derivative of that with respect to B L J. And so within this the only thing that varies with B is this and that derivative is simply equal to one.

And so again putting the pieces together we have that the derivative here of LI with respect to B LJ is simply equal to deltal L I J. Or if we want to form the full gradient of the loss for the E sample with respect to the biases at the L layer that is simply equal to delta I L. So once we did the weight matrices the biases were trivial. So now we have these expressions for our gradients that apply at each one of the layers.

What we want to do is perform stochastic gradient descent. Now as you can see the calculations for layer L depend on what's happening at the previous layer. And so this is where the back propagation algorithm comes in. Okay, so just to put everything on one board, what we have is that the gradient of L I with respect to the L layer is equal to delta L I* A L -1 um for the E sample transpose. And what we worked out is that delta I is going to have one of two values.

We can consider the case where we are not at the final layer or where we are at the final layer before the final layer. This is equal to simply w l + 1 transpose times delta i l + 1 and then we are going to take that whole thing and multiply it elementwise with sigma prime z iil vector. So when I have the little dot in a circle, this means elementwise multiplication. Uh so I've got two vectors the same length. I just multiply each pair of elements together to form a new vector of the same length.

And for the final layer, we simply have um the gradient of our loss with respect to um that final layer a l. And that also gets this element product with sigma prime z i l. And finally we've got that the gradient of li with respect to the vector of biases B is simply delta L I. Okay for the back propagation algorithm. Now what we are going to do is the following. So first t = 1 2 3 etc. As we integrate iterate, we are first going to have what's called the forward pass.

And for this, what we are going to do is take our current estimates of the weights and the biases. And we are going to feed in a training sample and calculate all of the A's and all of the Z's for every layer and every node in the network. So we select a sample it that is one of our n training samples that we have available. Then like we said before we are going to let our um a z uh a z i. So the a's corresponding to the inputs to simply be our feature vector x i t.

Then we are going to pass the information from the input to the output through the network. So for layer L equ= 1 2 all the way to capital L we calculate Z L for I subt by multiplying by our current estimate W for layer L and then I'm just going to add to the superscript here a T to indicate the fact that we're computing this at the t iteration but it's the same idea as what we've been doing all along. We multiply that times a l minus1 for sample it t and we add to that the biases for layer l and the current estimate of the biases at iteration t.

Then we compute our a's. So a l i t is simply sigma at z l i t. Okay. Okay. So that ends this loop. And so this whole part is called the forward pass. And now what I'm going to do, I want to keep this all on one board. So I'm now going to compute the backward pass. So I'm still on iteration t. And I am going to first compute the delta vector at the final layer. So I'm going to compute delta capital L for training sample it.

And we just use the formula that we have on the left hand side of the board. So it's simply going to be equal to the gradient of L I T with respect to A or uh for A I T for layer L element wise product with sigma prime Z I capital L and over here I should write a capital L as So in my former loop, I've calculated these A's and these Z's. So I'm now able to calculate delta using what I calculated in this earlier loop.

And now that I've got this delta, I can start working backwards through the network. Because when I've got the end layer delta, I can compute the previous layer delta from it. And so over here, so we're going to just loop on over here. And now we are going to loop over the layers, but we're going to start at the end and move backwards. And so first we are going to compute our new delta for layer L for sample I subt. And we are just going to use exactly the formula that we have over on the left.

So we have w l + 1 transpose times the delta l uh + 1 for sample it t and I take this the product of this elementwise with sigma prime z l for sample it t. So, I'm able to calculate this because I've already calculated the delta from the previous layer and I'm using that to calculate the delta from the new layer. And in my forward pass, I computed all of these Z's. And so now that I've got this delta, I have that my gradient of my loss for my E sample with respect to my weight matrix at layer L is simply equal to this delta IIL time A I L minus1 transpose.

And so again I've already calculated this delta I subtl and I computed the A's when I was going through this forward pass earlier. I can also compute the gradient of my loss at sample it with respect to my bias vector at layer L. And that is simply this delta I T L. So now that I've got my gradients, I can do my gradient update. I can say that WL for iteration t + 1 is going to be wl at iteration t minus a step or plus a step in this direction of the negative gradient or minus toao times this gradient we just calculated gradient l i subt at w l evaluated at iteration t and similarly I can take a stochastic gradient to step step step for my biases BL.

So at iteration t + 1, I will have the biases for the same layer at iteration t. And I take a step in the direction of the negative gradient. So minus some toao times this gradient l i subt b l t. And then I end this pass. And then I end. So I'm ending my iteration with respect to t. And here I'm ending my iteration with respect to L. So when I do this there's two kind of distinct chunks involved. First there is this loop here.

So this we would call the forward pass because we are taking our sample and we are passing it from the first layers to the last layers and as we go forward through the network we are computing the inputs and the outputs to every node at every layer and we're storing all those inputs and outputs those A's and those Z's. Then we start at the end of the network at the output end and work our way backwards towards the inputs.

So what we have over here now with this this is our backwards pass. So I compute this delta that shows up again and again at my final layer by thinking about how the final layer outputs are affecting my overall loss. And then I can use the deltas from the to compute the deltas from each previous layer. So I'm working my way backwards through the network. And once I have those deltas, then I have very simple expressions for the gradient of my loss with respect to the weights and with respect to the biases in terms of those deltas and these A's that we already computed.

And once I've got those, then I can compute these stochastic gradient updates. And that is going to give me my new sets of weights and my new steps of bias sets of biases. And so just like we talked about with stochastic gradient descent in the last lecture, we would continue iterating until we stop seeing changes in the W's and changes in the B's. So overall back propagation is simply an instantiation of stochastic gradient descent for a neural network where we compute our gradients for each layer by first taking our current weights and biases and propagating a sample through the network and computing all the inputs and outputs of all the nodes and then using that in order to compute gradients from one layer to the next starting at the end of the network or the output end and moving backwards through to the input end in order to get our gradient updates.

Thank you very much. This was a lot of notation and a lot of equations. I appreciate your patience, but hopefully this gives you a little bit of insight into the workings of back propagation to the extent that you'll be able to understand code and executing this when you look at it and figure out what the updates equations would be for um new activation functions and potentially new architectures. I'll see you after Thanksgiving.

Bye-bye.