Backpropagation derivation using Leibniz notation

From Machinelearning
Revision as of 22:25, 8 November 2018 by IssaRice (talk | contribs)

The cost function C depends on wjkl only through the activation of the jth neuron in the lth layer, i.e. on the value of ajl. Thus we can use the chain rule to expand:

∂C∂wjkl=∂C∂ajl∂ajl∂wjkl

We know that ∂ajl∂wjkl=σ′(zjl)akl−1 because ajl=σ(zjl)=σ(∑kwjklakl−1+bjl). We have used the chain rule again here.

In turn, C depends on ajl only through the activations of the (l+1)th layer.